Skip to content

perf(test): optimize Hydrogen test suite runtime (24.7% faster)#3620

Open
itsjustriley wants to merge 2 commits intomainfrom
autoresearch/test-runtime-2026-03-25
Open

perf(test): optimize Hydrogen test suite runtime (24.7% faster)#3620
itsjustriley wants to merge 2 commits intomainfrom
autoresearch/test-runtime-2026-03-25

Conversation

@itsjustriley
Copy link
Copy Markdown
Contributor

@itsjustriley itsjustriley commented Mar 25, 2026 •
edited
Loading

Summary

Optimizes Hydrogen's Vitest unit test suite runtime by 24.7% through configuration improvements.

Changes

Applied 5 Vitest configuration optimizations:

  1. Use threads pool instead of forks (14.8% improvement)
  2. Explicit test file pattern include glob (4.2%)
  3. Move jest-dom to per-file imports for 2/54 tests (1.6%)
  4. Remove unused coverage configuration (4.7%)
  5. Inline small common dependencies (1.7%)

Total improvement: 24.7% (4.52s → 3.40s for 478 tests)

Testing

  • ✅ All 478 tests continue to pass
  • ✅ Confidence: 6.3× noise floor (highly significant)
  • ✅ Config-only changes, no test behavior modifications

File Changes

packages/hydrogen/vitest.config.ts                             | 10 ++++++----
packages/hydrogen/vitest.setup.ts                              |  2 +-
packages/hydrogen/src/analytics-manager/AnalyticsProvider.test.tsx  |  1 +
packages/hydrogen/src/customer-privacy/useCustomerPrivacy.test.tsx  |  1 +

Checklist

  • All tests pass locally
  • Statistically significant improvement (6.3× noise floor)
  • Config-only changes
  • CI tests pass (needs PR run)

@shopify
Copy link
Copy Markdown
Contributor

shopify bot commented Mar 25, 2026 •
edited
Loading

Oxygen deployed a preview of your autoresearch/test-runtime-2026-03-25 branch. Details:

Storefront Status Preview link Deployment details Last update (UTC)
Skeleton (skeleton.hydrogen.shop) ✅ Successful (Logs) Preview deployment Inspect deployment March 27, 2026 1:30 PM

Learn more about Hydrogen's GitHub integration.

@itsjustriley
perf(test): optimize Hydrogen test suite runtime (24.7% faster)
23b5381 
Applied 5 Vitest configuration optimizations:
- Use threads pool instead of forks (14.8% improvement)
- Explicit test file pattern include glob (4.2%)
- Move jest-dom to per-file imports for 2/54 tests (1.6%)
- Remove unused coverage configuration (4.7%)
- Inline small common dependencies (1.7%)

Total improvement: 24.7% (4.52s → 3.40s for 478 tests)
Confidence: 6.3× noise floor

All 478 tests continue to pass. Config-only changes, no test
behavior modifications.
@itsjustriley itsjustriley force-pushed the autoresearch/test-runtime-2026-03-25 branch from 42dc0bb to ba77b84 Compare March 25, 2026 15:14
@fredericoo
Copy link
Copy Markdown
Contributor

fredericoo commented Mar 25, 2026

where is AUTORESEARCH_SUMMARY? i want to read it!

@itsjustriley
Copy link
Copy Markdown
Contributor Author

itsjustriley commented Mar 25, 2026

where is AUTORESEARCH_SUMMARY? i want to read it!

I'll post it on slack. I don't think it needs to be committed to hydrogen.

@itsjustriley itsjustriley changed the title perf(test): optimize Hydrogen test suite runtime (24.7% faster) test: improve E2E test reliability with retries and fixture improvements Mar 26, 2026
@itsjustriley itsjustriley force-pushed the autoresearch/test-runtime-2026-03-25 branch from dea630c to 016e4a9 Compare March 26, 2026 13:33
@itsjustriley itsjustriley force-pushed the autoresearch/test-runtime-2026-03-25 branch from 016e4a9 to 9efc4b6 Compare March 26, 2026 13:34
@itsjustriley itsjustriley changed the title test: improve E2E test reliability with retries and fixture improvements perf(test): optimize Hydrogen test suite runtime (24.7% faster) Mar 26, 2026
@itsjustriley itsjustriley reopened this Mar 26, 2026
@itsjustriley @claude
fix(test): move server.deps.inline to correct config location and rem…
fe33de9 
…ove no-op setup file

server.deps.inline was placed at the top level of defineConfig, but
Vitest's docs (https://vitest.dev/config/server) show it should be
under test.server.deps.inline. The top-level `server` key is Vite's
dev server options, not Vitest's vite-node module resolution config.
This means the dependency inlining for @shopify/hydrogen-react and
@testing-library/react was silently ignored.

Also removes vitest.setup.ts which contained only a comment after
jest-dom was moved to per-file imports. The setupFiles reference
in vitest.config.ts pointed to a no-op file loaded at startup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@itsjustriley itsjustriley marked this pull request as ready for review March 27, 2026 13:39
@itsjustriley itsjustriley requested a review from a team as a code owner March 27, 2026 13:39
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 27, 2026

We detected some changes in packages/*/package.json or packages/*/src, and there are no updates in the .changeset.
If the changes are user-facing and should cause a version bump, run pnpm run changeset add to track your changes and include them in the next release CHANGELOG.
If you are making simple updates to examples or documentation, you do not need to add a changeset.

kdaviduik
kdaviduik reviewed Apr 1, 2026
View reviewed changes
packages/hydrogen/vitest.config.ts
all: true,
include: ['src/**'],
exclude: ['src/vite/virtual-routes/**'],
pool: 'threads',
Copy link
Copy Markdown
Contributor

@kdaviduik kdaviduik Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall the change itself LGTM, but do we want to get rid of the coverage? Like is it not adding any value, or is it something we still want even if it slows things down slightly?

Copy link
Copy Markdown
Contributor Author

@itsjustriley itsjustriley Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing ever invoked it as far as I can tell.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kdaviduik kdaviduik kdaviduik approved these changes

@fredericoo fredericoo Awaiting requested review from fredericoo

@andguy95 andguy95 Awaiting requested review from andguy95

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@itsjustriley @fredericoo @kdaviduik