> [Agent navigation index](/llms.txt)

# End-to-end tests and hydration benchmarks

Use managed Playwright suites for complete application behavior and the Node-only hydration benchmark for controlled comparisons between equivalent production deployments.

Prefer route and component tests for isolated behavior. Add E2E when the requirement crosses the
real server/browser boundary, such as navigation, SSR-to-hydration behavior, or production service
integration. Managed defaults support full-stack Preact applications and managed React
applications; full-stack suites also need a provider-free `test-run` stage. Libraries need an explicit
`webServer`.

## Test the complete application

Add `e2e/products.spec.ts` and import the managed fixtures through Brix:

```ts
import { expect, test } from "brix/test/e2e";

test("opens a product", async ({ page }) => {
  await page.goto("/products/sku-123");
  await expect(page.getByRole("heading", { name: "Everyday backpack" })).toBeVisible();
});

```

No extra dependency, config, package script, or install command is required for an application. Brix starts and stops `brix dev` for the project, prepares Chromium only when tests execute, and stores failure traces and screenshots under `.brix/e2e/results`. Tests in `e2e/` run fully parallel in headless Chromium with the `list` reporter.

The managed server listens on a free `127.0.0.1` port that Brix selects once per run and shares with
Playwright workers, so parallel checkouts and other local servers do not collide. Set `BRIX_E2E_PORT`
to a TCP port from 1 to 65535 to pin it. The server never reuses an existing process: a port that is
already in use fails the run instead of testing an unrelated server. A full-stack server is ready
when `brix dev` has loaded the application and answers its framework-owned
`/__brix/dev-ready` probe, so the application need not serve `/`; a managed React server is ready
when `/` responds. Server startup may take up to two minutes before the run fails.

When `CI` is set, managed defaults retry each failing test twice and reject committed `.only` calls;
locally tests run once. Workspace-root `brix test` sets `CI=true` for member runs, so it uses the CI
policy too. Set `retries` or `forbidOnly` in `brix.e2e.ts` to choose another policy.

For a full-stack application, the server inherits an already resolved environment; otherwise managed
E2E loads the **test** stage once in the Playwright runner, including local and personal overrides.
Workers reuse the running server and do not load it again. That load can resolve providers
even without `--env`. Keep `env/.env.test-run` provider-free for offline suites and prevent local files
from selecting production services. Use `brix test --env --no-local --project e2e` for an
explicit isolated stage, or select another stage deliberately for integration coverage. Managed React
applications do not use this server environment loader. A custom `webServer` owns its own loading.

Managed traces capture DOM, accessibility (ARIA), and screen snapshots around actions and retain
the trace only when a test fails. Successful tests still incur capture overhead, but their traces
are discarded. Use Playwright's trace viewer to compare the action, rendered page, and accessibility
tree. Traces can contain page text, screenshots, network data, and credentials; use disposable test
data and review artifacts before sharing them. Brix does not redact traces.

```sh
brix test e2e/products.spec.ts -t "opens a product"
brix test --project e2e
brix test --project e2e --list
brix test --project e2e --headed
brix test e2e/products.spec.ts --debug

```

Unqualified `brix test` and `brix verify` include discovered `e2e/` suites. `-t` maps to Playwright's `--grep` after E2E selection. Keep a focused selection within one runner and workspace member.

The default suite runs against development output, not a production bundle. Use a configured
production service or deployed candidate when testing platform bindings, production asset URLs,
cache policy, or behavior that differs after bundling. A passing development suite is not a
deployment smoke check.

## Customize the E2E service and browser

Add `brix.e2e.ts` only when the managed defaults are insufficient:

```ts
import { defineBrixE2EConfig } from "brix/test/e2e";

export default defineBrixE2EConfig({
  webServer: [],
  use: { baseURL: "https://staging.example.test" },
});

```

The helper accepts native Playwright configuration. Top-level options such as `testDir`, `outputDir`, `retries`, `reporter`, and `projects` replace the managed values. `webServer` replaces the managed server, its port selection, and its environment loading, so also set `use.baseURL`; otherwise it defaults to `http://127.0.0.1:4173` (or `BRIX_E2E_PORT`). `use` merges with managed browser, trace, and screenshot defaults. You can also configure browser projects, retries, reporting, or a custom test directory. Select `--project e2e` explicitly when focusing a file outside conventional `e2e/` discovery.

An explicit `use.trace` replaces the entire trace policy. Set it to `"off"` to disable traces, or use
`{ mode: "on-first-retry", snapshots: { dom: true, aria: true, screen: true } }` to capture only a
first retry (configure `retries` when running locally). Screenshots have a separate policy; set
`use.screenshot: "off"` too when artifacts must not capture page contents.

Existing `playwright.config.*` files remain supported, but their authors own server settings and tests must still import from `brix/test/e2e` for automatic browser preparation. An existing `test:e2e` package script overrides automatic full-suite dispatch and must not call unqualified `brix test` recursively. Remote browsers, custom executables, and branded browsers remain caller-provisioned.

Use E2E tests for navigation, real browser/server integration, and behavior before hydration. Use the [route harness](/docs/testing-routes.md) for fast server-route coverage and [component tests](/docs/testing-components.md) for isolated browser interactions.

## Playwright behavior to keep in mind

- Device descriptors forward `screen`, which affects `window.screen` and media queries. Brix's plain
  Chromium default is not a desktop device descriptor; review responsive assertions and screenshots
  when you add one.
- JSX in E2E files compiles with the `jsx`, `jsxFactory`, `jsxFragmentFactory`, and
  `jsxImportSource` settings of the selected tsconfig, defaulting to React's automatic runtime when
  none are set. Fix Preact test compilation in the owning project's configuration instead of adding
  React.
- `--update-snapshots=missing` passes tests that only create missing snapshots, while the normal
  `default` mode writes them and fails. Keep CI's acceptance policy explicit and review new
  baselines rather than treating a green creation run as comparison evidence.
- Elements inside hidden iframes are hidden for actions and visibility assertions. Reveal the
  intended frame before interacting; do not force actions past a real visibility requirement.

Optional upstream capabilities, such as video frame-rate options, WebP screenshots, and
experimental WebMCP, are not Brix defaults. Browser-version changes in a Brix upgrade can alter
snapshots; review them with application behavior and disposable fixture data.

## Compare hydration architectures

`runHydrationBenchmark` from `brix/test/benchmark` compares selective islands with a full-page island using caller-supplied production URLs. It does not modify source, build, start, deploy, instrument, or rewrite applications.

Prepare two checkouts with the same Brix pin, lockfile, data, locale, CSS, environment, and application code. Keep selective islands in one. In the other, add `"use island";` to the actual page component only if its browser graph contains no server-only code. Build and serve both production applications at separate URLs.

Create an application script:

```ts
import { expect, runHydrationBenchmark } from "brix/test/benchmark";

const islandsURL = process.env.ISLANDS_URL;
const fullPageURL = process.env.FULL_PAGE_URL;
if (!islandsURL || !fullPageURL) throw new Error("Set benchmark URLs");

await runHydrationBenchmark({
  variants: [
    { name: "islands", baseURL: islandsURL },
    { name: "full-page", baseURL: fullPageURL },
  ],
  runs: 10,
  scenarios: [
    {
      name: "product",
      path: "/products/example",
      controls: { quantity: '[data-testid="quantity"]' },
      verify: async (page) => {
        const quantity = page.getByTestId("quantity");
        await quantity.fill("2");
        await expect(quantity).toHaveValue("2");
        return { quantity: await quantity.inputValue() };
      },
    },
  ],
});

```

Run it with `brix run scripts/benchmark-hydration.ts`. The promise resolves to the report also
written to `results.json`.

Required options:

- `variants`: at least two, each with a unique nonempty `name` and an HTTP(S) `baseURL` without
  embedded credentials.
- `scenarios`: at least one, each with a unique `name`, a `path` that stays on every variant's
  origin, a nonempty `controls` map of name to selector, and a `verify` callback.

Optional settings and defaults:

- `variants[].context`: Playwright browser-context options for that variant. Keep them identical
  unless the difference is deliberate.
- `variants[].setup`: optional async `(context) => void` for cookies or authentication before
  navigation. Do not install request routing.
- `runs`: `10` repetitions per condition; must be a positive integer. Use `1` only for smoke
  verification.
- `cache`: `["cold", "warm"]`; any nonempty subset without duplicates. Cold samples clear the
  browser cache first; warm samples prime by navigating and activating the scenario first.
- `profiles`: `hydrationBenchmarkProfiles`, which are `desktop` (no throttling), 4× `cpu`, and
  `constrained` (4× CPU, 80 ms latency, 200,000 B/s down, 93,750 B/s up).
- `timeoutMs`: `30000` for browser waits, actions, and navigation; not a deadline for your callbacks.
- `outputDir`: `.brix/benchmarks/hydration`, with a unique `<timestamp>-<uuid>` subdirectory per
  invocation.

Select presets with `profiles: { cpu: hydrationBenchmarkProfiles.cpu }`, or define
`{ cpu, network? }` profiles: `cpu` is a throttling factor of at least 1, and `network` takes
`latencyMs` (zero or more) plus positive `downloadBytesPerSecond` and `uploadBytesPerSecond`. Omit
`network` for no throttling. Invalid options fail before the browser launches. Ten runs × two
variants × three profiles × two cache modes is 120 measured navigations per scenario, run
sequentially, plus warm priming.

Control selectors must uniquely identify SSR elements inside Brix island boundaries. The probe associates them before mount and observes existing boundary events. It supports repeated and multi-root boundaries and independent slot children; a parent event does not make an unhydrated child ready. Client-only elements, portals, static controls, and elements absent from initial HTML are not valid readiness probes.

`verify` runs after readiness measurement and must return the same stable JSON-compatible functional snapshot for every sample and variant. Browser errors, ambiguous controls, missing readiness, assertion failures, and snapshot differences fail the run. Verification interactions are excluded from startup timing, so use E2E tests—not the benchmark—to specify pre-hydration behavior or interaction latency.

For lazy or idle controls, add a separate `deferred` object with named controls and an `activate(page)` callback. Deferred control names must differ from the scenario's `controls`. Its readiness includes caller-driven activation and is not automatic startup time. Avoid Playwright request routing because it disables HTTP caching and invalidates warm comparisons.

## Interpret benchmark output

`summary.md` reports median important-control readiness, script execution, JavaScript transfer, and per-control readiness. `results.json` retains raw samples, environment/profile metadata, boundary events, completion timing, TTFB, FCP, payload sizes, resource inventories, and long-task excess. A failed run still writes `results.json` with `status: "failed"` and the error, but no `summary.md`, then rejects.

Boundary readiness means synchronous mount completion—not paint, effects, data readiness, INP, or responsiveness. Resource Timing zeros can indicate cache hits or unavailable cross-origin timing; configure `Timing-Allow-Origin` where appropriate and compare encoded, decoded, and transferred values together.

One run is a smoke check only. Keep the machine quiet, use non-mutating or isolated test accounts, retain compared revisions, and review reports for errors or sensitive paths before sharing them.
