&w=3840&q=75)
Tester Army’s e2e runner has been doing the rounds, and I think it’s a good article because the interesting part isn’t “an agent writes your tests.” It’s a TypeScript runner that lets you mix ordinary locators with a goal, then treats a passing agent step as something you can replay.
The package is open source (Apache-2.0), still on the way to 1.0, and the maintainers say the API can still move between minor releases. You scaffold with npx e2e init (or npx skills add tester-army/e2e if a coding agent is doing the setup), and you run with npx e2e run. The quickstart wants Node 24.8 or newer, or 22.22.3 or newer if you’re still on the 22 line. Web goes through Playwright, and iOS simulators and Android emulators go through their mobile engine. Tests that never call the agent don’t need a model at all.
You bring the model yourself, whether that’s an API key, Vercel AI Gateway, OpenRouter, a local server, or a ChatGPT, GitHub Copilot, or SuperGrok subscription. I’m not going to tour the provider list. The point is the runner doesn’t ship with one opinion about which model clicks the buttons.
Goals where the screen is messy, checks where you already know
The writing docs are pretty blunt about the split. Use agent.act when you’re describing a goal (“upgrade the workspace to the Pro plan”, “add these two todos and complete the first”). Use a locator when you know the exact control or the exact value. agent.assert and agent.extract sit in the middle, for questions about what’s on screen. You give act one goal at a time, you keep the order of those goals, and the clicks inside a single goal are the model’s.
import { test, expect } from 'e2e';
test('a member upgrades to Pro', async ({ app, agent, screen }) => {
await app.open('/settings/billing');
await agent.act('upgrade the workspace to the Pro plan');
await agent.assert('the invoice preview shows a prorated amount');
await expect(screen.getByRole('status')).toContainText('Pro');
});That example is theirs, from the README, and it’s the shape I keep coming back to. The agent does the fiddly navigation, and the assertion at the end is still a role and a string, the sort of check a human reviewer can argue with. If the whole test was “the agent says checkout worked”, you’d be trusting the model’s opinion of its own work.
I haven’t put this on a suite. I’m writing about the design, not a war story. Natural-language end-to-end tests have usually annoyed me because the flake moves into the model and stays there, on every run, forever.
The cache is the actual idea
Caching agent steps is where this stops being another “AI browser” demo. After an agent.act step, a later check has to verify the result before the runner records the actions. On the next run it replays those actions and doesn’t call the model. If a control is missing, ambiguous, or the end state doesn’t match what was recorded, the agent takes over from the current screen (a hand-off) or runs the step from the start (a miss).
agent.assert, agent.waitFor, and agent.extract always run live. That’s the part I’d want if I were reviewing a pull request that added this, because the navigation can be cached while the judgment still has to be made live. Changing the model doesn’t bust the cache, but changing the instruction, the params, the agent, or the agent’s context does. Timestamps and fresh emails need wrapping in unique() or every run looks like a different step.
Locally the default mode is read-write. In CI, if you haven’t set cache, the default is read-only, so a pipeline replays and doesn’t quietly rewrite the recordings. There’s npx e2e run --no-cache when you want to know whether a failure is the app or a stale recording. Entries live as JSON under .e2e/cache/. The docs say they hold actions and target descriptions, not prompts, screenshots, or secret values (a secret fill stores the secret’s name). Typed text is stored as typed, so you wouldn’t commit a cache that has just pasted a customer email into a form.
The replay cache records the actions in an agent.act() step after a later check verifies the result. On the next run, the runner replays those actions without calling the model.
That’s their sentence, and it’s a better product description than “AI testing framework.”
What I’d actually want it for
The failure mode I don’t want is a prompt that says “log in, add a product, apply the code, and confirm the total” as one goal, with nothing deterministic underneath. The model will often manage it, and you still won’t be able to say which click regressed when it doesn’t. Their own tip is to write the instruction the way you’d say it out loud, one goal per call, and follow it with an assertion or a locator check. I’d take that seriously. A cached saga you can’t verify is just a flaky script with extra steps.
I’m also not pretending replay is free durability. The cache matches controls by role, name, test id, and surrounding context, and it gives up when a match is ambiguous or a bare point was recorded on a different viewport. Pre-1.0 means I’d pin a version before I let a suite depend on it. Anonymous telemetry is on unless you opt out (npx e2e telemetry disable or E2E_TELEMETRY_DISABLED=1), which is worth knowing before it lands in a work repo.
Thoughts
If your teams are looking at this because an agent might finally write the end-to-end suite, I’d look at the cache instead. Keep the goals short, put a locator or a live agent.assert after the ones that matter, and let a verified step replay without another model call. The model is a way to record a messy path the first time, not the thing you trust on every pull request.
