Playwright Agent

🐢 In short, Playwright Agent builds your CI pipeline, MCP config, and custom agents for you. Everything happens through conversation, step by step: plan → generate → fix

Official docs: https://playwright.dev/docs/test-agents

Setup

Use the init-agents command to add Playwright test agent definitions to your project. These should be regenerated whenever Playwright updates to adopt new tools and instructions.

The following uses VS Code, minimum version v1.105 (released October 9, 2025).

Command: npx playwright init-agents --loop=vscode

Run the command in the vscode terminal

Once the command completes, the following files are generated automatically:

Make sure your server is running (e.g. npm run dev) before proceeding. The official docs suggest the following flow:

Step 1. Plan

Use the Planner Agent The agent explores your app and generates a test plan for one or more scenarios and user flows.

I had already written tests for this flow before, so I'll walk through it directly.

Select the planner agent

I want to create a test plan for the print-on-demand file upload flow.
The current flow is:

Go to the file upload page, router path: `/ibonprinter`
Click the "Add File" button
Upload files from `/tests/test-data/` in the codebase
Test multiple error scenarios, e.g. uploading more than 8 files or exceeding the 15MB total size limit
The upload button is disabled until the user checks the consent checkbox
Clicking "Pre-select Print Settings" opens a settings panel
Clicking "Confirm Upload" shows a warning dialog if any non-image files are in the list
If only images are present, an API call is made — success is indicated by a pickup code

🐢 The agent starts exploring the feature I described, asking only to confirm permissions before acting. At the end it produces a complete test plan.

🐢 Step 1 thoughts

• Impressed that the agent found my existing file-upload.spec.ts
• It generated /e2e/file-upload-comprehensive.plan.md rather than /specs as the official docs suggest. It recommended keeping it in e2e because:
  1. Project consistency — the existing test file is already there
  2. Proximity — plan and code in the same directory is easier to manage
  3. No functional impact — .plan.md is just a doc file, location doesn't matter

Step 2. Generate

Use the Generator Agent The agent produces runnable Playwright tests from a Markdown plan, validating selectors and assertions in real time as it runs each scenario. Playwright supports generation prompts and an assertion catalog for efficient structure and behavior verification.

🐢 Tip

Before running, verify all file paths in file-upload-comprehensive.plan.md. The agent doesn't always get directory paths right, and a wrong path wastes time and tokens.

1. Generate the first test suite

Simple prompt — just the first section heading from the plan

# Run all file upload tests
npx playwright test tests/e2e/uploadfile/

# Run only basic functionality
npx playwright test tests/e2e/uploadfile/basic-functionality/

# Run a specific test
npx playwright test tests/e2e/uploadfile/basic-functionality/page-accessibility.spec.ts

🐢 Tip

Always run npx playwright test tests/e2e/uploadfile/ to verify — the agent tends to be overly optimistic about its output =___=

😫 If tests fail, make sure your server is running with npm run dev

If you normally use PowerShell, switch to the vscode terminal for both npx playwright test and npm run dev.

Run playwright test in a separate tab

2. Generate the second test suite

Same simple prompt — the second section heading

⏸️ The agent proactively asked whether to generate Dropbox integration tests. I've always avoided this — here's how our discussion went:

For features requiring third-party login, we adopted a "test the integration trigger, not the actual login" strategy:

✅ What to test:

• Whether the Dropbox button is clickable
• Whether the authorization dialog opens correctly
• Whether all copy text is present
• Whether clicking redirects to the Dropbox OAuth page

❌ What not to test:

• The actual Dropbox login flow
• File selection after login
• OAuth callback handling

What we found during testing

After clicking "Got it", the app opens a new tab to the Dropbox OAuth URL with the correct client_id and redirect_uri. The Dropbox button on the original page remains active.

🐢 Hit a problem!

The agent splits tests across separate directories, so to target only what was just generated: npx playwright test tests/e2e/uploadfile/advanced-features/

From this point on, tests started failing more often — but each test passes when run individually. My guess is the backend was getting hammered, causing slow responses.

Asked the agent how to increase timeout — adjusting playwright.config.ts fixed it:

actionTimeout: 10000,
navigationTimeout: 30000,

The agent did a lot, but the Context Window usage was lower than expected — 80.8k tokens

• Day 1
  • Used about 60% of the Context Window — like a margin call level
  • Better to stay under half, so I planned to clear this session and open a new one the next day
  • Final usage: 63.4%
• Day 2
  • Final usage: 74.7%

3. Generate test suites 3–5

Same simple prompt — section headings from the plan

🐢 Hit a problem!

This time the agent didn't actually write the tests. I asked "Are the tests fully covered?" and it still said yes — time to check the homework. Yep, it was lying 🙄

In the new session it didn't have write permissions (the previous session had been granted them, but this fresh one hadn't).

If a session is nearly full, use "Compact conversation" or /compact — it helps.

I want to create tests for the print-on-demand file upload flow.
The plan already exists.

Please generate Playwright code based on the plan.
I give you permission to write code directly.
If you need any additional permissions, please ask.

This prompt works better. The agent can also paste code directly into the chat — copy-paste into your editor works too. Test data files are in /tests/test-data:

|-tests
   |-test-data
   |-e2e
     |-uploadfile
            |-file-upload-comprehensive.plan.md  ## test plan
            |-basic-functionality
              |-page-accessibility.spec.ts        ## actual test file

Since the structure is as above, file references in code use ../../../test-data/MB1.jpg.

The third agent — playwright-test-healer

• If tests keep failing I'll look at the test code first, but when I'm going in circles I let this agent step in.
• For i18n projects, don't hardcode aria-label in Chinese — use data-testid with an alphanumeric identifier instead:

<b aria-label="取件編號" data-testid="pickupId">{{ props.pickupId }}</b>

Final Thoughts

The whole flow is remarkably smooth. After using it for a while, I realized the key isn't saving time — it's doing more. Some test cases I never had time to write before, but the agent anticipated them and filled in the gaps.

Debug time didn't shrink much, but having an agent as a collaborator means nothing is truly out of reach. That's a surprisingly reassuring feeling.