Skip to content

Improve Playwright workflow guidance and reconcile CLI examples #57730

Description

@lpcox

Summary

Improve .github/aw/playwright.md so its Playwright CLI guidance is complete, unambiguous, and consistent with the public documentation.

Proposed improvements

  • Clarify whether network.allowed: local is required for loopback access. The guide says localhost and 127.0.0.1 work by default, while its local-application example includes the local ecosystem.
  • Explain that playwright-cli is automatically added to restricted Bash allowlists and that only supporting lifecycle commands such as npm, curl, and kill need explicit entries.
  • Document the default browser, supported alternatives (Chromium, Firefox, and WebKit), and named sessions where multiple browser sessions are useful.
  • Explain that files under /tmp, including screenshots, are ephemeral and must be published with an appropriate safe output such as upload-artifact when users need to retrieve them.
  • Qualify the accessibility-testing guidance: snapshots support structural and manual inspection, while comprehensive WCAG checks such as axe-core or programmatic contrast analysis require dependencies prepared before the agent runs. Runtime installation remains prohibited by the AWF sandbox policy.
  • Add concise troubleshooting guidance covering browser console output, network requests, tracing, and redirected local-server logs.
  • Reconcile docs/src/content/docs/reference/playwright.md with the current CLI syntax. It still includes MCP-style commands such as playwright-cli browser_navigate, browser_snapshot, and browser_take_screenshot; examples should use current commands such as goto, snapshot, resize, and screenshot.

Acceptance criteria

  • .github/aw/playwright.md clearly distinguishes automatic Playwright CLI access from user-configured Bash commands.
  • Loopback network requirements are stated consistently across Playwright and network documentation.
  • Browser support, artifact persistence, accessibility limitations, and failure diagnostics are documented concisely.
  • Public Playwright reference examples use valid @playwright/cli commands and agree with .github/aw/playwright.md.
  • Updated examples are checked against the pinned default @playwright/cli version.

Additional requirements

  • Define browser provisioning semantics explicitly. The compiler currently installs Playwright CLI skills with browser download disabled, while the public guide says Chromium, Firefox, and WebKit are available. Document which browser binaries are guaranteed to be prepared before the agent runs, and ensure every claimed browser works inside AWF without runtime installation.
  • Add compiler coverage proving that playwright-cli is automatically available with a restricted Bash allowlist and that loopback access works without network.allowed: local.
  • Validate every documented Playwright CLI command, option form, and required open/named-session lifecycle against the pinned default @playwright/cli version.
  • Include a complete safe-output example that publishes a screenshot created under /tmp, rather than only stating that temporary files are ephemeral.

Additional acceptance criteria

  • Browser availability claims match compiler provisioning, and tests cover each browser claimed to work without agent-time installation.
  • A compiled fixture verifies automatic playwright-cli Bash permission and loopback behavior without the local network ecosystem.
  • Documentation snippets are executable with the pinned default CLI version, including session setup and exact option syntax.
  • The guidance contains a working screenshot publication example using upload-artifact.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions