Autopilot gives any stdio MCP client (Claude Code, Claude Desktop, Cursor, VS Code Copilot, Codex, Gemini CLI) 26 browser tools against the Chrome that is already open on the machine: accessibility-tree reads with ref_N handles, input the page sees as trusted, screenshots, console and network capture, file upload, GIF recording. It was built to match Anthropic’s Claude in Chrome tool for tool, with one difference that matters at my desk: it runs in the background against sessions that are already logged in, so I keep using the browser while an agent works in it.

architecture

Three processes. Each MCP client gets its own server process, which speaks JSON-RPC over stdio to the agent and connects to a named pipe (a unix socket elsewhere). The native host is spawned by Chrome through native messaging and is the listener on that pipe, so several agent sessions share one browser. The extension’s service worker owns the native port, splits payloads above 384 KB because Chrome caps a native message at 1 MB, and routes each tool either to a content script or to chrome.debugger. Reads and form fills go through the content script in about 4 to 6 ms. Input and capture go through the debugger so events carry isTrusted.

The host journals every action and parks a response whose requesting socket died mid-call, so a session that reconnects gets its answer instead of a timeout.

technical decisions

• An extension instead of a launched browser. Playwright and CDP-only tools start their own Chrome and cannot reach the user’s profile. The extension reaches Gmail, Notion, internal dashboards and localhost with no credentials handed to anything.
• The host is the listener. An earlier design had each MCP server bind the pipe, and the second browser lost it. Chrome spawns the host, the host owns one pipe per browser, and servers connect to it.
• Background mode. New tabs open unselected in the current window and are never activated. A hidden or minimized tab is woken with Emulation.setFocusEmulationEnabled and Page.setWebLifecycleState, so it answers input in about 1 ms while the user keeps working in another tab.
• Screenshots from one screencast frame. Surface capture takes seconds on a hidden tab and the debugger API refuses renderer captures. A single screencast frame takes about 58 ms, and images are bounded by area to roughly 1,600 tokens.
• Timed waits run inside the page. A pending setTimeout does not keep a Manifest V3 worker alive, so waits go through Runtime.evaluate.
• Find is local and lexical. Sub-millisecond, no model call and no tokens, with escalation to MCP sampling when lexical matching is not enough.
• Permissions live in the extension. A financial blocklist, three modes, and a per-action origin re-check. A client cannot bypass them because the client never talks to the page directly.

hard problems

Iframe coordinate translation and stale offsets. Shadow roots and clipped elements in hit tests. chrome.debugger.onDetach after navigation. Stale sockets and handshake races between host and server. Soft-navigation waits, done as network idle plus DOM settle. Input.dispatchMouseEvent blocking for 5 s on a throttled renderer. Window occlusion dropping input. Chrome’s stale module cache under Default/Extension Scripts. Ten bugs from the last verification pass are still open in the handoff notes.

what a check looks like

npm run doctor verifies the pinned extension key, the native host manifest and its Chrome and Edge registrations, and that a browser with the extension is attached. This is the real output from my machine, with the profile lines removed:

  ok   manifest carries the pinned key
  ok   native host manifest written
  ok   host wrapper exists
  ok   manifest allows this extension id
  ok   wrapper points at a real node binary
  ok   Chrome registration
  ok   Edge registration
  ok   a browser is connected
         Chrome 152.0.0.0
  ok   Chrome extension is attached
         extension v0.2.0, 19 handlers, 26 tools advertised
All checks passed.

numbers