Work project · Desktop app
Autonomous Browsing
A browser automation agent that saves its runs as routines
Why I built it
A lot of operations work is web chores: open a site, search, check something, capture the result, repeat. Scripted automations can do this until a site changes its layout, and then they break, often without anyone noticing.
The goal was runs that finish without anyone watching and that the team can trust afterwards.
What it does
- Type a goal, and Claude drives a real Chrome to reach it, one action at a time.
- Save a finished run as a routine with parameters, which replays later without calling the model.
- Hands off to a person at a CAPTCHA, a login or a bot wall.
- Proposed repairs wait in a review queue and are never written back silently. Routines keep their revision history, with undo.
- A run queue, and resume from failure that never repeats a step that placed an order.
- Assert steps, so a finished run means the result was checked.
- Per-site memory of page landmarks that users can inspect and clear.
- Run history with a token and cost meter, and a health view per routine.
How it works
The app starts its own Chrome on a dedicated profile and attaches to it with Playwright over the DevTools protocol. A second backend drives the user's everyday Chrome through an extension and a native-messaging relay. Both inject the same in-page engine, so their safety checks can't drift apart.
On each step the model gets a filtered DOM and a compressed screenshot and picks the next action through a tool call. The DOM is cut to an interactive skeleton, screenshots are capped at 1024×768, and only the last two pages stay in the thread, which keeps runs cheap. Page content reaches the model inside a fresh random fence each turn, as a defence against prompt injection.
A routine replays without the model while it works. When a step breaks, the fix goes through four layers in order: re-ranking selectors, checking that the page actually reacted, an LLM repair, and finally a hold for a person.
Why build this when Zapier, Selenium scripts and browser agents exist?
Each of them covers part of the job. Zapier needs the site to offer an API or an integration. Selenium and Playwright scripts are fast but break when a page changes, and someone has to notice and fix them. A pure browser agent copes with changes, but it calls the model on every step, so every run is slower, costs money and can do something different from the last one.
Autonomous Browsing uses the agent to work out the task once, then saves the steps as a routine that replays without the model. The model comes back only when a step breaks, and its fix waits for a person to approve it.
Why drive a real Chrome instead of a headless browser?
The sites need the team's own logins, and a browser that announces itself as automated gets refused by sign-in pages such as Google's. The app starts a normal Chrome on its own profile, without the automation flag, and can also drive the user's everyday Chrome through the extension.
Where it stands
Version 0.1, used internally, with builds for Mac and Windows. Schedules and failure notifications are next.