agent-browser
Public
Forked from mbagley/agent-browser
Forked from mbagley/agent-browser
Forked from mbagley/agent-browser
CONTEXT.md
CONTEXT.md
Snapshot of what was built, the bugs hit, and why the code looks the way it does. Written so a future session can pick up without rereading the chat.
An LM Studio plugin that exposes Vercel agent-browser
(vercel-labs/agent-browser, npm agent-browser, native Rust CDP CLI)
as model tools. Owner mbagley, plugin name agent-browser.
Built against the canonical LM Studio scaffold ā verified by cloning
lmstudio/wikipedia and lmstudio/dice via lms clone and matching
manifest.json, package.json, tsconfig.json, and the
PluginContext.withConfigSchematics / withToolsProvider registration shape.
manifest.json type=plugin, runner=node, owner=mbagley, name=agent-browser, revision=1 package.json @lmstudio/sdk 1.4.0, zod 3.24.1, typescript 5.6, @types/node scripts: dev, push, build, test:verify tsconfig.json CommonJS, ES2021, strict, rootDir=src, outDir=dist .gitignore .lmstudio, node_modules, dist README.md user-facing setup + tools table CONTEXT.md this file src/index.ts main(context) ā registers configSchematics + toolsProvider src/configSchematics.ts binCommand, session, headed, timeoutMs, screenshotDir src/agentBrowser.ts spawn-based CLI wrapper (PATH discovery + arg ordering) src/toolsProvider.ts 13 curated tools wrapping agent-browser subcommands test/verify-nytimes.js network integration test against nytimes.com
browser_open, browser_snapshot, browser_screenshot, browser_click,
browser_type, browser_press, browser_get_text, browser_page_info,
browser_wait, browser_find, browser_navigate, browser_eval,
browser_close. Curated from agent-browser's 100+ subcommand surface ā chosen
to be the set an LLM actually needs without polluting the model's tool list.
spawn npx ENOENTLM Studio's plugin runtime does not inherit the user's shell PATH.
spawn("npx", ...) fails because the plugin's process.env.PATH is something
like /usr/bin:/bin ā Homebrew's /opt/homebrew/bin (where npx lives on
Apple silicon) isn't on it.
Constraint from the user: do NOT bundle agent-browser as a plugin
dependency. The plugin must work via the host system's npx / agent-browser.
Fix (src/agentBrowser.ts): resolveAbsolute(cmd) resolves npx (or
whatever the user configures) to an absolute path before spawn:
Spawned children also receive an enriched PATH (enrichedPath()) so npx
itself can find node and resolve agent-browser.
WebSocket closed by the client on every tool callSymptom: plugin loaded fine, every tool call failed silently with the WS-close message in LM Studio.
Root cause: agent-browser's CLI parses subcommands first and global flags after the subcommand. The wrapper was emitting
agent-browser saw --timeout as the subcommand, exited with
Unknown command: --timeout in ~150ms, the tool returned an error string, and
LM Studio's tool-call layer surfaced "WebSocket closed by the client"
downstream.
Fix (src/agentBrowser.ts:152): subcommand args go first, global flags
last:
Verified by running the binary directly, then by the integration test below.
test/verify-nytimes.js, run via npm run test:verify:
| Field | Type | Default | Notes |
|---|---|---|---|
binCommand | string | npx agent-browser | Override to absolute path of npx or agent-browser if discovery fails. |
session | string | lmstudio | agent-browser --session name. |
headed | boolean | false | Adds --headed if true. |
timeoutMs | numeric (1000ā300000, int) | 30000 | Passed as --timeout (ms). |
screenshotDir | string | "" (cwd) | Where browser_screenshot writes files. |
userAgent | string | Chrome 131 / macOS UA | --user-agent. Blank disables. |
viewport | string | 1440x900 | WxH. Parsed into . |
createConfigSchematics() field types verified against the SDK's
basicKVValueTypesLibrary declaration in @lmstudio/sdk/dist/index.d.ts:
numeric accepts min, max, int, step; select takes options as
strings or {value, displayName} objects.
The five human-emulation fields are appended to globalFlags in
runAgentBrowser, so they tail the subcommand and respect invariant #2.
agent-browser applies these when its daemon launches Chrome; once a session
is already running they're effectively no-ops, so re-sending on every command
is safe (matches how --session/--timeout already work).
viewport and extraBrowserArgs both target --args and are merged into a
single comma-separated value via buildLaunchArgs. Empty fields drop out
individually ā clearing userAgent doesn't affect viewport, etc.
runAgentBrowser: [...prefix, ...args, ...globalFlags].
Do not reorder ā the CLI parses subcommands first.resolveCommand is async because the shell-fallback path uses execFile.npm run test:verify is meant for local validation; it makes
real network calls to nytimes.com and depends on Chrome for Testing being
installed via agent-browser install.browser_screenshot
and assert the file exists.LM Studio Hub: mbagley/agent-browser (revision 1, not yet pushed).
Snapshot of what was built, the bugs hit, and why the code looks the way it does. Written so a future session can pick up without rereading the chat.
An LM Studio plugin that exposes Vercel agent-browser
(vercel-labs/agent-browser, npm agent-browser, native Rust CDP CLI)
as model tools. Owner mbagley, plugin name agent-browser.
Built against the canonical LM Studio scaffold ā verified by cloning
lmstudio/wikipedia and lmstudio/dice via lms clone and matching
manifest.json, package.json, tsconfig.json, and the
PluginContext.withConfigSchematics / withToolsProvider registration shape.
manifest.json type=plugin, runner=node, owner=mbagley, name=agent-browser, revision=1 package.json @lmstudio/sdk 1.4.0, zod 3.24.1, typescript 5.6, @types/node scripts: dev, push, build, test:verify tsconfig.json CommonJS, ES2021, strict, rootDir=src, outDir=dist .gitignore .lmstudio, node_modules, dist README.md user-facing setup + tools table CONTEXT.md this file src/index.ts main(context) ā registers configSchematics + toolsProvider src/configSchematics.ts binCommand, session, headed, timeoutMs, screenshotDir src/agentBrowser.ts spawn-based CLI wrapper (PATH discovery + arg ordering) src/toolsProvider.ts 13 curated tools wrapping agent-browser subcommands test/verify-nytimes.js network integration test against nytimes.com
browser_open, browser_snapshot, browser_screenshot, browser_click,
browser_type, browser_press, browser_get_text, browser_page_info,
browser_wait, browser_find, browser_navigate, browser_eval,
browser_close. Curated from agent-browser's 100+ subcommand surface ā chosen
to be the set an LLM actually needs without polluting the model's tool list.
spawn npx ENOENTLM Studio's plugin runtime does not inherit the user's shell PATH.
spawn("npx", ...) fails because the plugin's process.env.PATH is something
like /usr/bin:/bin ā Homebrew's /opt/homebrew/bin (where npx lives on
Apple silicon) isn't on it.
Constraint from the user: do NOT bundle agent-browser as a plugin
dependency. The plugin must work via the host system's npx / agent-browser.
Fix (src/agentBrowser.ts): resolveAbsolute(cmd) resolves npx (or
whatever the user configures) to an absolute path before spawn:
Spawned children also receive an enriched PATH (enrichedPath()) so npx
itself can find node and resolve agent-browser.
WebSocket closed by the client on every tool callSymptom: plugin loaded fine, every tool call failed silently with the WS-close message in LM Studio.
Root cause: agent-browser's CLI parses subcommands first and global flags after the subcommand. The wrapper was emitting
agent-browser saw --timeout as the subcommand, exited with
Unknown command: --timeout in ~150ms, the tool returned an error string, and
LM Studio's tool-call layer surfaced "WebSocket closed by the client"
downstream.
Fix (src/agentBrowser.ts:152): subcommand args go first, global flags
last:
Verified by running the binary directly, then by the integration test below.
test/verify-nytimes.js, run via npm run test:verify:
| Field | Type | Default | Notes |
|---|---|---|---|
binCommand | string | npx agent-browser | Override to absolute path of npx or agent-browser if discovery fails. |
session | string | lmstudio | agent-browser --session name. |
headed | boolean | false | Adds --headed if true. |
timeoutMs | numeric (1000ā300000, int) | 30000 | Passed as --timeout (ms). |
screenshotDir | string | "" (cwd) | Where browser_screenshot writes files. |
userAgent | string | Chrome 131 / macOS UA | --user-agent. Blank disables. |
viewport | string | 1440x900 | WxH. Parsed into . |
createConfigSchematics() field types verified against the SDK's
basicKVValueTypesLibrary declaration in @lmstudio/sdk/dist/index.d.ts:
numeric accepts min, max, int, step; select takes options as
strings or {value, displayName} objects.
The five human-emulation fields are appended to globalFlags in
runAgentBrowser, so they tail the subcommand and respect invariant #2.
agent-browser applies these when its daemon launches Chrome; once a session
is already running they're effectively no-ops, so re-sending on every command
is safe (matches how --session/--timeout already work).
viewport and extraBrowserArgs both target --args and are merged into a
single comma-separated value via buildLaunchArgs. Empty fields drop out
individually ā clearing userAgent doesn't affect viewport, etc.
runAgentBrowser: [...prefix, ...args, ...globalFlags].
Do not reorder ā the CLI parses subcommands first.resolveCommand is async because the shell-fallback path uses execFile.npm run test:verify is meant for local validation; it makes
real network calls to nytimes.com and depends on Chrome for Testing being
installed via agent-browser install.browser_screenshot
and assert the file exists.LM Studio Hub: mbagley/agent-browser (revision 1, not yet pushed).
/opt/homebrew/bin, /usr/local/bin),
system bins, plus ~/.volta/bin, ~/.fnm/current/bin,
~/.nvm/current/bin, ~/.asdf/shims, ~/.bun/bin, ~/.local/bin,
~/.npm-global/bin, and ~/node_modules/.bin./bin/sh -lc 'command -v <cmd>' (login shell sources the
user's profile) if scanning misses.process.env.PATH to /usr/bin:/bin to mimic the LM Studio plugin
runtime (so PATH discovery is exercised, not bypassed by the developer's
shell).open https://www.nytimes.com ā snapshot -c -d 3 ā get title
ā get url ā close.[ref=e; title matches
/new york times/i; URL contains nytimes.com.--args "--window-size=W,H"acceptLanguage | string | en-US,en;q=0.9 | Becomes --headers '{"Accept-Language":"ā¦"}'. |
colorScheme | select | light | --color-scheme. no-preference is the CLI's own default, so it's omitted. |
extraBrowserArgs | string | --disable-blink-features=AutomationControlled | Comma-separated Chromium flags merged into --args. |
env: { ...process.env, PATH: enrichedPath() } so npx can
re-resolve descendants.signal.addEventListener("abort", ...) sends SIGTERM, and
the listener is removed on close/error. No SIGKILL escalation yet ā add one
if hangs surface.runAgentBrowser(settings, ["--version"], { includeSession: false, includeTimeout: false })
in toolsProvider (don't await) to pre-download.agent-browser --session lmstudio --timeout 30000 open https://...
agent-browser open https://... --session lmstudio --timeout 30000
cd /Users/mbagley/working/lmstudio-agent-browser-plugin
npm install
agent-browser install # one-time: download Chrome for Testing (host-side)
npm run dev # lms dev ā load into LM Studio
npm run test:verify # network integration test
/opt/homebrew/bin, /usr/local/bin),
system bins, plus ~/.volta/bin, ~/.fnm/current/bin,
~/.nvm/current/bin, ~/.asdf/shims, ~/.bun/bin, ~/.local/bin,
~/.npm-global/bin, and ~/node_modules/.bin./bin/sh -lc 'command -v <cmd>' (login shell sources the
user's profile) if scanning misses.process.env.PATH to /usr/bin:/bin to mimic the LM Studio plugin
runtime (so PATH discovery is exercised, not bypassed by the developer's
shell).open https://www.nytimes.com ā snapshot -c -d 3 ā get title
ā get url ā close.[ref=e; title matches
/new york times/i; URL contains nytimes.com.--args "--window-size=W,H"acceptLanguage | string | en-US,en;q=0.9 | Becomes --headers '{"Accept-Language":"ā¦"}'. |
colorScheme | select | light | --color-scheme. no-preference is the CLI's own default, so it's omitted. |
extraBrowserArgs | string | --disable-blink-features=AutomationControlled | Comma-separated Chromium flags merged into --args. |
env: { ...process.env, PATH: enrichedPath() } so npx can
re-resolve descendants.signal.addEventListener("abort", ...) sends SIGTERM, and
the listener is removed on close/error. No SIGKILL escalation yet ā add one
if hangs surface.runAgentBrowser(settings, ["--version"], { includeSession: false, includeTimeout: false })
in toolsProvider (don't await) to pre-download.agent-browser --session lmstudio --timeout 30000 open https://...
agent-browser open https://... --session lmstudio --timeout 30000
cd /Users/mbagley/working/lmstudio-agent-browser-plugin
npm install
agent-browser install # one-time: download Chrome for Testing (host-side)
npm run dev # lms dev ā load into LM Studio
npm run test:verify # network integration test