Skip to content

Feat/form update - #218

Open
ssricardo wants to merge 4 commits into
BrowserMCP:mainfrom
ssricardo:feat/form_update
Open

ssricardo wants to merge 4 commits into
BrowserMCP:mainfrom
ssricardo:feat/form_update

Conversation

@ssricardo

Copy link
Copy Markdown

Hey Namu!

Thanks for great idea, creating this extension + mcp.

Here's PR 1 (of 2) points that IMO can be improved.

Improvements for Forms

This PR aims to provide more info for the agents, in order fill out forms, especially the ones with dynamic behavior (like new fields created on the fly).

To do so (in short):

  1. Add also nearby labels (websites don't always the the label for'x' properly linked).
  2. Observe and react to changes in the page

In my tests in a dynamic form that we use in our company, it was way faster and correct than without the changes.
Without them actually it needed a lot of screenshots and sometimes couldn't make it.

Changes Description (from AI)

Description of the changes and their goals

What changed

This branch improves Browser MCP's form-filling workflow and gives agents clearer feedback after each interaction.

1. New form command

  • Lists all fields in a form, grouped by the question/label they answer.
  • Each field shows its ref, role, current value, required/invalid/disabled state, and available options (for selects and radio groups).
  • Also identifies "add row" buttons so agents can fill dynamic forms.

2. Better labels in snapshot -i

  • When Playwright's accessibility tree doesn't provide a proper label (common with <div> labels, legends, or placeholders), the snapshot now keeps the text right before a form control as a hint/question.
  • Makes controls readable in pages that don't use real <label> elements.

3. Page change reports after interactions

  • click, fill, press, select, check, and uncheck now report what appeared or disappeared on the page after the action.
  • This gives the agent immediate confirmation of whether the action had the intended effect.

Follow-up safety & performance fixes

Security

  • Change reports no longer echo typed field values, so passwords and other sensitive inputs never come back as (hidden).

Actually I wasn't sure about this in this PR, I thought it'd be in the other one

Performance

  • click and press report changed elements, note (no visible change) when nothing happens, and wait roughly a human reaction time.
  • fill, select, and check only report new/gone elements and skip waiting when the page is quiet, since forms are usually filled in batches.

Ref stability

  • The internal snapshot used for change reports no longer updates the document refs resolve against, so a ref from before a navigation correctly fails instead of accidentally pointing at the new page.

Files most affected

  • packages/mcp/src/commands/form.ts (new)
  • packages/mcp/src/commands/snapshot.ts
  • packages/mcp/src/commands/engine.ts
  • packages/mcp/src/commands/commands.ts

rsaturni and others added 4 commits October 28, 2026 20:13
…page change reports

- form: lists a form's fields grouped by the question they answer
- snapshot -i keeps the text right before a form control as its label
- click, fill, press, select, check and uncheck report what appeared or went away on the page
- reports no longer echo field values, so a typed password never comes back
- only click and press report changed elements and "(no visible change)", and wait a human's reaction time; fill, select and check report new and gone elements only and don't wait on a quiet page
- the report's snapshots no longer move the document refs resolve against, so a ref from before a navigation fails again
- form is named in the tool description, and shows password values as (hidden)
Keep page change reports while waiting for tabs the step opened.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant