-
Notifications
You must be signed in to change notification settings - Fork 4.7k
move data between web apps with an ai agent
When neither app has an API, an AI agent can move data between them: read each row on the source, transform it, type it into the target, then read the target back to confirm the write landed. It suits a few hundred rows, not a pipeline; where both sides have an API, the API always wins.
An agent earns its keep in three situations: neither app has an API (common on internal tools and smaller products), an API exists but sits behind a plan nobody pays for, or the job is a one-off, a few hundred rows moving from an export into a new system, not a recurring feed.
Outside those three, do not reach for an agent. If both apps expose an API, call it: a request succeeds or fails as a whole and costs a fraction of a model call. If the target is a spreadsheet rather than a second app, getting website data into Google Sheets covers that easier, more common case.
The task is four repeating stages: read a row from the source, transform its values into the shape the target expects, write it into the target's own form, then read the target back to confirm what landed. Extracting data to a CSV covers the first stage alone; getting an agent to fill out forms covers the third; this task chains both, plus the confirmation neither needs alone.
The transform stage is where migrations quietly go wrong. A source column named "full name" becomes "first" and "last" on the target. A date stored as "03/14/2026" needs to become "2026-03-14".
State the field mapping explicitly, one line per field, rather than trusting the model to guess a correspondence; a guessed mapping is the same wrong-value-in-wrong-field failure the forms page describes for filling alone.
A submit button returning a success page is not evidence the row landed the way you meant it to. Target apps silently drop fields they do not recognize, truncate a string past a length limit, or apply a default when a required field arrives empty. None of this produces an error the agent would see; it produces a saved row that looks fine and is not.
The fix costs one more step per row. After the write, have the agent reload the record on the target and read back the fields that matter, then compare them to the transformed source values. Two or three fields are usually enough, chosen from the ones that would hurt if wrong: dates, identifiers, anything with a length limit.
Batch the run and stop between batches; do not ask for the whole migration in one instruction. AIHawk's own loop has no turn ceiling, so "all of them" will not be cut off part way - it will simply run on, and that is the problem rather than the reassurance it sounds like. One full read-transform-write-confirm cycle is several turns, every turn resends the whole transcript, so the cost per row climbs the longer the batch runs; and a long unattended run is the one where a bad mapping writes forty wrong rows before anyone looks. "The next twenty rows, then stop" is short enough to read the log before the next twenty start.
Writing dozens of rows back to back is also the kind of steady, gap-free rhythm that gets a session rate-limited or logged out mid-batch. The order-of-checks for a session that stops responding is on why does my AI agent get blocked, and a short pause between batches, not only within one, is the cheapest fix; agent retry loops and rate limits covers what happens when a failed write gets retried instead of paused.
Some row will not fit the target's rules: a required field the source never populated, a value the target's validation rejects, a duplicate it refuses outright. Decide the policy before the first row runs, or the agent invents one, often "guess a plausible value," which is worse than skipping the row.
Three policies work: skip the row and log why, needing a human pass over the skipped list afterward; stop the whole batch at the first failure, safest and slowest; or write the row with the problem field left empty and flagged, which keeps the count moving. State the policy in the instruction itself: "if a required field is missing, skip the row and note it, never invent a value" is worth one sentence.
The deliverable of a migration like this is not only the rows that landed; it is a record of what happened to each one. Ask the agent to log, per row: the source identifier, the transform applied, the target's confirmation (an ID, or "confirmed by read-back"), and the outcome: written, skipped, or failed.
A plain text or CSV log next to the moved data turns "trust the agent ran" into "read forty lines and see exactly what it did." Verifying the output applies here twice: once per row during the run, and once more across the whole log before the migration is called done; that final check is a human's job.
Can an AI agent migrate data from one system to another? Yes, for a bounded job: it reads each row on the source, transforms it, types it into the target, and reads the target back to confirm the write. It is not a substitute for an API integration where one is available on both sides.
How do I know the data was actually written correctly? By having the agent read the target back after each write and compare the fields that matter to the source, rather than trusting a success message. A saved row that looks fine can still be missing a field the target quietly dropped.
What happens if a row does not fit the target's format? Whatever policy you stated in the instruction: skip and log it, stop the batch, or write it with the bad field flagged. Without a stated policy the agent tends to invent a value to make the row fit, which is the outcome to avoid.
See also: getting an agent to fill out forms, extracting data to a CSV with an AI agent, and getting website data into Google Sheets.
-
feder-cr/AIHawk, plus its source in this repository (
src/aihawk/agent.py), retrieved 2026-09-05 and the loop re-read 2026-09-08, for what the batching section above rests on: a loop with no turn ceiling, and the whole transcript resent on every turn.
From the AIHawk wiki. A success message and a correctly saved row are two different facts, and the read-back-to-confirm step exists because only one of them is worth trusting.
- OpenAI Operator alternatives
- Open-source Operator-style agents
- Is OpenAI Operator still available?
- OpenAI Operator vs Claude computer use
- browser-use alternatives
- Choosing an AI browser agent
- Open-source AI browser agents
- Open-source computer-use agents
- What is an AI web agent?
- AI browser agents vs traditional scraping
- Cloud browser infrastructure for AI agents, explained
- Browserbase alternatives
- Firecrawl vs an AI browser agent
- Skyvern alternatives
- Stagehand vs browser-use
- Project Mariner is gone: what replaced it
- Manus alternatives
- Gemini computer use vs Claude computer use
- AIHawk, reviewed honestly by its own wiki
- AI browser vs AI browser agent: which one do you want?
- AI browser agent vs RPA: which one fits the job
- AI browser agent vs n8n, Zapier and Make
- Vercel agent-browser alternatives, compared honestly
- What is an agentic browser? Definition and the two kinds
- Open-source agentic browsers: the three layers, compared
- Choosing an MCP server for browser automation: four axes
- Stealth MCP servers compared: Camoufox, nodriver, Patchright
- Playwright MCP alternatives, and the three you don't need
- Autonomous browser agents: the four rungs of autonomy
- What is actually free in the AI browser agent stack
- browser-use on GitHub: what the repo actually gives you
- Playwright MCP vs Chrome DevTools MCP: different jobs
- How to choose among MCP servers: a map by category
- Which MCP servers are worth adding to Claude Code
- MCP on GitHub: finding servers and judging them fast
- MCP vs an API: the decision, and what the wrapper costs
- MCP alternatives: when the protocol is the wrong shape
- Why does my AI agent get blocked?
- The timing signal AI agents give off
- Agent retry loops trip rate limits, not fingerprints
- Claude computer use detected as a bot
- browser-use getting blocked: what you can and cannot change
- Playwright MCP session blocked: four causes, four fixes
- Playwright MCP and captchas: what actually gets you past
- Cloudflare and a browser MCP server: what is being read
- Can an AI agent solve a captcha? The honest answer
- Getting an AI agent to fill out forms
- Which model to use with AIHawk
- Browser problem or model problem?
- Running AIHawk's browser from Claude Code
- Extracting data to a CSV with an AI agent
- Monitoring a page for changes with an AI agent
- Running AIHawk's browser from Claude Desktop
- Running AIHawk's browser from Cursor
- Using an AI agent to hunt for apartments
- Getting website data into Google Sheets with an AI agent
- Using an AI agent to download invoices from portals
- AI agents for web research
- Using an AI agent to test your own website
- Running AIHawk's browser from Cline
- Posting to social media with an AI agent
- Posting to Facebook with an AI agent
- Posting to Instagram with an AI agent
- Posting to X with an AI agent
- Automating LinkedIn posts: read this first
- Appointment bots: what they are and what an agent can legitimately do
- Track prices across sites with an AI agent
- Build a lead list with an AI browser agent
- Run an AI browser agent on a schedule
- AI browser agent with a local LLM: what changes
- Should you log your AI agent into your accounts?
- How to write a task an AI browser agent can follow
- Move data between two web apps with an AI agent
- The MCP server
- How the tools are shaped, and why
- Playwright MCP vs the Playwright CLI: which fits when
- Playwright MCP: browser is already in use, and the fix
- Playwright MCP best practices: four decisions that matter
- Playwright MCP with a proxy, and the three leaks it leaves
- A browser MCP server in GitHub Copilot: setup and limits
- Using a browser MCP server for web scraping: the pattern
- Which LLM for browser automation: the four properties
- How to build a browser agent, and what to take instead
- Getting an AI agent to log into a website: three routes
- MCP tools, resources and prompts: who controls each
- How many MCP tools is too many? The context arithmetic
- How to build an MCP server: the decisions, not the scaffold
- Local or remote MCP server: what changes, and what does not
- Writing an MCP client in Python: the thirty-line version
- Self-hosted AI agent: what one actually costs to run
- How long an AI browser agent takes per step, measured
- Text, HTML, snapshot or screenshot: what the agent should read
- Giving an AI browser agent a stopping condition
- Keeping an AI browser agent out of destructive actions
- Why did the AI agent click the wrong thing
- When the page changes under the AI agent
- Running one AI agent task across a list of sites
- Seeing a page as it appears in another country
- Getting data out of a dashboard with no export button
- Two browsers in one session: main and support
- Finding the dead links on a site with an AI agent
- Filling a CRM record from a company's website
- One form submission per spreadsheet row, with an AI agent
- Dated screenshots of a page as evidence
- Checking order and delivery status with an AI agent
- Reading a PDF that opens inside the browser
- Summarising a long page or thread with an AI agent
- Collecting every image on a page with its caption
- Collecting event and course listings with an AI agent
- Cancelling a subscription with an AI agent
- What an AI agent can and cannot do inside an iframe
- Shadow DOM and an AI agent: you can click it, you cannot read it
- What a page snapshot costs, per control
- Native selects and the ones that only look like selects
- Clicking by selector or by coordinates
- How long the agent waits before it gives up
- What a second browser costs
- Uploading a file with an AI agent, and why this one cannot
- Watching the agent work, and when it is worth it
- When not to use an AI browser agent
- Agent or script: deciding once instead of every time
- Using the keyboard instead of the mouse
- Secrets in an agent task: where they end up
- What an agent run should log
- Deduplicating what an AI agent collects
- Normalising values across sites
- Validating an AI agent's output
- Reading a table with an AI agent
- Driving a site's own search and filters
- The task works headed and fails headless