Web agents: macro vs micro work, and the routes to optimise each

TL;DR

• Web agents do two jobs. Macro work, open-web research: search, read many sites, answer. Micro work, in-site action: work inside one site and change its state. The field calls them deep research and web navigation (or computer use).
• They fail differently. On 365 open-web research runs, the main failure was a fluent wrong answer (157 runs, 43%). On 95 in-site action runs, the agent never changed a record it was not asked to touch; every wrong write was the right kind of change to the wrong thing (“which Dana?”).
• They use different tools. Research runs spend 8% of their tool calls on clicks, typing and key presses; action runs spend 35%.
• When one task does both (read a date on Wikipedia, then add it to a to-do app), the research half was the cheap part: 2 to 11 tool calls out of 37 to 143, in 6 runs on three cheap models. No run carried a wrong fact into the write.
• Verdict: the split holds, but on two axes, not one. Where the truth lives (the open web, or one site’s own data) decides how an agent fails. Whether the run writes decides what has to be gated.

Two jobs, one word

“Web agent” covers two different jobs:

• Open-web research. “Which gyms near Tompkins Square Park have classes before 7 am?” The agent searches, opens many sites, reads, cross-checks and writes an answer. Success is a correct, sourced answer.
• In-site action. “Move every contact at Acme still in Lead to SQL.” The agent works inside one site: logs in, finds records, fills forms, saves. Success is the site’s data, afterwards, being right, and nothing else having changed.

We wanted to know whether this split is real or just a naming habit, and whether a product should treat the two differently. We used four kinds of evidence: how public benchmarks and products divide the work, how our own runs fail, which tools each job reaches for, and a new task that does both.

Every run here used a cheap model: mostly glm-5.3-flash, with deepseek-v4.1-flash and gemini-3.8-flash where stated. The runs are from late September to 10 October 2026.

Read this first: the lanes are not equally hard. The research tasks are public AssistantBench questions on the live web, chosen because they are hard. The in-site tasks run on our own replicas of a to-do app and a CRM, and on four live news and shopping sites, and we wrote those tasks. So “18% vs 81%” does not mean research is four times harder. What does compare across lanes is how runs fail and which tools they use. The last section is the one comparison where we held difficulty fixed.

What the field calls these

The split is old and well established. It just has no shared name.

Where the two blur, grading gets harder. Online-Mind2Web (300 tasks on 136 live sites) mixes “find” and “do” tasks, and has to grade by human or model judgement of the trajectory, because nobody can read a live site’s database.⁹

What we did not find is a comparison of how the two jobs fail. Benchmarks grade one job or the other: Mind2Web 2 grades research on whether each claim is backed by a cited source,⁵ while ST-WebAgentBench grades in-site work on whether the final state is right and the site’s rules were kept.¹⁰ We could not find a survey or benchmark that runs both kinds of task through one agent loop and compares their failures. That comparison is what this post adds.

WebArena states the split most clearly. It sorts its tasks into information seeking, site navigation and content and configuration, grades the first by the answer and the last by “programmatic checks” of the final state.² The grading method follows the class. If the truth is on the web, you grade the answer. If the truth is in a site’s data, you grade the data.

Products drew the same line, then merged it:

The literature splits the jobs; the products merge them. OpenAI’s reason for merging is the hybrid: a research agent stalls at a login, an action agent cannot synthesise. We had no data on that hybrid, so we built a task for it (the last two sections).

On the name. Size is the wrong axis: one in-site task took one of our runs 85 tool calls. The field already has deep research (or agentic search) and web navigation (or computer use). We say open-web research and in-site action because the names say where the truth lives and match WebArena’s own split.

What our runs show

Pass rates by lane

An in-site action run cost a median of $0.0066 (mean $0.0111, p90 $0.0244, over 95 runs). Failed runs were cheaper than passing ones ($0.0043 vs $0.0086 median): in-site action mostly fails by stopping early with a wrong answer, not by grinding. An open-web research run cost about $0.032.

How each lane fails

Three findings.

1. Research fails on facts. Action fails on things. The main research failure is a fluent, wrong answer: 157 of 365 runs. Add the 62 abstentions and only 66 were right. In-site action never wrote outside its target. Its wrong writes were all entity resolution (which of two contacts called Dana?) or conditions (which contacts qualify for the bulk update?). None invented a record.

Both patterns match published work. BrowseComp measured calibration error, how far a model’s stated confidence sits from its accuracy, and found it higher with browsing: 82% for GPT-4o with browsing against 69% without, and 91% for Deep Research. Its authors suggest “access to web tools may increase the model’s confidence in incorrect answers.”¹ On the action side, ST-WebAgentBench found that three open agents completed 24.3% of its tasks on average, but only 15.0% while also keeping the site’s rules. Its examples include an agent asked to edit a CRM record that deleted a user instead, and one asked to create a single account that invented extra entries.¹⁰ Those agents date from 2024. Our runs are 2026 cheap models, and the failure has narrowed: no invented records and no stray deletions, but still the right change to the wrong thing.

So the two need different defences. Research needs source attribution and a grader that checks sources, which is why Mind2Web 2 grades attribution.⁵ Action needs a check before the write.

2. In-site reading is a third class, and it fails like research. The live in-site read lane never writes. Its worst failures look like open-web failures in miniature: links the page does not carry, wrong titles, and 4 runs that went off to a search engine when the answer was on the page. We added a check for that drift only because the agent does it on its own. So there are two axes, not one:

3. The tool mix flips. In research runs, 8.4% of tool calls are interactions (click, type, key press, select); the rest is navigating (30%), running scripts (28%) and reading the page (26%). In in-site action runs, interactions are 34.6% of 1,535 calls. The research-then-act runs land on the action side, at 31.1% of 463 calls. Research agents read. Action agents operate controls. A tool that makes reading cheaper mostly helps research; a tool that makes controls explicit mostly helps action.

Each job needs different tools

Open-web research needs search and a page cache. Live-web scores cannot be repeated: pages change, search results change. So we cache every page and every search, and charge each cache hit the time the real fetch took (see Agent performance and virtual clock). That tooling exists to make research measurable. In-site action does not need it, because a replica resets to a known state.

In-site reading needs typed readers. On the four live sites, a typed reader passed the same tasks for about a third of the tokens. Medians over 18 runs per arm, glm-5.3-flash:

This is the WebMCP idea, a page that declares its own tools (our post on it). Its first beneficiary is reading, which also helps open-web research once the agent lands on a known site.

In-site action needs approval before a change, and typed writes. Every write our action runs got wrong was a plausible write to the wrong thing. A person reading a card that says “set Dana Levi (haifaport-tech) lifecycle to SQL” catches that; a pass rate does not. Among seven agent products we surveyed, the ones that ask first ask before named risky classes (buy, publish, send).¹⁶¹¹ Ours asks per action, before any click that changes the page.

Typed tools for one site do not yet transfer to the next. A cheap-model generator of site tools passed 29 of 40 sites it had been developed on, and 2 of 7 it had never seen.

The hand-off: research, then act

OpenAI merged its products because of tasks that do both. The benchmarks hint that these are hard. BEARCUBS asks 111 questions that need real interaction on live pages to answer; people answer 84.7%, and the best agent it reports reached 65.8%.¹⁷ WebChoreArena adds 532 long, memory- and calculation-heavy chores to the WebArena sites.¹⁸ Neither isolates the moment a fact found in one place becomes a write in another. So we built a task that does.

The task. Read the Falkirk Wheel’s official opening date from its English Wikipedia article, then add a to-do “Falkirk Wheel anniversary” to the Personal project, due on that day and month in 2027. The date (24 May) is nowhere in the to-do app. The grader checks three things:

• the stored task: right project, due 2027-05-24, still open;
• nothing else in the app changed;
• the run reached Wikipedia and the text “24 May 2002” appeared in one of its tool results. This rules out a right answer from memory.

The replica normally refuses every connection to any other host. A task can now name the outside sites it needs, and only those are let through, via an outbound proxy system.

The runs. Six attempts across three cheap models, $0.66 in all:

It did not break at the hand-off. All six runs read 24 May from Wikipedia and carried it into the app. None wrote a wrong date. The failure we expected, a wrong fact turned into a correct-looking write, did not happen. The effort and the trouble were all on the write side:

• A login wall. In all five passing runs the app’s “Log in” dialog opened over the quick-add box mid-save; the agent signed in and carried on.
• The date picker. Three of the five passing runs gave up on the picker and set the date field’s value with a script.
• Checking its own work. Every passing run queried the app’s data 3 to 5 times to confirm the save.
• The one non-pass. gemini typed the task and date into quick-add, then started reading the app’s 324 KB JavaScript bundle to learn how it parses dates. Its input reached 306,210 tokens, it hit its $0.26 budget and was stopped before it pressed Enter. A budget cut, not a wrong write.

Same write, three lookups

To hold difficulty fixed, we ran the same write with three different lookups, graded by the same function:

• given: the date is in the prompt;
• URL given: the agent is told which Wikipedia page to read;
• find it: the agent is told to find the date on the web.

glm-5.3-flash:

The lookup is noise next to the write. With no lookup at all, the same write took 51 to 92 calls and cost $0.035 to $0.123. That spread is wider than any gap between rows. At this difficulty, adding research did not measurably change cost, calls or pass rate. With two or three runs per row, that is all these numbers can say.

The search engine was the wall, not the research. In the “find it” row, both agents first tried Google, which the task did not allow, then typed the Wikipedia address from memory. So that row never searched. We reran it on 2026-10-10 with the search engine named in the prompt and allowed. Through a basic proxy system, the search engine answered both runs with a bot challenge (“Unfortunately, bots use DuckDuckGo too”). One run gave up on it within two calls, searched Wikipedia’s own search box instead, and passed. The other spent 39 of its 50 tool calls running scripts against the challenge page, never reached the to-do app, and failed. This is the first research-side failure in any of our hybrid runs, and it is about access, not reasoning: on the open web, a search page can refuse an agent outright, and the agent has to notice and route around it.

What we did not prove

• Search access. Our one real search attempt hit a bot challenge on both runs. We have not yet measured search through an advanced proxy system or a search API, which is how a product would do it.
• The research half here was easy. One fact, an unambiguous date, on a page everyone knows. The harder case, where search returns a plausible wrong date, is not measured yet. That is where we expect the hand-off to break.
• Our research grader checks the answer, not its sources. Mind2Web 2 and DeepSearchQA show how to do better.⁵³
• We wrote the in-site tasks, so their pass rates are about our tasks, not about the web at large.
• Small samples. The hybrid and ladder rows are 2 to 6 runs each.

Prior art and how we differ

• WebArena already sorts tasks into seeking, navigating and changing, and grades each correctly.² We add a failure-mode comparison across those classes on one agent loop, a check that nothing else changed, and a research-then-act task with proof the fact was read off-site.
• Online-Mind2Web showed that scores from earlier live-site benchmarks are inflated: agents reported “close to 90%” on an earlier live benchmark, while on its 300 tasks the best, OpenAI’s Operator, reached 61.3% by human evaluation and most others about 30%.⁹ We avoid part of that by grading in-site work on data we can read, at the cost of writing the tasks ourselves.
• Mind2Web 2 and DeepSearchQA grade research on attribution and completeness.⁵³ Ours does not yet.

Verdict, in one line: the micro/macro split is real, but it is two axes, where the truth lives and whether the run writes, and at today’s difficulty the write, not the research, is where an agent spends its effort and makes its mistakes.