The Expanding Agent Toolkit: From Scaffolding to Native Capability

1. The scaffolding problem

A year ago, building a useful agent meant writing a lot of code around the model. You wired up your own retry logic for tool calls that the model would occasionally fumble. You maintained a custom context-compression pipeline because the model lost coherence near the edges of its window. You hand-rolled a code sandbox because asking the model to write and then run code reliably required careful orchestration. And if you wanted the model to interact with a GUI, you were building your own screen parsing stack.

All of that scaffolding was necessary — but it was also fragile, expensive to maintain, and a source of subtle bugs that lived outside the model and were hard to evaluate.

The central argument of this talk: that scaffolding is dissolving. Not because it was bad engineering, but because the underlying model capabilities have advanced to the point where the scaffolding is no longer needed. Capabilities that required heavy external frameworks in 2024 are native and reliable in current models.

This matters beyond convenience. When a capability moves from scaffolding into the model, it becomes composable with everything else the model can do. A model that can natively use tools and natively manage context can combine those capabilities in ways that external wiring cannot, because the reasoning happens in one coherent place instead of being split between model and wrapper code.

2. The four layers of the agent toolkit

Before tracing how each capability evolved, it helps to have a map. The toolkit that modern agents draw on organises into four distinct layers, each with a different role.

These layers are not a rigid architecture you wire up — they describe where different capabilities live. The key insight is that all four now operate within a single coherent reasoning context. When an agent uses computer use to observe a browser (interface layer), parses what it sees (reasoning layer), calls an API to fetch supporting data (communication layer), and runs a script to process the result (execution layer), it is doing all of that in one continuous flow of reasoning — not across four separate systems stitched together with message-passing.

3. Tool use: from fragile to first-class

Tool use — the ability to call functions and act on their results — was the first major capability to go from scaffolding to native. Early model generations could be coaxed into calling tools, but the reliability required wrapper logic: retry loops for malformed calls, validation layers to catch parameter errors, fallback prompts when the model ignored the tool schema.

Current models treat tool use as a first-class primitive. What changed:

Schema adherence. The model reliably produces tool calls that match the schema on the first try. Parameter types are respected. Required fields are present. This alone eliminates most of the wrapper logic that earlier agent frameworks existed to provide.

Multi-tool reasoning. Rather than selecting one tool per turn, the model can reason about which tools to call, in what order, and with what parameters — and then execute that plan across several turns without losing track of intermediate results. A model that can call search, then fetch_page, then extract_table, then compute_summary in sequence, where each step informs the next, is qualitatively different from one that calls tools mechanically one at a time.

Parallel tool calls. When a task benefits from parallel information gathering, the model can issue multiple tool calls simultaneously and wait for all results before reasoning about them. This dramatically reduces wall-clock time for research-style tasks and reflects a real understanding of which operations have dependencies and which do not.

Tool call transparency. Because tool use is now a first-class part of the model’s output (rather than parsed from prose), it is also easier to observe, log, and evaluate. You can see exactly what the model called, with what parameters, and what it got back — which is the foundation for debugging and improving agentic workflows.

4. Context management: the long-game capability

The second major shift is in how models handle long contexts. Context management used to be almost entirely externalised: you summarised as you went, you maintained external memory stores, you paginated your way through large documents because the model would degrade near the context limit.

What changed is both quantitative and qualitative. The quantitative side is obvious — context windows expanded substantially. The qualitative side is more important: models now exhibit much more uniform performance across the context window. Early long-context models were long but shallow — they reliably used information from the first and last few thousand tokens and were inconsistent about what they retrieved from the middle. Current models retrieve reliably from anywhere in a large context.

This changes what agents can hold in mind. A research agent working through a long document corpus, a coding agent maintaining the full state of a complex refactoring, an orchestrator tracking the outputs of multiple subagents — all of these depend on the model reliably using context from throughout the window, not just the edges.

It also changes how you should think about memory. The instinct from the scaffolding era was to compress aggressively — summarise before the model forgets. The better instinct now is to be selective about what goes into context in the first place, and to trust that what does go in will be usable throughout the session.

Extended thinking pairs with this. For tasks where careful multi-step reasoning matters — complex debugging, architectural decisions, tricky data transformations — the model can spend more tokens reasoning before responding. This is not a workaround for capability limits; it is a deliberate allocation of compute to quality. The words think, think hard, think harder, and ultrathink map to increasing thinking budgets in Claude models.

5. Code execution: closing the write-run loop

The third layer is execution. Writing code is useful; writing code and running it is transformative. For a long time, getting a model to write a script and then actually execute it required an external sandbox — a separate service, a managed container, permissions scaffolding, and security review.

When code execution moved into the model’s native toolkit, two things happened:

The loop tightened. A model that writes code and immediately runs it can debug its own work. It sees the error, understands it, corrects the code, and re-runs — all in one reasoning context. The human is not in the debugging loop unless something requires judgment beyond what the model can bring. This is the mechanism behind tasks like “analyse this dataset and produce a summary” that would previously have required multiple back-and-forth turns.

Output quality improved. Because the model sees actual outputs — real numbers, real errors, real file contents — rather than imagined ones, the chain from task to result is grounded in reality at every step. Hallucinated results are much harder to produce when the model can check its work.

The same principle applies to bash and shell commands. An agent that can write a shell pipeline and check whether it produced the right output is fundamentally more capable than one that must hand off that step to external tooling. Claude Code’s design is built almost entirely on this insight: give the model bash, file read/write, and search, and the agentic loop from context to action to verification runs entirely within the model’s reasoning.

6. Computer use: the interface frontier

The fourth layer is the newest and still the most demanding: computer use. Where tool use calls APIs and code execution runs programs, computer use interacts with the visual layer — the same GUIs that humans use.

The capability is straightforward to describe: the model sees a screenshot, moves the mouse, types, clicks, and observes the result. It can operate web browsers, desktop applications, and any system that has a visual interface but no programmatic API. The applications are significant: legacy enterprise software, complex web workflows, multi-step form filling, and anything where a human has historically been the interface layer.

What makes computer use hard — and what differentiates capable from weak implementations — is partial observability and irreversibility.

A model using a GUI does not see the full application state the way it sees a JSON response. It sees a slice of the screen. Actions have consequences that may not be immediately visible. Some actions are irreversible: once you’ve clicked “confirm delete”, you can’t undo it by going back to the previous screen. The mental model from the talk: a video game, not a chess board. Chess is fully observable — you always know the complete game state. A video game is partially observable — you see only what is on screen, there are one-way doors, and there are game-over states.

This is why computer use requires metacognition: the model must not just act, but model its own uncertainty. Before clicking an irreversible button, it should recognise the irreversibility and, if appropriate, pause and confirm. When the visual state is ambiguous, it should acknowledge the ambiguity rather than guessing.

The implication for builders: computer use is the right tool when there is no API and a human would have to click through a GUI. It is not the right tool when there is an API — use the API. Computer use has higher failure rates, higher latency, and more complex error states than programmatic access. It is a last resort that is increasingly capable, not a first choice.

7. How the layers compose into agents that finish

The reason this talk is worth studying is not any single capability — it is the argument that these four layers together change what agents can reliably accomplish.

Consider a task that was hard to automate a year ago: “Review our GitHub issues from last week, pull the relevant code from our repo, write and run tests to understand the bug, and prepare a fix with a summary.”

With scaffolding-era tools:

• Tool calls required retry logic → extra code
• The context spanning issues + code + test output would degrade → summarisation pipeline
• Running the tests required an external sandbox → more infrastructure
• The whole flow required careful orchestration → an orchestration framework

With native toolkit capabilities:

• Tool calls (GitHub API, file read) are reliable on first call
• Context spanning issues + code + test output is managed in one coherent window
• Tests run and error output comes back directly into reasoning
• The orchestration is the model — it plans, acts, observes, and iterates

The practical consequence: agents can now complete work instead of just starting it. The old pattern was “the agent does step 1, hands off, human does step 2, hands back.” The new pattern is “the agent does the whole thing and reports when it is done — or when it genuinely needs human judgment.”

This is not magic, and it does not mean agents are always right. It means the bar for what an agent can handle end-to-end has risen substantially, and the investment you make in good task design, clear permission scoping, and meaningful verification signals pays off in tasks actually reaching completion rather than stalling.

Look at this codebase. Find any function that takes a list as input and
returns a filtered result. Write a test that exercises each one with both
an empty list and a list of five elements. Run the tests and report which
ones pass and which ones fail.

claude mcp add github -- npx @modelcontextprotocol/server-github

Check if any of our open GitHub issues mention the functions you found.
If so, add a comment in the code next to that function referencing the issue number.

# Create a lightweight integration check
cat > check.sh << 'EOF'
#!/bin/bash
npm test -- --testPathPattern="the-functions-you-found" 2>&1 | tail -5
EOF
chmod +x check.sh

Run check.sh after every change you make. Only report done when all checks pass.

Read every file in this directory. For each file, note what it exports and
what it imports. Then produce a dependency graph in text form, showing which
files depend on which others. Finally, identify any circular dependencies.

Where to go next

• Watch the original talk to see the live capability demonstrations.
• Understand the communication layer deeply with MCP 201: How the Model Context Protocol Really Works.
• See how the execution layer powers the full agentic coding workflow in Mastering Claude Code.
• Learn what computer use looks like in practice in Giving Agents Their Own Computers.