The Capability Curve: What Accelerating AI Means for Developers

1. What the capability curve actually is

The term “capability curve” sounds abstract, but it has a precise empirical definition: it is the measured rate at which AI models are able to complete increasingly long, complex tasks on their own. The key metric comes from METR, an independent evaluation group that tracks task-horizon: the length of work an AI can finish reliably (at 50% success) in a single uninterrupted session.

In March 2024, Claude Opus 3 could handle tasks that take a skilled person about four minutes. Eighteen months later, Claude Sonnet 3.7 could sustain tasks taking about an hour and a half. By mid-2026, Claude Opus 4.6 reached twelve-hour tasks. Claude Mythos Preview, in METR’s own words, was “at the upper end of what METR can measure without new tasks” — working for at least sixteen hours.

That is not linear growth. The time horizon has been doubling roughly every four months, up from a prior trend of doubling every seven months. The curve is not just going up — it is steepening.

The same acceleration appears on coding benchmarks. SWE-bench — which gives a model a real open-source codebase and a real bug report and asks it to fix the issue — went from low single-digit scores to effectively saturated in two years. CORE-Bench, which tests whether a model can reproduce published research results, went from ~20% success in 2024 to saturated fifteen months later.

These are not cherry-picked metrics. Every capability that can be measured — code quality, task success on open-ended problems, research judgment — has followed the same curve. As of mid-2026, the curve shows no sign of bending.

2. Evidence from inside the machine

Abstract benchmark numbers become strategic information when you see what they look like from inside a company actually using the models. Anthropic has published unusually candid internal data on exactly this.

Before Claude Code launched in February 2025, Claude wrote code in the low single digits as a share of Anthropic’s production codebase. By May 2026, more than 80% of code merged to production was written by Claude. Engineers were not writing less — they were directing and reviewing an AI that was doing the typing. In Q2 2026, the median engineer was merging eight times as much code per day as in 2024.

A single concrete example makes the scale tangible. In April 2026, Claude shipped more than 800 fixes that reduced a class of API errors by a factor of one thousand. The human engineer overseeing the work estimated that doing it manually would have taken four years — not because the individual fixes were hard, but because holding that much unfamiliar context simultaneously exceeds human working memory. This is a category of problem AI can do that humans simply cannot, at any pace.

The same pattern shows up in research, not just engineering. In a live test, Claude-powered agents were given an open problem in AI safety — can a weaker model reliably supervise a stronger one? — with no predefined solution method. Two human researchers working a week recovered roughly 23% of the possible performance improvement. The agents recovered 97% over 800 cumulative hours at roughly $18,000 in compute cost. Direction-setting was the only meaningful human role.

🧠 Check your understanding

2 questions · your answers are saved in this browser only

1. 1. According to METR's task-horizon data, how often has the length of tasks AI can complete reliably been doubling (as of 2026)?

   Every 18 months Every 7 months Every 4 months Every 12 months

   As of mid-2026, the task horizon has been doubling roughly every four months, up from a prior trend of doubling every seven months — the curve is not just going up, it is steepening.

2. 2. Why did it take Claude — but not a human — to complete 800 API bug fixes in a tractable timeframe?

   Claude can write code faster line-for-line than any human The fixes required knowledge that humans do not have Humans struggle to hold the volume of unfamiliar context those fixes required simultaneously API bug fixing requires a specialized AI tool that humans cannot use

   The engineer overseeing the work estimated a human would have needed four years — not because individual fixes were hard, but because holding that much unfamiliar context in working memory at once is beyond human capacity. This is a category of work AI can do that humans cannot, at any pace.

Reset quiz

3. Three futures — and how to plan for each

The capability curve points to at least three distinct scenarios. Which one arrives — and when — is genuinely uncertain. Treating one as inevitable is a planning error; treating the uncertainty as an excuse not to plan is also a planning error.

Scenario 1: The curve is actually an S. Every exponential eventually bends. The judgment that separates a great researcher from a competent one might require a capability that does not emerge from simply scaling compute and data. A new architectural breakthrough — the equivalent of the Transformer in 2017 — might be needed to push past the current ceiling, and might not arrive. Anthropic considers this scenario unlikely given that every “squishy” qualitative capability (code quality, research judgment, open-ended task success) has followed the same upward arc with no sign of flattening, but it is possible.

Scenario 2: Compounding efficiency gains continue. AI development is largely automated, but humans continue to set research directions and judge results. A 100-person company does the work of a 10,000-person organization. The constraint is no longer execution speed — it is verification speed and judgment. Human comparative advantage lives in identifying which problems matter, interpreting ambiguous results, and catching AI errors before they propagate.

Scenario 3: Recursive self-improvement. AI systems become capable of fully designing their own successors. The pace of capability growth is determined not by human engineering cycles but by compute availability. This is the scenario Anthropic considers possible but not yet likely — and the one that justifies the “brake pedal” argument for coordinated industry slowdown.

For product builders, the critical observation is this: all three scenarios share the same near-term implication. In every scenario, the models available to you over the next one to two years will be substantially more capable than the ones available today. The question is not whether to plan for more capable models — it is how far out to extend that bet.

4. Betting on the model vs. engineering around limits

The strategic question the capability curve forces is concrete: when should you build a workaround for a current model limitation, and when should you bet that the next model release will make the workaround unnecessary?

Getting this wrong is expensive in both directions. Build too many workarounds and you ship complicated scaffolding that a model update makes obsolete — maintaining it costs more than it saves. Bet on the model too aggressively and you delay shipping real product behind a capability that may not arrive on your timeline.

A useful heuristic for drawing the line:

Engineer around limits when:

• The failure is structural, not capability-based (e.g., context window constraints, latency, cost per call)
• The workaround is simple and composable — it does not couple tightly to the current model’s specific behavior
• You have clear evidence the benchmark for this specific capability is not on the improvement curve (i.e., no measurable progress in recent model generations)
• The feature needs to ship this quarter

Bet on the model when:

• The failure is a “quality” problem — the model sometimes gets it right, and a better model would get it right more often
• The workaround is fragile — it requires enumerating edge cases that will grow with usage
• The capability is on a benchmark that has been improving rapidly (SWE-bench, METR task horizon, open-ended reasoning)
• The feature is for a 12-month roadmap item, not next sprint

One reliable test: ask whether the workaround depends on a specific model behavior or on a consistent model failure. Specific behaviors change between versions; consistent failures of a whole category (long multi-step reasoning, unseen API surfaces) tend to change only when a fundamentally new capability emerges.

🧠 Check your understanding

2 questions · your answers are saved in this browser only

1. 1. Your feature requires an AI model to reliably extract structured data from messy, variable-length PDFs. The current model succeeds about 60% of the time. What is the best strategic approach?

   Build a comprehensive rule-based fallback system since AI clearly cannot handle this reliably Ship nothing until the model reaches 100% accuracy on this task Assess whether success rates on this type of task have been improving across recent model versions; if so, add lightweight validation and bet on the model improving Switch to a different AI provider with better PDF support

   A 60% success rate with an improving trend is a "quality" problem, not a structural failure. The right move is to check whether this capability is on the improvement curve — if structured extraction accuracy has been rising across recent Claude versions, a modest validation layer plus a bet on future models is usually better than a brittle rule-based fallback system that will need rewriting anyway.

2. 2. Which of the following is a structural limit that justifies a workaround, rather than a model capability that is improving?

   The model sometimes hallucinates citations in long-form research tasks The model fails to plan correctly when given too many steps at once The API has a hard token limit per request that truncates inputs above a fixed length The model produces inconsistent tone when writing in a specific style

   A hard API token limit is a structural constraint, not a model capability. It will not be resolved by a more capable model alone — it requires chunking, summarization, or other architectural solutions. Hallucination, planning failure, and inconsistent tone are quality problems on the capability curve and may resolve with better models.

Reset quiz

5. Where human skills remain high-value

Even in the most optimistic reading of the capability curve, one pattern recurs in every future scenario: AI systems remain worst at the thing humans are best at last — judgment about which problems are worth solving.

Anthropic describes the narrowing human role with three hierarchical task types:

1. Execution tasks (“The export button is broken, fix it”) — these are almost fully automatable today. Claude handles them with a high and rising success rate.
2. Goal-with-method tasks (“Investigate why the network slows under load”) — Claude can now handle these reliably. The engineer specifies the goal; Claude designs and executes the approach.
3. Direction-setting tasks (“What should the team build next quarter?”) — this is where a meaningful capability gap remains. Claude can assist, generate options, analyze tradeoffs — but choosing which problems matter and when an answer is wrong still requires human judgment.

What makes this practically significant is Amdahl’s Law. When you automate the slow part of a process, the constraint moves to the next slowest part. Anthropic has already hit this wall: models generate code faster than engineers can review it. The bottleneck for an AI-heavy team stops being build speed and becomes verification speed and direction quality.

This points to specific skills that compound in value as model capabilities improve:

• Knowing what to ask for — prompt engineering and task decomposition remain load-bearing; a better model amplifies both good and bad instructions
• Recognizing when the answer is wrong — as AI writes more code, catches more bugs, and runs more experiments, the critical human skill becomes sanity-checking outputs that look right but aren’t
• Identifying which problems matter — the ability to spot the right question is now the primary rate-limiter on AI-assisted teams
• Managing verification pipelines — as Anthropic found, adding an automated Claude reviewer that pre-screens every code change is worth roughly a third of past incident prevention; designing these feedback loops is high-leverage work

6. Implications for product strategy

The capability curve is not an abstract observation about research benchmarks. It has direct implications for how you build AI products and how you think about the roadmap.

Design for model substitution. The model that powers your product today will not be the best model available in eighteen months. If your product logic is tightly coupled to specific model behaviors — a particular failure mode you’ve engineered around, a specific response format you’ve relied on — you will pay that debt at every upgrade. Build thin, stable interfaces to the model; let the prompts and evaluation criteria carry the weight.

Prioritize evaluation infrastructure early. As the model improves, your product improves for free — but only if you have evals that can tell you when it improved and when it regressed. Teams that built eval pipelines in 2024 are now seeing compounding returns. Teams that didn’t are flying blind every model release.

Shift budget from workaround engineering to judgment and verification. Given the trajectory, a dollar spent today on a workaround for a capability gap that will close in four months is a worse investment than a dollar spent on evals, review pipelines, and clear task decomposition that amplifies whatever the next model brings.

Give users leverage, not locks. As models improve, the ceiling on what AI can do for your users rises. Products that give users clear ways to direct and verify AI output will capture that rising ceiling automatically. Products that abstract the model into a fixed workflow will need to be redesigned at each step.

Plan for the constraint to move. If your team is bottlenecked on build speed today, that bottleneck will be resolved by AI. Plan now for the next constraint — code review, decision quality, customer trust in AI outputs — and build the infrastructure for it before you need it.

🧠 Check your understanding

1 question · your answers are saved in this browser only

1. 1. Anthropic's internal data showed that as Claude began generating more code, human code review became the new bottleneck. What principle does this illustrate?

   AI cannot fully replace human engineers Amdahl's Law: automating one bottleneck moves the constraint to the next slowest stage Code review is always the bottleneck in software development Claude-written code has more bugs than human-written code

   Amdahl's Law says the overall speed of a process is capped by its slowest stage. When Claude automated code writing, code review became the rate-limiter. The same pattern applies in any AI-augmented workflow: automating one step doesn't speed up the whole system unless you also address what comes next.

Reset quiz

🛠️ Build it yourself

Follow these exact steps to reproduce it yourself · estimated time: ~20 min

Prerequisites

• Access to Claude API or Claude.ai
• An existing AI feature or planned AI feature to evaluate
• Optionally: recent release notes for Claude model versions

Step 1: Map your current workarounds

List every place in your codebase or product logic where you have written code specifically to compensate for an AI model limitation. For each workaround, note:

• What model behavior it compensates for
• How complex the workaround is (lines of code, maintenance burden)
• Whether the behavior it compensates for is structural (token limits, API rate limits) or capability-based (quality of output, accuracy, reasoning)

Step 2: Check the trajectory

For each capability-based workaround, check whether that capability is on the improvement curve. Practical ways to do this:

• Find the relevant benchmark (SWE-bench for code, MMLU for knowledge, METR task horizon for autonomous work) and look at the last three model generations
• Run your specific test case against the current model and one version back — is accuracy improving?
• Search Anthropic’s model release notes and system cards for mention of the specific failure mode

Step 3: Classify each workaround

Sort your list into three buckets:

1. Keep: structural limits or capabilities with no improvement trend — maintain these
2. Monitor: capabilities that are improving but not yet good enough — set a threshold (e.g., “if success rate reaches 85% in production, remove the workaround”)
3. Delete soon: capabilities that have already improved past your threshold in the current model — schedule removal

Step 4: Build a lightweight eval for the “monitor” bucket

For each item in the “monitor” bucket, write 10–20 test cases that represent the real distribution of inputs your feature handles. Save the expected outputs. Run this eval:

• Against the current model
• Each time Anthropic releases a new model

This gives you automatic notification when the model has improved past your threshold — and protects you from regressions.

Step 5: Identify your next bottleneck

Ask: if AI handled everything in the “delete soon” bucket perfectly, what would the new bottleneck be? Common answers:

• Human review of AI-generated content
• Decision-making about which features to build
• Customer trust in AI outputs
• Cost of API calls at increased usage

For each identified next bottleneck, define one concrete investment you could make now to get ahead of it.