Picking the Right Model: A Practical Decision Framework

1. The model selection problem

Every time Anthropic ships a new model, developers face the same question: which one should I use? The instinct is often to reach for the most capable model — but that instinct is expensive, and frequently unnecessary.

The right framing is not “which model is best?” but “which model is best for this task?”. That distinction changes everything, because the answer depends entirely on:

1. What quality looks like for your specific output
2. How many tokens you process per day
3. How much latency your users will tolerate
4. What budget you have for inference

A model that is overkill for a classification task is exactly right for a multi-step reasoning task. A model fast enough for an interactive chatbot may be too slow for a synchronous API response in a checkout flow. Picking well means understanding both the task and the model family.

The Claude model family

As of mid-2026, Claude is organised into three capability tiers, each available as a point release (e.g., claude-haiku-4-5, claude-sonnet-4-6, claude-opus-4):

• Haiku — fastest, cheapest. Optimised for high-volume, latency-sensitive work where the task is well-defined: classification, extraction, routing, summarisation of short texts.
• Sonnet — balanced. The daily-driver for the majority of production workloads: code generation, multi-turn chat, document analysis, moderate-length reasoning.
• Opus — most capable, highest cost, higher latency. Reserved for tasks where quality is the only variable that matters: complex research, long-horizon agentic work, tasks that require deep multi-step reasoning.

2. What actually drives model choice

The selector between model tiers is not “difficulty” in the abstract — it is a set of measurable task properties. Learn to read these signals in your own workload:

Signal 1: Output sensitivity

Ask: “How bad is a wrong answer?” For a spam classifier, a wrong label costs almost nothing — you review flagged items anyway. For a contract clause analyzer, a wrong interpretation could carry real risk. High-sensitivity outputs push toward stronger models and always require an evaluation harness.

Signal 2: Reasoning depth

Count the number of independent reasoning steps the task requires. A sentiment label is one step. A strategy recommendation that must weigh competing constraints across a long document is ten. Tasks that require composing many sub-conclusions into a final judgment benefit most from Opus’s deeper reasoning.

Signal 3: Context length and coherence

Long-context tasks — summarising a 200-page report, reviewing a large codebase, multi-session research — stress working memory. At long contexts, quality gaps between model tiers widen. If your 90th-percentile request sends 50K+ tokens, test all tiers at that length, not at 2K.

Signal 4: Throughput and budget

Multiply your expected daily request volume by your average prompt + completion tokens, then by the per-token price for each tier. For high-volume pipelines, the cost difference between Haiku and Sonnet can be 10–20×. That spread often funds significant engineering work to make a smaller model perform adequately.

Signal 5: Latency budget

Streaming helps time-to-first-token feel fast, but total generation time matters for synchronous calls. Haiku generates tokens at roughly 3–5× the speed of Opus in practice. If users are waiting for a UI to unblock, measure end-to-end latency for your 95th-percentile token count on each tier.

3. Building a model comparison harness

The only way to make a confident model selection is to measure, not guess. You need a comparison harness: a script that runs the same inputs through multiple model tiers and produces comparable outputs you can score.

The three-layer evaluation stack

A minimal harness has three layers:

1. Golden dataset — 20–50 representative inputs sampled from your real distribution. Include edge cases and the hardest examples you know about. Do not cherry-pick easy inputs; you need your worst-case to be covered.

2. Automated metrics — whatever you can compute without human review: exact-match on classification labels, ROUGE or embedding similarity for summarization, JSON schema validation for structured outputs, unit tests for generated code. These give you fast signal on most failures.

3. Human review sample — 10–20 outputs per model tier, reviewed by someone who knows what “good” looks like. Focus reviewers on the cases where automated metrics diverge between tiers.

Running the comparison

import anthropic
import json

client = anthropic.Anthropic()

MODELS = {
    "haiku":  "claude-haiku-4-5",
    "sonnet": "claude-sonnet-4-6",
    "opus":   "claude-opus-4",
}

def run_on_model(model_id: str, system: str, user: str) -> dict:
    resp = client.messages.create(
        model=model_id,
        max_tokens=1024,
        system=system,
        messages=[{"role": "user", "content": user}],
    )
    return {
        "model": model_id,
        "output": resp.content[0].text,
        "input_tokens":  resp.usage.input_tokens,
        "output_tokens": resp.usage.output_tokens,
    }

# Run every golden-set example across all tiers
results = []
for example in golden_dataset:
    for tier, model_id in MODELS.items():
        result = run_on_model(model_id, SYSTEM_PROMPT, example["input"])
        result["tier"] = tier
        result["example_id"] = example["id"]
        results.append(result)

with open("comparison_results.json", "w") as f:
    json.dump(results, f, indent=2)

Once you have the results file, score each output against your golden answers using your automated metrics, compute per-tier aggregate scores, and calculate the cost for each tier at your production volume. The decision matrix usually becomes obvious.

4. Reading the cost model

After your comparison run, you have three numbers per tier: quality score, average latency, and token usage. Combine them with pricing to build a decision matrix:

In this fictional example, Sonnet buys a 7-point quality gain over Haiku for 7× the cost, while Opus buys only 3 more points for 5× the cost again. The marginal return on Opus is poor — unless those 3 points are the difference between acceptable and unacceptable for your use case.

The upgrade threshold

Use this rule of thumb: upgrade when the quality gap between tiers is larger than the noise in your evaluation. If your eval has ±3% variance (typical for a 20-example golden set), a 2-point quality difference between tiers is not a real signal — the difference could flip on the next run. A 10-point difference is real and worth paying for.

5. Model selection in agentic systems

Single-call tasks are the easy case. Agentic systems — where a model runs for many steps, calls tools, and accumulates a long context — add two complications:

Error propagation

In a multi-step agent, an early reasoning mistake compounds. A misread of step 2 leads to a wrong tool call at step 4, which surfaces as a completely wrong final answer. Stronger models make fewer early errors, so quality gaps are amplified in agentic pipelines compared to single-call tasks. This is why agentic orchestrators tend to use Sonnet or Opus even when individual steps seem simple.

Cost per task, not cost per call

A 10-step agentic task that calls the model 10 times with accumulating context costs 10× per run. At Opus prices, a task that feels inexpensive in a demo can become very expensive at scale. Common patterns:

• Orchestrator / subagent split: use Sonnet or Opus for the orchestrator (complex planning), Haiku for cheap subagents (web search parsing, data extraction, formatting).
• Speculative Haiku: try Haiku first; if the output fails a validation step, retry with Sonnet. This wins when most inputs are easy but a long tail is genuinely hard.
• Extended thinking on hard steps only: enable extended thinking for the one or two steps that require deep reasoning; use standard generation for the rest.

6. Staying current as models evolve

Anthropic ships model updates regularly. A selection you made six months ago may not be optimal today — in either direction. A newer Sonnet release might outperform the older Opus on your workload; a new Haiku might close the gap on tasks where you previously needed Sonnet.

When to re-evaluate

Trigger a re-evaluation whenever:

• A new point release ships for any tier you use
• Your workload composition changes significantly (e.g., average prompt length doubles)
• Your quality requirements change (e.g., you expand to a new use case)
• Your cost constraints change (e.g., you scale to 10× volume)

Keep your eval harness alive

The comparison harness you built in Chapter 3 is not a one-time script — it is a test suite for your model selection. Run it against new model versions the same way you run unit tests against new library versions. A harness that takes 5 minutes to run and costs $2 in API credits will save you far more in avoided mistakes.

mkdir model-selector && cd model-selector
python3 -m venv .venv && source .venv/bin/activate
pip install anthropic
export ANTHROPIC_API_KEY="sk-ant-..."

[
  {
    "id": "ex-001",
    "input": "Classify: 'I love this product, works perfectly!'",
    "expected": "positive"
  },
  {
    "id": "ex-002",
    "input": "Classify: 'Broke after two days, total waste of money.'",
    "expected": "negative"
  }
]

import json, time, anthropic

client = anthropic.Anthropic()

MODELS = {
    "haiku":  "claude-haiku-4-5",
    "sonnet": "claude-sonnet-4-6",
    "opus":   "claude-opus-4",
}

SYSTEM = "Classify the sentiment of the following text. Reply with exactly one word: positive, negative, or neutral."

with open("golden.json") as f:
    golden = json.load(f)

results = []
for example in golden:
    for tier, model_id in MODELS.items():
        t0 = time.time()
        resp = client.messages.create(
            model=model_id,
            max_tokens=10,
            system=SYSTEM,
            messages=[{"role": "user", "content": example["input"]}],
        )
        latency = time.time() - t0
        output = resp.content[0].text.strip().lower()
        results.append({
            "id": example["id"],
            "tier": tier,
            "model": model_id,
            "output": output,
            "expected": example["expected"],
            "correct": output == example["expected"],
            "latency_s": round(latency, 2),
            "input_tokens": resp.usage.input_tokens,
            "output_tokens": resp.usage.output_tokens,
        })
        print(f"{tier:8s} {example['id']} → {output!r:12s} {'✓' if output == example['expected'] else '✗'}")

with open("results.json", "w") as f:
    json.dump(results, f, indent=2)
print("Saved results.json")

import json
from collections import defaultdict

# Pricing in USD per million tokens (adjust to current rates)
PRICING = {
    "haiku":  {"input": 0.80,  "output": 4.00},
    "sonnet": {"input": 3.00,  "output": 15.00},
    "opus":   {"input": 15.00, "output": 75.00},
}

DAILY_REQUESTS = 10_000  # your expected daily volume

with open("results.json") as f:
    results = json.load(f)

by_tier = defaultdict(list)
for r in results:
    by_tier[r["tier"]].append(r)

print(f"\n{'Tier':8s} {'Accuracy':>10s} {'Avg latency':>13s} {'Est. daily cost':>17s}")
print("-" * 54)
for tier in ["haiku", "sonnet", "opus"]:
    rows = by_tier[tier]
    accuracy = sum(r["correct"] for r in rows) / len(rows) * 100
    avg_latency = sum(r["latency_s"] for r in rows) / len(rows)
    avg_in  = sum(r["input_tokens"]  for r in rows) / len(rows)
    avg_out = sum(r["output_tokens"] for r in rows) / len(rows)
    p = PRICING[tier]
    daily_cost = DAILY_REQUESTS * (avg_in * p["input"] + avg_out * p["output"]) / 1_000_000
    print(f"{tier:8s} {accuracy:>9.1f}% {avg_latency:>11.2f}s ${daily_cost:>15.2f}")

# Example: add to CI as a monthly scheduled job
# on: schedule: - cron: '0 9 1 * *'
python3 compare.py && python3 score.py