Getting More Out of the Claude Platform

1. The platform features developers overlook

Most developers start with the Claude API by making a messages.create call and iterating on their prompt. That gets you far. But there are four platform-level capabilities that can cut costs dramatically, extend what your agents can handle, and increase effective intelligence — without changing your prompt at all.

This lesson covers each one in detail:

1. Prompt caching — cache the expensive parts of your prompt and pay 90% less on re-reads.
2. Tool search — give agents access to thousands of tools without loading all their definitions into context.
3. Context compaction — let long-running conversations summarize their own history before hitting the context limit.
4. The advisor strategy — use a cheap model to generate candidates and a strong model only to select the best one.

Together these form a layered cost and intelligence optimization approach. You can adopt any of them independently, or combine them for compounding effect.

2. Prompt caching: pay once, re-read cheaply

Every time you make an API call, the full input is processed from scratch — unless you use prompt caching. Prompt caching lets you mark specific prefixes of your prompt as cacheable. On the first request, the prefix is written to a cache and costs 1.25× the normal input rate. Every subsequent request that hits the same cache reads those tokens for just 0.1× the normal rate: a 90% discount on cached tokens.

What can be cached

Nearly any block in your request can be cached: system prompts, long user messages, tool definitions, images, documents, and assistant turns. The one hard rule is that you mark the cache breakpoint at the last stable block — the last block whose content is identical across requests. Everything after the breakpoint is processed fresh each time.

The minimum cacheable size varies by model (1,024 tokens for Sonnet and Opus 4.x, 4,096 for Haiku 4.5), so very short system prompts won’t benefit. Check cache_creation_input_tokens and cache_read_input_tokens in the usage response to confirm caching is active.

The single most common mistake

Developers often put cache_control on the user message itself — the part that changes on every request. That guarantees zero cache hits because the prefix hash never matches.

# Wrong: breakpoint on content that changes every request
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": user_query,  # Changes on every request
                "cache_control": {"type": "ephemeral"},  # Never hits cache
            }
        ],
    }
]

# Correct: breakpoint on the stable system prompt
response = client.messages.create(
    model="claude-opus-4-8",
    system=[
        {
            "type": "text",
            "text": LARGE_STABLE_SYSTEM_PROMPT,  # Same for all users
            "cache_control": {"type": "ephemeral"},  # Cached after first request
        }
    ],
    messages=[{"role": "user", "content": user_query}],  # No cache_control here
    max_tokens=1024,
)

Multi-breakpoint caching

You can place up to four cache_control markers in a single request, one for each section that changes at a different frequency. A common pattern: one breakpoint after the system prompt (stable forever), one after a document that’s loaded per-user-session, and none on the per-turn messages.

response = client.messages.create(
    model="claude-opus-4-8",
    system=[
        {
            "type": "text",
            "text": BASE_SYSTEM_PROMPT,          # Changes: never
            "cache_control": {"type": "ephemeral"},
        },
        {
            "type": "text",
            "text": USER_SPECIFIC_DOCUMENT,      # Changes: per user session
            "cache_control": {"type": "ephemeral"},
        },
    ],
    messages=conversation_history,               # Changes: every turn
    max_tokens=1024,
)

Cache TTL and pre-warming

The default cache lifetime is five minutes (auto-refreshed to five minutes on every hit at no extra cost). For workloads where bursts arrive more than five minutes apart, you can pay 2× the base input rate for a one-hour TTL:

"cache_control": {"type": "ephemeral", "ttl": "1h"}

For the very first request of a cold deployment — before any users arrive — you can pre-warm the cache with max_tokens: 0. This absorbs the one-time write cost before users experience latency:

# Pre-warm: no output, just populate the cache
client.messages.create(
    model="claude-opus-4-8",
    max_tokens=0,
    system=[{"type": "text", "text": SYSTEM_PROMPT, "cache_control": {"type": "ephemeral"}}],
    messages=[{"role": "user", "content": "warmup"}],
)

3. Tool search: hundreds of tools, lean context

A typical multi-server agent setup — GitHub, Slack, Sentry, Grafana, Splunk — can consume 55,000 tokens in tool definitions before Claude has done any actual work. That’s expensive, and it hurts tool selection: Claude’s accuracy at picking the right tool degrades significantly once the catalog grows past 30–50 entries.

Tool search solves both problems. Instead of loading all tool definitions upfront, you mark most tools as defer_loading: true and add a special tool search tool. When Claude needs a tool, it issues a search query; the API returns the 3–5 most relevant definitions inline, and Claude calls from that focused set. Tool search typically reduces context consumed by tool definitions by over 85%.

Two search variants

The tool search tool comes in two flavors:

• Regex (tool_search_tool_regex_20251119): Claude constructs Python regex patterns to search tool names and descriptions. Good when tool names follow predictable conventions.
• BM25 (tool_search_tool_bm25_20251119): Claude uses natural language queries. More forgiving when descriptions are prose.

Both search tool names, descriptions, argument names, and argument descriptions.

Enabling tool search

response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=2048,
    messages=[{"role": "user", "content": "Send a Slack message to #incidents"}],
    tools=[
        # The search tool itself — never defer this
        {"type": "tool_search_tool_bm25_20251119", "name": "tool_search_tool_bm25"},

        # Frequently used tools: keep non-deferred (loaded every request)
        {"name": "read_file", "description": "Read a file from disk", "input_schema": {...}},

        # Large catalog: defer everything else
        {
            "name": "slack_send_message",
            "description": "Send a message to a Slack channel",
            "input_schema": {...},
            "defer_loading": True,  # Only loaded when discovered
        },
        {
            "name": "github_create_issue",
            "description": "Create a GitHub issue in a repository",
            "input_schema": {...},
            "defer_loading": True,
        },
        # ... hundreds more with defer_loading: True
    ],
)

Combining tool search with prompt caching

Deferred tools are not part of the system-prompt prefix, so adding defer_loading: true does not break your existing cache breakpoints. The tool search tool itself and any non-deferred tools can still be cached normally.

4. Context compaction: agents that never fill up

In an agentic loop, every tool call and response accumulates in the context window. Left unchecked, long-running agents will eventually hit the limit and fail. The naive fix — truncating history — loses important state. Context compaction is a better approach: when the conversation approaches the limit, Claude automatically summarizes the older history into a compact compaction block, preserving the gist without the token cost.

Enabling compaction

Add the beta header and set a context_management edit in your request:

response = client.beta.messages.create(
    betas=["compact-2026-01-12"],
    model="claude-opus-4-8",
    max_tokens=4096,
    messages=messages,
    context_management={
        "edits": [
            {
                "type": "compact_20260112",
                "trigger": {
                    "type": "input_tokens",
                    "value": 100_000,   # Compact when input exceeds 100k tokens
                },
            }
        ]
    },
)
# Append the full response (including any compaction block) back to messages
messages.append({"role": "assistant", "content": response.content})

When compaction fires, the response contains a compaction block. On subsequent requests the API automatically drops all message blocks that precede it — you don’t have to implement any history trimming yourself.

Tuning the trigger

The default trigger is 150,000 input tokens (minimum: 50,000). Set it lower for tighter memory management; higher if you want Claude to retain more raw history before compacting. For interactive chat where users expect continuity, pause_after_compaction: true lets you inject a user acknowledgement between the summary and the continued response.

Tracking usage across compaction iterations

The standard usage.input_tokens field only reports the current iteration. When compaction has fired one or more times, sum all entries in usage.iterations to see the true total token consumption across the conversation:

total_input = sum(it["input_tokens"] for it in response.usage.iterations)

5. The advisor strategy: cheap model, strong model

The advisor strategy is a multi-model prompting pattern that stretches your intelligence budget. Instead of routing every request directly to your most powerful (and most expensive) model, you use a cheaper model to generate a set of candidate answers and ask the stronger model only to select the best one. The selector does far less work — no generation, just evaluation — so it can usually run on a much smaller context, costing far less.

The pattern is especially valuable for:

• High-volume classification or ranking — you need the quality of an expensive model but can’t pay for it on every request.
• Evaluation pipelines — a small model generates N outputs; an expert model judges them.
• Redundancy on critical decisions — multiple independent model opinions reduce hallucination risk.

Implementation skeleton

import anthropic

client = anthropic.Anthropic()

ADVISOR_MODEL = "claude-haiku-4-5"   # Fast and cheap: generates candidates
EXPERT_MODEL  = "claude-opus-4-8"    # Smart and expensive: selects the best

def advisor_strategy(user_question: str, n_candidates: int = 3) -> str:
    # Step 1: advisor generates N candidate answers in parallel (or in one call)
    candidates = []
    for _ in range(n_candidates):
        resp = client.messages.create(
            model=ADVISOR_MODEL,
            max_tokens=512,
            messages=[{"role": "user", "content": user_question}],
        )
        candidates.append(resp.content[0].text)

    # Step 2: expert selects the best candidate — it never has to generate from scratch
    numbered = "\n\n".join(f"[{i+1}] {c}" for i, c in enumerate(candidates))
    selector_prompt = f"""You are an expert evaluator. Choose the best answer to the question below.
Reply with only the number of the best answer and a one-sentence reason.

Question: {user_question}

Candidates:
{numbered}"""

    selection = client.messages.create(
        model=EXPERT_MODEL,
        max_tokens=128,
        messages=[{"role": "user", "content": selector_prompt}],
        system=[{
            "type": "text",
            "text": "You evaluate answers for accuracy and clarity.",
            "cache_control": {"type": "ephemeral"},  # Cache the stable expert system prompt
        }],
    )
    return selection.content[0].text

Cost comparison

Suppose your baseline is sending every request to Claude Opus 4.8 at $5/MTok input. With the advisor strategy and three Haiku candidates:

The selection call is far shorter than an equivalent generation call because the expert model does not need to think through the problem — it only reads and judges. Across high-volume pipelines this can reduce expert-model costs by 60–80%.

import anthropic, json

client = anthropic.Anthropic()

def baseline_call(user_message: str) -> dict:
    resp = client.messages.create(
        model="claude-opus-4-8",
        max_tokens=1024,
        system=YOUR_SYSTEM_PROMPT,
        messages=[{"role": "user", "content": user_message}],
    )
    return {
        "input_tokens": resp.usage.input_tokens,
        "output_tokens": resp.usage.output_tokens,
        "cache_read_input_tokens": getattr(resp.usage, "cache_read_input_tokens", 0),
        "cache_creation_input_tokens": getattr(resp.usage, "cache_creation_input_tokens", 0),
    }

samples = [baseline_call(q) for q in TEST_QUERIES]
print(json.dumps(samples, indent=2))

CACHED_SYSTEM = [
    {
        "type": "text",
        "text": YOUR_SYSTEM_PROMPT,   # Must be >1,024 tokens for Claude Opus/Sonnet
        "cache_control": {"type": "ephemeral"},
    }
]

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    system=CACHED_SYSTEM,
    messages=[{"role": "user", "content": user_message}],
)
print("Cache write:", resp.usage.cache_creation_input_tokens)
print("Cache read: ", resp.usage.cache_read_input_tokens)

tools = [
    # Always-loaded: the search tool itself
    {"type": "tool_search_tool_bm25_20251119", "name": "tool_search_tool_bm25"},

    # Always-loaded: your 3–5 most commonly called tools
    {"name": "read_file", "description": "Read a file from disk", "input_schema": {...}},

    # Deferred: everything else
    *[
        {**tool_def, "defer_loading": True}
        for tool_def in LARGE_TOOL_CATALOG
    ],
]

messages = []

def agent_turn(user_input: str) -> str:
    messages.append({"role": "user", "content": user_input})

    resp = client.beta.messages.create(
        betas=["compact-2026-01-12"],
        model="claude-opus-4-8",
        max_tokens=4096,
        system=CACHED_SYSTEM,   # Reuse your cached system prompt
        messages=messages,
        context_management={
            "edits": [{"type": "compact_20260112", "trigger": {"type": "input_tokens", "value": 80_000}}]
        },
    )

    # IMPORTANT: append the full response, including any compaction block
    messages.append({"role": "assistant", "content": resp.content})

    # Track total tokens across compaction cycles
    total_input = sum(it["input_tokens"] for it in (resp.usage.iterations or [{"input_tokens": resp.usage.input_tokens}]))
    print(f"Turn tokens (total across iterations): {total_input}")

    return resp.content[-1].text if resp.content else ""