Principles of AI Token Efficiency: Count Cost per Completed Task

Learning goal: Explain what a token is and which line items you pay for, calculate why agent-loop cost compounds, and plan and measure savings in order, from free levers to trade-off levers.

In a one-person studio that builds several products with Claude Code and Codex, AI cost should be read as "how much to finish one feature", not "how much per request". This document covers the principles; tool-by-tool practice is in "Saving Tokens in Practice". Prices and features are as of 2026-09 and include only what official documentation confirms. Dollar-to-won conversions use 1 USD = 1,400 KRW (an assumption for easy arithmetic).

Key Concepts

Tokens and tokenizers

A token is the unit a model processes text in. The tokenizer cuts text into tokens, and billing, context limits and speed are all set in tokens. The same text yields different token counts under different tokenizers.

Whether Korean, English or code costs more, and by how much, must be measured per model. The "Korean costs N times more" figures that circulate online were usually measured with a different tokenizer. Count Claude tokens with the model-specific count_tokens, not with OpenAI's tokenizer (tiktoken). It is free (only a requests-per-minute limit applies), so prepare the same content in Korean, English and code and measure it once.

import anthropic

client = anthropic.Anthropic()
for path in ["sample_ko.txt", "sample_en.txt", "sample_code.ts"]:
    text = open(path, encoding="utf-8").read()
    for model in ["claude-opus-5-5", "claude-haiku-4-5"]:
        n = client.messages.count_tokens(
            model=model,
            messages=[{"role": "user", "content": text}],
        ).input_tokens
        print(f"{path}\t{model}\t{len(text)} chars\t{n} tokens\t{len(text)/n:.2f} chars/token")

What you pay for

An API response's usage reports four token counts separately, each at its own rate. Total prompt size is input_tokens + cache_creation_input_tokens + cache_read_input_tokens; input_tokens is only the uncached remainder.

Principles

1. An agent loop resends the whole history every turn

The model remembers nothing between requests. Every turn, Claude Code and Codex resend the system prompt, tool definitions, and all prior conversation and tool results. With a fixed prefix S, growth of Δ per turn, and N turns:

total_input = N*S + D*N*(N-1)/2
assumption: S = 20,000, D = 3,000, N = 30, output 800 per turn
total_input = 600,000 + 1,305,000 = 1,905,000 tokens
output      = 30 * 800 = 24,000 tokens

Double N and input roughly quadruples. Prompt caching does not stop the resending; it reprices the prefix already processed. Assume turn 1 writes S and each later turn reads the previous prompt and writes only the new Δ: that is 1,798,000 read and 107,000 written tokens (sum 1,905,000).

(Each ×$ multiplies by the per-1M-token price, then divides by 1,000,000.) With caching on, the gap between the two models shrinks from 2x to about 1.6x, because both read cache at $0.20. In its published measurements, Anthropic reports that caching cut agent-loop cost by a factor of 2.7 to 5.3.

2. The context window is working memory

The context window is the working memory the model sees at once. A 1M-token model bills a 900k-token request at the same per-token rate as a 9k one, but being able to fit something is not a reason to include it. Unneeded context costs twice: once in tokens, and again in retries caused by lower quality.

• Liu et al., "Lost in the Middle" (TACL 2024): performance dropped significantly when the relevant information sat in the middle of a long input.
• Anthropic, "Effective context engineering for AI agents" (2025-09): as tokens grow, the ability to recall information from context accurately declines, a context rot that appears across all models; it advises finding "the smallest possible set of high-signal tokens" that maximizes the chance of the desired outcome.

3. The unit of cost is the completed task, not the request

When a cheap model fails, you pay its tokens, the retry, and your own time fixing it. Assuming you retry until success, cost per completed task = cost per attempt ÷ success rate. A setup at $0.20 per attempt with 50% success costs $0.40 per task; one at $0.30 with 90% success costs $0.33. The attempt is pricier; the task is cheaper.

In Anthropic's measurements (SWE-bench Pro, Claude Opus 5.5), running everything at low effort and re-running only the 13% that failed at high passed about 97% for about $0.17 per task, against 95.3% for $0.29 running everything at high. It only works when you have a failure signal (tests, a validator).

4. Lever order: free levers first, trade-offs later

Anthropic's cost-optimization guidance splits levers into two kinds. Free levers lower the price without lowering quality. Trade-off levers exchange intelligence for cost. The order matters.

flowchart TD
    M[Measure: usage and cost per task] --> F{Free levers used up?}
    F -->|No| C[Prompt caching]
    C --> I[Input hygiene: load only what is needed, progressively]
    I --> L[Loop hygiene: prune and isolate]
    L --> O[Output control: specify the shape]
    O --> B[Batch: work that can wait]
    B --> M
    F -->|Yes| E[Adjust effort]
    E --> R[Choose the model]
    R --> V[Check quality with an eval]
    V --> M

5. How to measure

Codex's usage carries cached_input_tokens and reasoning_output_tokens separately, so cache hits and reasoning tokens can be told apart (openai/codex source, checked 2026-09). The dollar figure in Claude Code's /usage is a list-price estimate; on a subscription plan, read it as plan-limit consumption rather than cost.

Applied: A Day in a Solo Studio

All numbers are assumptions, and Claude is assumed to run on an API key, converted to dollars. The Claude total is $1.37 + $0.54 + $4.50 = $6.41 (about 8,970 KRW). The lines to notice are not model prices but one cache miss (about 25x a hit) and work that could run overnight (halved by Batch). For gaps of 5 to 60 minutes the 1-hour TTL pays off. On models with cheap cache reads such as Opus 5.5 and Fable 5.1, though, a 5-minute TTL plus a max_tokens: 0 keep-alive can be cheaper (see the Cache break-even section below). Past an hour, as here, neither TTL helps, so /clear before lunch or accept the cold miss. Tool-by-tool methods are in "Saving Tokens in Practice".

Going Deeper

A tokenizer change distorts price comparisons

Moving from Sonnet 4.6 ($3/$15) to Sonnet 5 ($2/$10) cuts the input price by 33%, but the new tokenizer produces about 30% more tokens for the same text. Text that was 1,000,000 tokens under the old tokenizer becomes about 1,300,000, so 1.3×$2 = $2.60: about 13% cheaper than $3. When switching models, compare usage on the same task, not the price table.

Cache break-even

The 5-minute TTL already pays on the second request: 1.25 + 0.1 = 1.35 < 2 (two uncached requests). The 1-hour TTL pays from the third: 2 + 0.1×2 = 2.2 < 3. On Opus 5.5, reads are 0.05x, so 1.25 + 0.05 = 1.30. If requests start less than 5 minutes apart, the 5-minute TTL is always cheaper; at 5 to 60 minutes, the 1-hour TTL usually wins. The exception is models with cheap reads, such as Opus 5.5 (0.05x reads) and Fable 5.1 (0.025x). A keep-alive that stays on the 5-minute TTL and, during a pause, re-sends the previous request with max_tokens: 0 every 4 minutes pays only a read per refresh. In Anthropic's "Optimizing for cost and intelligence" measurements, on Opus 5.5 this cost 8-13% less than the 1-hour TTL when only 5-10% of turns followed a pause of 6 to 32 minutes (at medium effort), but more when every turn followed a pause. The Opus 5.5 measurements sent the keep-alive with max_tokens: 1; whether max_tokens: 0 refreshes an existing entry was not measured on Opus 5.5. On Fable 5.1 it cost 13-20% less when pauses ran for minutes, and those figures were measured with max_tokens: 0. A max_tokens: 0 request is rejected with invalid_request_error when combined with stream: true, thinking.type: "enabled", output_config.format (structured outputs), or a tool_choice that forces a tool ({"type": "tool"} or {"type": "any"}), or when sent inside Message Batches. On those workloads, use the 1-hour TTL instead of a keep-alive.

Thinking tokens bill at the output rate

Even if the visible answer is 500 tokens, 4,000 thinking tokens make the Opus 5.5 output cost 4,500×$20 = $0.09. Counting only the visible part gives $0.01, a 9x underestimate. Opus 5.5, Sonnet 5.5 and the Fable models cannot turn thinking off; effort is the only control (Claude Code docs, 2026-09).

Common Misconceptions

• "The cheap model is always cheaper." Cost per task = cost per attempt ÷ success rate, and caching narrows the gap between models.
• "Turn caching on and it just works." A timestamp or UUID in the system prompt, unsorted JSON, or a tool list that changes per turn breaks the cache without any error. Confirm with cache_read_input_tokens.
• "Writing in Korean costs N times more." It depends on the tokenizer. Trust only what you measured with count_tokens.
• "With a big context window, include everything." The price per token is the same, but quality can drop (context rot).
• "Lowering max_tokens saves money." The model never sees it. A response that hits it is a truncated failure that costs a retry. To shorten answers, specify the shape; to shorten thinking, lower effort.
• "/compact is free." The summarization request itself reads the whole conversation. After the cache has gone cold, it is expensive.

Self-Check Questions

1. What do you miss if you look only at input_tokens among the four usage fields and conclude "input was only 4K"?
2. With S = 10,000, Δ = 2,000 and N = 20, what is the total uncached input in tokens?
3. Which is cheaper per task: $0.10 per attempt at 40% success, or $0.25 per attempt at 95% success?
4. Which TTL fits a feature whose requests arrive about 20 minutes apart, and why?
5. Why do effort and model choice come last in the lever order?

References

• Pricing — Anthropic, checked 2026-09-29
• Prompt caching — Anthropic, checked 2026-09-29
• Token counting — Anthropic, checked 2026-09-29
• Optimizing for cost and intelligence — Anthropic, checked 2026-09-29
• Manage costs effectively · Commands — Claude Code Docs, checked 2026-09-29
• Effective context engineering for AI agents — Anthropic Engineering, 2025-09
• Lost in the Middle: How Language Models Use Long Contexts — Liu et al., TACL 2024
• Language Model Tokenizers Introduce Unfairness Between Languages — Petrov et al., NeurIPS 2023
• openai/codex exec_events.rs — OpenAI, checked 2026-09-29