Token cost benchmark for an autonomous Test-Failure Triage agent, across 26 models. Prices as of 26 Jul 2026.
| Model | $/1M in | $/1M out | Cost / outcome | Cost / month* |
|---|---|---|---|---|
| GPT-4o mini | $0.15 | $0.60 | $0.0276 | $276 |
| Llama 4 Maverick | $0.27 | $0.85 | $0.0480 | $480 |
| Gemini 2.5 Flash | $0.30 | $2.50 | $0.0646 | $646 |
| GPT-4.1 mini | $0.40 | $1.60 | $0.0736 | $736 |
| DeepSeek V4 | $0.43 | $0.87 | $0.0738 | $738 |
| Mistral Large 3 | $0.50 | $1.50 | $0.0884 | $884 |
| Qwen3.5 397B | $0.60 | $3.60 | $0.119 | $1,190 |
| Kimi K2.6 | $0.95 | $4.00 | $0.176 | $1,762 |
| Claude Haiku 4.5 | $1.00 | $5.00 | $0.191 | $1,912 |
| Grok 4.3 | $1.25 | $2.50 | $0.212 | $2,120 |
| Qwen3.7 Max | $1.25 | $3.75 | $0.221 | $2,210 |
| GLM-5.2 | $1.40 | $4.40 | $0.249 | $2,490 |
| Gemini 2.5 Pro | $1.25 | $10.00 | $0.266 | $2,660 |
| Mistral Medium 3.5 | $1.50 | $7.50 | $0.287 | $2,868 |
| Gemini 3.5 Flash | $1.50 | $9.00 | $0.298 | $2,976 |
| GPT-4.1 | $2.00 | $8.00 | $0.368 | $3,680 |
| Claude Sonnet 5 | $2.00 | $10.00 | $0.382 | $3,824 |
| GPT-4o | $2.50 | $10.00 | $0.460 | $4,600 |
| GPT-5.4 | $2.50 | $15.00 | $0.496 | $4,960 |
| GPT-5.6 Terra | $2.50 | $15.00 | $0.496 | $4,960 |
| Claude Sonnet 4.6 | $3.00 | $15.00 | $0.574 | $5,736 |
| Kimi K3 | $3.00 | $15.00 | $0.574 | $5,736 |
| Claude Opus 4.8 | $5.00 | $25.00 | $0.956 | $9,560 |
| GPT-5.5 | $5.00 | $30.00 | $0.992 | $9,920 |
| GPT-5.6 Sol | $5.00 | $30.00 | $0.992 | $9,920 |
| Claude Fable 5 | $10.00 | $50.00 | $1.91 | $19,120 |
*At 10,000 outcomes per month. Cheapest model highlighted.
The clean-path steps this benchmark prices:
This path runs 9 steps: 3 tool calls, 1 reasoning step and 5 decision points. Tool steps make two model calls each, and the agent re-reads its growing context on every call. That compounding is why one Test-Failure Triage outcome costs about 25x a single chat message ($0.574 on Claude Sonnet 4.6), not the price of one message.
These figures are modeled estimates, not metered bills. We price a generic, representative Test-Failure Triage workflow across 26 models using the same cost engine as the live estimator, at each model’s published list price (checked 26 Jul 2026), under documented default assumptions for planning loops, tool calls, memory retrieval, sub-agents and context size. Your own process will differ, so use these as starting points, tune the assumptions in the estimator, and validate against your real usage. Illustrative estimates, not financial advice.
On the clean path with default assumptions, an agent for Test-Failure Triage costs about $0.0276 to $1.91 per outcome depending on the model, or roughly $276 to $19,120 per month at 10,000 outcomes. The cheapest model here is GPT-4o mini at $0.0276; the most expensive is Claude Fable 5 at $1.91.
An agent does not make one model call. It plans, calls tools, retrieves context and re-reads its growing working context on every step. For Test-Failure Triage that adds up to about 25x the cost of a single chat message.
Across the 26 models benchmarked, GPT-4o mini is cheapest at $0.0276 per outcome and Claude Fable 5 is the most expensive at $1.91. A cheaper model is not always the right choice, but it sets the floor for this workflow.
The biggest levers are prompt caching on the base context, fewer planning loops, smaller tool results, less retrieval, and choosing a cheaper model where quality allows. You can test each lever in the live estimator.
These are modeled estimates, not metered bills. Each figure prices a generic, representative Test-Failure Triage workflow across 26 models with the same cost engine as the live estimator, at each model's published list price (checked 26 Jul 2026), under documented default assumptions for planning loops, tool calls, memory retrieval, sub-agents and context size. Your own process will differ, so treat these as starting points, tune them in the estimator, and validate against your own usage.
Cost is one axis. Before you build, check the process is ready for AI and that you can prove you govern it. See how governed AI process animations work →