How to Calculate Cost Per Correct Answer for LLM Reasoning Tasks

Accuracy alone is no longer enough when evaluating Large Language Models (LLMs) for complex tasks. If a model solves 90% of math problems but costs ten times more than a competitor that solves 85%, you are likely wasting budget. The industry has shifted from asking "Is it right?" to asking "What does it cost to get it right?" This metric, known as cost per correct answer, combines accuracy with inference pricing to reveal the true efficiency of an AI system.

This approach is critical for reasoning tasks-like coding, advanced mathematics, and logical deduction-where models often use long, verbose thought processes (Chain-of-Thought) to arrive at a solution. These methods boost accuracy but explode token counts. Without tracking the dollar value of each correct output, you cannot optimize your deployment strategy effectively.

The Formula: Defining Cost Per Correct Answer

To benchmark this metric, you need a simple formula that bridges technical performance and financial reality. The core calculation is straightforward:

For example, if running a batch of 100 math problems costs $10.00 and the model answers 80 correctly, your cost per correct answer is $0.125. A cheaper model might cost $5.00 total but only answer 60 correctly, resulting in a higher cost per correct answer of $0.083. Wait-in this specific case, the cheaper model is actually more efficient ($0.083 vs $0.125). This counter-intuitive result highlights why raw accuracy percentages can be misleading without the cost context.

Step-by-Step Benchmarking Methodology

Calculating this metric requires rigorous control over variables. Small changes in prompting can drastically alter both token usage and accuracy. Follow these steps to ensure fair comparisons between models like GPT-4o is OpenAI's flagship multimodal model optimized for speed and lower cost compared to previous GPT-4 iterations. and Claude 3 Sonnet is Anthropic's mid-tier model designed to balance high intelligence with competitive pricing.

1. Select a Standardized Benchmark: Use established datasets with fixed questions and ground-truth answers. For reasoning, common choices include:
   • GSM8K: 8,500 grade-school math word problems.
   • MATH: 12,500 competition-style math problems requiring multi-step reasoning.
   • HumanEval: Coding tasks that test function generation.
   • MMLU: Massive Multitask Language Understanding covering diverse subjects.
2. Standardize Prompting: Keep temperature, max tokens, and few-shot examples identical across all models. As noted by researcher Cameron Wolfe, even minor formatting changes in prompts can skew scores. Use the same Chain-of-Thought (CoT) instructions for every model to ensure they "think" similarly before answering.
3. Track Token Usage Precisely: Log both input (prompt) and output (completion) tokens for every single query. Tools like vLLM or provider SDKs provide this data automatically. Do not rely on averages; sum the exact counts.
4. Apply Current Pricing: Convert tokens to USD using the latest provider rates. Prices change frequently. For instance, OpenAI reduced GPT-4o prices significantly in May 2024. Always verify current rates on provider dashboards.
5. Calculate Accuracy: Run the benchmark’s official scoring script to determine the number of correct answers.
6. Compute the Metric: Divide the total dollar cost by the number of correct answers.

Why Reasoning Tasks Are Different

Reasoning tasks differ from simple classification or summarization because they require intermediate steps. Models often generate hundreds or thousands of tokens of "scratchpad" work before producing the final answer. This verbosity drives up costs.

Consider the impact of Chain-of-Thought prompting. On the GSM8K dataset, adding CoT instructions can boost accuracy from ~70% to ~90%. However, it may also increase output tokens by 5x to 10x. If the price per token is high, the cost per correct answer might actually increase despite the higher accuracy. You must find the Pareto frontier-the point where additional spending yields diminishing returns in accuracy.

New benchmarks are emerging to address this complexity. OckBench is A benchmark introduced in 2025 that jointly measures accuracy and token efficiency for reasoning and coding tasks. explicitly reports metrics like "tokens per correct answer," which serves as a direct proxy for cost when multiplied by per-token rates. Similarly, MMMR is Massive Multi-Modal Reasoning, a benchmark targeting logic, math, code, and science across visual and text domains. expands this evaluation to multi-modal scenarios, where image processing costs must also be factored in.

Model Comparison: Efficiency vs. Capability

Not all models are created equal when it comes to cost efficiency. Frontier models offer high accuracy but at a premium. Mid-tier models often provide the best balance. Here is how different tiers typically perform based on 2024-2025 pricing trends:

In many cases, mid-tier models like GPT-4o achieve a lower cost per correct answer than frontier models. They sacrifice only a few percentage points of accuracy but reduce token costs by 5x to 10x. Entry-level models are cheap per token, but their lower accuracy means you need more retries or post-processing, which can negate the savings.

Pricing Trends and Market Context

The landscape of LLM pricing is shifting rapidly. According to Epoch AI’s 2024 analysis, the cost to achieve specific performance milestones on benchmarks has fallen by a median of 50x per year. In some extreme cases, prices dropped by 900x annually. This deflationary trend benefits users but complicates long-term budgeting.

Key drivers include:

• Hardware Improvements: Newer GPUs and specialized accelerators process tokens faster and cheaper.
• Model Architecture: Techniques like mixture-of-experts (MoE) allow models to activate only necessary parameters, reducing compute load.
• Competition: Providers like OpenAI, Anthropic, and Google aggressively cut prices to capture market share.

However, these trends are uneven. Complex reasoning tasks still require significant compute. As models become smarter, they may also become more verbose, potentially offsetting hardware gains. Always re-benchmark when new model versions are released.

Common Pitfalls to Avoid

When calculating cost per correct answer, avoid these frequent errors:

• Igoring Input Tokens: Long prompts (especially with many few-shot examples) can dominate costs. Ensure your prompt length is consistent across tests.
• Excluding Retries: In production, models often fail and require retrying. A model with 90% accuracy might have a much higher effective cost if 10% of queries trigger expensive second attempts.
• Mixing Prompt Styles: Comparing a model forced to give short answers against one allowed to reason extensively is unfair. Standardize the instruction set.
• Overlooking Latency: A cheap, slow model might not meet user expectations. Factor in time-to-first-token if real-time interaction is required.

Practical Implementation Tips

To implement this benchmarking in your workflow:

1. Automate Logging: Use infrastructure tools like LangSmith, Weights & Biases, or custom logging scripts to capture token counts and costs for every request.
2. Test Multiple Configurations: Evaluate each model with and without Chain-of-Thought, with different temperature settings, and varying max token limits. Plot accuracy vs. cost to find the optimal operating point.
3. Use Routing Strategies: Deploy a classifier to send easy queries to cheap models (e.g., Haiku) and hard reasoning tasks to powerful ones (e.g., Opus). This hybrid approach minimizes average cost per correct answer.
4. Monitor Price Changes: Set up alerts for API price updates. A sudden price hike can make your previously optimal model uneconomical overnight.