Reasoning Models
Reasoning models are LLMs that spend extra computation thinking before they answer — working through a problem step by step in an internal scratchpad, then producing a final response informed by that reasoning. This "thinking" phase is why they solve hard math, multi-step logic, complex coding, and planning tasks that trip up a model answering in one shot.
The underlying idea is test-time compute: instead of only making the model bigger (train-time compute), you let it spend more computation at inference on hard problems. A model that can "think longer" on a difficult question often beats a larger model answering instantly. This page explains how reasoning works, how to control it, and — importantly — when not to pay for it.
TL;DR
- Reasoning models generate an internal thinking phase before the final answer, improving accuracy on hard, multi-step problems.
- This is test-time compute: spending more inference computation on hard problems, complementary to making models bigger.
- Adaptive thinking lets the model decide how much to think per request; an effort control (low → max) tunes the depth/cost tradeoff.
- Reasoning costs tokens and latency — the thinking phase is billed and adds a pause before output. It's a lever, not a default for everything.
- Use it for math, logic, complex coding, planning, and agentic work; skip it for simple lookups, classification, and chat where it just adds cost.
- The raw chain of thought is often summarized or hidden; you get a readable summary, not the verbatim reasoning, and you don't prompt for the reasoning text directly.
Quick Example
Enable adaptive thinking and set an effort level; the model reasons, then answers:
The model works through the problem in the thinking phase, then gives a final answer grounded in that work.
Core Concepts
Test-Time Compute
Two ways to make a model smarter: spend more compute at training time (bigger models, more data) or at test time (let it think longer per query). Reasoning models exploit the second. On hard problems, allowing a model to generate a long internal reasoning trace before answering can match or exceed the accuracy of a much larger model responding instantly. The insight — that "thinking longer" is a distinct, powerful axis — reshaped how frontier models are built.
The Thinking Phase
A reasoning model produces two kinds of output: thinking blocks (the internal reasoning — planning, calculating, self-correcting) and the final response (the answer the user sees). The thinking is where the model catches its own errors, breaks the problem down, and explores approaches. Crucially, this reasoning is generated tokens — it costs money and time, and it counts against the output budget.
Adaptive Thinking and Effort
Modern reasoning models don't force a fixed thinking budget. Instead:
- Adaptive thinking lets the model decide per request how much to think — a trivial question gets little, a hard proof gets a lot. You turn it on and let the model self-pace.
- Effort (typically
lowthroughmax) is a separate dial for the overall depth and token spend. Higher effort means deeper reasoning, more tool calls in agentic settings, and higher cost; lower effort means faster, cheaper, more scoped work.
Together they replace the old "set a fixed thinking-token budget" approach with "let the model adapt, and tune the ceiling with effort."
Hidden vs Summarized Reasoning
Providers increasingly return a summary of the reasoning rather than the verbatim chain of thought, or omit it entirely by default. You typically opt in to a summarized view for display. Two consequences: don't build logic that depends on parsing exact reasoning text, and don't prompt the model to "output your reasoning as the answer" — that's not how you access it, and some models refuse attempts to extract raw reasoning.
When Reasoning Helps (and When It Doesn't)
Deeper reasoning has a clear cost, so match it to the task:
💡 Tip: Don't reflexively max out effort. Higher settings can cause a model to over-think simple tasks — burning tokens and adding latency for no gain. Start at a moderate setting and raise it only when you see shallow reasoning on genuinely hard problems.
Best Practices
Match Effort to Task Difficulty — and Measure
The right effort level is empirical. Sweep low/medium/high on your own eval set and pick per route by the accuracy-vs-cost tradeoff. The relationship isn't monotonic: higher effort up front can reduce total cost on agentic work by cutting the number of turns.
Give Room in max_tokens
The thinking phase consumes output tokens. If max_tokens is too small, the model can spend its whole budget thinking and get truncated before the final answer. Budget generously at high effort — and stream, so a large budget doesn't hit request timeouts.
Leave Thinking On for Hard Interactive Work
For coding assistants and agents, keeping adaptive thinking on generally improves quality more than the latency costs. If you disable it for speed, expect shallower answers on complex requests.
Don't Prompt for the Reasoning as the Answer
You access reasoning via the thinking/summary channel, not by asking the model to "show all your work in the response." Asking a reasoning model to dump its internal chain of thought as output is both ineffective and, on some models, refused.
Consider a Cheaper Model for Simple Steps
In a multi-step system, route trivial sub-steps to a fast, low-effort model and reserve deep reasoning for the hard steps. Reasoning is a premium capability — spend it where it moves the needle.
Common Mistakes
Maxing Effort on Everything
Starving the Answer of Tokens
Treating Thinking as Free
The thinking phase is billed and adds latency before the first visible token. In a latency-sensitive chat UI, the default "omitted" reasoning display looks like a long pause. Budget for it and consider a summarized display so users see progress.
FAQ
What's the difference between a reasoning model and a regular LLM?
A reasoning model spends extra computation on an internal thinking phase before answering, which sharply improves accuracy on multi-step problems. Regular one-shot generation answers immediately. Many modern frontier models are both — they reason when it helps (adaptive thinking) and answer directly when it doesn't.
Does more thinking always mean better answers?
No. Thinking helps on hard, multi-step problems where errors compound. On simple tasks — classification, lookups, transformation — it adds cost and latency without improving the answer, and can even cause over-thinking. Match effort to difficulty and measure the tradeoff on your own evals.
What is "effort" and how do I set it?
Effort (typically low to max) tunes how deeply the model reasons and how much it spends. Higher effort suits hard coding, math, and agentic tasks; lower effort suits routine or latency-sensitive work. Start moderate, sweep levels on your eval set, and raise it only where you see shallow reasoning on genuinely hard inputs.
Can I see the model's reasoning?
Usually only a summary, and often only if you opt in — the verbatim chain of thought is frequently hidden or summarized. Don't depend on parsing exact reasoning text, and don't prompt the model to output its reasoning as the answer; that's not how the channel works.
Why is there a long pause before the answer?
The model is in its thinking phase, generating internal reasoning before the visible response. With reasoning display off (a common default), you see nothing until the answer starts. Enable a summarized display to show thinking progress, or accept the pause as the cost of deeper reasoning.
Related Topics
- Large Language Models — The foundation reasoning models extend
- AI APIs & Tool Use — Where the thinking and effort parameters live
- AI Agents — Long-horizon agentic work leans hardest on reasoning
- AI Agent Frameworks — Frameworks that orchestrate reasoning across steps
- Prompt Engineering — Steering behavior alongside reasoning
- Context Engineering — Reasoning tokens consume the context/output budget
- LLM Evaluation — Measuring whether more reasoning actually helps