Author(s): Adi Insights and Innovations

Originally published on Towards AI.

I Tested Google’s Weird Discovery: “Thinking” Tokens Boost Accuracy Even When They Say Nothing Useful

A new Google Research paper found that reasoning models recall more facts correctly than instant models — even on questions that don’t require any reasoning at all. I reproduced the core experiment. Here’s the code, and what it actually means for how you should be calling LLM APIs.

The article explains Google’s findings that adding “thinking” tokens can improve factual recall even when the extra reasoning content is meaningless, and that this effect comes from two mechanisms: a computational buffer effect (extra internal computation boosts accuracy regardless of the reasoning text) and factual priming (reasoning through related facts helps retrieve the target answer). The author then builds a small reproduction using API calls with extended thinking enabled/disabled, scores responses with a lightweight judge model, and finds accuracy improves in reasoning mode while a filler-token control lands between instant and genuine reasoning. The author also flags a tradeoff: the same bridging mechanism that helps retrieve correct facts can amplify hallucinations, so outputs still need validation—especially for high-stakes use cases. Finally, the article offers practical takeaways for API builders and eval designers: don’t assume instant mode is sufficient for factual lookup, budget tokens/latency intentionally, pair reasoning with verification, and scale small evals up with larger question sets and additional judging before drawing conclusions.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI