The Best AI Agent Predicts Future World Events at 25% Accuracy.
Last Updated on June 14, 2026 by Editorial Team
Author(s): Dr Swarneendu AI
Originally published on Towards AI.
FutureSim replays real-world news events in chronological order and asks agents to predict what happens next. The Brier skill score for most agents is negative. Negative means worse than a coin flip.
Every major agentic AI benchmark tests whether an agent can complete a task.
After introducing why static benchmarks are misleading, the article explains how FutureSim evaluates agents on adaptive forecasting: replaying ordered real-world events, updating predictions as new information arrives, and scoring with Brier score and Brier skill score versus an uninformed baseline. The results show that even the best agent reaches only about 25% accuracy, while most agents get negative Brier skill scores—meaning they are not merely uncertain, but confidently wrong compared to abstaining. The author argues this failure mode is structural: LLMs have a training cutoff, so when events move beyond prior distribution, “Bayesian-style” updating can confidently compound an outdated world model into wrong posteriors. The article concludes that current agentic systems are being deployed for forecasting and decision support without evaluation that measures dynamic calibration and epistemic humility, and it suggests better calibration mechanisms and benchmark suites like FutureSim are needed to ensure predictions are reliable for real-world deployment.
Read the full blog for free on Medium.
Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.
Published via Towards AI
Towards AI Academy
We Build Enterprise-Grade AI. We'll Teach You to Master It Too.
15 engineers. 100,000+ students. Towards AI Academy teaches what actually survives production.
Start free — no commitment:
→ 6-Day Agentic AI Engineering Email Guide — one practical lesson per day
→ Agents Architecture Cheatsheet — 3 years of architecture decisions in 6 pages
Our courses:
→ AI Engineering Certification — 90+ lessons from project selection to deployed product. The most comprehensive practical LLM course out there.
→ Agent Engineering Course — Hands on with production agent architectures, memory, routing, and eval frameworks — built from real enterprise engagements.
→ AI for Work — Understand, evaluate, and apply AI for complex work tasks.
Note: Article content contains the views of the contributing authors and not Towards AI.
Related posts
Recent Posts
