Author(s): Hamza Boulahia

Originally published on Towards AI.

I put Jev through a series of tests to see whether Decision Models are actually worth caring about.

These last 7 days, social media has been taken by storm because of the newly released AI model, Jev. Even though we’re kind of used to the whole “circus” that comes with each new AI trend or big model release, it still always surprises me how much content, whether slop or not, gets published across all platforms.

The article introduces Jev as a “decision model” rather than a typical LLM, explaining how it uses typed question primitives (Choice, Score, Noul) to produce fast, calibrated probabilities and avoid the token-by-token generation and hallucination behavior associated with LLMs. It then highlights that Jev’s performance claims (speed, cost, and competitiveness on certain benchmarks) motivated the author to test it in practical settings, especially RAG context filtering and model routing, where quick, accurate decisions could replace slower LLM steps. However, after running evaluations on code review and CVE vulnerability scoring as well as limit tests in board games, the author reports that Jev’s accuracy and probability calibration were underwhelming despite strong latency and cost. Finally, the author argues that despite weaker results on the tested tasks, Jev’s value lies in its suitability for “System One” style decisions, and suggests that the broader paradigm shift toward decision models may matter more than chasing ever-bigger LLMs.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI