Author(s): Muhammad Soliman

Originally published on Towards AI.

The Number That Matters in Cloudflare’s Clef System-One Model Isn’t 38.8 ms — Jev and Laya compared

Cloudflare just open-sourced Clef and Clef-flash, two “System One” decision models built on Qwen. The headline is speed. I went in to prove the fast one couldn’t say “none of the above.” I ran it on my laptop against Laya, and that hypothesis didn’t survive. What broke instead was more interesting.

After explaining what decision models are (and how Clef fits into the “System One” routing-style protocol compatible with Jev), the article breaks down what’s actually new in Clef—especially its frozen Qwen backbone paired with a joint schema head for parallel scoring, along with explicit calibration objectives and multimodal inputs. The author then contrasts vendor benchmark claims with their own testing: Clef-flash’s routing performance is strong, but surprising weaknesses appear on out-of-scope intent detection (and execution latency varies heavily with hardware and option counts). Through three experiments—including CLINC150 OOS tests and real runs inside a browser agent—the author shows that safety depends on providing an explicit escape option and gating decisions using margin/escape probabilities rather than raw confidence alone. They also surface practical integration lessons (calibration differences across models, option-set size effects, drift from rendering differences, and why state/token cost matters more than decision-only latency), conclude with a recommended “gate you can ship” routing ladder, and caution readers to validate on their own agents because a crucial failure mode may not show up in public leaderboards.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI