Author(s): Rick Hightower

Originally published on Towards AI.

CCA-F Part 5: A passing JSON schema proves your output has the right shape. It proves nothing about whether the values are true. That gap is the single most punishing area on the live exam

A model response can pass every check in your JSON schema and still assert something that never happened. The skill that separates working systems from quiet disasters is knowing that syntactic validity is not semantic truth, and learning when to retry the call versus repair the schema itself.

The article argues that JSON-schema validation checks only syntax (the shape and required fields) and cannot guarantee semantic truth, using examples like fabricated refund reasons that still pass validation. It lays out a “domain 4” pipeline for passing the Claude CCA-F exam by separating absence from fabrication: make potentially missing fields nullable and use an explicit “unclear” enum value, rather than forcing required fields to be invented. It then emphasizes adding semantic gates in code for constraints schemas can’t express (e.g., cross-field arithmetic like line items summing to a stated total), using validation-retry only for recoverable mis-extraction (while routing genuinely absent data to null/“unclear”). Additional guidance covers why the tool_use mechanism is the structured-output workhorse, how to distinguish format stability (few-shot examples) from decision boundaries (explicit categorical criteria), and why an independent reviewer instance should approve outputs rather than self-review. Finally, it notes that Message Batches are appropriate for non-blocking overnight bulk work, not latency-sensitive checks, and concludes with a checklist for auditors and implementers: audit required fields for absence, add semantic validation, make retry feedback exact, add independent review, and use batches only for bulk.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI