Author(s): Chew Loong Nian – AI ENGINEER

Originally published on Towards AI.

I rendered the same chat — one system message, one user message, then again at three and five turns — through twelve production chat templates. Two turned out to be stale mirrors. Of the ten that survived, three never give the system prompt a role at all, one of those carries it three-quarters of the way down the conversation, and one refuses to run.

There is a question I had never once asked about my own code, and when I finally asked it I could not answer it.

After realizing that “system” messages are templated and may be transformed without logging, the author builds a minimal offline experiment using a sentinel string to locate where the system text actually lands in the final prompt. Across twelve production chat templates, the same system prompt is handled inconsistently: it can be role-marked and stay near the top, drift deeper as history grows, be stripped of the system role entirely, be merged into user text, have extra injected lines, or even cause refusal. The article shows how to verify verdicts by inspecting rendered text, highlights failures caused by stale mirrors, and demonstrates model-specific quirks such as Mistral attaching system instructions to the last user turn, DeepSeek hoisting later system messages to the beginning, and Llama templates injecting date lines regardless of whether a system message is provided. It concludes that the API “role” field is not a guarantee—what matters is the actual model’s chat template as executed by your runtime—so developers should inspect and test the template used in their stack.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI