Prompt Caching Issue in DeepSeek-V4-Flash-0731 and the 'latest_reminder' Workaround
Developers have identified that inserting system messages mid-conversation in DeepSeek-V4-Flash-0731 invalidates the prompt cache. This happens because the model's chat template hoists all system messages to the very top, disrupting the cached prefix. Prompt caching is crucial for reducing latency and API costs when handling long, multi-turn conversations. Understanding this template quirk allows developers to optimize performance and avoid unnecessary expenses when using DeepSeek-V4-Flash-0731. To prevent cache invalidation, developers should use the `latest_reminder` role instead of the standard `system` role for mid-conversation instructions. This role is natively supported by DeepSeek's training and works seamlessly with engines like llama.cpp.
## BACKGROUND
Prompt caching stores the prefix of a prompt (like system instructions and early conversation history) so the LLM doesn't have to reprocess it in subsequent turns. Chat templates, often written in Jinja, define how structured message histories (system, user, assistant) are formatted into a single raw text prompt for the model. If the template dynamically rearranges messages—such as moving mid-conversation system instructions to the beginning—the prefix changes, rendering the cache useless.