Notes · Field note ·
The timestamp that froze the cache
With long prompts, the cheapest token is the one you don't change.
Model servers reuse work for the part of a prompt they've seen before, the prefix. Change one early byte and everything after it is recomputed.
A memory briefing my agents receive at the start of each session included a timestamp that changed on every request. For a day, the engine's cache hits stopped at 51,712 tokens: everything after the timestamp was recomputed, every time.
The fix
The briefing is fetched once per session, saved to disk, and sent byte-identical for the rest of the session, even if the plugin reloads.
Put anything that changes at the end of the prompt, or leave it out.