SemiAnalysis Outlines Shared Prefix Caching for AI Agents

SemiAnalysis, a semiconductor and AI research firm, detailed an efficiency gain for AI agents. Previously, agents each saved their own copy of the model cache off-GPU and rewrote everything from scratch every turn. The new method saves the cache once per shared prefix and writes only new content on subsequent turns. This reuses the model's existing cache to skip rereading full history. The post included a split diagram comparing the approaches and forms part of a five-part thread.