Agent memory QA scores may hide errors in stored facts

A post describing an Amazon paper says its FEB-30 benchmark tracks the correct memory state across 30 scripted fact histories turned into 147 documents. In a test of a memory system built like Mem0 and Hindsight, direct memory and QA scores differed by 16.8 points on average, and pass/fail verdicts disagreed on 21% of fact histories. The post says the system chose the right memory action only 20.1% of the time—something QA scores did not reveal.