New research introduces a benchmark to test if AI memory systems leak private data into inappropriate contexts.
NEWS
Bruce Schneier highlights the CIMemories paper, which addresses risks in LLMs using persistent memory for personalization. The benchmark evaluates whether models correctly restrict sensitive information from flowing into unrelated conversation contexts. This follows growing concerns about AI privacy and data handling in long-term memory systems.
Why I Care
As AI agents retain more user history, the risk of accidental data exposure increases significantly. Organizations deploying personalized LLMs face potential compliance violations and trust erosion if memory controls fail. Users risk having private conversations exposed to unrelated tasks or third parties.
Next Steps
Security teams should evaluate current LLM deployments for persistent memory risks using frameworks like CIMemories by Q4 2026. Developers must implement strict context-aware access controls for stored user data before scaling personalization features.
I have been thinking a lot about AI and integrity. Part of that is contextual integrity. I recently found two papers on the topic.
“CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMs“:
Abstract: Large Language Models (LLMs) increasingly use persistent memory from past interactions to enhance personalization and task performance. However, this memory introduces critical risks when sensitive information is revealed in inappropriate contexts. We present CIMemories, a benchmark for evaluating whether LLMs appropriately control information flow from memory based on task context. CIMemories uses synthetic user profiles with over 100 attributes per user, paired with diverse task contexts in which each attribute may be essential for some tasks but inappropriate for others. Our evaluation reveals that frontier models exhibit up to 69% attribute-level violations (leaking information inappropriately), with lower violation rates often coming at the cost of task utility. Violations accumulate across both tasks and runs: as usage increases from 1 to 40 tasks, GPT-5’s violations rise from 0.1% to 9.6%, reaching 25.1% when the same prompt is executed 5 times, revealing arbitrary and unstable behavior in which models leak different attributes for identical prompts. Privacy-conscious prompting does not solve this—models overgeneralize, sharing everything or nothing rather than making nuanced, context-dependent decisions. These findings reveal fundamental limitations that require contextually aware reasoning capabilities, not just better prompting or scaling...