From a dataset of 179 incidents reported by 146 users, researchers at the University of Illinois Urbana-Champaign and Northeastern University have mapped a “negative space” of AI memory, identifying 15 categories of information people want kept private from conversational AI assistants. The work characterizes a growing need for proactive memory governance, where protection takes precedence over personalization.

Researchers operationalized this understanding in FENCE, a system that automatically applies user-defined rules to prevent sensitive information from entering AI memory, while still allowing exceptions. The findings demonstrate how taxonomy-based rules can support user-defined memory boundaries, and the team calls for collective efforts to establish policies governing what AI should not remember.

AI Memory Governance: Addressing Post-Sharing Protection Gaps

The research team developed a system called FENCE, a policy-level memory protection module designed to automatically apply user-selected category-based rules to conversations, preventing information from protected exchanges from entering AI memory by default. This approach shifts memory governance from reactive deletion to proactive control, addressing a gap in current AI personalization strategies. FENCE operates by establishing a standing rule based on a taxonomy of information categories users wish to exclude, making protection the default behavior for those categories.

The system allows for case-by-case overrides, offering flexibility while prioritizing user-defined boundaries. Researchers found that establishing memory governance at the policy level, focusing on system behavior rather than individual memory items, created a more stable yet revisable protection boundary. This design allows users to adjust the scope of protection as their AI usage evolves, addressing a key challenge in long-term AI interaction. The work highlights a need to define what AI should not remember, moving beyond simply maximizing data retention for personalization benefits.

179 Incident Analysis Reveals User-Defined Memory Boundaries

From 179 incidents reported by 146 users, the researchers developed a taxonomy of 15 information categories users want excluded from memory, revealing sensitivity concerns beyond conventional taxonomies. Disclosures covered by the policy are kept out of memory, while other information remains eligible for persistent memory. Participants found that FENCE made memory governance more usable, interpretable and trustworthy than conventional post-hoc management.

FENCE proactively filters disclosures before they reach persistent AI memory, a departure from typical management strategies that address privacy concerns after data exposure. The research team characterizes the goal as creating interactions safe enough to remain useful while instilling user confidence in how the system handles disclosures.

Taxonomy of 15 Sensitive Information Categories for AI Exclusion

Analysis of 179 reported incidents informed the development of a taxonomy structuring 15 categories of sensitive information, allowing for a shared vocabulary between users and the system without requiring complex hierarchical navigation. This structure contrasts with typical classification schemes, prioritizing interpretability and ease of use for defining memory boundaries. The research team intentionally designed the taxonomy to be a stable policy language, connecting current protection decisions to expectations about future AI system behavior.

FENCE Implementation: Filtering Disclosures from Persistent Memory

The FENCE system proactively filters information from AI memory using user-defined rules, a departure from conventional post-hoc management techniques, as demonstrated in a within-subjects study with 15 participants. Unlike reactive deletion of data, FENCE operates as a policy layer between conversation and persistent memory, preventing disclosures in protected categories from being stored. Participants in the study found FENCE more usable and trustworthy, reporting a clearer understanding of what the system would remember and why.

Policy-Level Protection: Rules Governing AI Memory Over Time

Establishing proactive memory boundaries requires a shift from reactive deletion to preventative measures, as evidenced by the design of FENCE, a system intended to govern AI memory at a policy level. This approach addresses a critical gap in current AI systems, where users often struggle to manage memory after disclosures occur, relying on post-hoc attempts to remove information.

The system’s design acknowledges that AI providers are already making implicit decisions about what to remember, but these rules are often provider-defined and subject to change, creating inconsistent user experiences. These categories inform FENCE’s policy framework, allowing users to establish a “standing protection policy” before sensitive information is shared, rather than attempting to manage it afterward.

AI Memory Personalization vs. User-Defined Negative Space

Researchers found that users desire protection of information beyond conventionally sensitive categories, necessitating a proactive approach to memory governance. The system’s design allows for case-by-case overrides, balancing protection with the possibility of personalization when users explicitly permit it. Their findings demonstrate how taxonomy-based rules can potentially support user-defined memory boundaries.

LLM Inference and the Expanding Scope of Memory Governance

The University of Illinois Urbana-Champaign team identified a need to proactively define what AI systems should not remember, shifting focus from simply deleting data after it’s stored. This approach moves beyond conventional data-storage privacy concerns by addressing information users actively wish to exclude from persistent AI memory. Researchers argue that current privacy frameworks are insufficient, as user sensitivities extend beyond personally identifiable information. The team proposes that AI systems should disclose the default rules governing this transition and provide users with a means to modify them, ensuring transparency and control.

Providers should also visibly communicate changes to these rules, rather than silently expanding the scope of persistent data. This framework complements existing data-protection principles by standardizing the governance of information entering memory, rather than attempting to manage individual data points retroactively. Such standardization would allow for verifiable protection and user revision of the scope of memory retention, addressing concerns that current approaches leave protection to individual retrospective management.

👉 More information
🗞 What should AI forget? A design and taxonomy for negative space governance in AI memory
✍️ Eryue Xu, Ke Ding, Ting-Yu Chang and Masooda Bashir (University of Illinois Urbana-Champaign, Champaign, Illinois, USA); Tianshi Li (Northeastern University, Boston, Massachusetts, USA)
🧠 ArXiv: https://arxiv.org/html/2610.27607

More like this