The ability of AI chatbots and agents to remember user preferences is becoming a major selling point, with companies like Google, OpenAI, Anthropic, and Meta developing features that draw on personal details and histories.

Personalized, interactive AI systems are designed to act on behalf of users, maintaining context across conversations and improving task completion, from booking travel to filing taxes. However, storing and retrieving intimate user details over time introduces significant privacy vulnerabilities.

Conversational interfaces and frequent context switching can lead to the collapse of data separation by context, purpose, or permissions, creating single, unstructured repositories of user information. When AI agents link to external apps or other agents, data can seep into shared pools, exacerbating privacy risks.