TL;DR: A New America policy brief warns that AI agents with persistent memory, connected through the Model Context Protocol, are outpacing every privacy safeguard we have. Agents that juggle your calendar, health records, and email in a single workflow create cross-service data exposure that existing consent models, encryption standards, and jurisdiction boundaries weren’t designed to handle. The window for getting governance right is closing fast.

AI Agents Remember Everything, And That’s a Privacy Problem

Picture this scenario. You ask your AI agent to renew your driver’s license. Straightforward enough. The agent pulls up your calendar for scheduling, checks transit routes, grabs your insurance details, and files the paperwork. Somewhere along the way it surfaces an outdated medical record. Now you’re flagged. Benefits suspended. No human reviewed the decision, and no interface tells you what went wrong.

That’s not science fiction. It’s the opening scenario from a New America policy brief published by Matt Steinberg and Prem M. Trivedi, and it shows why the combination of AI agents and persistent memory is a serious concern for privacy advocates.

From copilots to autopilots

We need to be precise about what we’re talking about here, because “AI agent” gets thrown around loosely.

A chatbot answers questions. An agent acts. It orchestrates multi-step workflows across multiple services with minimal human oversight. The New America authors put it bluntly: “If a chatbot is like a copilot for a single task, then an agent is more like an autopilot managing multi-step workflows with minimal oversight.”

Under the hood, an agent stacks several architectural layers: A large language model for reasoning, an orchestrator making real-time decisions, connections to external tools through APIs, authentication credentials, and (critically) persistent memory systems that carry context from one task to the next.

The connective tissue holding this together is increasingly the Model Context Protocol, or MCP. Anthropic open-sourced it in November 2024. By early 2026 it had been adopted by OpenAI, Google, and Microsoft, registered over 6,400 servers on its official registry, and been donated to the Linux Foundation’s Agentic AI Foundation. MCP is rapidly becoming the HTTP of the agent era, the default plumbing through which agents talk to the world.

And that plumbing has some serious leaks.

The memory problem nobody’s talking about

Traditional software keeps data in silos. Your calendar app knows your schedule. Your health portal knows your medical history. Your bank knows your finances. These silos have problems, sure, but they also provide a crude form of privacy protection: No single system sees everything.

Agents demolish those walls. An MCP-connected agent scheduling that doctor’s appointment simultaneously accesses calendar preferences, transit information, insurance data, and medical history, without separate user prompts for each service. Memory doesn’t stay local anymore. It becomes distributed, flowing across platforms in ways that are functionally invisible to the person who owns it.

This creates three categories of risk that the New America brief identifies with uncomfortable clarity.

You can’t see what it knows

First, opacity. Users have no reliable way to track what an agent “remembers” or where that information lives. The brief puts it starkly: “Even if permissions were originally granted, the lack of interface-level memory visibility makes it impossible to review, edit, or revoke.”

Think about that. You authorized access once, maybe during an eager setup flow when you connected six services to your shiny new agent. Now that agent carries fragments of your data across contexts you never anticipated, and there’s no dashboard, no audit trail, no memory map showing you the full picture.

Cross-service leakage

Second, data bleeds between services in ways nobody designed for. An agent might infer your health status from calendar patterns (repeated Wednesday appointments at a specific address) and then surface that inference to an entirely unrelated tool. The health data never left your health portal in the traditional sense. But the inference traveled everywhere.

This is qualitatively different from a data breach. No firewall was penetrated. No terms of service were technically violated. The agent simply did what agents do: Synthesize information across contexts. The privacy violation is architectural, baked into the operating model.

Behavioral profiling by default

Third, and this one gets less attention than it deserves, agents build behavioral profiles as a byproduct of memory. Stress patterns derived from calendar density. Spending impulsivity correlated with time of day. Emotional states inferred from communication tone. Scholars call this a threat to “cognitive autonomy,” which sounds academic until you realize it means your AI assistant might know you’re spiraling before you do, and might act on that knowledge in ways you never sanctioned.

Why existing law doesn’t cover this

Privacy regulation was built for a world of discrete services, clear data flows, and identifiable transactions. Agent architectures break every assumption.

Consent models fail because GDPR-style informed consent assumes users can understand and authorize specific data uses. When an agent dynamically chains together six services in a workflow the user never explicitly designed, what exactly was consented to? The New America brief calls this the “cascading authorization” problem, and nobody has solved it.

Purpose limitation breaks because privacy law typically restricts data use to the purpose for which it was collected. Agents recombine data across purposes in real-time. Your location data collected for transit routing becomes an input to a health recommendation. The purpose boundary evaporates.

Data minimization collapses because the entire value proposition of persistent agent memory is accumulating context over time. That’s the opposite of minimization. As the brief warns, “an agent’s context window becomes a growing repository of customer data, internal decisions, and system credentials.” That’s precisely the kind of data stockpile that regulators have spent years trying to prevent.

Jurisdictions stop making sense because a single agent task might span servers in three countries, invoke APIs governed by different regulatory regimes, and process data belonging to citizens of yet another jurisdiction. “Privacy laws emanate from geographies,” the authors write. “Agents do not.”

MCP’s security gaps are real

Beyond privacy theory, MCP has concrete security vulnerabilities that the Coalition for Secure AI flagged in a January 2026 white paper.

MCP currently lacks a protocol-level method for authenticating agents or delegating access to external APIs. In banking or healthcare, where identity is everything, this is a gaping hole. Without reliable identity protocols, agents can’t verify each other’s legitimacy, which opens the door to impersonation and unauthorized data access.

End-to-end encryption protects data in transit. That’s table stakes. But once an agent accesses plaintext content to do its job, that data becomes legible to every other component plugged into the orchestration layer. The encryption boundary stops at the agent’s front door.

Prompt injection is still unsolved, as the 2024 Echoleak incident proved. A hidden prompt in an email manipulated an agent into leaking private information from prior conversations. The agent couldn’t tell the difference between a legitimate instruction and a carefully crafted attack. Persistent memory makes this worse because there’s more sensitive context available to exfiltrate.

Cascading failures are another concern. Research cited in multiple security analyses found that in simulated agent networks, a single compromised agent poisoned 87% of downstream decision-making within four hours. Traditional incident response simply can’t contain failures that move at that speed.

MCP servers also tend to request broad permission scopes (full read/write access to Gmail and Google Drive, for instance) because granularity is harder to implement. A compromise of a single server with those permissions creates blast radius across the entire connected ecosystem.

The context flywheel and vendor lock-in

There’s a power dynamics angle here that the New America brief handles particularly well.

As foundational models converge in capability (GPT-5, Claude 4, Gemini 2 all performing at roughly comparable levels on standard benchmarks) the competitive advantage shifts from model quality to context. The more an agent knows about you, the more useful it becomes. The more useful it becomes, the harder it is to leave.

This is the context flywheel, and it mirrors the network effects that consolidated social media into a handful of platforms.

MCP could theoretically prevent this. As an open standard, it should enable portability, letting you switch providers without losing accumulated context, much like HTTP lets you move between websites. But “should” and “will” are doing very different work in that sentence. Without enforceable portability standards, MCP could just as easily become the mechanism through which context gets locked into proprietary ecosystems. The authors warn: “Without enforceable portability, MCP could recreate the very chokepoints it was meant to solve.”

Companies are already restricting API access while granting their own agents privileged integrations. The pattern is familiar. Open standard gets adopted. Dominant players extend it with proprietary features. Switching costs rise. Portability promises quietly expire.

What governance should look like

The New America brief doesn’t just diagnose. It prescribes, and the recommendations are unusually specific for a policy document.

On the user side, the brief calls for plain-language disclosures when linking agents to services. It recommends interoperable memory dashboards (not locked to a single vendor) that let people see, edit, and delete what agents know. It also proposes default retention limits for sensitive categories like health and finance, along with “memory-free” modes for high-stakes interactions where persistent context creates more risk than value.

For infrastructure, the brief recommends compartmentalized memory by default, segmented per-user, per-session, and per-tool. It calls for cryptographically signed action logs so agent decisions can be audited after the fact, plus NIST-level security guidance specific to MCP. The brief also argues for a framing shift: Treating orchestration infrastructure the same way we treat telecommunications or financial services infrastructure, as critical systems requiring federal coordination.

At the market level, the brief proposes Know-Your-Agent (KYA) requirements, modeled on Know-Your-Customer rules in finance, that validate agent identity without centralizing control. It pushes for memory portability standards with teeth, and accessible design defaults (multilingual interfaces, low-verification flows) so the governance overhead doesn’t become another barrier to participation.

One proposal that deserves more attention: Requiring agents to provide plain-language rationales for consequential decisions. Not a dump of model weights and attention scores. An actual explanation, in words a human can parse, of why the agent did what it did. Combined with dispute resolution mechanisms and user override capabilities, this would create something approaching accountability.

The governance gap is measured

Here’s a statistic that captures the problem in a single data point: 82% of executives believe their existing policies protect them from unauthorized agent actions. Only 21% have actual visibility into what their agents can access, which tools they call, or what data they touch.

That gap between confidence and reality is where the damage will happen.

The window is closing

The authors make a case that deserves repeating: “Regulating infrastructure, from telecommunications and financial services to transportation, is not new. The challenge is strategically adapting long-standing principles of privacy, security, and accountability to systems that increasingly act on our behalf.”

We’ve done this before. We regulated telephone networks. We regulated financial clearinghouses. We regulated air traffic control. Each time, the pattern was the same: New infrastructure emerged, early adoption outpaced governance, harms accumulated, and regulation arrived late and clumsy.

The MCP ecosystem is still young enough (barely 16 months old in its open-source form) that the governance conversation can happen before the harms compound. The Agentic AI Foundation under the Linux Foundation provides a governance structure for the protocol itself. But protocol governance and privacy governance are different animals. MCP’s maintainers can standardize authentication. They can’t decide what counts as informed consent.

That’s a job for policymakers. And based on the pace of agent adoption (97 million monthly SDK downloads, every major cloud provider participating, enterprise deployments accelerating) the runway for getting this right is shorter than it looks.

Agents should be treated, as the New America authors put it, as “experiences in need of evidence, not inevitabilities.” The technology is moving. The question is whether the rules can keep up.