Severe control over AI agents as a tool for mass-surveillance

post by Andrey Seryakov (andrey-seryakov) · 2025-04-24T20:27:50.860Z · LW · GW · 0 comments

Contents

  IDs for AI agents
  Who is the watcher?
  Andrey, you are criticising again without offering solutions?
None
No comments

Critique and Extension of the “IDs for AI Systems” proposal written by A. Chan, N. Kolt, P. Wills, U. Anwar, C. Schroeder de Witt, N. Rajkumar, L. Hammond, D. Krueger, L. Heim, and M. Anderljung 

IDs for AI agents

Soon, the internet will be populated with AI agents [LW · GW] —individual AI entities performing various actions. Personal agents, corporate agents, state agents... It’s likely that in just a few years, almost everyone will have at least one agent of their own, much like how nearly everyone today has social media profiles.

But how will you know who owns the agent you're about to interact with (or that your own agent is interacting with)? Can you trust it? What if it is a fake? Should you answer that call or buy tickets from that website?

An agent's ID is something like a phone number—but more complex. Agent's behaviour is shaped by several factors: the prompt, the knowledge base, and the model. Changing any of the three alters the agent’s behaviour. This makes it more like a versioned software system—similar, yet different. The authors argue that each such version deserves its own ID.

An ID is also crucial for untangling the complex web of agent interactions. If something goes wrong, only a unique ID can help identify the exact state the agent was in at the time. This becomes especially important when agents handle vast assets, like financial companies. And yes, with an ID, you can easily block a rogue agent from interactions if needed.

As the authors state and I agree, there will be far more agents than users. Moreover, agents themselves will create other agents—for instance, to parallelise tasks. When one agent creates another, the second gets its own ID.

Since an AI’s behaviour depends on the model, its knowledge, and the task it is given, an ID will only be issued if all three are deemed “safe.” But by whom?

Who is the watcher?

Technically, it sounds great. We just have to find an institution, that would provide IDs and check agents for safety. The authors propose that a "trusted-by-everyone" organisation should become such an "ID provider". It could be a corporation, an independent, transparent organisation, or a government entity. But let’s be honest—who do you think that would be? Information is power, and governments don’t give that up easily.

And here’s where the problems begin—at least for the 93% of humanity ^[1]who don’t live in liberal democracies, especially for the 53% under authoritarian regimes. To verify that an agent is truly safe, the provider must know everything about it: its knowledge base, task, and model. In other words, everything the agent knows about you and everything you’ve asked it to do. And that can include a lot: your illnesses, travel plans, eating habits, who you’re dating... Sure, corporations and governments already know a lot—but this would be orders of magnitude more, and more importantly, centralized and analyzed instantly.

Yes, in countries that respect human rights, you could try to make the system transparent, add oversight by civil organizations, and limit what data must be disclosed. But again, that would only work for the privileged 7%. What about the rest? We would get a hyper-powerful surveillance infrastructure.

Andrey, you are criticising again without offering solutions?

No, in fact, I believe this paper is a good start for the discussion that is needed. And I want to continue it, offering an extension.

Yes, we absolutely do need agent IDs so that agents can interact meaningfully. But those IDs should be strictly limited to business and government agents—the ones that need regulation. Just like governments inspect airline safety or restaurant hygiene, they should ensure that corporate agents are who they claim to be.

But I strongly oppose personal agent IDs. Information held by personal agents should be treated as private and legally protected. No one should know who I spend the evening with (except that person), where I buy tickets (except the airline), or what’s wrong with my mental health (except my doctor)—unless I choose to share it.

If you want personal agent verification, do it like social media currently does: a user ID and maybe the AI model version^[2]. That’s it. Each person should be responsible for the instructions they give to their agents and companies for the model. If a crime is committed, personal data can be disclosed with a court order, and only this way.

What I propose isn’t new. This is how we already manage political, economic, and military systems. There's a huge imbalance of power between institutions and individuals, so individuals must be protected, and institutions must be controlled.

This post is, in essence, a plea to AI developers: don’t just optimize your systems technically—think about the global consequences they might bring for humanity.

Thank you for reading.

1. ^{^}

   These numbers differ depending on the institute that provided them. However, they agree on < 10% of the human population lives in liberal democracies and around 50% in authoritarian regimes. 

2. ^{^}

   There should be a way to register custom models too. Probably, an alignment and safety procedure they have to pass. This should be similar how we may register a handmade car to use it on the roads.

0 comments

Comments sorted by top scores.