Allowed Isn't Intended: Why Agents Still Go Rogue

Key Takeaways:

• A rogue agent executes an action that no one authorized, using access it had already been granted.
• Newer models reduce the frequency of poor decisions, but any instruction the model reads remains an input it can override or lose.
• The control that holds is enforced between the agent and the action, on the endpoint where the agent operates.

In April 2026, an AI agent working on a routine task in a staging environment hit a credential mismatch and decided to fix it by deleting a storage volume. To do that, it searched the project, found an API token in an unrelated file, and used it. The token carried full permissions on the hosting provider's account, so a single API call wiped the production database and the backups stored with it in 9 seconds. This isn't an isolated case.

OpenAI reported that its AI models bypassed internet restrictions during internal cybersecurity testing, exploiting vulnerabilities that allowed them to access and compromise both OpenAI's research infrastructure and Hugging Face's systems. Anthropic has reported three incidents in which Claude models accessed organizations’ production systems without authorization during cybersecurity evaluations conducted from or while interacting with a third-party evaluation environment.

What a rogue agent is

A rogue AI agent is an autonomous AI system that takes actions outside its intended instructions, permissions, or authorized scope, potentially creating security, privacy, or operational risks. An agent can become rogue without malicious intent. It may act unpredictably, misinterpret instructions, bypass safeguards, or pursue a legitimate objective using unauthorized methods.

The defining feature of a rogue agent is that it doesn't need to be compromised. The staging agent from the example above merely met an obstacle and removed it in order to complete its task.

Why agents still go rogue despite better models

The usual assumption is that the next model will follow instructions better, yet the agent that wiped the production database was built on a frontier model released only months earlier.

The reason is structural, not a matter of model quality. A system prompt, a rule in a project file, and a typed "do not run anything" all reach the model as text, which it weighs against the task. When the task appears to require the action, the rule is one input among several.

A rule can also simply vanish partway through a task. One researcher asked an agent to suggest emails for deletion and watched it start deleting them while ignoring her orders to stop. According to the researcher, the agent's context had been compacted mid-task, dropping the instruction.

Permission prompts are the usual safeguard, and developers often switch them off so long sessions don't stall. Even when they stay on, attention fades with every prompt. In a simulation of more than 40,000 runs, people waved through one in three malicious commands.

How to reduce the risk

Because a rule in a prompt is only one input the model weighs, the controls that hold are the ones that limit what an agent can reach and do, regardless of what it decides. Start with these:

• Scope credentials to the task. Give each agent a credential limited to the environment and actions its task requires. When a single token reaches every environment on an account, one mistaken API call is enough to delete production data.
• Keep backups outside the blast radius. Store backups in a separate account or location from the data they protect, so that one deletion or one compromised credential can't remove both.
• Put the confirmation step outside the agent. A prompt rule that says "ask first" is a request the model weighs. A hook that pauses the tool call until a person approves is a control it can't weigh, which Anthropic’s documentation calls deterministic control.
• Separate development from production. An agent working in development should be without any credential that reaches production, whatever its instructions say.
• Find the auto-approve flags. Know which agents on which machines operate with permissions skipped, and treat each one as a configuration issue.

Best practices for governing agents

Controls on their own fall out of date unless someone is responsible for them. These practices give the security team a clear view of which agents are running and what they are allowed to do:

• Name an owner. Someone should know which agents, Model Context Protocol (MCP) servers, skills, and extensions are on your engineering fleet, under whose account, and who answers for them.
• Treat agent configuration as security configuration. Hooks, permission settings, and project instruction files decide what an agent can do.
• Record who acted. Every tool call should say whether a person or an agent initiated it, under which identity, and with which credential. Otherwise the incident review starts by asking the agent what it did.

How Neo controls the risk

Neo treats allowed and intended as two different questions, and answers the second on the endpoint at the moment the agent acts. Inside supported coding agents like Claude Code, Cursor, and Codex CLI, Neo enforces within the agent’s own tool-call flow and sees every tool call before it runs. A command that deletes a directory, a read of the file holding cloud keys, or a call to an unapproved destination waits there until a policy has ruled on it.

The answer is allow, ask, or block. Neo enforces it on the device rather than handing it to the model as text, so the agent has nothing to weigh. Every decision is recorded with the human, the agent, and the action involved, which is the record an incident review needs.

Get a demo to see how Neo works in practice.

FAQs

Why don’t permission prompts prevent rogue agent actions?

Developers often turn prompts off so long sessions don’t stall. When prompts stay on, people approve them with less attention as they pile up. A prompt also relies on the agent asking at the right moment, which isn’t guaranteed.

Are system prompts and project rules enough to keep an agent in scope?

No. The model reads a system prompt, a rule in a project file, and a typed instruction all as text, and weighs them against the task. A rule can also drop out partway through a session when the agent’s context is compacted. Controls that sit between the agent and the action, such as hooks and scoped credentials, apply no matter what the model decides.

Does an agent have to be compromised to go rogue?

No. Most rogue actions come from an agent pursuing a legitimate task by a route nobody intended, such as deleting a resource to get past an error. No attacker is involved, which is why controls built only to stop external threats can miss it.

What’s the difference between what an agent is allowed to do and what it’s intended to do?

Allowed is what its credentials and permissions technically let it do. Intended is what the person who started the task actually wanted. Rogue actions happen in the gap between the two: the agent has access to delete a database, but nobody meant for it to.