We have agents that can call APIs and take actions — what can stop them doing something unauthorized?

Why the prompt cannot be the control

An agent decides what to call by reading text. That text can come from several places: the user’s message, a document supplied to the agent, or a result returned by another tool. Whoever controls one of those inputs may be able to influence the agent’s next decision.

A system prompt can state that the agent should act only for the signed-in user. It can describe which tools are appropriate, which records belong to that user, and which actions need approval. Those instructions may help define the intended behaviour, but they do not enforce it outside the model. The model still interprets the instruction alongside the rest of the conversation and the content returned by tools.

OWASP names this Excessive Agency: damaging actions taken because an agent had more functionality, permission or autonomy than the task required. The relevant remedy is structural. Reduce the functionality available to the agent, narrow its permissions and require human approval where appropriate. Better wording does not replace those controls.

This distinction matters because an unauthorised action can result from a legitimate tool being used in an illegitimate way. An agent does not need access to an obviously dangerous capability if it can use an ordinary customer or administrative tool with the wrong target. Controls must therefore apply to the available tools, the credentials behind them, the arguments supplied to them and the point at which the call executes.

What are the five controls?

The controls operate at different points in the request path. They should not be treated as alternatives. An allowlist limits what the agent can request, credentials limit what the API can grant, argument checks validate the specific request, approval gates pause sensitive operations, and runtime controls block and record calls that still reach the enforcement layer.

Tool allowlists

A tool allowlist defines the capabilities available to the agent. It prevents the agent from calling tools that were never part of the intended task. This is the first boundary to establish because an unavailable capability cannot be selected through an ordinary tool call.

The allowlist should reflect the task rather than the full set of APIs reachable by the application. An agent handling order queries may need a status lookup, but that does not mean it needs every customer-management, administrative or data-deletion operation. The control is deliberately narrow: it stops access to an unapproved capability, not misuse of a permitted one.

Scoped credentials

An allowlisted tool still needs credentials. Those credentials should be scoped to the least the task needs. Where the API supports it, they should be per user so that a session cannot reach more than the signed-in user could reach.

This boundary is enforced in the API the tool calls. It remains important even when the agent configuration is correct, because a tool may be used with an unexpected target or an argument influenced by untrusted text. A single broad service credential makes every session as powerful as the most privileged one. Credentials scoped to the task reduce the reach of a hijacked session.

Why must checks validate arguments, not just tool names?

A tool name describes a capability, not the legitimacy of a particular call. An agent allowed to call checkOrderStatus can still be steered to call it with another customer’s order ID. The tool itself may be valid, while the customer, amount or target supplied to it is not.

The pre-call check should compare the arguments with the session. For an order lookup, that means verifying that the order belongs to the signed-in customer. The same principle applies to other permitted operations: the requested target must be consistent with the identity and scope of the current session.

This is where an allowlist alone stops being sufficient. It answers the question, “May this agent use this capability?” It does not answer, “May this session use it with these arguments?” Both questions need an enforcement point outside the model.

Enforce the decision before execution

The check must run before the call and outside the conversation. It should see the tool name and every argument exactly as the agent formed them, then return an enforcement outcome such as deny, require approval, warn or log.

In Redline, this policy runs at the pre_tool stage. The model can be persuaded to want a call, but it cannot persuade the external check to ignore the arguments. The check can compare the request with the session and determine whether the call should proceed.

See runtime guardrails at four stages for the enforcement model. The important design property is placement: the policy must sit between the agent’s decision and the API execution rather than existing only as text inside the agent’s instructions.

What the check should inspect

The policy should inspect the complete call, not merely the selected tool. Relevant values include:

• The tool name.
• Every argument supplied to the tool.
• The customer, account, tenant or other target represented by those arguments.
• The amount where the operation involves a monetary value.
• The session to which the call belongs.
• The policy outcome before execution.

The purpose is not to predict every possible attack. It is to establish an explicit relationship between the current session and the requested action. A call can be legitimate in one session and unauthorised in another.

How should credentials be scoped?

Credentials should provide no more access than the task requires. This applies to the tool’s API access as well as to the agent’s configuration. A tool may be approved, and its arguments may appear structurally valid, but broad credentials can still allow it to cross a tenant boundary or reach an administrative endpoint.

Per-user credentials are preferable where the API supports them. They preserve the user boundary at the API rather than relying only on the agent to remember which records the user may access. If a session is hijacked, the available credential should limit what that session can reach to what the user could reach.

A single broad service credential has a different failure mode. It gives every session the authority associated with that credential, including authority that may not be relevant to the current user or task. That makes the other controls carry more responsibility than they should.

Credential scope also complements argument checks. The argument policy can reject a request for another customer’s order. The API credential provides an additional boundary if an incorrect request gets through the application logic. These controls address different points in the path and should be designed together.

When should a person approve the action?

An approval gate should protect an irreversible action. Examples include a refund, an email to a customer or a deleted record. The action is held before the call and does not execute until a named person approves it.

This control is valuable because it does not depend on predicting the attack. It does not need to identify every way an agent might be persuaded. Instead, it pauses the operation at the point where a person can decide whether the exact action is acceptable.

Start with the most irreversible tool the agent has and require approval for it. The approval should be associated with the specific pending action rather than functioning as a general instruction to trust the agent. The person should be able to assess what the agent is about to do, for whom and with which arguments.

Approval is not a substitute for least privilege. An agent should still have only the tools and credentials required for its task. Nor does approval remove the need for audit. The system should retain the call, the policy decision and the session so that the action can be reconstructed.

How does runtime blocking make a hijack provable?

Rules catch the behaviour they describe. Runtime blocking provides a final enforcement point for calls that reach the execution path, while audit records preserve what happened. Every call should be evaluated and recorded per session, including the arguments and the verdict returned by policy.

A runtime control can block a call that violates the rules. It can also record a call that is allowed, warned about or sent for approval. This provides a consistent record of what the agent attempted, rather than only a record of successful API operations.

Use honeypot tools as evidence

For behaviour that the explicit rules do not describe, add a honeypot tool that fits the agent’s real toolset but that no legitimate conversation ever calls. A call to that tool is not merely an anomaly score. It is evidence that text somewhere redirected the agent, with the session attached.

The honeypot should be plausible in the context of the agent’s available tools while remaining outside legitimate conversation paths. Its value comes from the expected absence of legitimate calls. If it is called, the event can be blocked and investigated as part of the session record.

See AI agent honeypots for the related control. Honeypots do not replace allowlists, scoped credentials, argument checks or approval gates. They add evidence and a runtime response for redirection that the stated rules may not have anticipated.

How should the controls be tested before release?

The assembled agent should be tested rather than assessed only as a model or as a collection of individual tools. The relevant question is whether the complete agent preserves the rules when messages, documents and tool results influence its decisions.

Testing should check that:

• An unallowlisted capability cannot be called.
• A permitted tool cannot cross the session’s customer or tenant boundary.
• Arguments are checked before the API call runs.
• An irreversible action is held for approval.
• A policy violation is blocked and recorded.
• A honeypot call becomes evidence with the session attached.

Redline runs 11,204 cases across 16 families, including Excessive Agency, Risky Tool Use and Credential Theft. See the pre-production testing playbook.

The purpose of pre-release testing is to verify the assembled enforcement path. A tool may behave correctly in isolation while the agent forms an unsafe argument. A prompt may state the correct user boundary while a document or tool result attempts to redirect the agent. Testing should therefore exercise the agent’s ability to call tools and take actions, not just its ability to produce text.

Key takeaways

• A system prompt is an instruction to the model, not an enforcement boundary outside it.
• Use a tool allowlist to remove capabilities the agent was never meant to have.
• Scope credentials to the least the task needs, ideally per user where the API supports it.
• Check every tool name and argument before execution, against the current session.
• Require approval for irreversible actions such as refunds, customer emails and deletes.
• Block and audit calls at runtime, and use honeypot tools to make hijacks provable.

Common questions

Is a tool allowlist enough?

No. It stops calls to tools the agent should not have, but it does nothing about a permitted tool called with the wrong arguments. An agent may be allowed to call checkOrderStatus and still supply another customer’s order ID. Pair the allowlist with argument checks that compare the request with the current session.

Should the agent use the user’s credentials or its own?

Use credentials scoped to the least the task needs, and per user where the API supports it. This means a hijacked session can reach only what that user could reach. A single broad service credential makes every session as powerful as the most privileged one, increasing the consequence of an unauthorised tool call.

Where should the policy check run?

It should run outside the model and before the call. Inside the agent process, the check can see the tool name and arguments exactly as the agent formed them. The model can be persuaded to want a call, but it cannot persuade an external check to ignore the policy result.

What should be logged?

Log every tool call with its arguments, the verdict returned by policy and the session to which it belonged. That record should be sufficient to reconstruct what the agent did and why a call was allowed. Runtime records also provide the session context for blocked calls and honeypot events.

What should happen to an irreversible action?

Hold it before the call until a named person approves it. Refunds, emails to customers and deleted records are examples of actions for which an approval gate is appropriate. Begin with the most irreversible tool available to the agent and apply a require approval decision.

Why check arguments if the API has scoped credentials?

Scoped credentials and argument checks address different boundaries. Credentials limit what the API can grant, while the pre-call policy verifies that the requested customer, amount or target is appropriate for the current session. A permitted tool can still be called with the wrong identifier, so both controls are needed.

Sources

• OWASP Excessive Agency