What is tool governance?
Tool governance is the set of policies and technical controls that decide which tools an AI agent may call, which arguments it may pass, whose identity and permissions it acts under, which calls need a human approval, and how every call is recorded. It applies the ordinary discipline of access control to a new kind of actor. A person gets a role, a set of permissions, and an audit trail. Under tool governance an agent gets the same, with the added rule that the model’s own judgment is never the only thing standing between it and a write to a live system.
In practice the controls live in the platform layer that executes tool calls, outside the model. A policy might state that the support agent may read any ticket but may close only tickets assigned to it, that the finance agent may post journal entries below a threshold and must queue anything above it for a controller, and that no agent may call the payroll system at all. The platform enforces these rules whatever the prompt says.
Why tool governance matters
Tool use is what makes an agent useful and also what makes it dangerous. Prompt instructions are suggestions to a model, and a model can be talked out of them by a cleverly worded email or document it reads during a task. Governance that lives in code cannot be talked out of anything. It also answers the questions auditors and regulators ask: who authorized this action, under what policy, and where is the record. Without it, the honest answer is that the model decided.
Tool governance in practice
A regional bank deploys an agent to help branch staff with account servicing. The governance policy gives the agent read access to account and transaction data through the staff member’s own credentials, so it can see only what that person can see. It may draft address changes, fee reversals, and card replacement orders, but each one is a pending action that the staff member confirms. Fee reversals above a set amount route to a supervisor. Every tool call, including reads, is written to the audit log with the user, the agent, the arguments, and the result. When an auditor asks how the agent is controlled, the bank hands over the policy file and a sample of the log.
What to watch for
- Enforce tool permissions in the platform rather than in the system prompt, since prompts can be overridden by injected content.
- Run agents under the identity of the user they serve, or under a narrowly scoped service identity, so permissions and logs line up with existing access controls.
- Classify tools as read, write, or irreversible, and apply stricter approval at each step up.
- Review the tool list whenever a new MCP server or integration is added, because scope tends to grow quietly.