Putting AI agents to work on internal processes, with guardrails

What an agent is, in practical terms

Every major provider's API supports tool use (also called function calling): you describe a set of tools, the model decides which to call and with what arguments, your code runs the call and returns the result, and the model continues until it has an answer. An agent is that loop, plus instructions, plus a set of tools chosen for a job.

The model never touches your systems directly. Your code runs every tool call, which means your code decides what's allowed. That's the lever every guardrail in this guide pulls on. For an assistant that only reads documents, see connecting AI to your internal data safely.

Where agents help inside a company

Good first candidates share three traits: the work is mostly gathering and assembling information, a person already reviews the result, and a mistake is caught before it matters.

Notice that every output is a draft or a suggestion. That's deliberate for a first agent. Once its drafts are consistently accepted with few edits, you can consider letting it take some actions itself, with the controls below.

Tool permissions: the real guardrail

Instructions like "never delete records" are requests, not controls. The dependable guardrail is what the agent's tools can physically do. Design tools the way you'd design access for a new contractor.

• Narrow tools, not generic ones. get_open_quality_holds(line) is safer and more reliable than run_sql(query). The tool enforces the scope, so the prompt doesn't have to.
• Separate read and write tools. An agent typically needs many read tools and few, if any, write tools. Keep them in different lists so you can grant them separately.
• The agent's own credentials. Each agent runs as a service identity with only the permissions its tools need, never with an administrator's or the requesting user's full access.
• Respect the requester's access. When an agent acts for a person, check that person's permissions too, so the agent can't become a way around them.
• Validate every argument. Tool code checks inputs like any API would: allowed values, record ownership, sensible sizes.

The Model Context Protocol (MCP), an open standard for connecting AI applications to tools and data, is supported by a growing number of AI tools. It's a convenient way to package tools once and use them with several models, but the same rules apply: an MCP server's permissions are the agent's permissions.

Human approval for actions

Sort every tool the agent could use by what happens if it's used wrongly, and let that decide whether a person approves it first.

• Approval happens in code. When the agent calls a gated tool, the call is paused and queued; it runs only after a person approves it in a screen that shows exactly what will happen.
• Show the reasoning and the evidence. The approver sees the proposed action, the arguments and the sources the agent used, not just "Approve?".
• Approvals are specific. Approving one email doesn't approve the next one.

A reference design

Treat everything the agent reads as untrusted

An agent reads documents, tickets, emails and web pages, and any of them can contain text written to redirect it (prompt injection). OWASP's guidance on the risk notes that it's unclear whether any method prevents it completely, so the defense is in the design.

• Don't combine, in one agent, the ability to read untrusted outside content and the ability to send data outside the company.
• Keep instructions in the system prompt and treat retrieved content as data, clearly separated.
• Gate outbound and hard-to-reverse actions behind approval, as above, so an injected instruction still stops at a person.
• Include injection attempts in your test set, and check that tool permissions hold even when the model is fooled.

Audit logs and evaluation

When someone asks "why did the agent do that?", you need to be able to answer from the record. And when you change a prompt, a tool or a model, you need to know whether the agent got better or worse.

1. 1Build a set of realistic tasks with known good outcomes, including edge cases and a few deliberate injection attempts.
2. 2Score the final outputs, and check the path too: did it call the right tools, stay within its permissions and stop when it should?
3. 3Rerun the set before any change to the prompt, tools or model goes live, and compare with the last run.
4. 4Add real tasks where a person rejected or heavily edited the output, so the set tracks what actually goes wrong.

Choosing a model provider

Claude, OpenAI, Gemini and Grok all support tool use, and we've integrated all four providers' APIs. How well each handles your tools, instructions and edge cases is something to measure on your own evaluation set rather than assume, and the answer can change with each model release. Keep the provider behind one internal interface so switching or comparing is a configuration change, and weigh hosting options, such as access through Amazon Bedrock, Google Cloud Vertex AI or Microsoft Azure, and data-use terms alongside quality.

Our own product, Round Table, runs Claude, ChatGPT, Grok and Gemini side by side in one conversation, with usage tracking (token accounting) built in. Our experience also includes AI agents, agent skills, and AI-driven policies and procedures. See AI integration for how we'd approach your process, and AI Velocity Training if your developers want to work with coding agents themselves.

Sources

• Tool use with Claude (Anthropic)
• Function calling (OpenAI)
• Function calling with the Gemini API (Google)
• Function calling (xAI)
• What is the Model Context Protocol? (MCP project)
• LLM01: Prompt injection and LLM06:2025 Excessive Agency (OWASP GenAI Security Project)
• Cloud hosting: Claude on Amazon Bedrock, models on Vertex AI, Azure OpenAI