Track
Validate · Approve · Restrict · Recover
Agents don't just generate text — they call tools, read data, and take real-world actions. Every tool call is an opportunity for something to go wrong. Guardrails are the safety layer that decides what an agent is allowed to do, when it must ask for approval, and how unsafe behavior is blocked.
This track covers the four pillars of agent safety: input validation and tool permissions, human approval for risky actions, prompt injection defense, and safe failure handling.
Before calling a tool, validate every argument against its schema: right type, expected range, correct format. An agent that blindly passes user strings into a shell command is one injection away from disaster.
Tool permissions use an allowlist, not denylist: start with no permissions and grant only what's needed. Data analysis agents get search, read_file, calculate — never delete_file or send_email.
Classify tools by risk level: allowed (executes immediately), needs approval (pauses for human review), blocked (never allowed). When risky tools are called, the system pauses and presents the call to a human reviewer who can approve, deny, or modify. Default to denied on timeout.
Retrieved content may contain instructions that override the system prompt. Defense: separate instructions from retrieved content using different message roles, validate tool calls against allowed schemas regardless of context, detect instruction-like patterns in observations.
When guardrails trigger: stop execution, return a safe error message, log the event. Never return raw internal errors. Permission denied: 'This tool is not available.' Invalid arguments: explain the expected format. Log all events with caller, tool, args, and guardrail triggered.
Concrete Example
class Guardrail:
APPROVED_TOOLS = {"search", "read_file", "calculate"}
RISKY_TOOLS = {"delete_file", "send_email"}
def check(self, tool_name, args, user):
if tool_name not in self.APPROVED_TOOLS:
if tool_name in self.RISKY_TOOLS:
return {"status": "needs_approval", "message": f"Tool {tool_name} requires approval"}
return {"status": "blocked", "message": f"Unknown tool {tool_name}"}
if not self.validate_args(tool_name, args):
return {"status": "invalid", "message": "Argument validation failed"}
return {"status": "allowed"}Classifies tools by risk level. Unknown tools blocked. Risky tools require human approval. Argument validation catches malformed calls. This prevents agents from calling tools they shouldn't.
Define which tools and operations an agent is allowed to use before execution.
Require approval for risky operations with timeout fallback.
Detect and block instruction injection from untrusted sources.
Never crash or expose raw errors. Return safe, logged failure messages.
Log all guardrail events — blocked calls, approvals, failures — for review.
0 problems. Sign in to start solving.
Sign in to open a workspace and solve these problems.