Tool Creation · Deep Dive
Tools fail. Networks timeout, APIs return 500s, rate limits trip. In an AI agent, a tool failure is a conversation — the LLM needs to know what went wrong so it can recover, retry, or change strategy.
Building robust tooling means distinguishing between errors the model can recover from and errors it cannot, implementing retry logic, and returning structured error observations.
Validation errors (malformed args) — never retryable. Runtime errors (crashes) — may be retryable if transient. Timeout errors — retryable if idempotent. Rate limit errors — retryable with waiting. Permission errors — never retryable.
Retryable errors: use exponential backoff (1s, 2s, 4s) with jitter to prevent thundering herd. Cap at 30-60s. Set max 3-5 attempts. After exhaustion, return a structured error observation.
Concrete Example
import asyncio, random
async def run_with_retry(tool_name, tool_fn, arguments, max_retries=3, base_delay=1.0, max_delay=30.0, timeout=10.0):
for attempt in range(max_retries + 1):
try:
if attempt > 0:
delay = min(base_delay * (2 ** (attempt - 1)), max_delay)
await asyncio.sleep(random.uniform(0, delay))
result = await asyncio.wait_for(asyncio.to_thread(tool_fn, arguments), timeout=timeout)
return {"status": "success", "data": result}
except asyncio.TimeoutError:
pass
except Exception as e:
pass
return {"status": "error", "error": f"Failed after {max_retries} retries", "retryable": True, "suggestion": "Try a different approach."}Wraps any tool call with exponential backoff using jitter. Timeout and runtime errors trigger retries. On final failure, returns a structured error observation with a suggestion for the LLM.
Validation, runtime, timeout, rate limit, permission. Handle each differently.
Only timeout, rate limit, 5xx are retryable. Validation and permission fail fast.
Increase wait time between retries and add random jitter.
Return errors with retryable flag and suggested next action.