Inspect an agent config, execution trace, and eval results; classify failures; prioritise root causes; and produce a targeted review plan.
Full problem description visible. Upgrade to unlock the editor, test cases, and solution.
Implement diagnose_agent(config, trace, eval_results) that produces a structured diagnostic of an agent system.
config is a dict with keys like {"loop_type": str, "tools": list[str], "memory_enabled": bool, "guardrails_enabled": bool, "max_steps": int}.
trace is a list of step dicts, each with:
- step (int): iteration number.
- tool (str or null): tool called this step.
- status (str): "success", "failed", "blocked", or "timeout".
- latency_ms (int): step duration.
- tokens (int): tokens used.
eval_results is a dict with {"score": float, "categories": {"correctness": float, "safety": float, "efficiency": float, "robustness": float}}.
Requirements:
- Classify failures by dimension: loop, tools, memory, rag, safety, reliability, cost.
- Check config for missing components (no guardrails, no memory, low max_steps).
- Analyse trace for patterns: repeated tool calls (looping), high failure rate, timeouts, excessive token usage per step.
- Check eval_results for low-scoring categories.
- Prioritise root causes: missing components > critical failures in trace > low eval scores.
- Recommend remediations for each issue found.
- Return {"overall_rating": str, "issues": list[dict], "priorities": list[str], "review_plan": list[str]}.
- Each issue is {"dimension": str, "severity": "critical"|"major"|"minor", "finding": str, "remediation": str}.
₹999/monthLimited period launch pricing.
Get the full problem statement, test cases, interactive editor, solution explanation, and visual diagram.
SEVERITY_ORDER = {"critical": 0, "major": 1, "minor": 2}
DIMENSION_PRIORITY = {"safety": 0, "reliability": 1, "cost": 2}
def diagnose_agent(config, trace, eval_results):
issues = []Analyse config, trace, and eval results to produce diagnostic issues
issues.sort(key=lambda x: (SEVERITY_ORDER.get(x["severity"], 9),
DIMENSION_PRIORITY.get(x["dimension"], 9)))
priorities = [i["finding"] for i in issues[:3]]
overall = "pass" if not issues else "needs_review" if any(i["severity"] == "critical" for i in issues) else "needs_improvement"
review_plan = [i["remediation"] for i in issues[:5]]
return {"overall_rating": overall, "issues": issues,
"priorities": priorities, "review_plan": review_plan}Already have Pro access? Sign in