Aggregate seeded eval runs to expose pass rates, instability, and stochastic behavior.
Full problem description visible. Upgrade to unlock the editor, test cases, and solution.
Implement aggregate(results) that computes aggregate statistics from seeded eval runs.
results is a list of dicts: {"seed": int, "passed": int, "total": int}.
Return a dict with:
- mean_pass_rate: Average per-seed pass rate (each seed weighted equally).
- total_passed: Sum of all passed counts across all runs.
- total_runs: Sum of all total counts across all runs.
- num_seeds: Number of unique seeds present.
If results is empty, return {"mean_pass_rate": 0.0, "total_passed": 0, "total_runs": 0, "num_seeds": 0}.
₹999/monthLimited period launch pricing.
Get the full problem statement, test cases, interactive editor, solution explanation, and visual diagram.
def aggregate(results):
if not results:
return {"mean_pass_rate": 0.0, "total_passed": 0, "total_runs": 0, "num_seeds": 0}Aggregate per-seed results
Already have Pro access? Sign in