Group hidden eval failures into actionable feedback without exposing secret cases.
Full problem description visible. Upgrade to unlock the editor, test cases, and solution.
Implement summarize_failures(failures) that produces a readable summary
of hidden-test failures without exposing the raw detail text.
failures is a list of dicts: {"category": str, "detail": str}.
The detail field may contain sensitive test content that must not appear
verbatim in the output.
Return a dict mapping each category to a summary dict:
{"count": int, "failure_modes": int, "sample_modes": [str]}
where failure_modes is the number of distinct failure detail strings
and sample_modes is up to 3 example snippets (first 50 chars) to help
debug without revealing the complete hidden test.
₹999/monthLimited period launch pricing.
Get the full problem statement, test cases, interactive editor, solution explanation, and visual diagram.
def summarize_failures(failures):
groups = {}Group and summarize failures by category
Already have Pro access? Sign in