Score final answers while rejecting leaked scratchpad, tool, or evaluator content.
The full prompt is available without an account.
Implement exact_match(preds, refs) returning accuracy in [0, 1].
Normalize both sides: lowercase, strip whitespace, collapse internal whitespace,
remove punctuation. Return sum(matches) / len(preds).
Read-only Python 3.12 preview. Sign in to edit and execute it.
import string
_PUNCT = str.maketrans("", "", string.punctuation)
def normalize(s: str) -> str:
# Lowercase, strip, collapse whitespace, and remove punctuation.
return ""
def exact_match(preds, refs) -> float:
# TODO
return 0.0