Summarize a Growing Scratchpad

When the scratchpad exceeds a token limit, summarize the older portion while preserving the latest action and observation.

Problem statement

The full prompt is available without an account.

Implement run_agent(question, llm, tools, token_limit, summarizer) that compresses the scratchpad when it grows too long. Requirements: - Run a ReAct loop as usual, parsing Final Answer and Action lines. - Use the provided count_tokens(text) helper to check token usage. - After each observation is appended, if count_tokens(scratchpad) > token_limit, compress the scratchpad. - Compression strategy: keep the Question: line, use backward regex search to find the last Action: and Observation: lines, call summarizer(old_part) on everything before the last Action, and append a [Summary of earlier steps: <summary>]\n marker. - The most recent Action and Observation lines must remain intact after compression. - Stop on Final Answer or after MAX_STEPS=10. - Return "I could not find an answer." on exhaustion.

Starter code preview

Read-only Python 3.12 preview. Sign in to edit and execute it.

import re

MAX_STEPS = 10
ACTION_RE = re.compile(r"Action:\s*(\w+)\((.*)\)")
FINAL_RE = re.compile(r"Final Answer:\s*(.*)")

def count_tokens(text: str) -> int:
    return len(text.split())

def run_agent(question: str, llm, tools: dict,
              token_limit: int = 200, summarizer=None) -> str:
    scratchpad = f"Question: {question}\n"
    for _ in range(MAX_STEPS):
        output = llm(scratchpad)
        scratchpad += output + "\n"
        for line in output.split("\n"):
            m = FINAL_RE.search(line)
            if m:
                return m.group(1)
            m = ACTION_RE.search(line)
            if m:
                name, args = m.group(1), m.group(2)
                if name in tools:
                    result = tools[name](args)
                    scratchpad += f"Observation: {result}\n"
                else:
                    scratchpad += f"Observation: Unknown tool {name}\n"
                # TODO: check token count and compress if over limit
    return "I could not find an answer."

import re MAX_STEPS = 10 ACTION_RE = re.compile(r"Action:\s*(\w+)\((.*)\)") FINAL_RE = re.compile(r"Final Answer:\s*(.*)") def count_tokens(text: str) -> int: return len(text.split()) def run_agent(question: str, llm, tools: dict, token_limit: int = 200, summarizer=None) -> str: scratchpad = f"Question: {question}\n" for _ in range(MAX_STEPS): output = llm(scratchpad) scratchpad += output + "\n" for line in output.split("\n"): m = FINAL_RE.search(line) if m: return m.group(1) m = ACTION_RE.search(line) if m: name, args = m.group(1), m.group(2) if name in tools: result = tools[name](args) scratchpad += f"Observation: {result}\n" else: scratchpad += f"Observation: Unknown tool {name}\n" # TODO: check token count and compress if over limit return "I could not find an answer."