maodd commited on
Commit
43ea6cc
·
verified ·
1 Parent(s): 9190f43

Revert max_steps to 7, stop wasting steps on Wikipedia direct visits

Browse files

max_steps=12 blew Cerebras' daily token quota partway through the run, auto-failing the back half of questions (25% -> 10%). Reverted to 7. Also added an explicit instruction against visit_webpage on wikipedia.org URLs, which always 403s - use wikipedia_search instead.

Files changed (1) hide show
  1. app.py +10 -6
app.py CHANGED
@@ -40,6 +40,7 @@ If you are asked for a number, don't use commas to write it, and don't use units
40
  If you are asked for a string, don't use articles or abbreviations (e.g. for cities), and write digits in plain text unless specified otherwise.
41
  If you are asked for a comma separated list, apply the above rules to each element depending on whether it's a number or a string.
42
  Work efficiently: if a search or lookup doesn't find what you need after 1-2 tries, try a meaningfully different approach rather than repeating similar queries, and give your best-guess final_answer rather than exhausting all steps.
 
43
  """
44
 
45
  class TokenPacer:
@@ -137,12 +138,15 @@ class BasicAgent:
137
  "collections", "statistics", "datetime", "io", "openpyxl", "PIL",
138
  ]),
139
  ],
140
- # 12 (was 7): the 25% run showed several questions still making
141
- # real progress (finding the right source) when cut off at max
142
- # steps, forcing a garbled non-tool-call final answer instead of
143
- # a real one. TokenPacer sleeps stayed short (6-8s) at 7 steps,
144
- # so there's TPM budget headroom to let runs go longer.
145
- max_steps=12,
 
 
 
146
  step_callbacks=[
147
  MemoryTrimmer(),
148
  TokenPacer(tokens_per_minute_budget=int(os.getenv("RATE_LIMIT_TOKENS_PER_MINUTE", "25000"))),
 
40
  If you are asked for a string, don't use articles or abbreviations (e.g. for cities), and write digits in plain text unless specified otherwise.
41
  If you are asked for a comma separated list, apply the above rules to each element depending on whether it's a number or a string.
42
  Work efficiently: if a search or lookup doesn't find what you need after 1-2 tries, try a meaningfully different approach rather than repeating similar queries, and give your best-guess final_answer rather than exhausting all steps.
43
+ Never call visit_webpage on a wikipedia.org URL - it always returns 403 Forbidden. Use the wikipedia_search tool for Wikipedia content instead.
44
  """
45
 
46
  class TokenPacer:
 
138
  "collections", "statistics", "datetime", "io", "openpyxl", "PIL",
139
  ]),
140
  ],
141
+ # Reverted 12 -> 7: raising it caused several questions to run
142
+ # 12-13 steps each with growing context, and the cumulative
143
+ # token usage blew Cerebras' *daily* quota partway through the
144
+ # run (confirmed: "Tokens per day limit exceeded" starting
145
+ # around question 13) - every question after that auto-failed,
146
+ # dropping the score from 25% to 10%. 7 steps was empirically
147
+ # better: fewer wasted tokens per question, more questions
148
+ # actually get a real shot before the daily budget runs out.
149
+ max_steps=7,
150
  step_callbacks=[
151
  MemoryTrimmer(),
152
  TokenPacer(tokens_per_minute_budget=int(os.getenv("RATE_LIMIT_TOKENS_PER_MINUTE", "25000"))),