maodd commited on
Commit
eebc9ca
·
verified ·
1 Parent(s): c9fc314

Switch model backend from Gemini to Groq

Browse files

Gemini free tier caps at 20 requests/day for gemini-3.6-flash - too low for a multi-step agent benchmark. Groq's llama-3.3-70b-versatile free tier allows 1000/day, 30 RPM. Uses GROQ_API_KEY.

Files changed (1) hide show
  1. app.py +12 -10
app.py CHANGED
@@ -24,7 +24,7 @@ DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
24
  @spaces.GPU
25
  def _zerogpu_startup_check():
26
  # This Space runs on ZeroGPU hardware but the agent below only makes
27
- # network calls (Gemini API, web search) and never touches CUDA.
28
  # ZeroGPU requires at least one @spaces.GPU function to be declared,
29
  # so this no-op satisfies that check without spending any GPU quota
30
  # (it is never actually invoked).
@@ -43,11 +43,13 @@ class RateLimiter:
43
  """
44
  Enforces a minimum delay between successive agent LLM calls (one call per
45
  step) as a step_callback, to stay under the model provider's
46
- requests-per-minute limit. Gemini's free tier for gemini-3.6-flash caps
47
- at 5 RPM, so the default here (12s + margin) targets that; override via
48
- RATE_LIMIT_SECONDS_BETWEEN_CALLS if your tier/model allows more.
 
 
49
  """
50
- def __init__(self, min_seconds_between_calls: float = 13.0):
51
  self.min_seconds_between_calls = min_seconds_between_calls
52
  self._last_call_at: float | None = None
53
 
@@ -65,10 +67,10 @@ class RateLimiter:
65
  # ----- THIS IS WERE YOU CAN BUILD WHAT YOU WANT ------
66
  class BasicAgent:
67
  def __init__(self):
68
- model_id = os.getenv("AGENT_MODEL_ID", "gemini/gemini-3.6-flash")
69
- api_key = os.getenv("GEMINI_API_KEY")
70
  if not api_key:
71
- print("Warning: GEMINI_API_KEY is not set - the agent will fail to call the model.")
72
 
73
  self.model = LiteLLMModel(model_id=model_id, api_key=api_key, temperature=0)
74
  self.agent = CodeAgent(
@@ -80,7 +82,7 @@ class BasicAgent:
80
  "collections", "statistics", "datetime", "io", "openpyxl", "PIL",
81
  ],
82
  max_steps=12,
83
- step_callbacks=[RateLimiter(float(os.getenv("RATE_LIMIT_SECONDS_BETWEEN_CALLS", "13.0")))],
84
  )
85
  print("BasicAgent initialized.")
86
 
@@ -253,7 +255,7 @@ with gr.Blocks() as demo:
253
  Once clicking on the "submit button, it can take quite some time ( this is the time for the agent to go through all the questions).
254
  This space provides a basic setup and is intentionally sub-optimal to encourage you to develop your own, more robust solution. For instance for the delay process of the submit button, a solution could be to cache the answers and submit in a seperate action or even to answer the questions in async.
255
 
256
- **Setup:** This agent calls Google's Gemini via `smolagents`. Get a free key at https://aistudio.google.com/apikey and set it as the `GEMINI_API_KEY` secret in this Space's settings before running.
257
  """
258
  )
259
 
 
24
  @spaces.GPU
25
  def _zerogpu_startup_check():
26
  # This Space runs on ZeroGPU hardware but the agent below only makes
27
+ # network calls (Groq API, web search) and never touches CUDA.
28
  # ZeroGPU requires at least one @spaces.GPU function to be declared,
29
  # so this no-op satisfies that check without spending any GPU quota
30
  # (it is never actually invoked).
 
43
  """
44
  Enforces a minimum delay between successive agent LLM calls (one call per
45
  step) as a step_callback, to stay under the model provider's
46
+ requests-per-minute limit. Groq's free tier for llama-3.3-70b-versatile
47
+ caps at 30 RPM, so the default here (2.5s) targets that with a small
48
+ margin; override via RATE_LIMIT_SECONDS_BETWEEN_CALLS for other tiers.
49
+ Note this only protects against per-minute limits - free tiers also
50
+ often cap total requests/day, which this can't work around.
51
  """
52
+ def __init__(self, min_seconds_between_calls: float = 2.5):
53
  self.min_seconds_between_calls = min_seconds_between_calls
54
  self._last_call_at: float | None = None
55
 
 
67
  # ----- THIS IS WERE YOU CAN BUILD WHAT YOU WANT ------
68
  class BasicAgent:
69
  def __init__(self):
70
+ model_id = os.getenv("AGENT_MODEL_ID", "groq/llama-3.3-70b-versatile")
71
+ api_key = os.getenv("GROQ_API_KEY")
72
  if not api_key:
73
+ print("Warning: GROQ_API_KEY is not set - the agent will fail to call the model.")
74
 
75
  self.model = LiteLLMModel(model_id=model_id, api_key=api_key, temperature=0)
76
  self.agent = CodeAgent(
 
82
  "collections", "statistics", "datetime", "io", "openpyxl", "PIL",
83
  ],
84
  max_steps=12,
85
+ step_callbacks=[RateLimiter(float(os.getenv("RATE_LIMIT_SECONDS_BETWEEN_CALLS", "2.5")))],
86
  )
87
  print("BasicAgent initialized.")
88
 
 
255
  Once clicking on the "submit button, it can take quite some time ( this is the time for the agent to go through all the questions).
256
  This space provides a basic setup and is intentionally sub-optimal to encourage you to develop your own, more robust solution. For instance for the delay process of the submit button, a solution could be to cache the answers and submit in a seperate action or even to answer the questions in async.
257
 
258
+ **Setup:** This agent calls Groq (Llama 3.3 70B) via `smolagents`. Get a free key at https://console.groq.com/keys and set it as the `GROQ_API_KEY` secret in this Space's settings before running.
259
  """
260
  )
261