Spaces:
Sleeping
Sleeping
Switch model backend from Gemini to Groq
Browse filesGemini free tier caps at 20 requests/day for gemini-3.6-flash - too low for a multi-step agent benchmark. Groq's llama-3.3-70b-versatile free tier allows 1000/day, 30 RPM. Uses GROQ_API_KEY.
app.py
CHANGED
|
@@ -24,7 +24,7 @@ DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
|
|
| 24 |
@spaces.GPU
|
| 25 |
def _zerogpu_startup_check():
|
| 26 |
# This Space runs on ZeroGPU hardware but the agent below only makes
|
| 27 |
-
# network calls (
|
| 28 |
# ZeroGPU requires at least one @spaces.GPU function to be declared,
|
| 29 |
# so this no-op satisfies that check without spending any GPU quota
|
| 30 |
# (it is never actually invoked).
|
|
@@ -43,11 +43,13 @@ class RateLimiter:
|
|
| 43 |
"""
|
| 44 |
Enforces a minimum delay between successive agent LLM calls (one call per
|
| 45 |
step) as a step_callback, to stay under the model provider's
|
| 46 |
-
requests-per-minute limit.
|
| 47 |
-
at
|
| 48 |
-
|
|
|
|
|
|
|
| 49 |
"""
|
| 50 |
-
def __init__(self, min_seconds_between_calls: float =
|
| 51 |
self.min_seconds_between_calls = min_seconds_between_calls
|
| 52 |
self._last_call_at: float | None = None
|
| 53 |
|
|
@@ -65,10 +67,10 @@ class RateLimiter:
|
|
| 65 |
# ----- THIS IS WERE YOU CAN BUILD WHAT YOU WANT ------
|
| 66 |
class BasicAgent:
|
| 67 |
def __init__(self):
|
| 68 |
-
model_id = os.getenv("AGENT_MODEL_ID", "
|
| 69 |
-
api_key = os.getenv("
|
| 70 |
if not api_key:
|
| 71 |
-
print("Warning:
|
| 72 |
|
| 73 |
self.model = LiteLLMModel(model_id=model_id, api_key=api_key, temperature=0)
|
| 74 |
self.agent = CodeAgent(
|
|
@@ -80,7 +82,7 @@ class BasicAgent:
|
|
| 80 |
"collections", "statistics", "datetime", "io", "openpyxl", "PIL",
|
| 81 |
],
|
| 82 |
max_steps=12,
|
| 83 |
-
step_callbacks=[RateLimiter(float(os.getenv("RATE_LIMIT_SECONDS_BETWEEN_CALLS", "
|
| 84 |
)
|
| 85 |
print("BasicAgent initialized.")
|
| 86 |
|
|
@@ -253,7 +255,7 @@ with gr.Blocks() as demo:
|
|
| 253 |
Once clicking on the "submit button, it can take quite some time ( this is the time for the agent to go through all the questions).
|
| 254 |
This space provides a basic setup and is intentionally sub-optimal to encourage you to develop your own, more robust solution. For instance for the delay process of the submit button, a solution could be to cache the answers and submit in a seperate action or even to answer the questions in async.
|
| 255 |
|
| 256 |
-
**Setup:** This agent calls
|
| 257 |
"""
|
| 258 |
)
|
| 259 |
|
|
|
|
| 24 |
@spaces.GPU
|
| 25 |
def _zerogpu_startup_check():
|
| 26 |
# This Space runs on ZeroGPU hardware but the agent below only makes
|
| 27 |
+
# network calls (Groq API, web search) and never touches CUDA.
|
| 28 |
# ZeroGPU requires at least one @spaces.GPU function to be declared,
|
| 29 |
# so this no-op satisfies that check without spending any GPU quota
|
| 30 |
# (it is never actually invoked).
|
|
|
|
| 43 |
"""
|
| 44 |
Enforces a minimum delay between successive agent LLM calls (one call per
|
| 45 |
step) as a step_callback, to stay under the model provider's
|
| 46 |
+
requests-per-minute limit. Groq's free tier for llama-3.3-70b-versatile
|
| 47 |
+
caps at 30 RPM, so the default here (2.5s) targets that with a small
|
| 48 |
+
margin; override via RATE_LIMIT_SECONDS_BETWEEN_CALLS for other tiers.
|
| 49 |
+
Note this only protects against per-minute limits - free tiers also
|
| 50 |
+
often cap total requests/day, which this can't work around.
|
| 51 |
"""
|
| 52 |
+
def __init__(self, min_seconds_between_calls: float = 2.5):
|
| 53 |
self.min_seconds_between_calls = min_seconds_between_calls
|
| 54 |
self._last_call_at: float | None = None
|
| 55 |
|
|
|
|
| 67 |
# ----- THIS IS WERE YOU CAN BUILD WHAT YOU WANT ------
|
| 68 |
class BasicAgent:
|
| 69 |
def __init__(self):
|
| 70 |
+
model_id = os.getenv("AGENT_MODEL_ID", "groq/llama-3.3-70b-versatile")
|
| 71 |
+
api_key = os.getenv("GROQ_API_KEY")
|
| 72 |
if not api_key:
|
| 73 |
+
print("Warning: GROQ_API_KEY is not set - the agent will fail to call the model.")
|
| 74 |
|
| 75 |
self.model = LiteLLMModel(model_id=model_id, api_key=api_key, temperature=0)
|
| 76 |
self.agent = CodeAgent(
|
|
|
|
| 82 |
"collections", "statistics", "datetime", "io", "openpyxl", "PIL",
|
| 83 |
],
|
| 84 |
max_steps=12,
|
| 85 |
+
step_callbacks=[RateLimiter(float(os.getenv("RATE_LIMIT_SECONDS_BETWEEN_CALLS", "2.5")))],
|
| 86 |
)
|
| 87 |
print("BasicAgent initialized.")
|
| 88 |
|
|
|
|
| 255 |
Once clicking on the "submit button, it can take quite some time ( this is the time for the agent to go through all the questions).
|
| 256 |
This space provides a basic setup and is intentionally sub-optimal to encourage you to develop your own, more robust solution. For instance for the delay process of the submit button, a solution could be to cache the answers and submit in a seperate action or even to answer the questions in async.
|
| 257 |
|
| 258 |
+
**Setup:** This agent calls Groq (Llama 3.3 70B) via `smolagents`. Get a free key at https://console.groq.com/keys and set it as the `GROQ_API_KEY` secret in this Space's settings before running.
|
| 259 |
"""
|
| 260 |
)
|
| 261 |
|