maodd commited on
Commit
741cd88
·
verified ·
1 Parent(s): 05bcb11

Switch to llama-3.1-8b-instant: fresh, larger daily quota

Browse files

llama-3.3-70b-versatile's 100K tokens/day free-tier quota was exhausted by our own testing (98090/100000 used). Groq tracks quota per model - llama-3.1-8b-instant has a separate 500K/day budget, untouched.

Files changed (1) hide show
  1. app.py +2 -2
app.py CHANGED
@@ -66,7 +66,7 @@ class MemoryTrimmer:
66
  # ----- THIS IS WERE YOU CAN BUILD WHAT YOU WANT ------
67
  class BasicAgent:
68
  def __init__(self):
69
- model_id = os.getenv("AGENT_MODEL_ID", "groq/llama-3.3-70b-versatile")
70
  api_key = os.getenv("GROQ_API_KEY")
71
  if not api_key:
72
  print("Warning: GROQ_API_KEY is not set - the agent will fail to call the model.")
@@ -273,7 +273,7 @@ with gr.Blocks() as demo:
273
  Once clicking on the "submit button, it can take quite some time ( this is the time for the agent to go through all the questions).
274
  This space provides a basic setup and is intentionally sub-optimal to encourage you to develop your own, more robust solution. For instance for the delay process of the submit button, a solution could be to cache the answers and submit in a seperate action or even to answer the questions in async.
275
 
276
- **Setup:** This agent calls Groq (Llama 3.3 70B) via `smolagents`. Get a free key at https://console.groq.com/keys and set it as the `GROQ_API_KEY` secret in this Space's settings before running.
277
  """
278
  )
279
 
 
66
  # ----- THIS IS WERE YOU CAN BUILD WHAT YOU WANT ------
67
  class BasicAgent:
68
  def __init__(self):
69
+ model_id = os.getenv("AGENT_MODEL_ID", "groq/llama-3.1-8b-instant")
70
  api_key = os.getenv("GROQ_API_KEY")
71
  if not api_key:
72
  print("Warning: GROQ_API_KEY is not set - the agent will fail to call the model.")
 
273
  Once clicking on the "submit button, it can take quite some time ( this is the time for the agent to go through all the questions).
274
  This space provides a basic setup and is intentionally sub-optimal to encourage you to develop your own, more robust solution. For instance for the delay process of the submit button, a solution could be to cache the answers and submit in a seperate action or even to answer the questions in async.
275
 
276
+ **Setup:** This agent calls Groq (Llama 3.1 8B Instant) via `smolagents`. Get a free key at https://console.groq.com/keys and set it as the `GROQ_API_KEY` secret in this Space's settings before running.
277
  """
278
  )
279