atakan Claude Sonnet 5 commited on
Commit
1498459
·
1 Parent(s): d9203b7

fix: Raise ZeroGPU call duration budget; drop stale llama.cpp comment

Browse files

Live logs showed "GPU task is exceeding its requested duration and
might be aborted" at the 120s budget -- the 6.4k-token prompt plus
generation genuinely takes longer than that on unoptimized transformers
generate(). Raised to 300s so calls aren't killed mid-flight while the
prompt-size optimization is still pending. Also removed a leftover
comment about llama.cpp KV-cache warm-up that no longer applies now
that the Space runs on transformers/ZeroGPU.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Files changed (1) hide show
  1. app.py +3 -7
app.py CHANGED
@@ -34,10 +34,6 @@ async def lifespan(app: FastAPI):
34
  print("Pre-loading ControlAI Core Engine on startup...")
35
  get_agent()
36
  print("ControlAI Core Engine is online and ready for traffic.")
37
- # Note: the system prompt + tool schemas (~6.4k tokens) are only cached
38
- # in llama.cpp after the first generation. Send one chat message after
39
- # a fresh deploy or after the Space wakes from sleep to warm it up —
40
- # every request after that reuses the cached prefix and is fast.
41
  yield
42
 
43
 
@@ -86,12 +82,12 @@ def get_agent() -> ControlAIAgent:
86
  # decorated with @spaces.GPU (it requests physical GPU time for the call and
87
  # releases it afterward). Outside of a ZeroGPU Space this decorator is a
88
  # harmless no-op, so it's safe to always wrap these.
89
- @spaces.GPU(duration=120)
90
  def _run_stream_on_gpu(agent: ControlAIAgent, message: str, history: list[dict[str, str]]):
91
  yield from agent.run_stream(message, history=history)
92
 
93
 
94
- @spaces.GPU(duration=120)
95
  def _run_on_gpu(agent: ControlAIAgent, message: str, history: list[dict[str, str]]):
96
  return agent.run(message, history=history, verbose=False)
97
 
@@ -99,7 +95,7 @@ def _run_on_gpu(agent: ControlAIAgent, message: str, history: list[dict[str, str
99
  # ZeroGPU's startup check statically looks for a @spaces.GPU function wired
100
  # to a Gradio event handler -- must be a module-level function referenced by
101
  # name, not one defined inline inside a `with gr.Blocks():` block.
102
- @spaces.GPU(duration=120)
103
  def _zerogpu_registration_probe(message: str) -> str:
104
  result = get_agent().run(message or "hi", history=[], verbose=False)
105
  return result.final_response
 
34
  print("Pre-loading ControlAI Core Engine on startup...")
35
  get_agent()
36
  print("ControlAI Core Engine is online and ready for traffic.")
 
 
 
 
37
  yield
38
 
39
 
 
82
  # decorated with @spaces.GPU (it requests physical GPU time for the call and
83
  # releases it afterward). Outside of a ZeroGPU Space this decorator is a
84
  # harmless no-op, so it's safe to always wrap these.
85
+ @spaces.GPU(duration=300)
86
  def _run_stream_on_gpu(agent: ControlAIAgent, message: str, history: list[dict[str, str]]):
87
  yield from agent.run_stream(message, history=history)
88
 
89
 
90
+ @spaces.GPU(duration=300)
91
  def _run_on_gpu(agent: ControlAIAgent, message: str, history: list[dict[str, str]]):
92
  return agent.run(message, history=history, verbose=False)
93
 
 
95
  # ZeroGPU's startup check statically looks for a @spaces.GPU function wired
96
  # to a Gradio event handler -- must be a module-level function referenced by
97
  # name, not one defined inline inside a `with gr.Blocks():` block.
98
+ @spaces.GPU(duration=300)
99
  def _zerogpu_registration_probe(message: str) -> str:
100
  result = get_agent().run(message or "hi", history=[], verbose=False)
101
  return result.final_response