atakan Claude Sonnet 5 commited on
Commit
c01ad45
·
1 Parent(s): f22161c

revert: Don't block app startup on a warm-up generation

Browse files

Measured on Spaces' CPU tier, the ~6.4k-token warm-up prefill takes
much longer than on dev hardware and was blocking uvicorn from binding
at all, risking a platform startup timeout. Startup is fast again;
send one chat message after a fresh deploy or Space wake-up to warm
the KV cache manually instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Files changed (1) hide show
  1. app.py +5 -6
app.py CHANGED
@@ -29,13 +29,12 @@ from controlai_rag.index import ControlRAGIndex
29
  @asynccontextmanager
30
  async def lifespan(app: FastAPI):
31
  print("Pre-loading ControlAI Core Engine on startup...")
32
- agent = get_agent()
33
- print("Warming up KV cache for the system prompt + tool schemas...")
34
- try:
35
- agent.run("hi", verbose=False)
36
- except Exception as exc:
37
- print(f"Warning: warm-up generation failed (non-fatal): {exc}")
38
  print("ControlAI Core Engine is online and ready for traffic.")
 
 
 
 
39
  yield
40
 
41
 
 
29
  @asynccontextmanager
30
  async def lifespan(app: FastAPI):
31
  print("Pre-loading ControlAI Core Engine on startup...")
32
+ get_agent()
 
 
 
 
 
33
  print("ControlAI Core Engine is online and ready for traffic.")
34
+ # Note: the system prompt + tool schemas (~6.4k tokens) are only cached
35
+ # in llama.cpp after the first generation. Send one chat message after
36
+ # a fresh deploy or after the Space wakes from sleep to warm it up —
37
+ # every request after that reuses the cached prefix and is fast.
38
  yield
39
 
40