Serve Crello corpus aafdb0e86816 from the mounted bucket
Browse files- README.md +14 -0
- server/router.py +3 -1
README.md
CHANGED
|
@@ -92,6 +92,20 @@ Measured locally (`scripts/loadtest.py`, 4 workers, no errors):
|
|
| 92 |
|
| 93 |
The router itself saturates one core at about 300–450 tool calls per second, so 4–6 workers is the useful range.
|
| 94 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 95 |
## Talk to it
|
| 96 |
|
| 97 |
```python
|
|
|
|
| 92 |
|
| 93 |
The router itself saturates one core at about 300–450 tool calls per second, so 4–6 workers is the useful range.
|
| 94 |
|
| 95 |
+
On a private Space (`scripts/deploy_space.py`: cpu-upgrade, 8 vCPU / 32 GB, 4 workers × 48 sessions, the full
|
| 96 |
+
corpus read over HTTP), tested from India, where the network round trip is about 0.22 s:
|
| 97 |
+
|
| 98 |
+
| Load | Result | Throughput | Step p50 / p95 |
|
| 99 |
+
|---|---|---|---|
|
| 100 |
+
| Tools, 192 concurrent | 575/576 ok | 643 episodes/min, 110 calls/s | 0.26 / 0.8 s |
|
| 101 |
+
| Built-in GUI editor, 128 concurrent | 256/256 ok | 75 actions/s | 0.46 / 1.3 s |
|
| 102 |
+
|
| 103 |
+
- The one failure arrived at the capacity edge, just before a closing session was freed.
|
| 104 |
+
- At capacity, cold resets take 7–10 s; each new design is fetched once.
|
| 105 |
+
- More than 192 sessions are refused cleanly ("Server at capacity") after every worker has been tried.
|
| 106 |
+
- OpenEnv's client has no option for auth headers. `scripts/loadtest.py --hf-token-env HF_TOKEN` shows the two
|
| 107 |
+
wrappers a client needs to reach a private Space.
|
| 108 |
+
|
| 109 |
## Talk to it
|
| 110 |
|
| 111 |
```python
|
server/router.py
CHANGED
|
@@ -39,8 +39,10 @@ from starlette.responses import JSONResponse, Response, StreamingResponse
|
|
| 39 |
from starlette.routing import Route, WebSocketRoute
|
| 40 |
from starlette.websockets import WebSocket, WebSocketDisconnect
|
| 41 |
|
|
|
|
|
|
|
| 42 |
HOP = {"connection", "keep-alive", "proxy-authenticate", "proxy-authorization", "te", "trailers",
|
| 43 |
-
"transfer-encoding", "upgrade", "
|
| 44 |
EPISODE_ID = re.compile(rb'"episode_id"\s*:\s*"([A-Za-z0-9_.-]{1,64})"')
|
| 45 |
STICKY_PREFIXES = ("/web", "/gradio_api", "/gradio", "/theme.css", "/assets", "/file=", "/upload", "/queue",
|
| 46 |
"/config", "/info", "/heartbeat")
|
|
|
|
| 39 |
from starlette.routing import Route, WebSocketRoute
|
| 40 |
from starlette.websockets import WebSocket, WebSocketDisconnect
|
| 41 |
|
| 42 |
+
# hop-by-hop headers stay here. Host is forwarded: the playground (Gradio) builds its URLs from it, and with the
|
| 43 |
+
# worker's own address the browser is sent to 127.0.0.1 (an unstyled page on the Space)
|
| 44 |
HOP = {"connection", "keep-alive", "proxy-authenticate", "proxy-authorization", "te", "trailers",
|
| 45 |
+
"transfer-encoding", "upgrade", "content-length"}
|
| 46 |
EPISODE_ID = re.compile(rb'"episode_id"\s*:\s*"([A-Za-z0-9_.-]{1,64})"')
|
| 47 |
STICKY_PREFIXES = ("/web", "/gradio_api", "/gradio", "/theme.css", "/assets", "/file=", "/upload", "/queue",
|
| 48 |
"/config", "/info", "/heartbeat")
|