Spaces:
Running on Zero
Running on Zero
Bill the conditioner call to the requesting user's own ZeroGPU token, never an org token
Browse files
README.md
CHANGED
|
@@ -183,31 +183,29 @@ on is cold and a cold one pays the lazy 72.16 GiB `PIPE.to("cuda")` inside its f
|
|
| 183 |
|
| 184 |
## Whose GPU quota pays
|
| 185 |
|
| 186 |
-
Two cards are booked per request: this Space's denoise loop and the conditioner's forward
|
| 187 |
-
|
| 188 |
-
|
| 189 |
-
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
an `xlarge` booking costs **twice** its seconds, so the conditioner books the encode (45 s) and a prompt upsample
|
| 203 |
-
(60 s) as **two separate calls**, where one combined booking would be refused outright.
|
| 204 |
|
| 205 |
## Secrets
|
| 206 |
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
|
| 210 |
-
|
| 211 |
|
| 212 |
## Where diffusers comes from
|
| 213 |
|
|
|
|
| 183 |
|
| 184 |
## Whose GPU quota pays
|
| 185 |
|
| 186 |
+
Two cards are booked per request: this Space's denoise loop and the conditioner's forward, and **both are meant to be
|
| 187 |
+
billed to the requesting user**. ZeroGPU attributes a booking to the `X-IP-Token` of the request that triggered it —
|
| 188 |
+
its `/schedule` hands that token to the Spaces API together with the duration and the calling pod's IP, and nothing
|
| 189 |
+
else, so there is no Space identity in the decision and a valid token minted anywhere is honoured. This Space
|
| 190 |
+
therefore forwards the caller's header to the conditioner
|
| 191 |
+
(`gradio_client.Client(..., headers={"X-IP-Token": ...})`, off the `gr.Request` gradio injects on
|
| 192 |
+
the UI path and the `/generate` API path alike) rather than spending a token of its own.
|
| 193 |
+
|
| 194 |
+
A token ZeroGPU refuses — it answers `401`, which `spaces` surfaces as `Expired ZeroGPU proxy token` — falls back to
|
| 195 |
+
calling the conditioner with no token at all, billed to this Space's pod IP off a small shared quota. That is a safety
|
| 196 |
+
net rather than the intended path, and the log line `conditioner call paid for by ...` records which identity actually
|
| 197 |
+
paid, next to a decoded dump of what the incoming token claimed.
|
| 198 |
+
|
| 199 |
+
The conditioner is sized so that even the fallback is legal: an unattributed caller may book at most 120 credits at a
|
| 200 |
+
time and an `xlarge` booking costs **twice** its seconds (`_gpu_size_units`), so the conditioner books the encode
|
| 201 |
+
(45 s) and a prompt upsample (60 s) as **two separate calls**, where one combined booking would be refused outright.
|
|
|
|
|
|
|
| 202 |
|
| 203 |
## Secrets
|
| 204 |
|
| 205 |
+
None are required. Everything this Space downloads is public — the
|
| 206 |
+
[`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3) checkpoint and the
|
| 207 |
+
[`multimodalart/minimax-h3-aoti`](https://huggingface.co/multimodalart/minimax-h3-aoti) packages — and the conditioner
|
| 208 |
+
is a public Space called on the requesting user's own ZeroGPU token, never on an org token.
|
| 209 |
|
| 210 |
## Where diffusers comes from
|
| 211 |
|
app.py
CHANGED
|
@@ -319,46 +319,37 @@ def conditioner(ip_token: str | None = None, hf_token: str | None = None):
|
|
| 319 |
key = (ip_token, hf_token)
|
| 320 |
if key in CLIENTS:
|
| 321 |
return CLIENTS[key]
|
| 322 |
-
|
| 323 |
-
|
| 324 |
-
|
| 325 |
-
headers={"X-IP-Token": ip_token} if ip_token else None,
|
| 326 |
-
)
|
| 327 |
if len(CLIENTS) >= 32:
|
| 328 |
CLIENTS.pop(next(iter(CLIENTS)))
|
| 329 |
CLIENTS[key] = client
|
| 330 |
return client
|
| 331 |
|
| 332 |
|
| 333 |
-
# What ZeroGPU says when an identity cannot pay for
|
| 334 |
-
#
|
| 335 |
-
# exhausted quota. All three mean "try the next identity" rather than "fail the request".
|
| 336 |
_UNPAYABLE = ("proxy token", "ZeroGPU quota", "larger than the maximum allowed", "GPU limit")
|
| 337 |
|
| 338 |
|
| 339 |
def call_conditioner(ip_token, **arguments):
|
| 340 |
-
"""One conditioner call,
|
| 341 |
-
|
| 342 |
-
In order of preference:
|
| 343 |
|
| 344 |
-
|
| 345 |
-
|
| 346 |
-
|
| 347 |
-
|
| 348 |
-
|
| 349 |
-
3. **no token at all** — an IP-based free quota, shared by everything calling out of this Space's egress IP and
|
| 350 |
-
worth a couple of requests a day. A last resort, not a design.
|
| 351 |
|
| 352 |
-
|
| 353 |
-
|
|
|
|
| 354 |
"""
|
| 355 |
api_name = arguments.pop("api_name")
|
| 356 |
-
attempts = []
|
| 357 |
if ip_token:
|
| 358 |
-
attempts.
|
| 359 |
-
if HF_TOKEN:
|
| 360 |
-
attempts.append(("this Space's HF_TOKEN", {"hf_token": HF_TOKEN}))
|
| 361 |
-
attempts.append(("no token, on an IP quota", {}))
|
| 362 |
|
| 363 |
for index, (label, identity) in enumerate(attempts):
|
| 364 |
try:
|
|
@@ -368,23 +359,39 @@ def call_conditioner(ip_token, **arguments):
|
|
| 368 |
except Exception as error:
|
| 369 |
if index == len(attempts) - 1 or not any(reason in str(error) for reason in _UNPAYABLE):
|
| 370 |
raise
|
| 371 |
-
print(f"[{LOG_TAG}] {label}: {error};
|
| 372 |
CLIENTS.pop((identity.get("ip_token"), identity.get("hf_token")), None)
|
| 373 |
|
| 374 |
|
| 375 |
def ip_token_of(request) -> str | None:
|
| 376 |
-
"""The
|
| 377 |
|
| 378 |
-
|
| 379 |
-
|
| 380 |
-
|
|
|
|
| 381 |
"""
|
| 382 |
headers = getattr(request, "headers", None)
|
| 383 |
token = None if headers is None else headers.get("x-ip-token")
|
| 384 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 385 |
return token
|
| 386 |
|
| 387 |
-
|
| 388 |
def probe(path: str) -> tuple[float | None, float | None]:
|
| 389 |
"""`(video seconds, audio seconds)` of a media file, either being `None` when the stream is absent."""
|
| 390 |
import av
|
|
|
|
| 319 |
key = (ip_token, hf_token)
|
| 320 |
if key in CLIENTS:
|
| 321 |
return CLIENTS[key]
|
| 322 |
+
# No token of this Space's own: the booking is meant to be the caller's, and an unattributed call is the
|
| 323 |
+
# fallback rather than a second identity to spend.
|
| 324 |
+
client = Client(CONDITIONER_SPACE, headers={"X-IP-Token": ip_token} if ip_token else None)
|
|
|
|
|
|
|
| 325 |
if len(CLIENTS) >= 32:
|
| 326 |
CLIENTS.pop(next(iter(CLIENTS)))
|
| 327 |
CLIENTS[key] = client
|
| 328 |
return client
|
| 329 |
|
| 330 |
|
| 331 |
+
# What ZeroGPU says when an identity cannot pay for a booking: a proxy token its `/usage-approval` refused (`401`,
|
| 332 |
+
# surfaced as "Expired ZeroGPU proxy token"), a duration past what that identity may book, and an exhausted quota.
|
|
|
|
| 333 |
_UNPAYABLE = ("proxy token", "ZeroGPU quota", "larger than the maximum allowed", "GPU limit")
|
| 334 |
|
| 335 |
|
| 336 |
def call_conditioner(ip_token, **arguments):
|
| 337 |
+
"""One conditioner call, billed to **the requesting user**.
|
|
|
|
|
|
|
| 338 |
|
| 339 |
+
ZeroGPU attributes a booking to the `X-IP-Token` of the request that triggered it: `/schedule` hands that token to
|
| 340 |
+
the Spaces API's `/usage-approval` together with the duration and the calling pod's IP, and nothing else — there is
|
| 341 |
+
no Space identity in that call, so a *valid* token minted anywhere is honoured and the user's own quota pays for
|
| 342 |
+
both halves of their request. That is the whole reason this Space forwards the header instead of spending a token
|
| 343 |
+
of its own.
|
|
|
|
|
|
|
| 344 |
|
| 345 |
+
A token the Spaces API refuses (`401`) falls back to calling the conditioner with no token, which is billed to this
|
| 346 |
+
Space's pod IP off a small shared quota. That is a safety net, not the intended path: the log line
|
| 347 |
+
`conditioner call paid for by ...` records which one actually paid, so a Space that keeps falling back is visible.
|
| 348 |
"""
|
| 349 |
api_name = arguments.pop("api_name")
|
| 350 |
+
attempts = [("no token, on this Space's shared IP quota", {})]
|
| 351 |
if ip_token:
|
| 352 |
+
attempts.insert(0, ("the requesting user's own ZeroGPU token", {"ip_token": ip_token}))
|
|
|
|
|
|
|
|
|
|
| 353 |
|
| 354 |
for index, (label, identity) in enumerate(attempts):
|
| 355 |
try:
|
|
|
|
| 359 |
except Exception as error:
|
| 360 |
if index == len(attempts) - 1 or not any(reason in str(error) for reason in _UNPAYABLE):
|
| 361 |
raise
|
| 362 |
+
print(f"[{LOG_TAG}] {label} was refused: {error}; falling back", flush=True)
|
| 363 |
CLIENTS.pop((identity.get("ip_token"), identity.get("hf_token")), None)
|
| 364 |
|
| 365 |
|
| 366 |
def ip_token_of(request) -> str | None:
|
| 367 |
+
"""The requesting user's ZeroGPU identity, as the Spaces router put it on this request.
|
| 368 |
|
| 369 |
+
Logged, decoded, on every request — never the token itself, only what it claims. ZeroGPU refuses a token its
|
| 370 |
+
`/usage-approval` considers expired, and that refusal is indistinguishable from a missing one in the outcome, so
|
| 371 |
+
the claims are what tell the two apart when a request ends up on the fallback quota. Both the UI path and the
|
| 372 |
+
`/generate` API path reach this through the same `gr.Request` gradio injects for a parameter annotated with it.
|
| 373 |
"""
|
| 374 |
headers = getattr(request, "headers", None)
|
| 375 |
token = None if headers is None else headers.get("x-ip-token")
|
| 376 |
+
if token is None:
|
| 377 |
+
print(f"[{LOG_TAG}] no X-IP-Token on this request; the conditioner call cannot be billed to the caller", flush=True)
|
| 378 |
+
return None
|
| 379 |
+
try:
|
| 380 |
+
import base64
|
| 381 |
+
import json
|
| 382 |
+
import time
|
| 383 |
+
|
| 384 |
+
payload = json.loads(base64.urlsafe_b64decode(f"{token.split('.')[1]}=="))
|
| 385 |
+
left = payload.get("exp", 0) - time.time()
|
| 386 |
+
print(
|
| 387 |
+
f"[{LOG_TAG}] X-IP-Token present: {left:.0f}s to expiry, claims "
|
| 388 |
+
f"{ {k: v for k, v in payload.items() if k in ('exp', 'iat', 'sub', 'aud', 'error', 'user')} }",
|
| 389 |
+
flush=True,
|
| 390 |
+
)
|
| 391 |
+
except Exception as error: # a token that cannot be read is still worth forwarding; ZeroGPU is the judge
|
| 392 |
+
print(f"[{LOG_TAG}] X-IP-Token present but unreadable ({type(error).__name__}: {error})", flush=True)
|
| 393 |
return token
|
| 394 |
|
|
|
|
| 395 |
def probe(path: str) -> tuple[float | None, float | None]:
|
| 396 |
"""`(video seconds, audio seconds)` of a media file, either being `None` when the stream is absent."""
|
| 397 |
import av
|