multimodalart HF Staff commited on
Commit
691e8f4
·
verified ·
1 Parent(s): 08f770f

A real person as the subject reference (Pexels, Oliver Dohrn)

Browse files
Files changed (1) hide show
  1. README.md +31 -5
README.md CHANGED
@@ -20,7 +20,7 @@ This Space is the denoising half of the `ref2va` task: the 61.73 GiB `transforme
20
  autoencoders. The 62.14 GiB Qwen3-VL conditioner runs in
21
  [`minimax-h3-conditioner`](https://huggingface.co/spaces/diffusers-internal-dev/minimax-h3-conditioner), which this
22
  Space calls over the gradio API for every request — the same conditioner Space, and the same resident weights, that
23
- the keyframe half [`minimax-h3-generator`](https://huggingface.co/spaces/diffusers-internal-dev/minimax-h3-generator)
24
  uses.
25
 
26
  ## Why split
@@ -55,8 +55,10 @@ Rules the model imposes, enforced here before anything is uploaded:
55
 
56
  ## Example assets
57
 
58
- `examples/subject.png` is generated (FLUX.1-schnell), `examples/motion.mp4` is a synthetic clip from the parity
59
- fixtures, and `examples/voice.wav` is utterance `1462-170145-0022` of
 
 
60
  [LibriSpeech](https://www.openslr.org/12) `dev-clean` — CC BY 4.0, read from a public-domain LibriVox recording. It
61
  is 16 kHz mono on purpose: the audio VAE wants 32 kHz, so the example exercises the `torchaudio` resample the
62
  `ref2va` path needs.
@@ -139,6 +141,29 @@ no CFG and no negative prompt (it is guidance-distilled, so every step is one fo
139
  stops at 14 s because it is the *snapped* count that has to hold for the ceiling: 15 s is 360 frames, which rounds
140
  up to 362, i.e. 15.083 s, and is refused.
141
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
142
  ## Space variables
143
 
144
  | Variable | Default | Meaning |
@@ -146,8 +171,9 @@ up to 362, i.e. 15.083 s, and is refused.
146
  | `H3_CONDITIONER` | `diffusers-internal-dev/minimax-h3-conditioner` | The Space this one asks for embeddings. |
147
  | `H3_AOTI` | `0` | `1` loads the compiled block package. |
148
  | `H3_PLACEMENT` | `lazy` | `lazy` moves all 72.16 GiB onto the card on the first GPU call and leaves it there; `offload` hands placement to `ComponentsManager.enable_auto_cpu_offload` instead. |
149
- | `H3_ATTENTION` | `_native_cudnn` | cuDNN's fused kernel, 10–20% faster than the SDPA default and needs nothing installed. flash-attention 3 is sm90-only and this pool is sm120. |
150
- | `H3_GPU_DURATION` | `900` | Seconds per request; the pool applies a 1.5 duration factor. |
 
151
  | `H3_GPU_SIZE` | `xlarge` | ZeroGPU allocation size. `large` does not fit. |
152
 
153
  ## Required secret
 
20
  autoencoders. The 62.14 GiB Qwen3-VL conditioner runs in
21
  [`minimax-h3-conditioner`](https://huggingface.co/spaces/diffusers-internal-dev/minimax-h3-conditioner), which this
22
  Space calls over the gradio API for every request — the same conditioner Space, and the same resident weights, that
23
+ the keyframe half [`minimax-h3`](https://huggingface.co/spaces/diffusers-internal-dev/minimax-h3)
24
  uses.
25
 
26
  ## Why split
 
55
 
56
  ## Example assets
57
 
58
+ `examples/subject.png` is [a studio portrait by Oliver
59
+ Dohrn](https://www.pexels.com/photo/casual-portrait-of-smiling-woman-in-studio-31747504/) cropped to head and
60
+ shoulders — Pexels licence, free to use. `examples/motion.mp4` is a synthetic clip from the parity fixtures, and
61
+ `examples/voice.wav` is utterance `1462-170145-0022` of
62
  [LibriSpeech](https://www.openslr.org/12) `dev-clean` — CC BY 4.0, read from a public-domain LibriVox recording. It
63
  is 16 kHz mono on purpose: the audio VAE wants 32 kHz, so the example exercises the `torchaudio` resample the
64
  `ref2va` path needs.
 
141
  stops at 14 s because it is the *snapped* count that has to hold for the ceiling: 15 s is 360 frames, which rounds
142
  up to 362, i.e. 15.083 s, and is refused.
143
 
144
+ ## GPU time is reserved per request, not per Space
145
+
146
+ MiniMax-H3 attends over one packed sequence, so what a step costs is a function of that sequence's length alone —
147
+ and on this half the references dominate it. A single 1344x768 image reference is ~7168 conditioning rows plus the
148
+ vision block it puts in front of the prompt; a 2.5 s video reference is another ~17000. The same 960x544, 124-frame
149
+ request runs 2.4 s/step with no references and 16 s/step with an image and a video.
150
+
151
+ `get_duration` prices that before the call instead of reserving a flat ceiling for everything. It takes the
152
+ arguments of the `@spaces.GPU` function, so it has the conditioner's own `text_token_tags` (exact) and the reference
153
+ files (measured from metadata, no decode), and evaluates
154
+
155
+ ```
156
+ S = text rows + reference rows + target rows
157
+ seconds = placement + reference encode + steps * (LINEAR * S + QUADRATIC * S**2) * SAFETY + decode + pad
158
+ ```
159
+
160
+ fitted on the `t2va` half and checked against live `ref2va` requests to about 10%. It matters beyond tidiness: the
161
+ pool reserves whatever number it is given, and a flat 900 s is what makes a busy account fail admission with *"You
162
+ have too many ZeroGPU credits allocated to running tasks."* A typical single-image request now reserves ~460 s.
163
+
164
+ Every request carries the full placement allowance, because nothing on this side knows whether the worker it lands
165
+ on is cold and a cold one pays the lazy 72.16 GiB `PIPE.to("cuda")` inside its first GPU call.
166
+
167
  ## Space variables
168
 
169
  | Variable | Default | Meaning |
 
171
  | `H3_CONDITIONER` | `diffusers-internal-dev/minimax-h3-conditioner` | The Space this one asks for embeddings. |
172
  | `H3_AOTI` | `0` | `1` loads the compiled block package. |
173
  | `H3_PLACEMENT` | `lazy` | `lazy` moves all 72.16 GiB onto the card on the first GPU call and leaves it there; `offload` hands placement to `ComponentsManager.enable_auto_cpu_offload` instead. |
174
+ | `H3_ATTENTION` | `_native_cudnn` | cuDNN's fused kernel, 10–20% faster than the SDPA default and needs nothing installed. flash-attention 3 is sm90-only and this pool is sm120. The two VAEs are pinned to torch SDPA instead: they are float32, which cuDNN has no kernel for. |
175
+ | `H3_GPU_DURATION_MIN` / `_MAX` | `120` / `1500` | Bounds on what `get_duration` may reserve. |
176
+ | `H3_PLACEMENT_ALLOWANCE` | `90` | Seconds of the reservation set aside for a cold worker's placement. |
177
  | `H3_GPU_SIZE` | `xlarge` | ZeroGPU allocation size. `large` does not fit. |
178
 
179
  ## Required secret