topabaem commited on
Commit
9636ed0
·
verified ·
1 Parent(s): c9ecb07

Correct the VRAM figure: allocated bytes undercount the card by 2 GiB

Browse files
Files changed (1) hide show
  1. README.md +29 -9
README.md CHANGED
@@ -139,16 +139,36 @@ Individually, the forge under the vendor's own sampler:
139
  what these clips do and do not establish. All thirty clips are in `samples/`,
140
  and the Space plays them side by side under its **BF16 vs 4-bit** tab.
141
 
142
- ## Memory
143
 
144
- | mode | resident | encode |
145
- |---|---:|---|
146
- | `resident` (default) | **7.75 GiB** | dequantizes inside `forward` |
147
- | `dequantized` | ~26.3 GB | one dense build at load |
148
-
149
- The two produce `torch.equal` conditioning, so the choice is footprint against
150
- speed and never quality. `resident` measured at **8.34 GiB** peak on a 16 GB
151
- V100 — the encoder fits one consumer card for the first time.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
152
 
153
  ## Use it
154
 
 
139
  what these clips do and do not establish. All thirty clips are in `samples/`,
140
  and the Space plays them side by side under its **BF16 vs 4-bit** tab.
141
 
142
+ ## Memory — what the card actually needs
143
 
144
+ | | resident (default) | dequantized |
145
+ |---|---:|---:|
146
+ | PyTorch allocated, peak | 8.48 GiB | ~22.3 GiB |
147
+ | PyTorch reserved, peak | 9.33 GiB | — |
148
+ | **what `nvidia-smi` shows** | **9.70 GiB** | — |
149
+
150
+ **Size your card from the last row.** The first is `max_memory_allocated`, which
151
+ counts only live allocator blocks — it misses the CUDA context and everything
152
+ the caching allocator reserved and has not handed back, and it undercounts by
153
+ nearly 2 GiB here. Earlier versions of this card quoted that number, and anyone
154
+ who bought an 8 GB card on the strength of it would have been wrong.
155
+
156
+ Measured on a **Tesla V100-SXM2-16GB (cc 7.0)** with a 55-token prompt, through
157
+ the ComfyUI-faithful path that left-pads to 1024 tokens. So:
158
+
159
+ * **16 GB and up** — comfortable.
160
+ * **12 GB** — fits.
161
+ * **10 GB** — fits, with roughly 300 MB of headroom. Nothing else on the card.
162
+ * **8 GB** — does not fit.
163
+
164
+ The two modes produce `torch.equal` conditioning, so the choice is footprint
165
+ against speed and never quality. Resident dequantizes inside `forward`, which
166
+ costs time on short prompts; dequantized builds one dense BF16 model at load and
167
+ then needs a card that can hold 26 GB.
168
+
169
+ These figures are for the **text encoder alone**. Generating video also needs the
170
+ DiT and the VAEs, which this repository does not contain — the pipeline in the
171
+ Space loads a `Q3_K_M` DiT alongside it.
172
 
173
  ## Use it
174