File size: 3,116 Bytes
ca3d977 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | # Matched P-cache and retained-KV VRAM comparison
At the largest measured case, canonical P occupied 2.052 MiB while retained KV
tensors occupied 193.312 MiB. Canonical P was therefore about 94 times smaller
as a stored representation in this matched test. The two stores are not
equivalent. Canonical P keeps structured current facts, while retained KV keeps
recent token-level attention state.
Peak execution memory includes temporary computation. At 1,024 units, the
incremental peak was 103.491 MiB for P-only and 294.266 MiB for KV-only. These
peak values should not be confused with the cache tensor sizes above.
The comparison uses one frozen Pythia-1.4B model on one NVIDIA GeForce RTX
3050 Laptop GPU. Every row uses batch size 1, a float16 base, the same float32
TTL, greedy generation, eight generated tokens, and the same synthetic prompt
tokens at each workload size. All mechanisms are warmed once before measurement.
`P-cache only` disables retained runtime KV while leaving normal attention
working memory and the P-cache TTL path active. `KV only` retains model KV and
disables P-cache. `P-cache plus KV` enables both. This distinguishes the two
memory systems without claiming that their contents or purposes are
interchangeable.
Here, **retained KV** means token-level key-value tensors kept by the runtime.
**Incremental peak VRAM** means the additional maximum allocated GPU memory
above the same warmed model baseline.
| Prompt tokens and P slots | Condition | Canonical P | Retained KV | Baseline allocated | Peak allocated | Peak reserved | Incremental peak allocated | Runtime |
|---:|---|---:|---:|---:|---:|---:|---:|---:|
| 64 | P-cache only | 0.128 MiB | 0 | 2,717.183 MiB | 2,724.309 MiB | 2,776 MiB | 7.125 MiB | 0.2555 s |
| 64 | KV only | 0 | 13.312 MiB | 2,717.183 MiB | 2,735.575 MiB | 2,788 MiB | 18.392 MiB | 0.1754 s |
| 64 | P-cache plus KV | 0.128 MiB | 13.312 MiB | 2,717.183 MiB | 2,735.608 MiB | 2,788 MiB | 18.425 MiB | 0.2045 s |
| 256 | P-cache only | 0.513 MiB | 0 | 2,717.183 MiB | 2,744.347 MiB | 2,806 MiB | 27.164 MiB | 0.7415 s |
| 256 | KV only | 0 | 49.312 MiB | 2,717.183 MiB | 2,794.609 MiB | 2,852 MiB | 77.426 MiB | 0.1937 s |
| 256 | P-cache plus KV | 0.513 MiB | 49.312 MiB | 2,717.183 MiB | 2,794.739 MiB | 2,852 MiB | 77.556 MiB | 0.4156 s |
| 1,024 | P-cache only | 2.052 MiB | 0 | 2,717.183 MiB | 2,820.674 MiB | 2,938 MiB | 103.491 MiB | 2.7105 s |
| 1,024 | KV only | 0 | 193.312 MiB | 2,717.183 MiB | 3,011.449 MiB | 3,096 MiB | 294.266 MiB | 0.3753 s |
| 1,024 | P-cache plus KV | 2.052 MiB | 193.312 MiB | 2,717.183 MiB | 3,011.966 MiB | 3,114 MiB | 294.783 MiB | 1.2812 s |
All nine conditions completed. There were no OOM events, failures, fallbacks,
or estimated values. CUDA peak allocation includes transient attention, router,
TTL, output, and allocator behavior. Canonical P bytes and retained KV tensor
bytes are therefore reported separately from peak deltas.
Raw measurements are in [`vram-comparison.json`](../artifacts/vram-comparison.json). The generated table is
`vram_comparison.csv`, and the plot is `vram_comparison.svg`.
|