Spaces:
Sleeping
Sleeping
| TASK 1 — KV CACHE BENCHMARK | |
| ======================================== | |
| src_len standard(s) cached(s) speedup encoder% mem-save% | |
| 16 3.431 3.512 0.98x 133.1% 50.0% | |
| source-mem before=0.070MB after=0.035MB | |
| 32 3.626 3.555 1.02x 36.8% 50.0% | |
| source-mem before=0.141MB after=0.070MB | |
| 64 3.585 3.701 0.97x 53.3% 50.0% | |
| source-mem before=0.281MB after=0.141MB | |
| Encoder cost = % of one full forward pass | |
| Speedup = standard_time / cached_time | |
| Expected: speedup ≈ 1 / (1 - encoder_pct/100 * (T-1)/T) | |
| SUMMARY | |
| ------- | |
| src_len=16: 0.98x speedup (-2.4% time saved, encoder was 133.1% of total, estimated memory change 50.0%) | |
| src_len=32: 1.02x speedup (1.9% time saved, encoder was 36.8% of total, estimated memory change 50.0%) | |
| src_len=64: 0.97x speedup (-3.2% time saved, encoder was 53.3% of total, estimated memory change 50.0%) | |