Spaces:
Sleeping
Sleeping
File size: 984 Bytes
29e5bf8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | TASK 1 — KV CACHE BENCHMARK
========================================
src_len standard(s) cached(s) speedup encoder% mem-save%
16 3.431 3.512 0.98x 133.1% 50.0%
source-mem before=0.070MB after=0.035MB
32 3.626 3.555 1.02x 36.8% 50.0%
source-mem before=0.141MB after=0.070MB
64 3.585 3.701 0.97x 53.3% 50.0%
source-mem before=0.281MB after=0.141MB
Encoder cost = % of one full forward pass
Speedup = standard_time / cached_time
Expected: speedup ≈ 1 / (1 - encoder_pct/100 * (T-1)/T)
SUMMARY
-------
src_len=16: 0.98x speedup (-2.4% time saved, encoder was 133.1% of total, estimated memory change 50.0%)
src_len=32: 1.02x speedup (1.9% time saved, encoder was 36.8% of total, estimated memory change 50.0%)
src_len=64: 0.97x speedup (-3.2% time saved, encoder was 53.3% of total, estimated memory change 50.0%)
|