Commit History
-
Day 1 of 7-day Kimi K2.6 + RULER 128K sprint: transformers 5.x port + real-weights install validation
GENOMA LABS / research +1: Claude Opus 4.7 (1M context)Patch ported to dual-API (transformers 4.x + 5.x): - _maybe_evict reads / writes both DynamicCache.key_cache[i] (4.x) and DynamicCache.layers[i].keys / .values (5.x), detected by hasattr. - _make_evicting_forward accepts the union of 4.x and 5.x DeepseekV3Attention signatures via *args / **kwargs spreading. Calls original_forward positionally to avoid hidden_states= keyword collision under positional spread. - Defensive 2-tuple / 3-tuple unpacking of forward return. Validation: - B1 mock-cache multi-step: still PASSes on the ported patch. 500 steps x 4 layers, 1824 eviction events, cache stabilizes at exactly n_sink + budget + n_recent, 1029 steps/sec on CPU. (results/validate_eviction_random_init.csv) - New scripts/validate_install_on_real_kimi_layer.py exercises install_kv_eviction against a real DeepseekV3Attention layer instantiated with actual Kimi K2.6 layer-0 weights from the published checkpoint. Patched forward executes cleanly on TITAN RTX in 0.09s for seq_…95298f5 -
cleanup: gitignore pycache + remove stray empty CSV from earlier debug run
GENOMA LABS / researchd6db53d -
B1 validation: multi-step eviction test + transformers compatibility note
GENOMA LABS / researchAdds: - scripts/validate_eviction_random_init.py: 1000-step mock-cache validation exercising the H2O eviction logic across 4 layers with canonical DeepseekV3 cache dimensions (qk_dim=192, v_dim=128, FP16). - notebooks/02_validation_results.md: documented results, interpretation, reproduction instructions. - results/validate_eviction_random_init.csv: per-step cache size + eviction events, 1000 rows. Validation result: PASS. Cache stabilizes at exactly n_sink+budget+n_recent, 3664 eviction events triggered correctly across 916 post-cap steps, 913 steps/sec on CPU. Patch fixes (uncovered by attempting end-to-end with transformers 5.7.0): - Bound forward method called without re-passing self. - Defensive unpacking of original_forward return value (handles 2-tuple modern transformers and 3-tuple older versions). - Return arity matches what original_forward returned, so calling decoder layer unpacks correctly. Roadmap updates: - Multi-step validation marked complete. - API por…1ba26d6 -
initial release: H2O KV cache eviction for DeepseekV3 / MLA architectures
GENOMA LABS / researchReference implementation of importance-based KV cache eviction (H2O, Zhang et al. 2023) patched onto DeepseekV3Attention layers. Compatible with DeepSeek V3 / V3.2, Kimi K2 / K2.6, and any future MLA-based model that subclasses the same attention layer. Memory model: 82 GB -> 10.2 GB at budget=4096 on canonical 61-layer DeepseekV3 layout. Roughly 2-4x prefill speedup at 128K-1M context with no retraining and no architectural changes. Contents: - src/kv_eviction_mla.py H2O patch + smoke test (277 LOC) - notebooks/01_smoke_test_walkthrough.md annotated explanation + memory model - docs/HOW_IT_WORKS.md where eviction hooks into the layer + porting notes - LICENSE Apache 2.0 - README.md quick start + budget guidance Companion to GENOMA LABS / research handbook on sub-quadratic attention (github.com/genoma-labs/subquadratic-research, public on flip).a8d4591