Harmonic: Hierarchical State Space Models for Efficient Long-Context Language Modeling
Paper • 2606.24650 • Published
Hallamonic 1B — Phase 1 checkpoint (intermediate)
Intermediate checkpoint from Phase 1 (SSM warmup, 10K steps, FFN frozen) of training Hallamonic 1B. This is provided for reproducibility/research into the training dynamics. For the final model, use Omibranch/harmonic-checkpoints-phase2-final.
Paper: arxiv.org/abs/2606.24650. Code: Omibranch/Harmonic.
License
Dual-licensed: free for noncommercial use under the PolyForm Noncommercial License 1.0.0. Commercial use requires a separate license — see LICENSE-COMMERCIAL.md.
Model tree for Omibranch/harmonic-checkpoints-phase1
Base model
TinyLlama/TinyLlama-1.1B-Chat-v1.0