| license: other | |
| tags: [etm, attention-free, ssm, causal-lm, korean] | |
| library_name: pytorch | |
| # ETM-Backbone (attention-free ETM+SSM causal LM) | |
| - **Stage**: pretrain | **Checkpoint**: step_100000 | **val_ppl**: 7.4864104236873255 | |
| - **Arch**: attention-free (dilated causal conv ETM + Mamba-style SSM), no positional embeddings -> context-length unbounded. | |
| - **Files**: model.pt (raw PyTorch state_dict), tokenizer.json (HF tokenizers, ChatML specials for SFT). | |
| - Custom architecture - load with the repo's ETMSSMLangModel (not AutoModel). vocab/d_model/layers auto-detected from state_dict. | |
| _Rolling latest mirror; overwritten twice daily by watchdog._ | |