---
license: other
tags: [etm, attention-free, ssm, causal-lm, korean]
library_name: pytorch
---
# ETM-Backbone (attention-free ETM+SSM causal LM)

- **Stage**: pretrain  |  **Checkpoint**: step_100000  |  **val_ppl**: 7.4864104236873255
- **Arch**: attention-free (dilated causal conv ETM + Mamba-style SSM), no positional embeddings -> context-length unbounded.
- **Files**: model.pt (raw PyTorch state_dict), tokenizer.json (HF tokenizers, ChatML specials for SFT).
- Custom architecture - load with the repo's ETMSSMLangModel (not AutoModel). vocab/d_model/layers auto-detected from state_dict.

_Rolling latest mirror; overwritten twice daily by watchdog._