---
license: mit
tags:
  - block-sparse
  - recursive
  - language-model
  - parameter-offloading
  - research
library_name: pytorch
---

# SR-Core — Recursive Block-Sparse Language Models (checkpoints)

Trained checkpoints for the research monograph **Recursive Block-Sparse Language Models**
(SR-Core). Code, the full write-up, and all evaluation data are on GitHub:
**https://github.com/vijedich/sr-core-recursive-blocklm**

SR-Core is a block-sparse recursive LM with a hard working-set guarantee (WS=k): routing is
decided once (at recursion step r=1) and reused for all further steps, so exactly *k* blocks
are active per token, independent of bank size or recursion depth. This makes the per-token
weight footprint statically bounded — the property the monograph studies for cache-efficient
RAM→VRAM parameter streaming.

These are **research checkpoints**, not a deployment model. At this scale (~19M params, ~6.6M
token four-domain corpus) SR-Core does **not** match dense quality; it trails Dense d24 by
~0.5 nats (seen) / ~0.35 nats (held-out). Its contribution is transfer efficiency, not quality.
See the monograph for the full claim boundaries.

## Checkpoints

Each `.pt` is a full training snapshot (`model`, `optimizer`, `scheduler`, RNG state,
`step`, `config`) — self-describing, loadable via `rblm.model_io.load_checkpoint`, and
resumable for the convergence run (monograph §7.5.6).

| File pattern | What it is |
|---|---|
| `hm_cont_hm_srcore_b64_k8_R6_s{0-3}.pt` | SR-Core b64 k8 R6, 4 seeds — quality evidence |
| `hm_cont_hm_dense_d24_17k_s0.pt` | Dense d24 @17k — quality ceiling (seen 4.79) |
| `hm_cont_hm_dense_d24_s0.pt` | Dense d24 @10k — earlier snapshot |
| `hm_cont_hm_srcore_b64_k16_R4_d256h192_s{0-2}.pt` | Param/compute-matched — the ~0.5-nat gap evidence |
| `hm_cont_hm_srcore_b64_k8_R6_entmin_r1_lam003_s{0-3}.pt` | Entropy consolidation λ=0.003, 4 seeds |
| `hm_cont_hm_srcore_b64_k8_R6_entmin_r1_lam005_s{0-3}.pt` | Entropy consolidation λ=0.005, 4 seeds |
| `hm_cont_hm_srcore_b64_k8_R6_ctrl_17k_s{0,1}.pt` | Entropy-sweep control |
| `hm_cont_hm_srcore_b64_{k2,k8,k16}_R6_asw_s*.pt` | A-sweep depth-vs-capacity variants |

## Loading

```python
from rblm import model_io           # from the GitHub repo
model, arch, step = model_io.load_checkpoint("hm_cont_hm_srcore_b64_k8_R6_s0.pt")
```

## Citation

Archived on Zenodo with a citable DOI: **[10.5281/zenodo.21146547](https://doi.org/10.5281/zenodo.21146547)**
(concept DOI, resolves to the latest version; v1.0 is `10.5281/zenodo.21146548`).

```bibtex
@misc{jedich2026srcore,
  title  = {Entropy-based Router Consolidation for Cache-Efficient
            Recursive Block-Sparse Language Models},
  author = {Jedich, Viktor},
  year   = {2026},
  note   = {Self-published research monograph},
  doi    = {10.5281/zenodo.21146547},
  url    = {https://github.com/vijedich/sr-core-recursive-blocklm}
}
```

License: **MIT** (code and checkpoints). The monograph text is CC BY 4.0.