---
license: apache-2.0
base_model:
  - Qwen/Qwen3-4B-Base
  - Qwen/Qwen3-8B-Base
tags: [research-backup, memory-graft, product-key-memory, capacity-expansion]
---

# memsplit-checkpoints

Research checkpoints from the FCUO / post-training capacity-expansion project (Qwen3), backed up
before B200 server release. Weights-only + sparse learned-memory (optimizer state and regenerable
calib caches excluded).

## License
Weights derive from **Qwen3 (Apache-2.0)** → Apache-2.0. **Exception:** `ckd_*` (agentic) were
trained partly on **APIGen-MT-5k (CC-BY-NC-4.0)** → those subdirs are **non-commercial / research-only**.
Training data: OpenR1-Math-220k (Apache-2.0), openwebmath (ODC-By), xlam-function-calling-60k (CC-BY-4.0),
ToolACE (Apache-2.0), APIGen-MT-5k (CC-BY-NC-4.0), PopQA. Third-party datasets are NOT redistributed here.

## Headline result
`pk_p2_frozen` — a FROZEN Qwen3-4B-Base backbone + a large content-addressed product-key memory
(dq512 / 65536 keys / 1.6B params) reaches **52.5% direct fact recall (full 14k set; 77.5 on the
first-3000 subset)** vs small-PK 24.3 and bare-frozen 0 — the project's strongest near-zero-inference
capacity positive (direct-probe only; generalization untested).

## Format
- Weights: standard HF `*.safetensors` + config/tokenizer (+ `ctx_gate.pt` for grafts, `pk_memory.pt` for PK).
- `sparse_memory/layer_*.pt` (learned memory grafts only): nonzero rows only. Reconstruct dense:
  ```python
  import torch
  d = torch.load("sparse_memory/layer_00.pt")
  dense = torch.zeros(d["V"], d["I"], dtype=d["mean_rows"].dtype)
  dense[d["rows"].long()] = d["mean_rows"]
  ```
- Graft checkpoints need their calib table to run; calib dirs are regenerable via
  `fcuo/calibrate_selfgen_math.py` (see code repo). Frozen-graft `trained_memory` (= calib copy) is omitted.

Code: github.com/hyunseoklee-ai/memory_split. Full result docs: private backup repo.