README.md

---
license: apache-2.0
library_name: pytorch
tags:
- memsft
- memory-decoder
- router
- qwen3
- biology
- biology-instructions
- openswi
- law
---

# MemSFT-Qwen3-Routers

<p align="center">
  <a href="https://arxiv.org/abs/2607.25614">📄 Paper</a> •
  <a href="https://github.com/LUMIA-Group/MemSFT">💻 GitHub</a> •
  <a href="https://huggingface.co/collections/Jiarui-Wang/memsft">🤗 HF Collection</a>
</p>

## Overview

This repository contains the learned token-level routers for the Qwen3
MemSFT release. A router predicts how the frozen backbone and an external
domain memory should be combined at each decoding step.

The router checkpoints are auxiliary components, not standalone language
models. Each router must be used with its matching backbone and memory
checkpoint through the MemSFT implementation.

## Available Routers

| Domain | Backbone | Memory | Router directory |
|---|---|---|---|
| Biology-Instructions | [`Qwen3-8B`](https://huggingface.co/Qwen/Qwen3-8B) | [`MemSFT-Qwen3-Bio-Memory-8B`](https://huggingface.co/Jiarui-Wang/MemSFT-Qwen3-Bio-Memory-8B) | `Qwen3-8B-Bio-M8B-Router` |
| Biology-Instructions | [`Qwen3-14B`](https://huggingface.co/Qwen/Qwen3-14B) | [`MemSFT-Qwen3-Bio-Memory-8B`](https://huggingface.co/Jiarui-Wang/MemSFT-Qwen3-Bio-Memory-8B) | `Qwen3-14B-Bio-M8B-Router` |
| Biology-Instructions | [`Qwen3-32B`](https://huggingface.co/Qwen/Qwen3-32B) | [`MemSFT-Qwen3-Bio-Memory-8B`](https://huggingface.co/Jiarui-Wang/MemSFT-Qwen3-Bio-Memory-8B) | `Qwen3-32B-Bio-M8B-Router` |
| Biology-Instructions | [`Qwen3-235B-A22B`](https://huggingface.co/Qwen/Qwen3-235B-A22B) | [`MemSFT-Qwen3-Bio-Memory-8B`](https://huggingface.co/Jiarui-Wang/MemSFT-Qwen3-Bio-Memory-8B) | `Qwen3-235B-A22B-Bio-M8B-Router` |
| Biology-Instructions | [`Qwen3-14B`](https://huggingface.co/Qwen/Qwen3-14B) | [`MemSFT-Qwen3-Bio-Memory-1.7B`](https://huggingface.co/Jiarui-Wang/MemSFT-Qwen3-Bio-Memory-1.7B) | `Qwen3-14B-Bio-M1.7B-Router` |
| Biology-Instructions | [`Qwen3-14B`](https://huggingface.co/Qwen/Qwen3-14B) | [`MemSFT-Qwen3-Bio-Memory-4B`](https://huggingface.co/Jiarui-Wang/MemSFT-Qwen3-Bio-Memory-4B) | `Qwen3-14B-Bio-M4B-Router` |
| OpenSWI | [`Qwen3-8B`](https://huggingface.co/Qwen/Qwen3-8B) | [`MemSFT-Qwen3-OpenSWI-Memory-8B`](https://huggingface.co/Jiarui-Wang/MemSFT-Qwen3-OpenSWI-Memory-8B) | `Qwen3-8B-OpenSWI-M8B-Router` |
| OpenSWI | [`Qwen3-14B`](https://huggingface.co/Qwen/Qwen3-14B) | [`MemSFT-Qwen3-OpenSWI-Memory-8B`](https://huggingface.co/Jiarui-Wang/MemSFT-Qwen3-OpenSWI-Memory-8B) | `Qwen3-14B-OpenSWI-M8B-Router` |
| OpenSWI | [`Qwen3-32B`](https://huggingface.co/Qwen/Qwen3-32B) | [`MemSFT-Qwen3-OpenSWI-Memory-8B`](https://huggingface.co/Jiarui-Wang/MemSFT-Qwen3-OpenSWI-Memory-8B) | `Qwen3-32B-OpenSWI-M8B-Router` |
| OpenSWI | [`Qwen3-235B-A22B`](https://huggingface.co/Qwen/Qwen3-235B-A22B) | [`MemSFT-Qwen3-OpenSWI-Memory-8B`](https://huggingface.co/Jiarui-Wang/MemSFT-Qwen3-OpenSWI-Memory-8B) | `Qwen3-235B-A22B-OpenSWI-M8B-Router` |
| Law | [`Qwen3-14B`](https://huggingface.co/Qwen/Qwen3-14B) | [`MemSFT-Qwen3-Law-Memory-8B`](https://huggingface.co/Jiarui-Wang/MemSFT-Qwen3-Law-Memory-8B) | `Qwen3-14B-Law-M8B-Router` |

## Quick Start

Download the router that matches the selected backbone and memory:

```python
from pathlib import Path

from huggingface_hub import snapshot_download

router_subdir = "Qwen3-14B-Bio-M8B-Router"
router_root = snapshot_download(
    repo_id="Jiarui-Wang/MemSFT-Qwen3-Routers",
    revision="v1.0",
    allow_patterns=[f"{router_subdir}/*"],
)
router_path = str(Path(router_root) / router_subdir)
```

Pass `router_path` together with the loaded backbone and memory to MemSFT:

```python
from router.adaptive_memdec import AdaptiveMemoryDecoder

model = AdaptiveMemoryDecoder(
    base_lm=base,
    knn_generator=memory,
    router_path=router_path,
    router_device=device,
).eval()
model.set_tokenizer(tokenizer)
```

See the [MemSFT repository](https://github.com/LUMIA-Group/MemSFT) and the
model card for the selected memory checkpoint for complete installation,
loading, and generation examples.

## Intended Use and Limitations

These checkpoints are intended for reproducing the evaluated MemSFT Qwen3
configurations listed above. A router should not be used with a different
backbone, memory size, or domain unless that combination is separately
trained and evaluated. Performance outside the listed combinations has not
been established.

## License

The MemSFT router checkpoints are released under the Apache License 2.0.
Upstream models, software, and datasets remain subject to their respective
licenses and terms.

## Citation

If you find MemSFT helpful in your research, please consider citing:

```bibtex
@misc{wang2026memsftmitigatingalignmenttax,
      title={MemSFT: Mitigating Alignment Tax with an External Parametric Memory},
      author={Jiarui Wang and Xiang Shi and Jiaqi Cao and Rubin Wei and Xiquan Wang and Hao Sun and Jingzhi Wang and Zhiqi Yang and Qipeng Guo and Bowen Zhou and Zhouhan Lin},
      year={2026},
      eprint={2607.25614},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2607.25614},
}
```

## Contact

For questions and discussions, feel free to email
[wangjiarui1@sjtu.edu.cn](mailto:wangjiarui1@sjtu.edu.cn).