README.md

---
library_name: transformers
tags:
  - co-lmlm
  - retrieval
---

# CoLMLM-360M-FW

The 360M-parameter, SmolLM2-based **Co-LMLM** retriever from the paper
[**Co-LMLM: Continuous-Query Limited Memory Language Models**](https://arxiv.org/abs/2607.07707),
trained on FineWeb-Edu.

Co-LMLM is a retrieval-aware language model: at each `<FACT>` position it emits a continuous query
from its hidden state that retrieves the fact's content from an external index at inference time,
instead of storing it in its weights.

## Usage

The model is used together with a released retrieval index. See the code repository for setup,
index downloads, and a copy-pasteable quick start:

👉 **[github.com/lil-lab/Co-LMLM](https://github.com/lil-lab/Co-LMLM)**

This model is part of the [**Co-LMLM** collection](https://huggingface.co/collections/lil-lab/co-lmlm-6a4e8216d55eae83af348f57).

## Citation

```bibtex
@misc{feldman2026colmlmcontinuousquerylimitedmemory,
      title={Co-LMLM: Continuous-Query Limited Memory Language Models},
      author={Yair Feldman and Linxi Zhao and Nathan Godey and Dongyoung Go and Yilun Hua and Kilian Q. Weinberger and Jennifer J. Sun and Yoav Artzi},
      year={2026},
      eprint={2607.07707},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2607.07707},
}
```