README.md

metadata

library_name: transformers
tags:
  - co-lmlm
  - retrieval

CoLMLM-360M-FW

The 360M-parameter, SmolLM2-based Co-LMLM retriever from the paper Co-LMLM: Continuous-Query Limited Memory Language Models, trained on FineWeb-Edu.

Co-LMLM is a retrieval-aware language model: at each <FACT> position it emits a continuous query from its hidden state that retrieves the fact's content from an external index at inference time, instead of storing it in its weights.

Usage

The model is used together with a released retrieval index. See the code repository for setup, index downloads, and a copy-pasteable quick start:

👉 github.com/lil-lab/Co-LMLM

This model is part of the Co-LMLM collection.

Citation

@misc{feldman2026colmlmcontinuousquerylimitedmemory,
      title={Co-LMLM: Continuous-Query Limited Memory Language Models},
      author={Yair Feldman and Linxi Zhao and Nathan Godey and Dongyoung Go and Yilun Hua and Kilian Q. Weinberger and Jennifer J. Sun and Yoav Artzi},
      year={2026},
      eprint={2607.07707},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2607.07707},
}