LiquidAI

/

LFM2.5-ColBERT-350M

like 112

Follow

Liquid AI 5.01k

Hello!

The MultiVectorEncoder class ships in the next Sentence Transformers release, planned for around the 18th, so for now the install below pulls from source. I would love to feature this model in that release's blog post and documentation, especially once it loads without the revision pin (that is, once this PR is merged).

Heads up, this PR was AI-generated and human-reviewed. Here's a summary of the changes as reported by my agent:

Pull Request overview

• Fix a transformers 5.x modeling incompatibility (an unexpected seq_idx kwarg to the LFM2 short-convolution) that stops the model from loading.
• Lead the README with a Sentence Transformers (MultiVectorEncoder) usage section and add the multi-vector tag.

Details

This checkpoint loads through its own bidirectional LFM2 modeling code (trust_remote_code). On transformers 5.x that currently fails: 5.x passes a seq_idx keyword into the LFM2 short-convolution, and this repository's _noncausal_shortconv_forward did not accept it, so loading raised a TypeError. The fix absorbs it with **kwargs (a no-op here, since seq_idx is None on this path). Verified bit-exact against the published transformers 4.56.2: max absolute difference 0.0, including a padded batch. The trained weights are untouched.

With that applied, the model loads directly into Sentence Transformers as a ColBERT-style late-interaction retriever, Transformer(Lfm2BidirectionalModel) -> Dense(1024 -> 128) scored with MaxSim, so it works with the familiar model.encode_query(...) / model.encode_document(...) / model.similarity(...) API. I led the README with a "Using Sentence Transformers" section (kept above the PyLate section) and added the multi-vector tag.

pip install "sentence-transformers @ git+https://github.com/huggingface/sentence-transformers.git"

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("LiquidAI/LFM2.5-ColBERT-350M", revision="refs/pr/3", trust_remote_code=True)

query = "Which planet is known as the Red Planet?"
documents = [
    "Venus is often called Earth's twin because of its similar size and proximity.",
    "Mars, known for its reddish appearance, is often referred to as the Red Planet.",
    "Jupiter, the largest planet in our solar system, has a prominent red spot.",
    "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.",
]

query_embeddings = model.encode_query([query])
document_embeddings = model.encode_document(documents)
print(query_embeddings[0].shape, document_embeddings[0].shape)
# (32, 128) (17, 128)

scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[27.1621, 28.2578, 27.7266, 28.1992]])

• Tom Aarsen