README.md

metadata

library_name: sentence-transformers
tags:
  - colbert
  - late-interaction
  - retrieval
  - pylate
  - multi-vector
language:
  - en
base_model:
  - lightonai/LateOn-unsupervised
license: apache-2.0
pipeline_tag: sentence-similarity

GLInt

GLINT is a SOTA 149M-parameter English late-interaction retriever built from LateOn-unsupervised. It retains 128-dimensional token embeddings and uses MaxSim retrieval with 32 query tokens and 300 document tokens.

What is new in GLInt?

GLINT is designed around the mismatch between ordinary dense hard-negative mining and a late-interaction retriever. Dense mining selects documents that are difficult under one pooled vector; GLINT instead mines negatives under the same token-level MaxSim geometry used at retrieval time. This exposes lexical, compositional, and localized token matches that a single-vector miner can miss.

The training recipe has two stages:

1. supervised fine-tuning with multi-vector (MaxSim) hard negatives;
2. mixed listwise knowledge distillation over a diverse seven-source hard-negative mixture.

For the second stage, a frozen listwise teacher (jinaai/jina-reranker-v3.5) scores each 32-document candidate set jointly. GLINT distils that ordering with a sharpened listwise KL objective, while a false-negative-masked InfoNCE term preserves a direct retrieval signal.

Usage

Sentence Transformers

This model can be used with Sentence Transformers as a multi-vector (ColBERT-style late interaction) retriever via the MultiVectorEncoder:

pip install "sentence-transformers>=6.0.0"

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("chungimungi/GLInt")

query = "Which planet is known as the Red Planet?"
documents = [
    "Venus is often called Earth's twin because of its similar size and proximity.",
    "Mars, known for its reddish appearance, is often referred to as the Red Planet.",
    "Jupiter, the largest planet in our solar system, has a prominent red spot.",
    "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.",
]

query_embeddings = model.encode_query(query)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings[0].shape)
# torch.Size([12, 128]) torch.Size([18, 128])

# MaxSim late-interaction scoring (higher is more relevant)
scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[11.6192, 11.7344, 11.6513, 11.7105]], device='cuda:0')

PyLate

from pylate import models

model = models.ColBERT("chungimungi/GLInt")
query_embeddings = model.encode(["what causes a lunar eclipse?"], is_query=True)
document_embeddings = model.encode(
    ["A lunar eclipse happens when Earth passes between the Sun and the Moon."],
    is_query=False,
)

Use a late-interaction backend such as PyLate/PLAID for corpus-scale retrieval. Scores are computed by summing, over query tokens, the maximum similarity to a document token.

Results

BEIR (15 datasets, NDCG@10)

Model	Average	Size (M)	Embed dim	ArguAna	CQADupstackRetrieval	ClimateFEVER	DBPedia	FEVER	FiQA2018	HotpotQA	MSMARCO	NFCorpus	NQ	QuoraRetrieval	SCIDOCS	SciFact	TRECCOVID	Touche2020
ColBERTv2	48.63	110	128	46.50	38.30	17.60	45.20	78.50	35.40	67.50	46.00	33.70	52.40	85.50	15.40	68.90	72.60	26.00
Jina-ColBERT-v2	51.85	600	128	36.60	40.80	23.90	47.10	80.50	40.80	76.60	46.90	34.60	64.00	88.70	18.60	67.80	83.40	27.40
ColBERT-small	53.79	33	96	50.09	38.75	33.07	45.58	90.96	41.15	76.11	43.50	37.30	59.10	87.72	18.42	74.77	84.59	25.69
GTE-ModernColBERT-v1	54.75	149	128	47.52	41.08	31.33	47.56	87.67	45.25	77.48	45.60	37.83	61.62	86.71	19.22	76.33	84.84	31.25
ColBERT-Zero	55.39	149	128	52.82	41.41	35.90	47.43	90.52	42.50	79.45	45.95	37.21	61.82	85.19	19.84	76.33	78.27	36.24
LateOn-unsupervised	50.11	149	128	43.12	47.71	18.76	43.36	65.74	51.94	68.17	37.51	37.15	58.41	89.48	21.13	76.89	69.81	22.53
LateOn	57.22	149	128	50.52	47.36	39.67	45.99	92.02	53.12	79.98	45.67	37.79	63.91	89.67	21.90	76.61	83.60	30.52
GLInt	57.43	149	128	52.38	46.49	34.17	47.68	92.45	50.85	82.54	46.38	37.51	68.03	90.08	20.65	77.13	84.78	30.26

BEIR-Decontaminated (14 datasets, NDCG@10)

Model	Average	ArguAna	ClimateFEVER	DBPedia	FEVER	FiQA2018	HotpotQA	MS MARCO	NFCorpus	Natural Questions	Quora	SciDocs	SciFact	TREC-COVID	Touché-2020
GLInt	62.50	51.67	36.35	42.50	92.89	56.88	81.16	72.70	26.21	94.97	92.06	22.02	89.07	81.51	34.97
LateOn	61.4	52.2	42.1	31.7	92.7	57.9	78.9	70.3	27.0	93.1	91.5	15.1	88.9	80.9	36.8
DenseOn	58.8	40.0	39.5	28.8	91.2	55.9	73.7	68.9	28.5	92.1	91.1	14.7	85.4	82.5	31.0
pplx-embed-v1-0.6b	59.7	43.7	42.4	28.4	91.1	55.2	73.5	71.9	28.0	91.6	91.5	15.4	89.0	83.7	30.0
jina-v5-text-nano	58.8	47.2	41.6	30.2	90.0	51.5	67.5	68.6	29.4	92.3	91.3	14.9	89.4	76.8	33.2
harrier-oss-v1-0.6b	58.0	47.4	25.7	31.3	80.7	50.1	71.4	73.4	27.9	90.0	90.9	17.1	90.7	81.8	33.3
arctic-embed-l-v2	57.9	43.1	45.7	45.7	92.2	50.4	63.1	71.0	26.0	90.7	91.3	13.9	87.4	81.4	26.8
bge-large-en-v1.5	57.3	46.0	39.0	28.9	87.6	49.3	75.2	68.9	29.8	85.9	91.3	14.0	86.5	72.7	26.9
Qwen3-Embedding-0.6B	57.0	48.4	38.0	25.3	86.4	49.1	62.2	63.6	25.8	88.3	90.0	15.3	85.5	87.9	31.8
GTE-ModernBERT	56.6	52.5	47.5	25.9	94.1	55.5	65.5	64.8	26.1	84.5	90.8	11.6	88.6	62.4	23.1
bge-base-en-v1.5	56.2	45.6	32.9	26.7	86.8	44.5	72.7	66.8	27.4	85.6	91.1	13.8	87.6	76.6	28.1
Nomic v1.5	55.9	35.8	43.5	28.8	86.8	44.7	72.7	67.4	24.4	85.1	87.2	12.7	83.3	80.7	29.4
modernbert-embed-base	55.6	36.5	37.8	24.7	87.8	46.0	62.7	65.3	24.3	89.3	89.9	12.9	85.5	82.7	33.1
ColBERT-Zero	60.0	54.5	36.8	33.0	90.5	46.6	77.8	74.2	26.6	91.1	88.3	14.2	89.5	75.3	40.9
pplx-embed-v1-late-0.6b	59.8	60.9	36.4	29.9	89.7	50.9	78.6	69.2	27.9	92.8	83.8	13.5	89.3	80.2	34.7
GTE-ModernColBERT	59.3	48.8	33.5	33.2	88.1	50.2	77.3	71.6	27.3	93.1	89.1	13.6	87.7	81.4	35.3
colbert-small	58.1	47.7	35.7	31.7	89.3	45.6	77.1	71.4	25.0	86.2	90.1	13.1	89.2	81.5	29.0

Training data and reproducibility

The corresponding private training artifacts are in GLINT-data. It contains the complete prepared SFT data, the 1,046,009-row seven-source KD mixture, and Jina teacher-score parquet shards. The repository contains no BEIR evaluation corpus or evaluation labels.

Citation

@misc{aarush2026glint,
  title={GLInt: Geometry-Matched Hard Negatives for Late-Interaction Retrieval},
  author={Aarush},
  year={2026},
  howpublished={\url{https://huggingface.co/blog/chungimungi/glint}},
}