--- library_name: sentence-transformers tags: - colbert - late-interaction - retrieval - pylate - multi-vector language: - en base_model: - lightonai/LateOn-unsupervised license: apache-2.0 pipeline_tag: sentence-similarity --- # GLInt GLINT is a SOTA 149M-parameter English late-interaction retriever built from [LateOn-unsupervised](https://huggingface.co/lightonai/LateOn-unsupervised). It retains 128-dimensional token embeddings and uses MaxSim retrieval with 32 query tokens and 300 document tokens. ## What is new in GLInt? GLINT is designed around the mismatch between ordinary dense hard-negative mining and a late-interaction retriever. Dense mining selects documents that are difficult under one pooled vector; GLINT instead mines negatives under the same token-level MaxSim geometry used at retrieval time. This exposes lexical, compositional, and localized token matches that a single-vector miner can miss. The training recipe has two stages: 1. supervised fine-tuning with multi-vector (MaxSim) hard negatives; 2. mixed listwise knowledge distillation over a diverse seven-source hard-negative mixture. For the second stage, a frozen listwise teacher (`jinaai/jina-reranker-v3.5`) scores each 32-document candidate set jointly. GLINT distils that ordering with a sharpened listwise KL objective, while a false-negative-masked InfoNCE term preserves a direct retrieval signal. ## Usage ### Sentence Transformers This model can be used with [Sentence Transformers](https://www.sbert.net/) as a multi-vector (ColBERT-style late interaction) retriever via the `MultiVectorEncoder`: ```bash pip install "sentence-transformers>=6.0.0" ``` ```python from sentence_transformers import MultiVectorEncoder model = MultiVectorEncoder("chungimungi/GLInt") query = "Which planet is known as the Red Planet?" documents = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.", ] query_embeddings = model.encode_query(query) document_embeddings = model.encode_document(documents) print(query_embeddings.shape, document_embeddings[0].shape) # torch.Size([12, 128]) torch.Size([18, 128]) # MaxSim late-interaction scoring (higher is more relevant) scores = model.similarity(query_embeddings, document_embeddings) print(scores) # tensor([[11.6192, 11.7344, 11.6513, 11.7105]], device='cuda:0') ``` ### PyLate ```python from pylate import models model = models.ColBERT("chungimungi/GLInt") query_embeddings = model.encode(["what causes a lunar eclipse?"], is_query=True) document_embeddings = model.encode( ["A lunar eclipse happens when Earth passes between the Sun and the Moon."], is_query=False, ) ``` Use a late-interaction backend such as PyLate/PLAID for corpus-scale retrieval. Scores are computed by summing, over query tokens, the maximum similarity to a document token. ## Results ### BEIR (15 datasets, NDCG@10) | Model | Average | Size (M) | Embed dim | ArguAna | CQADupstackRetrieval | ClimateFEVER | DBPedia | FEVER | FiQA2018 | HotpotQA | MSMARCO | NFCorpus | NQ | QuoraRetrieval | SCIDOCS | SciFact | TRECCOVID | Touche2020 | |---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:| | [ColBERTv2](https://huggingface.co/colbert-ir/colbertv2.0) | 48.63 | 110 | 128 | 46.50 | 38.30 | 17.60 | 45.20 | 78.50 | 35.40 | 67.50 | 46.00 | 33.70 | 52.40 | 85.50 | 15.40 | 68.90 | 72.60 | 26.00 | | [Jina-ColBERT-v2](https://huggingface.co/jinaai/jina-colbert-v2) | 51.85 | 600 | 128 | 36.60 | 40.80 | 23.90 | 47.10 | 80.50 | 40.80 | 76.60 | **46.90** | 34.60 | 64.00 | 88.70 | 18.60 | 67.80 | 83.40 | 27.40 | | [ColBERT-small](https://huggingface.co/answerdotai/answerai-colbert-small-v1) | 53.79 | 33 | 96 | 50.09 | 38.75 | 33.07 | 45.58 | 90.96 | 41.15 | 76.11 | 43.50 | 37.30 | 59.10 | 87.72 | 18.42 | 74.77 | 84.59 | 25.69 | | [GTE-ModernColBERT-v1](https://huggingface.co/lightonai/GTE-ModernColBERT-v1) | 54.75 | 149 | 128 | 47.52 | 41.08 | 31.33 | 47.56 | 87.67 | 45.25 | 77.48 | 45.60 | **37.83** | 61.62 | 86.71 | 19.22 | 76.33 | **84.84** | 31.25 | | [ColBERT-Zero](https://huggingface.co/lightonai/ColBERT-Zero) | 55.39 | 149 | 128 | **52.82** | 41.41 | 35.90 | 47.43 | 90.52 | 42.50 | 79.45 | 45.95 | 37.21 | 61.82 | 85.19 | 19.84 | 76.33 | 78.27 | **36.24** | | [LateOn-unsupervised](https://huggingface.co/lightonai/LateOn-unsupervised) | 50.11 | 149 | 128 | 43.12 | **47.71** | 18.76 | 43.36 | 65.74 | 51.94 | 68.17 | 37.51 | 37.15 | 58.41 | 89.48 | 21.13 | 76.89 | 69.81 | 22.53 | | [LateOn](https://huggingface.co/lightonai/LateOn) | 57.22 | 149 | 128 | 50.52 | 47.36 | **39.67** | 45.99 | 92.02 | **53.12** | 79.98 | 45.67 | 37.79 | 63.91 | 89.67 | **21.90** | 76.61 | 83.60 | 30.52 | | GLInt | **57.43** | 149 | 128 | 52.38 | 46.49 | 34.17 | **47.68** | **92.45** | 50.85 | **82.54** | 46.38 | 37.51 | **68.03** | **90.08** | 20.65 | **77.13** | 84.78 | 30.26 | ### BEIR-Decontaminated (14 datasets, NDCG@10) | Model | Average | ArguAna | ClimateFEVER | DBPedia | FEVER | FiQA2018 | HotpotQA | MS MARCO | NFCorpus | Natural Questions | Quora | SciDocs | SciFact | TREC-COVID | Touché-2020 | |---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:| | **GLInt** | **62.50** | 51.67 | 36.35 | 42.50 | 92.89 | 56.88 | **81.16** | 72.70 | 26.21 | **94.97** | **92.06** | **22.02** | **89.07** | 81.51 | 34.97 | | LateOn | 61.4 | 52.2 | 42.1 | 31.7 | 92.7 | **57.9** | 78.9 | 70.3 | 27.0 | 93.1 | 91.5 | 15.1 | 88.9 | 80.9 | 36.8 | | DenseOn | 58.8 | 40.0 | 39.5 | 28.8 | 91.2 | 55.9 | 73.7 | 68.9 | 28.5 | 92.1 | 91.1 | 14.7 | 85.4 | 82.5 | 31.0 | | pplx-embed-v1-0.6b | 59.7 | 43.7 | 42.4 | 28.4 | 91.1 | 55.2 | 73.5 | 71.9 | 28.0 | 91.6 | 91.5 | 15.4 | 89.0 | 83.7 | 30.0 | | jina-v5-text-nano | 58.8 | 47.2 | 41.6 | 30.2 | 90.0 | 51.5 | 67.5 | 68.6 | 29.4 | 92.3 | 91.3 | 14.9 | 89.4 | 76.8 | 33.2 | | harrier-oss-v1-0.6b | 58.0 | 47.4 | 25.7 | 31.3 | 80.7 | 50.1 | 71.4 | **73.4** | 27.9 | 90.0 | 90.9 | 17.1 | **90.7** | 81.8 | 33.3 | | arctic-embed-l-v2 | 57.9 | 43.1 | **45.7** | **45.7** | 92.2 | 50.4 | 63.1 | 71.0 | 26.0 | 90.7 | 91.3 | 13.9 | 87.4 | 81.4 | 26.8 | | bge-large-en-v1.5 | 57.3 | 46.0 | 39.0 | 28.9 | 87.6 | 49.3 | 75.2 | 68.9 | **29.8** | 85.9 | 91.3 | 14.0 | 86.5 | 72.7 | 26.9 | | Qwen3-Embedding-0.6B | 57.0 | 48.4 | 38.0 | 25.3 | 86.4 | 49.1 | 62.2 | 63.6 | 25.8 | 88.3 | 90.0 | 15.3 | 85.5 | **87.9** | 31.8 | | GTE-ModernBERT | 56.6 | 52.5 | 47.5 | 25.9 | **94.1** | 55.5 | 65.5 | 64.8 | 26.1 | 84.5 | 90.8 | 11.6 | 88.6 | 62.4 | 23.1 | | bge-base-en-v1.5 | 56.2 | 45.6 | 32.9 | 26.7 | 86.8 | 44.5 | 72.7 | 66.8 | 27.4 | 85.6 | 91.1 | 13.8 | 87.6 | 76.6 | 28.1 | | Nomic v1.5 | 55.9 | 35.8 | 43.5 | 28.8 | 86.8 | 44.7 | 72.7 | 67.4 | 24.4 | 85.1 | 87.2 | 12.7 | 83.3 | 80.7 | 29.4 | | modernbert-embed-base | 55.6 | 36.5 | 37.8 | 24.7 | 87.8 | 46.0 | 62.7 | 65.3 | 24.3 | 89.3 | 89.9 | 12.9 | 85.5 | 82.7 | 33.1 | | ColBERT-Zero | 60.0 | 54.5 | 36.8 | 33.0 | 90.5 | 46.6 | 77.8 | **74.2** | 26.6 | 91.1 | 88.3 | 14.2 | 89.5 | 75.3 | **40.9** | | pplx-embed-v1-late-0.6b | 59.8 | **60.9** | 36.4 | 29.9 | 89.7 | 50.9 | 78.6 | 69.2 | 27.9 | 92.8 | 83.8 | 13.5 | 89.3 | 80.2 | 34.7 | | GTE-ModernColBERT | 59.3 | 48.8 | 33.5 | 33.2 | 88.1 | 50.2 | 77.3 | 71.6 | 27.3 | 93.1 | 89.1 | 13.6 | 87.7 | 81.4 | 35.3 | | colbert-small | 58.1 | 47.7 | 35.7 | 31.7 | 89.3 | 45.6 | 77.1 | 71.4 | 25.0 | 86.2 | 90.1 | 13.1 | 89.2 | 81.5 | 29.0 | ## Training data and reproducibility The corresponding private training artifacts are in [GLINT-data](https://huggingface.co/datasets/chungimungi/GLINT-data). It contains the complete prepared SFT data, the 1,046,009-row seven-source KD mixture, and Jina teacher-score parquet shards. The repository contains no BEIR evaluation corpus or evaluation labels. ## Citation ``` @misc{aarush2026glint, title={GLInt: Geometry-Matched Hard Negatives for Late-Interaction Retrieval}, author={Aarush}, year={2026}, howpublished={\url{https://huggingface.co/blog/chungimungi/glint}}, } ```