File size: 2,995 Bytes
e363b49 3f58295 e363b49 d84587b e363b49 9b148d4 e363b49 9b148d4 e9f30c8 9b148d4 e363b49 3f58295 e363b49 d84587b e363b49 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | ---
language:
- de
tags:
- ColBERT
- multi-vector
- PyLate
- sentence-transformers
- sentence-similarity
pipeline_tag: sentence-similarity
library_name: PyLate
datasets:
- samheym/ger-dpr-collection
base_model:
- deepset/gbert-base
---
# Model Overview
GerColBERT is a ColBERT-based retrieval model trained on German text. It is designed for efficient late interaction-based retrieval while maintaining high-quality ranking performance.
Training Configuration
- Base Model: [deepset/gbert-base](https://huggingface.co/deepset/gbert-base)
- Training Dataset: samheym/ger-dpr-collection
- Dataset: 10% of randomly selected triples from the final dataset
- Vector Length: 128
- Maximum Document Length: 256 Tokens
- Batch Size: 50
- Training Steps: 80,000
- Gradient Accumulation: 1 step
- Learning Rate: 5 × 10⁻⁶
- Optimizer: AdamW
- In-Batch Negatives: Included
## Usage
### Sentence Transformers
This model can be used with [Sentence Transformers](https://www.sbert.net/) as a multi-vector (ColBERT-style late interaction) retriever via the `MultiVectorEncoder`:
```bash
pip install "sentence-transformers>=6.0.0"
```
```python
from sentence_transformers import MultiVectorEncoder
model = MultiVectorEncoder("samheym/GerColBERT")
query = "Welcher Planet ist als der Rote Planet bekannt?"
documents = [
"Venus wird wegen ihrer ähnlichen Größe und Nähe oft als Zwilling der Erde bezeichnet.",
"Mars, bekannt für sein rötliches Aussehen, wird oft als der Rote Planet bezeichnet.",
"Jupiter, der größte Planet des Sonnensystems, hat einen markanten roten Fleck.",
"Saturn, berühmt für seine Ringe, wird manchmal mit dem Roten Planeten verwechselt.",
]
query_embeddings = model.encode_query(query)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings[0].shape)
# (32, 128) (18, 128)
# MaxSim late-interaction scoring (higher is more relevant)
scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[13.8994, 27.9157, 21.6549, 22.7871]])
```
### PyLate
First install the PyLate library:
```bash
pip install -U pylate
```
### Retrieval
PyLate provides a streamlined interface to index and retrieve documents using ColBERT models. The index leverages the Voyager HNSW index to efficiently handle document embeddings and enable fast retrieval.
```python
from pylate import indexes, models, retrieve
# Step 1: Load the ColBERT model
model = models.ColBERT(
model_name_or_path=samheym/GerColBERT,
)
```
<!--
## Citation
### BibTeX
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--> |