Sentence Similarity
Safetensors
sentence-transformers
PyLate
lfm2
liquid
lfm2.5
edge
ColBERT
feature-extraction
custom_code
Instructions to use LiquidAI/LFM2.5-ColBERT-350M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use LiquidAI/LFM2.5-ColBERT-350M with sentence-transformers:
from pylate import models queries = [ "Which planet is known as the Red Planet?", "What is the largest planet in our solar system?", ] documents = [ ["Mars is the Red Planet.", "Venus is Earth's twin."], ["Jupiter is the largest planet.", "Saturn has rings."], ] model = models.ColBERT(model_name_or_path="LiquidAI/LFM2.5-ColBERT-350M") queries_emb = model.encode(queries, is_query=True) docs_emb = model.encode(documents, is_query=False) - Notebooks
- Google Colab
- Kaggle
Fix short-conv padding masking on transformers >=5
#1
by Satyen - opened
README.md
CHANGED
|
@@ -26,7 +26,6 @@ library_name: PyLate
|
|
| 26 |
license: other
|
| 27 |
license_name: lfm1.0
|
| 28 |
license_link: LICENSE
|
| 29 |
-
base_model: LiquidAI/LFM2.5-350M-Base
|
| 30 |
---
|
| 31 |
|
| 32 |
<center>
|
|
@@ -55,10 +54,7 @@ Both models are 350M params and the first bidirectional members of the LFM famil
|
|
| 55 |
|
| 56 |
Find more details about the bidirectional architecture and training recipe in our [blog post](https://www.liquid.ai/blog/lfm2-5-retrievers).
|
| 57 |
|
| 58 |
-
|
| 59 |
-
> ๐ป **Demo**: https://huggingface.co/spaces/LiquidAI/colbert-tool-selection
|
| 60 |
-
|
| 61 |
-

|
| 62 |
|
| 63 |
## ๐ Model details
|
| 64 |
|
|
@@ -68,11 +64,13 @@ Find more details about the bidirectional architecture and training recipe in ou
|
|
| 68 |
| **Total parameters** | ~353M | ~354M |
|
| 69 |
| **Backbone** | [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base) + bi-directional patches | [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base) + bi-directional patches |
|
| 70 |
| **Layers** | 17 (10 conv + 6 attn + 1 dense) | 17 (10 conv + 6 attn + 1 pool) |
|
|
|
|
| 71 |
| **Vocabulary size** | 64,402 | 65,536 |
|
| 72 |
| **Output** | 128-dim per token | 1024-dim CLS vector |
|
| 73 |
| **Similarity** | MaxSim | Cosine |
|
| 74 |
| **Training precision**| BF16 | BF16 |
|
| 75 |
| **License** | LFM Open License v1.0 | LFM Open License v1.0 |
|
|
|
|
| 76 |
|
| 77 |
**Document length:** 512 tokens
|
| 78 |
|
|
|
|
| 26 |
license: other
|
| 27 |
license_name: lfm1.0
|
| 28 |
license_link: LICENSE
|
|
|
|
| 29 |
---
|
| 30 |
|
| 31 |
<center>
|
|
|
|
| 54 |
|
| 55 |
Find more details about the bidirectional architecture and training recipe in our [blog post](https://www.liquid.ai/blog/lfm2-5-retrievers).
|
| 56 |
|
| 57 |
+

|
|
|
|
|
|
|
|
|
|
| 58 |
|
| 59 |
## ๐ Model details
|
| 60 |
|
|
|
|
| 64 |
| **Total parameters** | ~353M | ~354M |
|
| 65 |
| **Backbone** | [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base) + bi-directional patches | [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base) + bi-directional patches |
|
| 66 |
| **Layers** | 17 (10 conv + 6 attn + 1 dense) | 17 (10 conv + 6 attn + 1 pool) |
|
| 67 |
+
| **Context length** | 32,768 tokens | 32,768 tokens |
|
| 68 |
| **Vocabulary size** | 64,402 | 65,536 |
|
| 69 |
| **Output** | 128-dim per token | 1024-dim CLS vector |
|
| 70 |
| **Similarity** | MaxSim | Cosine |
|
| 71 |
| **Training precision**| BF16 | BF16 |
|
| 72 |
| **License** | LFM Open License v1.0 | LFM Open License v1.0 |
|
| 73 |
+
| **Architecture** | <img src="https://cdn-uploads.huggingface.co/production/uploads/63f389fda096536aeaae0a66/neXdblhfapp1Oln8M9J4V.png" width="250">| <img src="https://cdn-uploads.huggingface.co/production/uploads/63f389fda096536aeaae0a66/y4Lx5eS3hZCRv1g3r1e9i.png" width="250"> |
|
| 74 |
|
| 75 |
**Document length:** 512 tokens
|
| 76 |
|