Fix short-conv padding masking on transformers >=5

#1
by Satyen - opened
Files changed (1) hide show
  1. README.md +3 -5
README.md CHANGED
@@ -26,7 +26,6 @@ library_name: PyLate
26
  license: other
27
  license_name: lfm1.0
28
  license_link: LICENSE
29
- base_model: LiquidAI/LFM2.5-350M-Base
30
  ---
31
 
32
  <center>
@@ -55,10 +54,7 @@ Both models are 350M params and the first bidirectional members of the LFM famil
55
 
56
  Find more details about the bidirectional architecture and training recipe in our [blog post](https://www.liquid.ai/blog/lfm2-5-retrievers).
57
 
58
- > [!NOTE]
59
- > ๐Ÿ’ป **Demo**: https://huggingface.co/spaces/LiquidAI/colbert-tool-selection
60
-
61
- ![colb](https://cdn-uploads.huggingface.co/production/uploads/63f389fda096536aeaae0a66/snq-O6VmWRJNMYCNHGtDD.png)
62
 
63
  ## ๐Ÿ“„ Model details
64
 
@@ -68,11 +64,13 @@ Find more details about the bidirectional architecture and training recipe in ou
68
  | **Total parameters** | ~353M | ~354M |
69
  | **Backbone** | [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base) + bi-directional patches | [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base) + bi-directional patches |
70
  | **Layers** | 17 (10 conv + 6 attn + 1 dense) | 17 (10 conv + 6 attn + 1 pool) |
 
71
  | **Vocabulary size** | 64,402 | 65,536 |
72
  | **Output** | 128-dim per token | 1024-dim CLS vector |
73
  | **Similarity** | MaxSim | Cosine |
74
  | **Training precision**| BF16 | BF16 |
75
  | **License** | LFM Open License v1.0 | LFM Open License v1.0 |
 
76
 
77
  **Document length:** 512 tokens &nbsp;&nbsp;
78
 
 
26
  license: other
27
  license_name: lfm1.0
28
  license_link: LICENSE
 
29
  ---
30
 
31
  <center>
 
54
 
55
  Find more details about the bidirectional architecture and training recipe in our [blog post](https://www.liquid.ai/blog/lfm2-5-retrievers).
56
 
57
+ ![colbert_retrieval](https://cdn-uploads.huggingface.co/production/uploads/63f389fda096536aeaae0a66/8J_3tsS-wDq77qeDpDgOd.png)
 
 
 
58
 
59
  ## ๐Ÿ“„ Model details
60
 
 
64
  | **Total parameters** | ~353M | ~354M |
65
  | **Backbone** | [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base) + bi-directional patches | [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base) + bi-directional patches |
66
  | **Layers** | 17 (10 conv + 6 attn + 1 dense) | 17 (10 conv + 6 attn + 1 pool) |
67
+ | **Context length** | 32,768 tokens | 32,768 tokens |
68
  | **Vocabulary size** | 64,402 | 65,536 |
69
  | **Output** | 128-dim per token | 1024-dim CLS vector |
70
  | **Similarity** | MaxSim | Cosine |
71
  | **Training precision**| BF16 | BF16 |
72
  | **License** | LFM Open License v1.0 | LFM Open License v1.0 |
73
+ | **Architecture** | <img src="https://cdn-uploads.huggingface.co/production/uploads/63f389fda096536aeaae0a66/neXdblhfapp1Oln8M9J4V.png" width="250">| <img src="https://cdn-uploads.huggingface.co/production/uploads/63f389fda096536aeaae0a66/y4Lx5eS3hZCRv1g3r1e9i.png" width="250"> |
74
 
75
  **Document length:** 512 tokens &nbsp;&nbsp;
76