Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,40 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: other
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
pipeline_tag: sentence-similarity
|
| 6 |
+
datasets:
|
| 7 |
+
- sentence-transformers/msmarco-msmarco-distilbert-base-v3
|
| 8 |
+
- microsoft/ms_marco
|
| 9 |
---
|
| 10 |
+
|
| 11 |
+
# constructai/taylor-v1-128-emb
|
| 12 |
+
|
| 13 |
+
This model is a fine-tuned version of a custom BERT-like encoder (hidden_size=384, 6 layers) trained on MS MARCO passage ranking dataset.
|
| 14 |
+
It uses **triplet hard negatives** from [`sentence-transformers/msmarco-msmarco-distilbert-base-v3`](https://huggingface.co/datasets/sentence-transformers/msmarco-msmarco-distilbert-base-v3) (Apache-2.0 license). The base MS MARCO data is subject to [Microsoft Research License](https://github.com/microsoft/MSMARCO-Passage-Ranking/blob/master/LICENSE).
|
| 15 |
+
|
| 16 |
+
The model produces **384-dimensional** embeddings (CLS token) optimized for cosine similarity.
|
| 17 |
+
|
| 18 |
+
# Training details
|
| 19 |
+
|
| 20 |
+
* Max sequence length: 128 tokens
|
| 21 |
+
|
| 22 |
+
* Loss: MultipleNegativesSymmetricRankingLoss (InfoNCE)
|
| 23 |
+
|
| 24 |
+
* Batch size: 128 (32 + gradient accumulation)
|
| 25 |
+
|
| 26 |
+
* Learning rate: 2e-5
|
| 27 |
+
|
| 28 |
+
* Data: ~500k triplets from MS MARCO
|
| 29 |
+
|
| 30 |
+
# Evaluation
|
| 31 |
+
|
| 32 |
+
On a small test set, the model achieves:
|
| 33 |
+
|
| 34 |
+
* Positive pair similarity: 0.58
|
| 35 |
+
|
| 36 |
+
* Negative pair similarity: 0.14
|
| 37 |
+
|
| 38 |
+
* Margin: 0.44
|
| 39 |
+
|
| 40 |
+
#
|