Sentence Similarity
PyTorch
English
taylor-m1
constructai commited on
Commit
7136be4
·
verified ·
1 Parent(s): e8a8222

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -1
README.md CHANGED
@@ -1,3 +1,40 @@
1
  ---
2
- license: apache-2.0
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: other
3
+ language:
4
+ - en
5
+ pipeline_tag: sentence-similarity
6
+ datasets:
7
+ - sentence-transformers/msmarco-msmarco-distilbert-base-v3
8
+ - microsoft/ms_marco
9
  ---
10
+
11
+ # constructai/taylor-v1-128-emb
12
+
13
+ This model is a fine-tuned version of a custom BERT-like encoder (hidden_size=384, 6 layers) trained on MS MARCO passage ranking dataset.
14
+ It uses **triplet hard negatives** from [`sentence-transformers/msmarco-msmarco-distilbert-base-v3`](https://huggingface.co/datasets/sentence-transformers/msmarco-msmarco-distilbert-base-v3) (Apache-2.0 license). The base MS MARCO data is subject to [Microsoft Research License](https://github.com/microsoft/MSMARCO-Passage-Ranking/blob/master/LICENSE).
15
+
16
+ The model produces **384-dimensional** embeddings (CLS token) optimized for cosine similarity.
17
+
18
+ # Training details
19
+
20
+ * Max sequence length: 128 tokens
21
+
22
+ * Loss: MultipleNegativesSymmetricRankingLoss (InfoNCE)
23
+
24
+ * Batch size: 128 (32 + gradient accumulation)
25
+
26
+ * Learning rate: 2e-5
27
+
28
+ * Data: ~500k triplets from MS MARCO
29
+
30
+ # Evaluation
31
+
32
+ On a small test set, the model achieves:
33
+
34
+ * Positive pair similarity: 0.58
35
+
36
+ * Negative pair similarity: 0.14
37
+
38
+ * Margin: 0.44
39
+
40
+ #