finetuned with additional names

README.md

@@ -3,30 +3,11 @@ tags:
 - sentence-transformers
 - cross-encoder
 - generated_from_trainer
-- dataset_size:32380
-- loss:BinaryCrossEntropyLoss
+- dataset_size:72905
+- loss:MultipleNegativesRankingLoss
 base_model: BAAI/bge-reranker-base
 pipeline_tag: text-ranking
 library_name: sentence-transformers
-metrics:
-- pearson
-- spearman
-model-index:
-- name: CrossEncoder based on BAAI/bge-reranker-base
-  results:
-  - task:
-      type: cross-encoder-correlation
-      name: Cross Encoder Correlation
-    dataset:
-      name: name similarity
-      type: name_similarity
-    metrics:
-    - type: pearson
-      value: 0.9803135847456451
-      name: Pearson
-    - type: spearman
-      value: 0.975407488053043
-      name: Spearman
 ---
 
 # CrossEncoder based on BAAI/bge-reranker-base
@@ -69,11 +50,11 @@ from sentence_transformers import CrossEncoder
 model = CrossEncoder("foochun/bge-reranker-ft")
 # Get scores for pairs of texts
 pairs = [
-    ['zach toh zhen bing', 'zach toh zhen bing'],
-    ['zach yap bing sheng', 'yap bing sheng zach'],
-    ['carmen chia zhen meng', 'carmen zhen chia meng'],
-    ['carmen lau zhen bing', 'carmen zhen bing lau'],
-    ['ajith s/o sockalingam', 'sockalingam ajith'],
+    ['zach koh yong liang', 'yong liang koh zach'],
+    ['zulkifli bin mohamad', 'zulkifli bin muhammad'],
+    ['rahman bin mohd rashid', 'rahman mohammed rashid'],
+    ['mohd syukri bin bakar', 'muhd syukri bakar'],
+    ['carmen tan fang kiat', 'tan fang kiat'],
 ]
 scores = model.predict(pairs)
 print(scores.shape)
@@ -81,13 +62,13 @@ print(scores.shape)
 
 # Or rank different texts based on similarity to a single text
 ranks = model.rank(
-    'zach toh zhen bing',
+    'zach koh yong liang',
     [
-        'zach toh zhen bing',
-        'yap bing sheng zach',
-        'carmen zhen chia meng',
-        'carmen zhen bing lau',
-        'sockalingam ajith',
+        'yong liang koh zach',
+        'zulkifli bin muhammad',
+        'rahman mohammed rashid',
+        'muhd syukri bakar',
+        'tan fang kiat',
     ]
 )
 # [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
@@ -117,20 +98,6 @@ You can finetune this model on your own dataset.
 *List how the model may foreseeably be misused and address what users ought not to do with the model.*
 -->
 
-## Evaluation
-
-### Metrics
-
-#### Cross Encoder Correlation
-
-* Dataset: `name_similarity`
-* Evaluated with [<code>CECorrelationEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CECorrelationEvaluator)
-
-| Metric       | Value      |
-|:-------------|:-----------|
-| pearson      | 0.9803     |
-| **spearman** | **0.9754** |
-
 <!--
 ## Bias, Risks and Limitations
 
@@ -149,24 +116,51 @@ You can finetune this model on your own dataset.
 
 #### Unnamed Dataset
 
-* Size: 32,380 training samples
-* Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
+* Size: 72,905 training samples
+* Columns: <code>query</code>, <code>pos</code>, and <code>neg</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                         | pos                                                                                           | neg                                                                                           |
+  |:--------|:----------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------|
+  | type    | string                                                                                        | string                                                                                        | string                                                                                        |
+  | details | <ul><li>min: 9 characters</li><li>mean: 19.91 characters</li><li>max: 45 characters</li></ul> | <ul><li>min: 9 characters</li><li>mean: 17.64 characters</li><li>max: 40 characters</li></ul> | <ul><li>min: 9 characters</li><li>mean: 17.95 characters</li><li>max: 37 characters</li></ul> |
+* Samples:
+  | query                                      | pos                                  | neg                                |
+  |:-------------------------------------------|:-------------------------------------|:-----------------------------------|
+  | <code>sim hong soon</code>                 | <code>sim hong soon</code>           | <code>sim soon hong</code>         |
+  | <code>raja mariam binti raja sharif</code> | <code>raja mariam raja sharif</code> | <code>zuraidah binti dollah</code> |
+  | <code>saw ann fui</code>                   | <code>fui saw ann</code>             | <code>ann saw fui</code>           |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 10.0,
+      "num_negatives": 4,
+      "activation_fn": "torch.nn.modules.activation.Sigmoid"
+  }
+  ```
+
+### Evaluation Dataset
+
+#### Unnamed Dataset
+
+* Size: 10,415 evaluation samples
+* Columns: <code>query</code>, <code>pos</code>, and <code>neg</code>
 * Approximate statistics based on the first 1000 samples:
-  |         | sentence_0                                                                                    | sentence_1                                                                                    | label                                                           |
-  |:--------|:----------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------|:----------------------------------------------------------------|
-  | type    | string                                                                                        | string                                                                                        | float                                                           |
-  | details | <ul><li>min: 10 characters</li><li>mean: 19.2 characters</li><li>max: 43 characters</li></ul> | <ul><li>min: 9 characters</li><li>mean: 17.93 characters</li><li>max: 40 characters</li></ul> | <ul><li>min: -0.3</li><li>mean: 0.53</li><li>max: 1.0</li></ul> |
+  |         | query                                                                                         | pos                                                                                          | neg                                                                                           |
+  |:--------|:----------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------|
+  | type    | string                                                                                        | string                                                                                       | string                                                                                        |
+  | details | <ul><li>min: 9 characters</li><li>mean: 19.95 characters</li><li>max: 43 characters</li></ul> | <ul><li>min: 9 characters</li><li>mean: 17.8 characters</li><li>max: 42 characters</li></ul> | <ul><li>min: 8 characters</li><li>mean: 18.33 characters</li><li>max: 36 characters</li></ul> |
 * Samples:
-  | sentence_0                         | sentence_1                         | label                           |
-  |:-----------------------------------|:-----------------------------------|:--------------------------------|
-  | <code>zach toh zhen bing</code>    | <code>zach toh zhen bing</code>    | <code>0.9999998211860657</code> |
-  | <code>zach yap bing sheng</code>   | <code>yap bing sheng zach</code>   | <code>0.9400546550750732</code> |
-  | <code>carmen chia zhen meng</code> | <code>carmen zhen chia meng</code> | <code>0.17237488925457</code>   |
-* Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
+  | query                               | pos                                 | neg                              |
+  |:------------------------------------|:------------------------------------|:---------------------------------|
+  | <code>zach koh yong liang</code>    | <code>yong liang koh zach</code>    | <code>liang yong koh zach</code> |
+  | <code>zulkifli bin mohamad</code>   | <code>zulkifli bin muhammad</code>  | <code>razak bin ibrahim</code>   |
+  | <code>rahman bin mohd rashid</code> | <code>rahman mohammed rashid</code> | <code>fauzi bin mohd</code>      |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#multiplenegativesrankingloss) with these parameters:
   ```json
   {
-      "activation_fn": "torch.nn.modules.linear.Identity",
-      "pos_weight": null
+      "scale": 10.0,
+      "num_negatives": 4,
+      "activation_fn": "torch.nn.modules.activation.Sigmoid"
   }
   ```
 
@@ -174,9 +168,15 @@ You can finetune this model on your own dataset.
 #### Non-Default Hyperparameters
 
 - `eval_strategy`: steps
-- `per_device_train_batch_size`: 16
-- `per_device_eval_batch_size`: 16
-- `num_train_epochs`: 4
+- `per_device_train_batch_size`: 64
+- `per_device_eval_batch_size`: 64
+- `learning_rate`: 1e-05
+- `warmup_ratio`: 0.1
+- `seed`: 12
+- `fp16`: True
+- `dataloader_num_workers`: 4
+- `load_best_model_at_end`: True
+- `batch_sampler`: no_duplicates
 
 #### All Hyperparameters
 <details><summary>Click to expand</summary>
@@ -185,24 +185,24 @@ You can finetune this model on your own dataset.
 - `do_predict`: False
 - `eval_strategy`: steps
 - `prediction_loss_only`: True
-- `per_device_train_batch_size`: 16
-- `per_device_eval_batch_size`: 16
+- `per_device_train_batch_size`: 64
+- `per_device_eval_batch_size`: 64
 - `per_gpu_train_batch_size`: None
 - `per_gpu_eval_batch_size`: None
 - `gradient_accumulation_steps`: 1
 - `eval_accumulation_steps`: None
 - `torch_empty_cache_steps`: None
-- `learning_rate`: 5e-05
+- `learning_rate`: 1e-05
 - `weight_decay`: 0.0
 - `adam_beta1`: 0.9
 - `adam_beta2`: 0.999
 - `adam_epsilon`: 1e-08
-- `max_grad_norm`: 1
-- `num_train_epochs`: 4
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 3
 - `max_steps`: -1
 - `lr_scheduler_type`: linear
 - `lr_scheduler_kwargs`: {}
-- `warmup_ratio`: 0.0
+- `warmup_ratio`: 0.1
 - `warmup_steps`: 0
 - `log_level`: passive
 - `log_level_replica`: warning
@@ -215,12 +215,12 @@ You can finetune this model on your own dataset.
 - `no_cuda`: False
 - `use_cpu`: False
 - `use_mps_device`: False
-- `seed`: 42
+- `seed`: 12
 - `data_seed`: None
 - `jit_mode_eval`: False
 - `use_ipex`: False
 - `bf16`: False
-- `fp16`: False
+- `fp16`: True
 - `fp16_opt_level`: O1
 - `half_precision_backend`: auto
 - `bf16_full_eval`: False
@@ -232,13 +232,13 @@ You can finetune this model on your own dataset.
 - `tpu_metrics_debug`: False
 - `debug`: []
 - `dataloader_drop_last`: False
-- `dataloader_num_workers`: 0
+- `dataloader_num_workers`: 4
 - `dataloader_prefetch_factor`: None
 - `past_index`: -1
 - `disable_tqdm`: False
 - `remove_unused_columns`: True
 - `label_names`: None
-- `load_best_model_at_end`: False
+- `load_best_model_at_end`: True
 - `ignore_data_skip`: False
 - `fsdp`: []
 - `fsdp_min_num_params`: 0
@@ -293,33 +293,18 @@ You can finetune this model on your own dataset.
 - `eval_use_gather_object`: False
 - `average_tokens_across_devices`: False
 - `prompts`: None
-- `batch_sampler`: batch_sampler
+- `batch_sampler`: no_duplicates
 - `multi_dataset_batch_sampler`: proportional
 
 </details>
 
 ### Training Logs
-| Epoch  | Step | Training Loss | name_similarity_spearman |
-|:------:|:----:|:-------------:|:------------------------:|
-| 0.2470 | 500  | 0.4855        | 0.9288                   |
-| 0.4941 | 1000 | 0.361         | 0.9507                   |
-| 0.7411 | 1500 | 0.3367        | 0.9563                   |
-| 0.9881 | 2000 | 0.3398        | 0.9633                   |
-| 1.0    | 2024 | -             | 0.9636                   |
-| 1.2352 | 2500 | 0.3286        | 0.9650                   |
-| 1.4822 | 3000 | 0.3267        | 0.9685                   |
-| 1.7292 | 3500 | 0.315         | 0.9702                   |
-| 1.9763 | 4000 | 0.3236        | 0.9719                   |
-| 2.0    | 4048 | -             | 0.9719                   |
-| 2.2233 | 4500 | 0.3081        | 0.9727                   |
-| 2.4704 | 5000 | 0.3172        | 0.9732                   |
-| 2.7174 | 5500 | 0.3121        | 0.9738                   |
-| 2.9644 | 6000 | 0.3037        | 0.9745                   |
-| 3.0    | 6072 | -             | 0.9745                   |
-| 3.2115 | 6500 | 0.3105        | 0.9745                   |
-| 3.4585 | 7000 | 0.2965        | 0.9750                   |
-| 3.7055 | 7500 | 0.3031        | 0.9751                   |
-| 3.9526 | 8000 | 0.2998        | 0.9754                   |
+| Epoch  | Step | Training Loss |
+|:------:|:----:|:-------------:|
+| 0.0009 | 1    | 0.5117        |
+| 0.8772 | 1000 | 0.0955        |
+| 1.7544 | 2000 | 0.005         |
+| 2.6316 | 3000 | 0.0039        |
 
 
 ### Framework Versions
@@ -328,7 +313,7 @@ You can finetune this model on your own dataset.
 - Transformers: 4.51.3
 - PyTorch: 2.6.0+cu124
 - Accelerate: 1.6.0
-- Datasets: 3.5.1
+- Datasets: 3.6.0
 - Tokenizers: 0.21.1
 
 ## Citation

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:df3df7ab41b95c380ef7801f7bd9085b327e2cc9279234c80b8445cff0540214
+oid sha256:edc64662e2fe56e8a890faf4992682b1605b018ba49b2acb609a13667cead4ce
 size 1112201932