Add new CrossEncoder model

7decb43 verified 7 days ago

53.4 kB

metadata

language:
  - en
license: apache-2.0
tags:
  - cross-encoder
  - sentence-transformers
  - text-classification
  - sentence-pair-classification
  - semantic-similarity
  - semantic-search
  - retrieval
  - reranking
  - generated_from_trainer
  - dataset_size:1452533
  - loss:MultipleNegativesRankingLoss
base_model: cross-encoder/ms-marco-MiniLM-L6-v2
datasets:
  - redis/langcache-sentencepairs-v1
pipeline_tag: text-ranking
library_name: sentence-transformers
metrics:
  - accuracy
  - accuracy_threshold
  - f1
  - f1_threshold
  - precision
  - recall
  - average_precision
model-index:
  - name: Redis fine-tuned CrossEncoder model for semantic caching on LangCache
    results:
      - task:
          type: cross-encoder-classification
          name: Cross Encoder Classification
        dataset:
          name: test cls
          type: test_cls
        metrics:
          - type: accuracy
            value: 0.8275422840650747
            name: Accuracy
          - type: accuracy_threshold
            value: 0.00318145751953125
            name: Accuracy Threshold
          - type: f1
            value: 0.8104219459514619
            name: F1
          - type: f1_threshold
            value: -0.298828125
            name: F1 Threshold
          - type: precision
            value: 0.7457510407211493
            name: Precision
          - type: recall
            value: 0.8873743016759776
            name: Recall
          - type: average_precision
            value: 0.8721928487901052
            name: Average Precision

Redis fine-tuned CrossEncoder model for semantic caching on LangCache

This is a Cross Encoder model finetuned from cross-encoder/ms-marco-MiniLM-L6-v2 on the LangCache Sentence Pairs (subsets=['all'], train+val=True) dataset using the sentence-transformers library. It computes scores for pairs of texts, which can be used for sentence pair classification.

Model Details

Model Description

Model Type: Cross Encoder
Base model: cross-encoder/ms-marco-MiniLM-L6-v2
Maximum Sequence Length: 512 tokens
Number of Output Labels: 1 label
Training Dataset:
- LangCache Sentence Pairs (subsets=['all'], train+val=True)
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Documentation: Cross Encoder Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("redis/langcache-reranker-v1-miniL6-softmnrl-triplet")
# Get scores for pairs of texts
pairs = [
    [' What high potential jobs are there other than computer science?', ' What high potential jobs are there other than computer science?'],
    [' Would India ever be able to develop a missile system like S300 or S400 missile?', ' Would India ever be able to develop a missile system like S300 or S400 missile?'],
    [' water from the faucet is being drunk by a yellow dog', 'A yellow dog is drinking water from the faucet'],
    [' water from the faucet is being drunk by a yellow dog', 'The yellow dog is drinking water from a bottle'],
    ['! colspan = `` 14 `` `` Players who appeared for Colchester who left during the season ``', '! colspan = `` 14 `` `` Players who appeared for Colchester who left during the season ``'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    ' What high potential jobs are there other than computer science?',
    [
        ' What high potential jobs are there other than computer science?',
        ' Would India ever be able to develop a missile system like S300 or S400 missile?',
        'A yellow dog is drinking water from the faucet',
        'The yellow dog is drinking water from a bottle',
        '! colspan = `` 14 `` `` Players who appeared for Colchester who left during the season ``',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Classification

Dataset: test_cls
Evaluated with CrossEncoderClassificationEvaluator

Metric	Value
accuracy	0.8275
accuracy_threshold	0.0032
f1	0.8104
f1_threshold	-0.2988
precision	0.7458
recall	0.8874
average_precision	0.8722

Training Details

Training Dataset

LangCache Sentence Pairs (subsets=['all'], train+val=True)

Dataset: LangCache Sentence Pairs (subsets=['all'], train+val=True)
Size: 1,452,533 training samples
Columns: anchor, positive, and negative_1

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative_1
type	string	string	string
details	min: 24 characters mean: 114.25 characters max: 268 characters	min: 19 characters mean: 114.1 characters max: 226 characters	min: 4 characters mean: 93.04 characters max: 234 characters

Samples:

anchor	positive	negative_1
`Any Canadian teachers (B.Ed. holders) teaching in U.S. schools?`	`Any Canadian teachers (B.Ed. holders) teaching in U.S. schools?`	`Are there many Canadians living and working illegally in the United States?`
`Are there any underlying psychological tricks/tactics that are used when designing the lines for rides at amusement parks?`	`Are there any underlying psychological tricks/tactics that are used when designing the lines for rides at amusement parks?`	`Is there any tricks for straight lines mcqs?`
`Can I pay with a debit card on PayPal?`	`Can I pay with a debit card on PayPal?`	`Can you transfer PayPal funds onto a debit card/credit card?`

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "num_negatives": 1,
    "activation_fn": "torch.nn.modules.activation.Sigmoid"
}

Evaluation Dataset

LangCache Sentence Pairs (split=test)

Dataset: LangCache Sentence Pairs (split=test)
Size: 110,066 evaluation samples
Columns: anchor, positive, and negative_1

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative_1
type	string	string	string
details	min: 3 characters mean: 97.95 characters max: 314 characters	min: 3 characters mean: 97.03 characters max: 314 characters	min: 11 characters mean: 74.49 characters max: 295 characters

Samples:

anchor	positive	negative_1
`What high potential jobs are there other than computer science?`	`What high potential jobs are there other than computer science?`	`Why IT or Computer Science jobs are being over rated than other Engineering jobs?`
`Would India ever be able to develop a missile system like S300 or S400 missile?`	`Would India ever be able to develop a missile system like S300 or S400 missile?`	`Should India buy the Russian S400 air defence missile system?`
`water from the faucet is being drunk by a yellow dog`	`A yellow dog is drinking water from the faucet`	`Do you get more homework in 9th grade than 8th?`

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "num_negatives": 1,
    "activation_fn": "torch.nn.modules.activation.Sigmoid"
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 48
per_device_eval_batch_size: 48
learning_rate: 0.0002
weight_decay: 0.001
num_train_epochs: 50
warmup_ratio: 0.1
load_best_model_at_end: True
optim: adamw_torch
ddp_find_unused_parameters: False
push_to_hub: True
hub_model_id: redis/langcache-reranker-v1-miniL6-softmnrl-triplet
eval_on_start: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 48
per_device_eval_batch_size: 48
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 0.0002
weight_decay: 0.001
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 50
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: True
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: False
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: True
resume_from_checkpoint: None
hub_model_id: redis/langcache-reranker-v1-miniL6-softmnrl-triplet
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: True
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss	test_cls_average_precision
0	0	-	0.3223	0.5734
0.1322	1000	0.4286	0.3215	0.5736
0.2644	2000	0.4241	0.3151	0.5742
0.3966	3000	0.4182	0.3038	0.5755
0.5288	4000	0.4036	0.2876	0.5770
0.6609	5000	0.3919	0.2619	0.5824
0.7931	6000	0.3694	0.2290	0.5908
0.9253	7000	0.3481	0.1966	0.6039
1.0575	8000	0.3109	0.1650	0.6231
1.1897	9000	0.2665	0.1384	0.6565
1.3219	10000	0.2281	0.1154	0.6911
1.4541	11000	0.1984	0.0928	0.7130
1.5863	12000	0.1794	0.0814	0.7341
1.7184	13000	0.1619	0.0698	0.7376
1.8506	14000	0.1498	0.0619	0.7523
1.9828	15000	0.1409	0.0581	0.7584
2.1150	16000	0.1315	0.0537	0.7699
2.2472	17000	0.1239	0.0495	0.7712
2.3794	18000	0.1157	0.0471	0.7847
2.5116	19000	0.1093	0.0415	0.7978
2.6438	20000	0.1026	0.0428	0.8013
2.7759	21000	0.0958	0.0393	0.8096
2.9081	22000	0.0922	0.0387	0.8105
3.0403	23000	0.0873	0.0415	0.8138
3.1725	24000	0.0823	0.0382	0.8178
3.3047	25000	0.0807	0.0369	0.8084
3.4369	26000	0.0772	0.0370	0.8199
3.5691	27000	0.0734	0.0348	0.8261
3.7013	28000	0.0709	0.0335	0.8286
3.8334	29000	0.067	0.0363	0.8374
3.9656	30000	0.0675	0.0359	0.8271
4.0978	31000	0.0629	0.0337	0.8275
4.2300	32000	0.0611	0.0350	0.8378
4.3622	33000	0.0618	0.0372	0.8441
4.4944	34000	0.0585	0.0341	0.8423
4.6266	35000	0.0569	0.0364	0.8469
4.7588	36000	0.055	0.0355	0.8398
4.8909	37000	0.0529	0.0316	0.8474
5.0231	38000	0.0522	0.0346	0.8442
5.1553	39000	0.0501	0.0384	0.8468
5.2875	40000	0.0503	0.0345	0.8534
5.4197	41000	0.0487	0.0321	0.8523
5.5519	42000	0.0465	0.0321	0.8519
5.6841	43000	0.0453	0.0316	0.8527
5.8163	44000	0.0426	0.0355	0.8600
5.9484	45000	0.043	0.0329	0.8527
6.0806	46000	0.0405	0.0358	0.8568
6.2128	47000	0.0398	0.0345	0.8514
6.3450	48000	0.0406	0.0336	0.8499
6.4772	49000	0.0381	0.0324	0.8589
6.6094	50000	0.0377	0.0322	0.8534
6.7416	51000	0.0357	0.0321	0.8518
6.8738	52000	0.035	0.0338	0.8554
7.0059	53000	0.035	0.0348	0.8585
7.1381	54000	0.033	0.0341	0.8582
7.2703	55000	0.0347	0.0341	0.8591
7.4025	56000	0.0339	0.0327	0.8575
7.5347	57000	0.0325	0.0315	0.8636
7.6669	58000	0.0313	0.0353	0.8628
7.7991	59000	0.0305	0.0353	0.8638
7.9313	60000	0.0296	0.0358	0.8641
8.0635	61000	0.0292	0.0348	0.8625
8.1956	62000	0.0301	0.0366	0.8521
8.3278	63000	0.03	0.0336	0.8608
8.4600	64000	0.0287	0.0336	0.8695
8.5922	65000	0.0279	0.0315	0.8627
8.7244	66000	0.027	0.0322	0.8617
8.8566	67000	0.026	0.0336	0.8613
8.9888	68000	0.0268	0.0369	0.8648
9.1210	69000	0.0259	0.0333	0.8646
9.2531	70000	0.0261	0.0350	0.8559
9.3853	71000	0.0261	0.0332	0.8613
9.5175	72000	0.0253	0.0336	0.8666
9.6497	73000	0.0252	0.0342	0.8629
9.7819	74000	0.0243	0.0348	0.8635
9.9141	75000	0.0244	0.0338	0.8656
10.0463	76000	0.0238	0.0349	0.8643
10.1785	77000	0.0239	0.0359	0.8650
10.3106	78000	0.0241	0.0337	0.8628
10.4428	79000	0.0236	0.0349	0.8689
10.5750	80000	0.0234	0.0348	0.8675
10.7072	81000	0.0225	0.0345	0.8668
10.8394	82000	0.0217	0.0354	0.8722
10.9716	83000	0.0226	0.0339	0.8706
11.1038	84000	0.0215	0.0354	0.8680
11.2360	85000	0.022	0.0364	0.8653
11.3681	86000	0.022	0.0348	0.8678
11.5003	87000	0.0217	0.0353	0.8712
11.6325	88000	0.0221	0.0338	0.8682
11.7647	89000	0.0213	0.0324	0.8642
11.8969	90000	0.021	0.0336	0.869
12.0291	91000	0.0206	0.0352	0.8707
12.1613	92000	0.0203	0.0344	0.8686
12.2935	93000	0.0207	0.0349	0.8658
12.4256	94000	0.0206	0.0339	0.8668
12.5578	95000	0.0199	0.0342	0.8687
12.6900	96000	0.0202	0.0323	0.8709
12.8222	97000	0.0192	0.0357	0.8697
12.9544	98000	0.0196	0.0359	0.8716
13.0866	99000	0.0196	0.0357	0.8723
13.2188	100000	0.0195	0.0347	0.8687
13.3510	101000	0.0198	0.0343	0.8681
13.4831	102000	0.0192	0.0329	0.8724
13.6153	103000	0.0191	0.0336	0.8680
13.7475	104000	0.0186	0.0326	0.8685
13.8797	105000	0.0183	0.0338	0.8708
14.0119	106000	0.0186	0.0346	0.8681
14.1441	107000	0.0177	0.0357	0.8698
14.2763	108000	0.0193	0.0344	0.8677
14.4085	109000	0.0186	0.0323	0.8692
14.5406	110000	0.018	0.0336	0.8676
14.6728	111000	0.0177	0.0353	0.8705
14.8050	112000	0.0176	0.0338	0.8704
14.9372	113000	0.0178	0.0348	0.8715
15.0694	114000	0.017	0.0353	0.8707
15.2016	115000	0.0181	0.0349	0.8698
15.3338	116000	0.0182	0.0341	0.8681
15.4660	117000	0.0171	0.0343	0.8689
15.5981	118000	0.0176	0.0341	0.8682
15.7303	119000	0.0173	0.0336	0.8703
15.8625	120000	0.0161	0.0342	0.8701
15.9947	121000	0.0174	0.0349	0.8714
16.1269	122000	0.0171	0.0341	0.8715
16.2591	123000	0.0171	0.0342	0.8669
16.3913	124000	0.0174	0.0336	0.8682
16.5235	125000	0.0167	0.0339	0.8709
16.6557	126000	0.0169	0.0344	0.8703
16.7878	127000	0.016	0.0341	0.8707
16.9200	128000	0.0163	0.0342	0.8717
17.0522	129000	0.0163	0.0342	0.8706
17.1844	130000	0.0163	0.0347	0.8679
17.3166	131000	0.017	0.0335	0.8683
17.4488	132000	0.0166	0.0337	0.8688
17.5810	133000	0.0165	0.0334	0.8706
17.7132	134000	0.0157	0.0334	0.8708
17.8453	135000	0.0154	0.0345	0.8692
17.9775	136000	0.0159	0.0340	0.8719
18.1097	137000	0.0156	0.0338	0.8698
18.2419	138000	0.0162	0.0333	0.8680
18.3741	139000	0.0161	0.0337	0.8694
18.5063	140000	0.0161	0.0345	0.8715
18.6385	141000	0.0163	0.0331	0.8722
18.7707	142000	0.015	0.0336	0.8733
18.9028	143000	0.0153	0.0350	0.8735
19.0350	144000	0.0152	0.0355	0.8722
19.1672	145000	0.0158	0.0354	0.8708
19.2994	146000	0.0158	0.0345	0.8690
19.4316	147000	0.0161	0.0327	0.8705
19.5638	148000	0.0155	0.0335	0.8721
19.6960	149000	0.015	0.0330	0.8709
19.8282	150000	0.0143	0.0339	0.8717
19.9603	151000	0.0156	0.0340	0.8712
20.0925	152000	0.0149	0.0337	0.8709
20.2247	153000	0.0154	0.0334	0.8701
20.3569	154000	0.0155	0.0337	0.8692
20.4891	155000	0.0156	0.0335	0.8708
20.6213	156000	0.0153	0.0337	0.8698
20.7535	157000	0.0149	0.0328	0.8699
20.8857	158000	0.0144	0.0331	0.8691
21.0178	159000	0.0148	0.0339	0.8729
21.1500	160000	0.0152	0.0331	0.8705
21.2822	161000	0.0156	0.0333	0.8690
21.4144	162000	0.0147	0.0328	0.8706
21.5466	163000	0.0148	0.0335	0.8691
21.6788	164000	0.0145	0.0342	0.8698
21.8110	165000	0.0142	0.0336	0.8701
21.9432	166000	0.0141	0.0346	0.8708
22.0753	167000	0.0148	0.0344	0.8713
22.2075	168000	0.0151	0.0335	0.8712
22.3397	169000	0.0147	0.0344	0.8715
22.4719	170000	0.0145	0.0343	0.8711
22.6041	171000	0.0144	0.0331	0.8709
22.7363	172000	0.014	0.0333	0.8716
22.8685	173000	0.0142	0.0341	0.8718
23.0007	174000	0.015	0.0344	0.8717
23.1328	175000	0.0141	0.0337	0.8713
23.2650	176000	0.0146	0.0336	0.8694
23.3972	177000	0.0143	0.0338	0.8700
23.5294	178000	0.0147	0.0330	0.8700
23.6616	179000	0.0141	0.0334	0.8711
23.7938	180000	0.0142	0.0329	0.8707
23.9260	181000	0.014	0.0338	0.8711
24.0582	182000	0.0141	0.0334	0.8726
24.1904	183000	0.0143	0.0350	0.8712
24.3225	184000	0.0144	0.0340	0.8710
24.4547	185000	0.015	0.0330	0.8707
24.5869	186000	0.0144	0.0341	0.8711
24.7191	187000	0.0143	0.0332	0.8707
24.8513	188000	0.014	0.0345	0.8720
24.9835	189000	0.0141	0.0353	0.8718
25.1157	190000	0.0137	0.0349	0.8716
25.2479	191000	0.0142	0.0345	0.8713
25.3800	192000	0.0143	0.0334	0.8706
25.5122	193000	0.0137	0.0332	0.8709
25.6444	194000	0.0143	0.0339	0.8692
25.7766	195000	0.0136	0.0338	0.8706
25.9088	196000	0.0134	0.0333	0.8705
26.0410	197000	0.0136	0.0350	0.8718
26.1732	198000	0.0136	0.0345	0.8713
26.3054	199000	0.0142	0.0340	0.8701
26.4375	200000	0.0141	0.0335	0.8707
26.5697	201000	0.0146	0.0343	0.8707
26.7019	202000	0.0136	0.0341	0.8700
26.8341	203000	0.0131	0.0348	0.8713
26.9663	204000	0.014	0.0345	0.8719
27.0985	205000	0.0135	0.0349	0.8713
27.2307	206000	0.0135	0.0337	0.8714
27.3629	207000	0.0146	0.0334	0.8713
27.4950	208000	0.0138	0.0337	0.8722
27.6272	209000	0.0136	0.0331	0.8709
27.7594	210000	0.0133	0.0343	0.8712
27.8916	211000	0.0137	0.0341	0.8716
28.0238	212000	0.0132	0.0340	0.8730
28.1560	213000	0.0136	0.0344	0.8718
28.2882	214000	0.0143	0.0337	0.8717
28.4204	215000	0.0136	0.0340	0.8716
28.5525	216000	0.014	0.0334	0.8713
28.6847	217000	0.0131	0.0338	0.8714
28.8169	218000	0.0131	0.0337	0.8716
28.9491	219000	0.0136	0.0346	0.8715
29.0813	220000	0.0132	0.0347	0.8722
29.2135	221000	0.0136	0.0344	0.8719
29.3457	222000	0.0137	0.0345	0.8710
29.4779	223000	0.0138	0.0337	0.8708
29.6100	224000	0.013	0.0337	0.8708
29.7422	225000	0.0134	0.0343	0.8714
29.8744	226000	0.0132	0.0338	0.8717
30.0066	227000	0.0133	0.0335	0.8718
30.1388	228000	0.013	0.0340	0.8718
30.2710	229000	0.0144	0.0332	0.8710
30.4032	230000	0.014	0.0346	0.8716
30.5354	231000	0.0137	0.0330	0.8717
30.6675	232000	0.0131	0.0342	0.8718
30.7997	233000	0.0128	0.0337	0.8721
30.9319	234000	0.0135	0.0342	0.8718
31.0641	235000	0.0138	0.0346	0.8720
31.1963	236000	0.0133	0.0347	0.8717
31.3285	237000	0.0137	0.0335	0.8712
31.4607	238000	0.0137	0.0337	0.8718
31.5929	239000	0.0131	0.0340	0.8719
31.7250	240000	0.0129	0.0334	0.8720
31.8572	241000	0.0133	0.0336	0.8725
31.9894	242000	0.0137	0.0343	0.8722
32.1216	243000	0.0132	0.0329	0.8720
32.2538	244000	0.0135	0.0338	0.8718
32.3860	245000	0.0129	0.0344	0.8724
32.5182	246000	0.0136	0.0342	0.8722
32.6504	247000	0.0133	0.0331	0.8716
32.7826	248000	0.0128	0.0337	0.8718
32.9147	249000	0.0127	0.0338	0.8724
33.0469	250000	0.013	0.0328	0.8724
33.1791	251000	0.0135	0.0337	0.8724
33.3113	252000	0.0131	0.0334	0.8723
33.4435	253000	0.0134	0.0339	0.8726
33.5757	254000	0.0135	0.0338	0.8725
33.7079	255000	0.013	0.0341	0.8730
33.8401	256000	0.0126	0.0334	0.8731
33.9722	257000	0.0136	0.0338	0.8730
34.1044	258000	0.0123	0.0338	0.8727
34.2366	259000	0.0135	0.0336	0.8724
34.3688	260000	0.0136	0.0343	0.8722
34.5010	261000	0.0134	0.0341	0.8723
34.6332	262000	0.0136	0.0343	0.8718
34.7654	263000	0.0131	0.0344	0.8721
34.8976	264000	0.0128	0.0343	0.8724
35.0297	265000	0.0129	0.0336	0.8725
35.1619	266000	0.0128	0.0334	0.8726
35.2941	267000	0.013	0.0340	0.8723
35.4263	268000	0.0133	0.0341	0.8723
35.5585	269000	0.0132	0.0331	0.8722
35.6907	270000	0.0127	0.0335	0.8721
35.8229	271000	0.0123	0.0334	0.8725
35.9551	272000	0.0135	0.0343	0.8726
36.0872	273000	0.0125	0.0345	0.8724
36.2194	274000	0.0134	0.0336	0.8722
36.3516	275000	0.0132	0.0338	0.8721
36.4838	276000	0.0136	0.0331	0.8722
36.6160	277000	0.0133	0.0335	0.8718
36.7482	278000	0.0125	0.0336	0.8721
36.8804	279000	0.0122	0.0344	0.8721
37.0126	280000	0.013	0.0336	0.8725
37.1447	281000	0.0132	0.0333	0.8726
37.2769	282000	0.0137	0.0333	0.8722
37.4091	283000	0.0133	0.0339	0.8723
37.5413	284000	0.013	0.0335	0.8723
37.6735	285000	0.0129	0.0329	0.8721
37.8057	286000	0.013	0.0327	0.8721
37.9379	287000	0.0124	0.0338	0.8722
38.0701	288000	0.0131	0.0338	0.8722
38.2022	289000	0.0129	0.0342	0.8722
38.3344	290000	0.013	0.0336	0.8721
38.4666	291000	0.0134	0.0335	0.8722
38.5988	292000	0.0129	0.0338	0.8720
38.7310	293000	0.0122	0.0337	0.8720
38.8632	294000	0.0123	0.0338	0.8722
38.9954	295000	0.0132	0.0335	0.8723
39.1276	296000	0.0128	0.0333	0.8722
39.2597	297000	0.0135	0.0336	0.8721
39.3919	298000	0.0132	0.0342	0.8722
39.5241	299000	0.0136	0.0328	0.8723
39.6563	300000	0.0125	0.0339	0.8722
39.7885	301000	0.0125	0.0343	0.8722
39.9207	302000	0.0126	0.0339	0.8723
40.0529	303000	0.0129	0.0338	0.8723
40.1851	304000	0.0133	0.0334	0.8723
40.3173	305000	0.0134	0.0336	0.8723
40.4494	306000	0.0127	0.0336	0.8724
40.5816	307000	0.0126	0.0342	0.8723
40.7138	308000	0.013	0.0340	0.8721
40.8460	309000	0.013	0.0332	0.8721
40.9782	310000	0.0129	0.0337	0.8723
41.1104	311000	0.0123	0.0328	0.8723
41.2426	312000	0.013	0.0336	0.8723
41.3748	313000	0.0132	0.0337	0.8722
41.5069	314000	0.0132	0.0335	0.8722
41.6391	315000	0.0131	0.0343	0.8722
41.7713	316000	0.0122	0.0339	0.8722
41.9035	317000	0.0125	0.0340	0.8722
42.0357	318000	0.0122	0.0342	0.8722
42.1679	319000	0.0129	0.0337	0.8721
42.3001	320000	0.013	0.0330	0.8721
42.4323	321000	0.013	0.0332	0.8721
42.5644	322000	0.0141	0.0349	0.8721
42.6966	323000	0.013	0.0334	0.8720
42.8288	324000	0.0125	0.0339	0.8721
42.9610	325000	0.0126	0.0342	0.8721
43.0932	326000	0.0127	0.0339	0.8721
43.2254	327000	0.0126	0.0330	0.8721
43.3576	328000	0.013	0.0343	0.8721
43.4898	329000	0.0135	0.0334	0.8721
43.6219	330000	0.0131	0.0327	0.8721
43.7541	331000	0.0124	0.0334	0.8722
43.8863	332000	0.0126	0.0344	0.8721
44.0185	333000	0.0131	0.0338	0.8722
44.1507	334000	0.0121	0.0340	0.8722
44.2829	335000	0.0131	0.0336	0.8721
44.4151	336000	0.0135	0.0340	0.8722
44.5473	337000	0.0131	0.0335	0.8722
44.6794	338000	0.0132	0.0340	0.8722
44.8116	339000	0.0128	0.0333	0.8722
44.9438	340000	0.0124	0.0333	0.8722
45.0760	341000	0.0131	0.0337	0.8722
45.2082	342000	0.0129	0.0341	0.8722
45.3404	343000	0.0133	0.0335	0.8722
45.4726	344000	0.0133	0.0341	0.8722
45.6048	345000	0.013	0.0334	0.8722
45.7369	346000	0.0129	0.0343	0.8722
45.8691	347000	0.0125	0.0335	0.8722
46.0013	348000	0.0133	0.0344	0.8722
46.1335	349000	0.013	0.0332	0.8722
46.2657	350000	0.0128	0.0337	0.8722
46.3979	351000	0.0132	0.0334	0.8722
46.5301	352000	0.0127	0.0343	0.8722
46.6623	353000	0.0127	0.0334	0.8722
46.7944	354000	0.0126	0.0332	0.8722
46.9266	355000	0.013	0.0339	0.8722
47.0588	356000	0.0126	0.0340	0.8722
47.1910	357000	0.0132	0.0336	0.8722
47.3232	358000	0.0138	0.0334	0.8722
47.4554	359000	0.0133	0.0336	0.8722
47.5876	360000	0.0135	0.0340	0.8722
47.7198	361000	0.0129	0.0341	0.8722
47.8519	362000	0.0123	0.0334	0.8722
47.9841	363000	0.0126	0.0334	0.8722
48.1163	364000	0.0121	0.0337	0.8722
48.2485	365000	0.0127	0.0342	0.8722
48.3807	366000	0.0124	0.0336	0.8722
48.5129	367000	0.0125	0.0338	0.8722
48.6451	368000	0.0125	0.0341	0.8721
48.7773	369000	0.0122	0.0333	0.8722
48.9095	370000	0.0123	0.0336	0.8722
49.0416	371000	0.0124	0.0341	0.8722
49.1738	372000	0.0132	0.0330	0.8722
49.3060	373000	0.0128	0.0342	0.8722
49.4382	374000	0.0132	0.0341	0.8722
49.5704	375000	0.013	0.0334	0.8722
49.7026	376000	0.0126	0.0340	0.8722
49.8348	377000	0.0126	0.0337	0.8722
49.9670	378000	0.0131	0.0337	0.8722

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.12.3
Sentence Transformers: 5.1.0
Transformers: 4.56.0
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.0.0
Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}