much1na/ettin-encoder-17m-denseon-embeddistill

SentenceTransformer based on jhu-clsp/ettin-encoder-17m

This is a sentence-transformers model finetuned from jhu-clsp/ettin-encoder-17m on the msmarco-embed-dense_on dataset. It maps sentences & paragraphs to a 256-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: jhu-clsp/ettin-encoder-17m
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 256 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Training Dataset:
  • msmarco-embed-dense_on

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'ModernBertModel'})
  (1): Pooling({'embedding_dimension': 256, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'Healthcare is involved, directly or indirectly, with the provision of health services to individuals. These services can occur in a variety of work settings, including hospitals, clinics, dental offices, out-patient surgery centers, birthing centers, emergency medical care, home healthcare, and nursing homes. Healthcare workers face a number of serious safety and health hazards.',
]
documents = [
    'In a right triangle, the orthocenter is the polygon vertex of the right angle. When the vertices of a triangle are combined with its orthocenter, any one of the points is the orthocenter of the other three, as first noted by Carnot (Wells 1991). These four points therefore form an orthocentric system. The circumcenter and orthocenter are isogonal conjugates. The orthocenter lies on the Euler line.',
    'who makes fire extinguishers',
    'what is powdery mildew cannabis',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 256] [3, 256]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.5302, 0.5367, 0.4435]])

Evaluation

Metrics

Information Retrieval

• Dataset: NanoMSMARCO
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.28
cosine_accuracy@3	0.48
cosine_accuracy@5	0.58
cosine_accuracy@10	0.78
cosine_precision@1	0.28
cosine_precision@3	0.16
cosine_precision@5	0.116
cosine_precision@10	0.078
cosine_recall@1	0.28
cosine_recall@3	0.48
cosine_recall@5	0.58
cosine_recall@10	0.78
cosine_ndcg@10	0.5032
cosine_mrr@10	0.4184
cosine_map@100	0.4276

Training Details

Training Dataset

msmarco-embed-dense_on

• Dataset: msmarco-embed-dense_on
• Size: 973,530 training samples
• Columns: text and label
• Approximate statistics based on the first 100 samples:

  	text	label
  type	string	list
  modality	text
  details	min: 4 tokens mean: 45.82 tokens max: 179 tokens	size: 768 elements

• Samples:

  text	label
  how long background check bank returned	[-1.896484375, 0.288330078125, -0.406005859375, -1.6953125, 1.98046875, ...]
  Humira® (adalimumab) is a prescription medication licensed to treat certain inflammatory conditions that affect the joints, spine, or digestive system. Specific Humira uses include the treatment of: Ankylosing spondylitis. Crohn's disease. Juvenile idiopathic arthritis, also known as juvenile rheumatoid arthritis.	[-1.77734375, -0.072265625, -0.301513671875, -0.66748046875, 0.80810546875, ...]
  Bonfim is a municipality located in the mideast of the state of Roraima in Brazil. Its population is 12,626 and its area is 8,095 km². The city lies opposite the Takutu River from Lethem, Guyana. The Takutu River Bridge links Bonfim and Roraima with the town of Lethem and the Atlantic port of Georgetown, Guyana.	[-0.411865234375, 1.58984375, 0.76123046875, 1.6171875, 0.34375, ...]

• Loss: EmbedDistillLoss with these parameters:

  {
      "distance_metric": "cosine",
      "projection_dim": 768
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• learning_rate: 0.0001
• warmup_steps: 0.1
• weight_decay: 0.01
• bf16: True
• load_best_model_at_end: True
• seed: 12
• dataloader_num_workers: 2

All Hyperparameters

Click to expand

• per_device_train_batch_size: 32
• num_train_epochs: 3
• max_steps: -1
• learning_rate: 0.0001
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 8
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: True
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 12
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 2
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	NanoMSMARCO_cosine_ndcg@10
-1	-1	-	0.01
0.0000	1	1.0217	-
0.0042	128	1.0103	-
0.0084	256	0.9761	-
0.0126	384	0.8144	-
0.0168	512	0.7116	-
0.0210	640	0.6541	-
0.0252	768	0.6331	-
0.0295	896	0.6174	-
0.0337	1024	0.6054	-
0.0379	1152	0.5923	-
0.0421	1280	0.5783	-
0.0463	1408	0.5659	-
0.0505	1536	0.5530	-
0.0547	1664	0.5433	-
0.0589	1792	0.5319	-
0.0631	1920	0.5211	-
0.0673	2048	0.5109	-
0.0715	2176	0.5060	-
0.0757	2304	0.4930	-
0.0799	2432	0.4891	-
0.0841	2560	0.4784	-
0.0884	2688	0.4739	-
0.0926	2816	0.4699	-
0.0968	2944	0.4638	-
0.1010	3072	0.4599	-
0.1052	3200	0.4509	-
0.1094	3328	0.4526	-
0.1136	3456	0.4470	-
0.1178	3584	0.4412	-
0.1220	3712	0.4403	-
0.1262	3840	0.4332	-
0.1304	3968	0.4310	-
0.1346	4096	0.4255	-
0.1388	4224	0.4276	-
0.1430	4352	0.4208	-
0.1473	4480	0.4164	-
0.1515	4608	0.4148	-
0.1557	4736	0.4117	-
0.1599	4864	0.4109	-
0.1641	4992	0.4061	-
0.1683	5120	0.4052	-
0.1725	5248	0.4016	-
0.1767	5376	0.3998	-
0.1809	5504	0.3970	-
0.1851	5632	0.3958	-
0.1893	5760	0.3969	-
0.1935	5888	0.3932	-
0.1977	6016	0.3934	-
0.2020	6144	0.3918	-
0.2062	6272	0.3900	-
0.2104	6400	0.3845	-
0.2146	6528	0.3841	-
0.2188	6656	0.3844	-
0.2230	6784	0.3806	-
0.2272	6912	0.3823	-
0.2314	7040	0.3779	-
0.2356	7168	0.3779	-
0.2398	7296	0.3773	-
0.2440	7424	0.3753	-
0.2482	7552	0.3756	-
0.2524	7680	0.3723	-
0.2566	7808	0.3726	-
0.2609	7936	0.3681	-
0.2651	8064	0.3681	-
0.2693	8192	0.3669	-
0.2735	8320	0.3652	-
0.2777	8448	0.3653	-
0.2819	8576	0.3642	-
0.2861	8704	0.3629	-
0.2903	8832	0.3639	-
0.2945	8960	0.3604	-
0.2987	9088	0.3577	-
0.3029	9216	0.3604	-
0.3071	9344	0.3577	-
0.3113	9472	0.3587	-
0.3156	9600	0.3556	-
0.3198	9728	0.3543	-
0.3240	9856	0.3538	-
0.3282	9984	0.3535	-
0.3324	10112	0.3512	-
0.3366	10240	0.3517	-
0.3408	10368	0.3480	-
0.3450	10496	0.3484	-
0.3492	10624	0.3449	-
0.3534	10752	0.3462	-
0.3576	10880	0.3443	-
0.3618	11008	0.3432	-
0.3660	11136	0.3454	-
0.3702	11264	0.3427	-
0.3745	11392	0.3413	-
0.3787	11520	0.3406	-
0.3829	11648	0.3404	-
0.3871	11776	0.3394	-
0.3913	11904	0.3368	-
0.3955	12032	0.3379	-
0.3997	12160	0.3381	-
0.4039	12288	0.3362	-
0.4081	12416	0.3379	-
0.4123	12544	0.3346	-
0.4165	12672	0.3325	-
0.4207	12800	0.3331	-
0.4249	12928	0.3311	-
0.4291	13056	0.3318	-
0.4334	13184	0.3283	-
0.4376	13312	0.3296	-
0.4418	13440	0.3282	-
0.4460	13568	0.3294	-
0.4502	13696	0.3270	-
0.4544	13824	0.3264	-
0.4586	13952	0.3290	-
0.4628	14080	0.3257	-
0.4670	14208	0.3238	-
0.4712	14336	0.3260	-
0.4754	14464	0.3223	-
0.4796	14592	0.3224	-
0.4838	14720	0.3234	-
0.4881	14848	0.3221	-
0.4923	14976	0.3202	-
0.4965	15104	0.3214	-
0.5007	15232	0.3207	-
0.5049	15360	0.3195	-
0.5091	15488	0.3165	-
0.5133	15616	0.3184	-
0.5175	15744	0.3189	-
0.5217	15872	0.3194	-
0.5259	16000	0.3162	-
0.5301	16128	0.3174	-
0.5343	16256	0.3151	-
0.5385	16384	0.3172	-
0.5427	16512	0.3158	-
0.5470	16640	0.3144	-
0.5512	16768	0.3141	-
0.5554	16896	0.3134	-
0.5596	17024	0.3135	-
0.5638	17152	0.3124	-
0.5680	17280	0.3118	-
0.5722	17408	0.3104	-
0.5764	17536	0.3102	-
0.5806	17664	0.3104	-
0.5848	17792	0.3078	-
0.5890	17920	0.3080	-
0.5932	18048	0.3088	-
0.5974	18176	0.3070	-
0.6017	18304	0.3067	-
0.6059	18432	0.3060	-
0.6101	18560	0.3059	-
0.6143	18688	0.3061	-
0.6185	18816	0.3043	-
0.6227	18944	0.3046	-
0.6269	19072	0.3041	-
0.6311	19200	0.3032	-
0.6353	19328	0.3039	-
0.6395	19456	0.3032	-
0.6437	19584	0.3043	-
0.6479	19712	0.3019	-
0.6521	19840	0.3015	-
0.6563	19968	0.3023	-
0.6606	20096	0.2995	-
0.6648	20224	0.3011	-
0.6690	20352	0.2992	-
0.6732	20480	0.2997	-
0.6774	20608	0.2993	-
0.6816	20736	0.2980	-
0.6858	20864	0.3010	-
0.6900	20992	0.2956	-
0.6942	21120	0.2988	-
0.6984	21248	0.2969	-
0.7026	21376	0.2961	-
0.7068	21504	0.2964	-
0.7110	21632	0.2968	-
0.7152	21760	0.2961	-
0.7195	21888	0.2941	-
0.7237	22016	0.2959	-
0.7279	22144	0.2943	-
0.7321	22272	0.2940	-
0.7363	22400	0.2933	-
0.7405	22528	0.2930	-
0.7447	22656	0.2927	-
0.7489	22784	0.2914	-
0.7531	22912	0.2906	-
0.7573	23040	0.2892	-
0.7615	23168	0.2882	-
0.7657	23296	0.2899	-
0.7699	23424	0.2897	-
0.7742	23552	0.2894	-
0.7784	23680	0.2912	-
0.7826	23808	0.2892	-
0.7868	23936	0.2880	-
0.7910	24064	0.2885	-
0.7952	24192	0.2908	-
0.7994	24320	0.2881	-
0.8036	24448	0.2862	-
0.8078	24576	0.2875	-
0.8120	24704	0.2867	-
0.8162	24832	0.2867	-
0.8204	24960	0.2876	-
0.8217	25000	-	0.3645
0.8246	25088	0.2868	-
0.8288	25216	0.2873	-
0.8331	25344	0.2849	-
0.8373	25472	0.2844	-
0.8415	25600	0.2852	-
0.8457	25728	0.2854	-
0.8499	25856	0.2858	-
0.8541	25984	0.2832	-
0.8583	26112	0.2842	-
0.8625	26240	0.2831	-
0.8667	26368	0.2832	-
0.8709	26496	0.2844	-
0.8751	26624	0.2830	-
0.8793	26752	0.2823	-
0.8835	26880	0.2796	-
0.8877	27008	0.2802	-
0.8920	27136	0.2808	-
0.8962	27264	0.2795	-
0.9004	27392	0.2804	-
0.9046	27520	0.2799	-
0.9088	27648	0.2801	-
0.9130	27776	0.2806	-
0.9172	27904	0.2803	-
0.9214	28032	0.2808	-
0.9256	28160	0.2778	-
0.9298	28288	0.2776	-
0.9340	28416	0.2799	-
0.9382	28544	0.2806	-
0.9424	28672	0.2797	-
0.9467	28800	0.2779	-
0.9509	28928	0.2782	-
0.9551	29056	0.2754	-
0.9593	29184	0.2782	-
0.9635	29312	0.2788	-
0.9677	29440	0.2751	-
0.9719	29568	0.2741	-
0.9761	29696	0.2751	-
0.9803	29824	0.2755	-
0.9845	29952	0.2744	-
0.9887	30080	0.2748	-
0.9929	30208	0.2744	-
0.9971	30336	0.2725	-
1.0013	30464	0.2717	-
1.0056	30592	0.2703	-
1.0098	30720	0.2719	-
1.0140	30848	0.2704	-
1.0182	30976	0.2719	-
1.0224	31104	0.2707	-
1.0266	31232	0.2700	-
1.0308	31360	0.2693	-
1.0350	31488	0.2687	-
1.0392	31616	0.2710	-
1.0434	31744	0.2716	-
1.0476	31872	0.2680	-
1.0518	32000	0.2686	-
1.0560	32128	0.2692	-
1.0603	32256	0.2681	-
1.0645	32384	0.2688	-
1.0687	32512	0.2681	-
1.0729	32640	0.2667	-
1.0771	32768	0.2682	-
1.0813	32896	0.2680	-
1.0855	33024	0.2670	-
1.0897	33152	0.2684	-
1.0939	33280	0.2671	-
1.0981	33408	0.2684	-
1.1023	33536	0.2676	-
1.1065	33664	0.2683	-
1.1107	33792	0.2670	-
1.1149	33920	0.2649	-
1.1192	34048	0.2657	-
1.1234	34176	0.2685	-
1.1276	34304	0.2656	-
1.1318	34432	0.2641	-
1.1360	34560	0.2646	-
1.1402	34688	0.2643	-
1.1444	34816	0.2644	-
1.1486	34944	0.2637	-
1.1528	35072	0.2642	-
1.1570	35200	0.2634	-
1.1612	35328	0.2638	-
1.1654	35456	0.2643	-
1.1696	35584	0.2633	-
1.1738	35712	0.2647	-
1.1781	35840	0.2626	-
1.1823	35968	0.2611	-
1.1865	36096	0.2639	-
1.1907	36224	0.2631	-
1.1949	36352	0.2621	-
1.1991	36480	0.2613	-
1.2033	36608	0.2621	-
1.2075	36736	0.2635	-
1.2117	36864	0.2627	-
1.2159	36992	0.2616	-
1.2201	37120	0.2635	-
1.2243	37248	0.2626	-
1.2285	37376	0.2615	-
1.2328	37504	0.2608	-
1.2370	37632	0.2624	-
1.2412	37760	0.2602	-
1.2454	37888	0.2599	-
1.2496	38016	0.2602	-
1.2538	38144	0.2599	-
1.2580	38272	0.2613	-
1.2622	38400	0.2593	-
1.2664	38528	0.2607	-
1.2706	38656	0.2595	-
1.2748	38784	0.2603	-
1.2790	38912	0.2591	-
1.2832	39040	0.2603	-
1.2874	39168	0.2606	-
1.2917	39296	0.2617	-
1.2959	39424	0.2603	-
1.3001	39552	0.2592	-
1.3043	39680	0.2583	-
1.3085	39808	0.2594	-
1.3127	39936	0.2593	-
1.3169	40064	0.2566	-
1.3211	40192	0.2588	-
1.3253	40320	0.2579	-
1.3295	40448	0.2562	-
1.3337	40576	0.2594	-
1.3379	40704	0.2568	-
1.3421	40832	0.2577	-
1.3463	40960	0.2579	-
1.3506	41088	0.2568	-
1.3548	41216	0.2557	-
1.3590	41344	0.2571	-
1.3632	41472	0.2552	-
1.3674	41600	0.2553	-
1.3716	41728	0.2574	-
1.3758	41856	0.2558	-
1.3800	41984	0.2560	-
1.3842	42112	0.2563	-
1.3884	42240	0.2559	-
1.3926	42368	0.2577	-
1.3968	42496	0.2547	-
1.4010	42624	0.2550	-
1.4053	42752	0.2551	-
1.4095	42880	0.2547	-
1.4137	43008	0.2537	-
1.4179	43136	0.2540	-
1.4221	43264	0.2552	-
1.4263	43392	0.2553	-
1.4305	43520	0.2530	-
1.4347	43648	0.2541	-
1.4389	43776	0.2564	-
1.4431	43904	0.2537	-
1.4473	44032	0.2534	-
1.4515	44160	0.2540	-
1.4557	44288	0.2541	-
1.4599	44416	0.2536	-
1.4642	44544	0.2539	-
1.4684	44672	0.2524	-
1.4726	44800	0.2548	-
1.4768	44928	0.2532	-
1.4810	45056	0.2532	-
1.4852	45184	0.2530	-
1.4894	45312	0.2524	-
1.4936	45440	0.2541	-
1.4978	45568	0.2516	-
1.5020	45696	0.2520	-
1.5062	45824	0.2533	-
1.5104	45952	0.2530	-
1.5146	46080	0.2521	-
1.5189	46208	0.2520	-
1.5231	46336	0.2510	-
1.5273	46464	0.2534	-
1.5315	46592	0.2534	-
1.5357	46720	0.2517	-
1.5399	46848	0.2519	-
1.5441	46976	0.2506	-
1.5483	47104	0.2514	-
1.5525	47232	0.2518	-
1.5567	47360	0.2521	-
1.5609	47488	0.2523	-
1.5651	47616	0.2501	-
1.5693	47744	0.2505	-
1.5735	47872	0.2516	-
1.5778	48000	0.2507	-
1.5820	48128	0.2492	-
1.5862	48256	0.2479	-
1.5904	48384	0.2497	-
1.5946	48512	0.2491	-
1.5988	48640	0.2501	-
1.6030	48768	0.2478	-
1.6072	48896	0.2493	-
1.6114	49024	0.2490	-
1.6156	49152	0.2504	-
1.6198	49280	0.2484	-
1.6240	49408	0.2502	-
1.6282	49536	0.2500	-
1.6324	49664	0.2503	-
1.6367	49792	0.2480	-
1.6409	49920	0.2490	-
1.6435	50000	-	0.4470
1.6451	50048	0.2475	-
1.6493	50176	0.2477	-
1.6535	50304	0.2489	-
1.6577	50432	0.2493	-
1.6619	50560	0.2469	-
1.6661	50688	0.2487	-
1.6703	50816	0.2483	-
1.6745	50944	0.2473	-
1.6787	51072	0.2485	-
1.6829	51200	0.2469	-
1.6871	51328	0.2464	-
1.6914	51456	0.2477	-
1.6956	51584	0.2459	-
1.6998	51712	0.2472	-
1.7040	51840	0.2476	-
1.7082	51968	0.2469	-
1.7124	52096	0.2483	-
1.7166	52224	0.2473	-
1.7208	52352	0.2459	-
1.7250	52480	0.2469	-
1.7292	52608	0.2448	-
1.7334	52736	0.2471	-
1.7376	52864	0.2440	-
1.7418	52992	0.2459	-
1.7460	53120	0.2470	-
1.7503	53248	0.2454	-
1.7545	53376	0.2474	-
1.7587	53504	0.2437	-
1.7629	53632	0.2458	-
1.7671	53760	0.2448	-
1.7713	53888	0.2451	-
1.7755	54016	0.2446	-
1.7797	54144	0.2449	-
1.7839	54272	0.2446	-
1.7881	54400	0.2445	-
1.7923	54528	0.2461	-
1.7965	54656	0.2438	-
1.8007	54784	0.2449	-
1.8050	54912	0.2444	-
1.8092	55040	0.2437	-
1.8134	55168	0.2411	-
1.8176	55296	0.2445	-
1.8218	55424	0.2448	-
1.8260	55552	0.2439	-
1.8302	55680	0.2445	-
1.8344	55808	0.2413	-
1.8386	55936	0.2432	-
1.8428	56064	0.2443	-
1.8470	56192	0.2420	-
1.8512	56320	0.2430	-
1.8554	56448	0.2425	-
1.8596	56576	0.2437	-
1.8639	56704	0.2419	-
1.8681	56832	0.2434	-
1.8723	56960	0.2424	-
1.8765	57088	0.2441	-
1.8807	57216	0.2449	-
1.8849	57344	0.2442	-
1.8891	57472	0.2418	-
1.8933	57600	0.2422	-
1.8975	57728	0.2441	-
1.9017	57856	0.2418	-
1.9059	57984	0.2442	-
1.9101	58112	0.2431	-
1.9143	58240	0.2408	-
1.9185	58368	0.2418	-
1.9228	58496	0.2437	-
1.9270	58624	0.2411	-
1.9312	58752	0.2416	-
1.9354	58880	0.2423	-
1.9396	59008	0.2419	-
1.9438	59136	0.2402	-
1.9480	59264	0.2402	-
1.9522	59392	0.2405	-
1.9564	59520	0.2400	-
1.9606	59648	0.2398	-
1.9648	59776	0.2427	-
1.9690	59904	0.2410	-
1.9732	60032	0.2422	-
1.9775	60160	0.2420	-
1.9817	60288	0.2410	-
1.9859	60416	0.2413	-
1.9901	60544	0.2413	-
1.9943	60672	0.2404	-
1.9985	60800	0.2375	-
2.0027	60928	0.2384	-
2.0069	61056	0.2369	-
2.0111	61184	0.2392	-
2.0153	61312	0.2377	-
2.0195	61440	0.2378	-
2.0237	61568	0.2369	-
2.0279	61696	0.2363	-
2.0321	61824	0.2374	-
2.0364	61952	0.2358	-
2.0406	62080	0.2399	-
2.0448	62208	0.2374	-
2.0490	62336	0.2380	-
2.0532	62464	0.2383	-
2.0574	62592	0.2381	-
2.0616	62720	0.2382	-
2.0658	62848	0.2366	-
2.0700	62976	0.2368	-
2.0742	63104	0.2361	-
2.0784	63232	0.2353	-
2.0826	63360	0.2373	-
2.0868	63488	0.2361	-
2.0910	63616	0.2384	-
2.0953	63744	0.2334	-
2.0995	63872	0.2360	-
2.1037	64000	0.2358	-
2.1079	64128	0.2368	-
2.1121	64256	0.2361	-
2.1163	64384	0.2362	-
2.1205	64512	0.2360	-
2.1247	64640	0.2360	-
2.1289	64768	0.2365	-
2.1331	64896	0.2347	-
2.1373	65024	0.2352	-
2.1415	65152	0.2366	-
2.1457	65280	0.2342	-
2.1500	65408	0.2369	-
2.1542	65536	0.2357	-
2.1584	65664	0.2344	-
2.1626	65792	0.2354	-
2.1668	65920	0.2365	-
2.1710	66048	0.2358	-
2.1752	66176	0.2367	-
2.1794	66304	0.2345	-
2.1836	66432	0.2348	-
2.1878	66560	0.2356	-
2.1920	66688	0.2353	-
2.1962	66816	0.2366	-
2.2004	66944	0.2356	-
2.2046	67072	0.2357	-
2.2089	67200	0.2359	-
2.2131	67328	0.2337	-
2.2173	67456	0.2328	-
2.2215	67584	0.2341	-
2.2257	67712	0.2360	-
2.2299	67840	0.2354	-
2.2341	67968	0.2349	-
2.2383	68096	0.2331	-
2.2425	68224	0.2346	-
2.2467	68352	0.2349	-
2.2509	68480	0.2355	-
2.2551	68608	0.2340	-
2.2593	68736	0.2355	-
2.2636	68864	0.2354	-
2.2678	68992	0.2343	-
2.2720	69120	0.2346	-
2.2762	69248	0.2336	-
2.2804	69376	0.2348	-
2.2846	69504	0.2321	-
2.2888	69632	0.2337	-
2.2930	69760	0.2342	-
2.2972	69888	0.2340	-
2.3014	70016	0.2352	-
2.3056	70144	0.2354	-
2.3098	70272	0.2339	-
2.3140	70400	0.2351	-
2.3182	70528	0.2328	-
2.3225	70656	0.2342	-
2.3267	70784	0.2327	-
2.3309	70912	0.2332	-
2.3351	71040	0.2350	-
2.3393	71168	0.2351	-
2.3435	71296	0.2343	-
2.3477	71424	0.2331	-
2.3519	71552	0.2351	-
2.3561	71680	0.2317	-
2.3603	71808	0.2320	-
2.3645	71936	0.2327	-
2.3687	72064	0.2341	-
2.3729	72192	0.2323	-
2.3771	72320	0.2338	-
2.3814	72448	0.2327	-
2.3856	72576	0.2331	-
2.3898	72704	0.2344	-
2.3940	72832	0.2336	-
2.3982	72960	0.2321	-
2.4024	73088	0.2338	-
2.4066	73216	0.2336	-
2.4108	73344	0.2319	-
2.4150	73472	0.2328	-
2.4192	73600	0.2316	-
2.4234	73728	0.2331	-
2.4276	73856	0.2339	-
2.4318	73984	0.2322	-
2.4361	74112	0.2318	-
2.4403	74240	0.2317	-
2.4445	74368	0.2339	-
2.4487	74496	0.2339	-
2.4529	74624	0.2336	-
2.4571	74752	0.2337	-
2.4613	74880	0.2341	-
2.4652	75000	-	0.4930
2.4655	75008	0.2328	-
2.4697	75136	0.2326	-
2.4739	75264	0.2340	-
2.4781	75392	0.2328	-
2.4823	75520	0.2314	-
2.4865	75648	0.2322	-
2.4907	75776	0.2314	-
2.4950	75904	0.2319	-
2.4992	76032	0.2323	-
2.5034	76160	0.2336	-
2.5076	76288	0.2300	-
2.5118	76416	0.2313	-
2.5160	76544	0.2324	-
2.5202	76672	0.2300	-
2.5244	76800	0.2321	-
2.5286	76928	0.2317	-
2.5328	77056	0.2325	-
2.5370	77184	0.2319	-
2.5412	77312	0.2323	-
2.5454	77440	0.2318	-
2.5496	77568	0.2329	-
2.5539	77696	0.2322	-
2.5581	77824	0.2321	-
2.5623	77952	0.2334	-
2.5665	78080	0.2308	-
2.5707	78208	0.2316	-
2.5749	78336	0.2304	-
2.5791	78464	0.2315	-
2.5833	78592	0.2308	-
2.5875	78720	0.2306	-
2.5917	78848	0.2328	-
2.5959	78976	0.2321	-
2.6001	79104	0.2311	-
2.6043	79232	0.2302	-
2.6086	79360	0.2318	-
2.6128	79488	0.2308	-
2.6170	79616	0.2326	-
2.6212	79744	0.2302	-
2.6254	79872	0.2307	-
2.6296	80000	0.2304	-
2.6338	80128	0.2315	-
2.6380	80256	0.2312	-
2.6422	80384	0.2321	-
2.6464	80512	0.2301	-
2.6506	80640	0.2284	-
2.6548	80768	0.2298	-
2.6590	80896	0.2318	-
2.6632	81024	0.2296	-
2.6675	81152	0.2285	-
2.6717	81280	0.2297	-
2.6759	81408	0.2308	-
2.6801	81536	0.2287	-
2.6843	81664	0.2308	-
2.6885	81792	0.2314	-
2.6927	81920	0.2302	-
2.6969	82048	0.2306	-
2.7011	82176	0.2269	-
2.7053	82304	0.2320	-
2.7095	82432	0.2283	-
2.7137	82560	0.2306	-
2.7179	82688	0.2292	-
2.7222	82816	0.2306	-
2.7264	82944	0.2299	-
2.7306	83072	0.2290	-
2.7348	83200	0.2310	-
2.7390	83328	0.2295	-
2.7432	83456	0.2305	-
2.7474	83584	0.2299	-
2.7516	83712	0.2299	-
2.7558	83840	0.2304	-
2.7600	83968	0.2297	-
2.7642	84096	0.2283	-
2.7684	84224	0.2314	-
2.7726	84352	0.2290	-
2.7768	84480	0.2278	-
2.7811	84608	0.2291	-
2.7853	84736	0.2297	-
2.7895	84864	0.2308	-
2.7937	84992	0.2309	-
2.7979	85120	0.2279	-
2.8021	85248	0.2302	-
2.8063	85376	0.2307	-
2.8105	85504	0.2297	-
2.8147	85632	0.2292	-
2.8189	85760	0.2295	-
2.8231	85888	0.2280	-
2.8273	86016	0.2309	-
2.8315	86144	0.2285	-
2.8357	86272	0.2284	-
2.8400	86400	0.2292	-
2.8442	86528	0.2300	-
2.8484	86656	0.2300	-
2.8526	86784	0.2292	-
2.8568	86912	0.2296	-
2.8610	87040	0.2284	-
2.8652	87168	0.2304	-
2.8694	87296	0.2302	-
2.8736	87424	0.2281	-
2.8778	87552	0.2286	-
2.8820	87680	0.2290	-
2.8862	87808	0.2297	-
2.8904	87936	0.2285	-
2.8947	88064	0.2287	-
2.8989	88192	0.2285	-
2.9031	88320	0.2293	-
2.9073	88448	0.2292	-
2.9115	88576	0.2274	-
2.9157	88704	0.2289	-
2.9199	88832	0.2307	-
2.9241	88960	0.2275	-
2.9283	89088	0.2293	-
2.9325	89216	0.2293	-
2.9367	89344	0.2283	-
2.9409	89472	0.2290	-
2.9451	89600	0.2286	-
2.9493	89728	0.2301	-
2.9536	89856	0.2292	-
2.9578	89984	0.2259	-
2.9620	90112	0.2286	-
2.9662	90240	0.2293	-
2.9704	90368	0.2312	-
2.9746	90496	0.2294	-
2.9788	90624	0.2280	-
2.9830	90752	0.2290	-
2.9872	90880	0.2283	-
2.9914	91008	0.2294	-
2.9956	91136	0.2301	-
2.9998	91264	0.2293	-
3.0	91269	-	0.5032
-1	-1	-	0.5032

• The bold row denotes the saved checkpoint.

Training Time

• Training: 37.8 minutes
• Evaluation: 12.9 seconds
• Total: 38.0 minutes

Framework Versions

• Python: 3.11.12
• Sentence Transformers: 5.6.1
• Transformers: 5.14.1
• PyTorch: 2.13.0+cu130
• Accelerate: 1.14.0
• Datasets: 5.0.1
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

EmbedDistillLoss

@article{kim2023embeddistill,
    title={EmbedDistill: A Geometric Knowledge Distillation for Information Retrieval},
    author={Kim, Seungyeon and Rawat, Ankit Singh and Zaheer, Manzil and Jayasumana, Sadeep and Sadhanala, Veeranjaneyulu and Jitkrittum, Wittawat and Menon, Aditya Krishna and Fergus, Rob and Kumar, Sanjiv},
    year={2023},
    eprint={2301.12005},
    archivePrefix={arXiv},
    primaryClass={cs.IR}
}