officialthinkingstack/Sentence-Matcher-v0.5

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'A 45-year-old male patient presented to the emergency department with a chief complaint of severe lower back pain.\nHe reports that the pain started suddenly about two hours ago while lifting a heavy object.\nThe pain is located in the lower back, radiates down the right leg, and is described as sharp and stabbing.\nHe also reports numbness and tingling in the right leg.\nMusculoskeletal: Severe lower back pain radiating down the right leg, numbness and tingling in the right leg\nMusculoskeletal: Limited range of motion in the lumbar spine due to pain.\nMuscle guarding is present in the paraspinal muscles.\nMotor strength and sensation are decreased in the right leg compared to the left.\nSuspected lumbar spine herniated disc\nX-rays of the lumbar spine to confirm the diagnosis\nMRI of the lumbar spine if X-rays are inconclusive\nNeurological consultation for further evaluation and management\nPain management with medication and physical therapy',
    'Other Intervertebral Disc Displacement, Lumbar Region',
    'Postconcussional Syndrome',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 1,554 training samples
• Columns: anchor, positive, and label
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	label
  type	string	string	float
  details	min: 3 tokens mean: 83.3 tokens max: 256 tokens	min: 3 tokens mean: 10.12 tokens max: 40 tokens	min: 1.0 mean: 1.0 max: 1.0

• Samples:

  anchor	positive	label
  Worsening sciatic pain despite conservative treatment She reports that her sciatic pain has been worsening despite conservative treatment with rest, ice, over-the-counter pain relievers, and lumbar spine exercises. She says that the pain is now radiating down her entire leg, from her lower back to her foot. She also reports that she has numbness and tingling in her toes. The pain is worse when she sits or stands for long periods of time. Musculoskeletal: Worsening sciatic pain radiating down the entire leg Neurological: Numbness and tingling in the toes of the affected leg Exacerbation of sciatica	Sciatica, Unspecified Side	1.0
  Chief Complaint: Malaria Patient presents with fever, chills, and headache. Reports a sudden onset of symptoms after returning from a recent trip to a malaria-endemic region. Positive for fever and chills. Recent travel to a malaria-endemic area. Fever (38.5°C), chills, and generalized body aches. High suspicion for malaria infection. Initiate prompt diagnostic testing and antimalarial treatment. Blood smear for malaria parasites. Prescribe antimalarial medications based on species identification. Consideration of hospitalization if severe symptoms or complications arise. Follow-up in two weeks for reassessment and treatment response. Blood smear for Plasmodium species identification. Complete blood count (CBC) for assessment of anemia and thrombocytopenia. Liver function tests to assess for malaria-related complications.	Unspecified Malaria	1.0
  The patient returns for a follow-up visit after being diagnosed with psychosis. He reports partial improvement in symptoms with medication but notes persistent paranoia and difficulty concentrating. Partial improvement in sleep but persistent paranoia. Some reduction in auditory hallucinations, persistent racing thoughts. Ongoing diagnosis of psychosis and mood disorder. Observable signs of residual psychosis and disorganized thoughts. Partial improvement in psychosis. Adjust medication dosage or consider alternative medications. Continue individual therapy for ongoing management. Monitor for side effects and therapeutic response. Attend scheduled therapy sessions. Take prescribed medications as directed. Report any new or worsening symptoms promptly.	Psychotic Disorder With Delusions Due To Known Physiological Condition	1.0

• Loss: ContrastiveLoss with these parameters:

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 5
• warmup_ratio: 0.1

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss
0.0102	1	0.1624
0.0204	2	0.1369
0.0306	3	0.1151
0.0408	4	0.1031
0.0510	5	0.097
0.0612	6	0.095
0.0714	7	0.1236
0.0816	8	0.1177
0.0918	9	0.0931
0.1020	10	0.1049
0.1122	11	0.0757
0.1224	12	0.0936
0.1327	13	0.0797
0.1429	14	0.0855
0.1531	15	0.079
0.1633	16	0.0666
0.1735	17	0.073
0.1837	18	0.0669
0.1939	19	0.0517
0.2041	20	0.0667
0.2143	21	0.0639
0.2245	22	0.0729
0.2347	23	0.0565
0.2449	24	0.0501
0.2551	25	0.0596
0.2653	26	0.0478
0.2755	27	0.0306
0.2857	28	0.0509
0.2959	29	0.0415
0.3061	30	0.0396
0.3163	31	0.0215
0.3265	32	0.0402
0.3367	33	0.0692
0.3469	34	0.0602
0.3571	35	0.0215
0.3673	36	0.0274
0.3776	37	0.0212
0.3878	38	0.0231
0.3980	39	0.0159
0.4082	40	0.0154
0.4184	41	0.013
0.4286	42	0.0144
0.4388	43	0.0353
0.4490	44	0.0169
0.4592	45	0.0055
0.4694	46	0.0098
0.4796	47	0.0071
0.4898	48	0.0167
0.5	49	0.0062
0.5102	50	0.0064
0.5204	51	0.0125
0.5306	52	0.0044
0.5408	53	0.0193
0.5510	54	0.0058
0.5612	55	0.0043
0.5714	56	0.0036
0.5816	57	0.0018
0.5918	58	0.0039
0.6020	59	0.0031
0.6122	60	0.0019
0.6224	61	0.003
0.6327	62	0.003
0.6429	63	0.0039
0.6531	64	0.0048
0.6633	65	0.0013
0.6735	66	0.0039
0.6837	67	0.0113
0.6939	68	0.0042
0.7041	69	0.0029
0.7143	70	0.0014
0.7245	71	0.0012
0.7347	72	0.001
0.7449	73	0.0128
0.7551	74	0.0076
0.7653	75	0.0031
0.7755	76	0.0012
0.7857	77	0.0014
0.7959	78	0.0015
0.8061	79	0.0017
0.8163	80	0.0014
0.8265	81	0.0015
0.8367	82	0.0013
0.8469	83	0.001
0.8571	84	0.0021
0.8673	85	0.0008
0.8776	86	0.0009
0.8878	87	0.0117
0.8980	88	0.003
0.9082	89	0.0008
0.9184	90	0.0068
0.9286	91	0.0014
0.9388	92	0.0014
0.9490	93	0.0007
0.9592	94	0.0011
0.9694	95	0.0009
0.9796	96	0.0008
0.9898	97	0.0011
1.0	98	0.0011
1.0102	99	0.0005
1.0204	100	0.0005
1.0306	101	0.0012
1.0408	102	0.0008
1.0510	103	0.0016
1.0612	104	0.0005
1.0714	105	0.0015
1.0816	106	0.0005
1.0918	107	0.0018
1.1020	108	0.0006
1.1122	109	0.0006
1.1224	110	0.0043
1.1327	111	0.0007
1.1429	112	0.0009
1.1531	113	0.0007
1.1633	114	0.0019
1.1735	115	0.0032
1.1837	116	0.0004
1.1939	117	0.0005
1.2041	118	0.0005
1.2143	119	0.0009
1.2245	120	0.0018
1.2347	121	0.0006
1.2449	122	0.0004
1.2551	123	0.0004
1.2653	124	0.0008
1.2755	125	0.0007
1.2857	126	0.0006
1.2959	127	0.0004
1.3061	128	0.0032
1.3163	129	0.0011
1.3265	130	0.0008
1.3367	131	0.0006
1.3469	132	0.0004
1.3571	133	0.0005
1.3673	134	0.0003
1.3776	135	0.0006
1.3878	136	0.0009
1.3980	137	0.0003
1.4082	138	0.0003
1.4184	139	0.0005
1.4286	140	0.0005
1.4388	141	0.0005
1.4490	142	0.0006
1.4592	143	0.0022
1.4694	144	0.0004
1.4796	145	0.0012
1.4898	146	0.0006
1.5	147	0.0003
1.5102	148	0.0008
1.5204	149	0.0004
1.5306	150	0.0004
1.5408	151	0.0004
1.5510	152	0.0004
1.5612	153	0.0007
1.5714	154	0.0022
1.5816	155	0.0005
1.5918	156	0.0003
1.6020	157	0.0005
1.6122	158	0.0003
1.6224	159	0.0004
1.6327	160	0.0004
1.6429	161	0.0002
1.6531	162	0.0005
1.6633	163	0.0005
1.6735	164	0.0003
1.6837	165	0.0005
1.6939	166	0.0005
1.7041	167	0.0004
1.7143	168	0.0003
1.7245	169	0.0003
1.7347	170	0.0003
1.7449	171	0.0005
1.7551	172	0.0005
1.7653	173	0.0002
1.7755	174	0.0005
1.7857	175	0.0003
1.7959	176	0.0006
1.8061	177	0.0003
1.8163	178	0.0004
1.8265	179	0.0004
1.8367	180	0.0002
1.8469	181	0.0002
1.8571	182	0.0005
1.8673	183	0.0003
1.8776	184	0.0003
1.8878	185	0.0002
1.8980	186	0.0003
1.9082	187	0.0032
1.9184	188	0.0006
1.9286	189	0.0003
1.9388	190	0.0005
1.9490	191	0.0003
1.9592	192	0.0004
1.9694	193	0.0004
1.9796	194	0.0004
1.9898	195	0.0003
2.0	196	0.0001
2.0102	197	0.0003
2.0204	198	0.0003
2.0306	199	0.0002
2.0408	200	0.0002
2.0510	201	0.0003
2.0612	202	0.0002
2.0714	203	0.0002
2.0816	204	0.0003
2.0918	205	0.0003
2.1020	206	0.0008
2.1122	207	0.0004
2.1224	208	0.0004
2.1327	209	0.0004
2.1429	210	0.0003
2.1531	211	0.0004
2.1633	212	0.0002
2.1735	213	0.0002
2.1837	214	0.0002
2.1939	215	0.0002
2.2041	216	0.0002
2.2143	217	0.0003
2.2245	218	0.0004
2.2347	219	0.0003
2.2449	220	0.0002
2.2551	221	0.0002
2.2653	222	0.0003
2.2755	223	0.0002
2.2857	224	0.0003
2.2959	225	0.0002
2.3061	226	0.0003
2.3163	227	0.0002
2.3265	228	0.0004
2.3367	229	0.0002
2.3469	230	0.0002
2.3571	231	0.001
2.3673	232	0.0002
2.3776	233	0.0006
2.3878	234	0.0003
2.3980	235	0.0003
2.4082	236	0.0005
2.4184	237	0.0004
2.4286	238	0.0011
2.4388	239	0.0009
2.4490	240	0.0004
2.4592	241	0.0003
2.4694	242	0.0003
2.4796	243	0.0002
2.4898	244	0.0004
2.5	245	0.0002
2.5102	246	0.0002
2.5204	247	0.0004
2.5306	248	0.0003
2.5408	249	0.0002
2.5510	250	0.0006
2.5612	251	0.0002
2.5714	252	0.0002
2.5816	253	0.0002
2.5918	254	0.0002
2.6020	255	0.0013
2.6122	256	0.0002
2.6224	257	0.0012
2.6327	258	0.0003
2.6429	259	0.0002
2.6531	260	0.0003
2.6633	261	0.0002
2.6735	262	0.0011
2.6837	263	0.0003
2.6939	264	0.0003
2.7041	265	0.0004
2.7143	266	0.0003
2.7245	267	0.0001
2.7347	268	0.0002
2.7449	269	0.0002
2.7551	270	0.0003
2.7653	271	0.0002
2.7755	272	0.0002
2.7857	273	0.0002
2.7959	274	0.0004
2.8061	275	0.0002
2.8163	276	0.0003
2.8265	277	0.0002
2.8367	278	0.0002
2.8469	279	0.0004
2.8571	280	0.0004
2.8673	281	0.0004
2.8776	282	0.0002
2.8878	283	0.0002
2.8980	284	0.0004
2.9082	285	0.0002
2.9184	286	0.0002
2.9286	287	0.0004
2.9388	288	0.0003
2.9490	289	0.0002
2.9592	290	0.0006
2.9694	291	0.0002
2.9796	292	0.0003
2.9898	293	0.0003
3.0	294	0.0002
3.0102	295	0.0002
3.0204	296	0.0001
3.0306	297	0.0002
3.0408	298	0.0005
3.0510	299	0.0004
3.0612	300	0.0005
3.0714	301	0.0002
3.0816	302	0.0002
3.0918	303	0.0002
3.1020	304	0.0004
3.1122	305	0.0002
3.1224	306	0.0002
3.1327	307	0.0002
3.1429	308	0.0002
3.1531	309	0.0003
3.1633	310	0.0003
3.1735	311	0.0002
3.1837	312	0.0004
3.1939	313	0.0002
3.2041	314	0.0001
3.2143	315	0.0002
3.2245	316	0.0004
3.2347	317	0.0004
3.2449	318	0.0003
3.2551	319	0.0002
3.2653	320	0.0002
3.2755	321	0.0002
3.2857	322	0.0003
3.2959	323	0.0003
3.3061	324	0.0003
3.3163	325	0.0002
3.3265	326	0.0002
3.3367	327	0.0001
3.3469	328	0.0002
3.3571	329	0.0004
3.3673	330	0.0002
3.3776	331	0.0002
3.3878	332	0.0002
3.3980	333	0.0001
3.4082	334	0.0002
3.4184	335	0.0002
3.4286	336	0.0001
3.4388	337	0.0005
3.4490	338	0.0001
3.4592	339	0.0003
3.4694	340	0.0003
3.4796	341	0.0002
3.4898	342	0.0002
3.5	343	0.0001
3.5102	344	0.0002
3.5204	345	0.0008
3.5306	346	0.0002
3.5408	347	0.0003
3.5510	348	0.0003
3.5612	349	0.0003
3.5714	350	0.0002
3.5816	351	0.0002
3.5918	352	0.0002
3.6020	353	0.0001
3.6122	354	0.0002
3.6224	355	0.0001
3.6327	356	0.0002
3.6429	357	0.0001
3.6531	358	0.0001
3.6633	359	0.0003
3.6735	360	0.0003
3.6837	361	0.0002
3.6939	362	0.0002
3.7041	363	0.0001
3.7143	364	0.0003
3.7245	365	0.0003
3.7347	366	0.0002
3.7449	367	0.0006
3.7551	368	0.0003
3.7653	369	0.0002
3.7755	370	0.0002
3.7857	371	0.0001
3.7959	372	0.0002
3.8061	373	0.0002
3.8163	374	0.0003
3.8265	375	0.0001
3.8367	376	0.0002
3.8469	377	0.0004
3.8571	378	0.0002
3.8673	379	0.0003
3.8776	380	0.0001
3.8878	381	0.0003
3.8980	382	0.0001
3.9082	383	0.0002
3.9184	384	0.0002
3.9286	385	0.0002
3.9388	386	0.0003
3.9490	387	0.0002
3.9592	388	0.0002
3.9694	389	0.0001
3.9796	390	0.0002
3.9898	391	0.0001
4.0	392	0.0001
4.0102	393	0.0001
4.0204	394	0.0002
4.0306	395	0.0001
4.0408	396	0.0007
4.0510	397	0.0002
4.0612	398	0.0002
4.0714	399	0.0001
4.0816	400	0.0001
4.0918	401	0.0002
4.1020	402	0.0002
4.1122	403	0.0001
4.1224	404	0.0001
4.1327	405	0.0002
4.1429	406	0.0004
4.1531	407	0.0004
4.1633	408	0.0006
4.1735	409	0.0001
4.1837	410	0.0002
4.1939	411	0.0002
4.2041	412	0.0001
4.2143	413	0.0001
4.2245	414	0.0001
4.2347	415	0.0001
4.2449	416	0.0003
4.2551	417	0.0001
4.2653	418	0.0002
4.2755	419	0.0001
4.2857	420	0.0002
4.2959	421	0.0003
4.3061	422	0.0004
4.3163	423	0.0002
4.3265	424	0.0003
4.3367	425	0.0001
4.3469	426	0.0001
4.3571	427	0.0002
4.3673	428	0.0002
4.3776	429	0.0002
4.3878	430	0.0002
4.3980	431	0.0002
4.4082	432	0.0001
4.4184	433	0.0003
4.4286	434	0.0002
4.4388	435	0.0003
4.4490	436	0.0003
4.4592	437	0.0003
4.4694	438	0.0001
4.4796	439	0.0002
4.4898	440	0.0002
4.5	441	0.0002
4.5102	442	0.0003
4.5204	443	0.0003
4.5306	444	0.0002
4.5408	445	0.0002
4.5510	446	0.0001
4.5612	447	0.0002
4.5714	448	0.0002
4.5816	449	0.0001
4.5918	450	0.0002
4.6020	451	0.0002
4.6122	452	0.0001
4.6224	453	0.0003
4.6327	454	0.0002
4.6429	455	0.0001
4.6531	456	0.0001
4.6633	457	0.0001
4.6735	458	0.0001
4.6837	459	0.0002
4.6939	460	0.0001
4.7041	461	0.0002
4.7143	462	0.0001
4.7245	463	0.0001
4.7347	464	0.0002
4.7449	465	0.0002
4.7551	466	0.0001
4.7653	467	0.0002
4.7755	468	0.0002
4.7857	469	0.0002
4.7959	470	0.0002
4.8061	471	0.0007
4.8163	472	0.0002
4.8265	473	0.0006
4.8367	474	0.0002
4.8469	475	0.0001
4.8571	476	0.0002
4.8673	477	0.0001
4.8776	478	0.0002
4.8878	479	0.0002
4.8980	480	0.0003
4.9082	481	0.0002
4.9184	482	0.0001
4.9286	483	0.0002
4.9388	484	0.0002
4.9490	485	0.0002
4.9592	486	0.0002
4.9694	487	0.0002
4.9796	488	0.0002
4.9898	489	0.0004
5.0	490	0.0002

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 4.1.0
• Transformers: 4.52.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.7.0
• Datasets: 2.14.4
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

ContrastiveLoss

@inproceedings{hadsell2006dimensionality,
    author={Hadsell, R. and Chopra, S. and LeCun, Y.},
    booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
    title={Dimensionality Reduction by Learning an Invariant Mapping},
    year={2006},
    volume={2},
    number={},
    pages={1735-1742},
    doi={10.1109/CVPR.2006.100}
}