ChenyuEcho/fine_tuned_model

SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-Embedding-0.6B
• Maximum Sequence Length: 768 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 768, 'do_lower_case': False, 'architecture': 'PeftModelForFeatureExtraction'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "What bottlenecks in the updated post-operative workflow are contributing to delays in surgical site infection specimen transfer and tracking?",
]
documents = [
    'Subject: Concerns Regarding Timeliness of Surgical Site Infection Tracking\nFrom: Xavier D. Brooks\nTo: David S. Wilson\nDate: 2025-11-10\n\nHi David,\n\nThank you for raising these concerns about the delays in surgical site infection tracking. We have indeed adjusted some aspects of our post-op patient flow in an attempt to enhance discharge efficiency, including new documentation checkpoints that might inadvertently be slowing the specimen transfer process. I’ll coordinate with our nursing and records teams to closely review recent workflow changes and identify any bottlenecks that could be contributing to extended turnaround times. I’ll share our findings and propose potential improvements by the end of this week, and I welcome any further details you notice from the lab side as well.\n\nBest regards,\nXavier',
    'Subject: Concern Regarding Allergy Documentation Accuracy and Glucose Meter Integration\nFrom: Daniel M. Evans\nTo: Gabriella I. Santos\nDate: 2026-01-26\n\nHi Gabriella,\n\nI wanted to bring to your attention a recurring issue we’ve noticed with our glucose meters not consistently syncing updated allergy information from the patient chart. During routine maintenance, I found discrepancies between recorded allergies on the device and what is documented in the EMR, which could lead to potential risks for patients with sensitivities, especially regarding test strip ingredients. I propose we review the current integration workflow and possibly schedule a troubleshooting session with IT to ensure seamless allergy data transfer. Please let me know if you’ve experienced similar concerns and if you’d be available to discuss this further.\n\nThanks,\nDaniel',
    'Subject: Inquiry Regarding Post-Operativ Care Documentation\nFrom: David R. Park\nTo: Inspector Helen R. Jacobs\nDate: 2026-01-26\n\nHello Inspector Jacobs,\n\nI am reaching out regarding the ongoing investigation tied to Mr. Hendricks’ recent case. We have been reviewing the patient records and noticed that the documentation for the post-operativ care period contains several ambiguities. We would appreciate your guidance on whether additional clarification or supplementary notes are required for compliance purposes. Please let me know how you would like us to proceed, or if you need copies of the relevant chart sections.\n\nBest regards,\nDavid R. Park',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.7188, 0.1221, 0.0596]], dtype=torch.bfloat16)

Evaluation

Metrics

Information Retrieval

• Dataset: val_real_corpus_thread_ir
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7613
cosine_accuracy@3	0.8297
cosine_accuracy@5	0.8614
cosine_accuracy@10	0.8948
cosine_precision@1	0.7613
cosine_precision@3	0.5275
cosine_precision@5	0.3349
cosine_precision@10	0.179
cosine_recall@1	0.3665
cosine_recall@3	0.7013
cosine_recall@5	0.7391
cosine_recall@10	0.7844
cosine_ndcg@10	0.752
cosine_mrr@10	0.8039
cosine_map@100	0.7154

Training Details

Training Dataset

Unnamed Dataset

• Size: 2,392 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 11 tokens mean: 26.85 tokens max: 62 tokens	min: 99 tokens mean: 159.15 tokens max: 364 tokens

• Samples:

  sentence_0	sentence_1
  What specific documents and timeline details are being requested for the medication administration incident involving the late husband (e.g., notes and observed discrepancy times)?	Subject: Clarification Needed Regarding Recent Medciation Administration Incident From: David R. Park To: Margaret L. Hendricks Date: 2025-10-16 Hello Mrs. Hendricks, Thank you for your prompt reply and for clarifying your experience regarding the medication administration incident involving your late husband. I acknowledge your willingness to provide further details and want to ensure that our review is thorough and respectful of your family's concerns. A call on Wednesday afternoon works for me, and I appreciate your flexibility in offering to share information by email. If you have any documentation, such as notes or times you observed discrepancies, that would be very helpful for our review. Please let me know your preferred time for the call, or if you wish to send information in writing, I am happy to review it carefully. Thank you again for your cooperation as we work to address these important concerns. I look forward to speaking with you and assisting however I can. Best r...
  What specific additional materials or documentation should my team prepare ahead of the meeting?	Subject: Re: Meeting Confirmation and Case Materials From: David R. Park To: Katherine E. Morrison Date: 2025-12-01 Hi Katherine, Thank you for confirming the meeting time and sharing the agenda. I appreciate your prompt coordination on this. Please let me know if there are any additional materials or documentation you would like from my team ahead of our discussion. I look forward to collaborating and ensuring all questions are addressed at our meeting. Best regards, David
  Who is assigned to coordinate the review of PACS-EHR interface error logs with radiology IT to address radiology report delays?	Subject: Radiology Report Turnaround Delays in EHR From: Angela R. Scott To: Laura A. Hughes Date: 2025-11-17 Hi Laura, I've noticed a consistent delay in radiology report turnaround times stemming from integration issues between the PACS interface and our EHR system. Reports are not always populating promptly in patient records, which is affecting timely communication with both care teams and patients. I suggest we collaborate with the radiology IT staff to review interface error logs and streamline the auto-notification features. If you have additional insight from recent patient feedback or workflow observations, please let me know so we can address this comprehensively. Thanks, Angela --- This email and any attachments are confidential and intended solely for the use of the individual or entity to whom they are addressed.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	val_real_corpus_thread_ir_cosine_ndcg@10
1.0	299	-	0.7464
1.6722	500	0.0176	-
2.0	598	-	0.7507
3.0	897	-	0.7520

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.2
• Transformers: 5.0.0
• PyTorch: 2.9.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}