Sentence Similarity
sentence-transformers
Safetensors
feature-extraction
dense
Generated from Trainer
dataset_size:2392
loss:MultipleNegativesRankingLoss
Eval Results (legacy)
Instructions to use ChenyuEcho/fine_tuned_model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use ChenyuEcho/fine_tuned_model with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("ChenyuEcho/fine_tuned_model") sentences = [ "What are the exact start and end times for overnight on-site IT coverage during the maintenance window?", "Subject: Issue Encountered with Insurance Verification Workflow\nFrom: Julian R. Torres\nTo: Rachel K. Martinez\nDate: 2025-10-20\n\nHi Rachel,\n\nI wanted to flag an ongoing issue with the insurance verification process that’s impacting our ED admissions, especially during peak hours. Sometimes, patient insurance details aren’t fully updated in the system, and it’s causing delays getting clearance from registration. Could we discuss ways to streamline the info handoff between the ED and registration, or is there a protocol update I might’ve missed? Any suggestions or insight from your end would be appreciated.\n\nThanks,\nJulian", "Subject: EHR Medication Documentation Concerns – Joint Commission Survey Preparation\nFrom: Katherine M. Walsh\nTo: Angela R. Scott\nDate: 2025-10-20\n\nHello Angela,\n\nAs we continue our preparations for the upcoming Joint Commission survey, I have identified a recurring issue with the EHR medication documentation process. Specifically, the current workflow does not require entry of medication batch numbers or precise dose changes during intraoperative adjustments, which is inconsistent with recent Joint Commission medication safety protocols. This gap could potentially lead to survey citations and, more importantly, compromises our ability to track medication safety accurately. Could you assist in reviewing and, if possible, updating the EHR fields so that batch numbers and intraoperative dose modifications are mandatory entries? If you need additional clinical detail, I am happy to collaborate.\n\nThank you for your attention to this patient safety concern.\n\nBest regards,\nKatherine", "Subject: Re: Scheduled System Maintenance Downtime – Main Hospital & Outpatient Clinics\nFrom: Richard T. Howard\nTo: David R. Park\nDate: 2025-10-16\n\nHi David,\n\nThank you for your prompt reply and for raising the question about tech support coverage during the maintenance window. I can confirm that our IT team will have on-site personnel available overnight to assist with any urgent issues that arise, particularly for clinical teams. Please feel free to direct your staff to extension 4471 if immediate support is required during downtime.\n\nLet me know if you need any additional details or have further concerns.\n\nBest,\nRichard" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
| tags: | |
| - sentence-transformers | |
| - sentence-similarity | |
| - feature-extraction | |
| - dense | |
| - generated_from_trainer | |
| - dataset_size:2392 | |
| - loss:MultipleNegativesRankingLoss | |
| base_model: Qwen/Qwen3-Embedding-0.6B | |
| widget: | |
| - source_sentence: What are the exact start and end times for overnight on-site IT | |
| coverage during the maintenance window? | |
| sentences: | |
| - 'Subject: Issue Encountered with Insurance Verification Workflow | |
| From: Julian R. Torres | |
| To: Rachel K. Martinez | |
| Date: 2025-10-20 | |
| Hi Rachel, | |
| I wanted to flag an ongoing issue with the insurance verification process that’s | |
| impacting our ED admissions, especially during peak hours. Sometimes, patient | |
| insurance details aren’t fully updated in the system, and it’s causing delays | |
| getting clearance from registration. Could we discuss ways to streamline the info | |
| handoff between the ED and registration, or is there a protocol update I might’ve | |
| missed? Any suggestions or insight from your end would be appreciated. | |
| Thanks, | |
| Julian' | |
| - 'Subject: EHR Medication Documentation Concerns – Joint Commission Survey Preparation | |
| From: Katherine M. Walsh | |
| To: Angela R. Scott | |
| Date: 2025-10-20 | |
| Hello Angela, | |
| As we continue our preparations for the upcoming Joint Commission survey, I have | |
| identified a recurring issue with the EHR medication documentation process. Specifically, | |
| the current workflow does not require entry of medication batch numbers or precise | |
| dose changes during intraoperative adjustments, which is inconsistent with recent | |
| Joint Commission medication safety protocols. This gap could potentially lead | |
| to survey citations and, more importantly, compromises our ability to track medication | |
| safety accurately. Could you assist in reviewing and, if possible, updating the | |
| EHR fields so that batch numbers and intraoperative dose modifications are mandatory | |
| entries? If you need additional clinical detail, I am happy to collaborate. | |
| Thank you for your attention to this patient safety concern. | |
| Best regards, | |
| Katherine' | |
| - 'Subject: Re: Scheduled System Maintenance Downtime – Main Hospital & Outpatient | |
| Clinics | |
| From: Richard T. Howard | |
| To: David R. Park | |
| Date: 2025-10-16 | |
| Hi David, | |
| Thank you for your prompt reply and for raising the question about tech support | |
| coverage during the maintenance window. I can confirm that our IT team will have | |
| on-site personnel available overnight to assist with any urgent issues that arise, | |
| particularly for clinical teams. Please feel free to direct your staff to extension | |
| 4471 if immediate support is required during downtime. | |
| Let me know if you need any additional details or have further concerns. | |
| Best, | |
| Richard' | |
| - source_sentence: 'Wrong-site surgery incident: what are the immediate disclosure | |
| obligations and communications strategy to the patient and family, in compliance | |
| with regulatory requirements?' | |
| sentences: | |
| - 'Subject: Einladung zum Physician Appreciation Luncheon – 26. Juni, Sicherheitshinweise | |
| bitte beachten | |
| From: David R. Park | |
| To: Kevin T. Murphy | |
| Date: 2026-01-12 | |
| Sehr geehrter Herr Murphy, | |
| vielen Dank für die Einladung zum Physician Appreciation Luncheon und die klaren | |
| Hinweise zu den Sicherheitsvorkehrungen. Ich begrüße die erhöhte Aufmerksamkeit | |
| für den Datenschutz und werde darauf achten, keine dienstlichen Geräte unbeaufsichtigt | |
| zu lassen und sensible Gesprächsthemen zu vermeiden. Die Maßnahmen der IT vor | |
| Ort geben zusätzliche Sicherheit. | |
| Mit freundlichen Grüßen | |
| David R. Park' | |
| - 'Subject: Medication Reconciliation EHR Issue: Immediate Attention Required | |
| From: Christopher P. Brown | |
| To: Isaiah T. Jackson | |
| Date: 2025-12-22 | |
| Hi Isaiah, | |
| I wanted to bring to your attention a recurring issue we''ve identified with the | |
| medication reconciliation feature in our EHR system. During peak usage hours, | |
| there are noticeable delays in loading patient medication histories, which has | |
| resulted in several incomplete reconciliations and workflow disruptions for clinical | |
| staff. I suspect this may be linked to the last unplanned EHR downtime, but we''re | |
| still analyzing the root cause. Could you coordinate with nursing and pharmacy | |
| teams to document specific impact cases from the last week? This data will help | |
| us escalate the issue with our EHR vendor and develop interim protocols to mitigate | |
| patient safety risks. | |
| Let me know a good time for a short call to discuss next steps. | |
| Best, | |
| Christopher' | |
| - 'Subject: Re: Re: URGENT: Incident Report - OR3 Surgical Case | |
| From: Patricia M. Vasquez | |
| To: David R. Park | |
| Date: 2025-09-12 | |
| Dear David, | |
| I am writing to advise you of a critical adverse event that occurred earlier today | |
| in OR3 involving patient Robert Hendricks. During a scheduled arthroscopy, the | |
| procedure was performed on the wrong site (right knee rather than the consented | |
| and marked left knee). The patient is increasingly agitated and has voiced significant | |
| distress over the error; his family members are now actively seeking information | |
| and have expressed concern about the care provided. | |
| Given the gravity of this situation, I am requesting immediate legal guidance | |
| regarding our incident management strategy, disclosure obligations to the patient | |
| and family, and best practices for documentation and information retention. I | |
| have instructed all involved staff to hold documentation pending our discussion | |
| and to refrain from further written communication until protocols are clarified. | |
| Please advise on next steps, including any immediate actions we should take to | |
| ensure compliance with regulatory requirements and to protect both the patient’s | |
| rights and the hospital’s interests. | |
| Your prompt attention to this matter is greatly appreciated. Please let me know | |
| if you require any additional information or wish to convene a call tonight to | |
| discuss further. | |
| Regards, | |
| Patricia M. Vasquez, RN, MBA, CPHRM | |
| Director of Risk Management & Patient Safety' | |
| - source_sentence: What are the specific gaps between the current patient-facing grievance | |
| script and the formal grievance procedure documentation in our department, and | |
| which points are not being conveyed? | |
| sentences: | |
| - 'Subject: Request for Support: Employee Wellness Initiative Documentation | |
| From: Chloe R. Anderson | |
| To: Linda R. Taylor | |
| Date: 2026-01-13 | |
| Hi Linda, | |
| I am reaching out regarding an issue we''ve encountered with tracking participation | |
| in the new employee wellness initiative. Several staff members have reported that | |
| their completed activity forms are not reflected in our records, possibly due | |
| to delays in processing or a system error. Would you be able to help review recent | |
| submissions and confirm that all entries from the past two weeks have been logged | |
| appropriately? If you notice any discrepancies, please let me know so we can address | |
| them promptly. | |
| Thank you for your assistance. | |
| Best, | |
| Chloe' | |
| - 'Subject: Re: Sending this your way | |
| From: Angela R. Scott | |
| To: Zoe M. Campbell | |
| Date: 2025-12-09 | |
| Hi Zoe, | |
| Thank you for passing along the documents and providing the details regarding | |
| the EHR issues. I’ve started reviewing the attached error logs and, based on some | |
| initial patterns, I suspect the API middleware might be bottlenecking when processing | |
| simultaneous attachment uploads. I plan to investigate further by running diagnostics | |
| during peak usage hours and testing middleware latency. I’ll circle back with | |
| more detailed findings and some preliminary recommendations by the end of the | |
| week. If you have any additional instances or timestamps where the failures were | |
| most severe, that information would be especially helpful for my analysis. | |
| Thanks for reaching out, and I’ll keep you posted as I dig deeper. | |
| Best regards, | |
| Angela' | |
| - 'Subject: Clarification Needed on Grievance Procedure Communication | |
| From: Elizabeth M. Turner | |
| To: Jasmine K. Patel | |
| Date: 2025-10-29 | |
| Hello Jasmine, | |
| During a recent audit of staff communications, I observed some inconsistencies | |
| in how the grievance procedure is being explained to patients within our department. | |
| It appears several steps specified in the formal documentation are not being fully | |
| outlined in verbal explanations, which could lead to misunderstandings. Could | |
| you assist by reviewing the current script with me, so we can ensure all required | |
| points are conveyed accurately going forward? Please let me know a convenient | |
| time for us to meet and update our process accordingly. | |
| Thank you for your attention to this matter. | |
| Best regards, | |
| Elizabeth M. Turner' | |
| - source_sentence: ¿Cuál es el estado actual y la fecha estimada de entrega de los | |
| registros médicos y la documentación solicitada para la revisión inicial del caso? | |
| sentences: | |
| - 'Subject: Re: Solicitud de Documentación Adicional para la Investigación | |
| From: Inspector Helen R. Jacobs | |
| To: David R. Park | |
| Date: 2026-01-01 | |
| Estimado Sr. Park, | |
| Agradezco su pronta respuesta y la confirmación del envío de los registros médicos | |
| y demás documentación solicitada. Por el momento, los documentos que menciona | |
| serán suficientes para continuar con la revisión inicial del caso; si surgiera | |
| la necesidad de información adicional, me pondré en contacto de inmediato. Quedamos | |
| atentos a la recepción de los archivos a finales de semana. | |
| Cordialmente, | |
| Helen R. Jacobs' | |
| - 'Subject: Need your input | |
| From: George M. Harris | |
| To: Zoe M. Campbell | |
| Date: 2026-01-22 | |
| Hi Zoe, | |
| Thanks for looping me in. Before I can provide a full response, could you clarify | |
| which specific billing codes are in question and whether the supporting clinical | |
| documentation has already been uploaded to the compliance system? I want to ensure | |
| that any input I provide aligns with the latest guidelines and audit standards. | |
| Please provide the relevant details when you have a moment. | |
| Thanks, | |
| George' | |
| - 'Subject: Update | |
| From: David R. Park | |
| To: Katherine E. Morrison | |
| Date: 2026-01-26 | |
| Hello, | |
| As requested, I am sending the update we discussed. Please find attached a summary | |
| of the current situation, along with all pertinent details that have come to light | |
| since our last conversation. If you have any further questions or need clarification | |
| on specific points, do not hesitate to reach out. Your input will be valuable | |
| as we move forward. | |
| Looking forward to your response. | |
| Best regards, | |
| David R. Park' | |
| - source_sentence: What bottlenecks in the updated post-operative workflow are contributing | |
| to delays in surgical site infection specimen transfer and tracking? | |
| sentences: | |
| - 'Subject: Inquiry Regarding Post-Operativ Care Documentation | |
| From: David R. Park | |
| To: Inspector Helen R. Jacobs | |
| Date: 2026-01-26 | |
| Hello Inspector Jacobs, | |
| I am reaching out regarding the ongoing investigation tied to Mr. Hendricks’ recent | |
| case. We have been reviewing the patient records and noticed that the documentation | |
| for the post-operativ care period contains several ambiguities. We would appreciate | |
| your guidance on whether additional clarification or supplementary notes are required | |
| for compliance purposes. Please let me know how you would like us to proceed, | |
| or if you need copies of the relevant chart sections. | |
| Best regards, | |
| David R. Park' | |
| - 'Subject: Concern Regarding Allergy Documentation Accuracy and Glucose Meter Integration | |
| From: Daniel M. Evans | |
| To: Gabriella I. Santos | |
| Date: 2026-01-26 | |
| Hi Gabriella, | |
| I wanted to bring to your attention a recurring issue we’ve noticed with our glucose | |
| meters not consistently syncing updated allergy information from the patient chart. | |
| During routine maintenance, I found discrepancies between recorded allergies on | |
| the device and what is documented in the EMR, which could lead to potential risks | |
| for patients with sensitivities, especially regarding test strip ingredients. | |
| I propose we review the current integration workflow and possibly schedule a troubleshooting | |
| session with IT to ensure seamless allergy data transfer. Please let me know if | |
| you’ve experienced similar concerns and if you’d be available to discuss this | |
| further. | |
| Thanks, | |
| Daniel' | |
| - 'Subject: Concerns Regarding Timeliness of Surgical Site Infection Tracking | |
| From: Xavier D. Brooks | |
| To: David S. Wilson | |
| Date: 2025-11-10 | |
| Hi David, | |
| Thank you for raising these concerns about the delays in surgical site infection | |
| tracking. We have indeed adjusted some aspects of our post-op patient flow in | |
| an attempt to enhance discharge efficiency, including new documentation checkpoints | |
| that might inadvertently be slowing the specimen transfer process. I’ll coordinate | |
| with our nursing and records teams to closely review recent workflow changes and | |
| identify any bottlenecks that could be contributing to extended turnaround times. | |
| I’ll share our findings and propose potential improvements by the end of this | |
| week, and I welcome any further details you notice from the lab side as well. | |
| Best regards, | |
| Xavier' | |
| pipeline_tag: sentence-similarity | |
| library_name: sentence-transformers | |
| metrics: | |
| - cosine_accuracy@1 | |
| - cosine_accuracy@3 | |
| - cosine_accuracy@5 | |
| - cosine_accuracy@10 | |
| - cosine_precision@1 | |
| - cosine_precision@3 | |
| - cosine_precision@5 | |
| - cosine_precision@10 | |
| - cosine_recall@1 | |
| - cosine_recall@3 | |
| - cosine_recall@5 | |
| - cosine_recall@10 | |
| - cosine_ndcg@10 | |
| - cosine_mrr@10 | |
| - cosine_map@100 | |
| model-index: | |
| - name: SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B | |
| results: | |
| - task: | |
| type: information-retrieval | |
| name: Information Retrieval | |
| dataset: | |
| name: val real corpus thread ir | |
| type: val_real_corpus_thread_ir | |
| metrics: | |
| - type: cosine_accuracy@1 | |
| value: 0.7612687813021702 | |
| name: Cosine Accuracy@1 | |
| - type: cosine_accuracy@3 | |
| value: 0.8297161936560935 | |
| name: Cosine Accuracy@3 | |
| - type: cosine_accuracy@5 | |
| value: 0.8614357262103506 | |
| name: Cosine Accuracy@5 | |
| - type: cosine_accuracy@10 | |
| value: 0.8948247078464107 | |
| name: Cosine Accuracy@10 | |
| - type: cosine_precision@1 | |
| value: 0.7612687813021702 | |
| name: Cosine Precision@1 | |
| - type: cosine_precision@3 | |
| value: 0.5275459098497496 | |
| name: Cosine Precision@3 | |
| - type: cosine_precision@5 | |
| value: 0.33489148580968287 | |
| name: Cosine Precision@5 | |
| - type: cosine_precision@10 | |
| value: 0.17896494156928214 | |
| name: Cosine Precision@10 | |
| - type: cosine_recall@1 | |
| value: 0.3664997217584864 | |
| name: Cosine Recall@1 | |
| - type: cosine_recall@3 | |
| value: 0.701307735114079 | |
| name: Cosine Recall@3 | |
| - type: cosine_recall@5 | |
| value: 0.7390651085141903 | |
| name: Cosine Recall@5 | |
| - type: cosine_recall@10 | |
| value: 0.7844462993878687 | |
| name: Cosine Recall@10 | |
| - type: cosine_ndcg@10 | |
| value: 0.7519775439563073 | |
| name: Cosine Ndcg@10 | |
| - type: cosine_mrr@10 | |
| value: 0.8039410922966848 | |
| name: Cosine Mrr@10 | |
| - type: cosine_map@100 | |
| value: 0.7154125325663795 | |
| name: Cosine Map@100 | |
| # SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B | |
| This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. | |
| ## Model Details | |
| ### Model Description | |
| - **Model Type:** Sentence Transformer | |
| - **Base model:** [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) <!-- at revision c54f2e6e80b2d7b7de06f51cec4959f6b3e03418 --> | |
| - **Maximum Sequence Length:** 768 tokens | |
| - **Output Dimensionality:** 1024 dimensions | |
| - **Similarity Function:** Cosine Similarity | |
| <!-- - **Training Dataset:** Unknown --> | |
| <!-- - **Language:** Unknown --> | |
| <!-- - **License:** Unknown --> | |
| ### Model Sources | |
| - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) | |
| - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers) | |
| - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) | |
| ### Full Model Architecture | |
| ``` | |
| SentenceTransformer( | |
| (0): Transformer({'max_seq_length': 768, 'do_lower_case': False, 'architecture': 'PeftModelForFeatureExtraction'}) | |
| (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True}) | |
| (2): Normalize() | |
| ) | |
| ``` | |
| ## Usage | |
| ### Direct Usage (Sentence Transformers) | |
| First install the Sentence Transformers library: | |
| ```bash | |
| pip install -U sentence-transformers | |
| ``` | |
| Then you can load this model and run inference. | |
| ```python | |
| from sentence_transformers import SentenceTransformer | |
| # Download from the 🤗 Hub | |
| model = SentenceTransformer("sentence_transformers_model_id") | |
| # Run inference | |
| queries = [ | |
| "What bottlenecks in the updated post-operative workflow are contributing to delays in surgical site infection specimen transfer and tracking?", | |
| ] | |
| documents = [ | |
| 'Subject: Concerns Regarding Timeliness of Surgical Site Infection Tracking\nFrom: Xavier D. Brooks\nTo: David S. Wilson\nDate: 2025-11-10\n\nHi David,\n\nThank you for raising these concerns about the delays in surgical site infection tracking. We have indeed adjusted some aspects of our post-op patient flow in an attempt to enhance discharge efficiency, including new documentation checkpoints that might inadvertently be slowing the specimen transfer process. I’ll coordinate with our nursing and records teams to closely review recent workflow changes and identify any bottlenecks that could be contributing to extended turnaround times. I’ll share our findings and propose potential improvements by the end of this week, and I welcome any further details you notice from the lab side as well.\n\nBest regards,\nXavier', | |
| 'Subject: Concern Regarding Allergy Documentation Accuracy and Glucose Meter Integration\nFrom: Daniel M. Evans\nTo: Gabriella I. Santos\nDate: 2026-01-26\n\nHi Gabriella,\n\nI wanted to bring to your attention a recurring issue we’ve noticed with our glucose meters not consistently syncing updated allergy information from the patient chart. During routine maintenance, I found discrepancies between recorded allergies on the device and what is documented in the EMR, which could lead to potential risks for patients with sensitivities, especially regarding test strip ingredients. I propose we review the current integration workflow and possibly schedule a troubleshooting session with IT to ensure seamless allergy data transfer. Please let me know if you’ve experienced similar concerns and if you’d be available to discuss this further.\n\nThanks,\nDaniel', | |
| 'Subject: Inquiry Regarding Post-Operativ Care Documentation\nFrom: David R. Park\nTo: Inspector Helen R. Jacobs\nDate: 2026-01-26\n\nHello Inspector Jacobs,\n\nI am reaching out regarding the ongoing investigation tied to Mr. Hendricks’ recent case. We have been reviewing the patient records and noticed that the documentation for the post-operativ care period contains several ambiguities. We would appreciate your guidance on whether additional clarification or supplementary notes are required for compliance purposes. Please let me know how you would like us to proceed, or if you need copies of the relevant chart sections.\n\nBest regards,\nDavid R. Park', | |
| ] | |
| query_embeddings = model.encode_query(queries) | |
| document_embeddings = model.encode_document(documents) | |
| print(query_embeddings.shape, document_embeddings.shape) | |
| # [1, 1024] [3, 1024] | |
| # Get the similarity scores for the embeddings | |
| similarities = model.similarity(query_embeddings, document_embeddings) | |
| print(similarities) | |
| # tensor([[0.7188, 0.1221, 0.0596]], dtype=torch.bfloat16) | |
| ``` | |
| <!-- | |
| ### Direct Usage (Transformers) | |
| <details><summary>Click to see the direct usage in Transformers</summary> | |
| </details> | |
| --> | |
| <!-- | |
| ### Downstream Usage (Sentence Transformers) | |
| You can finetune this model on your own dataset. | |
| <details><summary>Click to expand</summary> | |
| </details> | |
| --> | |
| <!-- | |
| ### Out-of-Scope Use | |
| *List how the model may foreseeably be misused and address what users ought not to do with the model.* | |
| --> | |
| ## Evaluation | |
| ### Metrics | |
| #### Information Retrieval | |
| * Dataset: `val_real_corpus_thread_ir` | |
| * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | |
| | Metric | Value | | |
| |:--------------------|:----------| | |
| | cosine_accuracy@1 | 0.7613 | | |
| | cosine_accuracy@3 | 0.8297 | | |
| | cosine_accuracy@5 | 0.8614 | | |
| | cosine_accuracy@10 | 0.8948 | | |
| | cosine_precision@1 | 0.7613 | | |
| | cosine_precision@3 | 0.5275 | | |
| | cosine_precision@5 | 0.3349 | | |
| | cosine_precision@10 | 0.179 | | |
| | cosine_recall@1 | 0.3665 | | |
| | cosine_recall@3 | 0.7013 | | |
| | cosine_recall@5 | 0.7391 | | |
| | cosine_recall@10 | 0.7844 | | |
| | **cosine_ndcg@10** | **0.752** | | |
| | cosine_mrr@10 | 0.8039 | | |
| | cosine_map@100 | 0.7154 | | |
| <!-- | |
| ## Bias, Risks and Limitations | |
| *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* | |
| --> | |
| <!-- | |
| ### Recommendations | |
| *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* | |
| --> | |
| ## Training Details | |
| ### Training Dataset | |
| #### Unnamed Dataset | |
| * Size: 2,392 training samples | |
| * Columns: <code>sentence_0</code> and <code>sentence_1</code> | |
| * Approximate statistics based on the first 1000 samples: | |
| | | sentence_0 | sentence_1 | | |
| |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | |
| | type | string | string | | |
| | details | <ul><li>min: 11 tokens</li><li>mean: 26.85 tokens</li><li>max: 62 tokens</li></ul> | <ul><li>min: 99 tokens</li><li>mean: 159.15 tokens</li><li>max: 364 tokens</li></ul> | | |
| * Samples: | |
| | sentence_0 | sentence_1 | | |
| |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | |
| | <code>What specific documents and timeline details are being requested for the medication administration incident involving the late husband (e.g., notes and observed discrepancy times)?</code> | <code>Subject: Clarification Needed Regarding Recent Medciation Administration Incident<br>From: David R. Park<br>To: Margaret L. Hendricks<br>Date: 2025-10-16<br><br>Hello Mrs. Hendricks,<br><br>Thank you for your prompt reply and for clarifying your experience regarding the medication administration incident involving your late husband. I acknowledge your willingness to provide further details and want to ensure that our review is thorough and respectful of your family's concerns. A call on Wednesday afternoon works for me, and I appreciate your flexibility in offering to share information by email. If you have any documentation, such as notes or times you observed discrepancies, that would be very helpful for our review. Please let me know your preferred time for the call, or if you wish to send information in writing, I am happy to review it carefully.<br><br>Thank you again for your cooperation as we work to address these important concerns. I look forward to speaking with you and assisting however I can.<br><br>Best r...</code> | | |
| | <code>What specific additional materials or documentation should my team prepare ahead of the meeting?</code> | <code>Subject: Re: Meeting Confirmation and Case Materials<br>From: David R. Park<br>To: Katherine E. Morrison<br>Date: 2025-12-01<br><br>Hi Katherine,<br><br>Thank you for confirming the meeting time and sharing the agenda. I appreciate your prompt coordination on this. Please let me know if there are any additional materials or documentation you would like from my team ahead of our discussion. I look forward to collaborating and ensuring all questions are addressed at our meeting.<br><br>Best regards,<br>David</code> | | |
| | <code>Who is assigned to coordinate the review of PACS-EHR interface error logs with radiology IT to address radiology report delays?</code> | <code>Subject: Radiology Report Turnaround Delays in EHR<br>From: Angela R. Scott<br>To: Laura A. Hughes<br>Date: 2025-11-17<br><br>Hi Laura,<br><br>I've noticed a consistent delay in radiology report turnaround times stemming from integration issues between the PACS interface and our EHR system. Reports are not always populating promptly in patient records, which is affecting timely communication with both care teams and patients. I suggest we collaborate with the radiology IT staff to review interface error logs and streamline the auto-notification features. If you have additional insight from recent patient feedback or workflow observations, please let me know so we can address this comprehensively.<br><br>Thanks,<br>Angela<br><br>---<br>This email and any attachments are confidential and intended solely for the use of the individual or entity to whom they are addressed.</code> | | |
| * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: | |
| ```json | |
| { | |
| "scale": 20.0, | |
| "similarity_fct": "cos_sim", | |
| "gather_across_devices": false | |
| } | |
| ``` | |
| ### Training Hyperparameters | |
| #### Non-Default Hyperparameters | |
| - `multi_dataset_batch_sampler`: round_robin | |
| #### All Hyperparameters | |
| <details><summary>Click to expand</summary> | |
| - `do_predict`: False | |
| - `eval_strategy`: no | |
| - `prediction_loss_only`: True | |
| - `per_device_train_batch_size`: 8 | |
| - `per_device_eval_batch_size`: 8 | |
| - `gradient_accumulation_steps`: 1 | |
| - `eval_accumulation_steps`: None | |
| - `torch_empty_cache_steps`: None | |
| - `learning_rate`: 5e-05 | |
| - `weight_decay`: 0.0 | |
| - `adam_beta1`: 0.9 | |
| - `adam_beta2`: 0.999 | |
| - `adam_epsilon`: 1e-08 | |
| - `max_grad_norm`: 1 | |
| - `num_train_epochs`: 3 | |
| - `max_steps`: -1 | |
| - `lr_scheduler_type`: linear | |
| - `lr_scheduler_kwargs`: None | |
| - `warmup_ratio`: None | |
| - `warmup_steps`: 0 | |
| - `log_level`: passive | |
| - `log_level_replica`: warning | |
| - `log_on_each_node`: True | |
| - `logging_nan_inf_filter`: True | |
| - `enable_jit_checkpoint`: False | |
| - `save_on_each_node`: False | |
| - `save_only_model`: False | |
| - `restore_callback_states_from_checkpoint`: False | |
| - `use_cpu`: False | |
| - `seed`: 42 | |
| - `data_seed`: None | |
| - `bf16`: False | |
| - `fp16`: False | |
| - `bf16_full_eval`: False | |
| - `fp16_full_eval`: False | |
| - `tf32`: None | |
| - `local_rank`: -1 | |
| - `ddp_backend`: None | |
| - `debug`: [] | |
| - `dataloader_drop_last`: False | |
| - `dataloader_num_workers`: 0 | |
| - `dataloader_prefetch_factor`: None | |
| - `disable_tqdm`: False | |
| - `remove_unused_columns`: True | |
| - `label_names`: None | |
| - `load_best_model_at_end`: False | |
| - `ignore_data_skip`: False | |
| - `fsdp`: [] | |
| - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} | |
| - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} | |
| - `parallelism_config`: None | |
| - `deepspeed`: None | |
| - `label_smoothing_factor`: 0.0 | |
| - `optim`: adamw_torch_fused | |
| - `optim_args`: None | |
| - `group_by_length`: False | |
| - `length_column_name`: length | |
| - `project`: huggingface | |
| - `trackio_space_id`: trackio | |
| - `ddp_find_unused_parameters`: None | |
| - `ddp_bucket_cap_mb`: None | |
| - `ddp_broadcast_buffers`: False | |
| - `dataloader_pin_memory`: True | |
| - `dataloader_persistent_workers`: False | |
| - `skip_memory_metrics`: True | |
| - `push_to_hub`: False | |
| - `resume_from_checkpoint`: None | |
| - `hub_model_id`: None | |
| - `hub_strategy`: every_save | |
| - `hub_private_repo`: None | |
| - `hub_always_push`: False | |
| - `hub_revision`: None | |
| - `gradient_checkpointing`: False | |
| - `gradient_checkpointing_kwargs`: None | |
| - `include_for_metrics`: [] | |
| - `eval_do_concat_batches`: True | |
| - `auto_find_batch_size`: False | |
| - `full_determinism`: False | |
| - `ddp_timeout`: 1800 | |
| - `torch_compile`: False | |
| - `torch_compile_backend`: None | |
| - `torch_compile_mode`: None | |
| - `include_num_input_tokens_seen`: no | |
| - `neftune_noise_alpha`: None | |
| - `optim_target_modules`: None | |
| - `batch_eval_metrics`: False | |
| - `eval_on_start`: False | |
| - `use_liger_kernel`: False | |
| - `liger_kernel_config`: None | |
| - `eval_use_gather_object`: False | |
| - `average_tokens_across_devices`: True | |
| - `use_cache`: False | |
| - `prompts`: None | |
| - `batch_sampler`: batch_sampler | |
| - `multi_dataset_batch_sampler`: round_robin | |
| - `router_mapping`: {} | |
| - `learning_rate_mapping`: {} | |
| </details> | |
| ### Training Logs | |
| | Epoch | Step | Training Loss | val_real_corpus_thread_ir_cosine_ndcg@10 | | |
| |:------:|:----:|:-------------:|:----------------------------------------:| | |
| | 1.0 | 299 | - | 0.7464 | | |
| | 1.6722 | 500 | 0.0176 | - | | |
| | 2.0 | 598 | - | 0.7507 | | |
| | 3.0 | 897 | - | 0.7520 | | |
| ### Framework Versions | |
| - Python: 3.12.12 | |
| - Sentence Transformers: 5.2.2 | |
| - Transformers: 5.0.0 | |
| - PyTorch: 2.9.0+cu128 | |
| - Accelerate: 1.12.0 | |
| - Datasets: 4.0.0 | |
| - Tokenizers: 0.22.2 | |
| ## Citation | |
| ### BibTeX | |
| #### Sentence Transformers | |
| ```bibtex | |
| @inproceedings{reimers-2019-sentence-bert, | |
| title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", | |
| author = "Reimers, Nils and Gurevych, Iryna", | |
| booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", | |
| month = "11", | |
| year = "2019", | |
| publisher = "Association for Computational Linguistics", | |
| url = "https://arxiv.org/abs/1908.10084", | |
| } | |
| ``` | |
| #### MultipleNegativesRankingLoss | |
| ```bibtex | |
| @misc{henderson2017efficient, | |
| title={Efficient Natural Language Response Suggestion for Smart Reply}, | |
| author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, | |
| year={2017}, | |
| eprint={1705.00652}, | |
| archivePrefix={arXiv}, | |
| primaryClass={cs.CL} | |
| } | |
| ``` | |
| <!-- | |
| ## Glossary | |
| *Clearly define terms in order to be accessible across audiences.* | |
| --> | |
| <!-- | |
| ## Model Card Authors | |
| *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* | |
| --> | |
| <!-- | |
| ## Model Card Contact | |
| *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* | |
| --> |