lsy9874205 commited on
Commit
07c9e13
·
verified ·
1 Parent(s): ce4635a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +32 -0
README.md CHANGED
@@ -496,7 +496,39 @@ You can finetune this model on your own dataset.
496
  url = "https://arxiv.org/abs/1908.10084",
497
  }
498
  ```
 
499
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
500
  <!--
501
  ## Glossary
502
 
 
496
  url = "https://arxiv.org/abs/1908.10084",
497
  }
498
  ```
499
+ # HEAL Protocol Embeddings
500
 
501
+ This model is fine-tuned from all-MiniLM-L6-v2 on HEAL Initiative clinical protocols.
502
+
503
+ ## Performance Evaluation
504
+
505
+ Comparison with OpenAI embeddings:
506
+
507
+ | Metric | OpenAI | Fine-tuned | Change |
508
+ |--------|--------|------------|---------|
509
+ | Faithfulness | 0.667 | 0.833 | ⬆️ +0.166 |
510
+ | Answer Relevancy | 0.986 | 0.986 | = |
511
+ | Context Precision | 1.000 | 1.000 | = |
512
+ | Context Recall | 1.000 | 0.000 | ⬇️ -1.000 |
513
+
514
+ ### Key Findings
515
+ - Improved faithfulness to source material
516
+ - Maintained high answer relevancy
517
+ - Trade-off in context recall
518
+
519
+ ## Future Improvements
520
+
521
+ 1. Retrieval Strategy
522
+ - Implement hybrid search combining semantic and keyword matching
523
+ - Add re-ranking for better result ordering
524
+
525
+ 2. Model Architecture
526
+ - Experiment with larger base models
527
+ - Fine-tune with domain-specific loss functions
528
+
529
+ 3. Data Processing
530
+ - Optimize chunking strategy
531
+ - Increase training data diversity
532
  <!--
533
  ## Glossary
534