Update README.md
Browse files
README.md
CHANGED
|
@@ -496,7 +496,39 @@ You can finetune this model on your own dataset.
|
|
| 496 |
url = "https://arxiv.org/abs/1908.10084",
|
| 497 |
}
|
| 498 |
```
|
|
|
|
| 499 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 500 |
<!--
|
| 501 |
## Glossary
|
| 502 |
|
|
|
|
| 496 |
url = "https://arxiv.org/abs/1908.10084",
|
| 497 |
}
|
| 498 |
```
|
| 499 |
+
# HEAL Protocol Embeddings
|
| 500 |
|
| 501 |
+
This model is fine-tuned from all-MiniLM-L6-v2 on HEAL Initiative clinical protocols.
|
| 502 |
+
|
| 503 |
+
## Performance Evaluation
|
| 504 |
+
|
| 505 |
+
Comparison with OpenAI embeddings:
|
| 506 |
+
|
| 507 |
+
| Metric | OpenAI | Fine-tuned | Change |
|
| 508 |
+
|--------|--------|------------|---------|
|
| 509 |
+
| Faithfulness | 0.667 | 0.833 | ⬆️ +0.166 |
|
| 510 |
+
| Answer Relevancy | 0.986 | 0.986 | = |
|
| 511 |
+
| Context Precision | 1.000 | 1.000 | = |
|
| 512 |
+
| Context Recall | 1.000 | 0.000 | ⬇️ -1.000 |
|
| 513 |
+
|
| 514 |
+
### Key Findings
|
| 515 |
+
- Improved faithfulness to source material
|
| 516 |
+
- Maintained high answer relevancy
|
| 517 |
+
- Trade-off in context recall
|
| 518 |
+
|
| 519 |
+
## Future Improvements
|
| 520 |
+
|
| 521 |
+
1. Retrieval Strategy
|
| 522 |
+
- Implement hybrid search combining semantic and keyword matching
|
| 523 |
+
- Add re-ranking for better result ordering
|
| 524 |
+
|
| 525 |
+
2. Model Architecture
|
| 526 |
+
- Experiment with larger base models
|
| 527 |
+
- Fine-tune with domain-specific loss functions
|
| 528 |
+
|
| 529 |
+
3. Data Processing
|
| 530 |
+
- Optimize chunking strategy
|
| 531 |
+
- Increase training data diversity
|
| 532 |
<!--
|
| 533 |
## Glossary
|
| 534 |
|