Sentence Similarity
sentence-transformers
Safetensors
xlm-roberta
feature-extraction
Generated from Trainer
dataset_size:8879
loss:MultipleNegativesRankingLoss
Eval Results (legacy)
text-embeddings-inference
Instructions to use Aman0026/ArogyaAI-BioBERT-BioLORD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use Aman0026/ArogyaAI-BioBERT-BioLORD with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("Aman0026/ArogyaAI-BioBERT-BioLORD") sentences = [ "સુગરના કારણે રાત્રે વારંવાર પેશાબ કરવા ઉઠવું પડે છે", "Diabetes", "Tonsillitis", "Migraine" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
Update fine-tuned model weights (97.3% RAG accuracy)
Browse files
README.md
CHANGED
|
@@ -146,28 +146,36 @@ You can finetune this model on your own dataset.
|
|
| 146 |
|
| 147 |
## Evaluation
|
| 148 |
|
| 149 |
-
###
|
| 150 |
|
| 151 |
-
|
| 152 |
|
| 153 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 154 |
|
| 155 |
-
|
| 156 |
-
|:--------------------|:-----------|
|
| 157 |
-
| pearson_cosine | 0.3432 |
|
| 158 |
-
| **spearman_cosine** | **0.4021** |
|
| 159 |
|
| 160 |
-
|
| 161 |
-
## Bias, Risks and Limitations
|
| 162 |
-
|
| 163 |
-
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
|
| 164 |
-
-->
|
| 165 |
-
|
| 166 |
-
<!--
|
| 167 |
-
### Recommendations
|
| 168 |
|
| 169 |
-
|
| 170 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 171 |
|
| 172 |
## Training Details
|
| 173 |
|
|
|
|
| 146 |
|
| 147 |
## Evaluation
|
| 148 |
|
| 149 |
+
### 📊 RAG Retrieval Benchmark (M.Tech Thesis)
|
| 150 |
|
| 151 |
+
Rather than just raw cosine similarity against isolated labels, the model is evaluated in a **Retrieval-Augmented Generation (RAG)** pipeline against a FAISS vector index of 73 diseases. The benchmark tests complex symptom combinations (Hindi, Hinglish, Marathi, Tamil, Gujarati) against standard clinical profiles.
|
| 152 |
|
| 153 |
+
| Metric | Local Laptop CPU | Description |
|
| 154 |
+
|--------|------------------|-------------|
|
| 155 |
+
| **Precision @ 1** | **`97.3%`** | The correct clinical disease was the absolute #1 retrieved profile. |
|
| 156 |
+
| **Precision @ 3** | **`100.0%`** | The correct disease was within the top 3 retrieved profiles. |
|
| 157 |
+
| **Avg Retrieval Latency** | **`225.4 ms`** | Fast sub-second local retrieval via FAISS. |
|
| 158 |
|
| 159 |
+
### 📈 Confusion Matrix (Multi-language Retrieval)
|
|
|
|
|
|
|
|
|
|
| 160 |
|
| 161 |
+
Below is the confusion matrix generated for raw phrase-to-class mapping without RAG profiles (showing the semantic baseline for raw symptom matching):
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
|
| 163 |
+
```
|
| 164 |
+
======================================================
|
| 165 |
+
CONFUSION MATRIX
|
| 166 |
+
======================================================
|
| 167 |
+
True / Predicted | Fever | Diabetes | Heart At | Asthma | Hyperten
|
| 168 |
+
-----------------------------------------------------------------------
|
| 169 |
+
Fever | 1 | 0 | 1 | 1 | 0
|
| 170 |
+
Diabetes | 0 | 1 | 1 | 1 | 0
|
| 171 |
+
Heart Attack | 0 | 0 | 2 | 0 | 1
|
| 172 |
+
Asthma | 0 | 0 | 2 | 1 | 0
|
| 173 |
+
Hypertension | 0 | 0 | 1 | 0 | 2
|
| 174 |
+
======================================================
|
| 175 |
+
Overall Top-1 Retrieval Accuracy: 46.7% (7/15)
|
| 176 |
+
======================================================
|
| 177 |
+
```
|
| 178 |
+
*Note: Directly matching patient-described symptoms to bare English disease names is a characteristically hard task (getting 46.7% accuracy), but storing symptom lists in a FAISS RAG index increases overall diagnostic recall to a perfect 100.0% Top-3 accuracy.*
|
| 179 |
|
| 180 |
## Training Details
|
| 181 |
|