namaste-asu-matcher

This is a sentence-transformers model fine-tuned from all-MiniLM-L6-v2 to specialize in clinical terminology mapping. It embeds patient symptoms and raw text complaints into a vector space optimized for matching against standard Ayurveda, Siddha, and Unani (ASU) classification codes from the NAMASTE dictionary.

Model Details

• Developed By: Soham Chakraborty
• Base Model: sentence-transformers/all-MiniLM-L6-v2
• Model Type: Sentence Embedding Transformer (Bi-Encoder / Feature Extraction)
• Model Version: v1.0.0
• Training Method: MultipleNegativesRankingLoss
• Use Case: Semantic search, typo-tolerant mapping, clinical vocabulary matching
• Max Sequence Length: 256 tokens
• Output Dimension: 384 dimensions

This is a sentence embedding (Bi-Encoder) model. It is designed solely for converting text strings into dense vector representations (384-dimensional floats) to calculate cosine similarity. It is not a generative conversational model (like Llama or GPT) and cannot generate text responses.

Intended Use & Capabilities

This model is designed to be loaded by the NAMASTE Interoperability Portal mapping engine to resolve semantic typos (e.g., mapping "pitta imbalance" or "vata agravation" to "Aggravation of vata pattern").

How to Use

1. Loading with SentenceTransformers

from sentence_transformers import SentenceTransformer

# Load directly from Hugging Face
model = SentenceTransformer("0xsoh/namaste-asu-matcher")

sentences = [
    "vata aggravation",
    "severe headache",
    "shivering fever"
]
embeddings = model.encode(sentences)
print(embeddings.shape)  # Output: (3, 384)

2. Running local Cosine Similarity

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("0xsoh/namaste-asu-matcher")

query = model.encode("high body temperature")
standard_terms = model.encode(["Fever with body pain disorder", "Headache disorders", "Constipation pattern"])

# Compute similarity scores
scores = util.cos_sim(query, standard_terms)
print(scores)

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Training Details

Training Dataset

• Size: 1,322 training samples
• Columns: anchor and positive
• Dataset Samples:

  anchor	positive
  Cephalalgia disorder	Cephalalgia disorder Head, brain, nerve and movement disorders -> Headache disorders (ICD-11: SK00)
  cephalalgia	Cephalalgia disorder Head, brain, nerve and movement disorders -> Headache disorders (ICD-11: SK00)
  Migraine disorder	Migraine disorder Head, brain, nerve and movement disorders -> Headache disorders (ICD-11: SK01)

Training Loss

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": ["query_to_doc"],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

• Optimizer: AdamW (adamw_torch)
• Initial Learning Rate: 5e-5
• Epochs: 5
• Batch Size: 32
• Warmup Steps: 50
• Warmup Ratio: 0.0
• Scheduler Type: linear
• Dataloader Pin Memory: True

Training Logs

Epoch	Step	Training Loss
0.2381	10	0.1055
0.4762	20	0.0672
0.7143	30	0.0387
0.9524	40	0.0346
1.1905	50	0.0301
1.4286	60	0.0241
1.6667	70	0.0341
1.9048	80	0.0388
2.1429	90	0.0212
2.3810	100	0.0658
2.6190	110	0.0197
2.8571	120	0.0411
3.0952	130	0.0482
3.3333	140	0.0417
3.5714	150	0.0253
3.8095	160	0.0569
4.0476	170	0.0211
4.2857	180	0.0496
4.5238	190	0.0342
4.7619	200	0.0286
5.0	210	0.0464

• Total Training Runtime: 2.8 minutes

Framework Versions

• Python: 3.13.7
• Sentence Transformers: 3.0.0 (and above)
• Transformers: 4.47.1
• PyTorch: 2.13.0+cpu
• Accelerate: 1.14.0
• Datasets: 5.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}