SheikhMiniMoE-Medical

A production-ready Mixture of Experts (MoE) model optimized for bilingual medical Q&A in English and Bengali.

Model Details

• Architecture: SheikhMiniMoE - Sparse Mixture of Experts with top-k routing
• Parameters: N/A
• Vocabulary Size: 5000
• Embedding Dimension: 256
• Hidden Dimension: 512
• Number of Experts: 8
• Top-K Routing: 2

Intended Use

This model is designed for humanitarian medical assistance, providing:

• Bilingual medical Q&A (English/Bengali)
• Safety-first responses with appropriate disclaimers
• Emergency keyword detection
• Knowledge base augmentation

Training Configuration

• Training Date: 2026-01-18
• Device: CPU-optimized for deployment in resource-constrained environments

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model_name = "OsamaBinLikhon/SheikhMiniMoE-Medical"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Medical query
prompt = "What are symptoms of fever?"
inputs = tokenizer(prompt, return_tensors="pt")

# Generate response
outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Safety Notice

⚠️ This model provides medical information for educational purposes only. It is not a substitute for professional medical advice, diagnosis, or treatment. Always consult qualified healthcare providers for medical concerns.

License

Apache 2.0 - See LICENSE file for details.

Citation

@misc{SheikhMiniMoE,
  title={SheikhMiniMoE: Bilingual Medical Assistant},
  author={MiniMax Agent},
  year={2025},
  url={https://huggingface.co/OsamaBinLikhon/SheikhMiniMoE-Medical}
}