README.md

metadata

title: MediSim
emoji: 🩺
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
pinned: false

MediSim: Multimodal Diagnostic and Agentic Triage System

MediSim is an AI-powered medical assistant web application designed to safely process health inputs. It is developed as an NLP research project focused on reducing clinical hallucination in generative healthcare applications using hybrid learning pipelines and multi-agent orchestration.

Core Features

1. Multimodal Diagnostic Assistant

• Purpose: Provides preliminary diagnostic assessments by combining medical image data and symptom descriptions.
• Input: Medical scans (for example, chest X-ray) plus symptom text.
• Architecture:
  • Vision encoder: ResNet-18.
  • Text encoder: biLSTM.
  • Fusion head: late-fusion classifier.
• Advantage: Better reliability and lower compute demands than large generic multimodal models in this domain.

2. Agentic Triage and Consultation

• Purpose: Interactively gathers symptoms and provides verified clinical guidance.
• Processing: Three-agent collaboration loop:
  • Triage Nurse: empathic intake and symptom collection.
  • Specialist Doctor: differential reasoning and next-step planning.
  • Fact Checker: verifies outputs against safety constraints.
• Advantage: Reduces hallucination risk through explicit multi-agent verification.

Architecture

• Frontend: React + TypeScript + Vite.
• Backend: FastAPI + PyTorch + LangChain orchestration.
• Authentication and Storage: Firebase Auth + Firestore.
• Deployment target: Hugging Face Space (Docker).

Directory Layout

MediSim/
|- web_app_pro/           # Production web application
|  |- frontend/           # React + Vite app
|  |- backend/            # FastAPI service and model logic
|- web_app/               # Legacy app entrypoint used for HF runtime
|- data/                  # Trained weights and supporting assets
|- notebooks/             # Training and experimentation notebooks
|- reports/               # Project reports and writeups
|- scripts/               # Deployment and utility scripts
`- README.md

Local Development

Backend

cd web_app_pro/backend
pip install -r requirements.txt
python main.py

Frontend

cd web_app_pro/frontend
npm install
npm run dev

Hugging Face Deployment Notes

This repository includes:

• A Docker-based Space configuration.
• Space runtime entrypoint through web_app/app.py.
• Environment-driven Firebase and backend configuration.

Team Members

• Htut Ko Ko (st126010)
• Imtiaz Ahmad (st126685)
• Michael R. Lacar (st126161)
• Aashutosh Raut (st126438)

References

See project reports under reports/ for methodology, literature review, and evaluation details.