|
|
--- |
|
|
license: openrail |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- CompVis/stable-diffusion-v1-4 |
|
|
pipeline_tag: text-to-image |
|
|
library_name: diffusers |
|
|
tags: |
|
|
- medical |
|
|
- X-ray |
|
|
- Diffusion |
|
|
- Generation |
|
|
- Text-to-image |
|
|
- stable-diffusion |
|
|
- lora |
|
|
- fine_tune |
|
|
|
|
|
widget: |
|
|
- text: >- |
|
|
Hey doc, I've been feeling really out of breath lately, |
|
|
especially when I'm walking up a flight of stairs or doing some light exercise. |
|
|
It's like my chest gets tight and I can't catch my breath. |
|
|
I've also been coughing up some stuff that's not quite right, it's been a few weeks now. |
|
|
And I've noticed a bit of weight loss, I'm not sure if that's related but it's been on my mind. |
|
|
I've been to a few doctors already, but they haven't been able to figure out what's going on. |
|
|
I'm hoping you can help. |
|
|
output: |
|
|
url: example.png |
|
|
|
|
|
--- |
|
|
|
|
|
# Symptom-to-Medical-Image Generator |
|
|
|
|
|
This project introduces a **text-to-image diffusion model** fine-tuned using **LoRA (Low-Rank Adaptation)** on top of [`CompVis/stable-diffusion-v1-4`](https://huggingface.co/CompVis/stable-diffusion-v1-4) for the task of **medical image generation**. The model generates **X-ray**, **CT**, or **MRI** scans based on **natural language descriptions of patient symptoms**, offering a novel way to visualize potential diagnostic outcomes. |
|
|
|
|
|
--- |
|
|
|
|
|
## What Is This Model? |
|
|
|
|
|
This is a **domain-adapted diffusion model** tailored to generate realistic medical scans conditioned on symptom prompts. The model was fine-tuned using LoRA, which allowed for: |
|
|
|
|
|
* Efficient training without modifying the original model weights. |
|
|
* Adaptation to a smaller, highly-specialized medical dataset. |
|
|
* Retention of high-quality generative capabilities from the base model. |
|
|
|
|
|
### Key Features |
|
|
|
|
|
* **Symptom-to-scan generation**: Input symptoms in plain English and receive a plausible X-ray, CT, or MRI image. |
|
|
* **Multi-modality support**: Generate different types of scans (e.g., chest X-rays, brain MRIs) depending on the prompt context. |
|
|
* **High realism**: Outputs are visually realistic and follow anatomical structure, trained using real medical datasets. |
|
|
|
|
|
--- |
|
|
|
|
|
## When Can You Use This Model? |
|
|
|
|
|
### Use Cases |
|
|
|
|
|
| Application Area | Description | |
|
|
| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | |
|
|
| **Medical Research** | Generate datasets for hypothesis testing or model training without using real patient data. | |
|
|
| **Education & Training** | Teach students about correlations between symptoms and imaging in an interactive way. | |
|
|
| **AI-Aided Prototyping** | Test downstream diagnostic pipelines on synthetic but realistic image data. | |
|
|
| **Data Augmentation** | Enrich datasets for training classification/segmentation models. | |
|
|
| **Prompt-Based Exploration** | Investigate how changes in symptoms affect image generation (e.g., how “fever + cough” differs from “chest pain + shortness of breath”). | |
|
|
|
|
|
### Not for Use In: |
|
|
|
|
|
* Real-world **clinical diagnosis or decision-making** |
|
|
* Generating scans for real patients or influencing treatment |
|
|
* Bypassing ethical or regulatory controls in medical AI |
|
|
|
|
|
--- |
|
|
|
|
|
## Example Usage |
|
|
|
|
|
### Input Prompt: |
|
|
|
|
|
> "I've been feeling really out of breath lately, especially when I'm walking up a flight of stairs or doing some light exercise. It's like my chest gets tight and I can't catch my breath. " |
|
|
|
|
|
### Output: |
|
|
|
|
|
<img src="example.png" alt="Generated Chest X-ray" width="512"/> |
|
|
|
|
|
> The model generates a chest X-ray image that corresponds to symptoms of a potential pulmonary issue. |
|
|
|
|
|
--- |
|
|
|
|
|
## Under the Hood |
|
|
|
|
|
* **Base Model**: `CompVis/stable-diffusion-v1-4` |
|
|
* **Fine-tuning Method**: LoRA (efficient, parameter-light adaptation) |
|
|
* **Dataset**: Custom dataset of symptom-to-image pairs, curated for medical imaging consistency |
|
|
* **Framework**: PyTorch + 🤗 Diffusers + Hugging Face Spaces |
|
|
|
|
|
--- |
|
|
|
|
|
## Ethical & Legal Disclaimer |
|
|
|
|
|
This model is strictly intended for **research and educational** use. It is **not a substitute for professional medical judgment**. Use of synthetic medical images should follow all local regulatory and ethical guidelines. |
|
|
|
|
|
--- |
|
|
|
