AtharAbbas993/tinyllama-pneumonia-detection

🩺 TinyLLaMA-1B Medical Fine-Tuned Model (Chest X-ray Pneumonia Detection)

This model is a TinyLLaMA-1.1B fine-tuned using QLoRA (4-bit) on a custom medical dataset focused on chest X-ray pneumonia detection reports.
It is designed to generate structured medical findings, explanations, and possible diagnoses from given chest X-ray text data.

---

📌 Model Details

• Base Model: TinyLLaMA/TinyLLaMA-1.1B-Chat-v1.0
• Fine-tuning Method: QLoRA (4-bit quantization)
• Frameworks Used: Transformers, PEFT, BitsAndBytes
• Domain: Medical NLP (Radiology Reports)
• Task: Text generation for chest X-ray interpretation and pneumonia detection

---

📂 Dataset

• Type: Medical instruction-response pairs
• Format: JSONL
• Example:

{
  "instruction": "Analyze the chest X-ray report and indicate if pneumonia is present",
  "input": "Findings: Left lower lobe opacity with consolidation signs.",
  "output": "The report indicates probable pneumonia affecting the left lower lobe."
}

---

🚀 Training Configuration

• Max sequence length: 512 tokens
• Per device batch size: 2
• Gradient accumulation steps: 4
• Learning rate: 2e-4
• Max steps: 200 (adjustable)
• Quantization: 4-bit for GPU efficiency
• Hardware Used: NVIDIA T4 (Colab Pro)

---

💻 How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "AtharAbbas993/tinyllama-custom"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

prompt = """Instruction: Analyze the chest X-ray report and indicate if pneumonia is present
Input: Findings: Bilateral lower lobe opacities with patchy consolidation.
Response:"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

📊 Example Output

Prompt:

Instruction: Analyze the chest X-ray report and indicate if pneumonia is present
Input: Findings: Left lower lobe opacity with consolidation signs.
Response:

Model Output:

The report suggests pneumonia in the left lower lobe, consistent with consolidation.

---

⚠️ Limitations & Disclaimer

• The model is not a substitute for professional medical diagnosis.
• Trained on a limited dataset; performance depends on data quality.
• Should be used only for research and educational purposes.
• Always consult a certified radiologist for final interpretation.

---

📜 License

This model follows the base model's license. Please refer to the TinyLLaMA License for details.

---

🙌 Acknowledgements

• TinyLLaMA Team for the base model.
• Hugging Face for hosting and tools.
• Dataset compiled from publicly available medical report data for pneumonia detection.