---
tags:
- text-generation
- gpt2
- healthcare
- clinical-report
- risk-assessment
datasets:
- HospitalReadmission
license: cc-by-nc-4.0
---

# HealthcareReadmissionGPT

## 📝 Overview

HealthcareReadmissionGPT is a fine-tuned **GPT-2 (8-layer)** model developed for generating structured, narrative clinical summaries and predicting readmission risk based on patient features. This model is trained on tokenized, structured clinical records (similar to Dataset 3), where all features (`Age`, `Diagnosis`, `LengthOfStay`, `ComorbidityCount`, `HadSurgery`, etc.) are converted into a sequential natural language prompt format.

The primary goal is **Structured Text Generation**, where the model generates a narrative that concludes with a binary risk prediction (e.g., "Risk: HIGH. Action: FOLLOW-UP") as the final output token sequence.

## 🧠 Model Architecture

This model is a smaller version of the **GPT-2** architecture, optimized for clinical text processing.

* **Base Model:** A custom GPT-2 variant with 8 layers (smaller than `gpt2-medium`).
* **Mechanism:** Decoder-only Transformer with a causal attention mask, allowing it to predict the next token based on all previous tokens.
* **Training Data Format:** Input data is structured as a sequence: `[BOS] Age: 68; Diagnosis: CHF; LOS: 5; Comorbidities: 3; Surgery: Yes; --> Patient is elderly with CHF. A 5-day stay indicates stability. However, multiple comorbidities pose high risk. Readmission: 1 [EOS]`
* **Output:** Generative text output, often used to bridge structured data and human-readable clinical summaries.

## 🚀 Intended Use

* **Risk Documentation Generation:** Quickly convert structured Electronic Health Record (EHR) data into a coherent, narrative summary suitable for handover or case notes.
* **Automated Risk Flagging:** When prompted with patient data, the model's generated output will conclude with the predicted readmission risk token (e.g., '1' for readmission, '0' for no readmission), acting as a text-based binary classifier.
* **Clinical Training Data Synthesis:** Generate realistic, synthetic clinical reports for training purposes (with appropriate safeguards).

## ⚠️ Limitations

* **Hallucination/Factual Accuracy:** As a generative model, it may occasionally "hallucinate" clinically inaccurate information or generate text that contradicts the input data. **It must not be used for actual clinical decision-making.**
* **Tokenization Sensitivity:** The performance is highly dependent on the quality and consistency of the structured data-to-text tokenization scheme.
* **Bias:** The model can inherit and amplify biases present in the training data (e.g., correlations between `Race` or `InsuranceType` and `Readmission_30d`). Use with extreme caution in predictive tasks.
* **License:** The **CC BY-NC 4.0** license restricts commercial use.

## 💻 Example Code

To use the model for generating a risk assessment narrative:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load the model and tokenizer
model_name = "your-username/HealthcareReadmissionGPT" # Replace with actual HuggingFace path
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Example structured prompt (must match training format)
prompt_text = "Age: 79; Race: Black; Diagnosis: Lung Cancer; LOS: 13; Comorbidities: 4; Surgery: Yes; --> Patient is elderly with complex cancer and long LOS. Multiple..."

# Encode the prompt
input_ids = tokenizer.encode(prompt_text, return_tensors='pt')

# Generate the continuation text (up to 50 new tokens)
output = model.generate(
    input_ids,
    max_length=len(input_ids[0]) + 50,
    num_return_sequences=1,
    do_sample=True,
    temperature=0.7,
    pad_token_id=tokenizer.eos_token_id # Important for GPT-like models
)

# Decode and print the result
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)

print(f"--- Input Prompt ---\n{prompt_text}\n")
print(f"--- Generated Assessment ---\n{generated_text}")
# Expected last token sequence: ... Readmission: 1