HealthcareReadmissionGPT

📝 Overview

HealthcareReadmissionGPT is a fine-tuned GPT-2 (8-layer) model developed for generating structured, narrative clinical summaries and predicting readmission risk based on patient features. This model is trained on tokenized, structured clinical records (similar to Dataset 3), where all features (Age, Diagnosis, LengthOfStay, ComorbidityCount, HadSurgery, etc.) are converted into a sequential natural language prompt format.

The primary goal is Structured Text Generation, where the model generates a narrative that concludes with a binary risk prediction (e.g., "Risk: HIGH. Action: FOLLOW-UP") as the final output token sequence.

🧠 Model Architecture

This model is a smaller version of the GPT-2 architecture, optimized for clinical text processing.

Base Model: A custom GPT-2 variant with 8 layers (smaller than gpt2-medium).
Mechanism: Decoder-only Transformer with a causal attention mask, allowing it to predict the next token based on all previous tokens.
Training Data Format: Input data is structured as a sequence: [BOS] Age: 68; Diagnosis: CHF; LOS: 5; Comorbidities: 3; Surgery: Yes; --> Patient is elderly with CHF. A 5-day stay indicates stability. However, multiple comorbidities pose high risk. Readmission: 1 [EOS]
Output: Generative text output, often used to bridge structured data and human-readable clinical summaries.

🚀 Intended Use

Risk Documentation Generation: Quickly convert structured Electronic Health Record (EHR) data into a coherent, narrative summary suitable for handover or case notes.
Automated Risk Flagging: When prompted with patient data, the model's generated output will conclude with the predicted readmission risk token (e.g., '1' for readmission, '0' for no readmission), acting as a text-based binary classifier.
Clinical Training Data Synthesis: Generate realistic, synthetic clinical reports for training purposes (with appropriate safeguards).

⚠️ Limitations

Hallucination/Factual Accuracy: As a generative model, it may occasionally "hallucinate" clinically inaccurate information or generate text that contradicts the input data. It must not be used for actual clinical decision-making.
Tokenization Sensitivity: The performance is highly dependent on the quality and consistency of the structured data-to-text tokenization scheme.
Bias: The model can inherit and amplify biases present in the training data (e.g., correlations between Race or InsuranceType and Readmission_30d). Use with extreme caution in predictive tasks.
License: The CC BY-NC 4.0 license restricts commercial use.

💻 Example Code

To use the model for generating a risk assessment narrative:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load the model and tokenizer
model_name = "your-username/HealthcareReadmissionGPT" # Replace with actual HuggingFace path
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Example structured prompt (must match training format)
prompt_text = "Age: 79; Race: Black; Diagnosis: Lung Cancer; LOS: 13; Comorbidities: 4; Surgery: Yes; --> Patient is elderly with complex cancer and long LOS. Multiple..."

# Encode the prompt
input_ids = tokenizer.encode(prompt_text, return_tensors='pt')

# Generate the continuation text (up to 50 new tokens)
output = model.generate(
    input_ids,
    max_length=len(input_ids[0]) + 50,
    num_return_sequences=1,
    do_sample=True,
    temperature=0.7,
    pad_token_id=tokenizer.eos_token_id # Important for GPT-like models
)

# Decode and print the result
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)

print(f"--- Input Prompt ---\n{prompt_text}\n")
print(f"--- Generated Assessment ---\n{generated_text}")
# Expected last token sequence: ... Readmission: 1

Downloads last month: 19