---
language:
- en
license: cc-by-nc-nd-4.0
library_name: pytorch
tags:
- transformer
- character-level
- medical-coding
- icd-10-cm
- healthcare
- gpt
- nanogpt
- visualization
- sequence-modeling
datasets:
- MIMIC-IV
- NHDS
- NHCS
metrics:
- perplexity
pipeline_tag: text-generation
---

# DgPg: Diagnosis Progression Transformer

**DgPg** ("**D**ia**g**nosis **P**ro**g**ression") is a compact (~2.5 MB) character-level, GPT-style transformer that learns representative *ordering patterns* of ICD-10-CM diagnostic (DX) codes within single-visit medical records. Given a "seed" diagnosis code, DgPg generates the codes most likely to precede or follow it, which can be rendered as graph-based visual summaries of an organization's coding practices.

DgPg was introduced in the paper [*Visualizing Medical Coding Practices Using Transformer Models*](https://doi.org/10.5220/0013257800003905) (Hobson & Huang, ICPRAM 2025), from the Department of Electrical Engineering and Computer Science at the University of Tennessee, Knoxville.

> ⚠️ **DgPg is an organizational-analytics tool, not a clinical or coding-assistance model.** It is not intended to improve medical coding, predict diagnoses, or support clinical decisions. Its purpose is to let hospital administrators, lead physicians, and other stakeholders *see* how coding is actually practiced in a given corpus of records.

## Model Details

| | |
|---|---|
| **Model type** | Decoder-only transformer (modified [nanoGPT](https://github.com/karpathy/nanoGPT)), character-level |
| **Task** | Next-character prediction over 4-character-normalized ICD-10-CM code sequences |
| **Parameters** | 1.3 M (recommended configuration, `T1.3M`) |
| **Architecture** | 3 layers, 3 attention heads, 192 embedding dims |
| **Precision** | float16 |
| **Model size on disk** | ~2.5 MB |
| **Training hardware** | Single NVIDIA T4 (Google Colab) |
| **Training time / cost** | ~1,092 s (~18 min), ≈ $0.05 per model |
| **License** | Paper published under CC BY-NC-ND 4.0 |
| **Authors** | Tanner Hobson, Jian Huang (University of Tennessee, Knoxville) |

### Base training configuration

```text
learning_rate  = 1e-3     dropout   = 0.2
lr_decay_iters = 30000    min_lr    = 1e-4
beta2          = 0.99     max_iters = 30000
batch_size     = 256      n_layer   = 3
n_head         = 3        n_embd    = 192
```

## How It Works

ICD-10-CM codes are inherently hierarchical: the first character encodes the top level of the hierarchy, the second character the next level, and so on (e.g., `S` → `S5` → `S52` → `S52.5` traces an injury from "injury/poisoning" all the way down to a specific radius fracture). DgPg exploits this by modeling code sequences **character by character** rather than treating each code as an atomic token.

- Every DX code is normalized to **4 characters** (truncated and/or padded with hyphens).
- Records are converted to training sequences of consecutive code pairs, e.g. the record `N12,Q610,N179` becomes `•----N12-`, `•N12-Q610`, `•Q610N179`, `•N179----`.
- At inference, the model is used **iteratively**: given a seed code, it predicts the next code one character at a time, naturally respecting the coding hierarchy.

Compared to a traditional code-level transition matrix (12,309 × 12,309 codes ≈ 290 MB in float16), the character-level formulation is >100× smaller and generalizes across code hierarchies rather than memorizing pairs.

### Visualization use case

The authors pair the model with a generative graph-building algorithm: starting from a seed code, a breadth-first search exhaustively scores candidate following (or preceding) codes by perplexity, keeps the top *k* = 3 at each step, and recurses to depth *n* = 3. The result is a left-to-right graph of the most representative code orderings around a diagnosis (e.g., what typically precedes and follows a sepsis code `A41.9`), generated in ~2 minutes per graph.

## Training Data

Four model variants were trained, one per dataset plus a combined corpus:

| Source | Dataset | # Records | # Patients (weighted) | Notes |
|---|---|---:|---:|---|
| MIT | [MIMIC-IV](https://physionet.org/content/mimiciv/) | 430,812 | 430,812 | Beth Israel Deaconess Medical Center; ICD-10-CM, truncated to 4 chars |
| CDC | NHDS | 756,953 | 111,514,788 | 1996–2010; ICD-9-CM converted to ICD-10-CM via the NBER crosswalk |
| CDC | NHCS | 123,565 | 33,595,765 | 2020 survey |
| — | Combined | 1,277,400 | 145,507,435 | Union of the above |

CDC survey records carry a frequency weight *w*; each record is emitted *w* times during training. Only ordered DX-code sequences are required — the framework applies to any corpus with ordered diagnosis codes.

**Note on data access:** MIMIC-IV is a restricted-access dataset requiring credentialed access via PhysioNet. No patient-level data is contained in the model weights beyond learned sequence statistics, but users retraining DgPg must obtain the datasets under their respective terms.

## Evaluation

Performance is measured as fixed-length sequence perplexity (PERF) on the second code of a two-code sequence, averaged over 10 K validation samples.

### Model size ablation

| Model | Layers | Heads | Embd | Size | Loss |
|---|---:|---:|---:|---:|---:|
| T11M | 6 | 6 | 384 | 20 MB | 1.16 |
| **T1.3M** ★ | 3 | 3 | 192 | **2.5 MB** | 1.17 |
| T230K | 2 | 2 | 96 | 440 KB | 1.19 |
| T30K | 1 | 1 | 48 | 58 KB | 1.25 |

T1.3M matches the 10×-larger T11M and is the recommended configuration. Model sizes were capped at 10% of the equivalent transition-matrix size to ensure generalizable (not memorized) patterns.

### Cross-dataset transferability

Evaluating each trained model against each dataset shows that models perform best on their own training distribution — evidence that **coding practices are provider- and scenario-specific**. The Combined model performs well across all constituent datasets; the NHDS-only model transfers worst, indicating NHDS coding patterns are the most dataset-unique. MIMIC and NHCS models are the most generalizable.

## Intended Uses

**Direct intended uses**

- Visualizing an organization's representative diagnostic-coding patterns (which codes typically precede/follow a given code; which codes tend to be coded first or last in a visit).
- Comparing coding practices across institutions, units, or clinical scenarios.
- Scoring how representative a given code sequence is relative to a corpus (classification use case).
- Operational insight for administrators: quality control, process improvement, burnout-related workflow analysis.

**Out-of-scope uses**

- ❌ Automated medical coding or coding suggestions for billing.
- ❌ Clinical diagnosis, prediction, or decision support.
- ❌ Auditing individual coders or labeling any coding practice as "good" or "bad" — the model summarizes shared practice; it does not evaluate deviations.
- ❌ Any use that would alter coding to affect claim payment. Modifying codes to increase reimbursement is fraudulent; any coding revision requires proper documentation.

## Limitations

- **4-character truncation.** For cross-dataset comparability, all codes are normalized to 4 characters, discarding finer ICD-10-CM specificity (the framework itself is not limited to 4 characters).
- **ICD-9 → ICD-10 crosswalk noise.** NHDS codes were converted via the NBER crosswalk, which is not lossless.
- **Single-visit scope.** DgPg models intra-visit code ordering only, not longitudinal, multi-visit patient trajectories.
- **Dataset-specific patterns.** A model trained on one provider's records does not transfer reliably to another's — this is by design, but means checkpoints are not general-purpose.
- **Hallucination.** As a generative model, DgPg can produce plausible-looking but unrepresentative sequences; the authors flag hallucination analysis as future work.
- **DX codes only.** Procedure and other medical code types are not yet covered.

## Citation

```bibtex
@inproceedings{hobson2025dgpg,
  author    = {Tanner Hobson and Jian Huang},
  title     = {Visualizing Medical Coding Practices Using Transformer Models},
  booktitle = {Proceedings of the 14th International Conference on Pattern
               Recognition Applications and Methods (ICPRAM 2025)},
  pages     = {725--732},
  year      = {2025},
  publisher = {SCITEPRESS},
  doi       = {10.5220/0013257800003905},
  isbn      = {978-989-758-730-6}
}
```

## Acknowledgments

Built on [nanoGPT](https://github.com/karpathy/nanoGPT) by Andrej Karpathy. Data courtesy of MIT-LCP (MIMIC-IV) and the US CDC National Center for Health Statistics (NHDS, NHCS).