Create README.md

README.md

@@ -0,0 +1,149 @@
+---
+language:
+- en
+license: cc-by-nc-nd-4.0
+library_name: pytorch
+tags:
+- transformer
+- character-level
+- medical-coding
+- icd-10-cm
+- healthcare
+- gpt
+- nanogpt
+- visualization
+- sequence-modeling
+datasets:
+- MIMIC-IV
+- NHDS
+- NHCS
+metrics:
+- perplexity
+pipeline_tag: text-generation
+---
+
+# DgPg: Diagnosis Progression Transformer
+
+**DgPg** ("**D**ia**g**nosis **P**ro**g**ression") is a compact (~2.5 MB) character-level, GPT-style transformer that learns representative *ordering patterns* of ICD-10-CM diagnostic (DX) codes within single-visit medical records. Given a "seed" diagnosis code, DgPg generates the codes most likely to precede or follow it, which can be rendered as graph-based visual summaries of an organization's coding practices.
+
+DgPg was introduced in the paper [*Visualizing Medical Coding Practices Using Transformer Models*](https://doi.org/10.5220/0013257800003905) (Hobson & Huang, ICPRAM 2025), from the Department of Electrical Engineering and Computer Science at the University of Tennessee, Knoxville.
+
+> ⚠️ **DgPg is an organizational-analytics tool, not a clinical or coding-assistance model.** It is not intended to improve medical coding, predict diagnoses, or support clinical decisions. Its purpose is to let hospital administrators, lead physicians, and other stakeholders *see* how coding is actually practiced in a given corpus of records.
+
+## Model Details
+
+| | |
+|---|---|
+| **Model type** | Decoder-only transformer (modified [nanoGPT](https://github.com/karpathy/nanoGPT)), character-level |
+| **Task** | Next-character prediction over 4-character-normalized ICD-10-CM code sequences |
+| **Parameters** | 1.3 M (recommended configuration, `T1.3M`) |
+| **Architecture** | 3 layers, 3 attention heads, 192 embedding dims |
+| **Precision** | float16 |
+| **Model size on disk** | ~2.5 MB |
+| **Training hardware** | Single NVIDIA T4 (Google Colab) |
+| **Training time / cost** | ~1,092 s (~18 min), ≈ $0.05 per model |
+| **License** | Paper published under CC BY-NC-ND 4.0 |
+| **Authors** | Tanner Hobson, Jian Huang (University of Tennessee, Knoxville) |
+
+### Base training configuration
+
+```text
+learning_rate  = 1e-3     dropout   = 0.2
+lr_decay_iters = 30000    min_lr    = 1e-4
+beta2          = 0.99     max_iters = 30000
+batch_size     = 256      n_layer   = 3
+n_head         = 3        n_embd    = 192
+```
+
+## How It Works
+
+ICD-10-CM codes are inherently hierarchical: the first character encodes the top level of the hierarchy, the second character the next level, and so on (e.g., `S` → `S5` → `S52` → `S52.5` traces an injury from "injury/poisoning" all the way down to a specific radius fracture). DgPg exploits this by modeling code sequences **character by character** rather than treating each code as an atomic token.
+
+- Every DX code is normalized to **4 characters** (truncated and/or padded with hyphens).
+- Records are converted to training sequences of consecutive code pairs, e.g. the record `N12,Q610,N179` becomes `•----N12-`, `•N12-Q610`, `•Q610N179`, `•N179----`.
+- At inference, the model is used **iteratively**: given a seed code, it predicts the next code one character at a time, naturally respecting the coding hierarchy.
+
+Compared to a traditional code-level transition matrix (12,309 × 12,309 codes ≈ 290 MB in float16), the character-level formulation is >100× smaller and generalizes across code hierarchies rather than memorizing pairs.
+
+### Visualization use case
+
+The authors pair the model with a generative graph-building algorithm: starting from a seed code, a breadth-first search exhaustively scores candidate following (or preceding) codes by perplexity, keeps the top *k* = 3 at each step, and recurses to depth *n* = 3. The result is a left-to-right graph of the most representative code orderings around a diagnosis (e.g., what typically precedes and follows a sepsis code `A41.9`), generated in ~2 minutes per graph.
+
+## Training Data
+
+Four model variants were trained, one per dataset plus a combined corpus:
+
+| Source | Dataset | # Records | # Patients (weighted) | Notes |
+|---|---|---:|---:|---|
+| MIT | [MIMIC-IV](https://physionet.org/content/mimiciv/) | 430,812 | 430,812 | Beth Israel Deaconess Medical Center; ICD-10-CM, truncated to 4 chars |
+| CDC | NHDS | 756,953 | 111,514,788 | 1996–2010; ICD-9-CM converted to ICD-10-CM via the NBER crosswalk |
+| CDC | NHCS | 123,565 | 33,595,765 | 2020 survey |
+| — | Combined | 1,277,400 | 145,507,435 | Union of the above |
+
+CDC survey records carry a frequency weight *w*; each record is emitted *w* times during training. Only ordered DX-code sequences are required — the framework applies to any corpus with ordered diagnosis codes.
+
+**Note on data access:** MIMIC-IV is a restricted-access dataset requiring credentialed access via PhysioNet. No patient-level data is contained in the model weights beyond learned sequence statistics, but users retraining DgPg must obtain the datasets under their respective terms.
+
+## Evaluation
+
+Performance is measured as fixed-length sequence perplexity (PERF) on the second code of a two-code sequence, averaged over 10 K validation samples.
+
+### Model size ablation
+
+| Model | Layers | Heads | Embd | Size | Loss |
+|---|---:|---:|---:|---:|---:|
+| T11M | 6 | 6 | 384 | 20 MB | 1.16 |
+| **T1.3M** ★ | 3 | 3 | 192 | **2.5 MB** | 1.17 |
+| T230K | 2 | 2 | 96 | 440 KB | 1.19 |
+| T30K | 1 | 1 | 48 | 58 KB | 1.25 |
+
+T1.3M matches the 10×-larger T11M and is the recommended configuration. Model sizes were capped at 10% of the equivalent transition-matrix size to ensure generalizable (not memorized) patterns.
+
+### Cross-dataset transferability
+
+Evaluating each trained model against each dataset shows that models perform best on their own training distribution — evidence that **coding practices are provider- and scenario-specific**. The Combined model performs well across all constituent datasets; the NHDS-only model transfers worst, indicating NHDS coding patterns are the most dataset-unique. MIMIC and NHCS models are the most generalizable.
+
+## Intended Uses
+
+**Direct intended uses**
+
+- Visualizing an organization's representative diagnostic-coding patterns (which codes typically precede/follow a given code; which codes tend to be coded first or last in a visit).
+- Comparing coding practices across institutions, units, or clinical scenarios.
+- Scoring how representative a given code sequence is relative to a corpus (classification use case).
+- Operational insight for administrators: quality control, process improvement, burnout-related workflow analysis.
+
+**Out-of-scope uses**
+
+- ❌ Automated medical coding or coding suggestions for billing.
+- ❌ Clinical diagnosis, prediction, or decision support.
+- ❌ Auditing individual coders or labeling any coding practice as "good" or "bad" — the model summarizes shared practice; it does not evaluate deviations.
+- ❌ Any use that would alter coding to affect claim payment. Modifying codes to increase reimbursement is fraudulent; any coding revision requires proper documentation.
+
+## Limitations
+
+- **4-character truncation.** For cross-dataset comparability, all codes are normalized to 4 characters, discarding finer ICD-10-CM specificity (the framework itself is not limited to 4 characters).
+- **ICD-9 → ICD-10 crosswalk noise.** NHDS codes were converted via the NBER crosswalk, which is not lossless.
+- **Single-visit scope.** DgPg models intra-visit code ordering only, not longitudinal, multi-visit patient trajectories.
+- **Dataset-specific patterns.** A model trained on one provider's records does not transfer reliably to another's — this is by design, but means checkpoints are not general-purpose.
+- **Hallucination.** As a generative model, DgPg can produce plausible-looking but unrepresentative sequences; the authors flag hallucination analysis as future work.
+- **DX codes only.** Procedure and other medical code types are not yet covered.
+
+## Citation
+
+```bibtex
+@inproceedings{hobson2025dgpg,
+  author    = {Tanner Hobson and Jian Huang},
+  title     = {Visualizing Medical Coding Practices Using Transformer Models},
+  booktitle = {Proceedings of the 14th International Conference on Pattern
+               Recognition Applications and Methods (ICPRAM 2025)},
+  pages     = {725--732},
+  year      = {2025},
+  publisher = {SCITEPRESS},
+  doi       = {10.5220/0013257800003905},
+  isbn      = {978-989-758-730-6}
+}
+```
+
+## Acknowledgments
+
+Built on [nanoGPT](https://github.com/karpathy/nanoGPT) by Andrej Karpathy. Data courtesy of MIT-LCP (MIMIC-IV) and the US CDC National Center for Health Statistics (NHDS, NHCS).