CASE_STUDY.md

Case Study — NephroScreen

An end-to-end applied machine-learning project: screening for chronic kidney disease from routine lab values, built and analysed solo.

---

At a glance

Problem	Binary classification — predict CKD vs. healthy from routine clinical/lab features
Data	UCI Chronic Kidney Disease dataset — 400 patients, 24 attributes, ~11% cells missing
Best model	Random Forest — 100% test accuracy, ROC-AUC 1.0000 (read the caveat below)
Stack	scikit-learn · FastAPI · vanilla JS frontend · Docker/Render
What makes it mature	Leakage-safe pipeline · SHAP interpretability · calibration · screening-tuned threshold · tests

1. The problem

Chronic kidney disease often progresses silently until it is advanced, yet the warning signs are already present in routine blood and urine tests. The task: given a patient's lab panel, flag likely CKD early enough to matter. This is a classic screening problem, which shapes every later decision — especially how errors are weighted.

2. The data — and an honest catch

The UCI CKD dataset is small (400 rows) and the two classes are highly separable on a few biomarkers. Every model I trained scored in the high 90s, and Random Forest reached 100% on the held-out test set.

That number is a red flag, not a trophy. With an 80-patient test set drawn from one clean, single-source dataset, near-perfect accuracy reflects the separability of this data, not real-world clinical performance. I treat it that way throughout — the engineering value here is doing the workflow correctly, not chasing a headline metric.

3. Approach

Leakage-safe preprocessing. All imputation, encoding, and feature engineering live in a custom scikit-learn transformer (CKDPreprocessor) wrapped in a Pipeline. Every statistic — medians, modes, the scaler, the age_group fill value — is fit on the training fold only, so cross-validation and the live API never see test-set information.

Domain feature engineering. Four clinically motivated features, e.g. kidney_stress_index = serum_creatinine × blood_urea / hemoglobin (the CKD triad in one number). SHAP later confirmed it earns its place in the top five.

Three models compared with 10-fold stratified CV:

Model	Accuracy	Recall	F1	ROC-AUC	10-Fold CV
Logistic Regression	98.75%	98%	98.99%	1.0000	99.38% ± 1.25%
Random Forest	100%	100%	100%	1.0000	98.75% ± 2.86%
Decision Tree	97.50%	98%	98.00%	0.9893	96.56% ± 2.19%

4. Interpretability — what the model actually uses

SHAP attributions are clinically coherent and build trust in the model:

• Low hemoglobin and PCV push strongly toward CKD (anemia of kidney disease).
• High serum creatinine and the engineered kidney_stress_index are top drivers.
• Hypertension and diabetes = "yes" push toward CKD — the two leading real-world causes.

This is the difference between "the model is accurate" and "I understand why the model is accurate" — the latter is what makes a result defensible.

5. Maturity signals

Calibration. Do the probabilities mean what they say? The reliability curve checks whether an "0.8" prediction is CKD ~80% of the time — essential before anyone acts on a probability.

A screening-tuned threshold. For screening, a missed CKD case (false negative) is far costlier than a false alarm. Instead of the default 0.5, the API serves a threshold tuned to keep recall ≥ 0.99, accepting more false positives as the safer trade.

6. From model to product

The trained pipeline is persisted with joblib and served by a FastAPI backend; a dependency-free web frontend collects a lab panel (missing fields are imputed) and shows the probability, a risk band, and which entered values fall outside normal clinical ranges. Containerised for free-tier deployment.

Browser → FastAPI /api/predict → joblib pipeline (CKDPreprocessor → RandomForest)

7. Limitations & what I'd do next

• Single small dataset → results will not transfer to real clinical populations without external validation on an independent cohort.
• No serving-time calibration yet; probabilities should be calibrated (e.g. isotonic) before being shown as clinical likelihoods.
• Next experiment: find a minimal, low-cost feature subset — how much accuracy is lost if the expensive labs are dropped? That's the question with real value for screening in resource-limited settings.

---

Built solo by Sadiq Mansoor. Dataset: UCI ML Repository — Chronic Kidney Disease (ID 336). Educational project, not a medical device.