README.md

---
title: CognitivePulse
emoji: 🔬
colorFrom: indigo
colorTo: blue
sdk: streamlit
sdk_version: "1.38.0"
python_version: "3.11"
app_file: app.py
pinned: false
---

# CognitivePulse

### Biomarker Intelligence & Coaching Assistant — Preventive Brain Health Platform

CognitivePulse is an end-to-end ML + RAG system that mirrors the core technical pipeline
of a preventive brain health product: biomarker intake → risk stratification → personalised
intervention priority ranking → grounded coaching brief → patient Q&A chatbot grounded
in the individual's specific profile.

**Live demo:** _add your HuggingFace Spaces link here after deploying_
**Author:** Kshamaa

---

## Disclaimer

This is a research and engineering demonstration prototype, not a validated clinical or
diagnostic tool. Model outputs are for illustrative purposes only. Any real deployment
in a health context would require clinical validation, regulatory review, IRB-approved
studies, and licensed clinical oversight.

---

## Overview

| Stage | What it does |
|---|---|
| **📊 Population Dashboard** | XGBoost global feature importance, diagnosis-stratified distributions, correlation matrix across the 2,149-patient dataset |
| **👤 Patient Risk Assessment** | Per-patient risk score (0–100) with SHAP waterfall attribution and a risk gauge, entered via a structured human-readable intake form |
| **🎯 Intervention Priority Engine** | Modifiability-weighted ranking of actionable risk factors: priority score = \|SHAP contribution\| × domain actionability weight |
| **🤖 AI Coaching Brief** | RAG-generated, citation-grounded coaching summary retrieved from a curated 14-paper corpus, with a RAGAS-style faithfulness evaluation |
| **💬 Patient Q&A** | Multi-turn chatbot grounded in the patient's specific biomarker profile and risk score — answers questions about their results in plain language |

---

## Dataset

**El Kharoua, R. (2024). Alzheimer's Disease Dataset.**
DOI: [10.34740/KAGGLE/DSV/8668279](https://doi.org/10.34740/KAGGLE/DSV/8668279)

- 2,149 patients, 33 features, binary diagnosis outcome (35.4% positive rate)
- Features span demographics, lifestyle, medical history, clinical measurements,
  cognitive assessments, and symptom indicators
- License: CC BY 4.0

The dataset covers the same feature categories as a BetterBrain-style preventive
assessment: cholesterol panel, blood pressure, BMI, sleep quality, physical activity,
diet quality, APOE family history, MMSE cognitive score, diabetes, hypertension, and depression.

A synthetic fallback (statistically matched to published feature distributions) is used
automatically if the Kaggle credentials are not configured.

---

## Architecture

```
data_loader.py
   → Kaggle API download (primary) / local file / synthetic fallback
   → Feature metadata, reference ranges, population statistics

risk_model.py
   → XGBoost classifier (5-fold stratified CV; AUC, F1 evaluation)
   → SHAP TreeExplainer for per-patient feature attribution
   → Model cached to data/model.pkl for fast inference at serve time

intervention_engine.py
   → Filters to modifiable features at adverse levels
   → Priority score = |SHAP contribution| × domain actionability weight
   → Domain mapping to literature tags for RAG retrieval
   → Generates structured coach brief as RAG context

rag_engine.py
   → 14-entry curated literature corpus (data/corpus.json)
   → Embedding backend: intfloat/e5-large-v2 + FAISS (TF-IDF fallback)
   → Coaching generation: openai/gpt-oss-120b via Groq inference API
   → Faithfulness evaluation: RAGAS-style claim-level verdict + score

app.py
   → 5-tab Streamlit interface
   → Plotly visualisations: feature importance bar, SHAP waterfall, risk gauge, correlation matrix
   → Multi-turn patient chatbot with profile-grounded system prompt
```

---

## Patient Intake Form

The risk assessment form covers six sections — Demographics, Lifestyle, Medical History,
Clinical Measurements, Cognitive & Functional, and Symptoms — with human-readable labels
throughout:

- **Gender:** Male / Female / Prefer to self-identify *(note: the underlying dataset uses
  a binary encoding; "Prefer to self-identify" maps to Male for model computation, which
  is a known limitation of the current dataset)*
- **Ethnicity:** Caucasian / African American / Asian / Other
- **Education Level:** No formal education / High school / Bachelor's degree / Higher degree
- All Yes/No fields (Smoking, Hypertension, Family History, etc.) displayed as plain language
- All continuous fields (BP, cholesterol, BMI, MMSE) as numeric inputs with reference-range bounds

---

## Patient Q&A Chatbot

The Q&A tab is a multi-turn conversational assistant that activates after the risk
assessment is completed. The system prompt is dynamically constructed from the patient's
actual profile — including their risk score, band, key biomarker values, top SHAP drivers,
and prioritised intervention areas — so every response is grounded in their specific data
rather than generic health advice.

Key design decisions:
- **Profile-grounded context:** the model sees the patient's actual numbers and refers to
  them directly (e.g. "Your LDL of 158 is above the optimal range...")
- **Suggested starter questions** are shown when the chat is empty to reduce friction
- **Non-diagnostic framing:** the system prompt explicitly prohibits diagnosis language
  and instructs the model to redirect medical decisions to a qualified clinician
- **Same inference backend** as the coaching brief (openai/gpt-oss-120b via Groq) —
  no additional credentials required
- **Full conversation history** is maintained in Streamlit session state across turns

---

## Literature Corpus

`data/corpus.json` contains 14 entries from peer-reviewed sources covering:

| Domain | Key sources |
|---|---|
| Cardiovascular risk | SPRINT MIND trial (JAMA, 2019); Lancet Commission 2024 (14 modifiable risk factors) |
| LDL/cholesterol | Systematic review, Journal of Neurology, 2025 |
| Sleep & glymphatic | Xie et al., Science, 2013; Nature Reviews Neuroscience, 2025 |
| Exercise | Exercise + BDNF + NfL review, MDPI, 2025 |
| Diet | MIND diet (Morris et al., 2015; NEJM RCT, 2023) |
| Homocysteine / B vitamins | Clarke et al., NEJM, 2010; meta-analysis update, 2025 |
| Diabetes / insulin | Type 3 diabetes review, 2025 |
| Mental health / social | Lancet Public Health network meta-analysis, 2025 |
| Smoking | Lancet Commission 2024; Zhong et al., Archives of Internal Medicine, 2015 |
| Alcohol | Rehm & Shield, Neuropsychology Review, 2019 |
| Multidomain lifestyle | FINGER trial (Lancet, 2015); APOE4 meta-analysis (PMC12726239, 2025) |
| Blood biomarkers | p-tau217 review (JAMA, 2023) |

---

## Risk Model

XGBoost with 5-fold stratified cross-validation. Key hyperparameters:

```python
n_estimators=300, max_depth=5, learning_rate=0.05,
subsample=0.8, colsample_bytree=0.8, scale_pos_weight=1.83
```

Performance on the Kaggle dataset (2,149 samples):
- CV AUC: reported at runtime
- CV F1: reported at runtime

SHAP TreeExplainer provides per-patient feature attribution. The waterfall chart
shows which biomarkers most increased or decreased the predicted risk score for
that specific individual — not population-level importance.

On GPU: pass `device="cuda"` to `train_model()` in `risk_model.py` to use XGBoost's
GPU-accelerated histogram method.

---

## Setup

### Local

```bash
pip install -r requirements.txt
export GROQ_API_KEY=your_key_here
export KAGGLE_USERNAME=your_username
export KAGGLE_KEY=your_kaggle_api_key
streamlit run app.py
```

A Groq API key is required for Tabs 4 and 5 (coaching brief and patient chatbot):
create one at [console.groq.com](https://console.groq.com) under API Keys.

Kaggle credentials are optional: if not set, the app runs on synthetic data.
To use the real dataset: create a Kaggle account and generate an API key at
[kaggle.com/settings/account](https://www.kaggle.com/settings/account).

### HuggingFace Spaces

1. Create a new Space (SDK: Streamlit, Hardware: CPU basic).
2. Push all project files.
3. Add `GROQ_API_KEY` as a Space secret under **Settings → Variables and secrets**.
4. Optionally add `KAGGLE_USERNAME` and `KAGGLE_KEY` to download the real dataset.

---

## Known Limitations

- The Kaggle dataset does not include blood-based Alzheimer's biomarkers such as
  p-tau217, Aβ42/40 ratio, NfL, or GFAP — which are the most sensitive early markers
  in the clinical literature and the core differentiator of products like BetterBrain.
  The model uses the 33 clinical and lifestyle features available, covering metabolic,
  cardiovascular, and lifestyle risk categories.
- Gender is encoded as a binary feature in the dataset (Male=0, Female=1). The
  "Prefer to self-identify" option maps to Male internally, which is a limitation
  of the current dataset encoding.
- SHAP values are computed on the synthetic dataset in fallback mode; absolute
  magnitudes will differ on the real Kaggle data.
- The XGBoost model is not calibrated (no Platt scaling or isotonic regression);
  the risk score should be interpreted as a relative ranking tool rather than an
  absolute probability estimate.
- The chatbot and coaching models (openai/gpt-oss-120b via Groq) are general-purpose
  open-weight models, not fine-tuned on clinical or coaching text.

## Future Work

- Integration of real blood biomarker panels (p-tau217, NfL, GFAP, Aβ42/40)
  when data access is available
- Model calibration for better-behaved probability outputs
- Longitudinal tracking: 3- and 6-month re-assessment trajectories with trend charts
- Fine-tuned coaching and Q&A models using curated clinical coaching transcripts
- LIME explanations as an alternative to SHAP for non-tree models
- Inclusive gender encoding when datasets with non-binary gender data become available