README.md

metadata

title: CognitivePulse
emoji: 🔬
colorFrom: indigo
colorTo: blue
sdk: streamlit
sdk_version: 1.38.0
python_version: '3.11'
app_file: app.py
pinned: false

CognitivePulse

Biomarker Intelligence & Coaching Assistant — Preventive Brain Health Platform

CognitivePulse is an end-to-end ML + RAG system that mirrors the core technical pipeline of a preventive brain health product: biomarker intake → risk stratification → personalised intervention priority ranking → grounded coaching brief → patient Q&A chatbot grounded in the individual's specific profile.

Live demo: add your HuggingFace Spaces link here after deploying Author: Kshamaa

---

Disclaimer

This is a research and engineering demonstration prototype, not a validated clinical or diagnostic tool. Model outputs are for illustrative purposes only. Any real deployment in a health context would require clinical validation, regulatory review, IRB-approved studies, and licensed clinical oversight.

---

Overview

Stage	What it does
📊 Population Dashboard	XGBoost global feature importance, diagnosis-stratified distributions, correlation matrix across the 2,149-patient dataset
👤 Patient Risk Assessment	Per-patient risk score (0–100) with SHAP waterfall attribution and a risk gauge, entered via a structured human-readable intake form
🎯 Intervention Priority Engine	Modifiability-weighted ranking of actionable risk factors: priority score = |SHAP contribution| × domain actionability weight
🤖 AI Coaching Brief	RAG-generated, citation-grounded coaching summary retrieved from a curated 14-paper corpus, with a RAGAS-style faithfulness evaluation
💬 Patient Q&A	Multi-turn chatbot grounded in the patient's specific biomarker profile and risk score — answers questions about their results in plain language

---

Dataset

El Kharoua, R. (2024). Alzheimer's Disease Dataset. DOI: 10.34740/KAGGLE/DSV/8668279

• 2,149 patients, 33 features, binary diagnosis outcome (35.4% positive rate)
• Features span demographics, lifestyle, medical history, clinical measurements, cognitive assessments, and symptom indicators
• License: CC BY 4.0

The dataset covers the same feature categories as a BetterBrain-style preventive assessment: cholesterol panel, blood pressure, BMI, sleep quality, physical activity, diet quality, APOE family history, MMSE cognitive score, diabetes, hypertension, and depression.

A synthetic fallback (statistically matched to published feature distributions) is used automatically if the Kaggle credentials are not configured.

---

Architecture

data_loader.py
   → Kaggle API download (primary) / local file / synthetic fallback
   → Feature metadata, reference ranges, population statistics

risk_model.py
   → XGBoost classifier (5-fold stratified CV; AUC, F1 evaluation)
   → SHAP TreeExplainer for per-patient feature attribution
   → Model cached to data/model.pkl for fast inference at serve time

intervention_engine.py
   → Filters to modifiable features at adverse levels
   → Priority score = |SHAP contribution| × domain actionability weight
   → Domain mapping to literature tags for RAG retrieval
   → Generates structured coach brief as RAG context

rag_engine.py
   → 14-entry curated literature corpus (data/corpus.json)
   → Embedding backend: intfloat/e5-large-v2 + FAISS (TF-IDF fallback)
   → Coaching generation: openai/gpt-oss-120b via Groq inference API
   → Faithfulness evaluation: RAGAS-style claim-level verdict + score

app.py
   → 5-tab Streamlit interface
   → Plotly visualisations: feature importance bar, SHAP waterfall, risk gauge, correlation matrix
   → Multi-turn patient chatbot with profile-grounded system prompt

---

Patient Intake Form

The risk assessment form covers six sections — Demographics, Lifestyle, Medical History, Clinical Measurements, Cognitive & Functional, and Symptoms — with human-readable labels throughout:

• Gender: Male / Female / Prefer to self-identify (note: the underlying dataset uses a binary encoding; "Prefer to self-identify" maps to Male for model computation, which is a known limitation of the current dataset)
• Ethnicity: Caucasian / African American / Asian / Other
• Education Level: No formal education / High school / Bachelor's degree / Higher degree
• All Yes/No fields (Smoking, Hypertension, Family History, etc.) displayed as plain language
• All continuous fields (BP, cholesterol, BMI, MMSE) as numeric inputs with reference-range bounds

---

Patient Q&A Chatbot

The Q&A tab is a multi-turn conversational assistant that activates after the risk assessment is completed. The system prompt is dynamically constructed from the patient's actual profile — including their risk score, band, key biomarker values, top SHAP drivers, and prioritised intervention areas — so every response is grounded in their specific data rather than generic health advice.

Key design decisions:

• Profile-grounded context: the model sees the patient's actual numbers and refers to them directly (e.g. "Your LDL of 158 is above the optimal range...")
• Suggested starter questions are shown when the chat is empty to reduce friction
• Non-diagnostic framing: the system prompt explicitly prohibits diagnosis language and instructs the model to redirect medical decisions to a qualified clinician
• Same inference backend as the coaching brief (openai/gpt-oss-120b via Groq) — no additional credentials required
• Full conversation history is maintained in Streamlit session state across turns

---

Literature Corpus

data/corpus.json contains 14 entries from peer-reviewed sources covering:

Domain	Key sources
Cardiovascular risk	SPRINT MIND trial (JAMA, 2019); Lancet Commission 2024 (14 modifiable risk factors)
LDL/cholesterol	Systematic review, Journal of Neurology, 2025
Sleep & glymphatic	Xie et al., Science, 2013; Nature Reviews Neuroscience, 2025
Exercise	Exercise + BDNF + NfL review, MDPI, 2025
Diet	MIND diet (Morris et al., 2015; NEJM RCT, 2023)
Homocysteine / B vitamins	Clarke et al., NEJM, 2010; meta-analysis update, 2025
Diabetes / insulin	Type 3 diabetes review, 2025
Mental health / social	Lancet Public Health network meta-analysis, 2025
Smoking	Lancet Commission 2024; Zhong et al., Archives of Internal Medicine, 2015
Alcohol	Rehm & Shield, Neuropsychology Review, 2019
Multidomain lifestyle	FINGER trial (Lancet, 2015); APOE4 meta-analysis (PMC12726239, 2025)
Blood biomarkers	p-tau217 review (JAMA, 2023)

---

Risk Model

XGBoost with 5-fold stratified cross-validation. Key hyperparameters:

n_estimators=300, max_depth=5, learning_rate=0.05,
subsample=0.8, colsample_bytree=0.8, scale_pos_weight=1.83

Performance on the Kaggle dataset (2,149 samples):

• CV AUC: reported at runtime
• CV F1: reported at runtime

SHAP TreeExplainer provides per-patient feature attribution. The waterfall chart shows which biomarkers most increased or decreased the predicted risk score for that specific individual — not population-level importance.

On GPU: pass device="cuda" to train_model() in risk_model.py to use XGBoost's GPU-accelerated histogram method.

---

Setup

Local

pip install -r requirements.txt
export GROQ_API_KEY=your_key_here
export KAGGLE_USERNAME=your_username
export KAGGLE_KEY=your_kaggle_api_key
streamlit run app.py

A Groq API key is required for Tabs 4 and 5 (coaching brief and patient chatbot): create one at console.groq.com under API Keys.

Kaggle credentials are optional: if not set, the app runs on synthetic data. To use the real dataset: create a Kaggle account and generate an API key at kaggle.com/settings/account.

HuggingFace Spaces

1. Create a new Space (SDK: Streamlit, Hardware: CPU basic).
2. Push all project files.
3. Add GROQ_API_KEY as a Space secret under Settings → Variables and secrets.
4. Optionally add KAGGLE_USERNAME and KAGGLE_KEY to download the real dataset.

---

Known Limitations

• The Kaggle dataset does not include blood-based Alzheimer's biomarkers such as p-tau217, Aβ42/40 ratio, NfL, or GFAP — which are the most sensitive early markers in the clinical literature and the core differentiator of products like BetterBrain. The model uses the 33 clinical and lifestyle features available, covering metabolic, cardiovascular, and lifestyle risk categories.
• Gender is encoded as a binary feature in the dataset (Male=0, Female=1). The "Prefer to self-identify" option maps to Male internally, which is a limitation of the current dataset encoding.
• SHAP values are computed on the synthetic dataset in fallback mode; absolute magnitudes will differ on the real Kaggle data.
• The XGBoost model is not calibrated (no Platt scaling or isotonic regression); the risk score should be interpreted as a relative ranking tool rather than an absolute probability estimate.
• The chatbot and coaching models (openai/gpt-oss-120b via Groq) are general-purpose open-weight models, not fine-tuned on clinical or coaching text.

Future Work

• Integration of real blood biomarker panels (p-tau217, NfL, GFAP, Aβ42/40) when data access is available
• Model calibration for better-behaved probability outputs
• Longitudinal tracking: 3- and 6-month re-assessment trajectories with trend charts
• Fine-tuned coaching and Q&A models using curated clinical coaching transcripts
• LIME explanations as an alternative to SHAP for non-tree models
• Inclusive gender encoding when datasets with non-binary gender data become available