README.md

---
title: PromptStat
emoji: 🎮
colorFrom: gray
colorTo: green
sdk: gradio
sdk_version: 6.6.0
app_file: space_app.py
pinned: false
license: apache-2.0
short_description: Rate your AI-collaboration skill — MiniCPM-8B + LoRA
tags:
  - thousand-token-wood
  - minicpm
  - modal
  - off-brand
  - track:wood
  - sponsor:openbmb
  - sponsor:modal
---

# 🎮 PromptStat

Upload your **ChatGPT or Claude export** (or paste a conversation / share link) and get an esports-style
**stat card** rating *how you collaborate with AI* across 5 observable axes:

**Focus · Technique · Critical Engagement · Interaction · Input Quality** → an overall score + tier (D→S).

> 🔗 **Live demo:** [demo link — TODO]  ·  🐦 **Write-up:** [social link — TODO]

## What makes it real
- **Scored by MiniCPM4.1-8B** (OpenBMB) using a validated observable-detection method — not vibes.
  The model is **served on Modal** (vLLM on an A100-40GB, the 4 LoRA adapters loaded warm and addressable
  by name). The card tags every score "✓ scored on MiniCPM-8B + LoRA hybrid"; if no model endpoint is
  configured it falls back to a transparent heuristic and says so (amber tag).
- **Per-category LoRA hybrid** (trained **on Modal**, H100): the 8B is fine-tuned only where it makes
  correctable errors — `decomposition` 0.26→**0.66 κ**, `critical·source_request` 0.47→**0.66**, plus
  `independent_verification` and `interaction` — and stays base everywhere it's already strong. Each adapter
  overrides only its own feature, so there's no catastrophic forgetting.
- **Honest validation**: every axis ships its measured **Cohen's κ** vs a 159-conversation human-labeled
  set. We document where the 8B hits its ceiling (it can't out-agree the human rater) instead of inflating.

## Two ways in
- **Upload export** — full profile across your whole history (representative 30-conversation sample, every
  turn within each scored; set `UI_MAX_CONVS=0` for the entire history).
- **Paste a conversation or a ChatGPT/Claude share link** — fast single-conversation sample for quick
  feedback / demo. Clearly labelled "Sample analysis — not your overall AI usage."

## Privacy
Your chat content is parsed **in memory only** — never written to disk, never logged. A pasted share link is
fetched once to read the conversation and is likewise never stored.

## Architecture (Modal end-to-end)
- **Inference:** `modal_serve_minicpm.py` — vLLM serves base MiniCPM4.1-8B **plus** all 4 LoRA adapters from a
  single warm A100; the app routes each axis to its adapter by model name over HTTP. Model weights are baked
  into the image so container starts don't touch the Hub.
- **Training:** `modal_train_lora.py` / `modal_eval_lora.py` — the LoRA adapters were trained and evaluated on
  Modal GPUs against the human-labeled validation set.

## Run locally
```bash
pip install -r requirements.txt
# real scoring needs a MiniCPM endpoint (auto-loaded from eval/.secrets.env if present):
export OPENBMB_BASE_URL=...   # OpenBMB free API, or your Modal vLLM URL
export OPENBMB_TOKEN=...
python -m ui.app              # or: python space_app.py   (or ./run_demo.sh for the LoRA hybrid)
python -m pytest ui/tests -q  # 19 UI tests
```
Without the env vars the app still runs (heuristic fallback, amber tag). See `DEPLOY.md` for the Space deploy
+ Modal model-serving + speed tuning, and `handoff_to_card.md` for the `card_data` schema.

## Built for the OpenBMB "Build Small" hackathon
Models ≤32B, English. Uses **MiniCPM4.1-8B** (reasoning) — eligible model — with **Modal** for both LoRA
training and warm inference. Maintainer notes: backend/ML workstream in `prompt_card/`, `modal_*.py`,
`SPEC.md`; UI shell in `ui/`.