README.md

metadata

title: Complaint Classifier
emoji: 📝
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
pinned: false
short_description: Multilingual complaint classifier with runtime labels

Complaint Classifier

A FastAPI app with a web UI that takes a public complaint in any language, detects the language, translates it to English with Google Translate, and classifies it into a problem type (Water Supply, Electricity, Road, Waste, …) using a small pretrained zero-shot model.

The labels are not baked into the model. You edit them in the UI and the very next classification uses them — no retraining, no restart.

---

Why zero-shot

A normal fine-tuned text classifier has its labels frozen at training time; adding "Street Lighting" would mean collecting data and retraining.

This app uses a zero-shot NLI classifier instead. The candidate labels are an input to every inference call. The model checks "does this complaint entail the hypothesis This complaint is about broken street lights?" for each label and ranks them. That is what makes runtime-editable labels possible.

Default model: typeform/distilbert-base-uncased-mnli — ~250 MB, CPU-friendly, ~100–400 ms per complaint on a laptop.

---

Pipeline

complaint (any language)
        │
        ▼
  langdetect            → ISO code + confidence (offline, instant)
        │
        ▼
  deep-translator       → Google Translate → English
  (skipped when already confidently English)
        │
        ▼
  zero-shot NLI model   → score every ACTIVE label from the database
        │
        ▼
  top label + confidence + full score breakdown  → saved to SQLite

---

Install & run

pip install -r requirements.txt

# Windows
run.bat
# macOS / Linux
./run.sh
# or directly
python -m uvicorn app.main:app --reload

Then open http://127.0.0.1:8000.

The first start downloads the model (~250 MB) in a background thread, so the UI is usable immediately — the dot in the top-right shows loading model… → model ready.

---

Pages

Page	What it does
/	Submit a complaint, see the predicted category, detected language, English translation and a score bar for every label.
/labels	Add / rename / describe / activate / delete labels. Changes are live instantly.
/history	Every classified complaint with per-category counts, plus a text filter.
/docs	Auto-generated OpenAPI docs (Swagger UI).

---

API

POST /api/classify

{
  "text": "අපේ ගමේ දින තුනක් තිස්සේ වතුර නැහැ",
  "translate": true,
  "multi_label": false,
  "threshold": 0.35,
  "save": true,
  "labels": null
}

Response:

{
  "id": 12,
  "predicted_label": "Water Supply",
  "confidence": 0.8421,
  "confident": true,
  "threshold": 0.35,
  "scores": [
    {"label": "Water Supply", "score": 0.8421},
    {"label": "Public Health", "score": 0.0612}
  ],
  "translation": {
    "source_lang": "si",
    "source_lang_name": "Sinhala",
    "detection_confidence": 0.9999,
    "was_translated": true,
    "original_text": "අපේ ගමේ දින තුනක් තිස්සේ වතුර නැහැ",
    "translated_text": "There is no water in our village for three days",
    "note": ""
  },
  "engine": "zero-shot:typeform/distilbert-base-uncased-mnli",
  "multi_label": false,
  "took_ms": 187
}

Pass "labels": ["Water", "Electricity"] to classify against an ad-hoc set for one request only, without touching the saved labels.

Labels

Method	Path	Purpose
GET	/api/labels?active_only=true	List labels
POST	/api/labels	Add {name, description, active}
PATCH	/api/labels/{id}	Update any subset of fields
DELETE	/api/labels/{id}	Remove a label

Other

GET /api/history · GET /api/stats · DELETE /api/history · GET /api/health · POST /api/model/reload

---

Configuration

All optional, via environment variables:

Variable	Default	Meaning
MODEL_NAME	typeform/distilbert-base-uncased-mnli	Any HF zero-shot/NLI model
HYPOTHESIS_TEMPLATE	This complaint is about {}.	Steers the NLI model
LEXICAL_WEIGHT	0.6	Weight of the description keyword prior (0 = pure model)
CONFIDENCE_THRESHOLD	0.35	Below this the result is flagged low-confidence
PRELOAD_MODEL	1	Load at startup instead of first request
TRANSLATE_ENABLED	1	Set 0 to skip Google Translate entirely
DATA_DIR	./data	Where complaints.db lives
DISABLE_XET	0	Set 1 if model downloads hang at 0 bytes (see below)
MODEL_OFFLINE	0	Set 1 to use only the local HF cache, never the network
MODEL_DOWNLOAD_TIMEOUT	30	Seconds a download chunk may stall before failing

Alternatives for MODEL_NAME — benchmark before switching:

• MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33 (146 MB). Far more confident (0.80 vs 0.24 on the same complaint) but measured at **41 s per inference** on the development CPU against distilbert's ~0.4 s. That is 100× slower and unusable for real-time intake, so it is not the default despite the better scores. Needs sentencepiece + protobuf.
• valhalla/distilbart-mnli-12-1 (~890 MB)
• facebook/bart-large-mnli (~1.6 GB, the reference model)

Run python scripts/bench.py <model> to check both accuracy and latency on your own hardware — the latency gap above may be specific to this CPU/torch build.

---

Getting good accuracy

Keep label names short and plain; put the vocabulary in the description.

The label name is the model's hypothesis ("This complaint is about Electricity.") — NLI models want a clean noun phrase there, so Electricity beats Electricity / power / outages etc.

The description is a separate keyword prior blended into the score (LEXICAL_WEIGHT, default 0.6). This is where the domain vocabulary belongs:

Electricity → "power cut, outage, voltage fluctuation, broken street light, damaged electric pole or hanging wire"

This split matters. Measured on the bundled 18-case benchmark:

candidates	accuracy
descriptions as the hypothesis (the obvious-looking choice)	0/18
label names, no blend (LEXICAL_WEIGHT=0)	10/18
label names + description prior at 0.6	16/18 (89%)

Reproduce with python scripts/bench.py.

Other levers: raise CONFIDENCE_THRESHOLD if you would rather route ambiguous complaints to a human than mislabel them; keep an Other label active so off-topic complaints have somewhere to land; use multi_label when a single complaint can legitimately belong to two categories.

---

Troubleshooting

The model download hangs at 0 bytes. Some corporate and ISP networks block Hugging Face's Xet storage backend while ordinary HTTPS to huggingface.co still works, so the download never starts and never errors. Force the classic CDN path:

# Windows
set DISABLE_XET=1
# macOS / Linux
export DISABLE_XET=1

This was hit on the development machine — 0 bytes after 15 minutes with Xet, normal throughput immediately after disabling it.

The status dot says keyword fallback. The model failed to load. GET /api/health returns the full error under model.error. The app keeps working on the keyword scorer until you fix it and call POST /api/model/reload.

---

Behaviour when things are offline

• No network for Google Translate → the complaint is classified in its original language and the response carries a note explaining why. Nothing is lost.
• Model can't be downloaded → a keyword-overlap fallback scorer keeps the app working, and engine reports keyword-fallback so you always know which path produced a result.

---

Deploying to Hugging Face Spaces

The repo ships a Dockerfile for a Space with sdk: docker (set in the README frontmatter above, together with app_port: 7860).

git clone https://huggingface.co/spaces/Arafath10/textclsy
cd textclsy
# copy this project in, then
git add -A && git commit -m "Complaint classifier" && git push

Pushing needs an HF token with write access:

huggingface-cli login          # or:
git remote set-url origin https://<user>:<hf_token>@huggingface.co/spaces/Arafath10/textclsy

Three things differ from the stock HF Docker template, all deliberate:

1. python:3.11-slim, not python:3.9. FastAPI and Pydantic evaluate annotations such as str | None at runtime; that syntax is a TypeError on 3.9.
2. CPU-only torch. The default PyPI wheel drags in ~2.5 GB of CUDA libraries a CPU Space can never use, so torch is installed from the PyTorch CPU index first.
3. The model is baked into the image at build time, so a cold start serves the first request immediately instead of downloading ~250 MB.

Note that a Space's filesystem is ephemeral — data/complaints.db resets when the Space restarts or rebuilds. Attach a persistent volume, or point DATA_DIR at one, if the complaint history has to survive.

---

Layout

app/
  main.py         FastAPI routes (pages + JSON API)
  classifier.py   zero-shot pipeline, keyword fallback, model state
  translator.py   langdetect + Google Translate, degrades gracefully
  store.py        SQLite: labels + complaint history
  config.py       env-var configuration and seed labels
  schemas.py      Pydantic request/response models
  templates/      Jinja2 pages
  static/         CSS + vanilla JS (no build step)
data/
  complaints.db   created on first run