--- title: Complaint Classifier emoji: 📝 colorFrom: blue colorTo: indigo sdk: docker app_port: 7860 pinned: false short_description: Multilingual complaint classifier with runtime labels --- # Complaint Classifier A FastAPI app with a web UI that takes a public complaint **in any language**, detects the language, translates it to English with Google Translate, and classifies it into a problem type (Water Supply, Electricity, Road, Waste, …) using a **small pretrained zero-shot model**. The labels are **not baked into the model**. You edit them in the UI and the very next classification uses them — no retraining, no restart. --- ## Why zero-shot A normal fine-tuned text classifier has its labels frozen at training time; adding "Street Lighting" would mean collecting data and retraining. This app uses a **zero-shot NLI classifier** instead. The candidate labels are an *input* to every inference call. The model checks "does this complaint entail the hypothesis *This complaint is about broken street lights*?" for each label and ranks them. That is what makes runtime-editable labels possible. Default model: [`typeform/distilbert-base-uncased-mnli`](https://huggingface.co/typeform/distilbert-base-uncased-mnli) — ~250 MB, CPU-friendly, ~100–400 ms per complaint on a laptop. --- ## Pipeline ``` complaint (any language) │ ▼ langdetect → ISO code + confidence (offline, instant) │ ▼ deep-translator → Google Translate → English (skipped when already confidently English) │ ▼ zero-shot NLI model → score every ACTIVE label from the database │ ▼ top label + confidence + full score breakdown → saved to SQLite ``` --- ## Install & run ```bash pip install -r requirements.txt # Windows run.bat # macOS / Linux ./run.sh # or directly python -m uvicorn app.main:app --reload ``` Then open **http://127.0.0.1:8000**. The first start downloads the model (~250 MB) in a background thread, so the UI is usable immediately — the dot in the top-right shows `loading model…` → `model ready`. --- ## Pages | Page | What it does | | --- | --- | | `/` | Submit a complaint, see the predicted category, detected language, English translation and a score bar for **every** label. | | `/labels` | Add / rename / describe / activate / delete labels. Changes are live instantly. | | `/history` | Every classified complaint with per-category counts, plus a text filter. | | `/docs` | Auto-generated OpenAPI docs (Swagger UI). | --- ## API ### `POST /api/classify` ```json { "text": "අපේ ගමේ දින තුනක් තිස්සේ වතුර නැහැ", "translate": true, "multi_label": false, "threshold": 0.35, "save": true, "labels": null } ``` Response: ```json { "id": 12, "predicted_label": "Water Supply", "confidence": 0.8421, "confident": true, "threshold": 0.35, "scores": [ {"label": "Water Supply", "score": 0.8421}, {"label": "Public Health", "score": 0.0612} ], "translation": { "source_lang": "si", "source_lang_name": "Sinhala", "detection_confidence": 0.9999, "was_translated": true, "original_text": "අපේ ගමේ දින තුනක් තිස්සේ වතුර නැහැ", "translated_text": "There is no water in our village for three days", "note": "" }, "engine": "zero-shot:typeform/distilbert-base-uncased-mnli", "multi_label": false, "took_ms": 187 } ``` Pass `"labels": ["Water", "Electricity"]` to classify against an ad-hoc set for one request only, without touching the saved labels. ### Labels | Method | Path | Purpose | | --- | --- | --- | | `GET` | `/api/labels?active_only=true` | List labels | | `POST` | `/api/labels` | Add `{name, description, active}` | | `PATCH` | `/api/labels/{id}` | Update any subset of fields | | `DELETE` | `/api/labels/{id}` | Remove a label | ### Other `GET /api/history` · `GET /api/stats` · `DELETE /api/history` · `GET /api/health` · `POST /api/model/reload` --- ## Configuration All optional, via environment variables: | Variable | Default | Meaning | | --- | --- | --- | | `MODEL_NAME` | `typeform/distilbert-base-uncased-mnli` | Any HF zero-shot/NLI model | | `HYPOTHESIS_TEMPLATE` | `This complaint is about {}.` | Steers the NLI model | | `LEXICAL_WEIGHT` | `0.6` | Weight of the description keyword prior (0 = pure model) | | `CONFIDENCE_THRESHOLD` | `0.35` | Below this the result is flagged low-confidence | | `PRELOAD_MODEL` | `1` | Load at startup instead of first request | | `TRANSLATE_ENABLED` | `1` | Set `0` to skip Google Translate entirely | | `DATA_DIR` | `./data` | Where `complaints.db` lives | | `DISABLE_XET` | `0` | Set `1` if model downloads hang at 0 bytes (see below) | | `MODEL_OFFLINE` | `0` | Set `1` to use only the local HF cache, never the network | | `MODEL_DOWNLOAD_TIMEOUT` | `30` | Seconds a download chunk may stall before failing | Alternatives for `MODEL_NAME` — **benchmark before switching**: - `MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33` (~146 MB). Far more confident (0.80 vs 0.24 on the same complaint) but measured at **~41 s per inference** on the development CPU against distilbert's **~0.4 s**. That is 100× slower and unusable for real-time intake, so it is not the default despite the better scores. Needs `sentencepiece` + `protobuf`. - `valhalla/distilbart-mnli-12-1` (~890 MB) - `facebook/bart-large-mnli` (~1.6 GB, the reference model) Run `python scripts/bench.py ` to check both accuracy and latency on your own hardware — the latency gap above may be specific to this CPU/torch build. --- ## Getting good accuracy **Keep label names short and plain; put the vocabulary in the description.** The label *name* is the model's hypothesis ("This complaint is about **Electricity**.") — NLI models want a clean noun phrase there, so `Electricity` beats `Electricity / power / outages etc`. The *description* is a separate keyword prior blended into the score (`LEXICAL_WEIGHT`, default `0.6`). This is where the domain vocabulary belongs: > `Electricity` → *"power cut, outage, voltage fluctuation, broken street light, > damaged electric pole or hanging wire"* This split matters. Measured on the bundled 18-case benchmark: | candidates | accuracy | | --- | --- | | descriptions as the hypothesis (the obvious-looking choice) | 0/18 | | label names, no blend (`LEXICAL_WEIGHT=0`) | 10/18 | | label names + description prior at `0.6` | **16/18 (89%)** | Reproduce with `python scripts/bench.py`. Other levers: raise `CONFIDENCE_THRESHOLD` if you would rather route ambiguous complaints to a human than mislabel them; keep an `Other` label active so off-topic complaints have somewhere to land; use `multi_label` when a single complaint can legitimately belong to two categories. --- ## Troubleshooting **The model download hangs at 0 bytes.** Some corporate and ISP networks block Hugging Face's Xet storage backend while ordinary HTTPS to `huggingface.co` still works, so the download never starts and never errors. Force the classic CDN path: ```bash # Windows set DISABLE_XET=1 # macOS / Linux export DISABLE_XET=1 ``` This was hit on the development machine — 0 bytes after 15 minutes with Xet, normal throughput immediately after disabling it. **The status dot says `keyword fallback`.** The model failed to load. `GET /api/health` returns the full error under `model.error`. The app keeps working on the keyword scorer until you fix it and call `POST /api/model/reload`. --- ## Behaviour when things are offline - **No network for Google Translate** → the complaint is classified in its original language and the response carries a `note` explaining why. Nothing is lost. - **Model can't be downloaded** → a keyword-overlap fallback scorer keeps the app working, and `engine` reports `keyword-fallback` so you always know which path produced a result. --- ## Deploying to Hugging Face Spaces The repo ships a `Dockerfile` for a Space with `sdk: docker` (set in the README frontmatter above, together with `app_port: 7860`). ```bash git clone https://huggingface.co/spaces/Arafath10/textclsy cd textclsy # copy this project in, then git add -A && git commit -m "Complaint classifier" && git push ``` Pushing needs an HF token with **write** access: ```bash huggingface-cli login # or: git remote set-url origin https://:@huggingface.co/spaces/Arafath10/textclsy ``` Three things differ from the stock HF Docker template, all deliberate: 1. **`python:3.11-slim`, not `python:3.9`.** FastAPI and Pydantic evaluate annotations such as `str | None` at runtime; that syntax is a `TypeError` on 3.9. 2. **CPU-only torch.** The default PyPI wheel drags in ~2.5 GB of CUDA libraries a CPU Space can never use, so torch is installed from the PyTorch CPU index first. 3. **The model is baked into the image** at build time, so a cold start serves the first request immediately instead of downloading ~250 MB. Note that a Space's filesystem is ephemeral — `data/complaints.db` resets when the Space restarts or rebuilds. Attach a persistent volume, or point `DATA_DIR` at one, if the complaint history has to survive. --- ## Layout ``` app/ main.py FastAPI routes (pages + JSON API) classifier.py zero-shot pipeline, keyword fallback, model state translator.py langdetect + Google Translate, degrades gracefully store.py SQLite: labels + complaint history config.py env-var configuration and seed labels schemas.py Pydantic request/response models templates/ Jinja2 pages static/ CSS + vanilla JS (no build step) data/ complaints.db created on first run ```