| ---
|
| title: Complaint Classifier
|
| emoji: 📝
|
| colorFrom: blue
|
| colorTo: indigo
|
| sdk: docker
|
| app_port: 7860
|
| pinned: false
|
| short_description: Multilingual complaint classifier with runtime labels
|
| ---
|
|
|
| # Complaint Classifier
|
|
|
| A FastAPI app with a web UI that takes a public complaint **in any language**,
|
| detects the language, translates it to English with Google Translate, and
|
| classifies it into a problem type (Water Supply, Electricity, Road, Waste, …)
|
| using a **small pretrained zero-shot model**.
|
|
|
| The labels are **not baked into the model**. You edit them in the UI and the very
|
| next classification uses them — no retraining, no restart.
|
|
|
| ---
|
|
|
| ## Why zero-shot
|
|
|
| A normal fine-tuned text classifier has its labels frozen at training time; adding
|
| "Street Lighting" would mean collecting data and retraining.
|
|
|
| This app uses a **zero-shot NLI classifier** instead. The candidate labels are an
|
| *input* to every inference call. The model checks "does this complaint entail the
|
| hypothesis *This complaint is about broken street lights*?" for each label and
|
| ranks them. That is what makes runtime-editable labels possible.
|
|
|
| Default model: [`typeform/distilbert-base-uncased-mnli`](https://huggingface.co/typeform/distilbert-base-uncased-mnli)
|
| — ~250 MB, CPU-friendly, ~100–400 ms per complaint on a laptop.
|
|
|
| ---
|
|
|
| ## Pipeline
|
|
|
| ```
|
| complaint (any language)
|
| │
|
| ▼
|
| langdetect → ISO code + confidence (offline, instant)
|
| │
|
| ▼
|
| deep-translator → Google Translate → English
|
| (skipped when already confidently English)
|
| │
|
| ▼
|
| zero-shot NLI model → score every ACTIVE label from the database
|
| │
|
| ▼
|
| top label + confidence + full score breakdown → saved to SQLite
|
| ```
|
|
|
| ---
|
|
|
| ## Install & run
|
|
|
| ```bash
|
| pip install -r requirements.txt
|
|
|
| # Windows
|
| run.bat
|
| # macOS / Linux
|
| ./run.sh
|
| # or directly
|
| python -m uvicorn app.main:app --reload
|
| ```
|
|
|
| Then open **http://127.0.0.1:8000**.
|
|
|
| The first start downloads the model (~250 MB) in a background thread, so the UI is
|
| usable immediately — the dot in the top-right shows `loading model…` → `model ready`.
|
|
|
| ---
|
|
|
| ## Pages
|
|
|
| | Page | What it does |
|
| | --- | --- |
|
| | `/` | Submit a complaint, see the predicted category, detected language, English translation and a score bar for **every** label. |
|
| | `/labels` | Add / rename / describe / activate / delete labels. Changes are live instantly. |
|
| | `/history` | Every classified complaint with per-category counts, plus a text filter. |
|
| | `/docs` | Auto-generated OpenAPI docs (Swagger UI). |
|
|
|
| ---
|
|
|
| ## API
|
|
|
| ### `POST /api/classify`
|
|
|
| ```json
|
| {
|
| "text": "අපේ ගමේ දින තුනක් තිස්සේ වතුර නැහැ",
|
| "translate": true,
|
| "multi_label": false,
|
| "threshold": 0.35,
|
| "save": true,
|
| "labels": null
|
| }
|
| ```
|
|
|
| Response:
|
|
|
| ```json
|
| {
|
| "id": 12,
|
| "predicted_label": "Water Supply",
|
| "confidence": 0.8421,
|
| "confident": true,
|
| "threshold": 0.35,
|
| "scores": [
|
| {"label": "Water Supply", "score": 0.8421},
|
| {"label": "Public Health", "score": 0.0612}
|
| ],
|
| "translation": {
|
| "source_lang": "si",
|
| "source_lang_name": "Sinhala",
|
| "detection_confidence": 0.9999,
|
| "was_translated": true,
|
| "original_text": "අපේ ගමේ දින තුනක් තිස්සේ වතුර නැහැ",
|
| "translated_text": "There is no water in our village for three days",
|
| "note": ""
|
| },
|
| "engine": "zero-shot:typeform/distilbert-base-uncased-mnli",
|
| "multi_label": false,
|
| "took_ms": 187
|
| }
|
| ```
|
|
|
| Pass `"labels": ["Water", "Electricity"]` to classify against an ad-hoc set for one
|
| request only, without touching the saved labels.
|
|
|
| ### Labels
|
|
|
| | Method | Path | Purpose |
|
| | --- | --- | --- |
|
| | `GET` | `/api/labels?active_only=true` | List labels |
|
| | `POST` | `/api/labels` | Add `{name, description, active}` |
|
| | `PATCH` | `/api/labels/{id}` | Update any subset of fields |
|
| | `DELETE` | `/api/labels/{id}` | Remove a label |
|
|
|
| ### Other
|
|
|
| `GET /api/history` · `GET /api/stats` · `DELETE /api/history` ·
|
| `GET /api/health` · `POST /api/model/reload`
|
|
|
| ---
|
|
|
| ## Configuration
|
|
|
| All optional, via environment variables:
|
|
|
| | Variable | Default | Meaning |
|
| | --- | --- | --- |
|
| | `MODEL_NAME` | `typeform/distilbert-base-uncased-mnli` | Any HF zero-shot/NLI model |
|
| | `HYPOTHESIS_TEMPLATE` | `This complaint is about {}.` | Steers the NLI model |
|
| | `LEXICAL_WEIGHT` | `0.6` | Weight of the description keyword prior (0 = pure model) |
|
| | `CONFIDENCE_THRESHOLD` | `0.35` | Below this the result is flagged low-confidence |
|
| | `PRELOAD_MODEL` | `1` | Load at startup instead of first request |
|
| | `TRANSLATE_ENABLED` | `1` | Set `0` to skip Google Translate entirely |
|
| | `DATA_DIR` | `./data` | Where `complaints.db` lives |
|
| | `DISABLE_XET` | `0` | Set `1` if model downloads hang at 0 bytes (see below) |
|
| | `MODEL_OFFLINE` | `0` | Set `1` to use only the local HF cache, never the network |
|
| | `MODEL_DOWNLOAD_TIMEOUT` | `30` | Seconds a download chunk may stall before failing |
|
|
|
| Alternatives for `MODEL_NAME` — **benchmark before switching**:
|
|
|
| - `MoritzLaurer/deberta-v3-xsmall-zeroshot-v1.1-all-33` (~146 MB). Far more
|
| confident (0.80 vs 0.24 on the same complaint) but measured at **~41 s per
|
| inference** on the development CPU against distilbert's **~0.4 s**. That is
|
| 100× slower and unusable for real-time intake, so it is not the default
|
| despite the better scores. Needs `sentencepiece` + `protobuf`.
|
| - `valhalla/distilbart-mnli-12-1` (~890 MB)
|
| - `facebook/bart-large-mnli` (~1.6 GB, the reference model)
|
|
|
| Run `python scripts/bench.py <model>` to check both accuracy and latency on your
|
| own hardware — the latency gap above may be specific to this CPU/torch build.
|
|
|
| ---
|
|
|
| ## Getting good accuracy
|
|
|
| **Keep label names short and plain; put the vocabulary in the description.**
|
|
|
| The label *name* is the model's hypothesis ("This complaint is about **Electricity**.")
|
| — NLI models want a clean noun phrase there, so `Electricity` beats
|
| `Electricity / power / outages etc`.
|
|
|
| The *description* is a separate keyword prior blended into the score
|
| (`LEXICAL_WEIGHT`, default `0.6`). This is where the domain vocabulary belongs:
|
|
|
| > `Electricity` → *"power cut, outage, voltage fluctuation, broken street light,
|
| > damaged electric pole or hanging wire"*
|
|
|
| This split matters. Measured on the bundled 18-case benchmark:
|
|
|
| | candidates | accuracy |
|
| | --- | --- |
|
| | descriptions as the hypothesis (the obvious-looking choice) | 0/18 |
|
| | label names, no blend (`LEXICAL_WEIGHT=0`) | 10/18 |
|
| | label names + description prior at `0.6` | **16/18 (89%)** |
|
|
|
| Reproduce with `python scripts/bench.py`.
|
|
|
| Other levers: raise `CONFIDENCE_THRESHOLD` if you would rather route ambiguous
|
| complaints to a human than mislabel them; keep an `Other` label active so
|
| off-topic complaints have somewhere to land; use `multi_label` when a single
|
| complaint can legitimately belong to two categories.
|
|
|
| ---
|
|
|
| ## Troubleshooting
|
|
|
| **The model download hangs at 0 bytes.** Some corporate and ISP networks block
|
| Hugging Face's Xet storage backend while ordinary HTTPS to `huggingface.co` still
|
| works, so the download never starts and never errors. Force the classic CDN path:
|
|
|
| ```bash
|
| # Windows
|
| set DISABLE_XET=1
|
| # macOS / Linux
|
| export DISABLE_XET=1
|
| ```
|
|
|
| This was hit on the development machine — 0 bytes after 15 minutes with Xet, normal
|
| throughput immediately after disabling it.
|
|
|
| **The status dot says `keyword fallback`.** The model failed to load. `GET /api/health`
|
| returns the full error under `model.error`. The app keeps working on the keyword
|
| scorer until you fix it and call `POST /api/model/reload`.
|
|
|
| ---
|
|
|
| ## Behaviour when things are offline
|
|
|
| - **No network for Google Translate** → the complaint is classified in its original
|
| language and the response carries a `note` explaining why. Nothing is lost.
|
| - **Model can't be downloaded** → a keyword-overlap fallback scorer keeps the app
|
| working, and `engine` reports `keyword-fallback` so you always know which path
|
| produced a result.
|
|
|
| ---
|
|
|
| ## Deploying to Hugging Face Spaces
|
|
|
| The repo ships a `Dockerfile` for a Space with `sdk: docker` (set in the README
|
| frontmatter above, together with `app_port: 7860`).
|
|
|
| ```bash
|
| git clone https://huggingface.co/spaces/Arafath10/textclsy
|
| cd textclsy
|
| # copy this project in, then
|
| git add -A && git commit -m "Complaint classifier" && git push
|
| ```
|
|
|
| Pushing needs an HF token with **write** access:
|
|
|
| ```bash
|
| huggingface-cli login # or:
|
| git remote set-url origin https://<user>:<hf_token>@huggingface.co/spaces/Arafath10/textclsy
|
| ```
|
|
|
| Three things differ from the stock HF Docker template, all deliberate:
|
|
|
| 1. **`python:3.11-slim`, not `python:3.9`.** FastAPI and Pydantic evaluate
|
| annotations such as `str | None` at runtime; that syntax is a `TypeError` on 3.9.
|
| 2. **CPU-only torch.** The default PyPI wheel drags in ~2.5 GB of CUDA libraries a
|
| CPU Space can never use, so torch is installed from the PyTorch CPU index first.
|
| 3. **The model is baked into the image** at build time, so a cold start serves the
|
| first request immediately instead of downloading ~250 MB.
|
|
|
| Note that a Space's filesystem is ephemeral — `data/complaints.db` resets when the
|
| Space restarts or rebuilds. Attach a persistent volume, or point `DATA_DIR` at one,
|
| if the complaint history has to survive.
|
|
|
| ---
|
|
|
| ## Layout
|
|
|
| ```
|
| app/
|
| main.py FastAPI routes (pages + JSON API)
|
| classifier.py zero-shot pipeline, keyword fallback, model state
|
| translator.py langdetect + Google Translate, degrades gracefully
|
| store.py SQLite: labels + complaint history
|
| config.py env-var configuration and seed labels
|
| schemas.py Pydantic request/response models
|
| templates/ Jinja2 pages
|
| static/ CSS + vanilla JS (no build step)
|
| data/
|
| complaints.db created on first run
|
| ```
|
|
|
