README.md

metadata

title: Vertical SubverticalClassifier
emoji: 👀
colorFrom: green
colorTo: gray
sdk: gradio
sdk_version: 6.20.0
python_version: '3.12'
app_file: app.py
pinned: false

Hybrid upgrade: TF-IDF + MiniLM embeddings -> logistic regression

Adds semantic understanding so cases like "Barbers" land near beauty, not books. Same app, same output columns, same thresholds — only the model behind it changes.

What each file is

• train_hybrid.py — trains the hybrid, saves hybrid_models.joblib. Run on GPU + internet.
• inference_hybrid.py — drop-in replacement for inference.py (same classify() output).
• requirements_hybrid.txt — adds sentence-transformers; use this on the Space.

Step 1 — train (Google Colab, free GPU, has HF access)

1. New Colab notebook, Runtime → change type → GPU.
2. Upload Raw_data_vertical_subvertical.xlsx and train_hybrid.py.
3. Run:

   !pip install -q scikit-learn==1.8.0 sentence-transformers openpyxl
   !python train_hybrid.py
   

   (Pinning scikit-learn==1.8.0 is required so the saved model loads on the Space.)
4. It prints held-out accuracy and writes hybrid_models.joblib. Download that file. (emb_all.npy is just a cache — you don't need to upload it. Keep it if you want to re-tune EMB_W later without re-embedding.)

Step 2 — deploy to the Space

1. Upload hybrid_models.joblib and inference_hybrid.py.
2. In app.py, change the one import line: from inference import classify, VERT_THR, SUB_THR -> from inference_hybrid import classify, VERT_THR, SUB_THR
3. Replace requirements.txt with requirements_hybrid.txt (rename it to requirements.txt).
4. Commit. First build is slower (it downloads the embedding model, ~470 MB).

You can delete the old inference.py, vertical_model.joblib, and subvertical_model.joblib once the hybrid is working — they're no longer used.

Tuning knob

EMB_W in train_hybrid.py (default 1.4) balances the two signals: higher = more semantic (helps barbers-type cases), lower = more lexical (closer to the old behavior). Because embeddings are cached in emb_all.npy, re-running with a new EMB_W is fast — no re-embedding.

Notes

• Inference embeds on CPU by default (fast enough for single names). Bulk uploads of many thousands of rows will be slower than the pure-TF-IDF version — that's the cost of semantics.
• Keep the scikit-learn==1.8.0 pin identical in Colab and on the Space, or the pickle won't load.