README.md

metadata

title: Breakdown risk
emoji: 🔧
colorFrom: red
colorTo: gray
sdk: docker
app_port: 7860
short_description: Breakdown risk from the caller's turns
pinned: false

Breakdown risk

Demo of the breakdown-risk classifier: from the caller's turns of an inbound call, the model answers

• risk — the vehicle is probably off the road, the call needs a tow truck or an urgent slot;
• no_risk — the customer can still drive, a normal appointment will do.

Calls are in French, so the transcripts, the examples and the dataset are French; the interface around them is English.

The model

SetFit: the sentence encoder sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 fine-tuned contrastively, plus a logistic regression on the embeddings. Published by train.py to bee2link/breakdown-risk-paraphrase-multilingual-MiniLM-L12-v2, trained on bee2link/breakdown-risk.

The Evaluation tab gives the current score: the test split keeps growing, so no number is copied in here. The demo's examples come from that same split — turns the model never saw during training.

The three-turn window

The model reads only the last three caller turns, and the demo reproduces that windowing: one turn per line, older ones are shown but ignored.

Filtering comes before slicing. Taking the last three messages of a dialogue would count the agent's replies: in an alternating exchange only two caller turns would be left, and after two follow-up questions the complaint itself would fall out of the window — the classifier would see nothing but "no / no".

Model access

The model repo is private. The Space reads it through an HF_TOKEN secret (Settings → Secrets), which needs read access to that repo. Secrets are not visible to visitors of the Space.

Picking the model and the revision

Repos in the organisation whose name starts with breakdown-risk- are listed at startup, and every commit of the chosen repo is offered as a revision — enough to compare two successive training runs without redeploying. ↻ reloads the list after a fresh train.py push.

The list stays sorted by name, so a repo keeps its place as you scan it, but the initial selection is the most recently pushed repo, on main: at startup the demo therefore shows the latest training run, whichever backbone it used. The date of that push is shown under the two menus and follows the selected repo, which makes the default legible. Careful, last_modified applies to the whole repo — a README fix is enough to move the default, and the shown date says so.

Every entry of the revision menu carries its timestamp, main (latest) included, which mirrors the tip commit's — so two runs pushed on the same day stay distinguishable. One format everywhere, %Y-%m-%d %H:%M in Paris time (config.stamp); the Hub reports UTC, and tzdata is a direct dependency so the conversion does not depend on the base image shipping a tz database.

Both outputs of train.py are accepted: the presence of model_head.pkl selects the SetFit path, otherwise the model is loaded as an AutoModelForSequenceClassification.

The Evaluation tab replays the test split and shows the confusion matrix, then the cases one by one, errors first.

Clicking Classify never downloads

Both action buttons start disabled, and the picked revision is fetched up front: on page load, on picking a model, and on picking a commit. While that runs, both buttons read Downloading the model… and are greyed out — they wait on the same weights, so they move together. A status line in the picker panel says the same thing, and Gradio's own progress animation sits on it via show_progress_on, so the feedback appears where the change was made rather than down in a tab.

That line then reports Ready · N ms per call, which is the point of putting it there: the number cannot exist until the weights do, so until then the component is simply loading. It comes from two inference calls at warm time — the first absorbs torch's lazy init, the second is the one measured — so it matches what a click will actually cost instead of reporting the init as if it were latency. For the same reason the ms beside a prediction is timed after load(): inference alone, not a download that happened to be first.

Picking again mid-download takes over. Gradio would otherwise drop the new pick (trigger_mode defaults to "once" everywhere except .change) or queue it behind the running download (concurrency_limit defaults to 1), so both are set explicitly. warm then decides which pick still owns the buttons: it records what the session is waiting on and, if that changed while it was loading, returns without touching anything and leaves the newer pick in charge. The abandoned download does finish in its thread — a blocking hf_hub_download cannot be interrupted — but it only fills the cache, and nothing stale reaches the screen.

Warming runs once per pick. model.change updates the revision menu and then warms, so it reads the revision the switch just chose rather than the previous repo's commit; the revision menu warms on .input, which is user-only, so the programmatic update model.change just made does not warm a second time. A revision that fails to load hands the buttons back and reports why, instead of leaving a dead interface.

⇧ Enter still submits while a model is loading — you can paste a transcript during the download, and an early keypress simply waits on the same load.

Nothing is ever stale

main is resolved to a commit on every call, and the caches are keyed on the commit — not on the branch name. A train.py --push is therefore picked up without a restart: without this, main would forever mean the model loaded on the first click. The test split is resolved the same way, which matters: it went from 46 to 51 cases in two days.

The reported score names both commits (model @ sha · data sha), so a number is always tied to what produced it.

The list of repos and revisions reloads every REFRESH_SECONDS seconds, keeping the current selection, and ↻ forces the reload.

Layout

main.py              startup
app/config.py        constants, display names, timestamp format
app/turns.py         window over the last turns
app/hub.py           repos, revisions, downloads
app/predictors.py    SetFit or AutoModel, depending on the repo's files
app/evaluation.py    test split, accuracy, tables
app/handlers.py      functions the interface calls
app/ui.py            Gradio Blocks
app/cmd_enter.js     ⌘/Ctrl + Enter

Shortcuts

⇧ Enter fires submit natively on a multiline Textbox. ⌘/Ctrl + Enter does not exist in Gradio and goes through app/cmd_enter.js, loaded via launch(js=...) — the method described in the Custom CSS and JS guide.

Dependencies

Docker SDK rather than Gradio, so that versions come from a uv.lock:

uv sync        # local environment, identical to the image's
uv lock        # after any change to pyproject.toml

setfit is not installed. A SetFit model is a SentenceTransformer followed by a pickled logistic regression, and app/predictors.py loads those two pieces — gradio 6 requires transformers>=5, which setfit 1.1 does not support (ImportError: default_logdir). Both paths give the same predictions, to within 6e-8.

torch comes from the CPU index on Linux, which keeps ~2 GB of unused CUDA out of the image.