---
license: mit
language:
- bn
- en
tags:
- text-classification
- intent-classification
- fasttext
- code-mixed
- banglish
- airline
pipeline_tag: text-classification
library_name: fasttext
---

# airline-intent-fasttext

A 19-class intent classifier for airline customer-support messages written in
**code-mixed Bengali / Banglish (romanised Bengali) / English**.

This is the **recommended model** for this domain. It outperforms the DistilBERT
student trained on the same data by **+17 points** on the held-out human-written
test set, while being **20x smaller** and **~230x faster**.

| artifact | size | use |
|---|---|---|
| `intent.ftz` | 6.0 MB | **ship this** — quantized, negligible accuracy cost |
| `intent.bin` | 81.5 MB | full precision, ~1.7 pts better |

## Why fastText beats a transformer here

The corpus is transliterated Banglish, where the same word arrives spelled many
ways — `koto` / `kotoo` / `kt`. fastText's character n-grams (`minn=3, maxn=5`)
share weight across those spellings by construction. A subword-tokenised
transformer has to learn the equivalences from data it does not have enough of.

## Intents (19)

`greeting`, `goodbye`, `thanks`, `flight_status`, `booking_new`, `booking_manage`,
`checkin_boarding`, `baggage_policy`, `baggage_issue`, `refund_compensation`,
`fare_payment`, `airport_info`, `special_assistance`, `loyalty_program`,
`travel_documents`, `disruption`, `complaint`, `agent_request`, `out_of_scope`

`out_of_scope` is the reject class — adjacent-service questions (train tickets,
hotels, visa agencies) that the airline does not handle.

## Usage

```python
import fasttext

from huggingface_hub import hf_hub_download

path = hf_hub_download("Badhon/airline-intent-fasttext", "intent.ftz")
model = fasttext.load_model(path)

labels, probs = model.predict("amar flight ta koto tay chare janaben?", k=3)
# ('__label__flight_status', ...)  0.94

intent = labels[0].removeprefix("__label__")
confidence = probs[0]
```

Inputs should be lowercased and stripped of trailing newlines; fastText treats a
newline as a document boundary.

**Confidence gating.** Route to a human below ~0.6. Two known failure shapes sit
above a 0.5 threshold and will pass it silently:

- adjacent-service questions leak into `booking_new` with high confidence
  (`apnara ki train er ticket o katen` → `booking_new` 0.88, gold `out_of_scope`)
- bare social tokens are low-confidence (`assalamu alaikum` → `greeting` 0.56)

`out_of_scope` recall on the synthetic split is 0.358. Do not rely on the reject
class alone to catch off-domain traffic.

## Evaluation

Two evaluation sets. Rank on **h-test** — the hand-written adversarial holdout,
unseen during any tuning. The synthetic `test` split is generated from the same
templates as training and overstates every model.

| model | test | h-dev | h-test | p50 | size |
|---|---|---|---|---|---|
| **fasttext (quantized)** | 0.772 | 0.869 | **0.729** | 0.03 ms | 6.0 MB |
| fasttext (full) | 0.774 | — | 0.746 | 0.02 ms | 81.5 MB |
| distilbert-int8 (student) | 0.846 | 0.656 | 0.559 | 6.95 ms | 121.6 MB |

Full 120-item holdout accuracy **0.800**, macro-F1 **0.798**.

Note DistilBERT scores *higher* on the synthetic split (0.846) and far lower on
real phrasing (0.559). That divergence is the whole argument for ranking on
h-test.





## Intended use

Front-line intent routing for an airline support bot or IVR, with a confidence
threshold and a human fallback. Not a decision system — it selects a reply
template or a queue, nothing more.