| ---
|
| license: apache-2.0
|
| base_model: distilbert-base-uncased
|
| tags:
|
| - text-classification
|
| - autonomous-vehicles
|
| pipeline_tag: text-classification
|
| ---
|
|
|
| # drivesignal-distilbert
|
|
|
| Fine-tuned `distilbert-base-uncased` that classifies free-text descriptions of
|
| autonomous-vehicle disengagement events into a 10-category safety scenario
|
| taxonomy. Trained on real California DMV Autonomous Vehicle Disengagement
|
| Reports (2022–2024).
|
|
|
| Part of [DriveSignal](https://github.com/jahnavidanda02/drivesignal), a
|
| project comparing a rule-based heuristic, this fine-tuned DistilBERT
|
| classifier, and a LoRA-fine-tuned Qwen3-1.7B on the same task.
|
|
|
| ## Taxonomy
|
|
|
| ```
|
| Perception Failure | Prediction Failure | Lane Keeping | Braking Behavior |
|
| Unwanted Maneuver | Construction/Environment | Precautionary |
|
| System/Hardware Fault | Localization/Mapping | Other
|
| ```
|
|
|
| ## Training data
|
|
|
| - 14,800 disengagement event descriptions from the CA DMV 2022–2024 reports
|
| (Waymo and other manufacturers).
|
| - **Weak-labeled**: training labels come from a keyword/regex heuristic
|
| classifier, not human annotation — a standard weak-supervision pattern for
|
| bootstrapping labels when ground truth doesn't exist. See training script:
|
| [`src/finetune_distilbert.py`](https://github.com/jahnavidanda02/drivesignal/blob/master/src/finetune_distilbert.py).
|
|
|
| ## Evaluation
|
|
|
| Evaluated against a 250-row hand-labeled gold set (not seen during training,
|
| and not derived from the heuristic labels):
|
|
|
| | Model | Accuracy (n=250) |
|
| |---|---|
|
| | Heuristic baseline | 80.0% |
|
| | **DistilBERT (this model)** | **80.0%** |
|
| | Qwen3-1.7B LoRA | 72.8% |
|
|
|
| Full breakdown: [`src/eval_distilbert_gold.py`](https://github.com/jahnavidanda02/drivesignal/blob/master/src/eval_distilbert_gold.py).
|
|
|
| ## Usage
|
|
|
| ```python
|
| from transformers import pipeline
|
|
|
| clf = pipeline("text-classification", model="jahnavidanda02/drivesignal-distilbert")
|
| clf("Vehicle disengaged after hesitating at an unprotected left turn due to "
|
| "misjudging the trajectory of an oncoming vehicle.")
|
| ```
|
|
|
| ## Limitations
|
|
|
| - Trained on weak (heuristic-derived) labels for the bulk of the data, so it
|
| inherits gaps in the heuristic's keyword coverage (e.g. rare categories
|
| like *Construction/Environment* and *Precautionary* are underrepresented).
|
| - Trained only on CA DMV disengagement report language; may not generalize
|
| to differently phrased AV incident text.
|
|
|
