---
license: apache-2.0
base_model: distilbert-base-uncased
tags:
  - text-classification
  - autonomous-vehicles
pipeline_tag: text-classification
---

# drivesignal-distilbert

Fine-tuned `distilbert-base-uncased` that classifies free-text descriptions of
autonomous-vehicle disengagement events into a 10-category safety scenario
taxonomy. Trained on real California DMV Autonomous Vehicle Disengagement
Reports (2022–2024).

Part of [DriveSignal](https://github.com/jahnavidanda02/drivesignal), a
project comparing a rule-based heuristic, this fine-tuned DistilBERT
classifier, and a LoRA-fine-tuned Qwen3-1.7B on the same task.

## Taxonomy

```
Perception Failure | Prediction Failure | Lane Keeping | Braking Behavior |
Unwanted Maneuver | Construction/Environment | Precautionary |
System/Hardware Fault | Localization/Mapping | Other
```

## Training data

- 14,800 disengagement event descriptions from the CA DMV 2022–2024 reports
  (Waymo and other manufacturers).
- **Weak-labeled**: training labels come from a keyword/regex heuristic
  classifier, not human annotation — a standard weak-supervision pattern for
  bootstrapping labels when ground truth doesn't exist. See training script:
  [`src/finetune_distilbert.py`](https://github.com/jahnavidanda02/drivesignal/blob/master/src/finetune_distilbert.py).

## Evaluation

Evaluated against a 250-row hand-labeled gold set (not seen during training,
and not derived from the heuristic labels):

| Model | Accuracy (n=250) |
|---|---|
| Heuristic baseline | 80.0% |
| **DistilBERT (this model)** | **80.0%** |
| Qwen3-1.7B LoRA | 72.8% |

Full breakdown: [`src/eval_distilbert_gold.py`](https://github.com/jahnavidanda02/drivesignal/blob/master/src/eval_distilbert_gold.py).

## Usage

```python
from transformers import pipeline

clf = pipeline("text-classification", model="jahnavidanda02/drivesignal-distilbert")
clf("Vehicle disengaged after hesitating at an unprotected left turn due to "
    "misjudging the trajectory of an oncoming vehicle.")
```

## Limitations

- Trained on weak (heuristic-derived) labels for the bulk of the data, so it
  inherits gaps in the heuristic's keyword coverage (e.g. rare categories
  like *Construction/Environment* and *Precautionary* are underrepresented).
- Trained only on CA DMV disengagement report language; may not generalize
  to differently phrased AV incident text.