metadata
language:
- en
license: apache-2.0
library_name: transformers
pipeline_tag: token-classification
tags:
- prompt-compression
- token-classification
- tiny-model
- synthetic-data
base_model: google/bert_uncased_L-2_H-128_A-2
datasets:
- aayushbist/saccade-100k
Saccade Tiny 100K
Saccade Tiny is a 4,369,666-parameter experimental token
classifier that predicts whether each word should be KEEP or DROP before a
prompt is sent to a larger language model.
Training
- Base model:
google/bert_uncased_L-2_H-128_A-2 - Training examples: 80,000
- Validation examples: 10,000
- Held-out test examples: 10,000
- Total generated dataset: 100,000 examples
- Dataset: aayushbist/saccade-100k
The examples were derived from human-written English OpenAssistant prompts by injecting synthetic filler, repetitions, false starts, and redundant politeness.
Held-out synthetic evaluation
| Metric | Result |
|---|---|
| Accuracy | 0.9393 |
| KEEP precision | 0.9363 |
| KEEP recall | 0.9684 |
| KEEP F1 | 0.9521 |
| Predicted DROP rate | 35.66% |
Intended use
Research and demonstrations involving conservative removal of obvious low-information wording from English prompts.
Honest limitations
- The source prompts are human-written, but the noise and labels are synthetic.
- High performance on this test set does not prove equivalent performance on naturally disfluent speech or unseen domains.
- This version has not established a general increase in downstream LLM accuracy.
- Do not use it for safety-critical, legal, medical, or financial text without extensive additional evaluation.
Usage
from transformers import pipeline
classifier = pipeline(
"token-classification",
model="aayushbist/saccade-tiny-100k",
aggregation_strategy="none",
)
print(classifier("Could you basically summarize this report?"))