yapwithai/yap-distilroberta-screenshot-intent

Screenshot Intent Classifier

This repository contains a DistilRoBERTa-based classifier fine-tuned to decide whether a conversational agent should trigger a screenshot tool for the latest user message.

Base Model

This model is fine-tuned from distilroberta-base, and inherits the base encoder's maximum context length and tokenizer. distilroberta-base is a distilled RoBERTa-family encoder intended for faster, cheaper inference than RoBERTa-base, while keeping the standard 512-token context window.

Classifier

• 0 / no_screenshot: do not call the screenshot tool.
• 1 / take_screenshot: call the screenshot tool.

The input is a text block representing the recent conversation history, formatted as one utterance per line (raw user messages separated by newlines), e.g.:

I'm wondering if blue goes well with yellow.
What's your take on this?

At inference time, the host application typically feeds the last few conversation turns (most importantly the latest user message) in this format and thresholds the classifier's take_screenshot probability to decide whether to trigger the tool.

Training Data

The classifier was trained on a curated, hand-labelled private dataset. It contains hundreds of single-turn and multi-turn examples specifying whether each user message should or should not trigger a screenshot, including:

• Clear positive triggers ("look at this", "check this out", "rate this pic").
• Clear negatives (off-topic chit-chat, abstract statements, idioms like "I'll look into it").
• Edge cases involving deictic pronouns, quantities ("take 2 screenshots"), negation ("don't look"), multi-turn context, and more.

No external user logs or third-party datasets were used; the training data is purely synthetic / curated for this intent task.

Training Setup

• Epochs: 4
• Batch size: 32 (per device)
• Learning rate: 5e-05
• Weight decay: 0.01
• Max sequence length: 512

The script builds examples by concatenating conversation history up to and including the current user message, one utterance per line. Multi-turn conversations therefore become multiple training examples with growing context.

Usage

Basic usage with the Transformers library:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

MODEL_ID = "yapwithai/yap-distilroberta-screenshot-intent"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID)
model.eval()

text = "look at this amazing sunset"
inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    padding="max_length",
    max_length=512,
)

with torch.no_grad():
    outputs = model(**inputs)
    probs = outputs.logits.softmax(dim=-1)[0]

p_no, p_yes = probs.tolist()
print("P(no_screenshot)=", p_no)
print("P(take_screenshot)=", p_yes)

In production, you would:

• Construct a conversation history string similar to the training format (recent user turns, each on its own line).
• Run the classifier once per latest user message.
• Threshold p_yes to decide whether to trigger the screenshot tool.

DistilRoBERTa / RoBERTa Citation

If you use DistilRoBERTa in your work, please cite:

@inproceedings{sanh2019distilbert,
  title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},
  author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf},
  booktitle={NeurIPS EMC^2 Workshop},
  year={2019}
}

@article{liu2019roberta,
  title={RoBERTa: A Robustly Optimized BERT Pretraining Approach},
  author={Yinhan Liu and Myle Ott and Naman Goyal and Jingfei Du and Mandar Joshi and Danqi Chen and Omer Levy and Mike Lewis and Luke Zettlemoyer and Veselin Stoyanov},
  journal={arXiv:1907.11692},
  year={2019}
}