FLAN-T5 Triggering Classifier

This model classifies whether conversational text should trigger one of two independent behaviors:

  • action_triggering
  • memory_triggering

For each task, the model returns exactly one of:

  • TRIGGER
  • NO_TRIGGER

The model only classifies intent. It does not authorize or execute actions and should be used together with a separate authorization and safety layer.

Intended input format

Provide the task and text in the following format:

task: action_triggering
text: Take a seat please

The expected output is:

TRIGGER

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_id = "tylermaister/flan-t5-small-triggering"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

prompt = "task: memory_triggering\ntext: What did I ask you to remember?"
inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=128)
outputs = model.generate(**inputs, max_new_tokens=8)
prediction = tokenizer.decode(outputs[0], skip_special_tokens=True).strip()
print(prediction)

Applications should accept only TRIGGER and NO_TRIGGER. Any other generated text should be treated as invalid rather than interpreted as a trigger decision.

Training data and evaluation

The model was fine-tuned from google/flan-t5-small on 476 deduplicated examples covering both tasks and both labels. The data includes synthetic examples and manually collected negative examples from game testing.

Evaluation was performed on an 80-example synthetic test set that was kept separate from training:

Metric Result
Overall accuracy 98.75%
Overall macro F1 98.75%
Invalid outputs 0

These results should not be treated as an estimate of production performance. The evaluation set is synthetic, and real-world language may contain ambiguity, typos, indirect phrasing, or examples outside the training distribution. Build and evaluate a manually reviewed real-world test set before deployment.

Limitations and safety

  • The model is not an authorization or policy engine.
  • Classification errors can result in false triggers or missed triggers.
  • Synthetic-data performance may not generalize to real user conversations.
  • Outputs must be normalized and validated against the two allowed labels.
  • Keep action execution outside the model and behind explicit authorization and safety checks.

Citation

This model is based on FLAN-T5 and uses the Apache-2.0 license inherited from the base model.

Downloads last month
59
Safetensors
Model size
77M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for tylermaister/flan-t5-small-triggering

Finetuned
(526)
this model