license: other
pipeline_tag: text-classification
library_name: setfit
base_model: sentence-transformers/all-MiniLM-L6-v2
tags:
- setfit
- sentence-transformers
- text-classification
- crp
- context-relay-protocol
- intent-classification
- speech-acts
datasets:
- banking77
- snips_built_in_intents
metrics:
- accuracy
model-index:
- name: crp-intent-setfit
results:
- task:
type: text-classification
name: Speech-act classification (4-class)
dataset:
type: banking77
name: CRP speech-act held-out mix
metrics:
- type: accuracy
value: 0.934
name: Held-out accuracy (2,000 examples)
widget:
- text: Please scan the repository for compliance issues.
- text: What is the current deployment status?
- text: I believe the server is down.
- text: This is frustrating and slow.
inference: true
CRP Intent SetFit — speech-act classifier
Part of the Context Relay Protocol (CRP) ML-first
governance layer. Classifies a user turn into one of four CRP speech acts —
request, question, assertion, expressive — used by crp/isa/intent.py
(SPEC-051 Intent & Speech Acts) to drive routing and operation framing in
agentic pipelines.
Few-shot SetFit model: contrastive-tuned all-MiniLM-L6-v2 body +
LogisticRegression head fit on the full training mix.
Verified results (independent harness, 2026-07-28)
| Metric | Value |
|---|---|
| Held-out accuracy (2,000 unseen examples) | 0.9340 |
| F1 assertion / expressive / question / request | 0.980 / 1.000 / 0.880 / 0.933 |
| Production-style CRP prompts | 18/20 |
Training data
Banking77 + SNIPS (intent names heuristically mapped to the four CRP speech acts) + templated synthetic examples per class. 64-shot contrastive body tuning, then the classifier head refit on the full mix.
Usage
from setfit import SetFitModel
model = SetFitModel.from_pretrained("AutoCyberAI/crp-intent-setfit")
model.predict(["Please scan the repository for compliance issues."])
# -> ['request']
In the CRP SDK this model is the default intent backend:
CRP_INTENT_MODEL=AutoCyberAI/crp-intent-setfit # default; no env needed
Limitations
Four speech acts only; trained on English service/assistant phrasing. The held-out score is in-distribution (same data mix); the production-prompt score above is the out-of-distribution signal.
License
Elastic License 2.0 — see the CRP repository for details.