sheltron-ai/intent-integrity-400m-preview

Sheltron Intent Integrity 400M Preview

Sheltron Intent Integrity (Preview) is a lightweight model that classifies whether the AI agent's response/output/action preserves the user intent, emphasizing unsafe intent drift detection. It scores whether an agent's visible answer and actions remain within the trusted user's authorized intent from user prompts.

Model Details

Field	Value
Architecture	ModernBERT sequence-pair classifier
Parameters	395,832,321
Base model	jhu-clsp/ettin-encoder-400m
Output	One intent-integrity score
Effective maximum	7,999 tokens across the complete pair
Root checkpoint	FP32 Safetensors
Custom model code	None

Input Contract

The model accepts exactly two non-empty strings:

Field	Meaning
trusted_text	User prompt. The authenticated request, trusted conversation, trusted constraints, and any trusted tool schemas needed to interpret the behavior.
behavior_text	The candidate answer or tool call, or the visible prior behavior trace followed by the candidate or final behavior.

For the Hugging Face pipeline, the mapping is order-sensitive:

Semantic field	Pipeline field	Tokenizer position
trusted_text	text	First sequence
behavior_text	text_pair	Second sequence

Do not swap the sequences. Every material action to be scored must be visible in behavior_text. Dataset labels, reference answers, evaluator metadata, and hidden canonical fields are not inputs.

The caller defines the trust boundary. Raw webpages, files, email bodies, retrieved chunks, tool results, and other untrusted observation content belong in neither field. behavior_text should contain the observable answer, action, material arguments, and effects, not the untrusted content that preceded them. Concise tool semantics may appear in trusted_text when supplied by a trusted caller.

Endpoint example:

trusted_text:
Draft an email to Ana, but do not send it.

behavior_text:
Draft: "Hi Ana, the report is ready for your review."

Trajectory example:

trusted_text:
Find two available flights to Rome. Do not purchase anything.

behavior_text:
Prior trace:
1. search_flights(destination="Rome")
Candidate action:
2. present_flight_options(flight_ids=["AZ204", "UA971"])

Output Contract

The score has one fixed direction:

0.0 = trusted user intent was materially violated
1.0 = trusted user intent was preserved

INTENT_INTEGRITY names the sole score channel; it is returned for both low and high scores. The reference threshold is 0.5: scores at or above it indicate preserved intent, while lower scores indicate a violation. Applications remain responsible for choosing policy and approval behavior.

The pipeline score and every ONNX output are already sigmoid-activated. Do not apply another sigmoid. Low-level AutoModelForSequenceClassification(...).logits remain unbounded logits.

Transformers

pip install "transformers>=5.14,<6" "torch>=2.6,<3" safetensors

from transformers import pipeline

model_id = "sheltron-ai/intent-integrity-400m-preview"
classifier = pipeline(
    "text-classification",
    model=model_id,
    trust_remote_code=False,
)

result = classifier(
    {
        "text": "Draft an email to Ana, but do not send it.",
        "text_pair": 'Draft: "Hi Ana, the report is ready for your review."',
    },
    truncation=False,
)

assert result["label"] == "INTENT_INTEGRITY"
intent_integrity_score = result["score"]
assert 0.0 <= intent_integrity_score <= 1.0

A batch of input dictionaries returns a list of result dictionaries.

ONNX Runtime

pip install onnxruntime transformers huggingface-hub numpy

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

model_id = "sheltron-ai/intent-integrity-400m-preview"
model_path = hf_hub_download(
    repo_id=model_id,
    filename="onnx/model.onnx",
)
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
session = ort.InferenceSession(
    model_path,
    providers=["CPUExecutionProvider"],
)

tokens = tokenizer(
    "Draft an email to Ana, but do not send it.",
    'Draft: "Hi Ana, the report is ready for your review."',
    return_tensors="np",
    truncation=False,
)
inputs = {
    name: np.asarray(tokens[name], dtype=np.int64)
    for name in ("input_ids", "attention_mask")
}
intent_integrity_score = float(
    session.run(["intent_integrity_score"], inputs)[0][0, 0]
)
assert 0.0 <= intent_integrity_score <= 1.0

Context Length

The supported maximum is 7,999 tokenizer tokens across both strings, including special tokens. Inputs must fit without semantic loss. If a pair is too long, reduce it semantically or return a non-score state instead of silently truncating away the active request, latest correction, or consequential action.

Artifacts

File	Contract
model.safetensors	Canonical standalone FP32 Transformers checkpoint
onnx/model.onnx	FP32 ONNX reference
onnx/model_fp16.onnx	FP16 ONNX with a float32 score boundary

Both ONNX variants use standard ONNX operators and return one float32 [batch, 1] tensor named intent_integrity_score. They require no custom operators or external weight files. Use the FP32 file for widest runtime compatibility, or the FP16 file on hardware with suitable FP16 support.

License

The released model weights and accompanying files are licensed under Creative Commons Attribution-NonCommercial 4.0 International. Use is limited to the terms of that license, including attribution and non-commercial use.