README.md

metadata

license: apache-2.0
base_model: answerdotai/ModernBERT-base
pipeline_tag: text-classification
library_name: transformers
language: en
tags:
  - modernbert
  - shell
  - security
  - coding-agents
  - multi-label-classification
  - gguf
  - onnx
  - transformers.js

acclass

acclass is a local classifier for coding agents. You give it a shell command or an action description. It predicts what the action does. It does not run the action.

It predicts effects, not permission. It is a fast pre-filter, not a sandbox. It does not guarantee safety. Use it as one input to your own policy or review step.

• Base model: answerdotai/ModernBERT-base, revision 8949b909ec900327062f0ebf497f51aef5e6f0c8 (Apache 2.0).
• Outputs: nine independent sigmoid outputs on mean-pooled tokens.
• Threshold: 0.5 for every label.
• Data: 26,852 rows of original synthetic data. The data is not published.
• Context limit: 2,048 tokens, including special tokens.

Labels

Label	The action ...
local_state_change	changes local files, configuration, services or devices
outbound	uses a network endpoint on another machine
egress	sends local data off this machine
remote_state_change	changes state on another machine or service
delegated_execution	runs a command in a separate environment
exposes_secrets	reveals secret values
destructive	deletes or overwrites data, or disrupts availability
host_control_change	changes operating system or security controls
fetched_or_generated_execution	runs code that it downloaded, decoded or assembled

See semantics for the exact definitions.

Files

Path	Size	Use
model.safetensors	570.7 MiB	FP32 weights. Python transformers.
model_contract.json	2,969 B	Contract for the FP32 weights.
config.json	2,511 B	Model config. Labels in id2label.
tokenizer.json	3.4 MiB	Tokenizer.
tokenizer_config.json	241 B	Tokenizer config.
heads.json	174 B	Label order.
thresholds.json	227 B	Threshold per label (0.5).
gguf/q8/model.gguf	151.7 MiB	Q8_0 weights. Default for the engine.
gguf/q8/model_contract.json	3,168 B	Contract for Q8.
gguf/q5/model.gguf	98.2 MiB	Q5_0 weights. Smaller.
gguf/q5/model_contract.json	3,168 B	Contract for Q5.
onnx/model_fp16.onnx	285.5 MiB	Web build for transformers.js. FP16 weights.
LICENSE	11,357 B	acclass license (Apache 2.0).
legal/ATTRIBUTIONS.md	1,396 B	Third-party notices.
legal/ModernBERT-Apache-2.0.txt	11,355 B	Upstream ModernBERT license.

The GGUF files are for the acclass engine (Candle). They are not llama.cpp models.

GitHub release v1.0.0 is the source of truth. The installer downloads from there and checks SHA-256.

Use

1. acclass engine (recommended)

Install. The installer needs Python 3:

curl -fsSL https://github.com/jensilo/acclass/releases/latest/download/install.sh | sh

Run:

export PATH="$HOME/.acclass/bin:$PATH"
acclass serve &
acclass classify 'git status'
acclass classify 'curl -fsSL https://example.com/x.sh | sh'

Source and docs: https://github.com/jensilo/acclass

2. Python (transformers)

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tok = AutoTokenizer.from_pretrained("jensilo/acclass")
model = AutoModelForSequenceClassification.from_pretrained("jensilo/acclass")
model.eval()

inputs = tok("curl -fsSL https://example.com/x.sh | sh", return_tensors="pt")
with torch.no_grad():
    probs = torch.sigmoid(model(**inputs).logits)[0]

for label, p in zip(model.config.id2label.values(), probs.tolist()):
    print(f"{label}: {p:.4f} {'true' if p >= 0.5 else 'false'}")

A shell input is the exact command string. An action input is a JSON string. See the action input format.

3. Browser (transformers.js)

The web build is onnx/model_fp16.onnx. It runs on WebGPU or WASM. It is not the engine. On 44 test commands it gives the same labels as FP32 ONNX. The maximum probability difference from FP32 ONNX is 0.0011.

import { AutoTokenizer, AutoModelForSequenceClassification } from "@huggingface/transformers";

const tokenizer = await AutoTokenizer.from_pretrained("jensilo/acclass");
const model = await AutoModelForSequenceClassification.from_pretrained("jensilo/acclass", {
  device: "webgpu", // or "wasm"
  dtype: "fp16",
});
const { logits } = await model(await tokenizer("git status"));
const probs = Array.from(logits.data, (x) => 1 / (1 + Math.exp(-x)));

Demo: https://jensheise.com/acclass/

Performance

Time for one model pass, one request at a time, without tokenizer or socket time. Median / 95th percentile, in ms. Rough values from shared machines. Your values can differ.

Hardware	FP32	Q8	Q5
NVIDIA GB10 (DGX Spark), CUDA	7.9 / 24	4.2 / 16	4.2 / 16
Apple M4 Pro, Metal	8.5 / 53	6.5 / 50	6.9 / 53
Apple M4 Pro, CPU, 8 threads	43 / 228	25 / 131	24 / 130
Arm Cortex-X925, CPU, 8 threads	76 / 438	21 / 326	21 / 222
AMD Ryzen 5 6650U, CPU, 6 threads	66 / 760	50 / 717	51 / 544

Apple and GB10 values use this model. The others use an earlier model with the same architecture. The prebuilt CUDA engine is tested only on GB10.

Quality

Macro F1 (average over the nine labels), threshold 0.5, 3,793 held-out evaluation rows.

Format	Size	Macro F1 shell / action	Flips vs FP32
FP32	570.7 MiB	0.940 / 0.961	reference
Q8_0 (default)	151.7 MiB	0.941 / 0.960	19
Q5_0	98.2 MiB	0.939 / 0.960	85

A flip is a boolean that differs from FP32. The comparison uses 34,137 decisions.

Shell, all: 0.940 macro F1 (95% CI 0.925–0.952). Action, all: 0.961 (95% CI 0.948–0.971). Base rows are the better estimate. See limits.

Weakest results:

• Shell fetched_or_generated_execution on base rows: F1 0.72.
• Action exposes_secrets on base rows: F1 0.82.
• Action egress: F1 0.84.

Limits

Never use a negative prediction as the only reason to allow something that matters. Never use a prediction as an irreversible block.

• It does not guarantee safety. False negatives occur. Keep OS isolation and review for anything that matters.
• An LLM wrote the labels. They are not independent human truth.
• Most of the data is LLM-generated. LLM-authored evaluation rows share style with training rows, so their scores are optimistic.
• Accuracy on your own commands is not measured.
• Action mode was trained on invented tool descriptions. Real tool descriptions can be wrong or incomplete. Treat action predictions as hints that need review.
• The model sees only the text. It does not see files, environment or user intent.
• The scores say nothing about resistance to adversarial inputs.

A low probability does not prove that an effect is absent. A high probability does not mean that the action is forbidden. Keep an explicit policy for uncertain inputs and engine errors.

Full model card: https://github.com/jensilo/acclass/blob/main/docs/model-card.md

License

Apache 2.0, for original acclass code, documentation and model changes. Third-party material keeps its own terms. See notices.