Upload folder using huggingface_hub

811d51e verified about 12 hours ago

8.33 kB

license: apache-2.0
language:
  - en
pipeline_tag: text-classification
inference: false
tags:
  - onnx
  - onnxruntime
  - browser
  - local-inference
  - prompt-classification
  - task-classification
  - complexity-estimation
  - ai-routing
  - green-ai
  - veritiana
datasets:
  - OpenAssistant/oasst1
  - grammarly/coedit
  - evalplus/mbppplus
  - google/IFEval
metrics:
  - accuracy
  - f1

Veritiana Prompt Task & Complexity Classifier

A compact ONNX classifier from Veritiana AI that identifies the type of work requested by an AI prompt and estimates its complexity before generative execution begins.

The model runs locally, including directly in a browser through ONNX Runtime Web. It does not generate text and does not require prompt content to be sent to a remote classification service.

Live browser test: https://www.veritiana.com/prompt_classifier.html

What it returns

The model produces two independent probability distributions.

Task — 9 classes

general_chat, writing, translation, summarization, research, coding, mathematics, document_analysis, high_stakes

Complexity — 3 classes

low, medium, high

Example result:

task: coding
complexity: high

Intended role

This release is a public recognition layer, not a complete AI router.

Prompt
  → local feature extraction
  → ONNX task and complexity classifier
  → probabilities and confidence
  → future routing policy
  → model, tools, context, safeguards and budget

The open model answers what kind of request this is and how demanding it appears. Commercial routing value begins when those signals are combined with provider capabilities, prices, latency, organizational policies, tool access, execution quality and feedback history to decide how the request should be executed.

The classifier is also part of the product direction behind Veritiana AI Meter, where local recognition helps distinguish the type and estimated complexity of visible AI work.

Model architecture

The input is a fixed vector of 1,544 float32 features:

1,024 hashed word and word-bigram bins
512 hashed character-trigram bins
8 numerical and structural features

Two independent multinomial logistic-regression heads produce task and complexity probabilities.

input
  features                  float32 [batch, 1544]

outputs
  task_probabilities        float32 [batch, 9]
  complexity_probabilities  float32 [batch, 3]

The released ONNX model is approximately 75 KB and uses ONNX opset 13.

Feature contract

The feature extractor is part of the model contract. Raw text cannot be passed directly to model.onnx.

The repository includes the exact implementation in features.py, with matching browser logic in examples/browser/src/main.js.

The eight numerical features encode:

normalized character length
normalized word count
question-mark count
newline count
technical-symbol count
URL presence
document/file-term presence
constant bias value

Hashing uses FNV-1a 32-bit over JavaScript UTF-16 code units so Python and browser implementations remain compatible.

Internal evaluation

The recorded internal split contains 1,287 examples.

Output	Accuracy	Macro-F1
Task	91.22%	91.04%
Complexity	86.79%	87.24%

These are internal weak-label evaluation results, not independently established real-world accuracy.

Important limitations:

labels include source-fixed labels, deterministic weak rules and deterministic lexical augmentation
augmented rows are variants, not independent human examples
minority-class support is uneven
the translation test support is only 4 examples and its reported score is not meaningful as a production claim
multilingual behavior has not been independently benchmarked
the classifier does not semantically reason like a large language model

Full per-class reports and confusion matrices are in evaluation.json.

Training data

The original normalized import contained 19,087 prompts:

Source	Imported rows	Recorded use
OpenAssistant/oasst1	10,170	filtered root prompter messages
grammarly/coedit	8,000	source text, task fixed to writing
evalplus/mbppplus	376	prompt field, task fixed to coding
google/IFEval	541	prompt field, weak task labels and instruction-count complexity signal

After filtering, balancing and deterministic augmentation, the prepared set contained 6,349 rows.

The datasets themselves are not redistributed. Data provenance, row counts, SHA-256 values and the known revision limitation are documented in dataset-manifest.json.

Browser use

A complete minimal Vite example is included in examples/browser.

cd examples/browser
npm install
npm run dev

The example loads model.onnx, reproduces the 1,544-feature contract and prints both probability distributions.

For the complete production interface, use the live Veritiana test:

https://www.veritiana.com/prompt_classifier.html

Python use

python -m venv .venv
source .venv/bin/activate
pip install -r requirements-inference.txt
python examples/python/classify.py "Refactor this API and add rollback tests."

Expected output structure:

{
  "task": {
    "label": "coding",
    "confidence": 0.0,
    "probabilities": {}
  },
  "complexity": {
    "label": "high",
    "confidence": 0.0,
    "probabilities": {}
  }
}

The numerical values depend on the supplied prompt. The example does not send data to an external service.

Reproducing training

Exact reproduction requires either:

dataset.jsonl with SHA-256 1d56520b25809488fa8be91bcd14219accdb31146e1ca56686ff805d34c21649, or
prepared training-balanced.jsonl with SHA-256 ab2665fcd75800b2350593b7af4222e81d8cc93a34f2bf67320514a92526f44c.

The datasets are not included in this repository.

python -m venv .venv
source .venv/bin/activate
pip install -r requirements-training.txt

python prepare_training_set.py dataset.jsonl \
  --output training-balanced.jsonl \
  --report training-balanced-report.json \
  --seed 42 \
  --confidence 0.65 \
  --target-per-task 800 \
  --max-per-task 2000

python train.py training-balanced.jsonl \
  --output-dir reproduced-output \
  --version 3.1.0-multisource-balanced \
  --test-size 0.20 \
  --seed 42 \
  --c 4.0

Reference model SHA-256:

dca7560742f19207d089d469c2907e7a1b7c06e2bbf14d2404a5d9359043f572

Exact byte reproduction can also depend on the numerical environment and BLAS implementation. The feature contract and output probabilities are the primary compatibility target.

Repository contents

README.md
model.onnx
config.json
classifier-meta.json
evaluation.json
dataset-manifest.json
training-config.json
features.py
prepare_training_set.py
train.py
requirements-inference.txt
requirements-training.txt
LICENSE
NOTICE
assets/
  veritiana-prompt-classifier-architecture.png
examples/
  python/
  browser/
upload_to_hub.py

Appropriate uses

local prompt analytics
task-distribution measurement
complexity estimation
pre-routing signals
browser-native AI usage instrumentation
research and prototyping around deterministic pre-LLM decision layers

Out-of-scope uses

Do not use the classifier as:

a medical, legal, financial or safety decision maker
proof that a prompt is safe
a factual-answer evaluator
an authorization layer
a universal language detector
a guaranteed model-selection engine
a replacement for human review in high-stakes workflows

Privacy

Inference can run entirely on the user device. The model itself does not upload, store or transmit prompt content. Integrators remain responsible for the behavior of the surrounding application.

License

The model and repository code are released under Apache License 2.0. See LICENSE and NOTICE.

Citation

@software{veritiana_prompt_classifier_2026,
  title        = {Veritiana Prompt Task and Complexity Classifier},
  author       = {Veritiana AI},
  year         = {2026},
  version      = {3.1.0},
  url          = {https://www.veritiana.com/prompt_classifier.html}
}