File size: 3,031 Bytes
861e002 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | ---
license: apache-2.0
library_name: onnx
pipeline_tag: text-classification
tags:
- perseval
- agent-evaluation
- task-completion
- onnx
- rust
---
# Perseval Task Completion — development runtime candidate
This repository contains the compact ONNX artifact used to exercise Perseval's
local Task Completion evaluator and model-management flow.
## Status
This is a **development candidate**, not a release-certified evaluator. It is
published so the macOS application can install, verify, update, and execute a
real local model end to end. Do not interpret its scores as production-quality
ground truth.
The frozen private comparison holdout remains sealed. A later release will be
marked certified only after it passes the documented holdout gates.
## Files
- `manifest.json` — versioned model, projector, feature, calibration, and file
integrity contract.
- `model.onnx` — compact binary Task Completion head.
- `tokenizer.json` — pinned tokenizer reference.
- `parity.json` — Python/ONNX/Rust parity fixtures.
- `catalog.json` — mutable release pointer whose artifact revision is immutable.
Perseval downloads files from the immutable revision named in `catalog.json`,
checks every size and SHA-256 digest, then validates the internal manifest
before making the installation active.
## Runtime contract
- Projector: `traceeval.compact-task-completion-projector.v1`
- Structured feature schema:
`traceeval.task_completion_structured_evidence.v2`
- Inputs: 39 structured evidence features
- Outputs: binary completion logits
- Calibration threshold: `0.745`
The application keeps trace projection, privacy enforcement, revision checks,
hash verification, inference, evidence resolution, and threshold application
in Rust.
## Intended use
Use this candidate to:
- test Perseval's local learned-assessment runtime;
- validate model installation and updates without manual folder entry;
- run development benchmarks and Python/ONNX/Rust parity checks.
Do not use it as an automated decision-maker or as a substitute for human
review in high-impact settings.
## Training data and limitations
The development pipeline combines source-balanced agent traces, structured
proof-bearing evidence, and group-safe splits. Synthetic, redistributable
examples are published separately at
[`Etolith/perseval-agent-evaluator-synthetic`](https://huggingface.co/datasets/Etolith/perseval-agent-evaluator-synthetic).
Known limitations include domain shift, unsupported-success claims, sparse
external-side-effect evidence, and incomplete calibration across long,
multi-agent traces. The compact head consumes a projected representation, not
raw screenshots or arbitrary full-context traces.
## Reproducibility
The manifest pins the model, tokenizer, parity fixtures, projector, feature
schema, training run, and calibration versions. The public application verifies
those contracts before loading the artifact. Private traces, credentials,
teacher prompts, and unreleased checkpoints are not included.
|