| --- |
| license: apache-2.0 |
| base_model: vllm-sr/vela3-fast |
| tags: |
| - coreml |
| - decision-making |
| - on-device |
| - system-one |
| - pii-detection |
| - hallucination-detection |
| - zero-shot-classification |
| --- |
| |
| # Vela 3.0 (vela3-fast, 0.4B) · Core ML |
|
|
| [vllm-sr/vela3-fast](https://huggingface.co/vllm-sr/vela3-fast) (vLLM Semantic Router × KR Labs, Vela 3.0 family) converted to |
| Core ML for Apple silicon: **decisions** (choice / yes-no over options you name) and **text spans** (17 types of personal |
| information, words of an answer its context does not support) from one UltraBERT-380M encoder, on device. |
|
|
| | | | |
| |---|---| |
| | Source | `vllm-sr/vela3-fast` @ `9e338f39` | |
| | `Vela3Decision.mlpackage` | decision backbone, fp16, 726 MB: functions `L128`, `L256`, `L512`, `L1024` (encoder tokens) | |
| | `Vela3SpanSets.mlpackage` | backbone + the release's `span_sets` adapter (LoRA merged), fp16, 725 MB: functions `L512`, `L1024` | |
| | Host side | `heads_decision.safetensors`, `heads_span_sets.safetensors` (fp32 readout heads), `tokenizer.json`, `config.json`, `vela3_config.json` (the release's thresholds) | |
| | Runtime | `vela3_coreml.py` (Python: coremltools, tokenizers, torch for the small heads, no PyTorch weights) + `upstream/` (the release's own rendering, span decoder and thresholds, unmodified) | |
| | Requires | macOS 15+ / iOS 18+ | |
|
|
| ```python |
| from vela3_coreml import Vela3CoreML |
| m = Vela3CoreML(".") # GPU by default |
| m.decide("My card was charged twice and I need this fixed today.", |
| {"intent": {"type": "choice", "instructions": "What does the customer want?", |
| "criteria": {"refund": "Money back for the duplicate charge", "cancel": "Cancel the order"}}, |
| "urgent": {"type": "noul", "instructions": "The customer needs a fast response."}}) |
| m.pii("Hi, I'm Dana Whitfield (dana@example.com, 312-555-0110).") # [{label, start, end, text, score}, ...] |
| m.unsupported(context="Adults may take up to 4 g of paracetamol in 24 hours.", |
| answer="Adults may take up to 6 g of paracetamol in 24 hours.") # -> "6 g" |
| ``` |
|
|
| ## Fidelity |
|
|
| Against the fp32 PyTorch release on the same inputs (`reports/verify.txt`): |
|
|
| - **Decisions:** typed-decisions TEST, all 1,200 choice and yes/no questions: **0 answers differ**, max probability difference 0.0026. |
| - **Personal information:** 25 varied passages (support tickets, clinic notes, HR reports, bank chats; 183 spans): **identical span sets**. |
| - **Unsupported claims:** identical spans on every context / answer pair tested. |
|
|
| On a 20,000-word synthetic case file with 2,971 planted values, the Core ML model catches 99.5 % (2,956), with 30 other flags. |
|
|
| ## Speed (MacBook Pro M5 Pro, GPU) |
|
|
| | Encoder function | Decision | Span sets | |
| |---|---:|---:| |
| | L128 | 6.7 ms | | |
| | L256 | 9.5 ms | | |
| | L512 | 17.6 ms | 18.7 ms | |
| | L1024 | 37.6 ms | 37.5 ms | |
|
|
| One encoder pass answers one question. The question (options, or the 17 PII label descriptions, ~250 tokens) counts toward |
| the length, so a PII pass over a paragraph usually lands in `L512`: about 0.5 ms per word end to end. The Neural Engine is |
| not used: it was slower at every length and drifted further from the fp32 answers (15 / 1,200 decisions flipped). |
|
|
| ## Scope and limits |
|
|
| - Covered: decisions on the decision backbone, and the two label sets of the `span_sets` adapter (17 PII types, `unsupported`). |
| The release's other adapters (open-label NER, relations, evidence, router signals, domain, fact-check, feedback) are not |
| converted; each retrains every layer, so each would be its own encoder package. |
| - The release's `score` question type is not part of the decision backbone and is not supported (as upstream). |
| - Inputs are limited to 1,024 tokens per encoder pass. Longer decision inputs raise `Unsupported`; longer span inputs are |
| windowed at 1,024 tokens (the release uses 8,192), so long-document spans can differ. |
|
|
| ## Files and licences |
|
|
| Apache-2.0, as the source release (`LICENSE`). `upstream/` is the release's own code, unmodified. The conversion: a |
| fixed-length export of the encoder (local layers' 128-token sliding window as a band mask, the two RoPE bases precomputed, fp16-safe |
| masks), the adapter's LoRA merged into plain weights, heads kept in fp32 on the host. `conversion/` holds the export, packaging |
| and verification scripts. |
|
|
