File size: 4,284 Bytes
deaf348 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | ---
license: apache-2.0
base_model: vllm-sr/vela3-fast
tags:
- coreml
- decision-making
- on-device
- system-one
- pii-detection
- hallucination-detection
- zero-shot-classification
---
# Vela 3.0 (vela3-fast, 0.4B) · Core ML
[vllm-sr/vela3-fast](https://huggingface.co/vllm-sr/vela3-fast) (vLLM Semantic Router × KR Labs, Vela 3.0 family) converted to
Core ML for Apple silicon: **decisions** (choice / yes-no over options you name) and **text spans** (17 types of personal
information, words of an answer its context does not support) from one UltraBERT-380M encoder, on device.
| | |
|---|---|
| Source | `vllm-sr/vela3-fast` @ `9e338f39` |
| `Vela3Decision.mlpackage` | decision backbone, fp16, 726 MB: functions `L128`, `L256`, `L512`, `L1024` (encoder tokens) |
| `Vela3SpanSets.mlpackage` | backbone + the release's `span_sets` adapter (LoRA merged), fp16, 725 MB: functions `L512`, `L1024` |
| Host side | `heads_decision.safetensors`, `heads_span_sets.safetensors` (fp32 readout heads), `tokenizer.json`, `config.json`, `vela3_config.json` (the release's thresholds) |
| Runtime | `vela3_coreml.py` (Python: coremltools, tokenizers, torch for the small heads, no PyTorch weights) + `upstream/` (the release's own rendering, span decoder and thresholds, unmodified) |
| Requires | macOS 15+ / iOS 18+ |
```python
from vela3_coreml import Vela3CoreML
m = Vela3CoreML(".") # GPU by default
m.decide("My card was charged twice and I need this fixed today.",
{"intent": {"type": "choice", "instructions": "What does the customer want?",
"criteria": {"refund": "Money back for the duplicate charge", "cancel": "Cancel the order"}},
"urgent": {"type": "noul", "instructions": "The customer needs a fast response."}})
m.pii("Hi, I'm Dana Whitfield (dana@example.com, 312-555-0110).") # [{label, start, end, text, score}, ...]
m.unsupported(context="Adults may take up to 4 g of paracetamol in 24 hours.",
answer="Adults may take up to 6 g of paracetamol in 24 hours.") # -> "6 g"
```
## Fidelity
Against the fp32 PyTorch release on the same inputs (`reports/verify.txt`):
- **Decisions:** typed-decisions TEST, all 1,200 choice and yes/no questions: **0 answers differ**, max probability difference 0.0026.
- **Personal information:** 25 varied passages (support tickets, clinic notes, HR reports, bank chats; 183 spans): **identical span sets**.
- **Unsupported claims:** identical spans on every context / answer pair tested.
On a 20,000-word synthetic case file with 2,971 planted values, the Core ML model catches 99.5 % (2,956), with 30 other flags.
## Speed (MacBook Pro M5 Pro, GPU)
| Encoder function | Decision | Span sets |
|---|---:|---:|
| L128 | 6.7 ms | |
| L256 | 9.5 ms | |
| L512 | 17.6 ms | 18.7 ms |
| L1024 | 37.6 ms | 37.5 ms |
One encoder pass answers one question. The question (options, or the 17 PII label descriptions, ~250 tokens) counts toward
the length, so a PII pass over a paragraph usually lands in `L512`: about 0.5 ms per word end to end. The Neural Engine is
not used: it was slower at every length and drifted further from the fp32 answers (15 / 1,200 decisions flipped).
## Scope and limits
- Covered: decisions on the decision backbone, and the two label sets of the `span_sets` adapter (17 PII types, `unsupported`).
The release's other adapters (open-label NER, relations, evidence, router signals, domain, fact-check, feedback) are not
converted; each retrains every layer, so each would be its own encoder package.
- The release's `score` question type is not part of the decision backbone and is not supported (as upstream).
- Inputs are limited to 1,024 tokens per encoder pass. Longer decision inputs raise `Unsupported`; longer span inputs are
windowed at 1,024 tokens (the release uses 8,192), so long-document spans can differ.
## Files and licences
Apache-2.0, as the source release (`LICENSE`). `upstream/` is the release's own code, unmodified. The conversion: a
fixed-length export of the encoder (local layers' 128-token sliding window as a band mask, the two RoPE bases precomputed, fp16-safe
masks), the adapter's LoRA merged into plain weights, heads kept in fp32 on the host. `conversion/` holds the export, packaging
and verification scripts.
|