Text Classification
Transformers
Safetensors
qwen3_5
image-text-to-text
decision-model
typed-decisions
classification
routing
calibrated-probabilities
Instructions to use ironbcc/Seb-9B-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ironbcc/Seb-9B-v2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="ironbcc/Seb-9B-v2")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("ironbcc/Seb-9B-v2") model = AutoModelForMultimodalLM.from_pretrained("ironbcc/Seb-9B-v2", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Remove blog draft (uploaded by mistake while the repo is public)
Browse files- blog/hero.png +0 -3
- blog/lab-triage.mp4 +0 -3
- blog/llm-table.png +0 -3
- blog/overview.mp4 +0 -3
- blog/post.md +0 -81
- blog/results-table.png +0 -3
- blog/router-frontier.png +0 -3
- blog/use-cases.png +0 -3
blog/hero.png
DELETED
Git LFS Details
|
blog/lab-triage.mp4
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:06bc7b3abcf8a1cadc867d4c308a8d35a08253ad5d004cb75e97559b657efdbe
|
| 3 |
-
size 1825315
|
|
|
|
|
|
|
|
|
|
|
|
blog/llm-table.png
DELETED
Git LFS Details
|
blog/overview.mp4
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:c473bbf17ff6948d38984a2e943ebe6227ac9e82096b13a543b92a766f57f0f7
|
| 3 |
-
size 1517275
|
|
|
|
|
|
|
|
|
|
|
|
blog/post.md
DELETED
|
@@ -1,81 +0,0 @@
|
|
| 1 |
-
# Seb-9B-v2: A small model for the decisions your app makes all day
|
| 2 |
-
|
| 3 |
-
Most of what an application asks a language model to do is not writing. It is choosing. Which team gets this ticket? Is this email a phishing attempt? Should the robot stop? Does this contract allow that? These calls happen thousands of times a day. They need answers in milliseconds, and they should not require a hosted API.
|
| 4 |
-
|
| 5 |
-
Seb-9B-v2 is built for exactly that. You give it a situation, a question, and a few options. In a single forward pass, it tells you how likely each option is. It generates no text, so there is nothing to parse. It runs on your own hardware, reads images, and supports both a standard chat API and the System One API.
|
| 6 |
-
|
| 7 |
-

|
| 8 |
-
|
| 9 |
-
## TL;DR
|
| 10 |
-
|
| 11 |
-
- **Quality.** 0.801 mean accuracy over 17 public suites (our v1 0.793, hosted Jev 1.13 0.786). BANKING77 0.973, jailbreak detection 0.940, model routing 0.980. On 420 public decision rows it is level with gpt-6-luna and Claude Haiku (medium effort), and behind Claude Opus.
|
| 12 |
-
- **Speed.** About 22 ms per decision on one RTX PRO 6000 and about 88 ms on an Apple M5 Max, up to roughly 87 decisions a second per GPU.
|
| 13 |
-
- **Decision Index.** 59.16 on the 0.3 public index, against 57.96 for hosted Jev 1.13 and 60.70 for Jev 1.14. Ahead of Jev 1.13 on 22 of 37 benchmarks and in 4 of 5 areas.
|
| 14 |
-
|
| 15 |
-
## Built for the edge
|
| 16 |
-
|
| 17 |
-
A decision model earns its keep where the data already lives: on a laptop, a gateway, a robot controller, or a server next to the database.
|
| 18 |
-
|
| 19 |
-
- **Local.** The weights are about 19 GB in bf16. On an Apple M5 Max, a decision takes about 88 ms (96 ms with 8-bit weights). On one RTX PRO 6000, it takes about 22 ms.
|
| 20 |
-
- **Private.** Nothing leaves your machine, and there is no per-call bill.
|
| 21 |
-
- **Probabilities, not prose.** Every option comes back with a probability so you can set a clear threshold: act when the model is confident, and escalate to a person or a larger model when it is not.
|
| 22 |
-
- **Images too.** Input states can include an image. Held-out image decisions scored 0.886 on 2,451 rows.
|
| 23 |
-
- **Two ways to call it.** You can use a regular chat API (vLLM, with structured choice and logprobs) or the System One API (`POST /v1/systemone`: one state, several typed questions, one call). The lightweight server for the second option ships directly in the model repo.
|
| 24 |
-
|
| 25 |
-

|
| 26 |
-
|
| 27 |
-
## Using Seb as the first tier of an LLM stack
|
| 28 |
-
|
| 29 |
-
The most practical job we found is serving as the front door. Seb reviews every request first. When it is confident, it answers on the spot. When it is uncertain, the request routes to a larger model.
|
| 30 |
-
|
| 31 |
-
To test this without inventing a synthetic benchmark, we took 420 public decision rows (20 from each of 21 public suites) and collected real, blind answers from four tiers of Claude models: Haiku (medium effort) as low, Sonnet (low effort) as medium, Sonnet as high, and Opus as max. Then we let Seb's own confidence score guide the routing.
|
| 32 |
-
|
| 33 |
-
- With a confidence threshold of 0.98, Seb answers **51%** of the rows itself, and the combined accuracy is **0.855** (identical to sending every request to Opus) at an **estimated 51% lower cost**.
|
| 34 |
-
- With a threshold of 0.90, it answers **70%** locally, combined accuracy is **0.845** (one point lower), and estimated cost is **70% lower**.
|
| 35 |
-
|
| 36 |
-

|
| 37 |
-
|
| 38 |
-
A few honest caveats. Cost here is based on a relative list-price ratio (Haiku 1, Sonnet 3, Opus 5), and we did not measure raw token counts. Seb's local calls are treated as free, which excludes hardware hosting costs. This benchmark also consists of public, mostly straightforward decision rows: 334 of the 420 were solved by the cheapest tier alone. We also asked Seb to pick a tier directly zero-shot, and the gain was small (0.807 at relative cost 1.7). The model has not been trained specifically for tier routing yet, and training a dedicated router on routing data remains the next logical step.
|
| 39 |
-
|
| 40 |
-
## Practical use cases
|
| 41 |
-
|
| 42 |
-
- **Guardrails and triage.** Jailbreak detection 0.940, phishing 0.895, model routing 0.980, and BANKING77 banking intents 0.973. Outputs provide probabilities you can threshold directly.
|
| 43 |
-
- **Documents.** On ContractNLI contract questions, it scores 78.0 on the Decision Index, compared to 59.1 for hosted Jev 1.13.
|
| 44 |
-
- **Robotics.** Seb acts as the high-level policy: deciding which object to pick next, where it should go, when to stop because a person walked in, and when to resume. In simulation, 421 of 428 arm decisions matched the written rules, and a warehouse robot delivered all packages in 4 of 4 episodes. Inverse kinematics and a path planner handle physical movement, while the grasp itself is simulated.
|
| 45 |
-
|
| 46 |
-
<video src="overview.mp4" controls width="100%"></video>
|
| 47 |
-
|
| 48 |
-
A harder scene: five objects, four bins, an unlabeled item that must go to quarantine, slow carrying for fragile and hazardous items, and two people events. Seb got 221 of 228 decisions right across 12 episodes. The misses were all in pick order.
|
| 49 |
-
|
| 50 |
-
<video src="lab-triage.mp4" controls width="100%"></video>
|
| 51 |
-
|
| 52 |
-
## Detailed numbers
|
| 53 |
-
|
| 54 |
-

|
| 55 |
-
|
| 56 |
-

|
| 57 |
-
|
| 58 |
-
The general models are much larger, need a hosted call per decision, and do not return native probabilities. On everyday public decision rows Claude Opus is clearly ahead of Seb-9B-v2 and Claude Sonnet is slightly ahead (within noise). Claude Haiku (medium effort) and gpt-6-luna are level with it.
|
| 59 |
-
|
| 60 |
-
## Where it falls short
|
| 61 |
-
|
| 62 |
-
- **Knowledge and reasoning.** Jev is clearly stronger here (GPQA Diamond, MMLU-Pro, and BBH show our largest gaps).
|
| 63 |
-
- **Calibration.** Our v1 model is still better calibrated (ECE 0.088 vs 0.110).
|
| 64 |
-
- **Public leaderboard.** The Decision Index board has moved forward, and newer open entries now score higher on the public index than Seb-9B-v2 (for example, OneJev1.6-9B at 65.3).
|
| 65 |
-
- **Not a game engine.** This is a decision model, not a game engine. It beats a random-move chess bot 7 games out of 10 (the rest end in stalemates) and loses to Stockfish.
|
| 66 |
-
- **Simulated robotics.** The robot evaluations are simulations, and options are provided with the structured facts that an upstream perception stack would supply.
|
| 67 |
-
|
| 68 |
-
## Try it
|
| 69 |
-
|
| 70 |
-
The model card has full setup details: `ironbcc/Seb-9B-v2`.
|
| 71 |
-
|
| 72 |
-
```bash
|
| 73 |
-
vllm serve ironbcc/Seb-9B-v2 --served-model-name seb \
|
| 74 |
-
--logprobs-mode processed_logprobs --max-logprobs 32 --generation-config vllm \
|
| 75 |
-
--max-model-len 32768 --dtype bfloat16 --max-num-seqs 128 --enable-prefix-caching
|
| 76 |
-
|
| 77 |
-
python serve/system1_vllm_backend.py --vllm-url http://127.0.0.1:8000 --served-model seb \
|
| 78 |
-
--tokenizer ./seb-9b-v2 --port 8020 --model-name seb-9b-v2
|
| 79 |
-
```
|
| 80 |
-
|
| 81 |
-
Seb-9B-v2 takes inputs of up to 32,768 tokens (set `SYSTEM1_MAX_CONTEXT_TOKENS=32768` for the System One server; the default is 1,024). It is released under the Apache-2.0 license, matching the base model. It is intended as a decision aid, not professional advice.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
blog/results-table.png
DELETED
Git LFS Details
|
blog/router-frontier.png
DELETED
Git LFS Details
|
blog/use-cases.png
DELETED
Git LFS Details
|