Upload 11 files

README.md

@@ -1,3 +1,226 @@
----
-license: mit
----
+# Glint-Router-1M
+
+a prompt router in 985,826 parameters. you hand it a user request, it hands back a domain, a complexity score, code/math/reasoning flags, and which model tier should take the job. one forward pass, around 0.06 ms on a gpu, 0.2 ms on cpu. the release: weights, tokenizer, two inference files, the routing policy as an editable json document, every score we measured. no training code, no pipeline, no framework. `pip install torch tokenizers safetensors` and you are routing.
+
+the target was SupraLabs/Supra-Router-51M, which does the same job with 51,786,240 parameters. we came in at 1/52nd the size and beat it on data neither model had seen.
+
+## what it actually is
+
+- a bidirectional encoder with typed heads. 3 layers, dim 128, 8 heads, ffn hidden 208, 4096-token bpe vocab, 256-token context.
+- the heads: domain over a closed 25-class taxonomy, complexity 1 to 5 as an ordinal head, four binary flags (code, math, reasoning, long_output), a routing head, and a 64-d projection used to add your own categories later.
+- 985,826 parameters. the embedding table is 524,288 of them, the three blocks are 418,560, the heads are the remaining 25k.
+- everything comes out of one forward pass. there is no decode loop and nothing to parse.
+
+## why an encoder
+
+supra generates its answer as text: `Domain: Programming | Complexity: 3 | Math: F | Code: T | Route: big model | Justification: ...`. about 40 greedy decode steps, one argmax at a time, under a causal mask.
+
+the causal mask is the expensive part. read "write a short poem about a Python" and the decision is already forming; the word "snake" arrives afterward and cannot reach back. an encoder reads the whole prompt before it commits. that is the entire architectural argument, and the trap slice below is where you can see it pay.
+
+the rest follows from dropping generation. no output format to malform, so no parse failures. probabilities instead of tokens, so the routing decision can be arithmetic. 40 sequential steps collapse to 1.
+
+## the numbers
+
+house rule, same as always: real scores, no dressing, including the ones that make us look bad.
+
+both obvious comparisons are rigged and we are going to say so before showing them. our own validation set uses our label pipeline, which this model trained on and supra never saw. supra's 992-row dataset is supra's training data, so scoring it there is asking a model to recite. neither answers the question.
+
+so the verdict comes from a set neither model has ever seen: dolly-15k, MBPP, HumanEval and AQuA-RAT, 4,272 prompts, labels taken from dataset provenance alone. every MBPP row is a code task because it is in MBPP. every AQuA row is math because it is in AQuA. no hand labelling, no model in the loop.
+
+### neutral holdout, 4,272 unseen prompts
+
+| | Glint-Router-1M | Supra-Router-51M |
+|---|---|---|
+| parameters | **985,826** | 51,786,240 |
+| route accuracy | **94.41** | 90.37 |
+| math detection | **95.26** | 92.07 |
+| code detection | **97.51** | 96.43 |
+| complexity within 1 | **95.90** | 91.63 |
+| complexity exact | **46.28** | 39.06 |
+| ms per prompt | **0.06** | 14.33 |
+| parse failures | **0** | 4 |
+
+239x faster, 52x smaller, ahead on every field.
+
+one caveat we owe you: the complexity *label rule* on that holdout is ours, so the two complexity rows lean our way. route, code and math do not. the route rule is supra's own published one, copied verbatim, and code/math come from which dataset a row lives in.
+
+### the other two views, for completeness
+
+on our own validation split (7,617 rows, our labels, we are in-distribution):
+
+| | Glint | Supra |
+|---|---|---|
+| route | **93.96** | 85.76 |
+| code | **99.15** | 97.61 |
+| math | **97.81** | 94.00 |
+| complexity exact | **79.23** | 46.29 |
+
+on supra's own 992 rows, held out of our training, which are supra's training data:
+
+| | Glint | Supra |
+|---|---|---|
+| route | 89.01 | **96.70** |
+| code | 90.11 | **100.00** |
+| math | 81.32 | **97.80** |
+| complexity exact | 50.55 | **85.71** |
+
+supra wins that table because it is reciting the set it was fine-tuned on. we publish it because leaving it out would be dressing.
+
+## the keyword trap
+
+supra's model card claims "keyword-trap evasion" and never measures it. a keyword trap is a prompt that mentions a technical token while being trivial: "write a short poem about a Python snake in the garden" is a poem, not a coding job.
+
+we generated 3,000 counterfactual pairs, trained on them, and scored the held-out slice separately because at 4% of the corpus it vanishes into any aggregate.
+
+| trap slice, 230 held-out prompts | Glint | Supra |
+|---|---|---|
+| route | **100.00** | 90.87 |
+| code | **100.00** | 98.70 |
+| complexity within 1 | **100.00** | 83.91 |
+
+100 is a suspicious number and it should be read as one. we generated those traps from templates, so the family was in training even though the specific rows were not. supra faces them completely cold. take it as evidence that traps are learnable, not that the problem is solved.
+
+## calibration
+
+the routing decision is expected-cost arithmetic over the model's probabilities, so those probabilities have to mean what they say. a head that is 90% confident should be right 90% of the time.
+
+after 42 epochs it was not. temperature scaling, fitted on held-out data after training, one scalar per head group:
+
+```
+expected calibration error, routing head
+  raw          0.0556
+  calibrated   0.0203
+```
+
+the fitted temperature on the routing head is 4.71, which is a blunt statement about how overconfident a small model gets when you let it see the same 95k rows forty times. the scalars ship inside the weights and `calibrated()` applies them for you.
+
+## routing is a policy, not a threshold
+
+supra decides with a boolean baked into its weights: complexity >= 3, or code, or math, means big model. two tiers. want three, or a different threshold, or "legal always goes to the frontier model"? you are collecting data and retraining 51M parameters.
+
+here the model stops at calibrated probabilities about the prompt. what to do with them is `policy.json`, which you own:
+
+```json
+{
+  "tiers": [
+    {"name": "local",    "cost": 0.0, "capability": 0.35, "tools": false},
+    {"name": "mid",      "cost": 1.0, "capability": 0.65, "tools": false},
+    {"name": "frontier", "cost": 8.0, "capability": 0.95, "tools": true}
+  ],
+  "failure_penalty": 50.0,
+  "sharpness": 20.0,
+  "abstain_margin": 0.05,
+  "overrides": []
+}
+```
+
+the decision is `argmin over tiers of cost + failure_penalty * P(tier fails)`, where P(fail) is a logistic in the gap between prompt difficulty and tier capability. n tiers, not 2. an abstain band escalates ties instead of coin-flipping them. overrides short-circuit the arithmetic on any predicted field:
+
+```json
+{"overrides": [{"when": {"domain": "legal"}, "tier": "frontier"},
+               {"when": {"code": true, "complexity_min": 4}, "tier": "frontier"}]}
+```
+
+two constants in there were tuned the hard way and we left the reasoning in the file. `failure_penalty` has to be several times the priciest call, or "pay least" beats "try hardest" when every tier is likely to fail, and the router quietly sends its hardest prompts to its weakest model. `sharpness` has to be steep: at 12, a free local tier still carries a 2.6% residual failure rate, which priced at the penalty is 1.33 and loses to the mid tier's flat cost of 1.0. the free tier won nothing until that was fixed.
+
+## adding a category it never trained on
+
+the 64-d projection head is trained alongside the classifier, so prompts that route alike land near each other. a new category is the mean of a handful of examples:
+
+```
+python infer.py --add-category refunds refund_examples.txt
+```
+
+six example prompts, one forward pass, no optimizer, no gradients. after that `decision.category` comes back as `refunds` on matching prompts and you can route on it with an override. this is the thing a 51M generator cannot do at all without a fine-tune.
+
+## where it fails
+
+a 1M parameter model has edges and we would rather you learn them here than in production.
+
+**math phrased without numbers or symbols.** the math training data is GSM8K word problems and LaTeX. symbol-free proofs read as easy:
+
+```
+"Prove that every finite integral domain is a field."   ->  math 0.01, complexity 2, local
+"Natalia sold clips to 48 friends in April, and half
+ as many in May. How many clips altogether?"            ->  math 0.99, complexity 3, frontier
+"Compute the integral of x^2 sin(x) dx from 0 to pi."   ->  math 0.95, frontier
+```
+
+the first one is wrong and it is wrong for a boring reason: nothing in 95k rows looks like it.
+
+**word problems with an unusual question verb.** "a train leaves at 3pm going 60mph, another at 4pm going 80mph, when do they meet?" scores math 0.03 and routes local. move it to "how far apart are they at 6pm" and it behaves. the corpus taught it "how many" and "calculate" and did not teach it "when do they meet".
+
+**raw code with no instruction.** HumanEval is our worst slice at 56.7% code detection, and supra's second worst at 67.7%. a bare function stub with a docstring has no instruction verb anywhere in it. neither of us handles it and we are not going to pretend the gap is small.
+
+**very short factual prompts.** "what is the capital of France" comes back domain `programming`, complexity 1, routed local. the routing is right and the domain label is noise. domain accuracy on prompts under six tokens is the weakest part of the model.
+
+both data gaps are one more training round away. the model shipped here got one 15-minute round and that was the whole budget.
+
+## how it was trained
+
+one round, 15 minutes, one RTX 3060 Ti, 14,479 steps, learning rate fully annealed inside the budget. the schedule measures its own step rate before it starts and sizes the cosine decay to fit, so the run finishes annealed instead of stopping mid-schedule. best-on-validation weights, not last-step weights.
+
+the biggest gap versus supra is on the data side. supra fine-tuned on 992 rows. we built 95,221 and never hand-labelled one of them:
+
+| source | rows | what it labels for free |
+|---|---|---|
+| alpaca | 35,000 | mixed domains, complexity from answer length |
+| code_instructions | 18,000 | code = true |
+| squad | 15,000 | factual, complexity 1 |
+| no_robots | 9,500 | human-written task category |
+| gsm8k | 7,473 | math = true |
+| hendrycks_math | 6,256 | math = true, complexity from level |
+| generated traps | 3,000 | keyword counterfactuals |
+| supra's own set | 992 | its labels, verbatim |
+
+labels come from provenance and from the reference answer, never from a human and never from a model. a row in MBPP is a code task. a 1,200-character reference answer means long_output. the model predicts those labels from the prompt alone and never sees the answer.
+
+the routing label is supra's published override rule copied exactly, so a head-to-head measures prediction quality rather than two people disagreeing about what "big model" means. our route split came out 51/49; supra's own set is 75/25.
+
+the tokenizer is a 4096-vocab bpe trained on routing prompts rather than on web prose, because `{`, `::`, `\frac` and `SELECT` are the evidence and a FineWeb vocabulary shreds all four into bytes.
+
+## files
+
+```
+model.safetensors     the weights (3.9 MB)
+config.json           architecture and label space
+tokenizer.json        4096-vocab bpe, trained on routing prompts
+modeling.py           the model, standalone, torch only
+policy.py             tiers, expected-cost decision, overrides, prototypes
+policy.json           the default 3-tier policy, edit this one
+infer.py              cli: route, batch, add a category
+scores.json           every number on this page, machine readable
+requirements.txt      torch, tokenizers, safetensors
+README.md             you are here
+```
+
+## quickstart
+
+```
+pip install torch tokenizers safetensors
+
+python infer.py --demo
+python infer.py "write a python function that reverses a linked list"
+python infer.py --file prompts.txt
+python infer.py --policy my_policy.json "summarize this thread"
+python infer.py --add-category refunds refund_examples.txt
+```
+
+in code:
+
+```python
+from modeling import encode_batch, load_router
+from policy import Policy, decide
+
+model, tokenizer = load_router()
+tokens = encode_batch(tokenizer, ["fix this SQL query"], model.config.max_len)
+decision = decide(model, tokens, "fix this SQL query", Policy.load("policy.json"))
+print(decision.tier, decision.domain, decision.complexity)
+```
+
+`decision.as_line()` prints supra's exact output format if you are swapping one for the other.
+
+runs on cpu. the model is 3.9 MB. it is smaller than the favicon budget of most websites.
+
+/lane
+glint research, 2026, 985,826 parameters, one 15-minute round on a 3060 Ti, 239x faster than the 51M model it beats, 0 parse failures, 3 known failure modes documented above

config.json

@@ -0,0 +1,28 @@
+{
+  "name": "Glint-Router-1M",
+  "author": "Glint-Research",
+  "task": "prompt routing / request triage",
+  "parameters": 985826,
+  "model": {
+    "vocab_size": 4096,
+    "dim": 128,
+    "n_heads": 8,
+    "layers": 3,
+    "ffn_hidden": 208,
+    "max_len": 256,
+    "rope_base": 10000.0,
+    "proj_dim": 64,
+    "n_domains": 25
+  },
+  "labels": {
+    "domains": 25,
+    "complexity": "ordinal 1-5 (CORAL)",
+    "binary": [
+      "code",
+      "math",
+      "reasoning",
+      "long_output"
+    ],
+    "route": "binary big/small, plus N-tier expected-cost policy"
+  }
+}

infer.py

@@ -0,0 +1,115 @@
+"""glint-router-1m: route prompts, edit the policy, teach it new categories.
+
+  python infer.py "write a python function that reverses a linked list"
+  python infer.py --file prompts.txt
+  python infer.py --policy my_policy.json "summarize this thread"
+  python infer.py --add-category refunds examples.txt
+  python infer.py --demo
+
+everything runs on cpu. one forward pass per prompt, around 0.2 ms.
+"""
+
+from __future__ import annotations
+
+import sys
+import time
+from pathlib import Path
+
+import torch
+
+from modeling import encode_batch, load_router
+from policy import Policy, Prototypes, decide
+
+HERE = Path(__file__).parent
+PROTOTYPES = HERE / "prototypes.json"
+
+
+def route(prompts: list[str], policy_path: Path | None = None,
+          prototypes_path: Path | None = None, device: str = "cpu"):
+    model, tokenizer = load_router(HERE, device)
+    policy = Policy.load(policy_path or HERE / "policy.json")
+    prototypes = Prototypes.load(prototypes_path)
+    tokens = encode_batch(tokenizer, prompts, model.config.max_len).to(device)
+    return [decide(model, tokens[i:i + 1], prompts[i], policy, prototypes)
+            for i in range(len(prompts))]
+
+
+def add_category(name: str, examples: list[str], device: str = "cpu") -> None:
+    """teach it a label it was never trained on. one forward pass, no optimizer."""
+    model, tokenizer = load_router(HERE, device)
+    tokens = encode_batch(tokenizer, examples, model.config.max_len).to(device)
+    with torch.no_grad():
+        projections = model(tokens)["proj"]
+    prototypes = Prototypes.load(PROTOTYPES)
+    prototypes.add(name, projections)
+    prototypes.save(PROTOTYPES)
+    print(f"added category {name!r} from {len(examples)} examples -> {PROTOTYPES}")
+
+
+def show(decisions, prompts) -> None:
+    for decision, prompt in zip(decisions, prompts, strict=True):
+        flags = [k for k, v in decision.signals.items() if v]
+        head = prompt if len(prompt) <= 58 else prompt[:55] + "..."
+        print(f"{decision.tier:<9} {decision.domain:<20} c={decision.complexity} "
+              f"d={decision.difficulty:.2f} {head}")
+        print(f"          {decision.reason}"
+              + (f" | signals={flags}" if flags else "")
+              + (f" | category={decision.category}" if decision.category else ""))
+
+
+DEMO = [
+    "What's the capital of France?",
+    "Write a short poem about a Python snake in the garden.",
+    "Write a Python function that reverses a linked list in place.",
+    "A train leaves at 3pm going 60mph, another at 4pm going 80mph, when do they meet?",
+    "Summarize this email thread in two sentences.",
+    "What are the latest news headlines today?",
+    "Explain the difference between a mutex and a semaphore, with an example.",
+]
+
+
+def main() -> int:
+    args = sys.argv[1:]
+    if not args or args[0] in ("-h", "--help"):
+        print(__doc__)
+        return 0
+
+    if args[0] == "--demo":
+        started = time.perf_counter()
+        decisions = route(DEMO)
+        elapsed = time.perf_counter() - started
+        show(decisions, DEMO)
+        print(f"\n{len(DEMO)} prompts in {elapsed * 1000:.0f} ms "
+              f"({elapsed * 1000 / len(DEMO):.2f} ms each, cpu, includes model load)")
+        return 0
+
+    if args[0] == "--add-category":
+        if len(args) < 3:
+            print("usage: python infer.py --add-category <name> <examples.txt>")
+            return 1
+        examples = [line.strip() for line in Path(args[2]).read_text().splitlines()
+                    if line.strip()]
+        add_category(args[1], examples)
+        return 0
+
+    policy_path = None
+    if args[0] == "--policy":
+        policy_path, args = Path(args[1]), args[2:]
+
+    if args and args[0] == "--file":
+        prompts = [line.strip() for line in Path(args[1]).read_text().splitlines()
+                   if line.strip()]
+    else:
+        prompts = [" ".join(args)]
+
+    decisions = route(prompts, policy_path,
+                      PROTOTYPES if PROTOTYPES.is_file() else None)
+    show(decisions, prompts)
+    if len(prompts) == 1:
+        print()
+        print(decisions[0].as_line())
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e49b3aef312902c7af0044336b17f97f7d5dd57075ae1d7d769adafea6018a4
+size 3946036

modeling.py

@@ -0,0 +1,198 @@
+"""glint-router-1m: standalone inference. torch + tokenizers + safetensors, nothing else.
+
+one forward pass over the prompt gives you every field at once: domain,
+complexity, code, math, reasoning, long_output, route, and a 64-d projection
+used for user-defined categories. no decoding loop, no output parsing.
+"""
+
+from __future__ import annotations
+
+import json
+import math
+from dataclasses import dataclass
+from pathlib import Path
+
+import torch
+from torch import Tensor, nn
+from torch.nn import functional
+
+HERE = Path(__file__).parent
+PAD_ID = 0
+BOS_ID = 1
+BINARY_FIELDS = ("code", "math", "reasoning", "long_output")
+COMPLEXITY_LEVELS = 5
+
+DOMAINS = (
+    "programming", "web_dev", "databases", "devops_sysadmin", "security",
+    "data_science_ml", "math", "science", "engineering", "reasoning_puzzle",
+    "creative_writing", "professional_writing", "editing_grammar", "translation",
+    "factual_qa", "education", "business", "finance", "legal", "health",
+    "travel", "food", "entertainment", "lifestyle", "other",
+)
+
+
+@dataclass(frozen=True)
+class RouterConfig:
+    vocab_size: int = 4096
+    dim: int = 128
+    n_heads: int = 8
+    layers: int = 3
+    ffn_hidden: int = 208
+    max_len: int = 256
+    rope_base: float = 10_000.0
+    proj_dim: int = 64
+    n_domains: int = len(DOMAINS)
+
+
+def build_rope_cache(config: RouterConfig) -> tuple[Tensor, Tensor]:
+    head_dim = config.dim // config.n_heads
+    positions = torch.arange(config.max_len, dtype=torch.float32)
+    inv_freq = 1.0 / (config.rope_base ** (torch.arange(0, head_dim, 2).float() / head_dim))
+    angles = torch.outer(positions, inv_freq)
+    return torch.cos(angles), torch.sin(angles)
+
+
+def apply_rope(x: Tensor, cos: Tensor, sin: Tensor) -> Tensor:
+    x_even, x_odd = x[..., 0::2], x[..., 1::2]
+    rotated_even = x_even * cos - x_odd * sin
+    rotated_odd = x_even * sin + x_odd * cos
+    return torch.stack((rotated_even, rotated_odd), dim=-1).flatten(-2)
+
+
+class SwiGlu(nn.Module):
+    def __init__(self, dim: int, hidden: int) -> None:
+        super().__init__()
+        self.gate_up = nn.Linear(dim, 2 * hidden, bias=False)
+        self.down = nn.Linear(hidden, dim, bias=False)
+
+    def forward(self, x: Tensor) -> Tensor:
+        gate, up = self.gate_up(x).chunk(2, dim=-1)
+        return self.down(functional.silu(gate) * up)
+
+
+class RouterAttention(nn.Module):
+    def __init__(self, config: RouterConfig) -> None:
+        super().__init__()
+        self.n_heads = config.n_heads
+        self.head_dim = config.dim // config.n_heads
+        self.qkv = nn.Linear(config.dim, 3 * config.dim, bias=False)
+        self.out = nn.Linear(config.dim, config.dim, bias=False)
+
+    def forward(self, x: Tensor, cos: Tensor, sin: Tensor, attn_mask: Tensor) -> Tensor:
+        batch, seq_len, dim = x.shape
+        q, k, v = self.qkv(x).split(dim, dim=-1)
+        shape = (batch, seq_len, self.n_heads, self.head_dim)
+        q = apply_rope(q.view(shape).transpose(1, 2), cos, sin)
+        k = apply_rope(k.view(shape).transpose(1, 2), cos, sin)
+        v = v.view(shape).transpose(1, 2)
+        attended = functional.scaled_dot_product_attention(q, k, v, attn_mask=attn_mask)
+        return self.out(attended.transpose(1, 2).reshape(batch, seq_len, dim))
+
+
+class RouterBlock(nn.Module):
+    def __init__(self, config: RouterConfig) -> None:
+        super().__init__()
+        self.attn_norm = nn.RMSNorm(config.dim)
+        self.attn = RouterAttention(config)
+        self.ffn_norm = nn.RMSNorm(config.dim)
+        self.ffn = SwiGlu(config.dim, config.ffn_hidden)
+
+    def forward(self, x: Tensor, cos: Tensor, sin: Tensor, attn_mask: Tensor) -> Tensor:
+        x = x + self.attn(self.attn_norm(x), cos, sin, attn_mask)
+        return x + self.ffn(self.ffn_norm(x))
+
+
+class GlintRouter(nn.Module):
+    def __init__(self, config: RouterConfig | None = None) -> None:
+        super().__init__()
+        config = config or RouterConfig()
+        self.config = config
+        self.embed = nn.Embedding(config.vocab_size, config.dim, padding_idx=PAD_ID)
+        self.blocks = nn.ModuleList(RouterBlock(config) for _ in range(config.layers))
+        self.final_norm = nn.RMSNorm(config.dim)
+        cos, sin = build_rope_cache(config)
+        self.register_buffer("rope_cos", cos, persistent=False)
+        self.register_buffer("rope_sin", sin, persistent=False)
+
+        pooled = 2 * config.dim
+        self.domain_head = nn.Linear(pooled, config.n_domains)
+        self.complexity_dir = nn.Linear(pooled, 1, bias=False)
+        self.complexity_bias = nn.Parameter(torch.zeros(COMPLEXITY_LEVELS - 1))
+        self.binary_head = nn.Linear(pooled, len(BINARY_FIELDS))
+        self.route_head = nn.Linear(pooled, 1)
+        self.proj_head = nn.Linear(pooled, config.proj_dim)
+        self.register_buffer("temperature", torch.ones(3), persistent=True)
+
+    def encode(self, tokens: Tensor) -> Tensor:
+        valid = tokens != PAD_ID
+        seq_len = tokens.shape[1]
+        cos = self.rope_cos[:seq_len].to(self.embed.weight.dtype)
+        sin = self.rope_sin[:seq_len].to(self.embed.weight.dtype)
+        attn_mask = torch.zeros(tokens.shape, dtype=self.embed.weight.dtype,
+                                device=tokens.device)
+        attn_mask = attn_mask.masked_fill(~valid, float("-inf"))[:, None, None, :]
+        x = self.embed(tokens)
+        for block in self.blocks:
+            x = block(x, cos, sin, attn_mask)
+        x = self.final_norm(x)
+        mask = valid.unsqueeze(-1)
+        mean = (x * mask).sum(dim=1) / mask.sum(dim=1).clamp(min=1)
+        maximum = torch.nan_to_num(x.masked_fill(~mask, float("-inf")).max(dim=1).values,
+                                   neginf=0.0)
+        return torch.cat((mean, maximum), dim=-1)
+
+    def forward(self, tokens: Tensor) -> dict[str, Tensor]:
+        pooled = self.encode(tokens)
+        return {
+            "domain": self.domain_head(pooled),
+            "complexity": self.complexity_dir(pooled) + self.complexity_bias,
+            "binary": self.binary_head(pooled),
+            "route": self.route_head(pooled).squeeze(-1),
+            "proj": functional.normalize(self.proj_head(pooled), dim=-1),
+        }
+
+    def calibrated(self, tokens: Tensor) -> dict[str, Tensor]:
+        """probabilities with the fitted temperatures applied. the routing
+        arithmetic in policy.py runs on these numbers, so they have to mean
+        something. temperatures were fitted on held-out data after training."""
+        out = self.forward(tokens)
+        route_t, complexity_t, binary_t = self.temperature.unbind()
+        return {
+            "domain": out["domain"].softmax(dim=-1),
+            "complexity": (out["complexity"] / complexity_t).sigmoid(),
+            "binary": (out["binary"] / binary_t).sigmoid(),
+            "route": (out["route"] / route_t).sigmoid(),
+            "proj": out["proj"],
+        }
+
+
+def complexity_from_cumulative(probabilities: Tensor) -> Tensor:
+    """coral decode. level = 1 + how many thresholds the prompt clears."""
+    return 1 + (probabilities > 0.5).sum(dim=-1)
+
+
+def encode_batch(tokenizer, texts: list[str], max_len: int) -> Tensor:
+    """bos-prefixed, right-padded. long prompts lose their tail, because the
+    instruction verb lives at the front and the pasted context lives at the back."""
+    out = torch.full((len(texts), max_len), PAD_ID, dtype=torch.long)
+    for row, text in enumerate(texts):
+        ids = [BOS_ID] + tokenizer.encode(text).ids[: max_len - 1]
+        out[row, : len(ids)] = torch.tensor(ids, dtype=torch.long)
+    return out
+
+
+def load_router(directory: Path = HERE, device: str = "cpu"):
+    """returns (model, tokenizer). reads config.json + model.safetensors + tokenizer.json."""
+    from safetensors.torch import load_file
+    from tokenizers import Tokenizer
+
+    directory = Path(directory)
+    config = RouterConfig(**json.loads((directory / "config.json").read_text())["model"])
+    model = GlintRouter(config).to(device)
+    model.load_state_dict(load_file(directory / "model.safetensors", device=device))
+    model.eval()
+    return model, Tokenizer.from_file(str(directory / "tokenizer.json"))
+
+
+def count_parameters(model: nn.Module) -> int:
+    return sum(p.numel() for p in model.parameters())

policy.json

@@ -0,0 +1,34 @@
+{
+  "tiers": [
+    {
+      "name": "local",
+      "cost": 0.0,
+      "capability": 0.35,
+      "tools": false
+    },
+    {
+      "name": "mid",
+      "cost": 1.0,
+      "capability": 0.65,
+      "tools": false
+    },
+    {
+      "name": "frontier",
+      "cost": 8.0,
+      "capability": 0.95,
+      "tools": true
+    }
+  ],
+  "failure_penalty": 50.0,
+  "sharpness": 20.0,
+  "abstain_margin": 0.05,
+  "difficulty_weights": [
+    0.6,
+    0.3,
+    0.1
+  ],
+  "overrides": [],
+  "freshness_needs_tools": true,
+  "multilingual_threshold": 0.05,
+  "prototype_threshold": 0.55
+}

policy.py

@@ -0,0 +1,239 @@
+"""Routing policy: everything you can change without touching the weights.
+
+Supra-Router-51M bakes its decision into the model. two tiers, one threshold,
+one domain vocabulary, all frozen at fine-tune time. Changing "route legal work
+to the frontier model regardless of complexity" means collecting data and
+retraining a 51M model.
+
+Here the model's job stops at *calibrated probabilities about the prompt*. What
+to do with them is this file, driven by a JSON document the user owns:
+
+  * N tiers, not 2, each with a cost and a capability
+  * expected-cost arithmetic instead of a hard-coded boolean
+  * an abstain band that escalates ties rather than coin-flipping them
+  * hard overrides on any predicted field
+  * user-defined categories matched by prototype, no retraining
+
+Also here: the freshness / tool-use / multilingual signals. Those are keyword
+and character-class rules. a learned head for them would only be memorising
+the regex below, which costs parameters and teaches nothing. They live where a
+rule belongs, and the user can edit them.
+"""
+
+from __future__ import annotations
+
+import json
+import re
+from dataclasses import asdict, dataclass, field
+from pathlib import Path
+
+import torch
+from torch import Tensor
+
+from modeling import BINARY_FIELDS, DOMAINS, GlintRouter, complexity_from_cumulative
+
+FRESHNESS = re.compile(
+    r"\b(today|tonight|current|currently|latest|right now|this (week|month|year)|"
+    r"recent|news|stock price|weather|who won|as of \d{4}|202\d|203\d)\b", re.I)
+TOOL_USE = re.compile(
+    r"\b(search the web|look ?up|browse|run this|execute|call the api|fetch|"
+    r"my file|this (csv|pdf|spreadsheet|repo)|book a|send an email|schedule)\b", re.I)
+
+
+def non_ascii_ratio(text: str) -> float:
+    letters = [c for c in text if c.isalpha()]
+    if not letters:
+        return 0.0
+    return sum(1 for c in letters if ord(c) > 127) / len(letters)
+
+
+@dataclass
+class Tier:
+    name: str
+    cost: float          # relative price of one call, any unit you like
+    capability: float    # 0..1; the difficulty this tier handles before it starts failing
+    tools: bool = False  # can this tier search / execute?
+
+
+@dataclass
+class Policy:
+    tiers: list[Tier] = field(default_factory=lambda: [
+        Tier("local", cost=0.0, capability=0.35),
+        Tier("mid", cost=1.0, capability=0.65),
+        Tier("frontier", cost=8.0, capability=0.95, tools=True),
+    ])
+    # A wrong answer must cost several times the most expensive call, or the
+    # arithmetic degenerates: when every tier is likely to fail, "pay least"
+    # beats "try hardest", and the router quietly sends its hardest prompts to
+    # the weakest model. Penalty >> max(cost) is what makes escalation rational.
+    failure_penalty: float = 50.0   # what a wrong answer costs, in the same units as `cost`
+    # Sharpness sets the residual failure rate a tier carries on prompts well
+    # inside its capability, and it has to be steep. At sharpness 12 the local
+    # tier still shows a 2.6% failure rate on a difficulty-0.05 prompt, which
+    # priced at `failure_penalty` (1.33) loses to the mid tier's flat cost of
+    # 1.0, so a free tier that handles the prompt perfectly never gets used.
+    # 20 puts that residual at 0.25% and the cheap tier wins what it should.
+    sharpness: float = 20.0
+    abstain_margin: float = 0.05    # ties inside this band escalate instead of coin-flipping
+    difficulty_weights: tuple[float, float, float] = (0.6, 0.3, 0.1)  # route, complexity, code/math
+    # {"when": {...}, "tier": "frontier"}. first match wins, checked before the arithmetic.
+    overrides: list[dict] = field(default_factory=list)
+    freshness_needs_tools: bool = True
+    multilingual_threshold: float = 0.05
+    prototype_threshold: float = 0.55
+
+    @classmethod
+    def load(cls, path: Path | None) -> Policy:
+        if path is None:
+            return cls()
+        blob = json.loads(Path(path).read_text())
+        tiers = [Tier(**t) for t in blob.pop("tiers", [])]
+        policy = cls(**blob)
+        if tiers:
+            policy.tiers = tiers
+        return policy
+
+    def save(self, path: Path) -> None:
+        Path(path).write_text(json.dumps(asdict(self), indent=2))
+
+
+@dataclass
+class Decision:
+    tier: str
+    domain: str
+    complexity: int
+    difficulty: float
+    probabilities: dict[str, float]
+    signals: dict[str, bool]
+    reason: str
+    category: str | None = None
+    escalated: bool = False
+
+    def as_line(self) -> str:
+        """Supra's output format, so the two are directly comparable."""
+        p = self.probabilities
+        return (f"Domain: {self.domain} | Complexity: {self.complexity} | "
+                f"Math: {'T' if p['math'] > 0.5 else 'F'} | "
+                f"Code: {'T' if p['code'] > 0.5 else 'F'} | "
+                f"Route: {self.tier} | Justification: {self.reason}")
+
+
+class Prototypes:
+    """User-defined categories, added from a handful of examples at runtime.
+
+    The projection head is trained jointly with the classifier, so prompts that
+    share a routing-relevant character land near each other. A new category is
+    the mean of its examples' projections. adding one is a forward pass over
+    5-10 prompts, not a training run.
+    """
+
+    def __init__(self, vectors: dict[str, list[float]] | None = None) -> None:
+        self.vectors = {k: torch.tensor(v) for k, v in (vectors or {}).items()}
+
+    def add(self, name: str, projections: Tensor) -> None:
+        centroid = projections.mean(dim=0)
+        self.vectors[name] = centroid / centroid.norm().clamp(min=1e-6)
+
+    def match(self, projection: Tensor, threshold: float) -> tuple[str | None, float]:
+        if not self.vectors:
+            return None, 0.0
+        names = list(self.vectors)
+        bank = torch.stack([self.vectors[n] for n in names]).to(projection.device)
+        scores = bank @ projection
+        best = int(scores.argmax())
+        score = float(scores[best])
+        return (names[best], score) if score >= threshold else (None, score)
+
+    def save(self, path: Path) -> None:
+        Path(path).write_text(json.dumps({k: v.tolist() for k, v in self.vectors.items()}))
+
+    @classmethod
+    def load(cls, path: Path | None) -> Prototypes:
+        if path is None or not Path(path).is_file():
+            return cls()
+        return cls(json.loads(Path(path).read_text()))
+
+
+def difficulty_of(route_p: float, complexity: float, code_p: float, math_p: float,
+                  weights: tuple[float, float, float]) -> float:
+    w_route, w_complexity, w_technical = weights
+    scaled_complexity = (complexity - 1.0) / 4.0
+    technical = max(code_p, math_p)
+    total = w_route + w_complexity + w_technical
+    return (w_route * route_p + w_complexity * scaled_complexity
+            + w_technical * technical) / max(total, 1e-6)
+
+
+def _override_hit(rule: dict, domain: str, complexity: int, probs: dict[str, float],
+                  signals: dict[str, bool], category: str | None) -> bool:
+    when = rule.get("when", {})
+    if "domain" in when and when["domain"] != domain:
+        return False
+    if "category" in when and when["category"] != category:
+        return False
+    if "complexity_min" in when and complexity < when["complexity_min"]:
+        return False
+    if "complexity_max" in when and complexity > when["complexity_max"]:
+        return False
+    for f in BINARY_FIELDS:
+        if f in when and bool(when[f]) != (probs[f] > 0.5):
+            return False
+    for s in ("freshness", "tool_use", "multilingual"):
+        if s in when and bool(when[s]) != signals[s]:
+            return False
+    return True
+
+
+def decide(model: GlintRouter, tokens: Tensor, text: str, policy: Policy,
+           prototypes: Prototypes | None = None) -> Decision:
+    """One prompt -> one routing decision. `tokens` is a (1, seq) batch."""
+    with torch.no_grad():
+        out = model.calibrated(tokens)
+    probs = {f: float(out["binary"][0, i]) for i, f in enumerate(BINARY_FIELDS)}
+    probs["route_big"] = float(out["route"][0])
+    domain = DOMAINS[int(out["domain"][0].argmax())]
+    complexity_p = out["complexity"][0]
+    complexity = int(complexity_from_cumulative(complexity_p))
+    expected_complexity = 1.0 + float(complexity_p.sum())
+
+    signals = {
+        "freshness": bool(FRESHNESS.search(text)),
+        "tool_use": bool(TOOL_USE.search(text)),
+        "multilingual": non_ascii_ratio(text) > policy.multilingual_threshold,
+    }
+    category, _ = (prototypes.match(out["proj"][0], policy.prototype_threshold)
+                   if prototypes else (None, 0.0))
+
+    for rule in policy.overrides:
+        if _override_hit(rule, domain, complexity, probs, signals, category):
+            return Decision(rule["tier"], domain, complexity, 0.0, probs, signals,
+                            f"override: {json.dumps(rule.get('when', {}))}", category)
+
+    difficulty = difficulty_of(probs["route_big"], expected_complexity,
+                               probs["code"], probs["math"], policy.difficulty_weights)
+
+    tiers = policy.tiers
+    if (signals["freshness"] or signals["tool_use"]) and policy.freshness_needs_tools:
+        with_tools = [t for t in tiers if t.tools]
+        if with_tools:
+            tiers = with_tools
+
+    costs = []
+    for tier in tiers:
+        fail = torch.sigmoid(torch.tensor(
+            (difficulty - tier.capability) * policy.sharpness)).item()
+        costs.append((tier.cost + policy.failure_penalty * fail, tier))
+    costs.sort(key=lambda pair: pair[0])
+
+    best_cost, best = costs[0]
+    escalated = False
+    if len(costs) > 1 and (costs[1][0] - best_cost) <= policy.abstain_margin * best_cost:
+        runner_up = costs[1][1]
+        if runner_up.capability > best.capability:
+            best, escalated = runner_up, True
+
+    reason = (f"difficulty={difficulty:.2f} complexity={complexity} "
+              f"code={probs['code']:.2f} math={probs['math']:.2f}"
+              + (" (escalated: tie inside abstain band)" if escalated else ""))
+    return Decision(best.name, domain, complexity, difficulty, probs, signals,
+                    reason, category, escalated)

refund_examples.txt

@@ -0,0 +1,6 @@
+I want a refund for my order, it arrived broken.
+How do I return this item and get my money back?
+My package never showed up, can I be refunded?
+Cancel my subscription and refund last month.
+The product is defective, I demand a refund.
+Requesting reimbursement for a duplicate charge.

requirements.txt

@@ -0,0 +1,3 @@
+torch
+tokenizers
+safetensors

scores.json

@@ -0,0 +1,179 @@
+{
+  "model": "Glint-Router-1M",
+  "author": "Glint-Research",
+  "parameters": 985826,
+  "baseline": "SupraLabs/Supra-Router-51M (51,786,240 params)",
+  "training": {
+    "params": 985826,
+    "train_rows": 87604,
+    "val_rows": 7617,
+    "steps": 14479,
+    "minutes": 15.0,
+    "domain": 0.9464356045687279,
+    "route": 0.939608769856899,
+    "complexity": 0.7923066824209006,
+    "code": 0.991466456610214,
+    "math": 0.978075357752396,
+    "reasoning": 0.8949717736641722,
+    "long_output": 0.9516870158855192,
+    "ece_raw": 0.05564934242829622,
+    "ece_calibrated": 0.02029288645098859,
+    "hardware": "RTX 3060 Ti",
+    "rounds": 1,
+    "wall_clock_minutes": 15.0,
+    "corpus_rows": 95221
+  },
+  "neutral_holdout": {
+    "val_rows": 3418,
+    "slice_rows": {
+      "aqua_rat": 254,
+      "dolly": 2500,
+      "humaneval": 164,
+      "mbpp": 500
+    },
+    "glint": {
+      "route_accuracy": 0.9441193680514921,
+      "code_accuracy": 0.9751316559391457,
+      "math_accuracy": 0.9526038619075483,
+      "complexity_exact": 0.46284376828554713,
+      "complexity_within_1": 0.9590403744880047,
+      "ms_per_prompt": 0.05992146021082586,
+      "parse_failures": 0
+    },
+    "glint_aqua_rat": {
+      "route_accuracy": 0.9094488188976378,
+      "code_accuracy": 0.9960629921259843,
+      "math_accuracy": 0.889763779527559,
+      "complexity_exact": 0.1141732283464567,
+      "complexity_within_1": 0.8070866141732284,
+      "ms_per_prompt": 0.05992146021082586,
+      "parse_failures": 0
+    },
+    "glint_dolly": {
+      "route_accuracy": 0.9424,
+      "code_accuracy": 0.9956,
+      "math_accuracy": 0.9612,
+      "complexity_exact": 0.5688,
+      "complexity_within_1": 0.9944,
+      "ms_per_prompt": 0.05992146021082586,
+      "parse_failures": 0
+    },
+    "glint_humaneval": {
+      "route_accuracy": 0.8719512195121951,
+      "code_accuracy": 0.5670731707317073,
+      "math_accuracy": 0.9024390243902439,
+      "complexity_exact": 0.4451219512195122,
+      "complexity_within_1": 0.9817073170731707,
+      "ms_per_prompt": 0.05992146021082586,
+      "parse_failures": 0
+    },
+    "glint_mbpp": {
+      "route_accuracy": 0.994,
+      "code_accuracy": 0.996,
+      "math_accuracy": 0.958,
+      "complexity_exact": 0.116,
+      "complexity_within_1": 0.852,
+      "ms_per_prompt": 0.05992146021082586,
+      "parse_failures": 0
+    },
+    "supra": {
+      "route_accuracy": 0.903744880046811,
+      "code_accuracy": 0.9643066120538326,
+      "math_accuracy": 0.9207138677589234,
+      "complexity_exact": 0.3905792861322411,
+      "complexity_within_1": 0.9163253364540667,
+      "ms_per_prompt": 14.334112498536923,
+      "parse_failures": 4
+    },
+    "supra_aqua_rat": {
+      "route_accuracy": 0.9133858267716536,
+      "code_accuracy": 1.0,
+      "math_accuracy": 0.9094488188976378,
+      "complexity_exact": 0.0,
+      "complexity_within_1": 0.09448818897637795,
+      "ms_per_prompt": 14.334112498536923,
+      "parse_failures": 4
+    },
+    "supra_dolly": {
+      "route_accuracy": 0.8896,
+      "code_accuracy": 0.996,
+      "math_accuracy": 0.9428,
+      "complexity_exact": 0.482,
+      "complexity_within_1": 0.9776,
+      "ms_per_prompt": 14.334112498536923,
+      "parse_failures": 4
+    },
+    "supra_humaneval": {
+      "route_accuracy": 0.8292682926829268,
+      "code_accuracy": 0.676829268292683,
+      "math_accuracy": 0.8414634146341463,
+      "complexity_exact": 0.16463414634146342,
+      "complexity_within_1": 1.0,
+      "ms_per_prompt": 14.334112498536923,
+      "parse_failures": 4
+    },
+    "supra_mbpp": {
+      "route_accuracy": 0.994,
+      "code_accuracy": 0.882,
+      "math_accuracy": 0.842,
+      "complexity_exact": 0.206,
+      "complexity_within_1": 1.0,
+      "ms_per_prompt": 14.334112498536923,
+      "parse_failures": 4
+    }
+  },
+  "in_corpus_val": {
+    "val_rows": 7617,
+    "glint": {
+      "route_accuracy": 0.939608769856899,
+      "code_accuracy": 0.991466456610214,
+      "math_accuracy": 0.978075357752396,
+      "complexity_exact": 0.7923066824209006,
+      "complexity_within_1": 0.9883156098201391,
+      "ms_per_prompt": 0.0748591950898926,
+      "parse_failures": 0
+    },
+    "glint_trap": {
+      "route_accuracy": 1.0,
+      "code_accuracy": 1.0,
+      "math_accuracy": 1.0,
+      "complexity_exact": 1.0,
+      "complexity_within_1": 1.0,
+      "parse_failures": 0
+    },
+    "supra": {
+      "route_accuracy": 0.8575554680320336,
+      "code_accuracy": 0.9761060785085992,
+      "math_accuracy": 0.9400026257056584,
+      "complexity_exact": 0.4629119075751608,
+      "complexity_within_1": 0.923723250623605,
+      "ms_per_prompt": 14.768008985689887,
+      "parse_failures": 1
+    },
+    "supra_trap": {
+      "route_accuracy": 0.908695652173913,
+      "code_accuracy": 0.9869565217391304,
+      "math_accuracy": 1.0,
+      "complexity_exact": 0.33043478260869563,
+      "complexity_within_1": 0.8391304347826087,
+      "parse_failures": 1
+    },
+    "note": "labels from the glint pipeline. glint is in-distribution here, supra is not. read the holdout numbers for the fair comparison."
+  },
+  "supra_own_rows": {
+    "val_rows": 91,
+    "glint": {
+      "route_accuracy": 0.8901098901098901,
+      "code_accuracy": 0.9010989010989011,
+      "math_accuracy": 0.8131868131868132,
+      "complexity_exact": 0.5054945054945055
+    },
+    "supra": {
+      "route_accuracy": 0.967032967032967,
+      "code_accuracy": 1.0,
+      "math_accuracy": 0.978021978021978,
+      "complexity_exact": 0.8571428571428571
+    },
+    "note": "supra's own 992-row dataset, held out of glint training. these rows are supra's TRAINING data, so supra is scoring a memorised set. published because leaving it out would be dishonest."
+  }
+}