Update README.md
Browse files
README.md
CHANGED
|
@@ -22,7 +22,7 @@ language:
|
|
| 22 |
|
| 23 |
# Glint-Router-1M
|
| 24 |
|
| 25 |
-
a prompt router in 985,826 parameters. you hand it a user request, it hands back a domain, a complexity score, code/math/reasoning flags, and which model tier should take the job. one forward pass, around 0.06 ms on a gpu, 0.2 ms on cpu.
|
| 26 |
|
| 27 |
the target was SupraLabs/Supra-Router-51M, which does the same job with 51,786,240 parameters. we came in at 1/52nd the size and beat it on data neither model had seen.
|
| 28 |
|
|
@@ -116,7 +116,7 @@ expected calibration error, routing head
|
|
| 116 |
|
| 117 |
the fitted temperature on the routing head is 4.71, which is a blunt statement about how overconfident a small model gets when you let it see the same 95k rows forty times. the scalars ship inside the weights and `calibrated()` applies them for you.
|
| 118 |
|
| 119 |
-
## routing
|
| 120 |
|
| 121 |
supra decides with a boolean baked into its weights: complexity >= 3, or code, or math, means big model. two tiers. want three, or a different threshold, or "legal always goes to the frontier model"? you are collecting data and retraining 51M parameters.
|
| 122 |
|
|
@@ -146,6 +146,7 @@ the decision is `argmin over tiers of cost + failure_penalty * P(tier fails)`, w
|
|
| 146 |
two constants in there were tuned the hard way and we left the reasoning in the file. `failure_penalty` has to be several times the priciest call, or "pay least" beats "try hardest" when every tier is likely to fail, and the router quietly sends its hardest prompts to its weakest model. `sharpness` has to be steep: at 12, a free local tier still carries a 2.6% residual failure rate, which priced at the penalty is 1.33 and loses to the mid tier's flat cost of 1.0. the free tier won nothing until that was fixed.
|
| 147 |
|
| 148 |
## adding a category it never trained on
|
|
|
|
| 149 |
|
| 150 |
the 64-d projection head is trained alongside the classifier, so prompts that route alike land near each other. a new category is the mean of a handful of examples:
|
| 151 |
|
|
@@ -155,6 +156,7 @@ python infer.py --add-category refunds refund_examples.txt
|
|
| 155 |
|
| 156 |
six example prompts, one forward pass, no optimizer, no gradients. after that `decision.category` comes back as `refunds` on matching prompts and you can route on it with an override. this is the thing a 51M generator cannot do at all without a fine-tune.
|
| 157 |
|
|
|
|
| 158 |
## where it fails
|
| 159 |
|
| 160 |
a 1M parameter model has edges and we would rather you learn them here than in production.
|
|
@@ -213,7 +215,7 @@ policy.json the default 3-tier policy, edit this one
|
|
| 213 |
infer.py cli: route, batch, add a category
|
| 214 |
scores.json every number on this page, machine readable
|
| 215 |
requirements.txt torch, tokenizers, safetensors
|
| 216 |
-
README.md you are here
|
| 217 |
```
|
| 218 |
|
| 219 |
## quickstart
|
|
|
|
| 22 |
|
| 23 |
# Glint-Router-1M
|
| 24 |
|
| 25 |
+
a prompt router in 985,826 parameters. you hand it a user request, it hands back a domain, a complexity score, code/math/reasoning flags, and which model tier should take the job. one forward pass, around 0.06 ms on a gpu, 0.2 ms on cpu.
|
| 26 |
|
| 27 |
the target was SupraLabs/Supra-Router-51M, which does the same job with 51,786,240 parameters. we came in at 1/52nd the size and beat it on data neither model had seen.
|
| 28 |
|
|
|
|
| 116 |
|
| 117 |
the fitted temperature on the routing head is 4.71, which is a blunt statement about how overconfident a small model gets when you let it see the same 95k rows forty times. the scalars ship inside the weights and `calibrated()` applies them for you.
|
| 118 |
|
| 119 |
+
## routing stuff
|
| 120 |
|
| 121 |
supra decides with a boolean baked into its weights: complexity >= 3, or code, or math, means big model. two tiers. want three, or a different threshold, or "legal always goes to the frontier model"? you are collecting data and retraining 51M parameters.
|
| 122 |
|
|
|
|
| 146 |
two constants in there were tuned the hard way and we left the reasoning in the file. `failure_penalty` has to be several times the priciest call, or "pay least" beats "try hardest" when every tier is likely to fail, and the router quietly sends its hardest prompts to its weakest model. `sharpness` has to be steep: at 12, a free local tier still carries a 2.6% residual failure rate, which priced at the penalty is 1.33 and loses to the mid tier's flat cost of 1.0. the free tier won nothing until that was fixed.
|
| 147 |
|
| 148 |
## adding a category it never trained on
|
| 149 |
+
This is the part that I played with for HOURS. Very happy & exited about it.
|
| 150 |
|
| 151 |
the 64-d projection head is trained alongside the classifier, so prompts that route alike land near each other. a new category is the mean of a handful of examples:
|
| 152 |
|
|
|
|
| 156 |
|
| 157 |
six example prompts, one forward pass, no optimizer, no gradients. after that `decision.category` comes back as `refunds` on matching prompts and you can route on it with an override. this is the thing a 51M generator cannot do at all without a fine-tune.
|
| 158 |
|
| 159 |
+
|
| 160 |
## where it fails
|
| 161 |
|
| 162 |
a 1M parameter model has edges and we would rather you learn them here than in production.
|
|
|
|
| 215 |
infer.py cli: route, batch, add a category
|
| 216 |
scores.json every number on this page, machine readable
|
| 217 |
requirements.txt torch, tokenizers, safetensors
|
| 218 |
+
README.md you are here. I hope.
|
| 219 |
```
|
| 220 |
|
| 221 |
## quickstart
|