CompactAI commited on
Commit
532b111
·
verified ·
1 Parent(s): d014f39

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -3
README.md CHANGED
@@ -22,7 +22,7 @@ language:
22
 
23
  # Glint-Router-1M
24
 
25
- a prompt router in 985,826 parameters. you hand it a user request, it hands back a domain, a complexity score, code/math/reasoning flags, and which model tier should take the job. one forward pass, around 0.06 ms on a gpu, 0.2 ms on cpu. the release: weights, tokenizer, two inference files, the routing policy as an editable json document, every score we measured. no training code, no pipeline, no framework. `pip install torch tokenizers safetensors` and you are routing.
26
 
27
  the target was SupraLabs/Supra-Router-51M, which does the same job with 51,786,240 parameters. we came in at 1/52nd the size and beat it on data neither model had seen.
28
 
@@ -116,7 +116,7 @@ expected calibration error, routing head
116
 
117
  the fitted temperature on the routing head is 4.71, which is a blunt statement about how overconfident a small model gets when you let it see the same 95k rows forty times. the scalars ship inside the weights and `calibrated()` applies them for you.
118
 
119
- ## routing is a policy, not a threshold
120
 
121
  supra decides with a boolean baked into its weights: complexity >= 3, or code, or math, means big model. two tiers. want three, or a different threshold, or "legal always goes to the frontier model"? you are collecting data and retraining 51M parameters.
122
 
@@ -146,6 +146,7 @@ the decision is `argmin over tiers of cost + failure_penalty * P(tier fails)`, w
146
  two constants in there were tuned the hard way and we left the reasoning in the file. `failure_penalty` has to be several times the priciest call, or "pay least" beats "try hardest" when every tier is likely to fail, and the router quietly sends its hardest prompts to its weakest model. `sharpness` has to be steep: at 12, a free local tier still carries a 2.6% residual failure rate, which priced at the penalty is 1.33 and loses to the mid tier's flat cost of 1.0. the free tier won nothing until that was fixed.
147
 
148
  ## adding a category it never trained on
 
149
 
150
  the 64-d projection head is trained alongside the classifier, so prompts that route alike land near each other. a new category is the mean of a handful of examples:
151
 
@@ -155,6 +156,7 @@ python infer.py --add-category refunds refund_examples.txt
155
 
156
  six example prompts, one forward pass, no optimizer, no gradients. after that `decision.category` comes back as `refunds` on matching prompts and you can route on it with an override. this is the thing a 51M generator cannot do at all without a fine-tune.
157
 
 
158
  ## where it fails
159
 
160
  a 1M parameter model has edges and we would rather you learn them here than in production.
@@ -213,7 +215,7 @@ policy.json the default 3-tier policy, edit this one
213
  infer.py cli: route, batch, add a category
214
  scores.json every number on this page, machine readable
215
  requirements.txt torch, tokenizers, safetensors
216
- README.md you are here
217
  ```
218
 
219
  ## quickstart
 
22
 
23
  # Glint-Router-1M
24
 
25
+ a prompt router in 985,826 parameters. you hand it a user request, it hands back a domain, a complexity score, code/math/reasoning flags, and which model tier should take the job. one forward pass, around 0.06 ms on a gpu, 0.2 ms on cpu.
26
 
27
  the target was SupraLabs/Supra-Router-51M, which does the same job with 51,786,240 parameters. we came in at 1/52nd the size and beat it on data neither model had seen.
28
 
 
116
 
117
  the fitted temperature on the routing head is 4.71, which is a blunt statement about how overconfident a small model gets when you let it see the same 95k rows forty times. the scalars ship inside the weights and `calibrated()` applies them for you.
118
 
119
+ ## routing stuff
120
 
121
  supra decides with a boolean baked into its weights: complexity >= 3, or code, or math, means big model. two tiers. want three, or a different threshold, or "legal always goes to the frontier model"? you are collecting data and retraining 51M parameters.
122
 
 
146
  two constants in there were tuned the hard way and we left the reasoning in the file. `failure_penalty` has to be several times the priciest call, or "pay least" beats "try hardest" when every tier is likely to fail, and the router quietly sends its hardest prompts to its weakest model. `sharpness` has to be steep: at 12, a free local tier still carries a 2.6% residual failure rate, which priced at the penalty is 1.33 and loses to the mid tier's flat cost of 1.0. the free tier won nothing until that was fixed.
147
 
148
  ## adding a category it never trained on
149
+ This is the part that I played with for HOURS. Very happy & exited about it.
150
 
151
  the 64-d projection head is trained alongside the classifier, so prompts that route alike land near each other. a new category is the mean of a handful of examples:
152
 
 
156
 
157
  six example prompts, one forward pass, no optimizer, no gradients. after that `decision.category` comes back as `refunds` on matching prompts and you can route on it with an override. this is the thing a 51M generator cannot do at all without a fine-tune.
158
 
159
+
160
  ## where it fails
161
 
162
  a 1M parameter model has edges and we would rather you learn them here than in production.
 
215
  infer.py cli: route, batch, add a category
216
  scores.json every number on this page, machine readable
217
  requirements.txt torch, tokenizers, safetensors
218
+ README.md you are here. I hope.
219
  ```
220
 
221
  ## quickstart