Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -56,10 +56,14 @@ risk: low | medium | high
|
|
| 56 |
- `task_type` β model specialty (e.g. design β Claude, systems β GPT, docs β small)
|
| 57 |
- `risk` β tier bumper (easy + high-risk still routes capable) and confirmation gate
|
| 58 |
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
|
| 64 |
Routing is **kickoff-only**: classify once at task start and lock the model for
|
| 65 |
the whole task cycle (no per-turn re-routing β no context thrash).
|
|
@@ -125,11 +129,19 @@ def classify(text: str) -> dict:
|
|
| 125 |
result[name.replace("_logits", "")] = {
|
| 126 |
"label": labels[i], "confidence": round(float(p[i]), 4),
|
| 127 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 128 |
return result
|
| 129 |
|
| 130 |
print(classify("refactor JWT key rotation in prod"))
|
| 131 |
-
# {'complexity': {'label':
|
| 132 |
-
# '
|
| 133 |
```
|
| 134 |
|
| 135 |
## Evaluation
|
|
@@ -184,7 +196,11 @@ action without a human gate; languages other than English (trained on English).
|
|
| 184 |
the intended path past this; this checkpoint predates that loop.
|
| 185 |
- **Synthetic-label ceiling.** Much training data is LLM-labeled; expect a
|
| 186 |
~75β80% ceiling per head until real disagreement signals are mixed in.
|
| 187 |
-
- **
|
|
|
|
|
|
|
|
|
|
|
|
|
| 188 |
|
| 189 |
## Training
|
| 190 |
|
|
|
|
| 56 |
- `task_type` β model specialty (e.g. design β Claude, systems β GPT, docs β small)
|
| 57 |
- `risk` β tier bumper (easy + high-risk still routes capable) and confirmation gate
|
| 58 |
|
| 59 |
+
The ONNX graphs emit the three classification logits plus all eight regression
|
| 60 |
+
sub-dimension scores. Output names are:
|
| 61 |
+
|
| 62 |
+
- `complexity_logits`, `task_type_logits`, `risk_logits`
|
| 63 |
+
- `complexity_sub_reasoning_depth`, `complexity_sub_scope_breadth`,
|
| 64 |
+
`complexity_sub_domain_knowledge`, `complexity_sub_spec_completeness`
|
| 65 |
+
- `risk_sub_security_surface`, `risk_sub_data_sensitivity`,
|
| 66 |
+
`risk_sub_production_exposure`, `risk_sub_reversal_cost`
|
| 67 |
|
| 68 |
Routing is **kickoff-only**: classify once at task start and lock the model for
|
| 69 |
the whole task cycle (no per-turn re-routing β no context thrash).
|
|
|
|
| 129 |
result[name.replace("_logits", "")] = {
|
| 130 |
"label": labels[i], "confidence": round(float(p[i]), 4),
|
| 131 |
}
|
| 132 |
+
result["complexity_sub"] = {
|
| 133 |
+
name.replace("complexity_sub_", ""): round(float(out[name][0]), 4)
|
| 134 |
+
for name in out if name.startswith("complexity_sub_")
|
| 135 |
+
}
|
| 136 |
+
result["risk_sub"] = {
|
| 137 |
+
name.replace("risk_sub_", ""): round(float(out[name][0]), 4)
|
| 138 |
+
for name in out if name.startswith("risk_sub_")
|
| 139 |
+
}
|
| 140 |
return result
|
| 141 |
|
| 142 |
print(classify("refactor JWT key rotation in prod"))
|
| 143 |
+
# {'complexity': {'label': ...}, 'task_type': {'label': ...}, 'risk': {'label': ...},
|
| 144 |
+
# 'complexity_sub': {'reasoning_depth': ...}, 'risk_sub': {'security_surface': ...}}
|
| 145 |
```
|
| 146 |
|
| 147 |
## Evaluation
|
|
|
|
| 196 |
the intended path past this; this checkpoint predates that loop.
|
| 197 |
- **Synthetic-label ceiling.** Much training data is LLM-labeled; expect a
|
| 198 |
~75β80% ceiling per head until real disagreement signals are mixed in.
|
| 199 |
+
- **Quantized serving tradeoff.** The fp32 ONNX graph matches the PyTorch model
|
| 200 |
+
on the locked battery, including sub-dimension scores. The int8 graph is the
|
| 201 |
+
recommended low-dependency serving artifact and preserves the established v6
|
| 202 |
+
serving behavior, but dynamic quantization can move borderline labels and
|
| 203 |
+
regression values.
|
| 204 |
|
| 205 |
## Training
|
| 206 |
|