| --- |
| title: quant-control-bench — how few bits before it falls over? |
| emoji: 🦿 |
| colorFrom: green |
| colorTo: gray |
| sdk: static |
| pinned: false |
| license: mit |
| --- |
| |
| <p align="center"><img src="assets/logo.svg" alt="quant-control-bench" width="820"></p> |
|
|
| # How few bits does a closed-loop robot controller need? |
|
|
| <p align="center"><img src="assets/demo.gif" alt="three precisions walking" width="820"></p> |
|
|
| The same trained Go1 policy, quantized several ways, walking in real MuJoCo |
| physics compiled to WebAssembly. Everything runs in your browser: the |
| simulation, the neural networks, and the timing. |
|
|
| Pick precisions, drive them with the joystick sliders, then break them: |
| push the robots, change torso mass and ground friction, add actuator delay |
| or observation noise. Every variant shares an initial state and a command, |
| so the only difference between them is the precision of the weights. |
|
|
| Weights are loaded from [happynood/quant-control-bench-policies](https://huggingface.co/happynood/quant-control-bench-policies), |
| which is the single source of truth for them. |
|
|
| Methodology, raw results and the code: https://github.com/Happynood/quant-control-bench |
|
|
| ## What the benchmark found |
|
|
| - int8 costs nothing measurable, on return or on any of five robustness axes. |
| - The interesting boundary is between int8 and int4 — and at 4 bits, *how* you |
| group the scales matters more than the bit width does. |
| - Quantizing the observation-normalization statistics, 0.05% of the parameters, |
| breaks the policy outright at 4 bits. |
|
|
| The live figures on this page (steps survived, whether the robot fell, action |
| jitter, inference milliseconds) are computed in your browser. The benchmark |
| tables come from recorded runs on an RTX 3050 Laptop and are not recomputed |
| here. |
|
|
