README.md

---
license: mit
library_name: onnx
tags:
  - robotics
  - reinforcement-learning
  - quantization
  - mujoco
pipeline_tag: robotics
---

<p align="center"><img src="assets/logo.svg" alt="quant-control-bench" width="820"></p>

# Go1 locomotion policies at nine precisions

<p align="center"><a href="https://huggingface.co/spaces/happynood/quant-control-bench-demo"><b>Try them in your browser</b></a> · <a href="https://github.com/Happynood/quant-control-bench"><b>Code and methodology</b></a></p>

<p align="center"><img src="assets/demo.gif" alt="three precisions walking" width="820"></p>

One PPO policy for `Go1JoystickFlatTerrain`, exported to ONNX nine times at
different weight precisions, plus the scene it was trained in. The point of the
set is the comparison: how far precision can be reduced before a closed-loop
controller stops working, and where that boundary actually sits.

Full methodology, code and raw results: https://github.com/Happynood/quant-control-bench

## Results

100 episodes x 5 fixed seeds, 1000-step horizon, deterministic policy (mean
action, no sampling). Return deltas are paired bootstrap 95% intervals over
10 000 resamples. `P50` is the perturbation magnitude at which the success rate
crosses 50%; on `friction_scale`, swept downward, a larger number is worse.

| scheme | bits/weight | mean return | Δreturn vs fp32 (95% CI) | P50 friction | P50 obs noise |
|---|---|---|---|---|---|
| `fp32` | 32.00 | 31.65 | — | 0.189 | 0.142 |
| `fp16` | 16.00 | 31.65 | no measurable loss | 0.192 | 0.143 |
| `int8-tensor` | 8.00 | 31.68 | no measurable loss | 0.200 | 0.142 |
| `int8-channel` | 8.00 | 31.66 | no measurable loss | 0.205 | 0.142 |
| `int4-channel` | 4.00 | 30.91 | -2.336% [-2.79%, -1.95%] | 0.241 | 0.127 |
| `int4-group32` | 4.00 | 31.42 | -0.728% [-0.88%, -0.45%] | 0.207 | 0.140 |
| `ternary` | 1.58 | 2.95 | -90.672% [-92.14%, -89.13%] | 1.000 | 0.000 |
| `mixed-head-fp16` | 4.19 | 31.36 | -0.902% [-1.26%, -0.70%] | 0.198 | 0.136 |
| `int8-act` | 8.00 | 31.65 | no measurable loss | 0.193 | 0.142 |

## What the numbers say

- **int8 is free.** Every int8 variant, including one that quantizes activations
  as well as weights, is statistically indistinguishable from fp32 on return and
  on all five robustness axes.
- **The boundary is between int8 and int4, and grouping matters more than bits.**
  `int4-channel` loses only 2.3% of return on flat ground yet its friction and
  observation-noise frontiers separate from fp32 with non-overlapping intervals.
  `int4-group32`, at the same 4.00 bits, does not.
- **Quantizing the observation-normalization statistics is catastrophic.** They
  are 0.05% of the parameters. Quantized with the same scheme as the weights,
  every int8 variant becomes measurably lossy and every 4-bit variant stops
  producing finite actions at all, because a strictly positive scale vector
  quantized symmetrically rounds entries to zero and the policy divides by them.

## Intended use and limitations

These are research artifacts for studying quantization of closed-loop control.
They are not tuned for deployment on hardware and have never been run on a
physical robot.

- **Quantization is simulated.** Weights are rounded to the target grid and
  stored back as float32, so every ONNX file is the same size and none of them
  runs faster. The benchmark measures what precision does to control, not what
  it does to storage or throughput. A deployment would need packed kernels, and
  the accuracy results here would carry over while the timing results would not.
- **One training seed.** Precision effects cannot be fully separated from the
  luck of a single checkpoint.
- **One task, flat terrain.** `Go1JoystickFlatTerrain` only. The headline
  hypothesis (that open-loop action error mispredicts closed-loop performance)
  is *not* supported on this task, and that negative result is reported with the
  same prominence as the positive ones.
- **Browser physics differs from training physics.** The demo runs MuJoCo 3.3.8
  compiled to WebAssembly; training used MJX with MuJoCo 3.10.0. Measured
  divergence and the reasoning behind it are in the repository's methodology.

## Training

| | |
|---|---|
| Environment steps executed | 206,438,400 |
| Wall clock | 140.0 min |
| Parallel environments | 8,192 |
| Peak VRAM | 2225 MiB |
| Final training reward | 31.772 ± 1.810 |
| Weight extraction vs Brax | 4.47e-06 |
| Hardware | NVIDIA RTX 3050 Laptop, 4096 MiB |

Every ONNX graph carries the observation normalization inside it, so the input
is the raw 48-dim observation and the output is the tanh-squashed action.