Update latency figures; document fp16/CUDA-graph/TensorRT deployment

README.md

@@ -41,13 +41,13 @@ F-measure.
 | `F_beta^w` (beta^2 = 0.3), this checkpoint | 0.8582 |
 | `F_beta^w` (beta^2 = 1), mean of 3 seeds | 0.8675 ± 0.0009 |
 | Mask quality on detected instances | 0.8933 |
-| Latency, 640x640, fp32, RTX 6000 Ada | 23.8 ms median |
-| Throughput | 41.7 img/s (42 FPS) |
+| Latency, 640x640, fp32, RTX 6000 Ada | 16.3 ms median |
+| Throughput | 61.5 img/s |
 
 Against published baselines on the same benchmark: Mask R-CNN 0.844 at 45 ms,
 deterministic Swin-T 0.883 at 42 ms, Bayesian Swin-T deep ensemble 0.906 at
-roughly 1015 ms. AffKernel beats the Mask R-CNN baseline at about half its
-latency and trails the deterministic Swin-T by 1.55 points at 57% of its
+roughly 1015 ms. AffKernel beats the Mask R-CNN baseline at about a third of its
+latency and trails the deterministic Swin-T by 1.55 points at 39% of its
 latency, with fully deterministic single-pass inference.
 
 **On seed selection.** This is seed 42, which is both the primary anchor seed
@@ -110,6 +110,81 @@ The first run downloads ImageNet-pretrained ResNet-50vd backbone weights from
 the RT-DETR release artefacts, so it needs network access. For an offline
 deployment, set `PResNet.pretrained: False` in the config.
 
+## Deployment
+
+All rows below use **these same weights** — fp16, CUDA graphs, BatchNorm folding
+and the TensorRT backbone are runtime configurations, not separate checkpoints.
+
+**RTX 6000 Ada**, 640x640, batch 1, end to end (preprocess, forward, mask decode,
+device-to-host copy):
+
+| Inference path | End to end | FPS | Peak GPU | `F_beta^w` (0.3 / 1) |
+|---|---:|---:|---:|---:|
+| fp32, as released | 35.7 ms | 28.0 | 0.43 GiB | 0.8582 / 0.8685 |
+| fp16 + CUDA graph + BN fold | **16.0 ms** | **62.7** | 0.29 GiB | 0.8577 / 0.8680 |
+
+**Jetson AGX Orin** Developer Kit (64 GB), JetPack 6.2, MAXN:
+
+| Inference path | End to end | FPS | Peak GPU | `F_beta^w` (0.3 / 1) |
+|---|---:|---:|---:|---:|
+| fp32, as released | 199.5 ms | 5.0 | 1.29 GiB | 0.8598 / 0.8680 |
+| `--half --cudagraph` | 53.2 ms | 18.8 | 0.21 GiB | 0.8596 / 0.8678 |
+| `--trt-backbone` | **43.4 ms** | **23.1** | 0.23 GiB | 0.8591 / 0.8673 |
+
+The fastest path costs **0.0007** `F_beta^w` at beta^2=1 — below the ±0.0009
+seed-to-seed spread of the training run itself. Dropping the input to 512x512 is
+*not* worth it by comparison: it costs 0.007, ten times more, to save less time.
+
+### `backbone_fp16.plan` — prebuilt TensorRT engine
+
+This repository also ships an fp16 TensorRT engine for the **backbone only**. The
+dynamic-kernel affordance head cannot be exported to ONNX, but the backbone is
+plain convolutions, exports cleanly, and is the largest single term of the forward
+pass: 17.7 ms → 5.3 ms. The encoder, decoder and affordance head stay in PyTorch,
+CUDA-graphed against the engine's output buffers.
+
+| | |
+|---|---|
+| Built on | Jetson AGX Orin (64 GB), JetPack 6.2 (L4T R36.4.7) |
+| TensorRT / CUDA | 10.3.0 / 12.6 |
+| Input | `images` — `(1, 3, 640, 640)`, fp16, fixed |
+| Outputs | `feat0..feat3` — fp16 |
+| Size | 46.4 MiB |
+
+> **Engine files are not portable.** A `.plan` is compiled for one specific GPU,
+> TensorRT version and input shape. On any other device, JetPack/TensorRT version
+> or input size it will fail to deserialize — or load and behave incorrectly. It
+> is published here as a convenience for setups identical to the table above.
+> **If yours differs in any respect, build your own** — one command, a few
+> minutes. That is the supported path.
+
+```bash
+# use the prebuilt engine (identical setups only)
+hf download anhrisn/affkernel-iit-aff backbone_fp16.plan --local-dir weights/
+
+# or build your own (recommended)
+python tools/build_trt_backbone.py \
+  -c configs/rtdetr/rtdetr_r50vd_6x_iit_v3_stride2_deepsup.yml \
+  -r weights/affkernel_iit_r50vd_stride2_deepsup_seed42.pth \
+  --out weights/backbone_fp16.plan
+
+python tools/infer.py \
+  -c configs/rtdetr/rtdetr_r50vd_6x_iit_v3_stride2_deepsup.yml \
+  -r weights/affkernel_iit_r50vd_stride2_deepsup_seed42.pth \
+  --input path/to/image.jpg --output outputs/ \
+  --trt-backbone weights/backbone_fp16.plan --gpu-preprocess
+```
+
+Keep the checkpoint either way — the engine replaces the backbone at inference
+only, and training is unaffected.
+
+## Files
+
+| File | What it is |
+|---|---|
+| `affkernel_iit_r50vd_stride2_deepsup_seed42.pth` | The model. fp32 EMA weights, 174 MB. |
+| `backbone_fp16.plan` | Optional deployment artifact. TensorRT fp16 backbone engine for AGX Orin / JetPack 6.2 / TensorRT 10.3 @ 640x640. Not portable; not usable on its own. |
+
 ## Classes
 
 **Objects (10):** bowl, tvm, pan, hammer, knife, cup, drill, racket, spatula, bottle
@@ -127,8 +202,9 @@ Intended for research on affordance perception and perception-guided grasping.
 - The residual error is dominated by **missed detections**, not by mask quality.
   On instances the detector does fire on, mask quality (0.893) already exceeds
   the deterministic Swin-T baseline's overall score.
-- Latency was measured at fp32 on an RTX 6000 Ada. fp16 and TensorRT are
-  untested; they would be expected to help but are not characterised here.
+- The headline latency is fp32 on an RTX 6000 Ada. fp16, CUDA graphs and a
+  TensorRT backbone are now characterised on both a workstation and a Jetson AGX
+  Orin — see [Deployment](#deployment) — and cost at most 0.0007 `F_beta^w`.
 - Not validated for safety-critical deployment. A predicted grasp region is a
   perception cue, not a guarantee of a safe grasp.