---
license: other
tags:
  - heal
  - horizon
---

# E2E Diffusion Planner + HENet-tiny

`e2e_planner` is an end-to-end autonomous-driving model on NAVSIM: an 8-camera BEV multitask perception front-end (HENet-tiny + single-frame BevFormer view transformer; dynamic/static 3D detection, occupancy, discrete map, online MapTR) feeding a BEV-feature-conditioned Diffusion Planner that outputs future ego trajectories. It covers the full pipeline from 5-stage float training → quantization calibration → HBIR export → HBM compilation → NAVSIM closed-loop metric evaluation.

---

## Deployment Metrics

### Model Parameters

| Model | Model Input | Backbone | Neck | Model Output |
|---|---|---|---|---|
| E2EPlanner (BEVFusionMultitask + Diffusion_Planner) | 8-camera multi-view images `(B,8,3,512,960)` + ego state + route lanes | HENet-tiny | FPN + SingleBevFormerViewTransformer | future ego trajectory `(1,8,4)` via diffusion |

### Accuracy Metrics

| March | Metric | float | calibration | hbm |
| --- | --- | --- | --- | --- |
| J6M | PDMS | 0.8223 | 0.8142 | — |

> Measured 2026-10-17 on NAVSIM navtest with `HAT_USE_NEIGHBORS_FUTURE=0` and `dequantize_bev_embed=False`. No QAT stage. HBM accuracy not yet measured.
>
> HEAL version: heal 0.6.0 / hbdk4-compiler 4.13.3 / horizon_plugin_pytorch 3.4.5 (board UCP 3.16.2).

### Performance Metrics

> **Performance benchmark**: FPS is measured with single-core 8 threads; latency is single-core single-thread; memory is peak DDR usage.
> Data source: `tools/perf.py` (`--perf-time 1`; DDR sampled by `hrt_ucp_monitor -d 200`). Measured 2026-10-19 (`dequantize_bev_embed=False`). Toolchain: hbdk4-compiler 4.13.3 / horizon_plugin_pytorch 3.4.5. No `native::` CPU ops.

| March | latency (ms) | fps | Memory Usage (MB) |
|---|---|---|---|
| J6M | 99.14 (BPU 99.07) | 10.11 | 306.30 |
| J6P | 37.84 (BPU 37.78) | 88.23 | 242.20 |

---

## Model Overview

### Core Design

The model has two halves joined at the BEV feature: an 8-camera multitask perception front-end producing a `128×192` BEV grid, and a BEV-conditioned Diffusion Planner. HENet-tiny extracts multi-view camera features; a single-frame `SingleBevFormerViewTransformer` lifts them to BEV; five task heads (`DenseBEVSparseHead` dynamic/static, `BEVOCCHead2D` occupancy, `BEVFormerDet2DDecoder` discrete map, `MaptrDetDecoder` online map) share the BEV feature. The planner encodes the pooled BEV feature into 384 tokens, conditions a 3-layer DiT via a route-lane `RouteEncoder`, and denoises future ego waypoints over 5 reverse diffusion steps.

- **Task type**: End-to-end planning (perception + planning joint).
- **Perception backbone**: HENet-tiny (`block_nums=[4,3,8,6]`, `embed_dims=[64,128,192,384]`), initialized from the ImageNet-pretrained `henet_tinym` float checkpoint.
- **Perception neck**: `FPN` + `SingleBevFormerViewTransformer` (`128×192` BEV, `[-51.2m, 102.4m] × [-51.2m, 51.2m]`).
- **Dynamic det head**: `DenseBEVSparseHead` (3 classes; `num_anchor=384`, `MemoryBank` 128 temporal instances).
- **Static det head**: `DenseBEVSparseHead` (4 classes; `num_anchor=200`).
- **Occupancy head**: `BevformerOccDetDecoder` + `BEVOCCHead2D` (`Dz=16`, 9 classes).
- **Discrete map head**: `BEVFormerDet2DDecoder` (2 classes: stop_line/crosswalk).
- **Online map head**: `MaptrDetDecoder` + `MapInstanceDetectorHead` (1 class `lanes`, SD-map fusion, 20 points per polyline).
- **Planner encoder**: `Diffusion_Planner_Encoder` (`BEVFusionEncoder` ×8 downsample → 384 tokens, 3-layer `FusionEncoder` transformer).
- **Planner decoder**: 3-layer `DiT` (`hidden_dim=256`, 8 heads, `RouteEncoder`), `model_type="x_start"`, `diffusion_pred_steps=5`, linear VP noise schedule.
- **Loss functions**: perception — `FocalLoss` + `L1Loss` + `CrossEntropyLoss` + `GaussianFocalLoss` / `CrossEntropyLoss` / `MapTRCriterion` / `BevFormer2DCriterion` per head; planner — `DiffusionLoss` (ego 2.0 + collision 1.0).
- **Model input**: 8-camera multi-view images `(B,8,3,512,960)` (YUV444 + `Normalize(128,128)`; camera-only) + ego state + route lanes.
- **Model output**: 8-step future ego trajectory `(1,8,4)`.

### Official Repo and Paper

NAVSIM benchmark: https://github.com/autonomousvision/navsim
Paper: https://arxiv.org/abs/2406.15349
SparseBEV-style det head repo: https://github.com/MCG-NJU/SparseBEV
Paper: https://arxiv.org/abs/2308.09244
MapTR repo: https://github.com/hustvl/MapTR
Paper: https://arxiv.org/abs/2208.14437

Note: The camera backbone HENet and the E2E assembly are HEAL in-house implementations; the official repos use different backbones and pipelines.