OpenExplorer/bev_multitask_henet_tinym

BEV Multitask + HENet-tiny

bev_multitask_henet_tinym is a camera-only BEV multitask perception model: HENet-tiny backbone + single-frame BevFormer view transformer, jointly performing five tasks — dynamic object 3D detection (with tracking), static object 3D detection, occupancy prediction, discrete map (stop line) detection, and online vectorized map (MapTR). It covers the full pipeline from 3-stage float training → quantization calibration → HBIR export → HBM compilation → inference.

---

Deployment Metrics

Model Parameters

Model	Model Input	Backbone	Neck	Model Output
BEVFusionMultitask	6-camera multi-view images (B,6,3,512,960)	HENet-tiny	FPN + SingleBevFormerViewTransformer	dynamic/static 3D boxes (B,N,cls+reg); occ voxel semantics; discrete-map 2D boxes; map polylines

Accuracy Metrics

March	Metric	float	calibration	qat	hbm
J6M	—	—	—	—	—

Accuracy numbers not yet measured; fill in after evaluation. Results use march = March.NASH_M (J6M) configuration; this task has no QAT stage (qat column is —).

HEAL version: heal 0.6.0 / hbdk4-compiler 4.13.1 / horizon_plugin_pytorch 3.4.5 (board UCP 3.16.2).

Performance Metrics

Performance benchmark: FPS is measured with single-core 8 threads; latency is single-core single-thread; memory is peak DDR usage. Data source: tools/test_perf_all.py (default sampling interval -d 200ms, perf-time 1).

March	latency (ms)	fps	Memory Usage (MB)
J6M	—	—	—

---

Model Overview

Core Design

HENet-tiny extracts multi-view camera features; a single-frame SingleBevFormerViewTransformer (with LearnedPositionalEncoding, HorizonSpatialCrossAttention and HorizonMultiScaleDeformableAttention) lifts them to a 128×128 BEV grid covering [-51.2m, 51.2m]². The BEV feature drives five task heads: dynamic/static 3D detection (DenseBEVSparseHead, SparseBEV-style sparse queries with MemoryBank temporal tracking), occupancy (BEVOCCHead2D, 18 classes), discrete map (BEVFormerDet2DDecoder, stop line), and online vectorized map (MaptrDetDecoder, 3 classes).

• Task type: BEV multitask perception (dynamic det + static det + occupancy + discrete map + online map).
• backbone: HENet-tiny (type=HENet, block_nums=[4,3,8,6], embed_dims=[64,128,192,384], GroupDWCB/AltDWCB/DWCB blocks, S2DDown downsampling), initialized from the ImageNet-pretrained henet_tinym float checkpoint.
• neck: FPN (in_strides=[32] → out_strides=[32], _dim_=256) + BevFuseModule (input_c=384+256 → fuse_c=256; lidar branch disabled, lidar_input=False).
• Dynamic det head: DenseBEVSparseHead (8 classes, num_anchor=384, num_decoder=6, MemoryBank with num_temp_instances=128 for tracking).
• Static det head: DenseBEVSparseHead (2 classes: traffic_cone, barrier; num_anchor=200).
• Occupancy head: BevformerOccDetDecoder + BEVOCCHead2D (Dz=16, 18 classes).
• Discrete map head: BEVFormerDet2DDecoder (1 class stop_line, num_query=100).
• Online map head: MaptrDetDecoder + MapInstanceDetectorHead (3 classes: divider, ped_crossing, boundary; 20 points per polyline).
• Loss functions: det — FocalLoss (cls) + L1Loss (reg) + CrossEntropyLoss (cns) + GaussianFocalLoss (yns); occ — CrossEntropyLoss; maptr — MapTRCriterion; discrete map — BevFormer2DCriterion.
• Model input: 6-camera multi-view images (B,6,3,512,960) (YUV444 + Normalize(128,128); camera-only, no lidar at inference).
• Model output: dynamic/static 3D boxes + occ voxel semantics + stop-line boxes + map polylines.

Official Repo and Paper

SparseBEV-style det head repo: https://github.com/MCG-NJU/SparseBEV Paper: https://arxiv.org/abs/2308.09244 MapTR repo: https://github.com/hustvl/MapTR Paper: https://arxiv.org/abs/2208.14437

Note: The camera backbone HENet and the multitask assembly are HEAL in-house implementations; the official repos use different backbones and single-task setups.