File size: 3,504 Bytes
176388f 6b4eaca | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | ---
license: other
tags:
- heal
- horizon
- bev
---
# SparseBEV + HENet-tiny
SparseBEV performs 3D detection in BEV space via sparse queries: HENet-tiny extracts multi-view camera features, `MMFPN` fuses multi-scale features, `DenseDepthNet` uses lidar point clouds for dense depth supervision during training; `SparseBEVHead` refines sparse queries layer by layer via `MemoryBank` + `SparseBEVEncoder` (with `AsymmetricFFN`, `DeformableFeatureAggregation`, `SparseBEVRefinementModule`) and outputs detection boxes.
---
## Deployment Metrics
### Model Parameters
| Model | Model Input | Backbone | Neck | Model Output |
|---|---|---|---|---|
| SparseBEV | 6-camera multi-view images `(B,6,3,256,704)` + lidar point cloud `(B,N,5)` | HENet-tiny | MMFPN + DenseDepthNet | 3D bounding boxes `(B,N,cls+reg)` |
### Accuracy Metrics
| March | Metric | float | calibration | qat | hbm |
| --- | --- | --- | --- | --- | --- |
| J6M | NDS | 0.5414 | 0.5319 | — | 0.5307 |
| | mAP | 0.4338 | 0.4265 | — | 0.4235 |
> Results measured with `march = March.NASH_M` (J6M) configuration; this task has no QAT stage (qat column is `—`).
>
> HEAL version: heal 0.0.2 / hbdk4-compiler 4.11.11 / horizon_plugin_pytorch 3.3.10.
### Performance Metrics
> **Performance benchmark**: FPS is measured with single-core 8 threads; latency is single-core single-thread; memory is peak DDR usage.
| March | latency (ms) | fps | Memory Usage |
|---|---|---|---|
| J6M | 10.44 | 98.74 | 74.70 |
| J6P | 7.39 | 524.54 | 87.80 |
| J6B | 28.57 | 36.16 | 72.00 |
---
## Model Overview
### Core Design
SparseBEV performs 3D detection in BEV space via sparse queries: HENet-tiny extracts multi-view camera features, `MMFPN` fuses multi-scale features, `DenseDepthNet` uses lidar point clouds for dense depth supervision during training; `SparseBEVHead` refines sparse queries layer by layer via `MemoryBank` + `SparseBEVEncoder` (with `AsymmetricFFN`, `DeformableFeatureAggregation`, `SparseBEVRefinementModule`) and outputs detection boxes.
- **Task type**: BEV 3D object detection (BEV 3D Object Detection).
- **backbone**: HENet-tiny (`type=HENet`, `in_channels=3`, 4 stages `[4,3,8,6]` blocks, `embed_dims=[64,128,192,384]`, `GroupDWCB`/`AltDWCB`/`DWCB` blocks, `S2DDown` downsampling; multi-view camera feature extraction).
- **neck**: `MMFPN` (`in_strides=[2,4,8,16,32]`→`out_strides=[4,8,16,32]`, multi-scale fusion) + `DenseDepthNet` (dense depth supervision from lidar point clouds during training, `num_depth_layers=3`).
- **Detection head**: `SparseBEVHead` (`MemoryBank` + `SparseBEVEncoder` + `SparseBEVEncoder`, with `AsymmetricFFN`, `DeformableFeatureAggregation`, `SparseBEVRefinementModule`, layer-wise sparse query refinement, `num_classes=10`, `num_decoder=6`, `num_anchors=384`).
- **Loss function**: `FocalLoss` (cls) + `L1Loss` (reg) + `CrossEntropyLoss` (cns) + `GaussianFocalLoss` (yns).
- **Model input**: 6-camera multi-view images `(B,6,3,256,704)` + lidar point cloud `(B,N,5)` (`load_dim=5`, `use_dim=[0,1,2,4]`, `num_lidar_sweeps=0`, dense depth GT auxiliary).
- **Model output**: 10-class 3D bounding boxes (`num_classes=10`), `(B,N,cls+reg)`.
### Official Repo and Paper
Official repo: https://github.com/MCG-NJU/SparseBEV
Paper: https://arxiv.org/abs/2308.09244
Note: The camera backbone HENet is a HEAL in-house implementation; the official repo uses a different backbone.
### Reference
For more J6 chip deployment details, see https://developer.horizon.auto/blog/14074
|