NIDAR

NIR-Guided Intrinsic Decomposition for Scalable Scene-Agnostic
LiDAR Intensity Reconstruction

IROS 2026 · Official model artifacts

Paper · Code · Project page · Demo guide

NIDAR adds synthesized intensity to an existing point cloud using RGB appearance, a pseudo-NIR prior, intrinsic decomposition, and calibration. It requires RGB images + XYZ geometry + camera/LiDAR calibration.

Ground-truth intensity	NIDAR intensity

Existing nuScenes research example from the project page. The first portable demo targets Waymo-style exports; this figure is not a new release-validation run.

Download And Use

File	Role	Local runtime destination
47.pth	RGB-to-pseudo-NIR STN	stn/47.pth
epoch_010.pth	U-Net + DeepLab IRNet	irnet/epoch_010.pth
wintensity_mapping_quantile_model.pkl	Waymo quantile calibration	quantile/wintensity_mapping_quantile_model.pkl
deep_intrinsic_public_model.yaml	Matching IRNet architecture	Select through the inference runner

Runtime destinations are relative to the chosen weights directory; files on the Hub are at the repository root. ARTIFACTS.json and SHA256SUMS provide sizes and checksums.

Release preview: the matching GitHub source is updated. A two-frame CPU acceptance run completed in 27.4 seconds on a ThinkPad, with full anonymous weight downloads, GT-independence checks, and an Open3D window test. Public sample access is still pending. See the acceptance scope and limits.

Follow the source demo guide with source revision 8803e880e08662b07409f4bc9a9f6fb487101c99 or a compatible later revision. Python 3.10 and PyTorch 2.6 or newer are required by the reviewed inference package; checkpoint release checks use PyTorch 2.8. This is a custom PyTorch pipeline. Training reproduction and simulator-specific adapters are follow-up releases.

Expected outputs include scalar-intensity NPY/PLY, a colorized PLY, a static preview, and GT/prediction comparisons when evaluation targets are present. The optional Open3D viewer opens an interactive point-cloud window locally.

Model Route

RGB -> STN pseudo-NIR -> IRNet R -> LiDAR-camera projection
    -> quantile remapping -> intensity point cloud

The default output uses R + remap, rather than the direct intensity head I_hat. The selected STN has local training evidence based on the cs-stereo architecture and is frozen for inference. Its metadata records selected epoch 47 of 48 configured epochs, batch size 16, learning rate 5e-5, and PittsStereo-RGBNIR sequence IDs. See PROVENANCE.md for the audit evidence and remaining limits.

Checkpoint downloads can pin revision 1390055c2c564d3276f29cc513b3ef3cdc355e40. Companion files were added later and must be fetched from a commit that contains them. Check sizes and SHA-256 before loading trusted checkpoints or calibration pickles. Matching hashes establish identity, not a security certification.

Scope And Limitations

NIDAR generates an intensity-like research signal, not absolute returned optical power, sensor-independent material reflectance, or new XYZ geometry. Exposure, spectral appearance, range, visibility, projection, and sensor shift can affect outputs. The bundled calibration is Waymo-domain specific. Intrinsic outputs can blur boundaries or vary between adjacent frames.

The paper contains research evaluations. Later controlled ablations use separately trained models and are not fresh benchmarks of these hosted files. No new manipulation-success claim is established by this artifact upload. The CPU check validates the inference workflow on two demo frames; it does not establish generalization, GPU performance, or robotic task success.

License And Attribution

The code, weights, calibration, and data have distinct terms. Read LICENSE_NOTICES.md and the retained upstream cs-stereo MIT notice. Final training-data and weight distribution reconciliation is pending; this card grants no blanket permissive or commercial weight license. No raw training dataset is included.

Citation And Contact

@misc{zhang2026nidar,
  author = {Junjie Zhang and Jie Yin and Kefei Qian and Jie Li and
            Mengpei Jia and Yajuan Dun and Wenbo Chu and Guofa Li},
  title = {{NIDAR}: {NIR}-Guided Intrinsic Decomposition for Scalable Scene-Agnostic {LiDAR} Intensity Reconstruction},
  year = {2026},
  eprint = {2609.36878},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2609.36878}
}

Questions: GitHub Issues. Project contact: Junjie Zhang.