Create README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,110 @@
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
base_model:
|
| 4 |
-
- prs-eth/marigold-depth-v1-1
|
| 5 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- radar
|
| 7 |
+
- lidar
|
| 8 |
+
- diffusion
|
| 9 |
+
- robotics
|
| 10 |
+
- point-cloud
|
| 11 |
+
- image-to-image
|
| 12 |
base_model:
|
| 13 |
+
- prs-eth/marigold-depth-v1-1
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# RadarSFD
|
| 17 |
+
|
| 18 |
+
RadarSFD reconstructs dense LiDAR-like point cloud representations from a single radar frame using conditional latent diffusion with pretrained priors. This repository contains the released RadarSFD checkpoint for the paper *RadarSFD: Single-Frame Diffusion with Pretrained Priors for Radar Point Clouds*.
|
| 19 |
+
|
| 20 |
+
The released weights are intended to be used together with the official project code:
|
| 21 |
+
|
| 22 |
+
- GitHub: https://github.com/phi-lab-rice/RadarSFD
|
| 23 |
+
|
| 24 |
+
## Model description
|
| 25 |
+
|
| 26 |
+
RadarSFD conditions a latent diffusion denoiser on a single radar range-azimuth BEV image and predicts a dense LiDAR-like BEV reconstruction. The model transfers geometric priors from a pretrained Marigold U-Net and uses latent-space conditioning from the radar input during denoising.
|
| 27 |
+
|
| 28 |
+
In the current release, the uploaded checkpoint corresponds to the trained U-Net weights used by the project codebase.
|
| 29 |
+
|
| 30 |
+
## Intended use
|
| 31 |
+
|
| 32 |
+
This model is intended for:
|
| 33 |
+
|
| 34 |
+
- research on radar-based 3D perception
|
| 35 |
+
- single-frame radar-to-LiDAR-like reconstruction
|
| 36 |
+
- benchmarking on the RadarHD dataset
|
| 37 |
+
- qualitative and quantitative evaluation using the official repository
|
| 38 |
+
|
| 39 |
+
This model is not intended as a plug-and-play general-purpose Hugging Face inference API model. It is designed to be loaded by the RadarSFD codebase for evaluation and research experiments.
|
| 40 |
+
|
| 41 |
+
## Input and output
|
| 42 |
+
|
| 43 |
+
Input:
|
| 44 |
+
|
| 45 |
+
- a single radar BEV image from the RadarHD-style preprocessing pipeline
|
| 46 |
+
- grayscale radar input repeated to 3 channels by the project dataloader
|
| 47 |
+
|
| 48 |
+
Output:
|
| 49 |
+
|
| 50 |
+
- a reconstructed LiDAR-like BEV image
|
| 51 |
+
- downstream evaluation artifacts produced by the repository evaluation scripts
|
| 52 |
+
|
| 53 |
+
Within the official codebase, radar images are loaded as grayscale images and repeated to 3 channels. The training pipeline pairs radar and LiDAR BEV images from the RadarHD dataset.
|
| 54 |
+
|
| 55 |
+
## How to use
|
| 56 |
+
|
| 57 |
+
Clone the official repository and follow its environment setup and evaluation instructions:
|
| 58 |
+
|
| 59 |
+
```bash
|
| 60 |
+
git clone https://github.com/phi-lab-rice/RadarSFD.git
|
| 61 |
+
cd RadarSFD
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
Place the checkpoint in your desired location, update the dataset path in `Code/config.yaml`, and run evaluation with:
|
| 65 |
+
|
| 66 |
+
```bash
|
| 67 |
+
python3 Code/Eval/eval.py --config Code/config.yaml --checkpoint /path/to/RadarSFD.safetensors
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
Please refer to the repository for the expected dataset structure, environment dependencies, and evaluation workflow.
|
| 71 |
+
|
| 72 |
+
## Training data
|
| 73 |
+
|
| 74 |
+
RadarSFD is trained and evaluated on the RadarHD dataset:
|
| 75 |
+
|
| 76 |
+
- https://github.com/akarsh-prabhakara/RadarHD
|
| 77 |
+
|
| 78 |
+
The project uses paired radar and LiDAR BEV images for training, validation, and testing.
|
| 79 |
+
|
| 80 |
+
## Architecture and training
|
| 81 |
+
|
| 82 |
+
- Backbone initialization: `prs-eth/marigold-depth-v1-1`
|
| 83 |
+
- VAE mode in the released code: TAESD (`madebyollin/taesd`)
|
| 84 |
+
- Diffusion scheduler for training: DDPM
|
| 85 |
+
- Inference scheduler in evaluation: DDIM
|
| 86 |
+
- Training objective: latent diffusion with additional perceptual losses in image space
|
| 87 |
+
|
| 88 |
+
## Limitations
|
| 89 |
+
|
| 90 |
+
- This release is research-oriented and has only been validated within the official RadarSFD codebase.
|
| 91 |
+
- Performance depends on using the same preprocessing and dataset conventions as the training setup.
|
| 92 |
+
- The model is designed for single-frame radar reconstruction and does not use temporal accumulation or SAR.
|
| 93 |
+
- Outputs are LiDAR-like BEV reconstructions rather than fully post-processed 3D point clouds ready for deployment.
|
| 94 |
+
|
| 95 |
+
## License
|
| 96 |
+
|
| 97 |
+
This model is released under the `CC-BY-NC-4.0` license. Please review the dataset and upstream model licenses as well before use.
|
| 98 |
+
|
| 99 |
+
## Citation
|
| 100 |
+
|
| 101 |
+
If you use this model, please cite:
|
| 102 |
+
|
| 103 |
+
```bibtex
|
| 104 |
+
@inproceedings{zhao2026radarsfd,
|
| 105 |
+
title = {RadarSFD: Single-Frame Diffusion with Pretrained Priors for Radar Point Clouds},
|
| 106 |
+
author = {Zhao, Bin and Garg, Nakul},
|
| 107 |
+
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
|
| 108 |
+
year = {2026}
|
| 109 |
+
}
|
| 110 |
+
```
|