code_snapshots/WiSER_public_release_code/README.md

# 📡 WiSER: A Wireless Scene Encoder for Geometry-Grounded Multi-View Wireless Prediction

<p align="center">
  <a href="https://scp10086.github.io/">Jing Qiao</a>,
  Yiyang Guo,
  Hao Ye
  <br>
  University of California Santa Cruz
</p>

<p align="center">
  <a href="https://scp10086.github.io/wiser-page/">Project Page</a> |
  <a href="https://github.com/scp10086/WiSER">Code</a> |
  <a href="https://huggingface.co/datasets/Jingqiao-ucsc/sionna-scannetpp-small-100">Dataset</a> |
  <a href="https://huggingface.co/Jingqiao-ucsc/WiSER">Checkpoint</a> |
  Paper coming soon
</p>

<p align="center">
  <img src="docs/assets/architecture.png" width="96%" alt="WiSER architecture">
</p>

WiSER turns a sparse 3D indoor scene into a transmitter-conditioned wireless
scene memory. From the same scene representation, it predicts dense radiomaps
over receiver planes and sparse multipath channel impulse response (CIR) taps
for transmitter--receiver pairs.

<p align="center">
  <strong>One scene encoder. Two wireless prediction tasks. Geometry-aware outputs.</strong>
</p>

<p align="center">
  <img src="docs/assets/dataset_pipeline.png" width="96%" alt="WiSER dataset generation pipeline">
</p>

<table>
  <tr>
    <td width="50%" align="center">
      <img src="docs/assets/radiomap_qualitative.png" width="100%" alt="Radiomap qualitative comparison"><br>
      <sub>Radiomap examples compare ground truth, WiSER, NeRF2, RF-3DGS, and ablations under a shared dB color scale.</sub>
    </td>
    <td width="50%" align="center">
      <img src="docs/assets/cir_qualitative.png" width="100%" alt="CIR qualitative comparison"><br>
      <sub>CIR examples show matched predicted taps against ground-truth delay--power taps.</sub>
    </td>
  </tr>
</table>

<p align="center">
  <img src="docs/assets/ray_corridor.png" width="52%" alt="Ray-corridor feature gathering">
</p>

The ray-corridor branch gathers receiver-specific scene evidence around the
transmitter, receiver, and the connecting corridor. This gives the radiomap
decoder and CIR decoder access to likely blockers, reflectors, and local
geometry without attending to the whole scene for every receiver query.

## 🔥 News

- **2026-05-27:** Initial WiSER code repository prepared and pushed to private
  GitHub for release staging.
- **2026-05-27:** Public small-100 processed dataset package uploaded to
  Hugging Face.
- **2026-05-27:** WiSER full checkpoint package staged for Hugging Face model
  upload.
- **Coming soon:** arXiv / paper PDF / BibTeX.

> Release note: this repository is currently private during paper review. It is
> prepared as the public code release and can be made public after the paper
> release decision.

## 🗂️ Repository Layout

```text
wiser/                    model, datasets, losses, metrics, and sparse modules
scripts/                  training, evaluation, checkpoint, and example tools
configs/                  public training and smoke-test configs
dataset_gen/              dataset-generation entrypoints and notes
example/                  compact WiSER-format example manifests
project_env/WiSER/        model inference / training environment
project_env/dataset_gen/  dataset-generation environment
tests/                    lightweight import / release checks
```

## 🛠️ Installation

Model inference / training environment:

```bash
conda env create -f project_env/WiSER/environment.yml
conda activate wiser
pip install -e .
```

Dataset-generation environment:

```bash
conda env create -f project_env/dataset_gen/environment.yml
conda activate wiser-dataset-gen
pip install -e .
```

The sparse backend is adapted from TRELLIS-2 sparse modules and may require
CUDA sparse dependencies such as `flash-attn`, `spconv`, `torchsparse`, or
`flex-gemm`, depending on your system. The included `dense_fallback` backend is
only intended for CPU smoke tests and small debugging runs.

## 📦 Data And Checkpoints

Processed small-100 dataset package:

```text
https://huggingface.co/datasets/Jingqiao-ucsc/sionna-scannetpp-small-100
```

WiSER checkpoint package:

```text
https://huggingface.co/Jingqiao-ucsc/WiSER
```

The released dataset package contains processed Sionna/Mitsuba material scenes,
10 cm voxel caches, radiomap labels, CIR labels, and metadata. It does not
contain original ScanNet++ raw meshes, RGB frames, depth frames, or original
semantic labels. Users who want to regenerate data from raw indoor scenes must
obtain ScanNet++ from the official source under its license terms, or use a
different indoor scene dataset with the provided generation scripts.

## 🚀 Quick Start

Run the bundled example without a checkpoint to validate the local data path:

```bash
python scripts/infer_example.py \
  --example-root example \
  --out-json outputs/example_summary.json
```

Run example inference with the WiSER checkpoint:

```bash
python scripts/infer_example.py \
  --example-root example \
  --checkpoint /path/to/wiser_sparse_scene_encoder_small100_full.pt \
  --out-json outputs/example_summary.json
```

The example folder contains processed WiSER-format assets only. It is intended
for smoke testing repository installation and model loading.

## 📊 Evaluation

Full dual-task evaluation:

```bash
python scripts/evaluate_dual.py \
  --ckpt /path/to/wiser_sparse_scene_encoder_small100_full.pt \
  --d22-ckpt /path/to/wiser_sparse_scene_encoder_small100_full.pt \
  --radiomap-manifest /path/to/radiomap_manifest.json \
  --cir-manifest /path/to/cir_manifest.json \
  --wireless-root /path/to/wireless/scannetpp \
  --scene3d-root /path/to/processed/3D/scannetpp \
  --out-json outputs/eval_summary.json
```

The model package includes `eval_summary.json`, which records the validation
summary associated with the released checkpoint.

## 🏋️ Training

For the paper-scale alternating schedule, first dry-run the launcher:

```bash
WISER_PYTHON=/path/to/python \
WISER_NUM_GPUS=8 \
bash scripts/run_paper_alternating_training.sh
```

After filling in the data manifests and warm-start checkpoint paths:

```bash
WISER_DRY_RUN=0 \
WISER_RADIOMAP_MANIFEST=/path/to/radiomap_train.json \
WISER_CIR_TRAIN_MANIFEST=/path/to/cir_train.json \
WISER_CIR_VAL_MANIFEST=/path/to/cir_val.json \
WISER_INIT_RADIOMAP_CKPT=/path/to/radiomap_warm_start.pt \
WISER_INIT_CIR_CKPT=/path/to/cir_warm_start.pt \
bash scripts/run_paper_alternating_training.sh
```

For a minimal configuration-read smoke test:

```bash
PYTHON=python3 bash scripts/run_full_training.sh configs/train_1gpu_debug.yaml
```

The one-GPU config validates the workflow but is not expected to reproduce the
paper-scale metrics.

## 🧱 Dataset Generation

Dataset generation converts indoor scenes into the WiSER training format:

1. Prepare an indoor scene mesh and semantic/material information.
2. Map scene labels to Sionna/Mitsuba radio-material groups.
3. Generate radiomap labels over receiver z-plane grids.
4. Generate CIR tap labels for selected TX/RX pairs.
5. Convert the scene into 10 cm sparse voxel features.

See [dataset_gen/README.md](dataset_gen/README.md) and
[dataset_gen/PIPELINE_NOTES.md](dataset_gen/PIPELINE_NOTES.md) for the current
entrypoints and release notes.

## ✅ Release Checks

Run the public-release audit:

```bash
python scripts/audit_release.py --root .
```

Run the import smoke test:

```bash
python -m pytest tests
```

## 📌 Roadmap

- Add arXiv / IEEE paper link after the public paper archive is available.
- Add BibTeX after the final public paper metadata is stable.
- Add fuller dataset-generation examples once the release scripts are frozen.
- Add pretrained-checkpoint loading examples for common workstation setups.

## 🙏 Acknowledgements

WiSER uses Sionna / Sionna RT and Mitsuba for wireless ray tracing and scene
simulation. The sparse 3D transformer backend includes an isolated subset of
TRELLIS-2 sparse modules; see
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) for details.

The README organization follows the common academic project-release style used
by projects such as 4DGaussians and Nerfies.

## 📄 Citation

BibTeX is coming soon. For now, please cite the project page if you need to
refer to this release before the paper archive is public:

```text
WiSER: A Wireless Scene Encoder for Geometry-Grounded Multi-View Wireless Prediction.
WiSER project page, 2026.
```

## License

This code release is under the MIT license. See [LICENSE](LICENSE).