# 📡 WiSER: A Wireless Scene Encoder for Geometry-Grounded Multi-View Wireless Prediction

Jing Qiao, Yiyang Guo, Hao Ye
University of California Santa Cruz

Project Page | Code | Dataset | Checkpoint | Paper coming soon

WiSER architecture

WiSER turns a sparse 3D indoor scene into a transmitter-conditioned wireless scene memory. From the same scene representation, it predicts dense radiomaps over receiver planes and sparse multipath channel impulse response (CIR) taps for transmitter--receiver pairs.

One scene encoder. Two wireless prediction tasks. Geometry-aware outputs.

WiSER dataset generation pipeline

_{Radiomap examples compare ground truth, WiSER, NeRF2, RF-3DGS, and ablations under a shared dB color scale.}

_{CIR examples show matched predicted taps against ground-truth delay--power taps.}

Ray-corridor feature gathering

The ray-corridor branch gathers receiver-specific scene evidence around the transmitter, receiver, and the connecting corridor. This gives the radiomap decoder and CIR decoder access to likely blockers, reflectors, and local geometry without attending to the whole scene for every receiver query. ## 🔥 News - **2026-05-27:** Initial WiSER code repository prepared and pushed to private GitHub for release staging. - **2026-05-27:** Public small-100 processed dataset package uploaded to Hugging Face. - **2026-05-27:** WiSER full checkpoint package staged for Hugging Face model upload. - **Coming soon:** arXiv / paper PDF / BibTeX. > Release note: this repository is currently private during paper review. It is > prepared as the public code release and can be made public after the paper > release decision. ## 🗂️ Repository Layout ```text wiser/ model, datasets, losses, metrics, and sparse modules scripts/ training, evaluation, checkpoint, and example tools configs/ public training and smoke-test configs dataset_gen/ dataset-generation entrypoints and notes example/ compact WiSER-format example manifests project_env/WiSER/ model inference / training environment project_env/dataset_gen/ dataset-generation environment tests/ lightweight import / release checks ``` ## 🛠️ Installation Model inference / training environment: ```bash conda env create -f project_env/WiSER/environment.yml conda activate wiser pip install -e . ``` Dataset-generation environment: ```bash conda env create -f project_env/dataset_gen/environment.yml conda activate wiser-dataset-gen pip install -e . ``` The sparse backend is adapted from TRELLIS-2 sparse modules and may require CUDA sparse dependencies such as `flash-attn`, `spconv`, `torchsparse`, or `flex-gemm`, depending on your system. The included `dense_fallback` backend is only intended for CPU smoke tests and small debugging runs. ## 📦 Data And Checkpoints Processed small-100 dataset package: ```text https://huggingface.co/datasets/Jingqiao-ucsc/sionna-scannetpp-small-100 ``` WiSER checkpoint package: ```text https://huggingface.co/Jingqiao-ucsc/WiSER ``` The released dataset package contains processed Sionna/Mitsuba material scenes, 10 cm voxel caches, radiomap labels, CIR labels, and metadata. It does not contain original ScanNet++ raw meshes, RGB frames, depth frames, or original semantic labels. Users who want to regenerate data from raw indoor scenes must obtain ScanNet++ from the official source under its license terms, or use a different indoor scene dataset with the provided generation scripts. ## 🚀 Quick Start Run the bundled example without a checkpoint to validate the local data path: ```bash python scripts/infer_example.py \ --example-root example \ --out-json outputs/example_summary.json ``` Run example inference with the WiSER checkpoint: ```bash python scripts/infer_example.py \ --example-root example \ --checkpoint /path/to/wiser_sparse_scene_encoder_small100_full.pt \ --out-json outputs/example_summary.json ``` The example folder contains processed WiSER-format assets only. It is intended for smoke testing repository installation and model loading. ## 📊 Evaluation Full dual-task evaluation: ```bash python scripts/evaluate_dual.py \ --ckpt /path/to/wiser_sparse_scene_encoder_small100_full.pt \ --d22-ckpt /path/to/wiser_sparse_scene_encoder_small100_full.pt \ --radiomap-manifest /path/to/radiomap_manifest.json \ --cir-manifest /path/to/cir_manifest.json \ --wireless-root /path/to/wireless/scannetpp \ --scene3d-root /path/to/processed/3D/scannetpp \ --out-json outputs/eval_summary.json ``` The model package includes `eval_summary.json`, which records the validation summary associated with the released checkpoint. ## 🏋️ Training For the paper-scale alternating schedule, first dry-run the launcher: ```bash WISER_PYTHON=/path/to/python \ WISER_NUM_GPUS=8 \ bash scripts/run_paper_alternating_training.sh ``` After filling in the data manifests and warm-start checkpoint paths: ```bash WISER_DRY_RUN=0 \ WISER_RADIOMAP_MANIFEST=/path/to/radiomap_train.json \ WISER_CIR_TRAIN_MANIFEST=/path/to/cir_train.json \ WISER_CIR_VAL_MANIFEST=/path/to/cir_val.json \ WISER_INIT_RADIOMAP_CKPT=/path/to/radiomap_warm_start.pt \ WISER_INIT_CIR_CKPT=/path/to/cir_warm_start.pt \ bash scripts/run_paper_alternating_training.sh ``` For a minimal configuration-read smoke test: ```bash PYTHON=python3 bash scripts/run_full_training.sh configs/train_1gpu_debug.yaml ``` The one-GPU config validates the workflow but is not expected to reproduce the paper-scale metrics. ## 🧱 Dataset Generation Dataset generation converts indoor scenes into the WiSER training format: 1. Prepare an indoor scene mesh and semantic/material information. 2. Map scene labels to Sionna/Mitsuba radio-material groups. 3. Generate radiomap labels over receiver z-plane grids. 4. Generate CIR tap labels for selected TX/RX pairs. 5. Convert the scene into 10 cm sparse voxel features. See [dataset_gen/README.md](dataset_gen/README.md) and [dataset_gen/PIPELINE_NOTES.md](dataset_gen/PIPELINE_NOTES.md) for the current entrypoints and release notes. ## ✅ Release Checks Run the public-release audit: ```bash python scripts/audit_release.py --root . ``` Run the import smoke test: ```bash python -m pytest tests ``` ## 📌 Roadmap - Add arXiv / IEEE paper link after the public paper archive is available. - Add BibTeX after the final public paper metadata is stable. - Add fuller dataset-generation examples once the release scripts are frozen. - Add pretrained-checkpoint loading examples for common workstation setups. ## 🙏 Acknowledgements WiSER uses Sionna / Sionna RT and Mitsuba for wireless ray tracing and scene simulation. The sparse 3D transformer backend includes an isolated subset of TRELLIS-2 sparse modules; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) for details. The README organization follows the common academic project-release style used by projects such as 4DGaussians and Nerfies. ## 📄 Citation BibTeX is coming soon. For now, please cite the project page if you need to refer to this release before the paper archive is public: ```text WiSER: A Wireless Scene Encoder for Geometry-Grounded Multi-View Wireless Prediction. WiSER project page, 2026. ``` ## License This code release is under the MIT license. See [LICENSE](LICENSE).