LaRenderer
LaRenderer is a neural renderer. It takes a 3D scene and renders it from any camera:
- Scene: meshes with PBR materials, glass, mirrors, point lights and an optional HDR environment map.
- Scene encoding: 1024ยณ sparse voxels, encoded with the TRELLIS.2 SC-VAE.
- Rendering: a flow-matching camera decoder predicts a FLUX.2 image latent, and the FLUX.2 VAE decodes it into an AgX-tonemapped sRGB image.
This repository holds the inference weights and the configs for four inference modes.
License: to be announced (applies to the LaRenderer weights in this repository). The third-party networks used at inference are not included and keep their own licences (see below).
Inference modes
| Mode | Weights | Sampling | Transformer calls / view |
|---|---|---|---|
5step |
teacher | 5 Euler steps, CFG 2, independent Gaussian noise per view (seed 0) | 10 |
5step_pairsync |
teacher | 5 Euler steps, CFG 2, antithetic pair-sync (+n / โn branches, guided x0 averaged every step), one shared noise for all views | 20 |
1step |
teacher scene encoder + student decoder | one conditional call from the fixed noise fixed106; deterministic |
1 |
1step_pairsync |
teacher scene encoder + student decoder | average of the calls from +fixed106 and โfixed106; deterministic | 2 |
Teacher: the caustic-mix fine-tune, EMA weights at step 5,000.
Student: a one-step regression distillation of the teacher (5,000 updates). It shares the teacher's scene encoder, so
its file holds only the camera decoder. The exact settings are in configs/modes/*.yaml.
Files
| File | Bytes | Content |
|---|---|---|
weights/larenderer_teacher_ema_step5000.safetensors |
4,285,478,920 | full model (scene encoder + flow camera decoder), fp32, 1.071 B parameters |
weights/larenderer_student_1step_step5000.safetensors |
2,888,663,108 | one-step student camera decoder (view_transformer.*), fp32, 0.722 B parameters; load over the teacher |
weights/fixed106_noise.safetensors |
524,552 | noise: the student's fixed input, [1, 32, 64, 64] fp32, std 2 |
configs/model_config.json |
model architecture config | |
configs/inference.yaml |
shared inference settings, weight file names, pinned third-party dependencies | |
configs/modes/*.yaml |
per-mode sampler settings | |
SHA256SUMS |
checksums of every file | |
| No optimizer states, training data or evaluation references are included. |
Third-party networks (download from their owners; not re-hosted here)
| Network | Role | Repository | Licence |
|---|---|---|---|
| FLUX.2 VAE | latent โ image decoder | black-forest-labs/FLUX.2-klein-4B, vae/diffusion_pytorch_model.safetensors (sha256 ca70d2202afe6415bdbcb8793ba8cd99fd159cfe6192381504d6c4d3036e0f04) |
Apache-2.0 |
| TRELLIS.2 SC-VAE | scene voxel encoding (shape + texture encoders) | microsoft/TRELLIS.2-4B, ckpts/{shape,tex}_{enc,dec}_next_dc_f16c32_fp16 |
MIT |
| Qwen-Image VAE | environment-map encoding (env-lit scenes only) | Qwen/Qwen-Image, vae/ |
Apache-2.0 |
Usage
The inference guide, docs/inference.md in the code repository eternity304/neural-rendering (scripts in scripts/inference/), covers:
- environment setup;
- scene preparation: scene scale, voxel features, light, camera and material ranges;
- the commands for every mode;
- a worked example that reproduces the caustic-benchmark numbers.
In short:
huggingface-cli download eternity304/LaRenderer --local-dir ./LaRenderer
python scripts/inference/prepare_scene.py --bundle my_scene --out shards/my_scene
python scripts/inference/larender_infer.py --release ./LaRenderer \
--flux2-vae ./deps/flux2_klein_4b/vae/diffusion_pytorch_model.safetensors \
--shard shards/my_scene/shard-00000.tar --mode 1step --out renders/my_scene
Results (caustic benchmark v1, 24 scenes / 72 views, 512 px, PSNR / SSIM / LPIPS-VGG)
| Mode | PSNR โ | SSIM โ | LPIPS โ |
|---|---|---|---|
5step |
22.53 | 0.901 | 0.127 |
5step_pairsync |
22.68 | 0.904 | 0.124 |
1step |
23.26 | 0.907 | 0.121 |
1step_pairsync |
23.48 | 0.910 | 0.121 |
These were reproduced from this repository's files on 2026-11-20. The 5step and 1step renders are byte-identical to
the reference runs.
Intended use and limitations
The model is for research on neural rendering of synthetic 3D scenes. It was trained on procedurally generated rooms about 4 m across, with point lights or HDRI sunlight, glassware and mirrors.
Input requirements
- Build scenes at that scale.
- Keep point lights at least 0.2 m from geometry, at 20โ2,000 W.
- Keep cameras within about two room widths of the scene.
Known failure modes
- dark discs around lights that are too close to a surface;
- dark rooms when the light is hidden from the room;
- under-estimated bounce light in closed rooms;
- thin glass rendered as solid glass;
- no HDR output: the output is display-referred AgX sRGB.
See the guide's failure-mode section.