LaRenderer

LaRenderer is a neural renderer. It takes a 3D scene and renders it from any camera:

  • Scene: meshes with PBR materials, glass, mirrors, point lights and an optional HDR environment map.
  • Scene encoding: 1024ยณ sparse voxels, encoded with the TRELLIS.2 SC-VAE.
  • Rendering: a flow-matching camera decoder predicts a FLUX.2 image latent, and the FLUX.2 VAE decodes it into an AgX-tonemapped sRGB image.

This repository holds the inference weights and the configs for four inference modes.

License: to be announced (applies to the LaRenderer weights in this repository). The third-party networks used at inference are not included and keep their own licences (see below).

Inference modes

Mode Weights Sampling Transformer calls / view
5step teacher 5 Euler steps, CFG 2, independent Gaussian noise per view (seed 0) 10
5step_pairsync teacher 5 Euler steps, CFG 2, antithetic pair-sync (+n / โˆ’n branches, guided x0 averaged every step), one shared noise for all views 20
1step teacher scene encoder + student decoder one conditional call from the fixed noise fixed106; deterministic 1
1step_pairsync teacher scene encoder + student decoder average of the calls from +fixed106 and โˆ’fixed106; deterministic 2

Teacher: the caustic-mix fine-tune, EMA weights at step 5,000. Student: a one-step regression distillation of the teacher (5,000 updates). It shares the teacher's scene encoder, so its file holds only the camera decoder. The exact settings are in configs/modes/*.yaml.

Files

File Bytes Content
weights/larenderer_teacher_ema_step5000.safetensors 4,285,478,920 full model (scene encoder + flow camera decoder), fp32, 1.071 B parameters
weights/larenderer_student_1step_step5000.safetensors 2,888,663,108 one-step student camera decoder (view_transformer.*), fp32, 0.722 B parameters; load over the teacher
weights/fixed106_noise.safetensors 524,552 noise: the student's fixed input, [1, 32, 64, 64] fp32, std 2
configs/model_config.json model architecture config
configs/inference.yaml shared inference settings, weight file names, pinned third-party dependencies
configs/modes/*.yaml per-mode sampler settings
SHA256SUMS checksums of every file
No optimizer states, training data or evaluation references are included.

Third-party networks (download from their owners; not re-hosted here)

Network Role Repository Licence
FLUX.2 VAE latent โ†’ image decoder black-forest-labs/FLUX.2-klein-4B, vae/diffusion_pytorch_model.safetensors (sha256 ca70d2202afe6415bdbcb8793ba8cd99fd159cfe6192381504d6c4d3036e0f04) Apache-2.0
TRELLIS.2 SC-VAE scene voxel encoding (shape + texture encoders) microsoft/TRELLIS.2-4B, ckpts/{shape,tex}_{enc,dec}_next_dc_f16c32_fp16 MIT
Qwen-Image VAE environment-map encoding (env-lit scenes only) Qwen/Qwen-Image, vae/ Apache-2.0

Usage

The inference guide, docs/inference.md in the code repository eternity304/neural-rendering (scripts in scripts/inference/), covers:

  • environment setup;
  • scene preparation: scene scale, voxel features, light, camera and material ranges;
  • the commands for every mode;
  • a worked example that reproduces the caustic-benchmark numbers.

In short:

huggingface-cli download eternity304/LaRenderer --local-dir ./LaRenderer
python scripts/inference/prepare_scene.py --bundle my_scene --out shards/my_scene
python scripts/inference/larender_infer.py --release ./LaRenderer \
    --flux2-vae ./deps/flux2_klein_4b/vae/diffusion_pytorch_model.safetensors \
    --shard shards/my_scene/shard-00000.tar --mode 1step --out renders/my_scene

Results (caustic benchmark v1, 24 scenes / 72 views, 512 px, PSNR / SSIM / LPIPS-VGG)

Mode PSNR โ†‘ SSIM โ†‘ LPIPS โ†“
5step 22.53 0.901 0.127
5step_pairsync 22.68 0.904 0.124
1step 23.26 0.907 0.121
1step_pairsync 23.48 0.910 0.121

These were reproduced from this repository's files on 2026-11-20. The 5step and 1step renders are byte-identical to the reference runs.

Intended use and limitations

The model is for research on neural rendering of synthetic 3D scenes. It was trained on procedurally generated rooms about 4 m across, with point lights or HDRI sunlight, glassware and mirrors.

Input requirements

  • Build scenes at that scale.
  • Keep point lights at least 0.2 m from geometry, at 20โ€“2,000 W.
  • Keep cameras within about two room widths of the scene.

Known failure modes

  • dark discs around lights that are too close to a surface;
  • dark rooms when the light is hidden from the room;
  • under-estimated bounce light in closed rooms;
  • thin glass rendered as solid glass;
  • no HDR output: the output is display-referred AgX sRGB.

See the guide's failure-mode section.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support