README.md

---
library_name: renderformer-studio
license: mit
pipeline_tag: other
tags:
- neural-rendering
- rendering
- transformers
- materials
---

# RenderFormer V2

This repository bundles the two RenderFormer V2 renderer checkpoints and the
material-latent models used by the data pipeline. Source code, installation
instructions, scene conversion, and inference examples live in
[RenderFormer Studio](https://github.com/iamNCJ/RenderFormer-Studio).

The model was presented in the paper [RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives](https://huggingface.co/papers/2609.05738). See the [project page](https://renderformer.github.io/v2/) for additional results and details.

If this repository is access-protected during release staging, authenticate
with Hugging Face before loading it. One repository revision selects every
component below as an atomic, compatible set.

## Components

| Path | Class | Resolution | Purpose |
| --- | --- | ---: | --- |
| `transformer_512` | `RenderFormerModel` | 512 | Native-512 V2 renderer |
| `transformer_2048` | `RenderFormerModel` | 2048 | Native-2048 V2 renderer |
| `material_autoencoder` | `MaterialAutoencoder` | 256 | Material image to/from 9-D latent |
| `diffspec_mapper` | `DiffuseSpecularToLatent` | — | Diffuse/specular BRDF to 9-D latent |
| `metallic_mapper` | `PrincipledBRDFToLatent` | — | Metallic/roughness BRDF to 9-D latent |
| `metallic_transmission_mapper` | `PrincipledBRDFToLatentWithTransmission` | — | Metallic/transmission BRDF to 9-D latent |

The renderer pipeline loads the Qwen-Image VAE configured by each transformer
for environment and raw-material encoding. Those auxiliary weights are an
external dependency and are not duplicated in this bundle.

## Load a renderer

`resolution` automatically selects `transformer_512` or `transformer_2048`:

```python
from renderformer import RenderFormerV2Pipeline

pipeline = RenderFormerV2Pipeline.from_pretrained(
    "RenderFormer/renderformer-v2",
    resolution=512,
    device="cuda",
)
result = pipeline("scene.h5", resolution=512, precision="fp16")
```

Advanced callers may select a component explicitly with
`subfolder="transformer_512"`. The CLI and pipeline documentation in
RenderFormer Studio cover H5 preparation, precision, view batching, and tone
mapping.

## Load material components

```python
from renderformer.models.material import (
    DiffuseSpecularToLatent,
    MaterialAutoencoder,
    PrincipledBRDFToLatent,
    PrincipledBRDFToLatentWithTransmission,
)

repo = "RenderFormer/renderformer-v2"
material_autoencoder = MaterialAutoencoder.from_pretrained(
    repo, subfolder="material_autoencoder", strict=True
).eval()
diffspec_mapper = DiffuseSpecularToLatent.from_pretrained(
    repo, subfolder="diffspec_mapper", strict=True
).eval()
metallic_mapper = PrincipledBRDFToLatent.from_pretrained(
    repo, subfolder="metallic_mapper", strict=True
).eval()
transmission_mapper = PrincipledBRDFToLatentWithTransmission.from_pretrained(
    repo, subfolder="metallic_transmission_mapper", strict=True
).eval()
```

The autoencoder's `preprocessor_config.json` records the physical-input
contract. Prefer `encode_hdr` and `decode_hdr` for finite, nonnegative,
alpha-premultiplied linear RGB.

## Citation

Please cite both RenderFormer papers when using this bundle. Full BibTeX
entries are provided in the RenderFormer Studio README.