---
license: apache-2.0
library_name: pytorch
tags:
  - novel-view-synthesis
  - gaussian-splatting
  - image-to-3d
  - diffusion
pipeline_tag: image-to-3d
---

# NovelGS

Official model release for [NovelGS: Consistent Novel-view Denoising via Large Gaussian Reconstruction Model](https://arxiv.org/abs/2411.16779).

This repository contains the official pretrained NovelGS weights.

## Files

- `model.safetensors`: 353 NovelGS tensors without optimizer, callbacks, trainer state, or frozen LPIPS/VGG state.
- `weights-manifest.json`: sizes, SHA-256 hashes, tensor names, dtypes, and shapes.
- `SHA256SUMS`: checksums for release artifacts.

The 353 NovelGS tensors in safetensors were compared tensor-by-tensor with the specified original research checkpoint: key sets, shapes, dtypes, and values are exactly equal.

## Sanitization and provenance

The published safetensors file is derived from `step=00010000.ckpt` with source SHA-256 `53eeda416228259db8226e8d9e6809717a45f9f104a1f6fcbf98e16759649f2f`. The release conversion removes 36 frozen `lpips.*` tensors (58,870,528 bytes) and clears Lightning callback state containing private machine paths. `weights-manifest.json` records the source hash, exclusions, tensor metadata, and artifact hashes.

## Architecture and input

NovelGS uses a 24-layer transformer of width 768 to predict pixel-aligned 3D Gaussians. The released 512-pixel checkpoint consumes four posed condition images and denoises one target view. The camera input consists of normalized camera-to-world matrices and normalized `[fx, fy, cx, cy]` intrinsics.

Use the code release at `https://github.com/moonsliu/NovelGS` for inference and training instructions.

## Loading

```python
from safetensors.torch import load_file
from omegaconf import OmegaConf
from src.utils.train_util import instantiate_from_config

config = OmegaConf.load("configs/train_512.yaml")
model = instantiate_from_config(config.model)
incompatible = model.load_state_dict(load_file("model.safetensors"), strict=False)
assert not incompatible.unexpected_keys
assert all(key.startswith("lpips.") for key in incompatible.missing_keys)
model.eval().cuda()
```
The LPIPS/VGG loss network is not bundled; the training code obtains it from the pinned upstream dependency.

## Training data

The model was trained on a filtered set of approximately 270K objects derived from Objaverse 1.0, rendered into 32 random 512×512 RGBA views per object. The original assets and rendered dataset are not distributed. Objaverse objects retain their individual licenses.

## Intended use and limitations

Intended for research on sparse-view reconstruction, Gaussian Splatting, and novel-view synthesis. Results may be geometrically incomplete or inconsistent for transparent, reflective, thin, heavily occluded, or out-of-distribution objects. The model does not establish ownership or licensing of its input or output content.

The optional single-image workflow uses Zero123++ weights under CC-BY-NC 4.0 and is therefore non-commercial. The core CUDA rasterizer distributed with the code has its own research-only license. These restrictions are separate from the Apache-2.0 license of NovelGS-authored code and NovelGS weights.

## Citation

```bibtex
@article{liu2024novelgs,
  title={NovelGS: Consistent Novel-view Denoising via Large Gaussian Reconstruction Model},
  author={Liu, Jinpeng and Xu, Jiale and Cheng, Weihao and Gao, Yiming and Wang, Xintao and Shan, Ying and Tang, Yansong},
  journal={arXiv preprint arXiv:2411.16779},
  year={2024}
}
```