FaceLift ConvRot INT8 / INT4 Runtimes

Quantized ConvRot runtimes and transformed checkpoints for FaceLift / OpenFaceLift, tested with native Comfy-Kitchen ConvRot kernels. Four tested configurations are included, ranging from full INT8 to mixed INT4 / INT8 setups.

FP16 reference

Runtime     28.204 s
Peak VRAM    7.828 GiB
Final SSIM   1.0000

Quantized configurations

Method	Checkpoint size	Peak VRAM	Runtime	Multiview SSIM vs FP16	Final SSIM vs FP16
Full INT8 W8A8	2.807 GiB	5.845 GiB	28.044 s	0.9973	0.9925
INT4 diffusion + INT8 GSLRM	2.235 GiB	5.268 GiB	25.780 s	0.9021	0.9036
INT4 diffusion + W4A8 GSLRM body + INT8 heads	2.094 GiB	5.130 GiB	26.089 s	0.9021	0.9024
INT8 diffusion + W4A8 GSLRM body + INT8 heads	2.666 GiB	5.700 GiB	28.139 s	0.9973	0.9847

Official FaceLift examples — turntable videos

All eight official FaceLift examples (000–007) rendered with all four quantization methods.

The four videos in each row use the same input face and the same inference settings.

Example 000

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 001

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 002

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 003

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 004

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 005

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 006

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 007

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Official FaceLift examples — multiview outputs

The corresponding six-view MVDiffusion outputs for the same eight official FaceLift examples.

Click any image to open the full-resolution PNG.

Example 000

INT4 Diff + W4A8 GSLRM example 000 multiview

INT8 Diff + W4A8 GSLRM example 000 multiview

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 001

INT4 Diff + W4A8 GSLRM example 001 multiview

INT8 Diff + W4A8 GSLRM example 001 multiview

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 002

INT4 Diff + W4A8 GSLRM example 002 multiview

INT8 Diff + W4A8 GSLRM example 002 multiview

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 003

INT4 Diff + W4A8 GSLRM example 003 multiview

INT8 Diff + W4A8 GSLRM example 003 multiview

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 004

INT4 Diff + W4A8 GSLRM example 004 multiview

INT8 Diff + W4A8 GSLRM example 004 multiview

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 005

INT4 Diff + W4A8 GSLRM example 005 multiview

INT8 Diff + W4A8 GSLRM example 005 multiview

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 006

INT4 Diff + W4A8 GSLRM example 006 multiview

INT8 Diff + W4A8 GSLRM example 006 multiview

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Example 007

INT4 Diff + W4A8 GSLRM example 007 multiview

INT8 Diff + W4A8 GSLRM example 007 multiview

_{Full INT8 W8A8}

_{INT4 Diff + INT8 GSLRM}

_{INT4 Diff + W4A8 GSLRM}

_{INT8 Diff + W4A8 GSLRM}

Upstream projects

FaceLift code: https://github.com/weijielyu/FaceLift
Official OpenFaceLift weights: https://huggingface.co/wlyu/OpenFaceLift
FaceLift paper: https://arxiv.org/abs/2412.17812
Comfy-Kitchen: https://github.com/Comfy-Org/comfy-kitchen

Benchmark environment

All benchmark results below used the same controlled FaceLift runtime.

GPU             NVIDIA L4
GPU capability  SM 8.9
Python          3.12.13
PyTorch         2.11.0+cu128
Torchvision     0.26.0+cu128
CUDA            12.8
comfy-kitchen   0.2.31
xformers        0.0.35

Benchmark image 004.png
Seed            4
Guidance        3.0
Diffusion steps 50

FaceLift multiview attention used:

XFormersMVAttnProcessor

GSLRM used PyTorch SDPA across all 24 transformer blocks.

The SSIM/PSNR figures below are from the controlled 004.png benchmark. The eight-example gallery above is a broader qualitative comparison across all official FaceLift examples.

1. Full INT8 W8A8

Folder:

01_full_int8_w8a8/

Configuration:

CLIP Vision     INT8 W8A8
CLIP Text       INT8 W8A8
UNet            INT8 W8A8
VAE             INT8 W8A8
GSLRM           INT8 W8A8

Quantized Linear layers:

CLIP Vision   193
CLIP Text     138
UNet          218
VAE             8
GSLRM           99
------------------
Total          656

Controlled benchmark:

Runtime          28.044 s
Peak VRAM         5.845 GiB

Multiview:
SSIM              0.997264
PSNR             46.621 dB
MAE               0.4247

Final render:
SSIM              0.992526

This is the highest-quality fully quantized configuration tested.

Official eight-example suite:

Total runtime       256.517 s
Mean / face          32.065 s
Examples completed    8 / 8

2. INT4 diffusion + INT8 GSLRM

Folder:

02_int4_diffusion_int8_gslrm/

Configuration:

CLIP Vision     W4A4
CLIP Text       W4A4
UNet            W4A4
VAE             W4A4

GSLRM           INT8 W8A8

Controlled benchmark:

Runtime          25.780 s
Peak VRAM         5.268 GiB

Multiview SSIM    0.902111
Final SSIM        0.903606

Official eight-example suite:

Total runtime       236.716 s
Mean / face          29.589 s
Examples completed    8 / 8

This test showed that the diffusion side can run at W4A4, but the multiview quality loss becomes the dominant bottleneck.

3. INT4 diffusion + W4A8 GSLRM body + INT8 heads

Folder:

03_int4_diffusion_w4a8_gslrm_int8_heads/

Configuration:

CLIP Vision                     W4A4
CLIP Text                       W4A4
UNet                            W4A4
VAE                             W4A4

GSLRM body                      W4A8
gaussian_upsampler.linear       INT8 W8A8
pixel_gaussian_decoder.1        INT8 W8A8

GSLRM split:

97 Linear layers   W4A8
 2 Linear layers   INT8 W8A8

Controlled benchmark:

Runtime          26.089 s
Peak VRAM         5.130 GiB

Multiview SSIM    0.902111
Final SSIM        0.902423

Official eight-example suite:

Total runtime       238.067 s
Mean / face          29.758 s
Examples completed    8 / 8

This configuration keeps the GSLRM body at packed 4-bit weights while avoiding A4 activation execution inside GSLRM.

4. INT8 diffusion + W4A8 GSLRM body + INT8 heads

Folder:

04_int8_diffusion_w4a8_gslrm_int8_heads/

Configuration:

CLIP Vision                     INT8 W8A8
CLIP Text                       INT8 W8A8
UNet                            INT8 W8A8
VAE                             INT8 W8A8

GSLRM body                      W4A8
gaussian_upsampler.linear       INT8 W8A8
pixel_gaussian_decoder.1        INT8 W8A8

GSLRM split:

97 Linear layers   W4A8
 2 Linear layers   INT8 W8A8

Controlled benchmark:

Runtime          28.139 s
Peak VRAM         5.700 GiB

Multiview:
SSIM              0.997264
PSNR             46.621 dB
MAE               0.4247

Final render:
SSIM              0.984692
PSNR             38.961 dB
MAE               1.2562

Compared directly with full INT8:

Multiview:
MSE               0.0
SSIM              1.0

Final:
SSIM              0.986710
PSNR             40.005 dB
MAE               1.1138

Official eight-example suite:

Total runtime       257.027 s
Mean / face          32.128 s
Examples completed    8 / 8

This is the best mixed INT8 / INT4 configuration tested so far.

Main findings

INT8 ConvRot is highly accurate

Full INT8 retained almost all FP16 quality:

FP16 vs INT8 final SSIM = 0.9925

Peak VRAM dropped from:

7.828 GiB -> 5.845 GiB

GSLRM is highly sensitive to A4 activations

Running the GSLRM transformer body with A4 activations caused a large reconstruction-quality loss.

Moving GSLRM from:

W4A4 -> W4A8

recovered most of the lost reconstruction quality while retaining packed 4-bit weights.

The Gaussian output heads benefit from INT8

The two protected projections are:

gaussian_upsampler.linear
pixel_gaussian_decoder.1

These directly produce the Gaussian reconstruction parameters.

Using INT8 for these layers improves the W4A8 GSLRM configuration.

Diffusion becomes the limiting stage at W4A4

With INT4 diffusion:

Multiview SSIM ~= 0.9021

Once information is lost during multiview generation, the downstream GSLRM cannot recover it.

Switching the diffusion stack back to INT8 restores:

Multiview SSIM = 0.9973

and raises final reconstruction quality to:

Final SSIM = 0.9847

Recommended configurations

Highest quality

Full INT8 W8A8

Peak VRAM  5.845 GiB
Final SSIM 0.9925

Best mixed quality / memory

INT8 diffusion
+
W4A8 GSLRM body
+
INT8 Gaussian heads

Peak VRAM  5.700 GiB
Final SSIM 0.9847

Lowest-memory mixed configuration tested

INT4 diffusion
+
W4A8 GSLRM body
+
INT8 Gaussian heads

Peak VRAM  5.130 GiB
Final SSIM 0.9024

Repository structure

facelift_convrot_int8_int4_runtimes/
│
├── README.md
├── LICENSE
├── build_report.json
│
├── 01_full_int8_w8a8/
│   ├── weights/
│   ├── runtime/
│   ├── outputs/004/
│   ├── examples/official_facelift/
│   │   ├── 000/
│   │   ├── 001/
│   │   ├── ...
│   │   └── 007/
│   ├── manifest.json
│   └── metrics.json
│
├── 02_int4_diffusion_int8_gslrm/
├── 03_int4_diffusion_w4a8_gslrm_int8_heads/
└── 04_int8_diffusion_w4a8_gslrm_int8_heads/

Each official example folder contains:

input.png
multiview.png
output.png
gaussians.ply
turntable.mp4

Runtime

Each method contains:

runtime/
├── facelift_convrot_runtime.py
├── run_inference.py
└── requirements.txt

Example:

python runtime/run_inference.py \
    --input_dir /path/to/images \
    --output_dir ./outputs_new

If OpenFaceLift is already downloaded locally:

export OPENFACELIFT_DIR=/path/to/OpenFaceLift

The runtime reconstructs the official model architecture, replaces eligible nn.Linear modules according to manifest.json, loads the transformed ConvRot checkpoints, enables FaceLift's xFormers multiview attention path, and runs the official inference pipeline.

ConvRot quantization details

Only eligible rank-2:

nn.Linear

weights are ConvRot quantized.

Convolutions, embeddings, normalization layers, biases, and unsupported tensors remain in their normal precision unless otherwise handled by the upstream model.

ConvRot group selection:

256 preferred
64  fallback
16  fallback

W4 quantization uses:

quant_group_size = 64

W4A4

packed W4 weights
A4 execution
linear_dtype="int4"

W4A8

packed W4 weights
A8 execution
linear_dtype="int8"

INT8 W8A8

INT8 ConvRot weights
INT8 activation execution

License

These transformed checkpoints are derived from the official OpenFaceLift model weights.

The official OpenFaceLift weights are distributed under the Adobe Research License.

Review the original license before redistribution or use:

https://huggingface.co/wlyu/OpenFaceLift/blob/main/LICENSE

FaceLift source code is distributed separately under the licensing terms of the official code repository.

This repository is an experimental quantization/runtime derivative and is not the official FaceLift release.

References

FaceLift — official code
https://github.com/weijielyu/FaceLift
OpenFaceLift — official weights
https://huggingface.co/wlyu/OpenFaceLift
FaceLift paper
https://arxiv.org/abs/2412.17812
Comfy-Kitchen
https://github.com/Comfy-Org/comfy-kitchen

Citation

If you use FaceLift, cite the original FaceLift work:

@InProceedings{FaceLift,
    author    = {Lyu, Weijie and Zhou, Yi and Yang, Ming-Hsuan and Shu, Zhixin},
    title     = {FaceLift: Learning Generalizable Single Image 3D Face Reconstruction from Synthetic Heads},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2025},
    pages     = {12691-12701}
}

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for ApacheOne/facelift_convrot_int8_int4_runtimes

FaceLift: Single Image to 3D Head with View Generation and GS-LRM

Paper • 2412.17812 • Published Dec 23, 2024 • 3