attashe/Bernini-2-INT8-ConvRot

Bernini-2 — INT8 ConvRot model artifacts

This public repository contains model weights, optional LoRAs, configs and tokenizers only. Inference and conversion code is maintained separately in the Bernini codebase. This is a community conversion of ByteDance/Bernini-Diffusers-v2, not an official ByteDance quantized release.

Both Wan experts use INT8 W8A8 + regular-Hadamard ConvRot, accelerated by comfy_kitchen. The v2 fine-tuned planner, connector and VIT decoder are included in BF16. No separate model repository or original FP32 release is needed.

Component	Storage	Size
High-noise Wan expert	INT8 ConvRot / BF16	13.54 GiB
Low-noise Wan expert	INT8 ConvRot / BF16	13.54 GiB
Fine-tuned Qwen2.5-VL planner	BF16	15.45 GiB
Connector, VIT decoder, mask tokens	BF16	2.84 GiB
Frozen UMT5 encoder	BF16	10.58 GiB
Wan VAE	FP32	0.47 GiB
Optional LightX2V high/low rank-64 adapters	BF16	1.14 GiB

Each component is a single .safetensors file, not split into shards: high_noise/model.safetensors, low_noise/model.safetensors, mllm/model.safetensors, glue/model.safetensors, t5_text_encoder/model.safetensors, and vae/diffusion_pytorch_model.safetensors. The two adapters are separate files under loras/.

Total download: approximately 57.58 GiB, or 56.44 GiB without loras/. All 800 Wan block linear layers are quantized, with FP32 per-row scales and ConvRot group size 256. Norms, embeddings, biases and output heads remain BF16. This is the full updated v2 planner and renderer, not a v1 planner combined with v2 diffusion weights.

Required inference code

Use a Bernini code checkout containing bernini/compact.py and scripts/run_full_bernini_local.py with the --model_dir / --int8_backend options. These INT8 support changes are awaiting publication to the maintainer's GitHub; the code-release link will be added when available. The unmodified upstream checkout cannot load this layout directly.

The commands below run from that code checkout, not from this model folder. Use Python 3.11 and CUDA-enabled PyTorch. The tested setup is RTX 5090 (32 GB), PyTorch 2.11.0+cu128, comfy-kitchen 0.2.31, diffusers 0.35.2, and transformers 4.57.3. Approximately 64 GiB host RAM is recommended.

cd /path/to/your/Bernini
pip install -e '.[int8]'
hf download attashe/Bernini-2-INT8-ConvRot \
  --local-dir models/bernini-v2-int8-convrot

The model is not loaded with DiffusionPipeline.from_pretrained(); use the custom full Bernini loader. FlashAttention is optional; PyTorch SDPA is supported.

Recommended Bernini-v2 generation settings

Verified against upstream Bernini-2 commit e6c2cf1 (2026-08-13), especially scripts/bernini_v2 and that commit's Gradio task defaults. Use v2-specific settings, not v1 presets.

Task	Render steps	Planner steps	vit_denoising_step	omega_txt	omega_tgt	omega_img	omega_vid	omega_scale	max_image_size
Text → video	50	50	1	5.0	1.2	1.0	1.0	1.0	842
References → video	40	50	1	4.5	1.5	3.0	1.0	0.75	842
Video / motion editing	40	50	1	4.0	1.2	1.25	1.25	0.75	848
Reference-guided video editing	40	50	1	3.6	1.5	3.0	1.5	0.5	848
Product placement (upstream Gradio)	40	50	1	4.0	1.2	1.25	1.25	0.75	848
Text → image	50	25	5	4.0	0.5	1.0	1.0	1.0	842
Image editing	40	25	5	4.0	0.5	1.25	1.25	0.75	842

Common settings: flow_shift=5, seed=42, fps=16, vit_txt_cfg=1.2, vit_img_cfg=1.0. Video defaults to 81 frames; images use one frame. Guidance is vae_txt_vit_wapg, except reference-guided video editing uses rv2v_wapg. Text-video defaults are 848×480; images are 512×512; video-editing tasks use height/width zero to follow source geometry. The R2V shell runner leaves height/width at CLI defaults (848×480). max_image_size=842 is intentional: it also controls media preprocessing and should not silently be replaced with 848.

For reference-to-video, video editing and motion editing, use the upstream negative prompt, rather than an empty string:

vivid tones, overexposed, static, blurry details, subtitles, style, artwork, painting, image, motionless, overall grayish, worst quality, low quality, JPEG compression artifacts, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn face, deformed, disfigured, malformed limbs, fused fingers, still frame, cluttered background, three legs, too many people in the background, walking backwards

Other tasks retain the codebase's DEFAULT_NEG_PROMPT. R2V system prompt: You are a helpful assistant specialized in subject-to-video generation. Video-editing system prompt: You are a helpful assistant specialized in video editing.

The local CLI selects these defaults from the v2 package metadata; leave the sampling arguments and --neg_prompt unset to inherit them. Gradio's Standard profile uses the same presets. In Gradio, choose Generation preset → Bernini v2 to apply its recommended controls immediately, including the negative prompt. Bernini v1 is a separate preset; Restore selected preset discards custom edits. The independent Rendering mode choice controls Standard versus LoRA inference. Choosing a preset does not load different weights; a version mismatch is shown. Imported recipes and explicit overrides remain user choices and may retain old settings.

LightX2V eight-step inference is experimental, not this upstream quality preset. It changes guidance to CFG-free distillation and uses external adapters. For a baseline quality evaluation, disable LoRAs and use Standard first.

Note about the September 5 local comparisons

The earlier v2 R2V comparisons already used the above numeric guidance values, 50 planner steps and one visual-plan denoising step; they were not simply v1 settings on v2 weights. However, they explicitly used an empty negative prompt and 864×480 / max_image_size=864, so they did not reproduce the complete upstream quality preset. The LoRA comparisons also changed adapters, and the v1/v2 weights used different quantization formats. Do not treat those examples as proof of a v2 model regression, or assume the negative prompt explains the observed failures. No weights were redownloaded and no new generation was performed for this settings audit; the effect of restoring the full recommended settings remains untested.

Generate an image

python scripts/run_full_bernini_local.py \
  --model_dir models/bernini-v2-int8-convrot --int8_backend kitchen \
  --task_type t2i \
  --prompt 'A small red sailboat on a calm blue lake at sunrise, soft golden light, distant mountains, realistic photograph' \
  --output outputs/sailboat.png

This runs the complete planner → Wan → VAE pipeline, with the 512×512, 50-denoising-step image preset.

Generate or edit a video

# Official v2 text-to-video preset: 81 frames, 480×848, 50 denoising steps.
python scripts/run_full_bernini_local.py \
  --model_dir models/bernini-v2-int8-convrot --task_type t2v \
  --prompt 'A red sailboat moves slowly across a calm blue lake at sunrise' \
  --output outputs/sailboat.mp4

# Video editing, preserving the source's aspect ratio.
python scripts/run_full_bernini_local.py \
  --model_dir models/bernini-v2-int8-convrot --task_type v2v \
  --video /path/to/source.mp4 --prompt 'Change the scene to a snowy winter landscape' \
  --output outputs/edited.mp4

Supported task families: t2i, i2i, t2v, v2v, mv2v, r2v, rv2v. The runner selects version-specific guidance and planning settings. Lower frame counts, resolution and step counts are preview settings, not the official preset. --metrics outputs/run.json saves timings and peak allocated GPU memory.

Eight-step video with LightX2V

python scripts/run_full_bernini_local.py \
  --model_dir models/bernini-v2-int8-convrot --int8_backend kitchen \
  --lightx2v_8step --task_type t2v \
  --prompt 'A red sailboat moves slowly across a calm blue lake at sunrise' \
  --output outputs/sailboat-8step.mp4 --metrics outputs/sailboat-8step.json

# Optional: launch the reusable UI with the same adapters and eight-step default.
python scripts/gradio_local.py \
  --model_dir models/bernini-v2-int8-convrot --lightx2v_8step \
  --host 127.0.0.1 --port 7860

This uses the official Wan2.2 T2V A14B rank-64 1217 high/low pair, the latest compatible T2V pair found in the LightX2V adapter repository on 2026-09-04, pinned to revision 570044187a5219776ef30a5c60c6f76428a3a10a. The source adapters are four-step-trained; this is an eight-step Bernini inference preset, not an official eight-step-trained Bernini checkpoint. Their transfer to Bernini-v2 is experimental.

The preset selects eight UniPC renderer steps, flow shift 5, and CFG-free vae_txt_vit_distill sampling: one conditioned Wan evaluation per step, four high-noise and four low-noise evaluations at the default boundary. Bernini's planner still uses its normal 50 planning iterations. Both adapters default to strength 1 (--lora_strength can adjust this). All 400 target linears per expert are validated strictly, with no skipped modules. The adapters are applied dynamically in the original activation basis; the base INT8 files are unchanged. Omit --lightx2v_8step to return to ordinary 50-step inference.

If converting locally instead of downloading this complete bundle, first run:

python scripts/download_lightx2v_loras.py --model_dir models/bernini-v2-int8-convrot

This downloads only the 1.14 GiB adapter pair and verifies the pinned SHA-256s. The preset never downloads anything implicitly. Do not combine it with manual --high_lora / --low_lora flags or a conflicting denoising step count.

Gradio and verification

Optimized attention launch profiles

The separate codebase also provides a preflight-checked launcher for either SageAttention or xFormers. Both profiles use the same model files and default to eight-step LightX2V video rendering:

python scripts/launch_bernini_optimized.py video --attention sage \
  --model_dir models/bernini-v2-int8-convrot \
  --prompt 'A red sailboat moves slowly across a calm lake at sunrise' \
  --output outputs/sage-video.mp4

python scripts/launch_bernini_optimized.py ui --attention xformers \
  --model_dir models/bernini-v2-int8-convrot --host 127.0.0.1 --port 7860

Swap sage / xformers to select the other backend. Add --full_steps to disable the adapters and use the ordinary 50-step preset. --preflight_only checks CUDA kernels without loading weights. The launcher will not silently fall back when the requested backend is unavailable. Planner/VAE attention remains unchanged.

Tested attention packages: SageAttention 2.2.0, xFormers 0.0.35, with the PyTorch/CUDA setup above. Use compatible CUDA builds; consult the official SageAttention and xFormers install instructions. No packages or models are downloaded automatically by the launcher. Sage uses hardware-dispatched CUDA INT8/FP8 attention and is approximate. xFormers uses memory-efficient exact attention and may dispatch to FlashAttention.

On RTX 5090, a 33,390-token, 40-head × 128-dimension BF16 self-attention benchmark took 109.88 ms with FA2, 110.17 ms with xFormers and 48.02 ms with Sage (2.29× attention-only speedup). Synthetic Sage attention-output relative L2 error was about 3.9%; perceptual equivalence is not guaranteed. xFormers matched FA2 on these benchmark inputs. These are not end-to-end video speedups.

Both optimized launch profiles passed full 81-frame, 480×848, eight-step video generation: 158.8 seconds / 26.33 GiB peak VRAM with Sage and 173.3 seconds / 26.33 GiB with xFormers. Individual run times depend on disk cache and I/O; the earlier FA2 runs ranged from 153 to 187 seconds. Sage's usual denoising steps took about 6.25 seconds versus 8.86 seconds with xFormers, excluding expert swaps. The Sage sample's first/middle/last frames were checked visually; this is a smoke test, not a perceptual-quality benchmark.

Standard UI and verification

pip install 'gradio[mcp]==6.15.0'
python scripts/gradio_local.py \
  --model_dir models/bernini-v2-int8-convrot --host 127.0.0.1 --port 7860

python scripts/run_full_bernini_local.py \
  --model_dir models/bernini-v2-int8-convrot --build_only
python scripts/verify_bernini_int8.py \
  --model_dir models/bernini-v2-int8-convrot --sha256 --forward

No public Gradio tunnel is enabled unless --share is requested. The UI retains weights for repeated requests; the single-shot CLI releases finished stages.

Provenance and reproducibility

Source revision: 399cf6a18a4c523b367b2b1ac25a2a61009e7df3. bernini_quantized.json records source provenance and converted weight hashes. artifact_manifest.json inventories the published model files.

The separate codebase includes scripts/convert_bernini_int8.py. It converts one source shard at a time, verifies source hashes, deletes its raw downloads after successful conversion, and resumes interruptions. It then packs each component into one safetensors file using bounded-memory byte copies and checksum checks. Re-conversion transfers about 145.6 GiB of combined source weights plus separate T5/VAE assets; allow about 82 GiB free, including consolidation temporary space, optional adapters and an 8 GiB reserve. Existing sharded compact packages can be repacked with scripts/consolidate_bernini_safetensors.py --model_dir PATH.

Performance and limitations

Synthetic Wan linear microbenchmarks on RTX 5090 measured 2.34–2.82× speedup over BF16 at 1024 tokens. This is not an end-to-end speedup. Both complete Wan experts passed finite-output forward tests; full image and video generation passed. The 512×512, 50-step image validation took 87.6 seconds with 27.13 GiB peak allocated VRAM, including disk reads, planning and CPU/GPU transfers.

A full 81-frame, 848×480, 16 fps video took 1,755.9 seconds at 50 steps (27.21 GiB peak allocated VRAM). The same prompt/seed/resolution with the eight-step LightX2V preset took 187.2 seconds (26.28 GiB): about 9.4× faster in this single run. This combines fewer steps, CFG removal, adapters and single-shot memory-placement improvements; it is not an isolated quantization benchmark or a promise of equal perceptual quality. The eight-step sample was also checked visually at its first, middle and last frames. These tests used exact FA2 Wan attention and SDPA planner attention on RTX 5090.

After packing the shards into single files, the same eight-step video was bit-for-bit identical (SHA-256 724d953aecd35cbd2983101cfe8727e1247117591207d2e46b83485df36345dd). That rerun took 153.0 seconds at the same 26.28 GiB peak; disk-cache and concurrent I/O differences affect total timings. The repacking itself does not change the tensor values or denoising computation. All 92 regression tests passed, including the CUDA Sage/xFormers self-attention, cross-attention and packed-batch checks.

--int8_backend auto prefers Kitchen; kitchen requires it; torch uses a slower independent reference. --block_offload streams Wan blocks only—it does not reduce the BF16 planner's memory requirement. Full-checkpoint disk streaming via --offload is not supported for this compact layout.

INT8 is lossy. Synthetic linear relative L2 error was about 1.3–1.4%; this is not a perceptual-quality evaluation. SageAttention and planner FP8 are separate, opt-in accuracy/performance tradeoffs. The eight-step preset was validated on text-to-video; adapter quality on editing/reference-conditioned tasks is not established. The source T2V adapters are chosen for Bernini's 16-channel renderer; the older I2V variants contain incompatible image-specific targets.

Original model: ByteDance Bernini. Kernels: Comfy Kitchen. Format: Comfy Quants INT8 W8A8. Model license: Apache-2.0; see LICENSE. LightX2V adapters: Apache-2.0, redistributed unmodified with provenance and hashes in loras/lightx2v_8step.json.