Instructions to use TrizZZZZ/tmp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use TrizZZZZ/tmp with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("TrizZZZZ/tmp", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -21,39 +21,44 @@ pipeline_tag: image-text-to-video
|
|
| 21 |
|
| 22 |
## π News
|
| 23 |
|
| 24 |
-
- **[2026-06-
|
| 25 |
- **[2026-05-22]** We released our paper [Bernini: Latent Semantic Planning for Video Diffusion](https://arxiv.org/abs/2605.22344).
|
| 26 |
|
| 27 |
## β¨ Highlights
|
| 28 |
|
| 29 |
Bernini is a unified framework for video generation and editing that combines an MLLM-based semantic planner with a DiT-based renderer.
|
| 30 |
|
| 31 |
-
|
|
|
|
|
|
|
| 32 |
|
| 33 |
## π§Ύ Model card
|
| 34 |
|
| 35 |
| Field | Description |
|
| 36 |
|-------|-------------|
|
| 37 |
| Model type | Full video generation/editing pipeline with an MLLM-based semantic planner and a DiT-based renderer. |
|
| 38 |
-
| Checkpoint | [`ByteDance/Bernini-Diffusers`](https://huggingface.co/ByteDance/Bernini-Diffusers) |
|
| 39 |
| Code | [`ByteDance/Bernini`](https://github.com/bytedance/Bernini) |
|
|
|
|
|
|
|
| 40 |
| Recommended use | Complex generation/editing requests that benefit from explicit latent semantic planning and stronger instruction following. |
|
| 41 |
-
|
|
|
|
|
| 42 |
|
| 43 |
### Benchmark snapshot
|
| 44 |
|
| 45 |
-
| Model | EditVerse | OpenVE | OpenS2V | VBench | Bernini-v2v (OS) | Bernini-
|
| 46 |
|---|---|---|---|---|---|---|
|
| 47 |
-
| [Bernini 7+14B](https://huggingface.co/ByteDance/Bernini-Diffusers) | 8.02 |
|
| 48 |
|
| 49 |
On video editing, Bernini reaches the first tier among leading closed-source commercial models in our internal arena evaluation based on blind human pairwise comparisons.
|
| 50 |
|
| 51 |
## π¦ Package layout
|
| 52 |
|
| 53 |
-
This release is a **self-contained diffusers-format directory**. Pass the downloaded `Bernini-Diffusers` directory directly to `--config`.
|
| 54 |
|
| 55 |
```text
|
| 56 |
-
Bernini-Diffusers/
|
| 57 |
bernini/
|
| 58 |
mllm/
|
| 59 |
scheduler/
|
|
@@ -76,8 +81,8 @@ At runtime:
|
|
| 76 |
|
| 77 |
```bash
|
| 78 |
pip install -U "huggingface_hub"
|
| 79 |
-
hf download ByteDance/Bernini-Diffusers \
|
| 80 |
-
--local-dir pretrained_models/Bernini-Diffusers
|
| 81 |
```
|
| 82 |
|
| 83 |
## π Usage
|
|
@@ -89,27 +94,24 @@ The official inference code is available in the [Bernini repository](https://git
|
|
| 89 |
```bash
|
| 90 |
git clone https://github.com/bytedance/Bernini.git bernini && cd bernini
|
| 91 |
pip install -r requirements.txt
|
|
|
|
|
|
|
|
|
|
| 92 |
```
|
| 93 |
|
| 94 |
Recommended environment:
|
| 95 |
|
| 96 |
- **Python** 3.11.2
|
| 97 |
-
- **PyTorch** 2.
|
| 98 |
-
- **CUDA toolkit** 12.
|
| 99 |
- **GPU** Hopper GPUs (H100/H800/H200) are recommended for best performance
|
| 100 |
|
| 101 |
-
For multi-GPU sequence parallel inference, install VeOmni:
|
| 102 |
-
|
| 103 |
-
```bash
|
| 104 |
-
pip install --no-deps git+https://github.com/ByteDance-Seed/VeOmni.git@v0.1.10
|
| 105 |
-
```
|
| 106 |
-
|
| 107 |
### Load the model
|
| 108 |
|
| 109 |
Pass the downloaded directory directly as `--config`:
|
| 110 |
|
| 111 |
```bash
|
| 112 |
-
python infer_single_gpu.py --config pretrained_models/Bernini-Diffusers \
|
| 113 |
--case assets/testcases/i2i/i2i.json --num_frames 1
|
| 114 |
```
|
| 115 |
|
|
@@ -127,17 +129,17 @@ export BERNINI_PE_MODEL=... # vision-capable chat model
|
|
| 127 |
|
| 128 |
```bash
|
| 129 |
# Single GPU
|
| 130 |
-
python gradio_demo.py --config pretrained_models/Bernini-Diffusers --port 7860
|
| 131 |
|
| 132 |
# 8 GPUs, 8-way Ulysses sequence parallel
|
| 133 |
torchrun --nproc-per-node 8 gradio_demo.py --ulysses 8 \
|
| 134 |
-
--config pretrained_models/Bernini-Diffusers \
|
| 135 |
--port 7860 --share
|
| 136 |
```
|
| 137 |
|
| 138 |
### Run scripts
|
| 139 |
|
| 140 |
-
The [`scripts/
|
| 141 |
|
| 142 |
- `run_t2i.sh`
|
| 143 |
- `run_i2i.sh`
|
|
@@ -150,7 +152,7 @@ The [`scripts/bernini/`](https://github.com/bytedance/Bernini/tree/master/script
|
|
| 150 |
You can override the model directory with:
|
| 151 |
|
| 152 |
```bash
|
| 153 |
-
export BERNINI_CONFIG=/path/to/Bernini-Diffusers
|
| 154 |
```
|
| 155 |
|
| 156 |
## π Citation
|
|
@@ -177,6 +179,3 @@ Bernini builds on several outstanding open-source projects:
|
|
| 177 |
## π License
|
| 178 |
|
| 179 |
Apache License 2.0.
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
hf upload TrizZZZZ/tmp /mnt/bn/editing/sunmingzhen.triz/codes/debug/Bernini/pretrained_models/Bernini-Diffusers
|
|
|
|
| 21 |
|
| 22 |
## π News
|
| 23 |
|
| 24 |
+
- **[2026-06-11]** We open-sourced the inference code and model weights of the full Bernini (**Bernini**).
|
| 25 |
- **[2026-05-22]** We released our paper [Bernini: Latent Semantic Planning for Video Diffusion](https://arxiv.org/abs/2605.22344).
|
| 26 |
|
| 27 |
## β¨ Highlights
|
| 28 |
|
| 29 |
Bernini is a unified framework for video generation and editing that combines an MLLM-based semantic planner with a DiT-based renderer.
|
| 30 |
|
| 31 |
+
**Bernini-Diffusers-v2** packages the full semantic-planning pipeline: a Qwen2.5-VL planner, Bernini planning weights, and Wan2.2 diffusion components in one self-contained diffusers-format directory.
|
| 32 |
+
|
| 33 |
+
Compared with the renderer-only Bernini-R releases, Bernini-Diffusers-v2 is recommended when you need stronger instruction following, multi-step semantic planning, and better handling of complex video generation or editing requests. Compared with the first Bernini-Diffusers release, v2 uses a training recipe that warms up the connector for thousands of steps before co-training, improving reference-guided video editing and OpenS2V performance.
|
| 34 |
|
| 35 |
## π§Ύ Model card
|
| 36 |
|
| 37 |
| Field | Description |
|
| 38 |
|-------|-------------|
|
| 39 |
| Model type | Full video generation/editing pipeline with an MLLM-based semantic planner and a DiT-based renderer. |
|
| 40 |
+
| Checkpoint | [`ByteDance/Bernini-Diffusers-v2`](https://huggingface.co/ByteDance/Bernini-Diffusers-v2) |
|
| 41 |
| Code | [`ByteDance/Bernini`](https://github.com/bytedance/Bernini) |
|
| 42 |
+
| Renderer base | [`Wan2.2-T2V-A14B`](https://huggingface.co/Wan-AI/Wan2.2-T2V-A14B) |
|
| 43 |
+
| Planner base | [`Qwen2.5-VL-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) |
|
| 44 |
| Recommended use | Complex generation/editing requests that benefit from explicit latent semantic planning and stronger instruction following. |
|
| 45 |
+
| Supported tasks | `t2i`, `i2i`, `t2v`, `v2v`, `rv2v`, and `r2v`. |
|
| 46 |
+
| Model behavior | Decomposes complex instructions and plans semantic changes before rendering, at the cost of a heavier checkpoint layout than Bernini-R. |
|
| 47 |
|
| 48 |
### Benchmark snapshot
|
| 49 |
|
| 50 |
+
| Model | EditVerse | OpenVE | OpenS2V | VBench | Bernini-v2v (OS) | Bernini-rv2v (OS) |
|
| 51 |
|---|---|---|---|---|---|---|
|
| 52 |
+
| [Bernini-v2 7+14B](https://huggingface.co/ByteDance/Bernini-Diffusers-v2) | 8.02 | 3.96 | 63.83 | 84.46 | 3.49 | 3.55 |
|
| 53 |
|
| 54 |
On video editing, Bernini reaches the first tier among leading closed-source commercial models in our internal arena evaluation based on blind human pairwise comparisons.
|
| 55 |
|
| 56 |
## π¦ Package layout
|
| 57 |
|
| 58 |
+
This release is a **self-contained diffusers-format directory**. Pass the downloaded `Bernini-Diffusers-v2` directory directly to `--config`.
|
| 59 |
|
| 60 |
```text
|
| 61 |
+
Bernini-Diffusers-v2/
|
| 62 |
bernini/
|
| 63 |
mllm/
|
| 64 |
scheduler/
|
|
|
|
| 81 |
|
| 82 |
```bash
|
| 83 |
pip install -U "huggingface_hub"
|
| 84 |
+
hf download ByteDance/Bernini-Diffusers-v2 \
|
| 85 |
+
--local-dir pretrained_models/Bernini-Diffusers-v2
|
| 86 |
```
|
| 87 |
|
| 88 |
## π Usage
|
|
|
|
| 94 |
```bash
|
| 95 |
git clone https://github.com/bytedance/Bernini.git bernini && cd bernini
|
| 96 |
pip install -r requirements.txt
|
| 97 |
+
# Open-VeOmni is required. Install it with --no-deps so it does not pull in a
|
| 98 |
+
# different torch build and override the pinned torch==2.7.1+cu126:
|
| 99 |
+
pip install --no-deps git+https://github.com/ByteDance-Seed/VeOmni.git@v0.1.11
|
| 100 |
```
|
| 101 |
|
| 102 |
Recommended environment:
|
| 103 |
|
| 104 |
- **Python** 3.11.2
|
| 105 |
+
- **PyTorch** 2.7.1+cu126
|
| 106 |
+
- **CUDA toolkit** 12.6
|
| 107 |
- **GPU** Hopper GPUs (H100/H800/H200) are recommended for best performance
|
| 108 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 109 |
### Load the model
|
| 110 |
|
| 111 |
Pass the downloaded directory directly as `--config`:
|
| 112 |
|
| 113 |
```bash
|
| 114 |
+
python infer_single_gpu.py --config pretrained_models/Bernini-Diffusers-v2 \
|
| 115 |
--case assets/testcases/i2i/i2i.json --num_frames 1
|
| 116 |
```
|
| 117 |
|
|
|
|
| 129 |
|
| 130 |
```bash
|
| 131 |
# Single GPU
|
| 132 |
+
python gradio_demo.py --config pretrained_models/Bernini-Diffusers-v2 --port 7860
|
| 133 |
|
| 134 |
# 8 GPUs, 8-way Ulysses sequence parallel
|
| 135 |
torchrun --nproc-per-node 8 gradio_demo.py --ulysses 8 \
|
| 136 |
+
--config pretrained_models/Bernini-Diffusers-v2 \
|
| 137 |
--port 7860 --share
|
| 138 |
```
|
| 139 |
|
| 140 |
### Run scripts
|
| 141 |
|
| 142 |
+
The [`scripts/bernini_v2/`](https://github.com/bytedance/Bernini/tree/master/scripts/bernini_v2) directory in the Bernini repo provides ready-to-run task launchers for Bernini-Diffusers-v2:
|
| 143 |
|
| 144 |
- `run_t2i.sh`
|
| 145 |
- `run_i2i.sh`
|
|
|
|
| 152 |
You can override the model directory with:
|
| 153 |
|
| 154 |
```bash
|
| 155 |
+
export BERNINI_CONFIG=/path/to/Bernini-Diffusers-v2
|
| 156 |
```
|
| 157 |
|
| 158 |
## π Citation
|
|
|
|
| 179 |
## π License
|
| 180 |
|
| 181 |
Apache License 2.0.
|
|
|
|
|
|
|
|
|