TrizZZZZ commited on
Commit
4e910b0
Β·
verified Β·
1 Parent(s): c072ec1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -26
README.md CHANGED
@@ -21,39 +21,44 @@ pipeline_tag: image-text-to-video
21
 
22
  ## πŸŽ‰ News
23
 
24
- - **[2026-06-10]** We open-sourced the inference code and model weights of the full Bernini (**Bernini**).
25
  - **[2026-05-22]** We released our paper [Bernini: Latent Semantic Planning for Video Diffusion](https://arxiv.org/abs/2605.22344).
26
 
27
  ## ✨ Highlights
28
 
29
  Bernini is a unified framework for video generation and editing that combines an MLLM-based semantic planner with a DiT-based renderer.
30
 
31
- Compared with the renderer-only Bernini-R release, **Bernini-Diffusers** packages the full semantic-planning pipeline: a Qwen2.5-VL planner, Bernini planning weights, and Wan2.2 diffusion components in one self-contained directory. This makes it the recommended release when you need stronger instruction following, multi-step semantic planning, and better handling of complex video editing requests.
 
 
32
 
33
  ## 🧾 Model card
34
 
35
  | Field | Description |
36
  |-------|-------------|
37
  | Model type | Full video generation/editing pipeline with an MLLM-based semantic planner and a DiT-based renderer. |
38
- | Checkpoint | [`ByteDance/Bernini-Diffusers`](https://huggingface.co/ByteDance/Bernini-Diffusers) |
39
  | Code | [`ByteDance/Bernini`](https://github.com/bytedance/Bernini) |
 
 
40
  | Recommended use | Complex generation/editing requests that benefit from explicit latent semantic planning and stronger instruction following. |
41
- | Model behavior | Better at decomposing complex instructions and planning semantic changes before rendering, at the cost of a heavier checkpoint layout than Bernini-R. |
 
42
 
43
  ### Benchmark snapshot
44
 
45
- | Model | EditVerse | OpenVE | OpenS2V | VBench | Bernini-v2v (OS) | Bernini-vr2v (OS) |
46
  |---|---|---|---|---|---|---|
47
- | [Bernini 7+14B](https://huggingface.co/ByteDance/Bernini-Diffusers) | 8.02 | 4.03 | 62.30 | 84.37 | 3.49 | 3.48 |
48
 
49
  On video editing, Bernini reaches the first tier among leading closed-source commercial models in our internal arena evaluation based on blind human pairwise comparisons.
50
 
51
  ## πŸ“¦ Package layout
52
 
53
- This release is a **self-contained diffusers-format directory**. Pass the downloaded `Bernini-Diffusers` directory directly to `--config`.
54
 
55
  ```text
56
- Bernini-Diffusers/
57
  bernini/
58
  mllm/
59
  scheduler/
@@ -76,8 +81,8 @@ At runtime:
76
 
77
  ```bash
78
  pip install -U "huggingface_hub"
79
- hf download ByteDance/Bernini-Diffusers \
80
- --local-dir pretrained_models/Bernini-Diffusers
81
  ```
82
 
83
  ## πŸš€ Usage
@@ -89,27 +94,24 @@ The official inference code is available in the [Bernini repository](https://git
89
  ```bash
90
  git clone https://github.com/bytedance/Bernini.git bernini && cd bernini
91
  pip install -r requirements.txt
 
 
 
92
  ```
93
 
94
  Recommended environment:
95
 
96
  - **Python** 3.11.2
97
- - **PyTorch** 2.5.1+cu124
98
- - **CUDA toolkit** 12.4
99
  - **GPU** Hopper GPUs (H100/H800/H200) are recommended for best performance
100
 
101
- For multi-GPU sequence parallel inference, install VeOmni:
102
-
103
- ```bash
104
- pip install --no-deps git+https://github.com/ByteDance-Seed/VeOmni.git@v0.1.10
105
- ```
106
-
107
  ### Load the model
108
 
109
  Pass the downloaded directory directly as `--config`:
110
 
111
  ```bash
112
- python infer_single_gpu.py --config pretrained_models/Bernini-Diffusers \
113
  --case assets/testcases/i2i/i2i.json --num_frames 1
114
  ```
115
 
@@ -127,17 +129,17 @@ export BERNINI_PE_MODEL=... # vision-capable chat model
127
 
128
  ```bash
129
  # Single GPU
130
- python gradio_demo.py --config pretrained_models/Bernini-Diffusers --port 7860
131
 
132
  # 8 GPUs, 8-way Ulysses sequence parallel
133
  torchrun --nproc-per-node 8 gradio_demo.py --ulysses 8 \
134
- --config pretrained_models/Bernini-Diffusers \
135
  --port 7860 --share
136
  ```
137
 
138
  ### Run scripts
139
 
140
- The [`scripts/bernini/`](https://github.com/bytedance/Bernini/tree/master/scripts/bernini) directory in the Bernini repo provides ready-to-run task launchers for the full pipeline:
141
 
142
  - `run_t2i.sh`
143
  - `run_i2i.sh`
@@ -150,7 +152,7 @@ The [`scripts/bernini/`](https://github.com/bytedance/Bernini/tree/master/script
150
  You can override the model directory with:
151
 
152
  ```bash
153
- export BERNINI_CONFIG=/path/to/Bernini-Diffusers
154
  ```
155
 
156
  ## πŸ“‘ Citation
@@ -177,6 +179,3 @@ Bernini builds on several outstanding open-source projects:
177
  ## πŸ“„ License
178
 
179
  Apache License 2.0.
180
-
181
-
182
- hf upload TrizZZZZ/tmp /mnt/bn/editing/sunmingzhen.triz/codes/debug/Bernini/pretrained_models/Bernini-Diffusers
 
21
 
22
  ## πŸŽ‰ News
23
 
24
+ - **[2026-06-11]** We open-sourced the inference code and model weights of the full Bernini (**Bernini**).
25
  - **[2026-05-22]** We released our paper [Bernini: Latent Semantic Planning for Video Diffusion](https://arxiv.org/abs/2605.22344).
26
 
27
  ## ✨ Highlights
28
 
29
  Bernini is a unified framework for video generation and editing that combines an MLLM-based semantic planner with a DiT-based renderer.
30
 
31
+ **Bernini-Diffusers-v2** packages the full semantic-planning pipeline: a Qwen2.5-VL planner, Bernini planning weights, and Wan2.2 diffusion components in one self-contained diffusers-format directory.
32
+
33
+ Compared with the renderer-only Bernini-R releases, Bernini-Diffusers-v2 is recommended when you need stronger instruction following, multi-step semantic planning, and better handling of complex video generation or editing requests. Compared with the first Bernini-Diffusers release, v2 uses a training recipe that warms up the connector for thousands of steps before co-training, improving reference-guided video editing and OpenS2V performance.
34
 
35
  ## 🧾 Model card
36
 
37
  | Field | Description |
38
  |-------|-------------|
39
  | Model type | Full video generation/editing pipeline with an MLLM-based semantic planner and a DiT-based renderer. |
40
+ | Checkpoint | [`ByteDance/Bernini-Diffusers-v2`](https://huggingface.co/ByteDance/Bernini-Diffusers-v2) |
41
  | Code | [`ByteDance/Bernini`](https://github.com/bytedance/Bernini) |
42
+ | Renderer base | [`Wan2.2-T2V-A14B`](https://huggingface.co/Wan-AI/Wan2.2-T2V-A14B) |
43
+ | Planner base | [`Qwen2.5-VL-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) |
44
  | Recommended use | Complex generation/editing requests that benefit from explicit latent semantic planning and stronger instruction following. |
45
+ | Supported tasks | `t2i`, `i2i`, `t2v`, `v2v`, `rv2v`, and `r2v`. |
46
+ | Model behavior | Decomposes complex instructions and plans semantic changes before rendering, at the cost of a heavier checkpoint layout than Bernini-R. |
47
 
48
  ### Benchmark snapshot
49
 
50
+ | Model | EditVerse | OpenVE | OpenS2V | VBench | Bernini-v2v (OS) | Bernini-rv2v (OS) |
51
  |---|---|---|---|---|---|---|
52
+ | [Bernini-v2 7+14B](https://huggingface.co/ByteDance/Bernini-Diffusers-v2) | 8.02 | 3.96 | 63.83 | 84.46 | 3.49 | 3.55 |
53
 
54
  On video editing, Bernini reaches the first tier among leading closed-source commercial models in our internal arena evaluation based on blind human pairwise comparisons.
55
 
56
  ## πŸ“¦ Package layout
57
 
58
+ This release is a **self-contained diffusers-format directory**. Pass the downloaded `Bernini-Diffusers-v2` directory directly to `--config`.
59
 
60
  ```text
61
+ Bernini-Diffusers-v2/
62
  bernini/
63
  mllm/
64
  scheduler/
 
81
 
82
  ```bash
83
  pip install -U "huggingface_hub"
84
+ hf download ByteDance/Bernini-Diffusers-v2 \
85
+ --local-dir pretrained_models/Bernini-Diffusers-v2
86
  ```
87
 
88
  ## πŸš€ Usage
 
94
  ```bash
95
  git clone https://github.com/bytedance/Bernini.git bernini && cd bernini
96
  pip install -r requirements.txt
97
+ # Open-VeOmni is required. Install it with --no-deps so it does not pull in a
98
+ # different torch build and override the pinned torch==2.7.1+cu126:
99
+ pip install --no-deps git+https://github.com/ByteDance-Seed/VeOmni.git@v0.1.11
100
  ```
101
 
102
  Recommended environment:
103
 
104
  - **Python** 3.11.2
105
+ - **PyTorch** 2.7.1+cu126
106
+ - **CUDA toolkit** 12.6
107
  - **GPU** Hopper GPUs (H100/H800/H200) are recommended for best performance
108
 
 
 
 
 
 
 
109
  ### Load the model
110
 
111
  Pass the downloaded directory directly as `--config`:
112
 
113
  ```bash
114
+ python infer_single_gpu.py --config pretrained_models/Bernini-Diffusers-v2 \
115
  --case assets/testcases/i2i/i2i.json --num_frames 1
116
  ```
117
 
 
129
 
130
  ```bash
131
  # Single GPU
132
+ python gradio_demo.py --config pretrained_models/Bernini-Diffusers-v2 --port 7860
133
 
134
  # 8 GPUs, 8-way Ulysses sequence parallel
135
  torchrun --nproc-per-node 8 gradio_demo.py --ulysses 8 \
136
+ --config pretrained_models/Bernini-Diffusers-v2 \
137
  --port 7860 --share
138
  ```
139
 
140
  ### Run scripts
141
 
142
+ The [`scripts/bernini_v2/`](https://github.com/bytedance/Bernini/tree/master/scripts/bernini_v2) directory in the Bernini repo provides ready-to-run task launchers for Bernini-Diffusers-v2:
143
 
144
  - `run_t2i.sh`
145
  - `run_i2i.sh`
 
152
  You can override the model directory with:
153
 
154
  ```bash
155
+ export BERNINI_CONFIG=/path/to/Bernini-Diffusers-v2
156
  ```
157
 
158
  ## πŸ“‘ Citation
 
179
  ## πŸ“„ License
180
 
181
  Apache License 2.0.