Hugging Face's logo

ByteDance-Seed
/
GenFirst

Text-to-Image
PyTorch
Diffusers
Safetensors
diffusion
latent-diffusion
end-to-end-training
variational-autoencoder
class-conditional-image-generation
sit
mmdit
Model card Files
xet
 Community
zhengzhou commited on
Commit
9907a59
·
verified ·
1 Parent(s): c3f0180

Update README.md with Hugging Face download links

Browse files
Files changed (1) hide show
  1. README.md +25 -24
README.md CHANGED
@@ -62,13 +62,13 @@ Two kinds of artifact are published, and they are used together:
62
  label in `[0, 999]` following the standard 1000-class label ordering; there is
63
  no text input. Epoch counts are prior-only training epochs.
64
 
65
- | Checkpoint | Resolution | VAE | gFID | IS | gFID (guided) | IS (guided) |
66
- |---|---|---|---|---|---|---|
67
- | EiT, 80 epochs | 256x256 | `f16d32` | 2.79 | 175.4 | 1.38 | 271.3 |
68
- | EiT, 480 epochs | 256x256 | `f16d32` | 1.60 | 216.8 | 0.988 | 270.7 |
69
- | EiT, 800 epochs | 256x256 | `f16d32` | **1.45** | 226.7 | **0.969** | 283.4 |
70
- | EiT, 80 epochs | 512x512 | `f16d32` | – | – | 1.63 | 278.9 |
71
- | EiT, 400 epochs | 512x512 | `f16d32` | – | – | **1.23** | 286.1 |
72
 
73
  The 512 models reuse the 256 `f16d32` VAE unchanged and frozen; only the prior
74
  is trained at 512.
@@ -79,10 +79,10 @@ is trained at 512.
79
  VAE, at 512x512. Scores are overall GenEval and DPG-Bench, with no prompt
80
  rewriting. The two variants differ only in which VAE they were trained on.
81
 
82
- | Checkpoint | Resolution | GenEval | DPG-Bench |
83
- |---|---|---|---|
84
- | EiT T2I, variant A | 512x512 | **0.8987** | **82.60** |
85
- | EiT T2I, variant B | 512x512 | 0.8920 | 82.06 |
86
 
87
  ### Shared visual latents (generation + understanding)
88
 
@@ -90,8 +90,8 @@ rewriting. The two variants differ only in which VAE they were trained on.
90
 
91
  | Checkpoint | VAE bundle | Supervision | Resolution | rFID ↓ | PSNR ↑ | GenEval ↑ | DPG-Bench ↑ | Linear Probe Top-1 (%) ↑ |
92
  |---|---|---|---|---|---|---|---|---|
93
- | `shared-latents/siglip/eit-qwenvit-siglip-t2i-512.pt` | `vae/qwenvit-siglip-b256-s2/` | E2E + SigLIP | 512x512 | **0.39** | **27.64** | **0.90** | **83.76** | 79.25 |
94
- | `shared-latents/vlm-nll/eit-qwenvit-vlm-nll-t2i-512.pt` | `vae/qwenvit-nlltrunk-s2/` | E2E + VLM-NLL | 512x512 | 0.46 | 27.19 | 0.89 | 83.49 | **81.69** |
95
 
96
  ### Unified text-image modeling (`JointMMDiT-XL/2`)
97
 
@@ -99,23 +99,24 @@ Bidirectional block-causal `JointMMDiT-XL/2` trained on the jointly E2E-shaped t
99
 
100
  | Checkpoints | Setting | GenEval ↑ | DPG-Bench ↑ | CIDEr ↑ | BLEU-4 ↑ | METEOR ↑ | ROUGE-L ↑ | SPICE ↑ | CLIPScore ↑ |
101
  |---|---|---|---|---|---|---|---|---|---|
102
- | `unified-text-image/e2e-joint/eit-joint-e2e-text-e2e-img-512-sft.pt` (T2I) & `eit-joint-e2e-text-e2e-img-512.pt` (I2T) | E2E-joint text + image VAEs | **86.57** | **72.68** | **33.28** | **7.76** | **17.74** | **34.16** | **11.81** | **57.97** |
103
 
104
  ### End-to-end VAEs
105
 
106
  Reconstruction quality of the posterior mean, measured at the resolution the
107
  VAE is used at.
108
 
109
- | VAE | Downsample / channels | Initialized from | rFID | PSNR | SSIM | LPIPS |
110
- |---|---|---|---|---|---|---|
111
- | `f8d4` | 8x, 4ch | SD-VAE (`stabilityai/sd-vae-ft-ema`) | **0.57** | 25.13 | 0.71 | 0.14 |
112
- | `f16d32` | 16x, 32ch | VA-VAE (LightningDiT) | 0.36 | 25.82 | 0.74 | 0.12 |
113
- | `f8d16` variant A | 8x, 16ch | FLUX.1 VAE | 0.23 | 29.43 | 0.85 | 0.07 |
114
- | `f8d16` variant B | 8x, 16ch | FLUX.1 VAE | 0.21 | 29.37 | 0.85 | 0.07 |
115
- | `qwenvit-siglip-b256-s2` | 8x, 16ch | Qwen3-VL ViT + ViT decoder (SigLIP) | 0.39 | 27.64 | – | – |
116
- | `qwenvit-nlltrunk-s2` | 8x, 16ch | Qwen3-VL ViT + ViT decoder (VLM-NLL) | 0.46 | 27.19 | – | – |
117
- | `e2e-joint-gan-l2-330k` | 8x, 16ch (image) + 16d (text) | FLUX.1 VAE + Cola Text VAE | – | – | – | – |
118
- | `f8d32` (`e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k`, @512) | 8x, 32ch | FLUX.2 VAE | **0.041** | **36.46** | **0.953** | **0.026** |
 
119
 
120
  **A VAE and its latent statistics must be kept together.** The prior was
121
  trained on latents normalized by those exact statistics; pairing a VAE with the
 
62
  label in `[0, 999]` following the standard 1000-class label ordering; there is
63
  no text input. Epoch counts are prior-only training epochs.
64
 
65
+ | Checkpoint | Resolution | VAE | gFID | IS | gFID (guided) | IS (guided) | Download |
66
+ |---|---|---|---|---|---|---|---|
67
+ | EiT, 80 epochs | 256x256 | `f16d32` | 2.79 | 175.4 | 1.38 | 271.3 | [checkpoint](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/class-conditional/imagenet256/eit-imagenet256-80ep.pt) · [weak-20ep](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/class-conditional/imagenet256/eit-imagenet256-weak-20ep.pt) |
68
+ | EiT, 480 epochs | 256x256 | `f16d32` | 1.60 | 216.8 | 0.988 | 270.7 | [checkpoint](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/class-conditional/imagenet256/eit-imagenet256-480ep.pt) · [weak-20ep](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/class-conditional/imagenet256/eit-imagenet256-weak-20ep.pt) |
69
+ | EiT, 800 epochs | 256x256 | `f16d32` | **1.45** | 226.7 | **0.969** | 283.4 | [checkpoint](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/class-conditional/imagenet256/eit-imagenet256-800ep.pt) · [weak-20ep](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/class-conditional/imagenet256/eit-imagenet256-weak-20ep.pt) |
70
+ | EiT, 80 epochs | 512x512 | `f16d32` | – | – | 1.63 | 278.9 | [checkpoint](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/class-conditional/imagenet512/eit-imagenet512-80ep.pt) |
71
+ | EiT, 400 epochs | 512x512 | `f16d32` | – | – | **1.23** | 286.1 | [checkpoint](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/class-conditional/imagenet512/eit-imagenet512-400ep.pt) |
72
 
73
  The 512 models reuse the 256 `f16d32` VAE unchanged and frozen; only the prior
74
  is trained at 512.
 
79
  VAE, at 512x512. Scores are overall GenEval and DPG-Bench, with no prompt
80
  rewriting. The two variants differ only in which VAE they were trained on.
81
 
82
+ | Checkpoint | Resolution | GenEval | DPG-Bench | Download |
83
+ |---|---|---|---|---|
84
+ | EiT T2I, variant A | 512x512 | **0.8987** | **82.60** | [checkpoint](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/text-to-image/variant-a/eit-t2i-variant-a-512.pt) · [VAE](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/e2e-flux-vae-sit-imagenet-s2) |
85
+ | EiT T2I, variant B | 512x512 | 0.8920 | 82.06 | [checkpoint](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/text-to-image/variant-b/eit-t2i-variant-b-512.pt) · [VAE](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/e2e-flux-vae-mmdit-l2-t2i-s2) |
86
 
87
  ### Shared visual latents (generation + understanding)
88
 
 
90
 
91
  | Checkpoint | VAE bundle | Supervision | Resolution | rFID ↓ | PSNR ↑ | GenEval ↑ | DPG-Bench ↑ | Linear Probe Top-1 (%) ↑ |
92
  |---|---|---|---|---|---|---|---|---|
93
+ | [`shared-latents/siglip/eit-qwenvit-siglip-t2i-512.pt`](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/shared-latents/siglip/eit-qwenvit-siglip-t2i-512.pt) | [`vae/qwenvit-siglip-b256-s2/`](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/qwenvit-siglip-b256-s2) | E2E + SigLIP | 512x512 | **0.39** | **27.64** | **0.90** | **83.76** | 79.25 |
94
+ | [`shared-latents/vlm-nll/eit-qwenvit-vlm-nll-t2i-512.pt`](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/shared-latents/vlm-nll/eit-qwenvit-vlm-nll-t2i-512.pt) | [`vae/qwenvit-nlltrunk-s2/`](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/qwenvit-nlltrunk-s2) | E2E + VLM-NLL | 512x512 | 0.46 | 27.19 | 0.89 | 83.49 | **81.69** |
95
 
96
  ### Unified text-image modeling (`JointMMDiT-XL/2`)
97
 
 
99
 
100
  | Checkpoints | Setting | GenEval ↑ | DPG-Bench ↑ | CIDEr ↑ | BLEU-4 ↑ | METEOR ↑ | ROUGE-L ↑ | SPICE ↑ | CLIPScore ↑ |
101
  |---|---|---|---|---|---|---|---|---|---|
102
+ | [`eit-joint-e2e-text-e2e-img-512-sft.pt`](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/unified-text-image/e2e-joint/eit-joint-e2e-text-e2e-img-512-sft.pt) (T2I) & [`eit-joint-e2e-text-e2e-img-512.pt`](https://huggingface.co/ByteDance-Seed/GenFirst/blob/main/unified-text-image/e2e-joint/eit-joint-e2e-text-e2e-img-512.pt) (I2T) | E2E-joint text + image VAEs ([`vae/e2e-joint-gan-l2-330k/`](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/e2e-joint-gan-l2-330k)) | **86.57** | **72.68** | **33.28** | **7.76** | **17.74** | **34.16** | **11.81** | **57.97** |
103
 
104
  ### End-to-end VAEs
105
 
106
  Reconstruction quality of the posterior mean, measured at the resolution the
107
  VAE is used at.
108
 
109
+ | VAE | Downsample / channels | Initialized from | rFID | PSNR | SSIM | LPIPS | Download |
110
+ |---|---|---|---|---|---|---|---|
111
+ | `f8d4` | 8x, 4ch | SD-VAE (`stabilityai/sd-vae-ft-ema`) | **0.57** | 25.13 | 0.71 | 0.14 | [Hugging Face](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/e2e-f8d4) |
112
+ | `f16d32` (decoder-ft, @256) | 16x, 32ch | VA-VAE (LightningDiT) | 0.36 | 25.82 | 0.74 | 0.12 | [Hugging Face](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/e2e-f16d32) |
113
+ | `f16d32` (no dec-ft, @512) | 16x, 32ch | VA-VAE (LightningDiT) | – | – | – | – | [Hugging Face](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/e2e-f16d32-nodecft) |
114
+ | `f8d16` variant A | 8x, 16ch | FLUX.1 VAE | 0.23 | 29.43 | 0.85 | 0.07 | [Hugging Face](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/e2e-flux-vae-sit-imagenet-s2) |
115
+ | `f8d16` variant B | 8x, 16ch | FLUX.1 VAE | 0.21 | 29.37 | 0.85 | 0.07 | [Hugging Face](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/e2e-flux-vae-mmdit-l2-t2i-s2) |
116
+ | `qwenvit-siglip-b256-s2` | 8x, 16ch | Qwen3-VL ViT + ViT decoder (SigLIP) | 0.39 | 27.64 | – | – | [Hugging Face](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/qwenvit-siglip-b256-s2) |
117
+ | `qwenvit-nlltrunk-s2` | 8x, 16ch | Qwen3-VL ViT + ViT decoder (VLM-NLL) | 0.46 | 27.19 | – | – | [Hugging Face](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/qwenvit-nlltrunk-s2) |
118
+ | `e2e-joint-gan-l2-330k` | 8x, 16ch (image) + 16d (text) | FLUX.1 VAE + Cola Text VAE | – | – | – | – | [Hugging Face](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/e2e-joint-gan-l2-330k) |
119
+ | `f8d32` (`e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k`, @512) | 8x, 32ch | FLUX.2 VAE | **0.041** | **36.46** | **0.953** | **0.026** | [Hugging Face](https://huggingface.co/ByteDance-Seed/GenFirst/tree/main/vae/e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k) |
120
 
121
  **A VAE and its latent statistics must be kept together.** The prior was
122
  trained on latents normalized by those exact statistics; pairing a VAE with the