# GenFirst model release manifest All paths below are relative to the root of this model release. These are the checkpoints used for the paper results, not the later validation runs. ## VAE bundles - `vae/e2e-f8d4/`: ImageNet f8d4 VAE initialized from SD-VAE. - `vae/e2e-f16d32/`: decoder-finetuned f16d32 VAE used by the ImageNet-256 class-conditional results. - `vae/e2e-f16d32-nodecft/`: pre-decoder-finetune f16d32 VAE used by the ImageNet-512 results. - `vae/e2e-flux-vae-sit-imagenet-s2/`: T2I variant A VAE (FLUX.1 init). - `vae/e2e-flux-vae-mmdit-l2-t2i-s2/`: T2I variant B VAE (FLUX.1 init). - `vae/qwenvit-siglip-b256-s2/`: Qwen3-VL ViT + ViT decoder VAE shaped with E2E + SigLIP supervision (Table 9). - `vae/qwenvit-nlltrunk-s2/`: Qwen3-VL ViT + ViT decoder VAE shaped with E2E + VLM-NLL supervision (Table 9). - `vae/e2e-joint-gan-l2-330k/`: Jointly E2E-shaped FLUX.1 image VAE + Cola text VAE (Table 10). - `vae/e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k/`: 32-channel f8d32 E2E FLUX.2 VAE (three-stage aspect-ratio-bucketed E2E + decoder finetune). Every VAE directory contains: - `.pt`: directly loadable VAE state dict. - `-latents-stats.pt`: required latent normalization statistics. - `-discriminator-ckpt.pt`: only needed to resume end-to-end training. In addition: - `vae/e2e-joint-gan-l2-330k/` includes `e2e-joint-gan-l2-330k-text-vae.pt` (standalone E2E-reshaped Cola text VAE state dict). - `vae/e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k/` includes: - `e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k.safetensors` and `diffusion_pytorch_model.safetensors`: diffusers-compatible safetensors weights. - `config.json` and `vae_params.json`: diffusers configuration and explicit per-channel `scaling_factor` and `shift_factor`: - `--vae_pth=vae/e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k/e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k.safetensors` - `--model.vae.scaling_factor=[0.887221,0.886926,0.887096,0.881914,0.886901,0.887307,0.886685,0.886833,0.886540,0.886646,0.887031,0.887073,0.887381,0.886768,0.887299,0.885674,0.886466,0.886795,0.886979,0.886519,0.886949,0.886949,0.887036,0.886828,0.887344,0.887004,0.886816,0.886799,0.887354,0.887523,0.887012,0.886643]` - `--model.vae.shift_factor=[-0.000623,0.000143,0.000282,0.001599,-0.001093,-0.000477,0.000855,-0.000830,-0.000883,0.001076,-0.000417,0.000200,0.000626,-0.000649,-0.000137,0.001453,-0.000816,-0.000307,-0.000290,0.000662,0.000484,0.000993,0.000574,-0.000139,-0.000914,0.000578,-0.000691,-0.000323,0.000619,-0.000606,-0.000372,0.000437]` - `e2e-vae-flux2-sitxl2-auxkl2p5e8-s3-20k-stage2-bn-latents-stats.pt`: Stage-2 70k SiT EMA BatchNorm statistics. Keep each VAE and its latent statistics together. ## Class-conditional priors ImageNet-256, paired with `vae/e2e-f16d32/`: - `class-conditional/imagenet256/eit-imagenet256-weak-20ep.pt`: weak model required for the paper's autoguidance evaluation. - `class-conditional/imagenet256/eit-imagenet256-80ep.pt` - `class-conditional/imagenet256/eit-imagenet256-480ep.pt` - `class-conditional/imagenet256/eit-imagenet256-800ep.pt` ImageNet-512, paired with `vae/e2e-f16d32-nodecft/`: - `class-conditional/imagenet512/eit-imagenet512-80ep.pt` - `class-conditional/imagenet512/eit-imagenet512-400ep.pt` These prior checkpoints are directly accepted by `Prior_only/generate.py --ckpt` and `Prior_only/scripts/eval_fid.sh` through its `CKPT` override. Do not convert or strip them. ## Text-to-image priors - `text-to-image/variant-a/eit-t2i-variant-a-512.pt`, paired with `vae/e2e-flux-vae-sit-imagenet-s2/`. - `text-to-image/variant-b/eit-t2i-variant-b-512.pt`, paired with `vae/e2e-flux-vae-mmdit-l2-t2i-s2/`. These checkpoints are directly accepted by `Prior_only/infer_t2i.py` and the GenEval/DPG generation scripts. Pass the matching VAE, latent statistics and `--text-encoder-name openai/clip-vit-large-patch14`; no checkpoint conversion is required. ## Shared visual latents (generation + understanding, Table 9) - `shared-latents/siglip/eit-qwenvit-siglip-t2i-512.pt`, paired with `vae/qwenvit-siglip-b256-s2/` (`0.90` GenEval, `83.76` DPG-Bench, `79.25%` IN1K linear probe). - `shared-latents/vlm-nll/eit-qwenvit-vlm-nll-t2i-512.pt`, paired with `vae/qwenvit-nlltrunk-s2/` (`0.89` GenEval, `83.49` DPG-Bench, `81.69%` IN1K linear probe). Directly accepted by `Prior_only/infer_t2i.py` and `Prior_only/gen_for_bench.py` with `--vae-arch qwenvit --vae-vit-path Qwen/Qwen3-VL-2B-Instruct`. ## Unified text-image modeling (`JointMMDiT-XL/2`, Table 10) - `unified-text-image/e2e-joint/eit-joint-e2e-text-e2e-img-512-sft.pt`: Phase-3 512x512 SFT checkpoint for text-to-image (`86.57` GenEval, `72.68` DPG-Bench). - `unified-text-image/e2e-joint/eit-joint-e2e-text-e2e-img-512.pt`: Phase-2 512x512 checkpoint before SFT for image-to-text COCO captioning (`33.28` CIDEr, `57.97` CLIPScore). - Both pair with `vae/e2e-joint-gan-l2-330k/` (`--image-vae-ckpt vae/e2e-joint-gan-l2-330k/e2e-joint-gan-l2-330k.pt`; the reshaped text VAE weights are already embedded inside `ckpt["text_vae"]`). ## Checkpoint formats produced by the repository - `train_prior_only.py`, `train_t2i_frozen.py`, and `train_joint_frozen.py` save final checkpoints in direct evaluation format. - Class-conditional `train_e2e.py` checkpoints can be used directly by the top-level `generate.py`; all `train_e2e.py`, `train_e2e_vae.py`, and `train_decoder_ft.py` checkpoints can be evaluated directly for reconstruction. Before using either checkpoint's VAE with a separate prior, run `Prior_only/scripts/run_prior_only_pipeline.sh extract` (or `save_joint_vae_weights.py` for `train_joint_e2e.py`), which produces a standalone VAE bundle. - A downloaded VAE bundle is already extracted and must not be extracted again. Weights are released under CC BY-NC 4.0. See the repository's `MODEL_CARD.md`, `NOTICE`, and documentation for complete terms and commands.