kimi000/Flux1-Dev-4Step-Raw2Q-1000

FLUX.1-dev Four-Step — Relative1 Raw2Q, step 1000

Complete standalone Diffusers-format merged EMA checkpoint after exactly 1000 optimizer updates. Includes the full transformer, CLIP and T5 text encoders, both tokenizers, VAE, custom interval-conditioned pipeline and scheduler, demo, pinned dependencies, validation reports and example images. No FAR checkout, base-model download, PEFT installation or adapter merging is needed for inference.

The generator uses one relative interval head, a frozen second time embedding (gate 0.25), and merged rank/alpha 256 LoRAv2 attention/MLP/output adapters. All 838 EMA tensors / 419 target modules are merged. Training used paired raw2q regression, distribution matching with a separate global 64-head fake model, AdamW betas (0, 0.999), global batch 128 and 16 GPUs. Only the generator EMA is exported.

Install and run

Use Python 3.10+ and a CUDA GPU. Install a CUDA-enabled PyTorch build for your hardware first, then the tested dependencies. Download this repository's demo.py and requirements.txt together, or clone the repository.

python -m pip install -r requirements.txt
python demo.py --cpu-offload --prompt 'A vintage pastel illustration of a tropical beach with palm trees and gentle waves.' --output beach.png
python demo.py --cpu-offload --prompt 'A white ceramic mug with the words "GOOD MORNING" printed in large blue letters.' --output mug.png
# Run from a downloaded directory:
python demo.py --model /path/to/Flux1-Dev-4Step-Raw2Q-1000 --cpu-offload

The demo defaults to kimi000/Flux1-Dev-4Step-Raw2Q-1000, downloads components automatically, and accepts --revision <commit> to pin both model and pipeline code. --cpu-offload moves components to GPU only when needed; omitting it keeps them on GPU and requires substantially more VRAM. Host RAM and GPU requirements depend on resolution and library versions; validation uses an H100 80GB. If transfers stall, retry with HF_HUB_DISABLE_XET=1.

Python API

import torch
from diffusers import DiffusionPipeline

repo = "kimi000/Flux1-Dev-4Step-Raw2Q-1000"
pipe = DiffusionPipeline.from_pretrained(
    repo, custom_pipeline=repo, trust_remote_code=True,
    torch_dtype=torch.bfloat16,
)
pipe.enable_model_cpu_offload()  # alternatively: pipe.to("cuda")
image = pipe(
    prompt="A vintage pastel illustration of a tropical beach.",
    num_inference_steps=4, guidance_scale=3.5,
    height=1024, width=1024,
    generator=torch.Generator(device="cpu").manual_seed(0),
).images[0]
image.save("result.png")

Required settings and limitations

• Four steps, guidance_scale=3.5. Here 3.5 is the native distilled guidance embedding, matching training. External two-pass CFG is disabled (teacher_guidance_scale=1 in training). This differs from the FLUX3 export's guidance_scale=1.0.
• Keep the custom pipeline and scheduler. This generator conditions on both ends of each interval. Standard FluxPipeline does not supply that second time. The scheduler uses shift exp(1.15)=3.158192909689767 and final-step scale 0.3: raw points 1-j/3.3 for j=0..3, followed by 0, then rational shifting. Do not substitute a standard FLUX scheduler.
• Load with custom_pipeline=repo, trust_remote_code=True. Review the included Python files before enabling remote code.
• Transformer and text encoders use BF16; VAE decoding remains FP32. The custom VAE loader preserves this. Do not call pipe.to(dtype=torch.bfloat16) on the whole pipeline.
• Use a CPU torch.Generator for training seed parity. Noise is BCHW BF16 rounded on CPU, integration is FP32, and transformer inputs are BF16. Output can vary across hardware and library versions.
• Training used 1024x1024. Other resolutions must be multiples of 16 and at least 64; they were not the training target. Default prompts use CLIP 77 / T5 512 tokens, with truncation and no prompt upsampler.
• English prompt mixture: 50% MJHQ, 25% sampled TextAtlas, 25% reward prompts; prompt-only/data-free training.
• Text rendering remains imperfect. Fixed validation samples show repeated/extra/cropped text and malformed long passages. Image integrity checks and numerical validation do not establish OCR accuracy. This is an experimental checkpoint.
• FP32 EMA adapters are merged into BF16 base weights with normal BF16 rounding. The exported sampler is checked against an independently composed training-style four-step sampler.

Examples and validation

See validation_report.json for actual GPU runtime and parity results. The training completed all 1000 updates, final sampling produced 32 images, and all 9870 saved training-state tensor entries passed finite-value readback. The full optimizer/fake-model training checkpoint is not needed for inference and is not included here.

License and provenance

The original FLUX.1 [dev] Non-Commercial License applies to the model weights; see LICENSE.md and the original base-model card in BASE_MODEL_CARD.md. This export does not change those terms.

Base revision: 3de623fc3c33e44ffbe2bad470d0f45bccf2eb21. Training implementation: 8f137b6ec8b582590c92ffb548afac5c18632b85. The source checkpoint SHA256, exact merged tensor hashes and training/export settings are in export_metadata.json.