ideogram-v4-instant / README.md
kaanakan's picture
Fix no-CFG example for Diffusers 0.39.0
d47b425 verified
|
Raw
History Blame Contribute Delete
7.65 kB
---
license: other
license_name: ideogram-4-non-commercial
license_link: https://huggingface.co/fal/ideogram-v4-instant/blob/main/LICENSE.md
library_name: diffusers
pipeline_tag: text-to-image
base_model: ideogram-ai/ideogram-4-fp8
base_model_relation: finetune
tags:
- diffusers
- safetensors
- text-to-image
- image-generation
- flow-matching
- ideogram4
- distillation
- cfg-distillation
- no-cfg
- no-qad
- instant
- 8-step
- bf16
extra_gated_prompt: By requesting access, you acknowledge the Ideogram Non-Commercial Model Agreement linked above.
extra_gated_fields:
I agree to use this model only as permitted by the Ideogram Non-Commercial Model Agreement: checkbox
---
# Ideogram 4 Instant β€” by fal
**8 steps. One transformer. No runtime CFG.**
Ideogram 4 Instant is an eight-step text-to-image checkpoint developed and released by
[fal](https://fal.ai), based on [`ideogram-ai/ideogram-4-fp8`](https://huggingface.co/ideogram-ai/ideogram-4-fp8).
This release contains the BF16 weights from immediately before the quantization-aware distillation
(QAD) stage. It combines timestep distillation with a single conditional branch for generation in
just eight denoising steps.
<video src="https://huggingface.co/fal/ideogram-v4-instant/resolve/main/assets/ideogram-v4-by-fal.mp4" controls autoplay loop muted playsinline width="100%"></video>
## Key features
- ⚑ **8-step inference** β€” the Instant schedule at 1024Γ—1024.
- 🎯 **No runtime CFG** β€” one conditional transformer call per denoising step; no negative branch or CFG blend.
- 🧠 **Pre-QAD BF16 weights** β€” approximately 9.28 billion parameters, captured immediately before QAD.
- 🧩 **Standard Diffusers components** β€” no repository Python code and no `trust_remote_code`.
- πŸ“¦ **Transformer-only release** β€” shared components come from Ideogram AI's public, gated Diffusers repository.
Read how fal built the single-branch, few-step Ideogram 4 serving path in
[Serving sub-second Ideogram v4 without quality loss](https://blog.fal.ai/serving-sub-second-ideogram-v4-without-quality-loss/).
## Hosted API
The production-optimized model is available on fal through
[`ideogram/v4/instant`](https://fal.ai/models/ideogram/v4/instant/api).
> The hosted endpoint uses the later QAD-trained, FP4-optimized production weights. This repository
> intentionally publishes the **BF16 checkpoint from before QAD**.
## Usage
This model expects Ideogram 4's structured JSON caption format. The hosted fal endpoint expands
natural-language prompts automatically; local Diffusers inference does not. Expand the prompt with
an Ideogram-compatible magic-prompt model first, or provide a complete structured caption like the
one below.
The component wiring below uses the official public, gated
[`ideogram-ai/ideogram-4-nf4-diffusers`](https://huggingface.co/ideogram-ai/ideogram-4-nf4-diffusers)
repository. Only its tokenizer, text encoder, VAE, and scheduler are used; neither of its diffusion
transformers is loaded. You must accept Ideogram's access gate before downloading the components.
Released Diffusers 0.39.0 still expects an unconditional transformer even for a distilled,
single-branch checkpoint. The zero-parameter compatibility module below satisfies that plumbing
without loading or running a second diffusion transformer. With `guidance_scale=1.0`, the stock
blend is exactly `1.0 * conditional + 0.0 * dummy_unconditional`.
This shim addresses the mandatory-CFG plumbing only. It does not apply fal's native terminal
timestep or frequency-table corrections, so stock Diffusers 0.39.0 is not bit-exact with the
optimized fal runtime.
```python
import json
import torch
from diffusers import Ideogram4Pipeline, Ideogram4Transformer2DModel
repo_id = "fal/ideogram-v4-instant"
components_repo_id = "ideogram-ai/ideogram-4-nf4-diffusers"
components_revision = "1874bc70267ba2c823a7239e1d70dd308c8d64dc"
class ZeroUnconditionalTransformer(torch.nn.Module):
"""Zero-parameter stand-in for Diffusers 0.39.0's mandatory CFG branch."""
def __init__(self, dtype=torch.bfloat16):
super().__init__()
self.register_buffer("_dtype_anchor", torch.empty(0, dtype=dtype), persistent=False)
@property
def dtype(self):
return self._dtype_anchor.dtype
def forward(self, *, hidden_states, **kwargs):
return (torch.zeros_like(hidden_states),)
transformer = Ideogram4Transformer2DModel.from_pretrained(
repo_id,
subfolder="transformer",
torch_dtype=torch.bfloat16,
)
pipe = Ideogram4Pipeline.from_pretrained(
components_repo_id,
revision=components_revision,
transformer=transformer,
unconditional_transformer=None,
torch_dtype=torch.bfloat16,
)
pipe.register_modules(unconditional_transformer=ZeroUnconditionalTransformer())
pipe.to("cuda")
prompt = json.dumps(
{
"high_level_description": (
"A bold typographic poster centered on the exact words INSTANT BY FAL, "
"printed in black and electric orange on warm white paper."
),
"compositional_deconstruction": {
"background": (
"Warm white textured paper with even studio lighting and generous negative space."
),
"elements": [
{
"type": "text",
"text": "INSTANT BY FAL",
"desc": (
"Large uppercase geometric sans-serif lettering with crisp print edges, "
"precisely centered."
),
}
],
},
},
ensure_ascii=False,
separators=(",", ":"),
)
generator = torch.Generator(device="cuda").manual_seed(42)
image = pipe(
prompt,
height=1024,
width=1024,
num_inference_steps=8,
guidance_scale=1.0,
guidance_schedule=None,
mu=0.0,
std=1.75,
generator=generator,
).images[0]
image.save("ideogram4-instant.png")
```
The compatibility module has no parameters and its trivial zero output is multiplied by zero; no
unconditional model is loaded and there is no effective runtime CFG.
## Repository layout
```text
.
β”œβ”€β”€ README.md
β”œβ”€β”€ LICENSE.md
β”œβ”€β”€ NOTICE
β”œβ”€β”€ assets/
β”‚ └── ideogram-v4-by-fal.mp4
└── transformer/
β”œβ”€β”€ config.json
β”œβ”€β”€ diffusion_pytorch_model-00001-of-00004.safetensors
β”œβ”€β”€ diffusion_pytorch_model-00002-of-00004.safetensors
β”œβ”€β”€ diffusion_pytorch_model-00003-of-00004.safetensors
β”œβ”€β”€ diffusion_pytorch_model-00004-of-00004.safetensors
└── diffusion_pytorch_model.safetensors.index.json
```
## Weights and provenance
This is the dense BF16 Instant checkpoint captured immediately before QAD. It is not a statically
quantized FP4 or NVFP4 export. During conversion, fused QKV tensors were split into the standard
Diffusers `to_q`, `to_k`, `to_v`, and `to_out` layout without changing their values.
The transformer was derived from `ideogram-ai/ideogram-4-fp8`. Shared inference components are
loaded from `ideogram-ai/ideogram-4-nf4-diffusers`; neither transformer in that repository is loaded
or used.
Ideogram 4 was created by Ideogram AI. This derivative checkpoint was developed and released by fal
and is not an official Ideogram product or endorsed by Ideogram AI.
## License
As a derivative of Ideogram 4, this model inherits the Ideogram 4 Non-Commercial Model Agreement.
The complete inherited license is included in
[`LICENSE.md`](https://huggingface.co/fal/ideogram-v4-instant/blob/main/LICENSE.md) and governs
use and redistribution of this model.