Preserve pinned official Ideogram 4 FP8 release, components, original terms and reference code
8ebf922 verified | # Model Architecture | |
| ``` | |
| prompt ─► Qwen3-VL-8B-Instruct (extract hidden states from layers (0,3,…,33,35) → concat) | |
| │ | |
| ▼ | |
| ┌─────────────────────────────────────────────────┐ | |
| │ Ideogram4Transformer │ | |
| │ • 34 × Ideogram4TransformerBlock │ | |
| │ – Ideogram4Attention (QK-RMSNorm, MRoPE) │ | |
| │ – Ideogram4MLP (SwiGLU) │ | |
| │ – adaln scale/gate from t-embedding │ | |
| │ • Ideogram4FinalLayer │ | |
| └─────────────────────────────────────────────────┘ | |
| │ | |
| │ velocity prediction | |
| │ | |
| ▼ | |
| Euler flow-matching sampler with asymmetric CFG | |
| │ | |
| │ denoised image latents | |
| │ | |
| ▼ | |
| VAE decode | |
| │ | |
| ▼ | |
| PIL.Image | |
| ``` | |
| The transformer is a single-stream DiT: text tokens (Qwen3-VL hidden states from | |
| the activation layers) and image latent tokens are concatenated into one | |
| sequence, modulated per-block by an AdaLN computed from the flow-matching | |
| timestep embedding. Attention uses QK-RMSNorm and 3D MRoPE so that text and | |
| image tokens share a unified positional space. | |
| Model spec: | |
| | field | value | | |
| |-------------------|---------------| | |
| | `emb_dim` | 4608 | | |
| | `num_layers` | 34 | | |
| | `num_heads` | 18 | | |
| | `intermediate` | 12288 | | |
| | `adanln_dim` | 512 | | |
| | `rope_theta` | 5_000_000 | | |
| | `mrope_section` | (24, 20, 20) | | |
| | latent channels | 32 × 2² = 128 | | |
| | max text tokens | 2048 | | |
| | sampler | Euler flow-matching, logit-normal schedule, asymmetric CFG | | |