feat: Ideogram-4 precision config (config-only, no weights)

README.md

@@ -0,0 +1,95 @@
+---
+language:
+- en
+- zh
+tags:
+- text-to-image
+- quantization
+- quantfunc
+- precision-config
+- ideogram
+license: apache-2.0
+---
+
+# QuantFunc
+
+<div align="center" style="margin-top: 50px;">
+  <img src="https://raw.githubusercontent.com/RealJonathanYip/ComfyUI-QuantFunc/main/assets/logo.webp" width="300" alt="Logo">
+</div>
+
+<p align="center">
+  🤗 <a href="https://huggingface.co/QuantFunc">Hugging Face</a> &nbsp;|&nbsp;
+  🤖 <a href="https://www.modelscope.cn/profile/QuantFunc">ModelScope</a> &nbsp;|&nbsp;
+  💻 <a href="https://github.com/RealJonathanYip/ComfyUI-QuantFunc">GitHub</a> &nbsp;|&nbsp;
+  💬 <a href="#wechat">WeChat (微信)</a> &nbsp;|&nbsp;
+  🎮 <a href="https://discord.gg/jCp9TpFWcn">Discord</a>
+</p>
+
+# Ideogram-4-Series
+
+> ⚠️ **Config-only repository — no model weights.**
+> This repo contains **only** a QuantFunc per-layer **precision config** (`precision-config/ideogram4_a4w4.json`).
+> It does **not** contain, mirror, or redistribute any Ideogram model weights. **You bring your own** officially-obtained Ideogram 4 model; this config only tells the QuantFunc engine how to quantize it **at load time, on your own machine**.
+
+**Powered by the [QuantFunc ComfyUI plugin](https://github.com/RealJonathanYip/ComfyUI-QuantFunc) — the fastest diffusion inference engine:**
+
+- 🚀 **2x–11x speedup** over standard BF16/FP16 Python pipelines.
+- ⚙️ **Native C++/CUDA** (`libquantfunc.so` / `quantfunc.dll`), **zero Python model dependencies**.
+- 🧩 **Universal format adapter** — loads **diffusers / BFL (Flux) / HF / nunchaku SVDQ** layouts directly, no manual conversion.
+- 🟢 **Full GPU coverage** — RTX 20/30/40/50 · A100/H100/H200/B100/B200/GB300 · RTX 6000 Ada / PRO Blackwell (CUDA 12 & 13); native **FP4** on Blackwell.
+
+👉 **Install the plugin:** **https://github.com/RealJonathanYip/ComfyUI-QuantFunc**
+
+## What this repository provides
+
+A single file — the precision config, nothing else:
+
+```
+Ideogram-4-Series/
+└── precision-config/
+    └── ideogram4_a4w4.json    # per-layer precision map for the Lighting backend (W4A4)
+```
+
+We deliberately **do not host Ideogram 4 weights**. The QuantFunc **Lighting** backend does **runtime** quantization: you load the *official* weights and they are quantized **in-memory at load**, so no pre-quantized checkpoint is ever distributed.
+
+## How to use
+
+1. **Obtain the official Ideogram 4 model yourself** in any QuantFunc-supported layout (**diffusers**, BFL/Flux-style, or HF). Follow Ideogram's official distribution channels and license terms.
+2. **Install the QuantFunc ComfyUI plugin:** https://github.com/RealJonathanYip/ComfyUI-QuantFunc
+3. **Load the official model** through the **Build Pipeline** node (universal format adapter).
+4. **Precision config** — leave the node on **`auto detect`** (it recognizes Ideogram 4 and applies `ideogram4_a4w4.json` automatically), or point it at this file manually. The Lighting engine then runtime-quantizes the transformer to **W4A4** (4-bit heavy GEMMs + 8-bit sensitive projections).
+
+## Precision config — `ideogram4_a4w4.json`
+
+Per-layer precision map (mirrors the Klein-style configs). **Measured** on a dual-transformer **24 GB** card (`cuda_overhead` 399 MB) to fit and render a coherent, prompt-matching image — with sharper detail than FP16-non-block.
+
+| Layer group | Precision | Why |
+|---|---|---|
+| `layers.attention.qkv` · `layers.attention.o` | **4-bit** (AUTO_4 → INT4 on SM89, FP4 on SM120) | self-attention projections; large K/N, quant-robust |
+| `layers.feed_forward.w1/w2/w3` | **4-bit** | SwiGLU MLP — largest matrices, primary memory target |
+| `input_proj` · `llm_cond_proj` · `t_embedding.mlp_in/out` · `adaln_proj` · `final_layer.linear` | **8-bit** (AUTO_8 → FP8 on SM89+, INT8 older; W8A8) | sensitive non-block projection GEMMs |
+| `layers.adaln_modulation` · `final_layer.adaln_modulation` | **FP16** | M=1 modulation GEMVs — per-token activation quant collapses conditioning; engine skips them |
+
+**Net:** 170 block GEMMs @ 4-bit · 5 non-block projection GEMMs @ AUTO_8 (FP8 on SM89) · 2 adaLN-modulation GEMVs @ FP16.
+
+Verified coherent on SM89 (INT8 dashboard-run + FP8 CLI-run, each `cuda_overhead` 399 MB). AUTO_8 picks **FP8** on SM89 for better dynamic range on these sensitive projections.
+
+## Hardware
+
+- NVIDIA **RTX 20-series and above** (CUDA 12 & 13). Native **FP4** on Blackwell (SM120); INT4 on SM89.
+- Fits a **24 GB** card with the a4w4 map (measured `cuda_overhead` 399 MB).
+
+## Legal / Attribution
+
+- This repository distributes **only** the QuantFunc precision-config JSON — our own work, Apache-2.0.
+- It contains **no Ideogram weights** and is **not affiliated with, nor endorsed by, Ideogram**.
+- "Ideogram" is a trademark of its respective owner. You are solely responsible for obtaining the official model and complying with its license and terms of use.
+
+## Community
+
+- 🎮 [Discord server](https://discord.gg/jCp9TpFWcn)
+- 💬 Scan the QR code below to join our WeChat group:
+
+<div align="center" id="wechat">
+  <img src="https://raw.githubusercontent.com/RealJonathanYip/ComfyUI-QuantFunc/main/assets/WeChat.jpg" alt="WeChat Group" width="300">
+</div>

precision-config/ideogram4_a4w4.json

@@ -0,0 +1,16 @@
+{
+  "_comment": "Ideogram-4 per-layer precision map (mirrors klein-style configs/flux2-klein-*.json). Heavy block GEMMs (attention + FFN) are 4-bit (AUTO_4 → INT4 on SM89, FP4 on SM120); the sensitive non-block PROJECTION GEMMs are 8-bit AUTO_8 ('8' → FP8 on SM89+, INT8 on older; W8A8) — MEASURED on the dual-transformer 24GB to fit (cuda_overhead 399 MB) and render a coherent prompt-matching image, with sharper detail than FP16-non-block. Both 8-bit variants verified coherent on SM89 (i8 dashboard-run + f8 CLI-run, each cuda_overhead 399 MB); AUTO_8 picks FP8 on SM89 (native FP8 tensor cores, better dynamic range than INT8 for these sensitive projections). Only the adaln_modulation GEMVs (block + final) stay FP16 (16): they are M=1 GEMVs whose per-token activation quantization is too lossy (conditioning collapse; mirrors ZImage's ZImageBlockPrecision.modulation=FP16 default), and the engine's quantizeWeights deliberately skips them. Keys match the layer-path prefixes used by Ideogram4TransformerLighting::quantizeBlockWeights + Ideogram4Block::quantizeWeights (the numeric block index is skipped by PrecisionMap::matchesPrefix, so 'layers.attention.qkv' matches layers.0..33). Per entry: (1) 'layers.attention.qkv/o' → self-attention projections, large K/N, quant-robust → 4; (2) 'layers.feed_forward.w1/w2/w3' → SwiGLU MLP, largest matrices, primary memory target → 4; (3) 'layers.adaln_modulation' → adaLN modulation GEMV (M=1), NOT called by quantizeWeights → 16; (4) 'input_proj'/(5)'llm_cond_proj'/(6)'t_embedding.*'/(7)'adaln_proj'/(8)'final_layer.linear' → non-block projection GEMMs → AUTO_8 ('8': FP8 on SM89+, INT8 older); (9) 'final_layer.adaln_modulation' → final conditioning GEMV → 16. Net: 170 block GEMMs at 4-bit, 5 non-block projection GEMMs at AUTO_8 (FP8 on SM89), 2 adaln-modulation GEMVs at FP16.",
+  "layers.attention.qkv": 4,
+  "layers.attention.o": 4,
+  "layers.feed_forward.w1": 4,
+  "layers.feed_forward.w2": 4,
+  "layers.feed_forward.w3": 4,
+  "layers.adaln_modulation": 16,
+  "input_proj": 8,
+  "llm_cond_proj": 8,
+  "t_embedding.mlp_in": 8,
+  "t_embedding.mlp_out": 8,
+  "adaln_proj": 8,
+  "final_layer.linear": 8,
+  "final_layer.adaln_modulation": 16
+}