Update README.md
Browse files
README.md
CHANGED
|
@@ -19,22 +19,22 @@ tags:
|
|
| 19 |
|
| 20 |
## Introduction
|
| 21 |
|
| 22 |
-
**Nucleus-Image** is a text-to-image generation model built on a sparse mixture-of-experts (MoE) diffusion transformer architecture. It scales to **17B total parameters** across 64 routed experts per layer while activating only **~2B parameters** per forward pass, establishing a new Pareto frontier in quality-versus-efficiency. Nucleus-Image matches or exceeds leading models
|
| 23 |
|
| 24 |
## Key Features
|
| 25 |
|
| 26 |
- **Sparse MoE efficiency**: 17B total capacity with only ~2B active parameters per forward pass, enabling high-quality generation at a fraction of the inference cost of dense models
|
| 27 |
- **Expert-Choice Routing**: Guarantees balanced expert utilization without auxiliary load-balancing losses, with a decoupled routing design that separates timestep-aware assignment from timestep-conditioned computation
|
| 28 |
-
- **Base model, no post-training**: This is a base model
|
| 29 |
- **Multi-aspect-ratio support**: Trained with aspect-ratio bucketing from the outset at every resolution stage, supporting a range of output dimensions
|
| 30 |
-
- **Text KV caching via diffusers**: Text tokens are excluded from the transformer backbone entirely and their KV projections are cached across all denoising steps. This caching is natively integrated into the `diffusers` pipeline
|
| 31 |
- **Progressive resolution training**: Three-stage curriculum (256 β 512 β 1024) with progressive sparsification of expert capacity
|
| 32 |
|
| 33 |
## Architecture
|
| 34 |
|
| 35 |

|
| 36 |
|
| 37 |
-
Nucleus-Image is a 32-layer diffusion transformer where 29 of the 32 blocks replace the dense FFN with a sparse MoE layer containing 64 routed experts and one shared expert (the first 3 layers use dense FFN for training stability). Image queries attend to concatenated image and text key-value pairs via joint attention
|
| 38 |
|
| 39 |
Routing uses **Expert-Choice** with a **decoupled design**: the router receives the unmodulated token representation concatenated with the timestep embedding, while expert MLPs receive the fully modulated representation. This prevents the adaptive modulation scale β which varies by an order of magnitude across timesteps β from collapsing expert selection into timestep-dependent routing, preserving spatial and semantic expert specialization.
|
| 40 |
|
|
@@ -117,11 +117,11 @@ image = pipe(
|
|
| 117 |
image.save("nucleus_output.png")
|
| 118 |
```
|
| 119 |
|
| 120 |
-
##
|
| 121 |
|
| 122 |
### Portraits & People
|
| 123 |
|
| 124 |
-
Nucleus-Image generations of human subjects and portraits, spanning diverse cultures, ages, and artistic styles
|
| 125 |
|
| 126 |

|
| 127 |

|
|
@@ -135,7 +135,7 @@ Nucleus-Image generations spanning fantasy, surrealism, animation, and the natur
|
|
| 135 |
|
| 136 |
### Commercial & Everyday Imagery
|
| 137 |
|
| 138 |
-
Nucleus-Image generations across product photography, architecture, typography, food, and world culture
|
| 139 |
|
| 140 |

|
| 141 |

|
|
|
|
| 19 |
|
| 20 |
## Introduction
|
| 21 |
|
| 22 |
+
**Nucleus-Image** is a text-to-image generation model built on a sparse mixture-of-experts (MoE) diffusion transformer architecture. It scales to **17B total parameters** across 64 routed experts per layer while activating only **~2B parameters** per forward pass, establishing a new Pareto frontier in quality-versus-efficiency. Nucleus-Image matches or exceeds leading models including Qwen-Image, GPT Image 1, Seedream 3.0, and Imagen4 on GenEval, DPG-Bench, and OneIG-Bench. This is a **base model** released without any post-training optimization (no DPO, no reinforcement learning, no human preference tuning). All reported results reflect pre-training performance only. We release the full model weights, training code, and dataset, making Nucleus-Image the first fully open-source MoE diffusion model at this quality tier.
|
| 23 |
|
| 24 |
## Key Features
|
| 25 |
|
| 26 |
- **Sparse MoE efficiency**: 17B total capacity with only ~2B active parameters per forward pass, enabling high-quality generation at a fraction of the inference cost of dense models
|
| 27 |
- **Expert-Choice Routing**: Guarantees balanced expert utilization without auxiliary load-balancing losses, with a decoupled routing design that separates timestep-aware assignment from timestep-conditioned computation
|
| 28 |
+
- **Base model, no post-training**: This is a base model. All benchmark results are from pre-training alone, without DPO, reinforcement learning, or human preference tuning
|
| 29 |
- **Multi-aspect-ratio support**: Trained with aspect-ratio bucketing from the outset at every resolution stage, supporting a range of output dimensions
|
| 30 |
+
- **Text KV caching via diffusers**: Text tokens are excluded from the transformer backbone entirely and their KV projections are cached across all denoising steps. This caching is natively integrated into the `diffusers` pipeline. Simply enable it with `TextKVCacheConfig` for automatic speedup with no code changes to the inference loop
|
| 31 |
- **Progressive resolution training**: Three-stage curriculum (256 β 512 β 1024) with progressive sparsification of expert capacity
|
| 32 |
|
| 33 |
## Architecture
|
| 34 |
|
| 35 |

|
| 36 |
|
| 37 |
+
Nucleus-Image is a 32-layer diffusion transformer where 29 of the 32 blocks replace the dense FFN with a sparse MoE layer containing 64 routed experts and one shared expert (the first 3 layers use dense FFN for training stability). Image queries attend to concatenated image and text key-value pairs via joint attention. Text tokens are excluded from the transformer backbone entirely, participating only as KV contributors. This eliminates MoE routing overhead for text and enables full text KV caching across denoising steps.
|
| 38 |
|
| 39 |
Routing uses **Expert-Choice** with a **decoupled design**: the router receives the unmodulated token representation concatenated with the timestep embedding, while expert MLPs receive the fully modulated representation. This prevents the adaptive modulation scale β which varies by an order of magnitude across timesteps β from collapsing expert selection into timestep-dependent routing, preserving spatial and semantic expert specialization.
|
| 40 |
|
|
|
|
| 117 |
image.save("nucleus_output.png")
|
| 118 |
```
|
| 119 |
|
| 120 |
+
## Highlights
|
| 121 |
|
| 122 |
### Portraits & People
|
| 123 |
|
| 124 |
+
Nucleus-Image generations of human subjects and portraits, spanning diverse cultures, ages, and artistic styles. From expressive character studies to fine-grained close-ups with intricate skin texture and detail.
|
| 125 |
|
| 126 |

|
| 127 |

|
|
|
|
| 135 |
|
| 136 |
### Commercial & Everyday Imagery
|
| 137 |
|
| 138 |
+
Nucleus-Image generations across product photography, architecture, typography, food, and world culture, demonstrating versatility in commercial, conceptual, and everyday imagery.
|
| 139 |
|
| 140 |

|
| 141 |

|