Toy-diffusion Ablation Results & Benchmark Analysis

Following the results presented in the i1 paper we conducted a series of architectural controlled experiments on our aipracticecafe/anime-faces-256px-v2 dataset (comprising 87,505 high-quality face crops) using Flux 2 VAE latents and using the Toy-diffusion training code.

Both Single and Dual stream dits completely remove the ADA-LN conditioning, instead the timestep is projected as a token and prepended to the text tokens. Training uses AdamW 8bit optimizer with warmup-constant learning rate of 4e-4 and a batch size of 256.

Below is the detailed benchmark performance FID-20K (Fréchet Inception Distance, ) across our evaluated checkpoints.

1. Unified Leaderboard of Evaluated Checkpoints

Checkpoint Name	Configuration & Parameters	Epoch	FID (↓)	Performance Summary
dual_stream_768_12_trans_2_bs_1024	Width: 768, Depth: 12, trans_2 Adapter, BS: 1024	1400	2.4240	Best Overall Configuration. Large-batch training combined with long training yields the best performance.
dual_stream_768_12_trans_2	Width: 768, Depth: 12, trans_2 Adapter, BS: 512	600	3.0635	Original baseline.
dual_stream_768_12_llm_te_log_normal	Width: 768, Depth: 12, LLM Text Encoder, Log-Normal	1000	3.1219	Good prompt following which makes the model prone to instabilities for poorly represented tags.
dual_stream_768_12_modern_cfm	Width: 768, Depth: 12, trans_2 Adapter, Contrastive FM	700	3.2707	Modern text adapter and CFM doesn't seem to provide improvements overbaseline as wall time is similar.
efficient_unet_bs_256	SnapGen Conv-Attn Hybrid, BS: 256	400	3.4479	CNN-Transformer hybrid. Fast early-stage convergence at the cost of more than double wall time training and more vram consumption.
single_stream_768_14_trans_2	Width: 768, Depth: 14, Single-Stream Backbone, BS: 512	600	3.8954	Slower training throughput. Struggles to drop below FID 3.7.
sprint_dual_stream_768_13_trans_2_fined	SPRINT (Sparse), Finetuned Dense on Full Sequence	1000	3.9647	Recovers remarkably well during dense finetuning, preserving visual fidelity while cutting pretraining costs.
dual_stream_768_16_mlp	Width: 768, Depth: 16, simple MLP text adapter	400	4.2485	Bottlenecked heavily by the weak MLP text adapter.
sprint_single_stream_768_14_fined	SPRINT Single-Stream, Finetuned Dense	1000	7.3793	Does not recover completely from sparse training, lagging significantly behind dual-stream variants.
sprint_dual_stream_768_13_trans_2	SPRINT (Sparse), No Finetuning (75% drop ratio)	700	13.8281	High sparse degradation. Evaluating on dense inputs without dense finetuning is not recommended.
sprint_single_stream_768_14_trans_img_2_rope_text	SPRINT (Sparse), No Finetuning (75% drop ratio)	750	9.7705	Lower fid than dual-stream version.

---

Architectural Insights

Text adapter

Our results provide clear, empirical validation of the importantece of a strong text encoder adapter:

• Comparing dual_stream_768_16_mlp (depth 16, FID 4.57 at 500 epochs) to dual_stream_768_12_trans_2 (depth 12, FID 3.54 at 500 epochs) reveals a clear pattern. Investing capacity into a 2-layer Transformer Text Adapter (trans_2) rather than raw MLP produces a massive visual upgrade.

B. Single-Stream vs. Dual-Stream Backbones

• Training Efficiency: Dual-stream blocks completed training epochs in ~3.5 minutes, whereas Single-stream blocks required ~5.1 minutes—under identical VRAM constraints (~23GB). By processing text and image tokens independently, Dual-stream models seem to be more hardware-friendly.
• Visual Fidelity: Dual-stream models drop FID significantly faster and settle at a much lower ultimate floor (FID 3.06 vs. Single-stream's 3.69 plateau).
• The Overfitting & Cosplay Tradeoff: We observed a distinct qualitative divergence:
  • Dual-stream models are prone to localized concept overfitting (e.g., struggling to render Asuka in Misato's uniform, consistently over-associating characters with their native outfits).
  • Single-stream models handle complex conditional blending and "cosplay" prompts with far better zero-shot composition, indicating stronger cross-modality token integration.

C. SPRINT: Efficient Sparse Pre-Training

• The sparse token-dropping framework (SPRINT) with a 75% drop ratio achieved a ~50% reduction in training time and lowered active GPU memory requirements from 23GB to 17GB. Even with the Finetuning stage, the overall training wall time is shorter than the dense version.
• Finetuning recovers performance: Evaluating SPRINT models directly results in highly degraded FID scores (FID ~13.8) due to the dense-sparse distribution shift. However, following the dense finetuning recipe (_fined), the model recovers rapidly, achieving an FID of 3.96 by epoch 1000.
• Dual Stream is favored: Despite the good FID scores exhibited by the Single stream architecture, during the finetuning stage only the dual stream architecture is able to recover the high performance.

Developed by: aipracticecafe

Content Warning & Rights

• NSFW Content: This dataset contains sensitive and NSFW (Not Safe For Work) material. It is intended for research and generative modeling purposes.
• Legal Disclaimer: I do not own the rights to any of the images in this dataset. All images are the property of their respective creators and were scraped from Danbooru. This dataset is provided for educational and research purposes under fair use.