Upload architecture.md

architecture.md

@@ -0,0 +1,132 @@
+# ARTIGEN V1.0 — Novel Architecture Specification
+
+## Overview
+**ARTIGEN** (Adaptive Reasoning-based Token-Informed Generative ENgine) is a lightweight, modular image generation architecture designed for anime/illustration generation on mobile devices (<3GB RAM). It combines insights from DiM (Diffusion Mamba), Zigzag Mamba, RWKV diffusion, MobileMamba, unified latent reasoning (MILR, Unified Thinker), flow matching in latent space (LFM), disentangled modular representation learning, and liquid time-constant dynamics.
+
+## Core Innovations
+1. **CARTEL** (CAscading Reasoning Token Ensemble Layer) — Modular latent reasoning backbone using hybrid SSM+RWKV blocks
+2. **PHI-SCAN** (Physics-Informed multi-directional scan) — No parameter scanning scheme preserving 2D geometric continuity
+3. **Art-Style Disentangled Latent Space (ASDL)** — Modular decomposition of style/content/concept/mood in latent vectors
+4. **Flow-Matching in Wavelet Space** — Training using rectified flow matching with Haar wavelet decomposition
+5. **Progressive Modular Curriculum** — Freeze/thaw training stages forcing specialization
+
+## Mathematical Formulation
+
+### 1. Pretrained VAE (Frozen)
+Use `madebyollin/taesd` (1M params) or `stabilityai/sd-vae-ft-mse` for latent encoding. Image x → z ∈ R^(h×w×c) where h=w=32, c=4 for 1024px images.
+
+### 2. CARTEL Backbone
+Each CARTEL block consists of:
+- **SSM branch**: Bidirectional selective state space model (Mamba-style)
+- **RWKV branch**: Linear attention with spatial shift
+- **LTC residual**: Liquid Time-Constant gate for adaptive information routing
+
+### 3. PHI-SCAN Pattern
+Physics-informed zigzag scan minimizing geodesic distance distortion when flattening 2D→1D:
+- Uses Hilbert curve ordering (preserves locality)
+- Alternates between row-major, column-major, and diagonal scans per layer
+- Zero additional parameters (permutation matrices are fixed)
+
+### 4. ASDL Latent Space
+The model learns to decompose latent representations into:
+- **Style vector**: s ∈ R^d_s — artistic style (colored pencil, watercolor, anime, etc.)
+- **Content vector**: c ∈ R^d_c — semantic content (characters, objects, scenes)
+- **Concept vector**: n ∈ R^d_n — abstract concept relationships
+- **Mood vector**: m ∈ R^d_m — emotional tone, atmosphere, lighting feel
+- **Composition vector**: p ∈ R^d_p — spatial layout, perspective, framing
+
+### 5. Training: Flow Matching with Auxiliary Reasoning Losses
+Instead of diffusion, use flow matching (Rectified Flow) in latent space, more stable and faster converging for small models.
+
+Loss function:
+L = λ_flow * L_flow + λ_concept * L_concept + λ_style * L_style + λ_mood * L_mood + λ_consistency * L_VAE_recon + λ_physics * L_spectral_smoothness
+
+where L_flow is the standard flow-matching loss predicting velocity field v_t(z_t).
+
+## Architecture Diagram (Simplified)
+
+```
+Text Prompt → CLIP/Text Encoder → Semantic Embedding φ_t
+Timestep t → Time Embedding φ_t
+Image Latent z_t → PHI-SCAN → [CARTEL Block × N] → Pred z_0 / v_t(z_t)
+Long skip connections (U-Net style)
+```
+
+## Training Stages (Progressive Modular)
+
+### Stage 1: Foundation + Style Learning
+Dataset: High-quality anime illustrations with strong style tags
+Objective: Learn ASDL style vectors. Style head predicts s from z.
+Loss: L_style + L_flow + L_VAE_recon
+Duration: ~50K steps
+
+### Stage 2: Content Understanding
+Dataset: Detailed caption dataset (e.g., Danbooru captions)
+Freeze style head, train content head.
+Objective: Content head predicts c from text embedding.
+Loss: L_concept + cross-modal alignment
+
+### Stage 3: Concept & Logic
+Dataset: Reasoning datasets + attribute counting + spatial relationship data
+Train concept head with structured reasoning traces.
+
+### Stage 4: Mood & Philosophy
+Dataset: Art with mood labels, emotional analysis data
+Train mood head to predict m from latent z.
+
+### Stage 5: Full End-to-End
+Unfreeze entire model, train with all losses balanced.
+Use classifier-free guidance during training (10% dropout of text).
+
+## Key Design Decisions
+
+1. **Why Flow Matching over Diffusion?**
+   - Fewer function evaluations for sampling (fewer steps = faster mobile inference)
+   - More stable training dynamics (no exploding loss from t→0 in DDPM)
+   - Compatible with rectified flow for 1-4 step generation
+
+2. **Why SSM+RWKV over Transformers?**
+   - Linear complexity in sequence length: O(N) vs O(N²)
+   - For 32×32 latent = 1024 tokens, transformer attention needs ~1M attention ops per layer
+   - SSM needs ~1024 state updates = 1000x fewer ops
+   - Memory usage: ~50MB vs ~2GB for equivalent context
+
+3. **Why Modular Heads?**
+   - Forces disentanglement in latent space
+   - Enables controllable generation: user can tweak style vectors post-training
+   - Prevents catastrophic forgetting during curriculum
+   - Each module can be trained/fine-tuned independently
+
+4. **Why LTC (Liquid Time-Constant) residual?**
+   - Adapts effective receptive field dynamically
+   - For structured content (faces, buildings): slower dynamics (larger time constant)
+   - For texture/noise: faster dynamics (smaller time constant)
+   - Adds only 2 extra parameters per channel
+
+5. **Why Wavelet Flow Matching?**
+   - Haar wavelet decomposes image into frequency bands
+   - Different CARTEL blocks can specialize on different frequencies
+   - LL subband: global structure (handled by deep SSM)
+   - LH/HL/HH: edges and textures (handled by RWKV with high-frequency bias)
+
+## Implementation Notes
+
+- Total model size target: **<150M parameters** (fits in ~600MB fp16, ~300MB int8)
+- With frozen VAE (~50M) + CARTEL backbone (~80M) + modular heads (~20M) = ~150M
+- In fp16: ~300MB RAM
+- In int8: ~150MB RAM
+- Plus overhead for KV cache/sampling: <3GB total on device
+
+## References (Inspiration Sources)
+
+1. DiM (2405.14224): SSM for diffusion, multi-directional scan, skip connections
+2. Zigzag Mamba (2403.13802): Zigzag scanning for spatial continuity
+3. Diffusion-RWKV (2404.04478): RWKV for image generation, adaLN conditioning
+4. MobileMamba (2411.15941): Three-stage network, wavelet-enhanced, multi-receptive field
+5. MILR (2509.22761): Test-time latent reasoning, policy gradients in unified space
+6. Unified Thinker (2601.03127): Reasoning-decoupled generation, task-agnostic reasoning core
+7. LatentMorph (2602.02227): Implicit latent reasoning without decode-encode loops
+8. LFM / Flow Matching in Latent Space (2307.08698): Flow matching with pretrained VAE
+9. Liquid Time-Constant Networks (2006.04439): Continuous-time adaptive dynamics
+10. Disentanglement via Latent Quantization (2305.18378): Modular latent decomposition
+11. Modular Deep Learning (2302.11529): Parameter-efficient routing modules