Academic rewrite: full Method section (equations + notation) + pipeline figure

README.md

@@ -22,8 +22,10 @@ maximum likelihood** — the marginal over *all* derivation trees, computed by a
 inside dynamic program (log-semiring DP). No noise process, no adversary, no
 ELBO, no token ordering.
 
-The current release is **v0.1 at 64×64**: a proof-of-concept for the mechanism. ~16M trainable
-parameters on top of a frozen T5-base caption encoder.
+The current release is **v0.1 at 64×64**: a proof-of-concept for the mechanism.
+~16M trainable parameters on top of a frozen T5-base caption encoder.
+
+<p align="center"><img src="figures/pipeline.png" width="820" alt="SPRIG pipeline: caption → grammar modulation → top-down derivation → texel rendering → image"></p>
 
 <p align="center"><img src="samples.jpg" width="360" alt="SPRIG v0.1 samples"></p>
 
@@ -40,7 +42,157 @@ Because analysis and synthesis are the *same* grammar run in two directions, the
 model can also **parse** a real image (infer its most likely derivation) — see
 `parses.png`. This is the strongest, most novel capability and it works well.
 
-## Scorecard (v0.1, 50k steps, held-out procedural scenes)
+## Method
+
+### Formal object
+
+SPRIG defines the conditional image distribution as a **text-modulated
+probabilistic context-free scene grammar** $G_c = (\Sigma, N, A_0, \Pi_c)$ whose
+derivations tile the canvas. $N$ is a set of $S$ *nonterminal* symbols (the
+"syntactic categories of vision"), $\Sigma$ is a set of $V$ *terminal* symbols
+(*texels*, i.e. learned material primitives), $A_0 \in N$ is the axiom, and
+$\Pi_c$ is a caption-conditioned set of production probabilities. The conditional
+density marginalizes over every derivation tree $\tau$:
+
+$$
+p(x \mid c) \;=\; \sum_{\tau \in \mathcal{T}(G_c)}\;
+\underbrace{\prod_{(A \to \langle s, B, C\rangle,\, r)\, \in\, \tau}
+\pi\big(A \to \langle s, B, C\rangle \mid c, r\big)}_{\text{internal productions (splits)}}
+\;\;
+\underbrace{\prod_{(A \to T,\, r)\, \in\, \mathrm{leaves}(\tau)}
+\pi\big(A \to T \mid c, r\big)\; p_{\mathrm{emit}}(x_r \mid T, r, c)}_{\text{leaf productions (emission)}}
+$$
+
+The single latent object is the **derivation tree** $\tau$: a variable-topology
+binary tree whose internal nodes are (nonterminal, region) pairs and whose leaves
+are (texel, region) pairs. There is no time index, no corruption path, and no
+fixed-dimensional state that gets revised — every generation step *creates* new
+latent variables (a split choice $s$, two child symbols $B, C$, two child
+regions) and never revisits old ones. Generation randomness is exactly the
+sequence of categorical production choices.
+
+### Derivation and the region lattice
+
+A production $A \to \langle s, B, C\rangle$ applied at region $r$ chooses a split
+$s$ from a finite **cut dictionary** and partitions $r$ into two child regions
+$r_1(s), r_2(s)$, which receive child symbols $B, C$. Regions live on a fixed
+**binary-space-partition (BSP) lattice**: at 64×64 on an 8-px grid, every
+axis-aligned rectangle with grid-aligned corners is a region ($|\mathcal{R}| =
+1296$), and each region has $14$ cut types (2 axes × 7 relative-offset buckets).
+A region is summarized for conditioning by a deterministic geometry code
+$\rho(r)$ (Fourier features of position, size, aspect) — *not* a sampled
+quantity, so subtrees remain conditionally independent given (symbol, region) and
+the sum over trees stays a valid sum-product.
+
+Finiteness is deliberate: a *finite* cut dictionary is what makes the region
+lattice finite and the marginal likelihood **exactly** computable. Continuous
+split parameters would break tractability (they are a target for later versions).
+
+### Text-modulated productions
+
+The load-bearing design decision is that **text deforms the grammar itself, it
+does not steer a sampler.** The internal production tensor is factorized in a
+low-rank (tensor-decomposition / TN-PCFG) form with $R$ rule components:
+
+$$
+\pi\big(A \to \langle s, B, C\rangle \mid c\big) \;=\;
+\sum_{k=1}^{R} \; p(k \mid A, c)\; p(s \mid k, c)\; p(B \mid k)\; p(C \mid k)
+$$
+
+Here $p(B \mid k), p(C \mid k)$ are static softmax rows of learned tables
+$V, W \in \mathbb{R}^{R \times S}$; $p(s \mid k, c)$ comes from rule-component
+embeddings that cross-attend once to the caption; and the **only** text-dependent
+factor is the parent mixture $p(k \mid A, c)$, produced by a **Grammar-Modulation
+Transformer (GMT)** whose *queries are the $S$ symbol embeddings* and whose
+keys/values are the frozen caption tokens. Because $p(k\mid A,c)$ is computed once
+per caption (a single $S \times R$ matrix), every production probability anywhere
+in the tree is thereafter a cheap contraction — the caption reweights *which
+rewrites are probable* rather than pushing a fixed state along a trajectory. The
+terminal (texel) prior shares this mixture, $p(T \mid A, c) = \sum_k p(k \mid A,
+c)\,\mathrm{softmax}(P_T[k])$.
+
+### Area-conditioned termination
+
+Each symbol carries a **termination head** that decides expand-vs-emit at region
+$r$:
+
+$$
+\pi(\mathrm{term} \mid A, c, r) \;=\; \sigma\!\Big( f_\theta\big(E_N[A], \rho(r)\big) \;+\; \gamma \cdot \log \tfrac{a_{\min}}{\mathrm{area}(r)} \Big)
+$$
+
+The explicit area term (learned gain $\gamma > 0$) makes termination more likely
+as regions shrink. Combined with a hard floor (regions $\le 8\times8$ must
+terminate) and a leaf ceiling ($\le 16\times16$), this **guarantees almost-sure
+termination** of every derivation at bounded depth — the grammar cannot run away,
+unlike a general PCFG.
+
+### Terminal emission
+
+A leaf $(T, r)$ is rendered by a shared, deliberately **weak** decoder that maps
+the texel embedding, geometry $\rho(r)$, and caption to per-pixel
+discretized-logistic mixture parameters ($K = 4$ components), fully factorized
+over pixels *given the texel*:
+
+$$
+p_{\mathrm{emit}}(x_r \mid T, r, c) \;=\; \prod_{p \in r} \sum_{j=1}^{4} \lambda_{p,j}\;\mathrm{DLogistic}\big(x_p; \mu_{p,j}, \sigma_{p,j}\big)
+$$
+
+A caption **illumination field** $\Phi(c)$ FiLM-modulates the means so global tone
+can cross region boundaries. The decoder is kept narrow on purpose: if a leaf
+could model arbitrary $16\times16$ crops unconditionally, maximum likelihood would
+let the grammar collapse to a vestigial uniform splitter. Capacity starvation at
+the leaves *forces* explanatory power up into the tree.
+
+### Exact likelihood via the inside DP
+
+Because the lattice and cut dictionary are finite, the marginal over all trees is
+computed **exactly** by a CYK-style inside pass in the log-semiring, bottom-up in
+area order over the region lattice:
+
+$$
+\beta(A, r) = \mathrm{logaddexp}\Big[\;
+\underbrace{\log \pi(\mathrm{term}\mid A,c,r) + \mathrm{logsumexp}_{T}\big(\log \pi(T\mid A,c) + \log p_{\mathrm{emit}}(x_r \mid T, r, c)\big)}_{\text{terminate}},\;\;
+\underbrace{\mathrm{logsumexp}_{s,B,C}\big(\log \pi(A{\to}\langle s,B,C\rangle\mid c) + \beta(B, r_1(s)) + \beta(C, r_2(s))\big)}_{\text{expand}}
+\;\Big]
+$$
+
+The low-rank factorization turns the $S \times S \times S$ contraction at each
+(region, split) into a pair of $S \times R$ matmuls in a numerically stabilized
+exp domain. The training loss is the **exact** conditional negative log-likelihood
+
+$$
+\mathcal{L}(x, c) \;=\; -\,\beta(A_0, \text{canvas}),
+$$
+
+and a single smooth backward pass through the DP reaches every parameter. There is
+no encoder, no ELBO gap, and no sampling in the training loop. The *same* DP with
+a max-semiring yields the **Viterbi parse** of any image — this is why analysis
+and synthesis are the same object, and why SPRIG can explain an image as well as
+draw one.
+
+### Configuration (v0.1)
+
+| Symbol | Meaning | v0.1 value |
+|---|---|---|
+| $S$ | nonterminal symbols | 1024 |
+| $V = T_v$ | terminal symbols (texels) | 256 |
+| $R$ | rule-mixture components | 64 |
+| $d$ | model width | 384 |
+| — | canvas / grid | 64×64 / 8 px |
+| $\lvert\mathcal{R}\rvert$ | BSP region lattice | 1296 |
+| — | cut types | 14 (2 axes × 7 offsets) |
+| — | leaf cap / hard-terminate | 16×16 / 8×8 |
+| — | caption encoder (frozen) | T5-base (768-d) |
+| — | emission | 4-comp discretized logistic |
+| — | trainable params | ~15.9M |
+
+Notation: $x$ image, $c$ caption, $\tau$ derivation tree, $A,B,C \in N$
+nonterminals, $T \in \Sigma$ texel, $s$ split, $r$ region, $\rho(r)$ geometry
+code, $\beta$ inside score, $A_0$ axiom, $E_N$ symbol embedding table.
+
+## Results
+
+Success criteria were fixed in advance (50k steps, held-out procedural scenes):
 
 | Gate | Target | Result | |
 |---|---|---|---|
@@ -54,12 +206,12 @@ model can also **parse** a real image (infer its most likely derivation) — see
 | Compositional holdout (unseen combos) | ≥ 0.60 | 0.01 | ❌ |
 | Grammar health (S_eff / alive texels) | ≥256 / ≥50% | 968 / 43% | ⚠️ texels over-pruned |
 
-The architecture's structural claims prove out: it models
-data far better than a no-grammar baseline, routes caption information, recovers
-scene structure by parsing, and (after a targeted fix) paints real objects. The
-open problem is **caption→object binding**: the model can draw objects and binds
-size perfectly, but does not yet reliably paint the *specific* object a prompt
-asks for, and places too many per scene. 
+The architecture's structural claims prove out: it models data far better than a
+no-grammar baseline, routes caption information, recovers scene structure by
+parsing, and (after a targeted fix) paints real objects. The open problem is
+**caption→object binding**: the model can draw objects and binds size perfectly,
+but does not yet reliably paint the *specific* object a prompt asks for, and places
+too many per scene.
 
 ## Usage
 
@@ -85,6 +237,7 @@ was drawn.
 - `config.json` — architecture config + release metadata
 - `inference.py` — minimal load + sample + T5 caption encoding
 - `metrics.json` — full evaluation numbers
+- `figures/pipeline.svg`, `figures/pipeline.png` — the method schematic
 - `samples.jpg`, `texel_atlas.png`, `parses.png` — qualitative outputs
 - `DESIGN.md` — the concrete v0.1 architecture specification