cebeuq commited on
Commit
571246f
·
verified ·
1 Parent(s): d05497a

Math rendering: inline uses HF KaTeX \\( \\); display $$ on own lines; no paren-adjacent delimiters

Browse files
Files changed (1) hide show
  1. README.md +7 -5
README.md CHANGED
@@ -44,15 +44,17 @@ model can also **parse** a real image (infer its most likely derivation) — see
44
 
45
  ## Method
46
 
47
- SPRIG is a **text-modulated probabilistic scene grammar** $G_c=(\Sigma, N, A_0, \Pi_c)$: nonterminal symbols $N$, texels (learned material primitives) $\Sigma$, an axiom $A_0$, and caption-conditioned productions $\Pi_c$. An image is one **derivation** $\tau$ — a binary tree that recursively splits the 64×64 canvas (a finite 1296-region binary-space-partition lattice, leaves ≤16px) and paints each leaf region with a texel. The conditional density marginalizes over *all* derivation trees:
48
 
49
- $$ p(x \mid c) = \sum_{\tau} \; \prod_{\text{splits}} \pi\big(A \to \langle s,B,C\rangle \mid c\big) \; \prod_{\text{leaves}} \pi(T \mid A, c)\, p_{\mathrm{emit}}(x_r \mid T, r, c) $$
 
 
50
 
51
- Text enters through a low-rank rule factorization $\pi(A \to \langle s,B,C\rangle \mid c) = \sum_k p(k \mid A, c)\, p(s \mid k, c)\, p(B \mid k)\, p(C \mid k)$, whose only caption-dependent factor — the mixture $p(k \mid A, c)$ — is produced by a **Grammar-Modulation Transformer** (queries = symbol embeddings, cross-attention to the frozen T5-base caption). *Text deforms the grammar; it does not steer a sampler.* Each leaf emits a 4-component discretized-logistic mixture over its pixels.
52
 
53
- Because the lattice and the cut dictionary are finite, the marginal is computed **exactly** by a log-semiring inside dynamic program over regions, and the training loss is the exact negative log-likelihood $\mathcal{L} = -\beta(A_0, \text{canvas})$ — no encoder, no ELBO, no sampling in the loop. The *same* DP with a max-semiring yields the **Viterbi parse** of any image, which is why analysis and synthesis are the same object.
54
 
55
- | $S$ | $T_v$ | $R$ | $d$ | canvas / grid | lattice | encoder | params |
56
  |---|---|---|---|---|---|---|---|
57
  | 1024 | 256 | 64 | 384 | 64² / 8px | 1296 regions | T5-base (frozen) | ~15.9M |
58
 
 
44
 
45
  ## Method
46
 
47
+ SPRIG is a **text-modulated probabilistic scene grammar** \\(G_c=(\Sigma, N, A_0, \Pi_c)\\): nonterminal symbols \\(N\\), texels (learned material primitives) \\(\Sigma\\), an axiom \\(A_0\\), and caption-conditioned productions \\(\Pi_c\\). An image is one **derivation** \\(\tau\\) — a binary tree that recursively splits the 64×64 canvas (a finite 1296-region binary-space-partition lattice, leaves ≤16px) and paints each leaf region with a texel. The conditional density marginalizes over *all* derivation trees:
48
 
49
+ $$
50
+ p(x \mid c) = \sum_{\tau} \; \prod_{\text{splits}} \pi\big(A \to \langle s,B,C\rangle \mid c\big) \; \prod_{\text{leaves}} \pi(T \mid A, c)\, p_{\mathrm{emit}}(x_r \mid T, r, c)
51
+ $$
52
 
53
+ Text enters through a low-rank rule factorization \\(\pi(A \to \langle s,B,C\rangle \mid c) = \sum_k p(k \mid A, c)\, p(s \mid k, c)\, p(B \mid k)\, p(C \mid k)\\), whose only caption-dependent factor — the mixture \\(p(k \mid A, c)\\) — is produced by a **Grammar-Modulation Transformer** (queries = symbol embeddings, cross-attention to the frozen T5-base caption). *Text deforms the grammar; it does not steer a sampler.* Each leaf emits a 4-component discretized-logistic mixture over its pixels.
54
 
55
+ Because the lattice and the cut dictionary are finite, the marginal is computed **exactly** by a log-semiring inside dynamic program over regions, and the training loss is the exact negative log-likelihood \\(\mathcal{L} = -\beta(A_0, \text{canvas})\\) — no encoder, no ELBO, no sampling in the loop. The *same* DP with a max-semiring yields the **Viterbi parse** of any image, which is why analysis and synthesis are the same object.
56
 
57
+ | \\(S\\) | \\(T_v\\) | \\(R\\) | \\(d\\) | canvas / grid | lattice | encoder | params |
58
  |---|---|---|---|---|---|---|---|
59
  | 1024 | 256 | 64 | 384 | 64² / 8px | 1296 regions | T5-base (frozen) | ~15.9M |
60