update

app/.astro/astro/content.d.ts

@@ -209,12 +209,12 @@ declare module 'astro:content' {
   data: any
 } & { render(): Render[".mdx"] };
 };
-"embeds": {
+"embeds2": {
 "vibe-code-d3-embeds-directives.md": {
 	id: "vibe-code-d3-embeds-directives.md";
   slug: "vibe-code-d3-embeds-directives";
   body: string;
-  collection: "embeds";
+  collection: "embeds2";
   data: any
 } & { render(): Render[".md"] };
 };

app/scripts/latex-to-mdx/filters/equation-ids.lua

@@ -69,9 +69,10 @@ function Math(el)
         
         local new_math
         if has_align then
-            -- For align environments, add the ID differently - KaTeX doesn't support \htmlId with align
-            -- Instead, we'll add a span with the ID right before the align
-            new_math = clean_math
+            -- For align environments, KaTeX doesn't support \htmlId with align
+            -- Instead, we add a special marker that the post-processor will convert to a span
+            -- This span will serve as an anchor for references
+            new_math = "%%ALIGN_ANCHOR_ID{" .. clean_id .. "}%%\n" .. clean_math
         else
             -- For other math, wrap with \htmlId{}
             new_math = "\\htmlId{" .. clean_id .. "}{" .. clean_math .. "}"

app/scripts/latex-to-mdx/output/main.md

@@ -248,6 +248,8 @@ Deriving the end-effector’s *pose*--position *and* orientation--in some $`m`$-
 
 In the simplified case here considered (for which $`\boldsymbol{p} \equiv p`$, as the orientation of the end-effector is disregarded for simplicity), one can solve the problem of controlling the end-effector’s location to reach a goal position $`p^*`$ by solving analytically for $`q: p(q) = f_{\text{FK}}(q) = p^*`$. However, in the general case, one might not be able to solve this problem analytically, and can typically resort to iterative optimization methods comparing candidate solutions using a loss function (in the simplest case, $`\Vert p(q) - p^* \Vert_2^2`$ is a natural candidate), yielding:
 
+<span id="ik_problem" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
 \min_{q \in \mathcal Q} \Vert p(q) - p^* \Vert_2^2 \, .
@@ -262,6 +264,8 @@ For instance, the robot in Figure <a href="#planar-manipulator-floor" data-refe
 However, IK--solving eq. <a href="#ik_problem" data-reference-type="ref" data-reference="ik_problem">[ik_problem]</a> for a feasible $`q`$--only proves useful in determining information regarding the robot’s configuration in the goal pose, and crucially does not provide information on the *trajectory* to follow over time to reach a target pose. Expert-defined trajectories obviate to this problem providing a length-$`K`$ succession of goal poses $`\tau_K = [p^*_0, p^*_1, \dots p^*_K]`$ for tracking. In practice, trajectories can also be obtained automatically through *motion planning* algorithms, thus avoiding expensive trajectory definition from human experts. However, tracking $`\tau_K`$ via IK can prove prohibitively expensive, as tracking would require $`K`$ resolutions of eq. <a href="#ik_problem" data-reference-type="ref" data-reference="ik_problem">[ik_problem]</a> (one for each target pose). *Differential* inverse kinematics (diff-IK) complements IK via closed-form solution of a variant of eq. <a href="#ik_problem" data-reference-type="ref" data-reference="ik_problem">[ik_problem]</a>. Let $`J(q)`$ denote the Jacobian matrix of (partial) derivatives of the FK-function $`f_\text{FK}- \mathcal Q \mapsto \mathcal P`$, such that $`J(q) = \frac{\partial f_{FK}(q)}{\partial q }`$. Then, one can apply the chain rule to any $`p(q) = f_{\text{FK}}(q)`$, deriving $`\dot p = J(q) \dot q`$, and thus finally relating variations in the robot configurations to variations in pose, thereby providing a platform for control.
 
 Given a desired end-effector trajectory $`\dot {p}^*(t)`$ (1) indicating anchor regions in space and (2) how much time to spend in each region, diff-IK finds $`\dot q(t)`$ solving for joints’ *velocities* instead of *configurations*,
+<span id="reg_ik_velocity" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
 \dot q(t) = \arg\min_\nu \; \lVert J(q(t)) \nu - \dot {p}^*(t) \rVert_2^2
@@ -390,6 +394,8 @@ A length-$`T`$ *trajectory* is the (random) sequence
 \htmlId{trajectory_definition}{\tau = (s_0, a_0, r_0, s_1, a_1, r_1, \dots, s_{T-1}, a_{T-1}, r_{T-1}, s_T),}
 ```
 with per-step rewards defined as $`r_t = r (s_t, a_t, s_{t+1})`$ for ease of notation.Interestingly, assuming both the environment dynamics and conditional distribution over actions given states--the *policy*--to be *Markovian*:
+<span id="dynamics_markovian" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
 \mathbb P(s_{t+1}\vert s_t, a_t, s_{t-1}, a_{t-1}, \dots s_0, a_0 ) &= \mathbb P (s_{t+1}\vert s_t, a_t) \\
@@ -406,6 +412,8 @@ Policies $`\mathbb P(a_t\vert s_t)`$ are typically indicated as $`\pi(a_t\vert s
 G(\tau) = \sum_{t=0}^{T-1} \gamma^{t} r_t.
 ```
 In that, agents seek to learn control strategies (*policies*, $`\pi_\theta`$) maximizing the expected return $`\mathbb E_{\tau \sim \pi_\theta} G(\tau)`$. For a given dynamics $`\mathcal D`$--i.e., for a given problem--taking the expectation over the (possibly random) trajectories resulting from acting according to a certain policy provides a direct, goal-conditioned ordering in the space of all the possible policies $`\Pi`$, yielding the (maximization) target $`J : \Pi \mapsto \mathbb R`$
+<span id="RL-j-function" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
     J(\pi_\theta) &= \mathbb E_{\tau \sim \mathbb P_{\theta; \mathcal D}} [G(\tau)],  \\
@@ -422,6 +430,8 @@ can be used to discriminate between desirable and undesirable state in terms of
 Q_\pi(s,a) = \mathbb E_{\tau \sim \pi} [G (\tau) \big \vert s_0 = s, a_0=a]
 ```
 Crucially, value functions are interrelated:
+<span id="q-as-v" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
 Q_\pi(s_t, a_t) &= \mathbb{E}_{s_{t+1}\sim \mathbb P(\bullet \vert s_t, a_t)} [r_t + \gamma V_\pi(s_{t+1})]  \\
@@ -493,6 +503,8 @@ Q_{i+1}(s_t, a_t) \leftarrow \mathbb E_{s_{t+1} \sim \mathbb P(\bullet \vert s_t
 Then, one can derive the (ideally, near-optimal) policy by explicitly maximizing over the action space the final (ideally, near-optimal) estimate $`Q_K \approx Q^*`$ at each timestep. In fact, under certain assumptions on the MDP considered, $`Q_K \to Q^* \, \text{as } K \to \infty`$.
 
 Effective in its early applications to small-scale discrete problems and theoretically sound, vanilla Q-learning was found complicated to scale to large $`\mathcal S\times \mathcal A`$ problems, in which the storing of $`Q : \mathcal S\times \mathcal A\mapsto \mathbb R`$ alone might result prohibitive. Also, vanilla Q-learning is not directly usable for *continuous*, unstructured state-action space MPDs, such as those considered in robotics. In their seminal work on *Deep Q-Learning* (DQN), @mnihPlayingAtariDeep2013 propose learning Q-values using deep convolutional neural networks, thereby accomodating for large and even unstructured *state* spaces. DQN parametrizes the Q-function using a neural network with parameters $`\theta`$, updating the parameters by sequentially minimizing the expected squared temporal-difference error (TD-error, $`\delta_i`$):
+<span id="dqn-loss" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
 \mathcal L(\theta_i) &= \mathbb E_{(s_t, a_t) \sim \chi(\bullet)} 
@@ -515,6 +527,8 @@ Provably, <a href="#deterministic-pg" data-reference-type="ref" data-reference="
 Similarily to DQN, DDPG also employs the same replay buffer mechanism, to reuse past transitions over training for increased sample efficiency and estimate the loss function via MC-estimates.
 
 Soft Actor-Critic (SAC) @haarnojaSoftActorCriticOffPolicy2018 is a derivation of DDPG in the max-entropy (MaxEnt) RL framework, in which RL agents are tasked with \<span class="highlight"\>maximizing the discounted cumulative reward, while acting as randomly as possible\</span\>. MaxEnt RL @haarnojaReinforcementLearningDeep2017 has proven particularly robust thanks to the development of diverse behaviors, incentivized by its entropy-regularization formulation. In that, MaxEnt revisits the RL objective $`J (\pi)`$ to specifically account for the policy entropy,
+<span id="J-soft" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
     J(\pi) &= \sum_{t=0}^T \mathbb{E}_{(s_t, a_t) \sim \chi} [r_t + \alpha \mathcal H(\pi (\bullet \vert s_t))] 
@@ -643,6 +657,8 @@ Intuitively, in the case of observation-action pairs $`(o, a)`$ for a robotics a
 </figure>
 
 Given a dataset $`\mathcal D`$ consisting of $`N`$ i.i.d. observation-action pairs, the log-likelihood of all datapoints under $`\theta`$ (in Bayesian terms, the *evidence* $`p_\theta(\mathcal D)`$) can thus be written as:
+<span id="evidence-definition-1" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
     \log p_\theta(\mathcal D) &= \log \sum_{i=0}^N p_\theta ((o,a)_i) \\
@@ -658,6 +674,8 @@ In the special case where one assumes distributions to be tractable, $`p_\theta
 In their seminal work on Variational Auto-Encoders (VAEs), @kingmaAutoEncodingVariationalBayes2022 present two major contributions to learn complex latent-variable GMs on unstructured data, proposing (1) a tractable, variational lower-bound to <a href="#evidence-definition" data-reference-type="ref" data-reference="evidence-definition">[evidence-definition]</a> as an optimization target to jointly learn likelihood and posterior and (2) high-capacity function approximators to model the likelihood $`p_\theta(o,a\vert z)`$ and (approximate) posterior distribution $`q_\phi(z \vert o,a) \approx q_\theta(z \vert o,a)`$.
 
 In particular, the lower bound on <a href="#evidence-definition" data-reference-type="ref" data-reference="evidence-definition">[evidence-definition]</a> (Evidence LOwer Bound, *ELBO*) can be derived from <a href="#evidence-definition" data-reference-type="ref" data-reference="evidence-definition">[evidence-definition]</a> applying Jensen’s inequality--$`\log \mathbb{E}[\bullet] \geq \mathbb{E} [\log (\bullet)]`$--yielding:
+<span id="ELBO-intractable" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
     \log p_\theta(\mathcal D) &\geq \sum_{i=0}^{N} \left(
@@ -671,6 +689,8 @@ In particular, the lower bound on <a href="#evidence-definition" data-reference
 \end{align}
 ```
 The true, generally intractable posterior $`p_\theta (z \vert o,a)`$ prevents computing both the expectation and KL divergence terms in <a href="#ELBO-intractable" data-reference-type="ref" data-reference="ELBO-intractable">[ELBO-intractable]</a>, and therefore @kingmaAutoEncodingVariationalBayes2022 propose deriving the ELBO using an *approximate* posterior $`q_\phi(z \vert o,a)`$, resulting in the final, tractable ELBO objective,
+<span id="ELBO" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
 \text{ELBO}_{\mathcal D}(\theta, \phi) = \sum_{i=0}^{N} \left(
@@ -684,6 +704,8 @@ From Jensen’s inequality, maximizing ELBO results in maximizing the log-likeli
 
 An intuitive explanation of the learning dynamics of VAEs can be given considering the equivalent case of *minimizing the negative ELBO*, which admits a particularly interpretable factorization
 
+<span id="VAE-min-neg-ELBO" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
 \min_{\theta, \phi} - \text{ELBO}_{\mathcal (o,a) \sim \mathcal D}(\theta, \phi) &= \min_{\theta, \phi}\mathbf{L^{\text{rec}}}(\theta) + \mathbf{L^{\text{reg}}}(\phi) \\
@@ -705,6 +727,8 @@ Indeed, it is very common in practice to approximate from the learned likelihood
 #### Diffusion Models
 
 VAEs approximate probability distributions via a *single* latent variable model, assuming the underlying unknown distribution can be factored according to <a href="#BC-latent-variable" data-reference-type="ref" data-reference="BC-latent-variable">[BC-latent-variable]</a>, and solve the variational inference problem of jointly learning the likelihood $`p_\theta`$ and (approximate) posterior $`q_\phi`$ for such model. In that, the unknown data distribution $`p(o,a)`$ is effectively approximated via $`\int_Z p(z) p_\theta(o,a \vert z)`$, and the underlying generative process reproduced by (1) sampling a latent variable and (2) learning to decode it into a (ideally) high-likelihood sample under the (unknown) $`p(o,a)`$. Diffusion Models (DMs) @hoDenoisingDiffusionProbabilistic2020 are another class of GMs which treat the similar problem of approximating an underlying unknown data distribution--*variational inference*--by *partially* extending VAEs to the case where *multiple* latent variables influence each other and the generative process underlying $`o,a`$ itself. In particular, DMs posit the generative process can be decomposed to a series of piece-wise (Markovian) interactions between (latent) variables (Figure <a href="#ch4-many-latents" data-reference-type="ref" data-reference="ch4-many-latents">[ch4-many-latents]</a>), resulting in
+<span id="BC-multi-latent-model-1" style="position- absolute;"></span>
+
 ``` math
 \begin{align}
     p(\underbrace{o,a}_{= z_0}) &= \int_{\text{supp}({Z_0})} \int_{\text{supp}({Z_1})} \ldots \int_{\text{supp}({Z_T})} p(z_0, z_1, \dots z_T)  \\ 
@@ -724,6 +748,8 @@ Similarily to VAEs, providing an exact interpretation for the latent variables i
 Just like VAEs, DMs attemp to learn to reproduce an underlying data distribution $`p (o,a)`$ given a collection of i.i.d. samples approximating the model posited to have generated the data in the first place ( <a href="#BC-multi-latent-model-1" data-reference-type="ref" data-reference="BC-multi-latent-model-1">[BC-multi-latent-model-1]</a>). Similarily to VAEs, DMs approximate the process of sampling from the unknown $`p(o,a)`$ (1) sampling from an easy-to-sample distribution (e.g., Gaussian) and (2) learning to reconstruct high-likelihood samples under the unknown distribution. However, in stark contrast with VAEs, the easy-to-sample distribution contains *no mutual information* regarding the data distribution $`p(o,a)`$. Crucially, as no information from the sample $`(o,a)`$ (denoted as $`z_0 \equiv (o,a)`$ for the sake of notation) is assumed to be propagated throughout the chain of latents, the posterior $`q(z_t \vert z_{t-1})`$ assumes a relatively amicable structure in DMs, reducing complexity. The *true* likelihood $`p(z_{t-1} \vert z_t)`$ is instead typically approximated using the parametrization $`p_\theta (z_{t-1} \vert z_t)`$. In that, the information contained in the unknwon data distribution is *reconstructed* via a process in which samples from a fixed distribution are turned into (ideally) high-likelihood samples under $`p(o,a)`$--a process referred to as *denoising*.
 
 Under such model, we can express the log-likelihood of an arbitrary sample as[^4]
+<span id="diffusion-likelihood" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
     \log p_\theta (\underbrace{o,a}_{= z_0}) = 
@@ -751,6 +777,8 @@ Finally, adopting Gaussian posteriors permits a particularly pleasing interpreta
 </figure>
 
 Because the recorded behavior is teleoperated, measurements mostly distribute along the line $`a = o + \eta, \eta \sim N(0,1)`$, with $`\eta`$-variability accouting for minor control inconsistencies (Figure <a href="#ch4-action-vs-observation-distribution" data-reference-type="ref" data-reference="ch4-action-vs-observation-distribution">[ch4-action-vs-observation-distribution]</a>). Using Gaussian posteriors--i.e., adding Gaussian noise--effectively simulates a *Brownian motion* for the elements in the distribution’s support (in Figure <a href="#diffusion-robot-actions" data-reference-type="ref" data-reference="diffusion-robot-actions">[diffusion-robot-actions]</a>, $`\mathcal O\times \mathcal A`$), whereby information *diffuses away* from the samples, and comparing the diffused samples to the original data points one can derive an estimate of the total displacement induced by diffusion. Under the only assumption that the likelihood of the diffused samples is low under the original unknown data distribution, then one can effectively approximate the unkwown distribution by learning to *reverse* such displacement. This key intuition allows to write a simplified training objective:
+<span id="diffusion-simplified-loss" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
 
@@ -764,6 +792,8 @@ Because the recorded behavior is teleoperated, measurements mostly distribute al
 In this simplified (minimization) objective, the optimization process differs from <a href="#diffusion-likelihood" data-reference-type="ref" data-reference="diffusion-likelihood">[diffusion-likelihood]</a> in that, rather than maxizing $`p_\theta`$ directly, the parameters $`\theta`$ of the pairwise likelihood $`p_\theta(z_{t-1} \vert z_t)`$ are adjusted to *predict the total displacement* $`\epsilon`$ for a randomly long ($`t \sim \mathcal{U}(\{1,\dots,T\}`$ )) diffusion process starting from a sample of the target distribution.
 
 By learning the total displacement from a generally, uninformative corrupted sample obtained diffusing information and a sample from an unknown distribution--significant ($`\Vert \epsilon \Vert > 0`$) whenever input and target distribution are sufficiently different-- @hoDenoisingDiffusionProbabilistic2020 show that one can approximate the underlying distribution reversing the displacement, *denoising* samples. Interestingly, under the hypothesis real-world data belongs to a single higher dimensional manifold (Manifold Hypothesis), @permenterInterpretingImprovingDiffusion2024 show that diffusion learns the gradient of a distance function from any off-point manifold (such as perturbed, uniformative samples), and the data manifold itself. Following this gradient--i.e., denoising a sample from an uninformative distribution--corresponds to projecting back into the manifold, yielding a procedure to sample from unknown distributions by means of Euclidean projection. Indeed, under the assumption that $`p_\theta (z_{t-1} \vert z_t)`$ is Gaussian, then sampling $`z_{t-1} \sim p_\theta(\bullet \vert z_{t})`$ corresponds to computing
+<span id="diffusion-denoising-definition" style="position- absolute;"></span>
+
 ``` math
 \begin{align}
     z_{t-1} = \frac{1}{\sqrt{\alpha_t}} \left( z_t - \frac{\beta_t}{\sqrt{1 - \bar\alpha_t}} \epsilon_\theta(z_t, t) \right) + \sigma_t \epsilon, \quad \epsilon \sim \mathcal N(\mathbf{0}, \mathbf{I}), 
@@ -804,6 +834,8 @@ While the noising schedule of DMs results in a stochastic process that resembles
 </figure>
 
 In practice, FM can be applied to generative modeling by learning a vector field regressor $`v_\theta(z, t)`$ to approximate a given target vector field $`u(t, z)`$. In the particular case of DMs, $`u(t, z)`$ is defined as in <a href="#fm-diffusion-vector-field" data-reference-type="ref" data-reference="fm-diffusion-vector-field">[fm-diffusion-vector-field]</a>, while in priciple the target vector field can be learned to induce a particular transportation, or fixed according to OT. Given a sample from the data distribution $`z_1 \sim p_1`$ and a sample from an easy-to-sample prior $`z_0 \sim p_0`$, CFM defines a simple path between them using *linear interpolation* between samples $`z_t = (1-t)z_0 + t z_1`$, resulting in the target vector field $`u(t, z_t) = z_1 - z_0`$. Then, a FM model can be trained with the simple regression objective defined as
+<span id="flow-matching-objective" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
 
@@ -820,6 +852,8 @@ While GMs prove useful in learning complex, high-dimensional multi-modal distrib
 On the robot learning side of their contributions, @zhaoLearningFineGrainedBimanual2023 adopt transformers as the architectural backbone to learn a *Conditional* VAE @sohnLearningStructuredOutput2015. Conditional VAEs are a variation of the more standard VAE formulation introducing a conditioning variable on sampling from the latent prior, allowing the modeling of *one-to-many* relationships between latent and data samples. Further, in stark contrast with previous work @florenceImplicitBehavioralCloning2022, @jannerPlanningDiffusionFlexible2022, @zhaoLearningFineGrainedBimanual2023 do not learn a full joint $`p_\theta(o,a)`$ on observation and actions. While the *policy* distribution $`p_\theta(a \vert o)`$ can in principle be entirely described from its joint $`p_\theta(o,a)`$, it is often the case that the conditional distribution is intractable when using function approximators, as $`p_\theta(a \vert o) = \tfrac{p_\theta(o,a)}{\int_\mathcal Ap_\theta(o,a)}`$ and the integral in the denominator is typically intractable. Instead of modeling the full joint using a vanilla VAE, @zhaoLearningFineGrainedBimanual2023 propose learning a *conditional* VAE @sohnLearningStructuredOutput2015 modeling the policy distribution directly $`p (a \vert o)`$.
 
 In practice, when learning from demonstrations adopting CVAEs results in a slight modification to the VAE objective in <a href="#ELBO" data-reference-type="ref" data-reference="ELBO">[ELBO]</a>, which is adapted to
+<span id="c-ELBO" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
 
@@ -862,6 +896,8 @@ However, the authors claim using a deterministic procedure to derive $`z`$ may b
 DMs proved very effective in approximating complex highly dimensional distributions, such as distributions over images @hoDenoisingDiffusionProbabilistic2020 or videos @polyakMovieGenCast2025, thanks to their inherent capability to deal with multimodal data and training stability. In Diffusion Policy (DP), @chiDiffusionPolicyVisuomotor2024 present an application of DMs the field of robot learning, leveraging diffusion to model human expert demonstrations in a variety of simulated and real-world tasks. Similarily to Action Chunking with Transformer @zhaoLearningFineGrainedBimanual2023, @chiDiffusionPolicyVisuomotor2024 (1) adopt a modified *observation-conditioned target distribution* instead of the full joint $`p(o,a)`$ and (2) predict multiple actions into the future instead of a single action. Besides the intractability of the observations’ marginal $`p_\theta(o)`$ given $`p_\theta(o,a)`$, DP’s rationale for modeling the data distribution via $`p_\theta(a \vert o)`$ stems from the rather test-time compute intensive nature of diffusion, whereby generating actions *alongside* observations is likely to result in higher complexity and thus a likely larger number of denoising operations, which would prove ultimately pointless considering robotics applications rely on the capability to generate controls rather than reproducing observations.
 
 In practice, conditioning on observation data is achieved conditioning the added noise regressor $`\epsilon_\theta`$ introduced in <a href="#diffusion-simplified-loss" data-reference-type="ref" data-reference="diffusion-simplified-loss">[diffusion-simplified-loss]</a> on a stack of $`T_o`$ observations, resulting in the *conditional* simplified diffusion objective
+<span id="diffusion-policy-objective" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
     \mathcal L(\theta) &= \mathbb{E}_{t, a_{t:t+H_a}, \epsilon} \big[
@@ -1033,6 +1069,8 @@ Concretely, $`\pi_0`$ is a unified transformer with two disjoint sets of weights
     \quad \mathbf{1}: \text{Bidirectional Attention}, \ \mathbf{0}: \text{Masked Attention}`$ Note how *intra*-block directional attention allows tokens to communicate freely, while *inter*-block communication is mediated by the attention mask $`\mathbf{A}`$. *Blockwise causal masking* effectively prevents the pre-trained perception-language tokens from attending to robotics-tokens, likely out of distribution for VLM backbones traditionally trained on large corpora of internet, non-robotics, data. Crucially, because communication is obstructed between image-language tokens, proprioperceptive and action tokens, one can cache keys and values across denoising steps at runtime time, incuring in a reduced computational footprint and faster inference.
 
 In $`\pi_0`$, both the VLM backbone and action expert are update using a *flow matching* loss, and in particular are updated minimizing:
+<span id="pi0-loss" style="position: absolute;"></span>
+
 ``` math
 \begin{align}
     \mathcal{L}(\phi, \theta) &= 

app/scripts/latex-to-mdx/output/main.mdx

@@ -347,6 +347,9 @@ Deriving the end-effector’s *pose*--position *and* orientation--in some $m$-di
 
 In the simplified case here considered (for which $\boldsymbol{p} \equiv p$, as the orientation of the end-effector is disregarded for simplicity), one can solve the problem of controlling the end-effector’s location to reach a goal position $p^*$ by solving analytically for $q: p(q) = f_{\text{FK}}(q) = p^*$. However, in the general case, one might not be able to solve this problem analytically, and can typically resort to iterative optimization methods comparing candidate solutions using a loss function (in the simplest case, $\Vert p(q) - p^* \Vert_2^2$ is a natural candidate), yielding:
 
+<span id="ik_problem" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 \min_{q \in \mathcal Q} \Vert p(q) - p^* \Vert_2^2 \, .
@@ -361,6 +364,9 @@ For instance, the robot in Figure <a href="#planar-manipulator-floor" data-refe
 However, IK--solving eq. <a href="#ik_problem" data-reference-type="ref" data-reference="ik_problem">[ik_problem]</a> for a feasible $q$--only proves useful in determining information regarding the robot’s configuration in the goal pose, and crucially does not provide information on the *trajectory* to follow over time to reach a target pose. Expert-defined trajectories obviate to this problem providing a length-$K$ succession of goal poses $\tau_K = [p^*_0, p^*_1, \dots p^*_K]$ for tracking. In practice, trajectories can also be obtained automatically through *motion planning* algorithms, thus avoiding expensive trajectory definition from human experts. However, tracking $\tau_K$ via IK can prove prohibitively expensive, as tracking would require $K$ resolutions of eq. <a href="#ik_problem" data-reference-type="ref" data-reference="ik_problem">[ik_problem]</a> (one for each target pose). *Differential* inverse kinematics (diff-IK) complements IK via closed-form solution of a variant of eq. <a href="#ik_problem" data-reference-type="ref" data-reference="ik_problem">[ik_problem]</a>. Let $J(q)$ denote the Jacobian matrix of (partial) derivatives of the FK-function $f_\text{FK}- \mathcal Q \mapsto \mathcal P$, such that $J(q) = \frac{\partial f_{FK}(q)}{\partial q }$. Then, one can apply the chain rule to any $p(q) = f_{\text{FK}}(q)$, deriving $\dot p = J(q) \dot q$, and thus finally relating variations in the robot configurations to variations in pose, thereby providing a platform for control.
 
 Given a desired end-effector trajectory $\dot {p}^*(t)$ (1) indicating anchor regions in space and (2) how much time to spend in each region, diff-IK finds $\dot q(t)$ solving for joints’ *velocities* instead of *configurations*,
+<span id="reg_ik_velocity" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 \dot q(t) = \arg\min_\nu \; \lVert J(q(t)) \nu - \dot {p}^*(t) \rVert_2^2
@@ -520,8 +526,9 @@ A length-$T$ *trajectory* is the (random) sequence
 ``` math
 \htmlId{trajectory_definition}{\tau = (s_0, a_0, r_0, s_1, a_1, r_1, \dots, s_{T-1}, a_{T-1}, r_{T-1}, s_T),}
 ```
-
 with per-step rewards defined as $r_t = r (s_t, a_t, s_{t+1})$ for ease of notation.Interestingly, assuming both the environment dynamics and conditional distribution over actions given states--the *policy*--to be *Markovian*:
+<span id="dynamics_markovian" style="position: absolute;">
+</span>
 
 ``` math
 \begin{align}
@@ -529,17 +536,21 @@ with per-step rewards defined as $r_t = r (s_t, a_t, s_{t+1})$ for ease of notat
 \mathbb P(a_t\vert s_t, a_{t-1}, s_{t-1}, s_0, a_0) &= \mathbb P(a_t\vert s_t) 
 \end{align}
 ```
+
 The probability of observing a given trajectory $\tau$ factorizes into
+
 ``` math
 \htmlId{traj_prob}{\mathbb P(\tau) = \mathbb P (s_0) \prod_{t=0}^{T-1} \mathbb P (s_{t+1}\vert s_t, a_t)\ \mathbb P(a_t\vert s_t).}
 ```
 
 Policies $\mathbb P(a_t\vert s_t)$ are typically indicated as $\pi(a_t\vert s_t)$, and often parametrized via $\theta$, yielding $\pi_\theta (a_t\vert s_t)$. Policies are trained optimizing the (discounted) *return* associated to a given $\tau$, i.e. the (random) sum of measured rewards over trajectory:
-
 ``` math
 G(\tau) = \sum_{t=0}^{T-1} \gamma^{t} r_t.
 ```
 In that, agents seek to learn control strategies (*policies*, $\pi_\theta$) maximizing the expected return $\mathbb E_{\tau \sim \pi_\theta} G(\tau)$. For a given dynamics $\mathcal D$--i.e., for a given problem--taking the expectation over the (possibly random) trajectories resulting from acting according to a certain policy provides a direct, goal-conditioned ordering in the space of all the possible policies $\Pi$, yielding the (maximization) target $J : \Pi \mapsto \mathbb R$
+<span id="RL-j-function" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     J(\pi_\theta) &= \mathbb E_{\tau \sim \mathbb P_{\theta; \mathcal D}} [G(\tau)],  \\
@@ -556,8 +567,9 @@ can be used to discriminate between desirable and undesirable state in terms of
 ``` math
 Q_\pi(s,a) = \mathbb E_{\tau \sim \pi} [G (\tau) \big \vert s_0 = s, a_0=a]
 ```
-
 Crucially, value functions are interrelated:
+<span id="q-as-v" style="position: absolute;">
+</span>
 
 ``` math
 \begin{align}
@@ -648,6 +660,9 @@ Q_{i+1}(s_t, a_t) \leftarrow \mathbb E_{s_{t+1} \sim \mathbb P(\bullet \vert s_t
 Then, one can derive the (ideally, near-optimal) policy by explicitly maximizing over the action space the final (ideally, near-optimal) estimate $Q_K \approx Q^*$ at each timestep. In fact, under certain assumptions on the MDP considered, $Q_K \to Q^* \, \text{as } K \to \infty$.
 
 Effective in its early applications to small-scale discrete problems and theoretically sound, vanilla Q-learning was found complicated to scale to large $\mathcal S\times \mathcal A$ problems, in which the storing of $Q : \mathcal S\times \mathcal A\mapsto \mathbb R$ alone might result prohibitive. Also, vanilla Q-learning is not directly usable for *continuous*, unstructured state-action space MPDs, such as those considered in robotics. In their seminal work on *Deep Q-Learning* (DQN), @mnihPlayingAtariDeep2013 propose learning Q-values using deep convolutional neural networks, thereby accomodating for large and even unstructured *state* spaces. DQN parametrizes the Q-function using a neural network with parameters $\theta$, updating the parameters by sequentially minimizing the expected squared temporal-difference error (TD-error, $\delta_i$):
+<span id="dqn-loss" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 \mathcal L(\theta_i) &= \mathbb E_{(s_t, a_t) \sim \chi(\bullet)} 
@@ -672,6 +687,9 @@ Provably, <a href="#deterministic-pg" data-reference-type="ref" data-reference="
 Similarily to DQN, DDPG also employs the same replay buffer mechanism, to reuse past transitions over training for increased sample efficiency and estimate the loss function via MC-estimates.
 
 Soft Actor-Critic (SAC) @haarnojaSoftActorCriticOffPolicy2018 is a derivation of DDPG in the max-entropy (MaxEnt) RL framework, in which RL agents are tasked with <span class="highlight">maximizing the discounted cumulative reward, while acting as randomly as possible</span>. MaxEnt RL @haarnojaReinforcementLearningDeep2017 has proven particularly robust thanks to the development of diverse behaviors, incentivized by its entropy-regularization formulation. In that, MaxEnt revisits the RL objective $J (\pi)$ to specifically account for the policy entropy,
+<span id="J-soft" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     J(\pi) &= \sum_{t=0}^T \mathbb{E}_{(s_t, a_t) \sim \chi} [r_t + \alpha \mathcal H(\pi (\bullet \vert s_t))] 
@@ -835,6 +853,9 @@ Intuitively, in the case of observation-action pairs $(o, a)$ for a robotics app
 </figure>
 
 Given a dataset $\mathcal D$ consisting of $N$ i.i.d. observation-action pairs, the log-likelihood of all datapoints under $\theta$ (in Bayesian terms, the *evidence* $p_\theta(\mathcal D)$) can thus be written as:
+<span id="evidence-definition-1" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     \log p_\theta(\mathcal D) &= \log \sum_{i=0}^N p_\theta ((o,a)_i) \\
@@ -850,6 +871,9 @@ In the special case where one assumes distributions to be tractable, $p_\theta (
 In their seminal work on Variational Auto-Encoders (VAEs), @kingmaAutoEncodingVariationalBayes2022 present two major contributions to learn complex latent-variable GMs on unstructured data, proposing (1) a tractable, variational lower-bound to <a href="#evidence-definition" data-reference-type="ref" data-reference="evidence-definition">[evidence-definition]</a> as an optimization target to jointly learn likelihood and posterior and (2) high-capacity function approximators to model the likelihood $p_\theta(o,a\vert z)$ and (approximate) posterior distribution $q_\phi(z \vert o,a) \approx q_\theta(z \vert o,a)$.
 
 In particular, the lower bound on <a href="#evidence-definition" data-reference-type="ref" data-reference="evidence-definition">[evidence-definition]</a> (Evidence LOwer Bound, *ELBO*) can be derived from <a href="#evidence-definition" data-reference-type="ref" data-reference="evidence-definition">[evidence-definition]</a> applying Jensen’s inequality--$\log \mathbb{E}[\bullet] \geq \mathbb{E} [\log (\bullet)]$--yielding:
+<span id="ELBO-intractable" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     \log p_\theta(\mathcal D) &\geq \sum_{i=0}^{N} \left(
@@ -862,8 +886,9 @@ In particular, the lower bound on <a href="#evidence-definition" data-reference
         \right) 
 \end{align}
 ```
-
 The true, generally intractable posterior $p_\theta (z \vert o,a)$ prevents computing both the expectation and KL divergence terms in <a href="#ELBO-intractable" data-reference-type="ref" data-reference="ELBO-intractable">[ELBO-intractable]</a>, and therefore @kingmaAutoEncodingVariationalBayes2022 propose deriving the ELBO using an *approximate* posterior $q_\phi(z \vert o,a)$, resulting in the final, tractable ELBO objective,
+<span id="ELBO" style="position: absolute;">
+</span>
 
 ``` math
 \begin{align}
@@ -878,6 +903,9 @@ From Jensen’s inequality, maximizing ELBO results in maximizing the log-likeli
 
 An intuitive explanation of the learning dynamics of VAEs can be given considering the equivalent case of *minimizing the negative ELBO*, which admits a particularly interpretable factorization
 
+<span id="VAE-min-neg-ELBO" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 \min_{\theta, \phi} - \text{ELBO}_{\mathcal (o,a) \sim \mathcal D}(\theta, \phi) &= \min_{\theta, \phi}\mathbf{L^{\text{rec}}}(\theta) + \mathbf{L^{\text{reg}}}(\phi) \\
@@ -900,6 +928,9 @@ Indeed, it is very common in practice to approximate from the learned likelihood
 #### Diffusion Models
 
 VAEs approximate probability distributions via a *single* latent variable model, assuming the underlying unknown distribution can be factored according to <a href="#BC-latent-variable" data-reference-type="ref" data-reference="BC-latent-variable">[BC-latent-variable]</a>, and solve the variational inference problem of jointly learning the likelihood $p_\theta$ and (approximate) posterior $q_\phi$ for such model. In that, the unknown data distribution $p(o,a)$ is effectively approximated via $\int_Z p(z) p_\theta(o,a \vert z)$, and the underlying generative process reproduced by (1) sampling a latent variable and (2) learning to decode it into a (ideally) high-likelihood sample under the (unknown) $p(o,a)$. Diffusion Models (DMs) @hoDenoisingDiffusionProbabilistic2020 are another class of GMs which treat the similar problem of approximating an underlying unknown data distribution--*variational inference*--by *partially* extending VAEs to the case where *multiple* latent variables influence each other and the generative process underlying $o,a$ itself. In particular, DMs posit the generative process can be decomposed to a series of piece-wise (Markovian) interactions between (latent) variables (Figure <a href="#ch4-many-latents" data-reference-type="ref" data-reference="ch4-many-latents">[ch4-many-latents]</a>), resulting in
+<span id="BC-multi-latent-model-1" style="position- absolute;">
+</span>
+
 ``` math
 \begin{align}
     p(\underbrace{o,a}_{= z_0}) &= \int_{\text{supp}({Z_0})} \int_{\text{supp}({Z_1})} \ldots \int_{\text{supp}({Z_T})} p(z_0, z_1, \dots z_T)  \\ 
@@ -925,6 +956,9 @@ Similarily to VAEs, providing an exact interpretation for the latent variables i
 Just like VAEs, DMs attemp to learn to reproduce an underlying data distribution $p (o,a)$ given a collection of i.i.d. samples approximating the model posited to have generated the data in the first place ( <a href="#BC-multi-latent-model-1" data-reference-type="ref" data-reference="BC-multi-latent-model-1">[BC-multi-latent-model-1]</a>). Similarily to VAEs, DMs approximate the process of sampling from the unknown $p(o,a)$ (1) sampling from an easy-to-sample distribution (e.g., Gaussian) and (2) learning to reconstruct high-likelihood samples under the unknown distribution. However, in stark contrast with VAEs, the easy-to-sample distribution contains *no mutual information* regarding the data distribution $p(o,a)$. Crucially, as no information from the sample $(o,a)$ (denoted as $z_0 \equiv (o,a)$ for the sake of notation) is assumed to be propagated throughout the chain of latents, the posterior $q(z_t \vert z_{t-1})$ assumes a relatively amicable structure in DMs, reducing complexity. The *true* likelihood $p(z_{t-1} \vert z_t)$ is instead typically approximated using the parametrization $p_\theta (z_{t-1} \vert z_t)$. In that, the information contained in the unknwon data distribution is *reconstructed* via a process in which samples from a fixed distribution are turned into (ideally) high-likelihood samples under $p(o,a)$--a process referred to as *denoising*.
 
 Under such model, we can express the log-likelihood of an arbitrary sample as[^4]
+<span id="diffusion-likelihood" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     \log p_\theta (\underbrace{o,a}_{= z_0}) = 
@@ -964,6 +998,9 @@ Finally, adopting Gaussian posteriors permits a particularly pleasing interpreta
 </figure>
 
 Because the recorded behavior is teleoperated, measurements mostly distribute along the line $a = o + \eta, \eta \sim N(0,1)$, with $\eta$-variability accouting for minor control inconsistencies (Figure <a href="#ch4-action-vs-observation-distribution" data-reference-type="ref" data-reference="ch4-action-vs-observation-distribution">[ch4-action-vs-observation-distribution]</a>). Using Gaussian posteriors--i.e., adding Gaussian noise--effectively simulates a *Brownian motion* for the elements in the distribution’s support (in Figure <a href="#diffusion-robot-actions" data-reference-type="ref" data-reference="diffusion-robot-actions">[diffusion-robot-actions]</a>, $\mathcal O\times \mathcal A$), whereby information *diffuses away* from the samples, and comparing the diffused samples to the original data points one can derive an estimate of the total displacement induced by diffusion. Under the only assumption that the likelihood of the diffused samples is low under the original unknown data distribution, then one can effectively approximate the unkwown distribution by learning to *reverse* such displacement. This key intuition allows to write a simplified training objective:
+<span id="diffusion-simplified-loss" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 
@@ -977,6 +1014,9 @@ Because the recorded behavior is teleoperated, measurements mostly distribute al
 In this simplified (minimization) objective, the optimization process differs from <a href="#diffusion-likelihood" data-reference-type="ref" data-reference="diffusion-likelihood">[diffusion-likelihood]</a> in that, rather than maxizing $p_\theta$ directly, the parameters $\theta$ of the pairwise likelihood $p_\theta(z_{t-1} \vert z_t)$ are adjusted to *predict the total displacement* $\epsilon$ for a randomly long ($t \sim \mathcal{U}(\{1,\dots,T\}$ )) diffusion process starting from a sample of the target distribution.
 
 By learning the total displacement from a generally, uninformative corrupted sample obtained diffusing information and a sample from an unknown distribution--significant ($\Vert \epsilon \Vert > 0$) whenever input and target distribution are sufficiently different-- @hoDenoisingDiffusionProbabilistic2020 show that one can approximate the underlying distribution reversing the displacement, *denoising* samples. Interestingly, under the hypothesis real-world data belongs to a single higher dimensional manifold (Manifold Hypothesis), @permenterInterpretingImprovingDiffusion2024 show that diffusion learns the gradient of a distance function from any off-point manifold (such as perturbed, uniformative samples), and the data manifold itself. Following this gradient--i.e., denoising a sample from an uninformative distribution--corresponds to projecting back into the manifold, yielding a procedure to sample from unknown distributions by means of Euclidean projection. Indeed, under the assumption that $p_\theta (z_{t-1} \vert z_t)$ is Gaussian, then sampling $z_{t-1} \sim p_\theta(\bullet \vert z_{t})$ corresponds to computing
+<span id="diffusion-denoising-definition" style="position- absolute;">
+</span>
+
 ``` math
 \begin{align}
     z_{t-1} = \frac{1}{\sqrt{\alpha_t}} \left( z_t - \frac{\beta_t}{\sqrt{1 - \bar\alpha_t}} \epsilon_\theta(z_t, t) \right) + \sigma_t \epsilon, \quad \epsilon \sim \mathcal N(\mathbf{0}, \mathbf{I}), 
@@ -1030,6 +1070,9 @@ While the noising schedule of DMs results in a stochastic process that resembles
 </figure>
 
 In practice, FM can be applied to generative modeling by learning a vector field regressor $v_\theta(z, t)$ to approximate a given target vector field $u(t, z)$. In the particular case of DMs, $u(t, z)$ is defined as in <a href="#fm-diffusion-vector-field" data-reference-type="ref" data-reference="fm-diffusion-vector-field">[fm-diffusion-vector-field]</a>, while in priciple the target vector field can be learned to induce a particular transportation, or fixed according to OT. Given a sample from the data distribution $z_1 \sim p_1$ and a sample from an easy-to-sample prior $z_0 \sim p_0$, CFM defines a simple path between them using *linear interpolation* between samples $z_t = (1-t)z_0 + t z_1$, resulting in the target vector field $u(t, z_t) = z_1 - z_0$. Then, a FM model can be trained with the simple regression objective defined as
+<span id="flow-matching-objective" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 
@@ -1046,6 +1089,9 @@ While GMs prove useful in learning complex, high-dimensional multi-modal distrib
 On the robot learning side of their contributions, @zhaoLearningFineGrainedBimanual2023 adopt transformers as the architectural backbone to learn a *Conditional* VAE @sohnLearningStructuredOutput2015. Conditional VAEs are a variation of the more standard VAE formulation introducing a conditioning variable on sampling from the latent prior, allowing the modeling of *one-to-many* relationships between latent and data samples. Further, in stark contrast with previous work @florenceImplicitBehavioralCloning2022, @jannerPlanningDiffusionFlexible2022, @zhaoLearningFineGrainedBimanual2023 do not learn a full joint $p_\theta(o,a)$ on observation and actions. While the *policy* distribution $p_\theta(a \vert o)$ can in principle be entirely described from its joint $p_\theta(o,a)$, it is often the case that the conditional distribution is intractable when using function approximators, as $p_\theta(a \vert o) = \tfrac{p_\theta(o,a)}{\int_\mathcal Ap_\theta(o,a)}$ and the integral in the denominator is typically intractable. Instead of modeling the full joint using a vanilla VAE, @zhaoLearningFineGrainedBimanual2023 propose learning a *conditional* VAE @sohnLearningStructuredOutput2015 modeling the policy distribution directly $p (a \vert o)$.
 
 In practice, when learning from demonstrations adopting CVAEs results in a slight modification to the VAE objective in <a href="#ELBO" data-reference-type="ref" data-reference="ELBO">[ELBO]</a>, which is adapted to
+<span id="c-ELBO" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 
@@ -1106,6 +1152,9 @@ However, the authors claim using a deterministic procedure to derive $z$ may ben
 DMs proved very effective in approximating complex highly dimensional distributions, such as distributions over images @hoDenoisingDiffusionProbabilistic2020 or videos @polyakMovieGenCast2025, thanks to their inherent capability to deal with multimodal data and training stability. In Diffusion Policy (DP), @chiDiffusionPolicyVisuomotor2024 present an application of DMs the field of robot learning, leveraging diffusion to model human expert demonstrations in a variety of simulated and real-world tasks. Similarily to Action Chunking with Transformer @zhaoLearningFineGrainedBimanual2023, @chiDiffusionPolicyVisuomotor2024 (1) adopt a modified *observation-conditioned target distribution* instead of the full joint $p(o,a)$ and (2) predict multiple actions into the future instead of a single action. Besides the intractability of the observations’ marginal $p_\theta(o)$ given $p_\theta(o,a)$, DP’s rationale for modeling the data distribution via $p_\theta(a \vert o)$ stems from the rather test-time compute intensive nature of diffusion, whereby generating actions *alongside* observations is likely to result in higher complexity and thus a likely larger number of denoising operations, which would prove ultimately pointless considering robotics applications rely on the capability to generate controls rather than reproducing observations.
 
 In practice, conditioning on observation data is achieved conditioning the added noise regressor $\epsilon_\theta$ introduced in <a href="#diffusion-simplified-loss" data-reference-type="ref" data-reference="diffusion-simplified-loss">[diffusion-simplified-loss]</a> on a stack of $T_o$ observations, resulting in the *conditional* simplified diffusion objective
+<span id="diffusion-policy-objective" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     \mathcal L(\theta) &= \mathbb{E}_{t, a_{t:t+H_a}, \epsilon} \big[
@@ -1306,6 +1355,9 @@ $\pi_0$ @blackp0VisionLanguageActionFlow2024 introduce a VLA consisting of a Mo
 Concretely, $\pi_0$ is a unified transformer with two disjoint sets of weights $\phi, \theta$. A larger VLM backbone $p_\phi$ initialized from Gemma 2.6B processes multiple image frames obtained from multiple cameras points $[\{ I_t \}_{t=1}^n]$, as well as a language instruction $[\ell_t]$ used to describe the task considered. Concurrently, a 300M-parameter *action expert* based on a similar transformer architecture is used processes the robot proprioperceptive state $q_t$ and an action chunk $a_{t:t+H_a}$ (Figure <a href="#ch5-pi0" data-reference-type="ref" data-reference="ch5-pi0">[ch5-pi0]</a>). The different expert networks operate separately in processing the respective inputs and turning them into query, key and value matrices, and only share information between each other via self-attention layers. The outputs from the VLM backbone are disregarded, while the vector field regressed by the action expert is used to iteratively refine the action process. In particular, $\pi_0$uses a *blockwise causal attention mask* over tokens belonging to three separate blocks: (1) image and language tokens $\mathcal T_i$ obtained from $[\{ I_t \}_{t=1}^n, \ell_t]$, (2) proprioperceptive tokens $\mathcal T_q$ obtained from $q_t$, and (3) the action tokens $\mathcal T_a$ for items in the chunk $a^{\tau}_{t:t+H_a}$ at time $\tau$ in the flow-matching process. Notably, *within* each block the attention operations are bidirectional, while across blocks, future blocks are masked out. Formally, this corresponds to using the attention mask $\mathbf{A} = \bordermatrix{ \mathcal{T}_i \mathcal{T}_q \mathcal{T}_a \cr \mathcal{T}_i \mathbf{1} \mathbf{0} \mathbf{0} \cr \mathcal{T}_q \mathbf{1} \mathbf{1} \mathbf{0} \cr \mathcal{T}_a \mathbf{1} \mathbf{1} \mathbf{1} \cr }, \quad \mathbf{1}: \text{Bidirectional Attention}, \ \mathbf{0}: \text{Masked Attention}$ Note how *intra*-block directional attention allows tokens to communicate freely, while *inter*-block communication is mediated by the attention mask $\mathbf{A}$. *Blockwise causal masking* effectively prevents the pre-trained perception-language tokens from attending to robotics-tokens, likely out of distribution for VLM backbones traditionally trained on large corpora of internet, non-robotics, data. Crucially, because communication is obstructed between image-language tokens, proprioperceptive and action tokens, one can cache keys and values across denoising steps at runtime time, incuring in a reduced computational footprint and faster inference.
 
 In $\pi_0$, both the VLM backbone and action expert are update using a *flow matching* loss, and in particular are updated minimizing:
+<span id="pi0-loss" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     \mathcal{L}(\phi, \theta) &= 

app/scripts/latex-to-mdx/post-processor.mjs

@@ -300,6 +300,29 @@ function fixLinkTextContent(content) {
     return cleanedContent;
 }
 
+/**
+ * Convert align anchor markers to proper HTML spans outside math blocks
+ * @param {string} content - Markdown content
+ * @returns {string} - Content with converted anchor spans
+ */
+function convertAlignAnchors(content) {
+    console.log('  🏷️  Converting align anchor markers to HTML spans...');
+
+    let convertedCount = 0;
+
+    // Find and replace align anchor markers with proper spans outside math blocks
+    content = content.replace(/``` math\n%%ALIGN_ANCHOR_ID\{([^}]+)\}%%\n([\s\S]*?)\n```/g, (match, anchorId, mathContent) => {
+        convertedCount++;
+        return `<span id="${anchorId}" style="position: absolute;"></span>\n\n\`\`\` math\n${mathContent}\n\`\`\``;
+    });
+
+    if (convertedCount > 0) {
+        console.log(`    ✅ Converted ${convertedCount} align anchor marker(s) to spans`);
+    }
+
+    return content;
+}
+
 /**
  * Main post-processing function that applies all cleanup steps
  * @param {string} content - Raw Markdown content from Pandoc
@@ -315,6 +338,7 @@ export function postProcessMarkdown(content, inputDir = null) {
     processedContent = removeTexGroupingCommands(processedContent);
     processedContent = simplifyLatexDelimiters(processedContent);
     processedContent = removeOrphanedLabels(processedContent);
+    processedContent = convertAlignAnchors(processedContent);
     processedContent = fixMathCommands(processedContent);
     processedContent = fixMatrixCommands(processedContent);
     processedContent = fixUnicodeIssues(processedContent);

app/src/components/Hero.astro

@@ -202,7 +202,7 @@ const pdfFilename = `${slugify(pdfBase)}.pdf`;
   /* Hero (full-width) */
   .hero {
     width: 100%;
-    padding: 48px 16px 16px;
+    padding: 48px 16px 48px;
     text-align: center;
   }
   .hero-title {

app/src/content/article.mdx

@@ -347,6 +347,9 @@ Deriving the end-effector’s *pose*--position *and* orientation--in some $m$-di
 
 In the simplified case here considered (for which $\boldsymbol{p} \equiv p$, as the orientation of the end-effector is disregarded for simplicity), one can solve the problem of controlling the end-effector’s location to reach a goal position $p^*$ by solving analytically for $q: p(q) = f_{\text{FK}}(q) = p^*$. However, in the general case, one might not be able to solve this problem analytically, and can typically resort to iterative optimization methods comparing candidate solutions using a loss function (in the simplest case, $\Vert p(q) - p^* \Vert_2^2$ is a natural candidate), yielding:
 
+<span id="ik_problem" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 \min_{q \in \mathcal Q} \Vert p(q) - p^* \Vert_2^2 \, .
@@ -361,6 +364,9 @@ For instance, the robot in Figure <a href="#planar-manipulator-floor" data-refe
 However, IK--solving eq. <a href="#ik_problem" data-reference-type="ref" data-reference="ik_problem">[ik_problem]</a> for a feasible $q$--only proves useful in determining information regarding the robot’s configuration in the goal pose, and crucially does not provide information on the *trajectory* to follow over time to reach a target pose. Expert-defined trajectories obviate to this problem providing a length-$K$ succession of goal poses $\tau_K = [p^*_0, p^*_1, \dots p^*_K]$ for tracking. In practice, trajectories can also be obtained automatically through *motion planning* algorithms, thus avoiding expensive trajectory definition from human experts. However, tracking $\tau_K$ via IK can prove prohibitively expensive, as tracking would require $K$ resolutions of eq. <a href="#ik_problem" data-reference-type="ref" data-reference="ik_problem">[ik_problem]</a> (one for each target pose). *Differential* inverse kinematics (diff-IK) complements IK via closed-form solution of a variant of eq. <a href="#ik_problem" data-reference-type="ref" data-reference="ik_problem">[ik_problem]</a>. Let $J(q)$ denote the Jacobian matrix of (partial) derivatives of the FK-function $f_\text{FK}- \mathcal Q \mapsto \mathcal P$, such that $J(q) = \frac{\partial f_{FK}(q)}{\partial q }$. Then, one can apply the chain rule to any $p(q) = f_{\text{FK}}(q)$, deriving $\dot p = J(q) \dot q$, and thus finally relating variations in the robot configurations to variations in pose, thereby providing a platform for control.
 
 Given a desired end-effector trajectory $\dot {p}^*(t)$ (1) indicating anchor regions in space and (2) how much time to spend in each region, diff-IK finds $\dot q(t)$ solving for joints’ *velocities* instead of *configurations*,
+<span id="reg_ik_velocity" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 \dot q(t) = \arg\min_\nu \; \lVert J(q(t)) \nu - \dot {p}^*(t) \rVert_2^2
@@ -520,8 +526,9 @@ A length-$T$ *trajectory* is the (random) sequence
 ``` math
 \htmlId{trajectory_definition}{\tau = (s_0, a_0, r_0, s_1, a_1, r_1, \dots, s_{T-1}, a_{T-1}, r_{T-1}, s_T),}
 ```
-
 with per-step rewards defined as $r_t = r (s_t, a_t, s_{t+1})$ for ease of notation.Interestingly, assuming both the environment dynamics and conditional distribution over actions given states--the *policy*--to be *Markovian*:
+<span id="dynamics_markovian" style="position: absolute;">
+</span>
 
 ``` math
 \begin{align}
@@ -529,17 +536,21 @@ with per-step rewards defined as $r_t = r (s_t, a_t, s_{t+1})$ for ease of notat
 \mathbb P(a_t\vert s_t, a_{t-1}, s_{t-1}, s_0, a_0) &= \mathbb P(a_t\vert s_t) 
 \end{align}
 ```
+
 The probability of observing a given trajectory $\tau$ factorizes into
+
 ``` math
 \htmlId{traj_prob}{\mathbb P(\tau) = \mathbb P (s_0) \prod_{t=0}^{T-1} \mathbb P (s_{t+1}\vert s_t, a_t)\ \mathbb P(a_t\vert s_t).}
 ```
 
 Policies $\mathbb P(a_t\vert s_t)$ are typically indicated as $\pi(a_t\vert s_t)$, and often parametrized via $\theta$, yielding $\pi_\theta (a_t\vert s_t)$. Policies are trained optimizing the (discounted) *return* associated to a given $\tau$, i.e. the (random) sum of measured rewards over trajectory:
-
 ``` math
 G(\tau) = \sum_{t=0}^{T-1} \gamma^{t} r_t.
 ```
 In that, agents seek to learn control strategies (*policies*, $\pi_\theta$) maximizing the expected return $\mathbb E_{\tau \sim \pi_\theta} G(\tau)$. For a given dynamics $\mathcal D$--i.e., for a given problem--taking the expectation over the (possibly random) trajectories resulting from acting according to a certain policy provides a direct, goal-conditioned ordering in the space of all the possible policies $\Pi$, yielding the (maximization) target $J : \Pi \mapsto \mathbb R$
+<span id="RL-j-function" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     J(\pi_\theta) &= \mathbb E_{\tau \sim \mathbb P_{\theta; \mathcal D}} [G(\tau)],  \\
@@ -556,8 +567,9 @@ can be used to discriminate between desirable and undesirable state in terms of
 ``` math
 Q_\pi(s,a) = \mathbb E_{\tau \sim \pi} [G (\tau) \big \vert s_0 = s, a_0=a]
 ```
-
 Crucially, value functions are interrelated:
+<span id="q-as-v" style="position: absolute;">
+</span>
 
 ``` math
 \begin{align}
@@ -648,6 +660,9 @@ Q_{i+1}(s_t, a_t) \leftarrow \mathbb E_{s_{t+1} \sim \mathbb P(\bullet \vert s_t
 Then, one can derive the (ideally, near-optimal) policy by explicitly maximizing over the action space the final (ideally, near-optimal) estimate $Q_K \approx Q^*$ at each timestep. In fact, under certain assumptions on the MDP considered, $Q_K \to Q^* \, \text{as } K \to \infty$.
 
 Effective in its early applications to small-scale discrete problems and theoretically sound, vanilla Q-learning was found complicated to scale to large $\mathcal S\times \mathcal A$ problems, in which the storing of $Q : \mathcal S\times \mathcal A\mapsto \mathbb R$ alone might result prohibitive. Also, vanilla Q-learning is not directly usable for *continuous*, unstructured state-action space MPDs, such as those considered in robotics. In their seminal work on *Deep Q-Learning* (DQN), @mnihPlayingAtariDeep2013 propose learning Q-values using deep convolutional neural networks, thereby accomodating for large and even unstructured *state* spaces. DQN parametrizes the Q-function using a neural network with parameters $\theta$, updating the parameters by sequentially minimizing the expected squared temporal-difference error (TD-error, $\delta_i$):
+<span id="dqn-loss" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 \mathcal L(\theta_i) &= \mathbb E_{(s_t, a_t) \sim \chi(\bullet)} 
@@ -672,6 +687,9 @@ Provably, <a href="#deterministic-pg" data-reference-type="ref" data-reference="
 Similarily to DQN, DDPG also employs the same replay buffer mechanism, to reuse past transitions over training for increased sample efficiency and estimate the loss function via MC-estimates.
 
 Soft Actor-Critic (SAC) @haarnojaSoftActorCriticOffPolicy2018 is a derivation of DDPG in the max-entropy (MaxEnt) RL framework, in which RL agents are tasked with <span class="highlight">maximizing the discounted cumulative reward, while acting as randomly as possible</span>. MaxEnt RL @haarnojaReinforcementLearningDeep2017 has proven particularly robust thanks to the development of diverse behaviors, incentivized by its entropy-regularization formulation. In that, MaxEnt revisits the RL objective $J (\pi)$ to specifically account for the policy entropy,
+<span id="J-soft" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     J(\pi) &= \sum_{t=0}^T \mathbb{E}_{(s_t, a_t) \sim \chi} [r_t + \alpha \mathcal H(\pi (\bullet \vert s_t))] 
@@ -835,6 +853,9 @@ Intuitively, in the case of observation-action pairs $(o, a)$ for a robotics app
 </figure>
 
 Given a dataset $\mathcal D$ consisting of $N$ i.i.d. observation-action pairs, the log-likelihood of all datapoints under $\theta$ (in Bayesian terms, the *evidence* $p_\theta(\mathcal D)$) can thus be written as:
+<span id="evidence-definition-1" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     \log p_\theta(\mathcal D) &= \log \sum_{i=0}^N p_\theta ((o,a)_i) \\
@@ -850,6 +871,9 @@ In the special case where one assumes distributions to be tractable, $p_\theta (
 In their seminal work on Variational Auto-Encoders (VAEs), @kingmaAutoEncodingVariationalBayes2022 present two major contributions to learn complex latent-variable GMs on unstructured data, proposing (1) a tractable, variational lower-bound to <a href="#evidence-definition" data-reference-type="ref" data-reference="evidence-definition">[evidence-definition]</a> as an optimization target to jointly learn likelihood and posterior and (2) high-capacity function approximators to model the likelihood $p_\theta(o,a\vert z)$ and (approximate) posterior distribution $q_\phi(z \vert o,a) \approx q_\theta(z \vert o,a)$.
 
 In particular, the lower bound on <a href="#evidence-definition" data-reference-type="ref" data-reference="evidence-definition">[evidence-definition]</a> (Evidence LOwer Bound, *ELBO*) can be derived from <a href="#evidence-definition" data-reference-type="ref" data-reference="evidence-definition">[evidence-definition]</a> applying Jensen’s inequality--$\log \mathbb{E}[\bullet] \geq \mathbb{E} [\log (\bullet)]$--yielding:
+<span id="ELBO-intractable" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     \log p_\theta(\mathcal D) &\geq \sum_{i=0}^{N} \left(
@@ -862,8 +886,9 @@ In particular, the lower bound on <a href="#evidence-definition" data-reference
         \right) 
 \end{align}
 ```
-
 The true, generally intractable posterior $p_\theta (z \vert o,a)$ prevents computing both the expectation and KL divergence terms in <a href="#ELBO-intractable" data-reference-type="ref" data-reference="ELBO-intractable">[ELBO-intractable]</a>, and therefore @kingmaAutoEncodingVariationalBayes2022 propose deriving the ELBO using an *approximate* posterior $q_\phi(z \vert o,a)$, resulting in the final, tractable ELBO objective,
+<span id="ELBO" style="position: absolute;">
+</span>
 
 ``` math
 \begin{align}
@@ -878,6 +903,9 @@ From Jensen’s inequality, maximizing ELBO results in maximizing the log-likeli
 
 An intuitive explanation of the learning dynamics of VAEs can be given considering the equivalent case of *minimizing the negative ELBO*, which admits a particularly interpretable factorization
 
+<span id="VAE-min-neg-ELBO" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 \min_{\theta, \phi} - \text{ELBO}_{\mathcal (o,a) \sim \mathcal D}(\theta, \phi) &= \min_{\theta, \phi}\mathbf{L^{\text{rec}}}(\theta) + \mathbf{L^{\text{reg}}}(\phi) \\
@@ -900,6 +928,9 @@ Indeed, it is very common in practice to approximate from the learned likelihood
 #### Diffusion Models
 
 VAEs approximate probability distributions via a *single* latent variable model, assuming the underlying unknown distribution can be factored according to <a href="#BC-latent-variable" data-reference-type="ref" data-reference="BC-latent-variable">[BC-latent-variable]</a>, and solve the variational inference problem of jointly learning the likelihood $p_\theta$ and (approximate) posterior $q_\phi$ for such model. In that, the unknown data distribution $p(o,a)$ is effectively approximated via $\int_Z p(z) p_\theta(o,a \vert z)$, and the underlying generative process reproduced by (1) sampling a latent variable and (2) learning to decode it into a (ideally) high-likelihood sample under the (unknown) $p(o,a)$. Diffusion Models (DMs) @hoDenoisingDiffusionProbabilistic2020 are another class of GMs which treat the similar problem of approximating an underlying unknown data distribution--*variational inference*--by *partially* extending VAEs to the case where *multiple* latent variables influence each other and the generative process underlying $o,a$ itself. In particular, DMs posit the generative process can be decomposed to a series of piece-wise (Markovian) interactions between (latent) variables (Figure <a href="#ch4-many-latents" data-reference-type="ref" data-reference="ch4-many-latents">[ch4-many-latents]</a>), resulting in
+<span id="BC-multi-latent-model-1" style="position- absolute;">
+</span>
+
 ``` math
 \begin{align}
     p(\underbrace{o,a}_{= z_0}) &= \int_{\text{supp}({Z_0})} \int_{\text{supp}({Z_1})} \ldots \int_{\text{supp}({Z_T})} p(z_0, z_1, \dots z_T)  \\ 
@@ -925,6 +956,9 @@ Similarily to VAEs, providing an exact interpretation for the latent variables i
 Just like VAEs, DMs attemp to learn to reproduce an underlying data distribution $p (o,a)$ given a collection of i.i.d. samples approximating the model posited to have generated the data in the first place ( <a href="#BC-multi-latent-model-1" data-reference-type="ref" data-reference="BC-multi-latent-model-1">[BC-multi-latent-model-1]</a>). Similarily to VAEs, DMs approximate the process of sampling from the unknown $p(o,a)$ (1) sampling from an easy-to-sample distribution (e.g., Gaussian) and (2) learning to reconstruct high-likelihood samples under the unknown distribution. However, in stark contrast with VAEs, the easy-to-sample distribution contains *no mutual information* regarding the data distribution $p(o,a)$. Crucially, as no information from the sample $(o,a)$ (denoted as $z_0 \equiv (o,a)$ for the sake of notation) is assumed to be propagated throughout the chain of latents, the posterior $q(z_t \vert z_{t-1})$ assumes a relatively amicable structure in DMs, reducing complexity. The *true* likelihood $p(z_{t-1} \vert z_t)$ is instead typically approximated using the parametrization $p_\theta (z_{t-1} \vert z_t)$. In that, the information contained in the unknwon data distribution is *reconstructed* via a process in which samples from a fixed distribution are turned into (ideally) high-likelihood samples under $p(o,a)$--a process referred to as *denoising*.
 
 Under such model, we can express the log-likelihood of an arbitrary sample as[^4]
+<span id="diffusion-likelihood" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     \log p_\theta (\underbrace{o,a}_{= z_0}) = 
@@ -964,6 +998,9 @@ Finally, adopting Gaussian posteriors permits a particularly pleasing interpreta
 </figure>
 
 Because the recorded behavior is teleoperated, measurements mostly distribute along the line $a = o + \eta, \eta \sim N(0,1)$, with $\eta$-variability accouting for minor control inconsistencies (Figure <a href="#ch4-action-vs-observation-distribution" data-reference-type="ref" data-reference="ch4-action-vs-observation-distribution">[ch4-action-vs-observation-distribution]</a>). Using Gaussian posteriors--i.e., adding Gaussian noise--effectively simulates a *Brownian motion* for the elements in the distribution’s support (in Figure <a href="#diffusion-robot-actions" data-reference-type="ref" data-reference="diffusion-robot-actions">[diffusion-robot-actions]</a>, $\mathcal O\times \mathcal A$), whereby information *diffuses away* from the samples, and comparing the diffused samples to the original data points one can derive an estimate of the total displacement induced by diffusion. Under the only assumption that the likelihood of the diffused samples is low under the original unknown data distribution, then one can effectively approximate the unkwown distribution by learning to *reverse* such displacement. This key intuition allows to write a simplified training objective:
+<span id="diffusion-simplified-loss" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 
@@ -977,6 +1014,9 @@ Because the recorded behavior is teleoperated, measurements mostly distribute al
 In this simplified (minimization) objective, the optimization process differs from <a href="#diffusion-likelihood" data-reference-type="ref" data-reference="diffusion-likelihood">[diffusion-likelihood]</a> in that, rather than maxizing $p_\theta$ directly, the parameters $\theta$ of the pairwise likelihood $p_\theta(z_{t-1} \vert z_t)$ are adjusted to *predict the total displacement* $\epsilon$ for a randomly long ($t \sim \mathcal{U}(\{1,\dots,T\}$ )) diffusion process starting from a sample of the target distribution.
 
 By learning the total displacement from a generally, uninformative corrupted sample obtained diffusing information and a sample from an unknown distribution--significant ($\Vert \epsilon \Vert > 0$) whenever input and target distribution are sufficiently different-- @hoDenoisingDiffusionProbabilistic2020 show that one can approximate the underlying distribution reversing the displacement, *denoising* samples. Interestingly, under the hypothesis real-world data belongs to a single higher dimensional manifold (Manifold Hypothesis), @permenterInterpretingImprovingDiffusion2024 show that diffusion learns the gradient of a distance function from any off-point manifold (such as perturbed, uniformative samples), and the data manifold itself. Following this gradient--i.e., denoising a sample from an uninformative distribution--corresponds to projecting back into the manifold, yielding a procedure to sample from unknown distributions by means of Euclidean projection. Indeed, under the assumption that $p_\theta (z_{t-1} \vert z_t)$ is Gaussian, then sampling $z_{t-1} \sim p_\theta(\bullet \vert z_{t})$ corresponds to computing
+<span id="diffusion-denoising-definition" style="position- absolute;">
+</span>
+
 ``` math
 \begin{align}
     z_{t-1} = \frac{1}{\sqrt{\alpha_t}} \left( z_t - \frac{\beta_t}{\sqrt{1 - \bar\alpha_t}} \epsilon_\theta(z_t, t) \right) + \sigma_t \epsilon, \quad \epsilon \sim \mathcal N(\mathbf{0}, \mathbf{I}), 
@@ -1030,6 +1070,9 @@ While the noising schedule of DMs results in a stochastic process that resembles
 </figure>
 
 In practice, FM can be applied to generative modeling by learning a vector field regressor $v_\theta(z, t)$ to approximate a given target vector field $u(t, z)$. In the particular case of DMs, $u(t, z)$ is defined as in <a href="#fm-diffusion-vector-field" data-reference-type="ref" data-reference="fm-diffusion-vector-field">[fm-diffusion-vector-field]</a>, while in priciple the target vector field can be learned to induce a particular transportation, or fixed according to OT. Given a sample from the data distribution $z_1 \sim p_1$ and a sample from an easy-to-sample prior $z_0 \sim p_0$, CFM defines a simple path between them using *linear interpolation* between samples $z_t = (1-t)z_0 + t z_1$, resulting in the target vector field $u(t, z_t) = z_1 - z_0$. Then, a FM model can be trained with the simple regression objective defined as
+<span id="flow-matching-objective" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 
@@ -1046,6 +1089,9 @@ While GMs prove useful in learning complex, high-dimensional multi-modal distrib
 On the robot learning side of their contributions, @zhaoLearningFineGrainedBimanual2023 adopt transformers as the architectural backbone to learn a *Conditional* VAE @sohnLearningStructuredOutput2015. Conditional VAEs are a variation of the more standard VAE formulation introducing a conditioning variable on sampling from the latent prior, allowing the modeling of *one-to-many* relationships between latent and data samples. Further, in stark contrast with previous work @florenceImplicitBehavioralCloning2022, @jannerPlanningDiffusionFlexible2022, @zhaoLearningFineGrainedBimanual2023 do not learn a full joint $p_\theta(o,a)$ on observation and actions. While the *policy* distribution $p_\theta(a \vert o)$ can in principle be entirely described from its joint $p_\theta(o,a)$, it is often the case that the conditional distribution is intractable when using function approximators, as $p_\theta(a \vert o) = \tfrac{p_\theta(o,a)}{\int_\mathcal Ap_\theta(o,a)}$ and the integral in the denominator is typically intractable. Instead of modeling the full joint using a vanilla VAE, @zhaoLearningFineGrainedBimanual2023 propose learning a *conditional* VAE @sohnLearningStructuredOutput2015 modeling the policy distribution directly $p (a \vert o)$.
 
 In practice, when learning from demonstrations adopting CVAEs results in a slight modification to the VAE objective in <a href="#ELBO" data-reference-type="ref" data-reference="ELBO">[ELBO]</a>, which is adapted to
+<span id="c-ELBO" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
 
@@ -1106,6 +1152,9 @@ However, the authors claim using a deterministic procedure to derive $z$ may ben
 DMs proved very effective in approximating complex highly dimensional distributions, such as distributions over images @hoDenoisingDiffusionProbabilistic2020 or videos @polyakMovieGenCast2025, thanks to their inherent capability to deal with multimodal data and training stability. In Diffusion Policy (DP), @chiDiffusionPolicyVisuomotor2024 present an application of DMs the field of robot learning, leveraging diffusion to model human expert demonstrations in a variety of simulated and real-world tasks. Similarily to Action Chunking with Transformer @zhaoLearningFineGrainedBimanual2023, @chiDiffusionPolicyVisuomotor2024 (1) adopt a modified *observation-conditioned target distribution* instead of the full joint $p(o,a)$ and (2) predict multiple actions into the future instead of a single action. Besides the intractability of the observations’ marginal $p_\theta(o)$ given $p_\theta(o,a)$, DP’s rationale for modeling the data distribution via $p_\theta(a \vert o)$ stems from the rather test-time compute intensive nature of diffusion, whereby generating actions *alongside* observations is likely to result in higher complexity and thus a likely larger number of denoising operations, which would prove ultimately pointless considering robotics applications rely on the capability to generate controls rather than reproducing observations.
 
 In practice, conditioning on observation data is achieved conditioning the added noise regressor $\epsilon_\theta$ introduced in <a href="#diffusion-simplified-loss" data-reference-type="ref" data-reference="diffusion-simplified-loss">[diffusion-simplified-loss]</a> on a stack of $T_o$ observations, resulting in the *conditional* simplified diffusion objective
+<span id="diffusion-policy-objective" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     \mathcal L(\theta) &= \mathbb{E}_{t, a_{t:t+H_a}, \epsilon} \big[
@@ -1306,6 +1355,9 @@ $\pi_0$ @blackp0VisionLanguageActionFlow2024 introduce a VLA consisting of a Mo
 Concretely, $\pi_0$ is a unified transformer with two disjoint sets of weights $\phi, \theta$. A larger VLM backbone $p_\phi$ initialized from Gemma 2.6B processes multiple image frames obtained from multiple cameras points $[\{ I_t \}_{t=1}^n]$, as well as a language instruction $[\ell_t]$ used to describe the task considered. Concurrently, a 300M-parameter *action expert* based on a similar transformer architecture is used processes the robot proprioperceptive state $q_t$ and an action chunk $a_{t:t+H_a}$ (Figure <a href="#ch5-pi0" data-reference-type="ref" data-reference="ch5-pi0">[ch5-pi0]</a>). The different expert networks operate separately in processing the respective inputs and turning them into query, key and value matrices, and only share information between each other via self-attention layers. The outputs from the VLM backbone are disregarded, while the vector field regressed by the action expert is used to iteratively refine the action process. In particular, $\pi_0$uses a *blockwise causal attention mask* over tokens belonging to three separate blocks: (1) image and language tokens $\mathcal T_i$ obtained from $[\{ I_t \}_{t=1}^n, \ell_t]$, (2) proprioperceptive tokens $\mathcal T_q$ obtained from $q_t$, and (3) the action tokens $\mathcal T_a$ for items in the chunk $a^{\tau}_{t:t+H_a}$ at time $\tau$ in the flow-matching process. Notably, *within* each block the attention operations are bidirectional, while across blocks, future blocks are masked out. Formally, this corresponds to using the attention mask $\mathbf{A} = \bordermatrix{ \mathcal{T}_i \mathcal{T}_q \mathcal{T}_a \cr \mathcal{T}_i \mathbf{1} \mathbf{0} \mathbf{0} \cr \mathcal{T}_q \mathbf{1} \mathbf{1} \mathbf{0} \cr \mathcal{T}_a \mathbf{1} \mathbf{1} \mathbf{1} \cr }, \quad \mathbf{1}: \text{Bidirectional Attention}, \ \mathbf{0}: \text{Masked Attention}$ Note how *intra*-block directional attention allows tokens to communicate freely, while *inter*-block communication is mediated by the attention mask $\mathbf{A}$. *Blockwise causal masking* effectively prevents the pre-trained perception-language tokens from attending to robotics-tokens, likely out of distribution for VLM backbones traditionally trained on large corpora of internet, non-robotics, data. Crucially, because communication is obstructed between image-language tokens, proprioperceptive and action tokens, one can cache keys and values across denoising steps at runtime time, incuring in a reduced computational footprint and faster inference.
 
 In $\pi_0$, both the VLM backbone and action expert are update using a *flow matching* loss, and in particular are updated minimizing:
+<span id="pi0-loss" style="position: absolute;">
+</span>
+
 ``` math
 \begin{align}
     \mathcal{L}(\phi, \theta) &=