File size: 3,132 Bytes
db39dbc | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | \section{Beyond Compositional Data}\label{sec:ml}
We establish a connection between compositional data and probabilistic modeling via shared underlying geometry. Further analysis and validation may be of independent interest.
\textbf{Aitchison geometry as quotient.}
Compositional data identifies vectors up to equivalence classes $[\mathbf{c}]$ induced by $\mathbf{c} \sim_{c} \lambda\mathbf{c}$ for $\lambda > 0$ (i.e., scaling), since only ratios carry information. We observe that the quotient $\mathbb{R}^d_{>0} / {\sim_{c}}$ is the Aitchison simplex, whose tangent space is $\mathcal{H}$ via CLR {\color{black}\citep{aitchison1982statistical, egozcue2003isometric}}.
\textbf{Probabilistic modeling.}
Consider a model $f_\theta$ outputting logits $\mathbf{z} = f_\theta(\mathbf{x}) \in \mathbb{R}^d$, with predicted distribution $\mathbf{p} = \mathrm{softmax}(\mathbf{z})$ trained via cross-entropy. Since softmax is \emph{shift-invariant}, i.e., $\mathrm{softmax}(\mathbf{z} + c\mathbf{1}) = \mathrm{softmax}(\mathbf{z})$, this induces an equivalence relation $\mathbf{z} \sim_{\ell} \mathbf{z} + c\mathbf{1}$. The loss and predictions are invariant to shifts along $\mathbf{1}$.
\begin{proposition}\label{prop:logit-clr}
Let $\mathcal{L} = \mathbb{R}^d / {\sim_{\ell}}$ be the quotient of logits under shift equivalence. Then $\mathcal{L} \cong \mathcal{H}$ (isomorphism). For $\mathbf{p} = \mathrm{softmax}(\mathbf{z})$, we have $\mathbf{z} - \bar{z}\mathbf{1} = \mathrm{clr}(\mathbf{p}), \quad \text{where } \bar{z} = \tfrac{1}{d}\textstyle\sum_i z_i$, and $[\mathbf{z}] \to \mathbf{z} - \bar{z}\mathbf{1}$ is well-defined.
\end{proposition}
{\color{black}The individual components (i.e., the CLR hyperplane, centering map~\citep{egozcue2003isometric}, and softmax shift-invariance) are well-known. Proposition~\ref{prop:logit-clr} newly establishes that} after quotienting by the shift symmetry, logit space aligns with the CLR/Aitchison tangent space.
\textbf{Implications.}
Many datasets have tree structure over classes, e.g., a superclass hierarchy for \texttt{CIFAR-100} and WordNet for \texttt{ImageNet} \citep{deng2009imagenet, miller1995wordnet}. Given such a tree $\mathcal{T}$, PolyILR can transform model logits as $\mathbf{a} = \mathbf{V}^\top \mathbf{z}$ in tree-aligned coordinates where each component corresponds to a node and contrast. Since $\mathbf{p} = \mathrm{softmax}(\mathbf{V}\mathbf{a})$, predictions can be analyzed in this interpretable space. For instance, the gradient w.r.t.\ logits is $\nabla_{\mathbf{z}} \ell = \mathbf{p} - \mathbf{e}_y$, with one-hot target $\mathbf{e}_y$. The gradient w.r.t.\ PolyILR coordinates thus is $\nabla_{\mathbf{a}} \ell = \mathbf{V}^\top \nabla_{\mathbf{z}} \ell = \mathbf{V}^\top (\mathbf{p} - \mathbf{e}_y)$. Each $(\nabla_{\mathbf{a}} \ell)_{(u,m)}$ measures how strongly the loss pushes probability mass along contrast $m$ at node $u$, localizing model errors in $\mathcal{T}$. See Appendix~\ref{appdx:proof_algo} and~\ref{appdx:ml} for proof and preliminary results. Practical applications to model training or analysis remain an open direction.
|