Commit ·
e5bc30f
1
Parent(s): af89cf1
add fig:token-subtask figure
Browse files
app.py
CHANGED
|
@@ -393,12 +393,17 @@ Shuffle (identity-preserving, position-destroying) is more harmful than random o
|
|
| 393 |
\subsection{Token-subtask heatmap}
|
| 394 |
\label{app:heatmap}
|
| 395 |
|
| 396 |
-
|
| 397 |
-
|
| 398 |
-
|
| 399 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 400 |
|
| 401 |
-
Of the 30 tokens in the codebook,
|
| 402 |
Each active token concentrates on a narrow slice of the subtask space: the dominant subtask accounts for ${\geq}70\%$ of that token's occurrences in the majority of cases.
|
| 403 |
Tokens are also \emph{position-locked}: each token appears predominantly at one or two answer positions ($d_0$-$d_6$), rarely crossing position boundaries.
|
| 404 |
Representative examples are shown in Table~\ref{tab:token-profiles}:
|
|
|
|
| 393 |
\subsection{Token-subtask heatmap}
|
| 394 |
\label{app:heatmap}
|
| 395 |
|
| 396 |
+
\begin{figure}[h]
|
| 397 |
+
\centering
|
| 398 |
+
\includegraphics[width=0.95\linewidth]{experiments/03_token_subtask_heatmap/fig_token_subtask.pdf}
|
| 399 |
+
\caption{Token--subtask heatmap for \texttt{2L/1H/128d} (100K).
|
| 400 |
+
Each cell shows $P(\text{subtask} \mid \text{token})$ over 2{,}600 held-out examples.
|
| 401 |
+
Rows are the 23 active tokens (of 30); columns are the 10 Quirke subtask labels.
|
| 402 |
+
Most tokens concentrate strongly on 1--2 subtasks, confirming specialist routing.}
|
| 403 |
+
\label{fig:token-subtask}
|
| 404 |
+
\end{figure}
|
| 405 |
|
| 406 |
+
Of the 30 tokens in the codebook, 23 appear in the held-out evaluation set.
|
| 407 |
Each active token concentrates on a narrow slice of the subtask space: the dominant subtask accounts for ${\geq}70\%$ of that token's occurrences in the majority of cases.
|
| 408 |
Tokens are also \emph{position-locked}: each token appears predominantly at one or two answer positions ($d_0$-$d_6$), rarely crossing position boundaries.
|
| 409 |
Representative examples are shown in Table~\ref{tab:token-profiles}:
|