amirali1985 commited on
Commit
e5bc30f
·
1 Parent(s): af89cf1

add fig:token-subtask figure

Browse files
Files changed (1) hide show
  1. app.py +10 -5
app.py CHANGED
@@ -393,12 +393,17 @@ Shuffle (identity-preserving, position-destroying) is more harmful than random o
393
  \subsection{Token-subtask heatmap}
394
  \label{app:heatmap}
395
 
396
- % [PLACEHOLDER: insert token-subtask heatmap figure here]
397
- % Figure: P(subtask | token) heatmap for all active tokens × 10 subtask labels.
398
- % Generate with: python experiments/03_token_subtask_heatmap/run.py \
399
- % -model add_sub_sorl_v1_abs30_K1_100K_2L1H128d
 
 
 
 
 
400
 
401
- Of the 30 tokens in the codebook, 18 appear in the held-out evaluation set.
402
  Each active token concentrates on a narrow slice of the subtask space: the dominant subtask accounts for ${\geq}70\%$ of that token's occurrences in the majority of cases.
403
  Tokens are also \emph{position-locked}: each token appears predominantly at one or two answer positions ($d_0$-$d_6$), rarely crossing position boundaries.
404
  Representative examples are shown in Table~\ref{tab:token-profiles}:
 
393
  \subsection{Token-subtask heatmap}
394
  \label{app:heatmap}
395
 
396
+ \begin{figure}[h]
397
+ \centering
398
+ \includegraphics[width=0.95\linewidth]{experiments/03_token_subtask_heatmap/fig_token_subtask.pdf}
399
+ \caption{Token--subtask heatmap for \texttt{2L/1H/128d} (100K).
400
+ Each cell shows $P(\text{subtask} \mid \text{token})$ over 2{,}600 held-out examples.
401
+ Rows are the 23 active tokens (of 30); columns are the 10 Quirke subtask labels.
402
+ Most tokens concentrate strongly on 1--2 subtasks, confirming specialist routing.}
403
+ \label{fig:token-subtask}
404
+ \end{figure}
405
 
406
+ Of the 30 tokens in the codebook, 23 appear in the held-out evaluation set.
407
  Each active token concentrates on a narrow slice of the subtask space: the dominant subtask accounts for ${\geq}70\%$ of that token's occurrences in the majority of cases.
408
  Tokens are also \emph{position-locked}: each token appears predominantly at one or two answer positions ($d_0$-$d_6$), rarely crossing position boundaries.
409
  Representative examples are shown in Table~\ref{tab:token-profiles}: