Project intermediate hidden states directly into vocabulary space layer-by-layer using the unembed matrix $W_U$. Watch the model's intermediate "best guesses" shift from early syntax representations to late semantic target resolution.
{/* Input Interface */}Projecting hidden states via the logit lens illustrates that the transformer does not solve sequence completion instantly. In early layers (0–4), representations are highly syntactic (predicting punctuation or connector words). The correct name prediction (e.g. {lensData?.io_name}) emerges sharply around layers 7–9, directly coinciding with S-inhibition and name-mover attention interventions.
For layer {"$L$"}, the logit distribution is extracted by projecting the residual stream representation through the final layer normalization followed by the standard unembed weight: {"$\\text{logits}_L = \\text{LN}(x_L) \\cdot W_U$"}. Projecting onto intermediate layers provides a highly precise approximation of the model's internal coordinate updates.