Method

LatentASR adds continuous latent test-time scaling to a frozen ASR backbone. The implementation wraps Qwen/Qwen3-ASR-0.6B with two lightweight modules:

Latent Adapter: converts decoder hidden states into bounded latent deltas.
Value Head: predicts whether additional latent compute is useful and enables dynamic halting.

Prefix Layout

The default layout is prefix mode:

system/user/audio prompt
assistant language <asr_text>
<|latent|> <|latent|> <|latent|> <|latent|>
transcript tokens

Transcript loss masks all prompt and latent positions.

Stable Injection

Each latent delta is stabilized by three mechanisms:

Bounded delta: delta_proj(h_k) is L2-normalized and scaled by a learned per-step scalar.
Sigmoid gate: a zero-initialized gate starts at 0.5 and learns how much of the delta to apply.
Fixed embedding anchor: the injected vector is embedding(<|latent|>) + gate * delta, keeping the input near a real token embedding.

These mechanisms correspond to the ablation flags:

LATENT_USE_BOUNDED_DELTA=1
LATENT_USE_INJECTION_GATE=1
LATENT_USE_EMBEDDING_ANCHOR=1

Value Head Target

For each training utterance, the value head predicts the latent-vs-baseline accuracy gain. The default target is:

0.9 * tanh(3.0 * (latent_accuracy - baseline_accuracy))

If both baseline and latent accuracies are zero for an utterance, the target uses a clamped CE-difference fallback:

0.9 * tanh(0.5 * clamp(baseline_ce - latent_ce, -2, 2))

With probability VALUE_FORCED_NEG_PROB=0.3 per minibatch, the target is flipped to -|target| for conservative calibration.

The current implementation supervises the value prediction at every produced latent state, matching the fact that inference can halt after any step.

Dynamic Halting

At inference time:

The initial value prediction gates the whole loop.
If v_0 < theta, all latent tokens are removed and generation falls back to the frozen baseline.
After each latent step, the value head is re-evaluated.
If v_k < theta, unused latent tokens are removed before decoding.

The paper uses theta=0.0 for the deployed setting.