lf2ar-speech-360m

Speech LF²AR checkpoint from LF²AR: Accounting for Layerwise Dynamics to Improve Multimodal Adaptation of Language Models. This model has 401,665,984 parameters and uses dynamic attention residuals.

from interleaved_lm import PerceptionExpressionAdaptedTextLM
from interleaved_lm.audio import generate_speech

model = PerceptionExpressionAdaptedTextLM.from_pretrained(
    "tiagoCuervo/lf2ar-speech-360m", device="cuda"
)
codes = generate_speech(model, "A short story about the moon", max_new_tokens=250)

Speech is represented by consecutive-run-collapsed 25 Hz mHuBERT layer-11 K-means units (K=500; EOS=500; PAD=501). A vocoder is not bundled.

The text backbone is initialized from SmolLM. Speech-tokenizer and vocoder artifacts are credited in the Interleaved-LM third-party notices.

Evaluation

Task	Direction	Accuracy
sstorycloze	audio-audio	57.19
sstorycloze	text-text	68.63
sstorycloze	audio-text	61.89
sstorycloze	text-audio	62.27
tstorycloze	audio-audio	84.61
tstorycloze	text-text	91.39
tstorycloze	audio-text	84.98
tstorycloze	text-audio	80.81

See provenance.json for exact hashes and evaluation metadata.