mcamara/conformer-landmark-detector

Conformer Acoustic Landmark Detector

Frame-level detector of acoustic landmarks (Stevens, 2002) in speech audio: abrupt acoustic events tied to articulatory gestures. Given a waveform, the model outputs per-frame probabilities (10 ms resolution) for 8 landmark types plus background, from which landmark times are extracted by per-class peak picking.

This is the base variant, trained only on the real manually-annotated corpus (1,839 recordings: 678 VCV syllables + 1,161 English words, 3 speakers). A variant pretrained on ~265k synthetic words is available at conformer-landmark-detector-synthetic-pretrained.

Landmark types

Type	Description	Precision	Recall	F1@20ms	Threshold
V	Vowel — peak of vowel/sonorant	0.545	0.532	0.538	0.45
G	Glide — vowel/consonant transition	0.743	0.647	0.692	0.75
Sc	Stop closure	0.826	0.849	0.837	0.30
Sr	Stop release burst	0.849	0.854	0.852	0.40
Fc	Fricative closure (frication onset)	0.833	0.787	0.809	0.55
Fr	Fricative release (frication offset)	0.867	0.867	0.867	0.50
Nc	Nasal closure (murmur onset)	0.824	0.955	0.884	0.65
Nr	Nasal release (murmur offset)	0.826	0.844	0.835	0.70

Macro F1 @ 20 ms tolerance (held-out test set): 0.789

Precision/recall/F1 are landmark-level at ±20 ms tolerance on a held-out test split (10%, speaker-stratified). Detection thresholds were tuned per class on a separate validation split (10%) — the test set was never used for model selection or threshold tuning.

Architecture

Conformer encoder (12 blocks, d_model 256, 4 heads, FF 1024, conv kernel 31) over 80-band log-mel spectrograms (16 kHz, 25 ms window, 10 ms hop), with a linear classification head over 9 classes. ~18.3M parameters. Trained with Gaussian soft labels (per-class σ: V 20 ms, G 15 ms, Fc/Fr 12 ms, Sc/Sr/Nc/Nr 10 ms), class-weighted cross-entropy and label smoothing 0.1.

Usage

With the training repository (MateoCamara/conformer-landmark-detector):

pip install -r requirements.txt
python scripts/predict.py --audio path/to/audio.wav --checkpoint mcamara/conformer-landmark-detector

Or programmatically:

from src.utils.hub import load_landmark_model

model, config, thresholds = load_landmark_model("mcamara/conformer-landmark-detector", device="cuda")
# model: LandmarkClassifier over (batch, frames, 80) log-mel features
# config: mel front-end + label map + sigmas; thresholds: per-class operating points

The output JSON of predict.py contains {type, time_ms, confidence} entries.

Files

• model.safetensors — model weights
• config.json — architecture, mel front-end, label map, soft-label sigmas
• thresholds.json — per-class detection thresholds tuned on validation

Citation

@inproceedings{camara2026conformer-landmarks,
  title={Conformer-based Acoustic Landmark Detection},
  author={C\'amara, Mateo},
  booktitle={Interspeech},
  year={2026}
}

References

• Stevens, K. N. (2002). Toward a model for lexical access based on acoustic landmarks and distinctive features. JASA, 111(4), 1872-1891.
• Gulati, A., et al. (2020). Conformer: Convolution-augmented Transformer for Speech Recognition. Interspeech 2020.