wav2vec2-lv-60-espeak-cv-ft (fp16 ONNX) + CMU Pronouncing Dictionary, mirrored for Rostrum
Meta's wav2vec2 phoneme recogniser fine-tuned on CommonVoice (espeak phones), as the fp16 ONNX export by onnx-community, with its vocabulary; plus the CMU Pronouncing Dictionary (cmudict-7479086/cmudict.dict) for English spellings.
This repository is a pinned, public copy of the exact files the Rostrum apps download. Rostrum fetches each file at a fixed commit of this repository and checks its SHA-256 before use.
What Rostrum uses it for
2D's lip sync: it hears the speech sounds in a recorded voice and when each is said, so the mouth shapes are keyed to them. It only listens.
Rostrum uses it for analysis only, never generation: it reads what is in a recording or a picture and makes no new sound, voice or image.
Files
| File | Bytes | SHA-256 |
|---|---|---|
model_fp16.onnx |
632,331,247 | 7ff53b55abe98dc26da93f445e39d33b95491d4f0292e1fef5bcbfa40aa22932 |
vocab.json |
4,637 | d732ab2456c0c017930001dc9af0b41b3b93d25b2eb9740bf9d925508d7d87d0 |
cmudict-7479086/cmudict.dict |
3,618,488 | 81917843c7f44ce2b094ac63873c2c7a4cf802040792c455ba3ca406891c3d22 |
In Rostrum's models folder they go under wav2vec2-lv60-espeak-fp16/ (and cmudict-7479086/ for the dictionary).
Source
| What | Where | Commit / version |
|---|---|---|
| Exact model files (unchanged copy) | https://huggingface.co/onnx-community/wav2vec2-lv-60-espeak-cv-ft-ONNX | c69750f5043e5e1f8a71ab95dd3b98338c280c92 (onnx/model_fp16.onnx, vocab.json) |
| Original model, Meta | https://huggingface.co/facebook/wav2vec2-lv-60-espeak-cv-ft | ae45363bf3413b374fecd9dc8bc1df0e24c3b7f4 (main when mirrored) |
| CMU Pronouncing Dictionary (unchanged copy) | https://github.com/cmusphinx/cmudict | 74790861f652b15e4ac49015a90074ad62a27690 |
Authors
Qiantong Xu, Alexei Baevski, Michael Auli (Meta AI, "Simple and Effective Zero-shot Cross-lingual Phoneme Recognition", arXiv:2109.11680); onnx-community (ONNX export); Carnegie Mellon University (CMU Pronouncing Dictionary).
Licence
The model and its export are Apache-2.0 (LICENSE). The CMU Pronouncing Dictionary is BSD-2-Clause, Copyright (C) 1993-2015 Carnegie Mellon University (cmudict-7479086/LICENSE). Files redistributed unchanged.
Model tree for rostrumstudios/lipsync-phones
Base model
facebook/wav2vec2-lv-60-espeak-cv-ft