Agreement to the Terms of Use
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
Please read the The Terms of Use in Model Card before requesting access to the models and source code.
Log in or Sign Up to review the conditions and access this model content.
FIRNet: Fast Neural Vocoder With Fundamental Frequency Control Using Finite Impulse Response Filters
We release official pretrained models of FIRNet, a DSP-based fast neural vocoder with voice factor control. FIRNet can control pitch bias and scale, voice depth (formant shift), and aperiodicity by modifying vocoder parameters. We provide two pretrained models for 24-kHz and 48-kHz sampling rates. FIRNet can be used not only for speech generation but also for data augmentation and perturbation during training.
Requirements
- Python 3.10+
- PyTorch (
torch >=2.7)
Install project dependencies:
pip install -r ./requirements.txt
Repository layout
- checksum: checksum information for .pt files of the pretrained models
- models: pretrained model weights and model definition files
- src: Python script set
- Network blocks for FIRNet
- Waveform analysis and synthesis
- sample script for voice factor control
Examples of usage for sample script
- Analysis-synthesis (24 kHz sampling rate)
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav
- Analysis-synthesis (48 kHz sampling rate)
python3 ./src/bin/sample_anasyn.py --sample_rate 48000 -I input48kHz.wav -O output48kHz.wav
- Pitch-modified voice
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --f0_shift 1.2 --f0_emph 0.7
- Constant F0 voice
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --fixed_f0 100
- Deeper voice
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --alpha -0.15
- Hoarse voice
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --beta 0.20
Model specifications
Model architecture
These pretrained models are parallel FIRNet models based on ASRU2025 version. The differences from the original version are as follows:
- Use ConvNeXt blocks instead of its core blocks in shared encoder.
- Frame shift size (hop size) is changed to 10 ms from 5 ms.
- The number of filter taps is changed to 384 (24 kHz) and 768 (48 kHz).
- Use a combination of spectral criteria instead of the mel-spectrogram loss for the generator loss.
Training data
For traning the pretrained models, we employ the following speech datasets:
- A part of commonvoice22_sidon, which includes 134 languages.
- Hi-Fi-Captain (currently unpublished)
The Terms of Use
Source code
- The FIRNet source code is distributed under the BSD 3-Clause license.
- The BSD license applies only to the source code.
- See
LICENSE_FOR_PYTHON_CODESfor the full license terms.
Pretrained models
- The pretrained models are distributed under CC BY-NC-SA 4.0, in accordance with the licensing conditions of the training corpora.
- The pretrained models are intended for research and educational purposes.
- For commercial use, please contact us in advance at astl-service@ml.nict.go.jp. Higher-quality pretrained models are available for commercial applications, particularly for 48-kHz speech synthesis.
- See
LICENSE_FOR_MODELSfor the full license terms.
References
When you cite the FIRNet in your paper, please cite the following papers:
[1] Yamato Ohtani, Takuma Okamoto, Tomoki Toda and Hisashi Kawai, "Fast Neural Vocoder with Fundamental Frequency Control Using Finite Impulse Response Filters," in IEEE Transactions on Audio, Speech and Language Processing, vol. 33, pp. 1893-1906, 2025. [link]
[2] Yamato Ohtani, Takuma Okamoto, Tomoki Toda, Hisashi Kawai, "Voice factor control using FIR-based fast neural vocoder for speech generation applications," Proc. ASRU, pp. 1-4, Dec. 2025. [link]
[3] Takuma Okamoto, Yamato Ohtani, Sota Shimizu, Tomoki Toda, Hisashi Kawai, "Challenge of Singing Voice Synthesis Using Only Text-To-Speech Corpus With FIRNet Source-Filter Neural Vocoder," Proc. Interspeech 2024, pp. 1870-1874, September 2024. [link]
[4] Yamato Ohtani, Takuma Okamoto, Tomoki Toda and Hisashi Kawai, "FIRNet: Fundamental frequency controllable fast neural vocoder with trainable finite impulse response filter," Proc. ICASSP 2024, pp. 10871-10875, April 2024. [link]