Hugging Face's logo

ASTREC-NICT
/
FIRNet

speech
audio
vocoder
speech-synthesis
neural-vocoder
firnet
Model card Files
xet
FIRNet / README.md
MSCL-NICT's picture
MSCL-NICT
Update README.md
8ebffc4 verified
preview code
|
Raw
History Blame
5.16 kB
metadata 
license: cc-by-nc-sa-4.0
tags:
  - speech
  - audio
  - vocoder
  - speech-synthesis
  - neural-vocoder
  - firnet
gated: true
extra_gated_heading: Agreement to the Terms of Use
extra_gated_prompt: >
  Please read the **The Terms of Use in Model Card** before requesting access to
  the models and source code.
extra_gated_fields:
  I have read and agree to The Terms of Use in Model Card: checkbox
extra_gated_button_content: Agree and request access

FIRNet: Fast Neural Vocoder With Fundamental Frequency Control Using Finite Impulse Response Filters

We release official pretrained models of FIRNet, a DSP-based fast neural vocoder with voice factor control. FIRNet can control pitch bias and scale, voice depth (formant shift), and aperiodicity by modifying vocoder parameters. We provide two pretrained models for 24-kHz and 48-kHz sampling rates. FIRNet can be used not only for speech generation but also for data augmentation and perturbation during training.

Requirements

  • Python 3.10+
  • PyTorch (torch >=2.7)

Install project dependencies:

pip install -r ./requirements.txt

Repository layout

  • checksum: checksum information for .pt files of the pretrained models
  • models: pretrained model weights and model definition files
  • src: Python script set
    • Network blocks for FIRNet
    • Waveform analysis and synthesis
    • sample script for voice factor control

Examples of usage for sample script

  • Analysis-synthesis (24 kHz sampling rate)
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav
  • Analysis-synthesis (48 kHz sampling rate)
python3 ./src/bin/sample_anasyn.py --sample_rate 48000 -I input48kHz.wav -O output48kHz.wav
  • Pitch-modified voice
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --f0_shift 1.2 --f0_emph 0.7
  • Constant F0 voice
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --fixed_f0 100
  • Deeper voice
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --alpha -0.15
  • Hoarse voice
python3 ./src/bin/sample_anasyn.py -I input.wav -O output.wav --beta 0.20

Model specifications

Model architecture

These pretrained models are parallel FIRNet models based on ASRU2025 version. The differences from the original version are as follows:

  • Use ConvNeXt blocks instead of its core blocks in shared encoder.
  • Frame shift size (hop size) is changed to 10 ms from 5 ms.
  • The number of filter taps is changed to 384 (24 kHz) and 768 (48 kHz).
  • Use a combination of spectral criteria instead of the mel-spectrogram loss for the generator loss.

Training data

For traning the pretrained models, we employ the following speech datasets:

  • A part of commonvoice22_sidon, which includes 134 languages.
  • Hi-Fi-Captain (currently unpublished)

The Terms of Use

Source code

  • The FIRNet source code is distributed under the BSD 3-Clause license.
  • The BSD license applies only to the source code.
  • See LICENSE_FOR_PYTHON_CODES for the full license terms.

Pretrained models

  • The pretrained models are distributed under CC BY-NC-SA 4.0, in accordance with the licensing conditions of the training corpora.
  • The pretrained models are intended for research and educational purposes.
  • For commercial use, please contact us in advance at astl-service@ml.nict.go.jp. Higher-quality pretrained models are available for commercial applications, particularly for 48-kHz speech synthesis.
  • See LICENSE_FOR_MODELS for the full license terms.

References

When you cite the FIRNet in your paper, please cite the following papers:

[1] Yamato Ohtani, Takuma Okamoto, Tomoki Toda and Hisashi Kawai, "Fast Neural Vocoder with Fundamental Frequency Control Using Finite Impulse Response Filters," in IEEE Transactions on Audio, Speech and Language Processing, vol. 33, pp. 1893-1906, 2025. [link]

[2] Yamato Ohtani, Takuma Okamoto, Tomoki Toda, Hisashi Kawai, "Voice factor control using FIR-based fast neural vocoder for speech generation applications," Proc. ASRU, pp. 1-4, Dec. 2025. [link]

[3] Takuma Okamoto, Yamato Ohtani, Sota Shimizu, Tomoki Toda, Hisashi Kawai, "Challenge of Singing Voice Synthesis Using Only Text-To-Speech Corpus With FIRNet Source-Filter Neural Vocoder," Proc. Interspeech 2024, pp. 1870-1874, September 2024. [link]

[4] Yamato Ohtani, Takuma Okamoto, Tomoki Toda and Hisashi Kawai, "FIRNet: Fundamental frequency controllable fast neural vocoder with trainable finite impulse response filter," Proc. ICASSP 2024, pp. 10871-10875, April 2024. [link]