| --- |
| license: mit |
| language: |
| - ru |
| - en |
| - ja |
| - zh |
| tags: |
| - singing-voice-conversion |
| - svc |
| - reflow |
| - bigvgan |
| - voice-cloning |
| - no-encoder |
| - mel-spectrogram |
| - audio |
| - music |
| pipeline_tag: audio-to-audio |
| --- |
| |
| # NICO_SVC — Singing Voice Conversion without Encoders |
| |
| ## 🎤 What is NICO_SVC? |
|
|
| A custom Singing Voice Conversion (SVC) system built from scratch. |
| **No ContentVec. No HuBERT. No Whisper. No encoders at all.** |
|
|
| # NICO_SVC — Singing Voice Conversion without Encoders |
| |
| ## 🎤 What is NICO_SVC? |
|
|
| A custom Singing Voice Conversion (SVC) system built from scratch. |
| **No ContentVec. No HuBERT. No Whisper. No encoders at all.** |
|
|
| Unlike every existing SVC (RVC, Diff-SVC, SO-VITS-SVC, DDSP-SVC, Reflow-VAE-SVC), |
| NICO_SVC does NOT pass audio through a 16kHz speech encoder. |
| Instead, it works directly with Mel spectrograms at native 44.1kHz sample rate. |
| |
| ## 🔥 Why? |
| |
| Every SVC system in 2023-2025 uses a speech encoder (ContentVec/HuBERT/Whisper) |
| that internally operates at 16kHz. This creates a bottleneck: |
| |
| ``` |
| Voice 48kHz → Encoder 16kHz → Lost upper frequencies → Vocoder guesses → Plastic sound |
| ``` |
| |
| NICO_SVC eliminates this entirely: |
|
|
| ``` |
| Voice 44.1kHz → Mel spectrogram (direct) → Reflow model → BigVGAN v2 → Real 44.1kHz output |
| ``` |
|
|
| ## 🏗️ Architecture |
|
|
| | Component | Details | |
| |-----------|---------| |
| | **Input** | Mel spectrogram 44.1kHz (128 bands, hop=512, n_fft=2048) | |
| | **F0 Extractor** | RMVPE | |
| | **Speaker Embedding** | Learnable (trained per speaker) | |
| | **Model** | Reflow (Rectified Flow Matching) with Conv + Attention blocks | |
| | **Parameters** | ~100M | |
| | **Vocoder** | BigVGAN v2 44kHz 128band (NVIDIA, pretrained) | |
| | **Output** | 44.1kHz WAV | |
| | **Training** | Single-speaker fine-tune, no pretrain needed | |
| | **Inference** | ~50 Euler steps, real-time capable | |
| |
| ## 📊 Key Differences from Existing SVC |
| |
| | Feature | RVC / Diff-SVC / SO-VITS | NICO_SVC | |
| |---------|--------------------------|----------| |
| | Speech Encoder | ContentVec 16kHz | ❌ None | |
| | Max Audio Frequency | ~8kHz (encoder limit) | **22kHz** (full Mel) | |
| | Pretrain Required | Yes (multi-speaker) | **No** | |
| | Mel Source | From encoder output | **Direct from WAV** | |
| | Vocoder | HiFi-GAN / NSF-HiFiGAN | **BigVGAN v2** (NVIDIA) | |
| | Architecture | Diffusion / DDSP | **Reflow** (faster, cleaner) | |
| | Sound Quality | Plastic, missing harmonics | **Natural, full spectrum** | |
|
|
| ## 🎯 Design Principles |
|
|
| 1. **No encoders** — they all operate at 16kHz and destroy timbre |
| 2. **Native 44.1kHz** — at every stage of the pipeline |
| 3. **Single-speaker focus** — one model = one voice = perfect copy |
| 4. **Fine-tune only** — no pretrain means no "average voice" contamination |
| 5. **Free tools only** — runs on Google Colab free tier (T4 GPU) |
| 6. **Fast inference** — 2-3 minutes per cover, not 18 minutes |
|
|
| ## 📦 Pipeline |
|
|
| ### Preprocessing: |
| ``` |
| WAV 44.1kHz → Mel spectrogram (torchaudio) |
| → F0 extraction (RMVPE) |
| → Speaker embedding (learnable) |
| ``` |
|
|
| ### Training: |
| ``` |
| Mel (target) + noise → Reflow model learns to denoise |
| Conditioning: F0 + Speaker Embedding |
| Loss: MSE between predicted and target flow vectors |
| Optimizer: AdamW + fp16 mixed precision |
| ``` |
|
|
| ### Inference (Cover): |
| ``` |
| Input acapella → RMVPE (extract F0) |
| → F0 * 2^(key/12) (pitch shift) |
| → Reflow model (generate Mel with target speaker) |
| → BigVGAN v2 (Mel → WAV) |
| → Output cover |
| ``` |
|
|
| ## 🎤 Training Data |
|
|
| - **Speaker:** SomNICO_Ai (lyric tenor, natural flat tone without vibrato) |
| - **Dataset:** up to 1951 files, ~3.92 hours |
| - **Languages:** Russian, English, Japanese, Chinese |
| - **Quality:** 48kHz WAV, single microphone, clean recording |
| - **Content:** Pure singing, no background noise |
| |
| ## 🔧 Requirements |
| |
| - Google Colab (free tier, T4 GPU) |
| - Google Drive (for checkpoints and data) |
| - Python 3.12 |
| - PyTorch 2.x |
| - torchaudio, librosa, BigVGAN v2, RMVPE |
| |
| ## 📈 Results |
| |
| | Metric | Value | |
| |--------|-------| |
| | Training loss | < 0.5 (272 files, 19M model) | |
| | Perceptual quality | Near-indistinguishable from original on test set | |
| | Inference speed | ~50 steps, real-time on T4 | |
| | Checkpoint size | ~400MB (100M model) | |
| |
| ## 👤 Author |
| |
| - **Developer:** SomNICO_Ai / Kedo |
| - **Age:** 20 |
| - **Vocal experience:** 12 years (since age 6) |
| - **AI voice experience:** 5 years, 143+ RVC models trained |
| - **YouTube:** SomNICO_Ai |
| - **HuggingFace:** 0qwpifs |
| |
| ## 📜 Philosophy |
| |
| > "I'm tired of using other people's tools. |
| > Every SVC crushes my voice through 16kHz encoders. |
| > I hear my voice differently — it has more detail, more character. |
| > NICO_SVC was built to preserve EVERYTHING." |
|
|
| ## 🗓️ Timeline |
|
|
| - **Day 1:** Full pipeline from scratch — Mel, F0, Reflow, BigVGAN working |
| - **Week 1:** 100M model + Attention, first covers |
| - **Week 2-3:** Volume conditioning, TensorBoard, production quality |
| - **Goal:** Voice clone indistinguishable from real recording |
|
|
| ## License |
|
|
| MIT |
|
|
| ## Credits |
|
|
| - BigVGAN v2: NVIDIA (pretrained vocoder) |
| - RMVPE: yxlllc (F0 extraction) |
| - Architecture inspired by Rectified Flow Matching papers |
| - Built entirely in Google Colab, free tier |
| ``` |
