# 🐟 Fish Speech S2 Pro — Quantization Experiments

**Comprehensive multi-phase quantization of Fish Audio S2 Pro (4.56B params) with voice cloning samples.**

## Model Architecture

| Component | Layers | Dim | Heads | Params | Size |
|-----------|--------|-----|-------|--------|------|
| **Slow AR** (LLM backbone) | 36 | 2560 | 32 (GQA, 8 local) | ~4.0B | ~8.5 GB |
| **Fast AR** (acoustic decoder) | 4 | 2560 | 32 (GQA, 8 local) | ~0.4B | ~0.8 GB |
| **DAC Codec** (RVQ) | — | — | — | — | 1.7 GB |
| **Total** | — | — | — | **4.56B** | **~10.8 GB** |

## Quantization Experiments

### Phase 1: Proven Approaches (Zero/Near-Zero Quality Loss)

| ID | Method | Target | Expected Size | Compression | Status |
|----|--------|--------|--------------|-------------|--------|
| **1a** | FP8 (per-row symmetric) | Slow AR | **~6.8 GB** | 1.60x | ✅ Proven ([drbaph/s2-pro-fp8](https://huggingface.co/drbaph/s2-pro-fp8)) |
| **1b** | INT4 (group=128) | Slow AR | **~4.8 GB** | 2.24x | ✅ Proven ([baicai1145/s2-pro-w4a16](https://huggingface.co/baicai1145/s2-pro-w4a16)) |

### Phase 2: Aggressive Approaches (Potential Quality Tradeoffs)

| ID | Method | Target | Expected Size | Compression | Status |
|----|--------|--------|--------------|-------------|--------|
| **2a** | INT4 (group=128) | All | **~4.9 GB** | 2.19x | 🔬 Experimental |
| **2b** | INT8 (per-row) | Slow AR | **~6.8 GB** | 1.60x | ✅ Safe |
| **2c** | INT3 (group=128) | Slow AR | **~4.3 GB** | 2.52x | ⚠️ Risky |

### Phase 3: Extreme Approaches (Quality Degradation Expected)

| ID | Method | Target | Expected Size | Compression | Status |
|----|--------|--------|--------------|-------------|--------|
| **3a** | INT2 (group=64) | Slow AR | **~3.8 GB** | 2.88x | ❌ Likely degraded |
| **3b** | INT2 (group=64) | All | **~3.8 GB** | 2.88x | ❌ Likely degraded |

## Quick Start

### Prerequisites
- CUDA GPU with ≥24GB VRAM (A100 40/80GB recommended)
- Python 3.10+

### Run All Phases
```bash
# Clone fish-speech and this experiment repo
git clone https://github.com/fishaudio/fish-speech.git
cd fish-speech-experiments

# Install dependencies
pip install torch einops loguru ormsgpack hydra-core omegaconf safetensors torchaudio soundfile

# Run all phases
python scripts/quantize.py --phase all --output ./output

# Or run individual phases
python scripts/quantize.py --phase 1a    # FP8 only
python scripts/quantize.py --phase 1b    # INT4 only
python scripts/quantize.py --phase 2c    # INT3 only
```

### Upload to Hub
```bash
# Requires HF write token
huggingface-cli login
python scripts/upload_to_hub.py --output ./output
```

## Voice Cloning

Each phase generates two audio samples:
1. **`{phase}_tts.wav`** — Text-to-speech without reference
2. **`{phase}_clone.wav`** — Voice cloning from celebrity reference

The reference audio is generated from the base model using a Morgan Freeman-style deep narration:
> *"Good morning. I want to tell you something about the universe. Every atom in your body came from a star that exploded. We are all made of star stuff."`

## Existing Quantized Models on HuggingFace

| Model | Method | Size | Link |
|-------|--------|------|------|
| fishaudio/s2-pro | BF16 (original) | 10.8 GB | [Link](https://huggingface.co/fishaudio/s2-pro) |
| drbaph/s2-pro-fp8 | FP8 | 6.2 GB | [Link](https://huggingface.co/drbaph/s2-pro-fp8) |
| baicai1145/s2-pro-w4a16 | GPTQ INT4 | ~5.5 GB | [Link](https://huggingface.co/baicai1145/s2-pro-w4a16) |
| rodrigomt/s2-pro-gguf | GGUF (q2-q8) | 2.4-9.2 GB | [Link](https://huggingface.co/rodrigomt/s2-pro-gguf) |

### GGUF Sizes (from rodrigomt/s2-pro-gguf)
| Quant | Size | Notes |
|-------|------|-------|
| f16 | 9.2 GB | Lossless |
| q8_0 | 5.2 GB | Near-lossless |
| q6_k | 4.2 GB | Minimal loss |
| q5_k_m | 3.8 GB | Slight loss |
| q4_k_m | 3.3 GB | Good tradeoff |
| q3_k | 2.8 GB | Noticeable loss |
| q2_k | 2.4 GB | Significant loss |

## Quantization Details

### FP8 (Phase 1a)
- **Method**: Per-row symmetric FP8 (float8_e4m3fn)
- **What's quantized**: All `nn.Linear` weights in Slow AR
- **What's kept in bf16**: Embeddings, layer norms, Fast AR, codec
- **Scale**: Per-row float32 (captures per-channel variation)
- **Dequant**: `W_bf16 = W_fp8.to(bfloat16) * scale`
- **Quality**: Zero perceptible loss

### INT4 (Phase 1b)
- **Method**: Group-wise symmetric INT4 (group_size=128)
- **Range**: [-7, 7] per weight
- **Scale**: Per-group float32
- **Target**: Slow AR only (Fast AR + codec in bf16)
- **Quality**: Near-zero loss with group_size=128

### INT3 (Phase 2c)
- **Method**: Group-wise symmetric INT3 (group_size=128)
- **Range**: [-3, 3] per weight
- **Expected**: Some quality loss, especially on prosody

### INT2 (Phase 3)
- **Method**: Group-wise symmetric INT2 (group_size=64)
- **Range**: [-1, 0, 1] per weight (ternary!)
- **Expected**: Significant quality degradation

## Files

```
fish-speech-experiments/
├── scripts/
│   ├── quantize.py           # Main quantization + sample generation script
│   ├── run_all_phases.py     # Alternative all-in-one script (for HF Jobs)
│   └── upload_to_hub.py      # Upload results to HuggingFace Hub
├── output/                   # Generated quantized models + samples
│   ├── samples/              # Audio samples from each phase
│   ├── phase1a/              # FP8 quantized model
│   ├── phase1b/              # INT4 quantized model
│   ├── phase2a/              # INT4 all layers
│   ├── phase2b/              # INT8 quantized model
│   ├── phase2c/              # INT3 quantized model
│   ├── phase3a/              # INT2 quantized model
│   ├── phase3b/              # INT2 all layers
│   └── all_results.json      # Combined results
├── size_analysis.json        # Theoretical size analysis
└── README.md                 # This file
```

## Citation

```bibtex
@misc{liao2026fishaudios2technical,
      title={Fish Audio S2 Technical Report}, 
      author={Shijia Liao and Yuxuan Wang and others},
      year={2026},
      eprint={2603.08823},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
}
```

## License

Quantized models inherit the [Fish Audio Research License](https://huggingface.co/fishaudio/s2-pro/blob/main/LICENSE.md). 
Research and non-commercial use only.