FP8 quantization: drop stale KD note, refresh PTQ metrics table
Browse filesAll variants are PTQ (no KD); refresh the FP8-vs-fp16 table on 1000 held-out
clips measured against the regenerated PTQ packages.
README.md
CHANGED
|
@@ -44,16 +44,16 @@ The `e2e_ctc`, `ctc`, `e2e_rnnt`, and `rnnt` branches additionally carry FP8 (E4
|
|
| 44 |
- `model.safetensors` — original fp16 weights
|
| 45 |
- `model_fp8.safetensors` — FP8 E4M3 weights (per-output-channel scales) + per-tensor activation scales (`model_fp8.safetensors.activation_scales.json`)
|
| 46 |
|
| 47 |
-
Quantization targets the GEMM layers (encoder feed-forward and attention projections; RNNT joint `enc`/`pred`).
|
| 48 |
|
| 49 |
-
Measured over 1000 held-out audio samples, FP8 transcription closely tracks the fp16 baseline — transcripts are
|
| 50 |
|
| 51 |
| Variant | Word disagreement (FP8 vs fp16) | Transcripts identical | ΔWER vs fp16 |
|
| 52 |
|:----------|--------------------------------:|----------------------:|-------------:|
|
| 53 |
-
| `e2e_ctc` | 1.
|
| 54 |
-
| `ctc` | 1.
|
| 55 |
-
| `e2e_rnnt`|
|
| 56 |
-
| `rnnt` | 0.
|
| 57 |
|
| 58 |
**License:** MIT
|
| 59 |
|
|
|
|
| 44 |
- `model.safetensors` — original fp16 weights
|
| 45 |
- `model_fp8.safetensors` — FP8 E4M3 weights (per-output-channel scales) + per-tensor activation scales (`model_fp8.safetensors.activation_scales.json`)
|
| 46 |
|
| 47 |
+
Quantization targets the GEMM layers (encoder feed-forward and attention projections; RNNT joint `enc`/`pred`). All variants use post-training quantization (PTQ) with per-tensor activation calibration — no fine-tuning is required. FP8 PTQ tracks the fp16 model closely for both CTC and RNNT.
|
| 48 |
|
| 49 |
+
Measured over 1000 held-out audio samples, FP8 transcription closely tracks the fp16 baseline — transcripts are identical for 93–99% of samples, and FP8 WER vs ground truth stays within ±0.2% of fp16:
|
| 50 |
|
| 51 |
| Variant | Word disagreement (FP8 vs fp16) | Transcripts identical | ΔWER vs fp16 |
|
| 52 |
|:----------|--------------------------------:|----------------------:|-------------:|
|
| 53 |
+
| `e2e_ctc` | 1.55% | 93.6% | +0.00 |
|
| 54 |
+
| `ctc` | 1.59% | 93.4% | +0.06 |
|
| 55 |
+
| `e2e_rnnt`| 0.85% | 97.1% | −0.16 |
|
| 56 |
+
| `rnnt` | 0.26% | 99.1% | +0.06 |
|
| 57 |
|
| 58 |
**License:** MIT
|
| 59 |
|