vpermilp commited on
Commit
55c9d20
·
1 Parent(s): 411962a

FP8 quantization: drop stale KD note, refresh PTQ metrics table

Browse files

All variants are PTQ (no KD); refresh the FP8-vs-fp16 table on 1000 held-out
clips measured against the regenerated PTQ packages.

Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -44,16 +44,16 @@ The `e2e_ctc`, `ctc`, `e2e_rnnt`, and `rnnt` branches additionally carry FP8 (E4
44
  - `model.safetensors` — original fp16 weights
45
  - `model_fp8.safetensors` — FP8 E4M3 weights (per-output-channel scales) + per-tensor activation scales (`model_fp8.safetensors.activation_scales.json`)
46
 
47
- Quantization targets the GEMM layers (encoder feed-forward and attention projections; RNNT joint `enc`/`pred`). CTC variants use post-training quantization (PTQ); RNNT variants add grid-level knowledge distillation against the fp16 teacher to recover accuracy.
48
 
49
- Measured over 1000 held-out audio samples, FP8 transcription closely tracks the fp16 baseline — transcripts are byte-identical for 93–99% of samples, and FP8 WER vs ground truth is within ±0.02% of fp16:
50
 
51
  | Variant | Word disagreement (FP8 vs fp16) | Transcripts identical | ΔWER vs fp16 |
52
  |:----------|--------------------------------:|----------------------:|-------------:|
53
- | `e2e_ctc` | 1.71% | 93.1% | +0.02 |
54
- | `ctc` | 1.67% | 93.2% | +0.00 |
55
- | `e2e_rnnt`| 1.35% | 96.1% | +0.02 |
56
- | `rnnt` | 0.29% | 98.7% | 0.02 |
57
 
58
  **License:** MIT
59
 
 
44
  - `model.safetensors` — original fp16 weights
45
  - `model_fp8.safetensors` — FP8 E4M3 weights (per-output-channel scales) + per-tensor activation scales (`model_fp8.safetensors.activation_scales.json`)
46
 
47
+ Quantization targets the GEMM layers (encoder feed-forward and attention projections; RNNT joint `enc`/`pred`). All variants use post-training quantization (PTQ) with per-tensor activation calibration — no fine-tuning is required. FP8 PTQ tracks the fp16 model closely for both CTC and RNNT.
48
 
49
+ Measured over 1000 held-out audio samples, FP8 transcription closely tracks the fp16 baseline — transcripts are identical for 93–99% of samples, and FP8 WER vs ground truth stays within ±0.2% of fp16:
50
 
51
  | Variant | Word disagreement (FP8 vs fp16) | Transcripts identical | ΔWER vs fp16 |
52
  |:----------|--------------------------------:|----------------------:|-------------:|
53
+ | `e2e_ctc` | 1.55% | 93.6% | +0.00 |
54
+ | `ctc` | 1.59% | 93.4% | +0.06 |
55
+ | `e2e_rnnt`| 0.85% | 97.1% | 0.16 |
56
+ | `rnnt` | 0.26% | 99.1% | +0.06 |
57
 
58
  **License:** MIT
59