Report official Open ASR H200 results

Update the seven public split WER values and add composite H200 RTFx from the reproducible HF Jobs evaluation.

.eval_results/open_asr_leaderboard.yaml

@@ -2,7 +2,17 @@
     id: hf-audio/open-asr-leaderboard
     task_id: mean_wer
   value: 7.03
-  date: '2026-07-13'
+  date: '2026-07-14'
+  source:
+    url: https://huggingface.co/datasets/hf-audio/open-asr-leaderboard
+    name: open-asr-leaderboard
+    user: hf-audio
+
+- dataset:
+    id: hf-audio/open-asr-leaderboard
+    task_id: rtfx
+  value: 741.15
+  date: '2026-07-14'
   source:
     url: https://huggingface.co/datasets/hf-audio/open-asr-leaderboard
     name: open-asr-leaderboard
@@ -11,8 +21,8 @@
 - dataset:
     id: hf-audio/open-asr-leaderboard
     task_id: ami_wer
-  value: 11.02
-  date: '2026-07-13'
+  value: 10.99
+  date: '2026-07-14'
   source:
     url: https://huggingface.co/datasets/hf-audio/open-asr-leaderboard
     name: open-asr-leaderboard
@@ -22,7 +32,7 @@
     id: hf-audio/open-asr-leaderboard
     task_id: earnings22_wer
   value: 12.31
-  date: '2026-07-13'
+  date: '2026-07-14'
   source:
     url: https://huggingface.co/datasets/hf-audio/open-asr-leaderboard
     name: open-asr-leaderboard
@@ -31,8 +41,8 @@
 - dataset:
     id: hf-audio/open-asr-leaderboard
     task_id: gigaspeech_wer
-  value: 8.49
-  date: '2026-07-13'
+  value: 8.48
+  date: '2026-07-14'
   source:
     url: https://huggingface.co/datasets/hf-audio/open-asr-leaderboard
     name: open-asr-leaderboard
@@ -42,7 +52,7 @@
     id: hf-audio/open-asr-leaderboard
     task_id: librispeech_clean_wer
   value: 2.70
-  date: '2026-07-13'
+  date: '2026-07-14'
   source:
     url: https://huggingface.co/datasets/hf-audio/open-asr-leaderboard
     name: open-asr-leaderboard
@@ -51,8 +61,8 @@
 - dataset:
     id: hf-audio/open-asr-leaderboard
     task_id: librispeech_other_wer
-  value: 6.61
-  date: '2026-07-13'
+  value: 6.59
+  date: '2026-07-14'
   source:
     url: https://huggingface.co/datasets/hf-audio/open-asr-leaderboard
     name: open-asr-leaderboard
@@ -62,7 +72,7 @@
     id: hf-audio/open-asr-leaderboard
     task_id: spgispeech_wer
   value: 3.73
-  date: '2026-07-13'
+  date: '2026-07-14'
   source:
     url: https://huggingface.co/datasets/hf-audio/open-asr-leaderboard
     name: open-asr-leaderboard
@@ -71,8 +81,8 @@
 - dataset:
     id: hf-audio/open-asr-leaderboard
     task_id: voxpopuli_wer
-  value: 4.34
-  date: '2026-07-13'
+  value: 4.39
+  date: '2026-07-14'
   source:
     url: https://huggingface.co/datasets/hf-audio/open-asr-leaderboard
     name: open-asr-leaderboard

README.md

@@ -54,25 +54,28 @@ peak runtime memory footprint, depending on device, iOS version, and workload.
 
 ## Evaluation Results
 
-| Evaluation suite | Dataset / split | Language | Metric | Score (%) |
-| --- | --- | :---: | :---: | ---: |
-| Open ASR Leaderboard | AMI Cleaned | EN | WER | 11.02 |
-| Open ASR Leaderboard | Earnings22 | EN | WER | 12.31 |
-| Open ASR Leaderboard | GigaSpeech Cleaned | EN | WER | 8.49 |
-| Open ASR Leaderboard | LibriSpeech test.clean | EN | WER | 2.70 |
-| Open ASR Leaderboard | LibriSpeech test.other | EN | WER | 6.61 |
-| Open ASR Leaderboard | SPGISpeech | EN | WER | 3.73 |
-| Open ASR Leaderboard | VoxPopuli Cleaned AA | EN | WER | 4.34 |
-| **Open ASR Leaderboard** | **Seven-split mean** | **EN** | **WER** | **7.03** |
-| Internal canonical ASR eval | WenetSpeech meeting | ZH | CER | 8.842 |
-| Internal canonical ASR eval | WenetSpeech net | ZH | CER | 7.976 |
+| Evaluation suite | Dataset / split | Language | Metric | Score (%) | H200 RTFx |
+| --- | --- | :---: | :---: | ---: | ---: |
+| Open ASR Leaderboard | AMI Cleaned | EN | WER | 10.99 | 396.91 |
+| Open ASR Leaderboard | Earnings22 | EN | WER | 12.31 | 654.17 |
+| Open ASR Leaderboard | GigaSpeech Cleaned | EN | WER | 8.48 | 641.19 |
+| Open ASR Leaderboard | LibriSpeech test.clean | EN | WER | 2.70 | 687.84 |
+| Open ASR Leaderboard | LibriSpeech test.other | EN | WER | 6.59 | 610.52 |
+| Open ASR Leaderboard | SPGISpeech | EN | WER | 3.73 | 870.32 |
+| Open ASR Leaderboard | VoxPopuli Cleaned AA | EN | WER | 4.39 | 686.14 |
+| **Open ASR Leaderboard** | **Seven-split mean / composite** | **EN** | **WER / RTFx** | **7.03** | **741.15** |
+| Internal canonical ASR eval | WenetSpeech meeting | ZH | CER | 8.842 | - |
+| Internal canonical ASR eval | WenetSpeech net | ZH | CER | 7.976 | - |
 
 The Open ASR results use the seven current public splits from
 [`hf-audio/open-asr-leaderboard`](https://huggingface.co/datasets/hf-audio/open-asr-leaderboard)
 at dataset revision `b6bdcd0beb34f8975dc659796176d88f43aff502`. They were
-measured with the standalone Transformers package using BF16, eager attention,
-greedy singleton decoding, `max_new_tokens=256`, and the documented 30-second
-audio cap. The corresponding machine-readable results are provided in
+measured with the standalone Transformers package on standardized H200 Hugging
+Face Jobs using BF16, eager attention, greedy decoding, `max_new_tokens=256`,
+and the documented 30-second audio cap. Per-split batch sizes were 1152, 1024,
+1408, 1024, 1024, 2048, and 628. Raw manifests are stored in
+`hf://buckets/AutoArk-AI/audio8-asr-open-asr-results`, and the corresponding
+machine-readable results are provided in
 `.eval_results/open_asr_leaderboard.yaml`.
 
 The internal canonical WenetSpeech results come from the reproducibility-checked