Files changed (1) hide show
  1. README.md +22 -2
README.md CHANGED
@@ -234,9 +234,29 @@ curl -X POST http://localhost:8000/v1/audio/transcriptions \
234
  | `max_new_tokens` | int | `5120` | Max generated tokens; raise for long audio (e.g. `65536`) |
235
  | `prompt` | string | unset | Optional instruction override; omit to use the built-in transcribe+diarize prompt |
236
 
237
- `verbose_json` parses the model markup into OpenAI-style `segments` with `start`, `end`, and speaker-prefixed `text` (for example `[S01]...`). `json` / `text` return the full transcript string without segment parsing.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
238
 
239
- For benchmarking, performance numbers, and more details, see the [SGLang Omni cookbook](https://github.com/sgl-project/sglang-omni/blob/main/docs/cookbook/moss_transcribe_diarize.md).
240
 
241
  ### Serving with Native Hugging Face Transformers
242
 
 
234
  | `max_new_tokens` | int | `5120` | Max generated tokens; raise for long audio (e.g. `65536`) |
235
  | `prompt` | string | unset | Optional instruction override; omit to use the built-in transcribe+diarize prompt |
236
 
237
+ For benchmarking, performance numbers, and more details, see the [SGLang Omni cookbook](https://github.com/sgl-project/sglang-omni/blob/main/docs/cookbook/moss_transcribe_diarize.md), here we list the performance of short/long-sequence mutli-speaker ASR tasks on single H100:
238
+
239
+
240
+ 1. `movies800times` for short sequence ASR:
241
+
242
+ | Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio_s/s |
243
+ |---:|---:|---:|---:|---:|
244
+ | 1 | 2.57 | 0.388 | 0.0612 | 29.76 |
245
+ | 2 | 4.89 | 0.409 | 0.0659 | 56.55 |
246
+ | 4 | 6.62 | 0.513 | 0.0790 | 76.64 |
247
+ | 8 | 6.80 | 0.533 | 0.0810 | 78.70 |
248
+ | 16 | 7.08 | 0.659 | 0.0922 | 81.98 |
249
+
250
+ 2. `aishell4_long` for long sequence ASR:
251
+
252
+ | Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio_s/s |
253
+ |---:|---:|---:|---:|---:|
254
+ | 1 | 0.022 | 45.2 | 0.0197 | 50.64 |
255
+ | 2 | 0.032 | 60.7 | 0.0265 | 74.25 |
256
+ | 4 | 0.036 | 105.6 | 0.0461 | 81.64 |
257
+ | 8 | 0.040 | 172.6 | 0.0754 | 90.62 |
258
+ | 16 | 0.043 | 282.8 | 0.1237 | 98.83 |
259
 
 
260
 
261
  ### Serving with Native Hugging Face Transformers
262