docs: explain the Apple split engine
Browse files
README.md
CHANGED
|
@@ -111,10 +111,14 @@ shared cache. The synthesis checkpoint will be at
|
|
| 111 |
|---|---|---:|
|
| 112 |
| PyTorch with CUDA graphs | RTX 3090 | 7.47x |
|
| 113 |
| PyTorch with CUDA graphs | Jetson Orin Nano | 1.83x |
|
| 114 |
-
| split PyTorch engine | Apple M3 Pro | 3.43x |
|
| 115 |
| ONNX Runtime, CPU provider | Apple M3 Pro | 1.21x |
|
| 116 |
| PyTorch CPU reference | Apple M3 Pro | 0.33x |
|
| 117 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
Higher is faster, and 1.0x means real time. ONNX Runtime on the measured M3 Pro
|
| 119 |
CPU is faster than real time. The PyTorch CPU reference path on the same machine
|
| 120 |
is not.
|
|
|
|
| 111 |
|---|---|---:|
|
| 112 |
| PyTorch with CUDA graphs | RTX 3090 | 7.47x |
|
| 113 |
| PyTorch with CUDA graphs | Jetson Orin Nano | 1.83x |
|
| 114 |
+
| split PyTorch engine\* | Apple M3 Pro | 3.43x |
|
| 115 |
| ONNX Runtime, CPU provider | Apple M3 Pro | 1.21x |
|
| 116 |
| PyTorch CPU reference | Apple M3 Pro | 0.33x |
|
| 117 |
|
| 118 |
+
\* "Split" describes device placement, not a different model or checkpoint.
|
| 119 |
+
The token generator runs on the CPU while the mel and vocoder renderer runs on
|
| 120 |
+
the Apple GPU through MPS. Adjacent windows can overlap across the two devices.
|
| 121 |
+
|
| 122 |
Higher is faster, and 1.0x means real time. ONNX Runtime on the measured M3 Pro
|
| 123 |
CPU is faster than real time. The PyTorch CPU reference path on the same machine
|
| 124 |
is not.
|