IgorSwat commited on
Commit ยท
bb76836
1
Parent(s): 3c8def5
Add MLX model & restructurize the repository
Browse files- .gitattributes +2 -1
- README.md +44 -0
- config.json +3 -0
- xnnpack/tts.json โ mlx/config.json +2 -2
- xnnpack/duration_predictor.pte โ mlx/duration_predictor_mlx_fp32.pte +2 -2
- xnnpack/text_encoder.pte โ mlx/text_encoder_mlx_fp32.pte +2 -2
- mlx/vector_estimator_mlx_fp32.pte +3 -0
- mlx/vocoder_mlx_fp32.pte +3 -0
- unicode_indexer.json +0 -0
- xnnpack/{unicode_indexer.json โ config.json} +2 -2
- xnnpack/duration_predictor_xnnpack_fp32.pte +3 -0
- xnnpack/text_encoder_xnnpack_fp32.pte +3 -0
- xnnpack/vector_estimator.pte +0 -3
- xnnpack/vector_estimator_xnnpack_fp32.pte +3 -0
- xnnpack/vocoder.pte +0 -3
- xnnpack/vocoder_xnnpack_fp32.pte +3 -0
.gitattributes
CHANGED
|
@@ -33,4 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
-
xnnpack/** filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
xnnpack/** filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
mlx/** filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,3 +1,47 @@
|
|
| 1 |
---
|
| 2 |
license: openrail
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: openrail
|
| 3 |
---
|
| 4 |
+
|
| 5 |
+
# Supertonic 3 TTS
|
| 6 |
+
|
| 7 |
+
This repository hosts the **Supertonic 3** model for the [React Native ExecuTorch](https://github.com/software-mansion/react-native-executorch) library. It performs text-to-speech synthesis supporting **30+ languages** (๐ฌ๐ง English, ๐ช๐ธ Spanish, ๐ซ๐ท French, ๐ฉ๐ช German, ๐ต๐ฑ Polish, ๐ฎ๐ณ Hindi, ๐ฏ๐ต Japanese, ๐จ๐ณ Chinese, and many more), with a single voice style per language.
|
| 8 |
+
|
| 9 |
+
The model is composed of four sub-models that run sequentially:
|
| 10 |
+
|
| 11 |
+
1. **Duration predictor** โ estimates speech duration from text
|
| 12 |
+
2. **Text encoder** โ encodes text into a style-conditioned representation
|
| 13 |
+
3. **Vector estimator** โ flow-matching denoiser that generates the audio latent
|
| 14 |
+
4. **Vocoder** โ decodes the latent into a 44.1 kHz waveform
|
| 15 |
+
|
| 16 |
+
## Compatibility
|
| 17 |
+
|
| 18 |
+
These models were exported using **v1.3.1** of ExecuTorch and no forward compatibility is guaranteed. Older versions of the runtime may not work with these files.
|
| 19 |
+
|
| 20 |
+
The models are intended to be used within the React Native ExecuTorch package. If you want to use them outside the package, make sure your runtime is compatible with the ExecuTorch version used to export the `.pte` files and follow the example scripts to run the models.
|
| 21 |
+
|
| 22 |
+
## Backends
|
| 23 |
+
|
| 24 |
+
| Backend | Description | RTF (Apple Silicon) |
|
| 25 |
+
|---------|-------------|---------------------|
|
| 26 |
+
| `xnnpack` | CPU-optimized via XNNPACK delegate | ~0.07 (14ร faster than real-time) |
|
| 27 |
+
| `mlx` | Apple Silicon GPU via MLX delegate | ~0.026 (38ร faster than real-time) |
|
| 28 |
+
|
| 29 |
+
## Repository Structure
|
| 30 |
+
|
| 31 |
+
```
|
| 32 |
+
.
|
| 33 |
+
โโโ config.json # Backend-agnostic model manifest
|
| 34 |
+
โโโ unicode_indexer.json # Character-to-id mapping for text preprocessing
|
| 35 |
+
โโโ voices/ # Pre-computed speaker embeddings
|
| 36 |
+
โ โโโ M1.json
|
| 37 |
+
โ โโโ M2.json
|
| 38 |
+
โ โโโ F1.json
|
| 39 |
+
โ โโโ ...
|
| 40 |
+
โโโ xnnpack/ # XNNPACK-exported .pte files
|
| 41 |
+
โโโ mlx/ # MLX-exported .pte files
|
| 42 |
+
```
|
| 43 |
+
|
| 44 |
+
Each `.pte` file exposes two methods:
|
| 45 |
+
|
| 46 |
+
- `forward` โ the sub-model inference
|
| 47 |
+
- `get_dynamic_dims_forward` โ returns per-input `[rank, 3]` shape constraints (`[min, max, step]`) for runtime input validation
|
config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"modelName": "supertonic"
|
| 3 |
+
}
|
xnnpack/tts.json โ mlx/config.json
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c779e14e47a87185026c5adccfc46263fd46d31367c82cd7ff6e9b62c7a5faa5
|
| 3 |
+
size 6460
|
xnnpack/duration_predictor.pte โ mlx/duration_predictor_mlx_fp32.pte
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fd6eb5abeedb9def30b04a225f0c3816aa8c21383b5c552c6f64c91845170b50
|
| 3 |
+
size 3518884
|
xnnpack/text_encoder.pte โ mlx/text_encoder_mlx_fp32.pte
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f5455739ff38100edc04cbe0fcca42a60792f3ba61e1edcd2426d8db47222db2
|
| 3 |
+
size 36093604
|
mlx/vector_estimator_mlx_fp32.pte
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:80e1c81f86b927e389fd0d6506e7bfd837c984a40ad4e0344342676bdbb414f6
|
| 3 |
+
size 256235916
|
mlx/vocoder_mlx_fp32.pte
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b478056c9143ea531954d822cc15892f6d01e9a75ddd4c4788cafadd70c5b65a
|
| 3 |
+
size 101400868
|
unicode_indexer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
xnnpack/{unicode_indexer.json โ config.json}
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:78dc916dc0035e57ffc4fb227b2937c48f1010834343032816b7b021a799ec02
|
| 3 |
+
size 6484
|
xnnpack/duration_predictor_xnnpack_fp32.pte
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:77638ffde9137f060b4ed2fd024894e1ff812ec1098a22363bdf2aa72202e046
|
| 3 |
+
size 3572484
|
xnnpack/text_encoder_xnnpack_fp32.pte
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6647a2ff007022577111482c5735c5b756b3e09df3a649a1427295ef9a210f2c
|
| 3 |
+
size 36198528
|
xnnpack/vector_estimator.pte
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:5ecbc66791174a484f4eddb73eefd3aa308605dc305887a2e7f89e63350c9ddc
|
| 3 |
-
size 256463744
|
|
|
|
|
|
|
|
|
|
|
|
xnnpack/vector_estimator_xnnpack_fp32.pte
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:48fcb112df733302c1cc305c4cb8de64e18cfda04373397d17f77e30dc2d745b
|
| 3 |
+
size 256437888
|
xnnpack/vocoder.pte
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:d47129247148fd01870188de5e7f3e18381a8088f73c4715457836a7fd5cf68e
|
| 3 |
-
size 101435136
|
|
|
|
|
|
|
|
|
|
|
|
xnnpack/vocoder_xnnpack_fp32.pte
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7933c5a2b6cccedb5851be5de0d549dfda5cc228f4bfd9096726182a9b04d08a
|
| 3 |
+
size 101429632
|