IgorSwat commited on
Commit
bb76836
ยท
1 Parent(s): 3c8def5

Add MLX model & restructurize the repository

Browse files
.gitattributes CHANGED
@@ -33,4 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
- xnnpack/** filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ xnnpack/** filter=lfs diff=lfs merge=lfs -text
37
+ mlx/** filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,3 +1,47 @@
1
  ---
2
  license: openrail
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: openrail
3
  ---
4
+
5
+ # Supertonic 3 TTS
6
+
7
+ This repository hosts the **Supertonic 3** model for the [React Native ExecuTorch](https://github.com/software-mansion/react-native-executorch) library. It performs text-to-speech synthesis supporting **30+ languages** (๐Ÿ‡ฌ๐Ÿ‡ง English, ๐Ÿ‡ช๐Ÿ‡ธ Spanish, ๐Ÿ‡ซ๐Ÿ‡ท French, ๐Ÿ‡ฉ๐Ÿ‡ช German, ๐Ÿ‡ต๐Ÿ‡ฑ Polish, ๐Ÿ‡ฎ๐Ÿ‡ณ Hindi, ๐Ÿ‡ฏ๐Ÿ‡ต Japanese, ๐Ÿ‡จ๐Ÿ‡ณ Chinese, and many more), with a single voice style per language.
8
+
9
+ The model is composed of four sub-models that run sequentially:
10
+
11
+ 1. **Duration predictor** โ€” estimates speech duration from text
12
+ 2. **Text encoder** โ€” encodes text into a style-conditioned representation
13
+ 3. **Vector estimator** โ€” flow-matching denoiser that generates the audio latent
14
+ 4. **Vocoder** โ€” decodes the latent into a 44.1 kHz waveform
15
+
16
+ ## Compatibility
17
+
18
+ These models were exported using **v1.3.1** of ExecuTorch and no forward compatibility is guaranteed. Older versions of the runtime may not work with these files.
19
+
20
+ The models are intended to be used within the React Native ExecuTorch package. If you want to use them outside the package, make sure your runtime is compatible with the ExecuTorch version used to export the `.pte` files and follow the example scripts to run the models.
21
+
22
+ ## Backends
23
+
24
+ | Backend | Description | RTF (Apple Silicon) |
25
+ |---------|-------------|---------------------|
26
+ | `xnnpack` | CPU-optimized via XNNPACK delegate | ~0.07 (14ร— faster than real-time) |
27
+ | `mlx` | Apple Silicon GPU via MLX delegate | ~0.026 (38ร— faster than real-time) |
28
+
29
+ ## Repository Structure
30
+
31
+ ```
32
+ .
33
+ โ”œโ”€โ”€ config.json # Backend-agnostic model manifest
34
+ โ”œโ”€โ”€ unicode_indexer.json # Character-to-id mapping for text preprocessing
35
+ โ”œโ”€โ”€ voices/ # Pre-computed speaker embeddings
36
+ โ”‚ โ”œโ”€โ”€ M1.json
37
+ โ”‚ โ”œโ”€โ”€ M2.json
38
+ โ”‚ โ”œโ”€โ”€ F1.json
39
+ โ”‚ โ””โ”€โ”€ ...
40
+ โ”œโ”€โ”€ xnnpack/ # XNNPACK-exported .pte files
41
+ โ””โ”€โ”€ mlx/ # MLX-exported .pte files
42
+ ```
43
+
44
+ Each `.pte` file exposes two methods:
45
+
46
+ - `forward` โ€” the sub-model inference
47
+ - `get_dynamic_dims_forward` โ€” returns per-input `[rank, 3]` shape constraints (`[min, max, step]`) for runtime input validation
config.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "modelName": "supertonic"
3
+ }
xnnpack/tts.json โ†’ mlx/config.json RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:42078d3aef1cd43ab43021f3c54f47d2d75ceb4e75f627f118890128b06a0d09
3
- size 8253
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c779e14e47a87185026c5adccfc46263fd46d31367c82cd7ff6e9b62c7a5faa5
3
+ size 6460
xnnpack/duration_predictor.pte โ†’ mlx/duration_predictor_mlx_fp32.pte RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:11688dd5b20661ccf0c419d5744241cfb45a194754d60b4dd7b86e9b0d888264
3
- size 3578372
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd6eb5abeedb9def30b04a225f0c3816aa8c21383b5c552c6f64c91845170b50
3
+ size 3518884
xnnpack/text_encoder.pte โ†’ mlx/text_encoder_mlx_fp32.pte RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:94ef59b924e13ee9d01ec2135c09a223de81fc81dabaf81ae86b6dd75a3c7e1f
3
- size 36208640
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f5455739ff38100edc04cbe0fcca42a60792f3ba61e1edcd2426d8db47222db2
3
+ size 36093604
mlx/vector_estimator_mlx_fp32.pte ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80e1c81f86b927e389fd0d6506e7bfd837c984a40ad4e0344342676bdbb414f6
3
+ size 256235916
mlx/vocoder_mlx_fp32.pte ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b478056c9143ea531954d822cc15892f6d01e9a75ddd4c4788cafadd70c5b65a
3
+ size 101400868
unicode_indexer.json ADDED
The diff for this file is too large to render. See raw diff
 
xnnpack/{unicode_indexer.json โ†’ config.json} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9bf7346e43883a81f8645c81224f786d43c5b57f3641f6e7671a7d6c493cb24f
3
- size 277676
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78dc916dc0035e57ffc4fb227b2937c48f1010834343032816b7b021a799ec02
3
+ size 6484
xnnpack/duration_predictor_xnnpack_fp32.pte ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77638ffde9137f060b4ed2fd024894e1ff812ec1098a22363bdf2aa72202e046
3
+ size 3572484
xnnpack/text_encoder_xnnpack_fp32.pte ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6647a2ff007022577111482c5735c5b756b3e09df3a649a1427295ef9a210f2c
3
+ size 36198528
xnnpack/vector_estimator.pte DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:5ecbc66791174a484f4eddb73eefd3aa308605dc305887a2e7f89e63350c9ddc
3
- size 256463744
 
 
 
 
xnnpack/vector_estimator_xnnpack_fp32.pte ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:48fcb112df733302c1cc305c4cb8de64e18cfda04373397d17f77e30dc2d745b
3
+ size 256437888
xnnpack/vocoder.pte DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d47129247148fd01870188de5e7f3e18381a8088f73c4715457836a7fd5cf68e
3
- size 101435136
 
 
 
 
xnnpack/vocoder_xnnpack_fp32.pte ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7933c5a2b6cccedb5851be5de0d549dfda5cc228f4bfd9096726182a9b04d08a
3
+ size 101429632