Fix dynamic Kokoro profiles and Supertonic baseline lookup
Browse files
README.md
CHANGED
|
@@ -1,3 +1,140 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# TTS Core AI Lab
|
| 2 |
+
|
| 3 |
+
Local iOS workbench for incrementally porting and profiling the split Kokoro
|
| 4 |
+
pipeline in Swift.
|
| 5 |
+
|
| 6 |
+
The app integrates the official KittenTTS Swift package and Supertonic 3
|
| 7 |
+
ONNX pipeline, and provides four modes:
|
| 8 |
+
|
| 9 |
+
- KittenTTS: ONNX Runtime CPU, eight voices, configurable speed.
|
| 10 |
+
- Kokoro: the existing 11-stage split Core AI pipeline.
|
| 11 |
+
- Supertonic 3: four Core AI models, ten voices, configurable flow steps,
|
| 12 |
+
44.1 kHz.
|
| 13 |
+
- Compare: sequentially generates all three models for the same text, preserves
|
| 14 |
+
PCM results for independent playback, and displays wall time, inference
|
| 15 |
+
time, RTF, Core AI coverage, and speedup.
|
| 16 |
+
|
| 17 |
+
KittenTTS downloads its Nano model and phonemizer assets on first use. Kokoro
|
| 18 |
+
and Supertonic 3 use Core AI model assets hosted separately on Hugging Face:
|
| 19 |
+
<https://huggingface.co/augustZheng/TTS-Core-AI>.
|
| 20 |
+
|
| 21 |
+
## Model Assets
|
| 22 |
+
|
| 23 |
+
The Xcode project expects these asset directories:
|
| 24 |
+
|
| 25 |
+
- `TTSCoreAILab/Models`
|
| 26 |
+
- `TTSCoreAILab/SupertonicResources`
|
| 27 |
+
|
| 28 |
+
They are not committed to GitHub because the full asset bundle is about 4.2 GB
|
| 29 |
+
and contains files larger than GitHub's regular 100 MB file limit.
|
| 30 |
+
|
| 31 |
+
Download them from Hugging Face before building:
|
| 32 |
+
|
| 33 |
+
```bash
|
| 34 |
+
./scripts/download-assets.sh
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
The script uses `uv` to run the Hugging Face CLI without creating a project
|
| 38 |
+
virtual environment.
|
| 39 |
+
|
| 40 |
+
The current stage runs a fixed `Hi.` text-to-acoustic-feature chain:
|
| 41 |
+
|
| 42 |
+
```text
|
| 43 |
+
fixed token ids + attention mask
|
| 44 |
+
-> ALBERT
|
| 45 |
+
-> BERT projection
|
| 46 |
+
-> fixed-length duration encoder
|
| 47 |
+
-> duration head
|
| 48 |
+
-> Swift duration/alignment calculation
|
| 49 |
+
-> text encoder CNN + LSTM
|
| 50 |
+
-> Swift ASR alignment
|
| 51 |
+
-> F0/noise shared LSTM + residual blocks
|
| 52 |
+
```
|
| 53 |
+
|
| 54 |
+
Bundled assets:
|
| 55 |
+
|
| 56 |
+
- `kokoro_bert_eager_64.aimodel`
|
| 57 |
+
- `kokoro_bert_projection.aimodel`
|
| 58 |
+
- `kokoro_duration_encoder_no_pack_64_intmask.aimodel`
|
| 59 |
+
- `kokoro_duration_head.aimodel`
|
| 60 |
+
- `kokoro_text_encoder_conv_64_intmask.aimodel`
|
| 61 |
+
- `kokoro_text_encoder_lstm.aimodel`
|
| 62 |
+
- `kokoro_f0n_shared_lstm_64.aimodel`
|
| 63 |
+
- `kokoro_f0n_blocks_64.aimodel`
|
| 64 |
+
|
| 65 |
+
## How to Try
|
| 66 |
+
|
| 67 |
+
1. Open `TTSCoreAILab.xcodeproj` in Xcode.
|
| 68 |
+
2. Select the same physical iPhone target that passed the original smoke test.
|
| 69 |
+
3. Run the `TTSCoreAILab` app target.
|
| 70 |
+
4. Tap `Run Acoustic Stage`.
|
| 71 |
+
|
| 72 |
+
Expected reference result from the Python CoreAI runtime:
|
| 73 |
+
|
| 74 |
+
- shapes: `[1,64,768]`, `[1,512,64]`, `[1,64,640]`, `[1,64,50]`
|
| 75 |
+
- valid-token durations: `[18, 2, 5, 9, 6, 1]`
|
| 76 |
+
- raw predicted frames: `41`
|
| 77 |
+
- fixed target frames: `64`
|
| 78 |
+
- aligned ASR: `[1, 512, 64]`
|
| 79 |
+
- F0: `[1, 128]`
|
| 80 |
+
- noise: `[1, 128]`
|
| 81 |
+
|
| 82 |
+
Python CoreAI reference mean-absolute values:
|
| 83 |
+
|
| 84 |
+
- shared features: `0.35923`
|
| 85 |
+
- F0: `72.4452`
|
| 86 |
+
- noise: `6.59444`
|
| 87 |
+
- text conv: `0.02399`
|
| 88 |
+
- text hidden: `0.16430`
|
| 89 |
+
- aligned ASR: `0.18732`
|
| 90 |
+
|
| 91 |
+
The current build now also includes:
|
| 92 |
+
|
| 93 |
+
- `kokoro_decoder_pre.aimodel`
|
| 94 |
+
- `kokoro_generator_core.aimodel`
|
| 95 |
+
- `kokoro_istft.aimodel`
|
| 96 |
+
- fixed `Hi.` decoder style and harmonic-source fixtures
|
| 97 |
+
- Swift PCM-to-WAV playback through `AVAudioPlayer`
|
| 98 |
+
|
| 99 |
+
The current UI mirrors the browser Local Lab:
|
| 100 |
+
|
| 101 |
+
- editable text and `af_heart` voice selection;
|
| 102 |
+
- selectable bundled Core AI budgets (`64 / 64`, `128 / 256`, and `256 / 512`), with automatic profile escalation for longer segments;
|
| 103 |
+
- optional automatic segmentation;
|
| 104 |
+
- total time, Core AI inference time, audio duration, and RTF;
|
| 105 |
+
- per-stage Core AI timing bars;
|
| 106 |
+
- generated audio playback controls and a detailed run log.
|
| 107 |
+
|
| 108 |
+
The PyTorch reference and native Kokoro comparison controls are visible but
|
| 109 |
+
disabled because those runtimes are not bundled in the iOS app.
|
| 110 |
+
|
| 111 |
+
The bundled profiles increase the text/frame budget from `64 / 64` to
|
| 112 |
+
`128 / 256` and `256 / 512`; automatic mode chooses the smallest profile that
|
| 113 |
+
fits a segment. Supertonic ships its validated `64 / 32` baseline assets under
|
| 114 |
+
`SupertonicResources/coreai-assets-baseline`.
|
| 115 |
+
|
| 116 |
+
Tap `Generate Locally`. A successful run should play generated audio and report:
|
| 117 |
+
|
| 118 |
+
- generator input: `[1, 512, 128]`
|
| 119 |
+
- spec/phase: `[1, 22, 7681]`
|
| 120 |
+
- audio: `[1, 1, 38400]`
|
| 121 |
+
- audio mean absolute value: approximately `0.01756`
|
| 122 |
+
- audio min/max: approximately `-0.2076 / 0.3555`
|
| 123 |
+
|
| 124 |
+
- ## RTF Result
|
| 125 |
+
|
| 126 |
+
The current benchmark on iPhone 17 Pro.
|
| 127 |
+
|
| 128 |
+
**RTF (Real-Time Factor)** measures how long it takes to generate one second of audio.
|
| 129 |
+
|
| 130 |
+
- **RTF < 1.0** → Faster than real time (suitable for interactive TTS)
|
| 131 |
+
|
| 132 |
+
- **Lower is better**
|
| 133 |
+
|
| 134 |
+
The benchmark below compares the same prompt on the same device across different runtimes.
|
| 135 |
+
|
| 136 |
+
<p align="center">
|
| 137 |
+
|
| 138 |
+
<img src="docs/images/rtf-result.png" width="900">
|
| 139 |
+
|
| 140 |
+
</p>
|
TTSCoreAILab/TTSCoreAILab.xcodeproj/project.pbxproj
ADDED
|
@@ -0,0 +1,466 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
// !$*UTF8*$!
|
| 2 |
+
{
|
| 3 |
+
archiveVersion = 1;
|
| 4 |
+
classes = {
|
| 5 |
+
};
|
| 6 |
+
objectVersion = 56;
|
| 7 |
+
objects = {
|
| 8 |
+
|
| 9 |
+
/* Begin PBXBuildFile section */
|
| 10 |
+
A10000000000000000000001 /* TTSCoreAILabApp.swift in Sources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000011 /* TTSCoreAILabApp.swift */; };
|
| 11 |
+
A10000000000000000000002 /* ContentView.swift in Sources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000012 /* ContentView.swift */; };
|
| 12 |
+
A10000000000000000000003 /* CoreAIModelSmokeRunner.swift in Sources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000013 /* CoreAIModelSmokeRunner.swift */; };
|
| 13 |
+
A10000000000000000000004 /* Models/kokoro_bert_projection.aimodel in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000015 /* Models/kokoro_bert_projection.aimodel */; };
|
| 14 |
+
A10000000000000000000005 /* HiFixture.swift in Sources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000016 /* HiFixture.swift */; };
|
| 15 |
+
A10000000000000000000006 /* Models/kokoro_bert_eager_64.aimodel in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000017 /* Models/kokoro_bert_eager_64.aimodel */; };
|
| 16 |
+
A10000000000000000000007 /* Models/kokoro_duration_encoder_no_pack_64_intmask.aimodel in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000018 /* Models/kokoro_duration_encoder_no_pack_64_intmask.aimodel */; };
|
| 17 |
+
A10000000000000000000008 /* Models/kokoro_duration_head.aimodel in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000019 /* Models/kokoro_duration_head.aimodel */; };
|
| 18 |
+
A10000000000000000000009 /* Models/kokoro_text_encoder_conv_64_intmask.aimodel in Resources */ = {isa = PBXBuildFile; fileRef = A1000000000000000000001A /* Models/kokoro_text_encoder_conv_64_intmask.aimodel */; };
|
| 19 |
+
A1000000000000000000000A /* Models/kokoro_text_encoder_lstm.aimodel in Resources */ = {isa = PBXBuildFile; fileRef = A1000000000000000000001B /* Models/kokoro_text_encoder_lstm.aimodel */; };
|
| 20 |
+
A1000000000000000000000B /* Models/kokoro_f0n_shared_lstm_64.aimodel in Resources */ = {isa = PBXBuildFile; fileRef = A1000000000000000000001C /* Models/kokoro_f0n_shared_lstm_64.aimodel */; };
|
| 21 |
+
A1000000000000000000000C /* Models/kokoro_f0n_blocks_64.aimodel in Resources */ = {isa = PBXBuildFile; fileRef = A1000000000000000000001D /* Models/kokoro_f0n_blocks_64.aimodel */; };
|
| 22 |
+
A1000000000000000000000D /* AudioPlayback.swift in Sources */ = {isa = PBXBuildFile; fileRef = A1000000000000000000001E /* AudioPlayback.swift */; };
|
| 23 |
+
A1000000000000000000000E /* Models/kokoro_decoder_pre.aimodel in Resources */ = {isa = PBXBuildFile; fileRef = A1000000000000000000001F /* Models/kokoro_decoder_pre.aimodel */; };
|
| 24 |
+
A1000000000000000000000F /* Models/kokoro_generator_core.aimodel in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000021 /* Models/kokoro_generator_core.aimodel */; };
|
| 25 |
+
A10000000000000000000022 /* Models/kokoro_istft.aimodel in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000023 /* Models/kokoro_istft.aimodel */; };
|
| 26 |
+
A10000000000000000000024 /* Models/hi_decoder_style.f32 in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000025 /* Models/hi_decoder_style.f32 */; };
|
| 27 |
+
A10000000000000000000026 /* Models/hi_harmonic_features.f32 in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000027 /* Models/hi_harmonic_features.f32 */; };
|
| 28 |
+
A10000000000000000000028 /* Models/LongDemo in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000029 /* Models/LongDemo */; };
|
| 29 |
+
A1000000000000000000002A /* LongDemoFixture.swift in Sources */ = {isa = PBXBuildFile; fileRef = A1000000000000000000002B /* LongDemoFixture.swift */; };
|
| 30 |
+
A1000000000000000000002C /* DynamicTextPipeline.swift in Sources */ = {isa = PBXBuildFile; fileRef = A1000000000000000000002D /* DynamicTextPipeline.swift */; };
|
| 31 |
+
A1000000000000000000002E /* Models/DynamicText in Resources */ = {isa = PBXBuildFile; fileRef = A1000000000000000000002F /* Models/DynamicText */; };
|
| 32 |
+
A10000000000000000000035 /* PlainTextView.swift in Sources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000036 /* PlainTextView.swift */; };
|
| 33 |
+
A10000000000000000000037 /* Models/Profiles/128x256 in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000038 /* Models/Profiles/128x256 */; };
|
| 34 |
+
A10000000000000000000094 /* Models/Profiles/256x512 in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000095 /* Models/Profiles/256x512 */; };
|
| 35 |
+
A10000000000000000000039 /* UnifiedTTSLabRunner.swift in Sources */ = {isa = PBXBuildFile; fileRef = A1000000000000000000003A /* UnifiedTTSLabRunner.swift */; };
|
| 36 |
+
A1000000000000000000003B /* KittenTTS in Frameworks */ = {isa = PBXBuildFile; productRef = A1000000000000000000003D /* KittenTTS */; };
|
| 37 |
+
A10000000000000000000080 /* SupertonicTTSService.swift in Sources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000081 /* SupertonicTTSService.swift */; };
|
| 38 |
+
A10000000000000000000084 /* coreai-assets in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000085 /* coreai-assets */; };
|
| 39 |
+
A10000000000000000000086 /* voice_styles in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000087 /* voice_styles */; };
|
| 40 |
+
A1000000000000000000008B /* unicode_indexer.json in Resources */ = {isa = PBXBuildFile; fileRef = A1000000000000000000008C /* unicode_indexer.json */; };
|
| 41 |
+
A10000000000000000000090 /* KokoroCoreMLService.swift in Sources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000091 /* KokoroCoreMLService.swift */; };
|
| 42 |
+
A10000000000000000000092 /* Models/KokoroCoreML in Resources */ = {isa = PBXBuildFile; fileRef = A10000000000000000000093 /* Models/KokoroCoreML */; };
|
| 43 |
+
/* End PBXBuildFile section */
|
| 44 |
+
|
| 45 |
+
/* Begin PBXFileReference section */
|
| 46 |
+
A10000000000000000000010 /* TTSCoreAILab.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = TTSCoreAILab.app; sourceTree = BUILT_PRODUCTS_DIR; };
|
| 47 |
+
A10000000000000000000011 /* TTSCoreAILabApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TTSCoreAILabApp.swift; sourceTree = "<group>"; };
|
| 48 |
+
A10000000000000000000012 /* ContentView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ContentView.swift; sourceTree = "<group>"; };
|
| 49 |
+
A10000000000000000000013 /* CoreAIModelSmokeRunner.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CoreAIModelSmokeRunner.swift; sourceTree = "<group>"; };
|
| 50 |
+
A10000000000000000000014 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = "<group>"; };
|
| 51 |
+
A10000000000000000000015 /* Models/kokoro_bert_projection.aimodel */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/kokoro_bert_projection.aimodel; sourceTree = "<group>"; };
|
| 52 |
+
A10000000000000000000016 /* HiFixture.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HiFixture.swift; sourceTree = "<group>"; };
|
| 53 |
+
A10000000000000000000017 /* Models/kokoro_bert_eager_64.aimodel */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/kokoro_bert_eager_64.aimodel; sourceTree = "<group>"; };
|
| 54 |
+
A10000000000000000000018 /* Models/kokoro_duration_encoder_no_pack_64_intmask.aimodel */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/kokoro_duration_encoder_no_pack_64_intmask.aimodel; sourceTree = "<group>"; };
|
| 55 |
+
A10000000000000000000019 /* Models/kokoro_duration_head.aimodel */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/kokoro_duration_head.aimodel; sourceTree = "<group>"; };
|
| 56 |
+
A1000000000000000000001A /* Models/kokoro_text_encoder_conv_64_intmask.aimodel */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/kokoro_text_encoder_conv_64_intmask.aimodel; sourceTree = "<group>"; };
|
| 57 |
+
A1000000000000000000001B /* Models/kokoro_text_encoder_lstm.aimodel */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/kokoro_text_encoder_lstm.aimodel; sourceTree = "<group>"; };
|
| 58 |
+
A1000000000000000000001C /* Models/kokoro_f0n_shared_lstm_64.aimodel */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/kokoro_f0n_shared_lstm_64.aimodel; sourceTree = "<group>"; };
|
| 59 |
+
A1000000000000000000001D /* Models/kokoro_f0n_blocks_64.aimodel */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/kokoro_f0n_blocks_64.aimodel; sourceTree = "<group>"; };
|
| 60 |
+
A1000000000000000000001E /* AudioPlayback.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AudioPlayback.swift; sourceTree = "<group>"; };
|
| 61 |
+
A1000000000000000000001F /* Models/kokoro_decoder_pre.aimodel */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/kokoro_decoder_pre.aimodel; sourceTree = "<group>"; };
|
| 62 |
+
A10000000000000000000021 /* Models/kokoro_generator_core.aimodel */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/kokoro_generator_core.aimodel; sourceTree = "<group>"; };
|
| 63 |
+
A10000000000000000000023 /* Models/kokoro_istft.aimodel */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/kokoro_istft.aimodel; sourceTree = "<group>"; };
|
| 64 |
+
A10000000000000000000025 /* Models/hi_decoder_style.f32 */ = {isa = PBXFileReference; lastKnownFileType = file; path = Models/hi_decoder_style.f32; sourceTree = "<group>"; };
|
| 65 |
+
A10000000000000000000027 /* Models/hi_harmonic_features.f32 */ = {isa = PBXFileReference; lastKnownFileType = file; path = Models/hi_harmonic_features.f32; sourceTree = "<group>"; };
|
| 66 |
+
A10000000000000000000029 /* Models/LongDemo */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/LongDemo; sourceTree = "<group>"; };
|
| 67 |
+
A1000000000000000000002B /* LongDemoFixture.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LongDemoFixture.swift; sourceTree = "<group>"; };
|
| 68 |
+
A1000000000000000000002D /* DynamicTextPipeline.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DynamicTextPipeline.swift; sourceTree = "<group>"; };
|
| 69 |
+
A1000000000000000000002F /* Models/DynamicText */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/DynamicText; sourceTree = "<group>"; };
|
| 70 |
+
A10000000000000000000036 /* PlainTextView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PlainTextView.swift; sourceTree = "<group>"; };
|
| 71 |
+
A10000000000000000000038 /* Models/Profiles/128x256 */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/Profiles/128x256; sourceTree = "<group>"; };
|
| 72 |
+
A10000000000000000000095 /* Models/Profiles/256x512 */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/Profiles/256x512; sourceTree = "<group>"; };
|
| 73 |
+
A1000000000000000000003A /* UnifiedTTSLabRunner.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = UnifiedTTSLabRunner.swift; sourceTree = "<group>"; };
|
| 74 |
+
A10000000000000000000081 /* SupertonicTTSService.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SupertonicTTSService.swift; sourceTree = "<group>"; };
|
| 75 |
+
A10000000000000000000085 /* coreai-assets */ = {isa = PBXFileReference; lastKnownFileType = folder; name = "coreai-assets"; path = "SupertonicResources/coreai-assets-baseline"; sourceTree = "<group>"; };
|
| 76 |
+
A10000000000000000000087 /* voice_styles */ = {isa = PBXFileReference; lastKnownFileType = folder; name = voice_styles; path = SupertonicResources/voice_styles; sourceTree = "<group>"; };
|
| 77 |
+
A1000000000000000000008C /* unicode_indexer.json */ = {isa = PBXFileReference; lastKnownFileType = text.json; name = unicode_indexer.json; path = SupertonicResources/unicode_indexer.json; sourceTree = "<group>"; };
|
| 78 |
+
A10000000000000000000091 /* KokoroCoreMLService.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = KokoroCoreMLService.swift; sourceTree = "<group>"; };
|
| 79 |
+
A10000000000000000000093 /* Models/KokoroCoreML */ = {isa = PBXFileReference; lastKnownFileType = folder; path = Models/KokoroCoreML; sourceTree = "<group>"; };
|
| 80 |
+
/* End PBXFileReference section */
|
| 81 |
+
|
| 82 |
+
/* Begin PBXFrameworksBuildPhase section */
|
| 83 |
+
A10000000000000000000020 /* Frameworks */ = {
|
| 84 |
+
isa = PBXFrameworksBuildPhase;
|
| 85 |
+
buildActionMask = 2147483647;
|
| 86 |
+
files = (
|
| 87 |
+
A1000000000000000000003B /* KittenTTS in Frameworks */,
|
| 88 |
+
);
|
| 89 |
+
runOnlyForDeploymentPostprocessing = 0;
|
| 90 |
+
};
|
| 91 |
+
/* End PBXFrameworksBuildPhase section */
|
| 92 |
+
|
| 93 |
+
/* Begin PBXGroup section */
|
| 94 |
+
A10000000000000000000030 = {
|
| 95 |
+
isa = PBXGroup;
|
| 96 |
+
children = (
|
| 97 |
+
A10000000000000000000031 /* TTSCoreAILab */,
|
| 98 |
+
A10000000000000000000032 /* Products */,
|
| 99 |
+
);
|
| 100 |
+
sourceTree = "<group>";
|
| 101 |
+
};
|
| 102 |
+
A10000000000000000000031 /* TTSCoreAILab */ = {
|
| 103 |
+
isa = PBXGroup;
|
| 104 |
+
children = (
|
| 105 |
+
A10000000000000000000011 /* TTSCoreAILabApp.swift */,
|
| 106 |
+
A10000000000000000000012 /* ContentView.swift */,
|
| 107 |
+
A10000000000000000000036 /* PlainTextView.swift */,
|
| 108 |
+
A10000000000000000000013 /* CoreAIModelSmokeRunner.swift */,
|
| 109 |
+
A1000000000000000000003A /* UnifiedTTSLabRunner.swift */,
|
| 110 |
+
A10000000000000000000081 /* SupertonicTTSService.swift */,
|
| 111 |
+
A10000000000000000000091 /* KokoroCoreMLService.swift */,
|
| 112 |
+
A10000000000000000000016 /* HiFixture.swift */,
|
| 113 |
+
A1000000000000000000002B /* LongDemoFixture.swift */,
|
| 114 |
+
A1000000000000000000002D /* DynamicTextPipeline.swift */,
|
| 115 |
+
A1000000000000000000001E /* AudioPlayback.swift */,
|
| 116 |
+
A10000000000000000000014 /* Info.plist */,
|
| 117 |
+
A10000000000000000000015 /* Models/kokoro_bert_projection.aimodel */,
|
| 118 |
+
A10000000000000000000017 /* Models/kokoro_bert_eager_64.aimodel */,
|
| 119 |
+
A10000000000000000000018 /* Models/kokoro_duration_encoder_no_pack_64_intmask.aimodel */,
|
| 120 |
+
A10000000000000000000019 /* Models/kokoro_duration_head.aimodel */,
|
| 121 |
+
A1000000000000000000001A /* Models/kokoro_text_encoder_conv_64_intmask.aimodel */,
|
| 122 |
+
A1000000000000000000001B /* Models/kokoro_text_encoder_lstm.aimodel */,
|
| 123 |
+
A1000000000000000000001C /* Models/kokoro_f0n_shared_lstm_64.aimodel */,
|
| 124 |
+
A1000000000000000000001D /* Models/kokoro_f0n_blocks_64.aimodel */,
|
| 125 |
+
A1000000000000000000001F /* Models/kokoro_decoder_pre.aimodel */,
|
| 126 |
+
A10000000000000000000021 /* Models/kokoro_generator_core.aimodel */,
|
| 127 |
+
A10000000000000000000023 /* Models/kokoro_istft.aimodel */,
|
| 128 |
+
A10000000000000000000025 /* Models/hi_decoder_style.f32 */,
|
| 129 |
+
A10000000000000000000027 /* Models/hi_harmonic_features.f32 */,
|
| 130 |
+
A10000000000000000000029 /* Models/LongDemo */,
|
| 131 |
+
A1000000000000000000002F /* Models/DynamicText */,
|
| 132 |
+
A10000000000000000000038 /* Models/Profiles/128x256 */,
|
| 133 |
+
A10000000000000000000095 /* Models/Profiles/256x512 */,
|
| 134 |
+
A10000000000000000000093 /* Models/KokoroCoreML */,
|
| 135 |
+
A10000000000000000000085 /* coreai-assets */,
|
| 136 |
+
A10000000000000000000087 /* voice_styles */,
|
| 137 |
+
A1000000000000000000008C /* unicode_indexer.json */,
|
| 138 |
+
);
|
| 139 |
+
path = TTSCoreAILab;
|
| 140 |
+
sourceTree = "<group>";
|
| 141 |
+
};
|
| 142 |
+
A10000000000000000000032 /* Products */ = {
|
| 143 |
+
isa = PBXGroup;
|
| 144 |
+
children = (
|
| 145 |
+
A10000000000000000000010 /* TTSCoreAILab.app */,
|
| 146 |
+
);
|
| 147 |
+
name = Products;
|
| 148 |
+
sourceTree = "<group>";
|
| 149 |
+
};
|
| 150 |
+
/* End PBXGroup section */
|
| 151 |
+
|
| 152 |
+
/* Begin PBXNativeTarget section */
|
| 153 |
+
A10000000000000000000040 /* TTSCoreAILab */ = {
|
| 154 |
+
isa = PBXNativeTarget;
|
| 155 |
+
buildConfigurationList = A10000000000000000000070 /* Build configuration list for PBXNativeTarget "TTSCoreAILab" */;
|
| 156 |
+
buildPhases = (
|
| 157 |
+
A10000000000000000000050 /* Sources */,
|
| 158 |
+
A10000000000000000000020 /* Frameworks */,
|
| 159 |
+
A10000000000000000000051 /* Resources */,
|
| 160 |
+
);
|
| 161 |
+
buildRules = (
|
| 162 |
+
);
|
| 163 |
+
dependencies = (
|
| 164 |
+
);
|
| 165 |
+
name = TTSCoreAILab;
|
| 166 |
+
packageProductDependencies = (
|
| 167 |
+
A1000000000000000000003D /* KittenTTS */,
|
| 168 |
+
);
|
| 169 |
+
productName = TTSCoreAILab;
|
| 170 |
+
productReference = A10000000000000000000010 /* TTSCoreAILab.app */;
|
| 171 |
+
productType = "com.apple.product-type.application";
|
| 172 |
+
};
|
| 173 |
+
/* End PBXNativeTarget section */
|
| 174 |
+
|
| 175 |
+
/* Begin PBXProject section */
|
| 176 |
+
A10000000000000000000060 /* Project object */ = {
|
| 177 |
+
isa = PBXProject;
|
| 178 |
+
attributes = {
|
| 179 |
+
BuildIndependentTargetsInParallel = 1;
|
| 180 |
+
LastSwiftUpdateCheck = 1600;
|
| 181 |
+
LastUpgradeCheck = 2700;
|
| 182 |
+
TargetAttributes = {
|
| 183 |
+
A10000000000000000000040 = {
|
| 184 |
+
CreatedOnToolsVersion = 16.0;
|
| 185 |
+
};
|
| 186 |
+
};
|
| 187 |
+
};
|
| 188 |
+
buildConfigurationList = A10000000000000000000061 /* Build configuration list for PBXProject "TTSCoreAILab" */;
|
| 189 |
+
compatibilityVersion = "Xcode 14.0";
|
| 190 |
+
developmentRegion = en;
|
| 191 |
+
hasScannedForEncodings = 0;
|
| 192 |
+
knownRegions = (
|
| 193 |
+
en,
|
| 194 |
+
Base,
|
| 195 |
+
);
|
| 196 |
+
mainGroup = A10000000000000000000030;
|
| 197 |
+
packageReferences = (
|
| 198 |
+
A1000000000000000000003C /* XCRemoteSwiftPackageReference "KittenTTS-swift" */,
|
| 199 |
+
);
|
| 200 |
+
productRefGroup = A10000000000000000000032 /* Products */;
|
| 201 |
+
projectDirPath = "";
|
| 202 |
+
projectRoot = "";
|
| 203 |
+
targets = (
|
| 204 |
+
A10000000000000000000040 /* TTSCoreAILab */,
|
| 205 |
+
);
|
| 206 |
+
};
|
| 207 |
+
/* End PBXProject section */
|
| 208 |
+
|
| 209 |
+
/* Begin PBXResourcesBuildPhase section */
|
| 210 |
+
A10000000000000000000051 /* Resources */ = {
|
| 211 |
+
isa = PBXResourcesBuildPhase;
|
| 212 |
+
buildActionMask = 2147483647;
|
| 213 |
+
files = (
|
| 214 |
+
A10000000000000000000004 /* Models/kokoro_bert_projection.aimodel in Resources */,
|
| 215 |
+
A10000000000000000000006 /* Models/kokoro_bert_eager_64.aimodel in Resources */,
|
| 216 |
+
A10000000000000000000007 /* Models/kokoro_duration_encoder_no_pack_64_intmask.aimodel in Resources */,
|
| 217 |
+
A10000000000000000000008 /* Models/kokoro_duration_head.aimodel in Resources */,
|
| 218 |
+
A10000000000000000000009 /* Models/kokoro_text_encoder_conv_64_intmask.aimodel in Resources */,
|
| 219 |
+
A1000000000000000000000A /* Models/kokoro_text_encoder_lstm.aimodel in Resources */,
|
| 220 |
+
A1000000000000000000000B /* Models/kokoro_f0n_shared_lstm_64.aimodel in Resources */,
|
| 221 |
+
A1000000000000000000000C /* Models/kokoro_f0n_blocks_64.aimodel in Resources */,
|
| 222 |
+
A1000000000000000000000E /* Models/kokoro_decoder_pre.aimodel in Resources */,
|
| 223 |
+
A1000000000000000000000F /* Models/kokoro_generator_core.aimodel in Resources */,
|
| 224 |
+
A10000000000000000000022 /* Models/kokoro_istft.aimodel in Resources */,
|
| 225 |
+
A10000000000000000000024 /* Models/hi_decoder_style.f32 in Resources */,
|
| 226 |
+
A10000000000000000000026 /* Models/hi_harmonic_features.f32 in Resources */,
|
| 227 |
+
A10000000000000000000028 /* Models/LongDemo in Resources */,
|
| 228 |
+
A1000000000000000000002E /* Models/DynamicText in Resources */,
|
| 229 |
+
A10000000000000000000037 /* Models/Profiles/128x256 in Resources */,
|
| 230 |
+
A10000000000000000000094 /* Models/Profiles/256x512 in Resources */,
|
| 231 |
+
A10000000000000000000084 /* coreai-assets in Resources */,
|
| 232 |
+
A10000000000000000000086 /* voice_styles in Resources */,
|
| 233 |
+
A1000000000000000000008B /* unicode_indexer.json in Resources */,
|
| 234 |
+
);
|
| 235 |
+
runOnlyForDeploymentPostprocessing = 0;
|
| 236 |
+
};
|
| 237 |
+
/* End PBXResourcesBuildPhase section */
|
| 238 |
+
|
| 239 |
+
/* Begin PBXSourcesBuildPhase section */
|
| 240 |
+
A10000000000000000000050 /* Sources */ = {
|
| 241 |
+
isa = PBXSourcesBuildPhase;
|
| 242 |
+
buildActionMask = 2147483647;
|
| 243 |
+
files = (
|
| 244 |
+
A10000000000000000000001 /* TTSCoreAILabApp.swift in Sources */,
|
| 245 |
+
A10000000000000000000002 /* ContentView.swift in Sources */,
|
| 246 |
+
A10000000000000000000003 /* CoreAIModelSmokeRunner.swift in Sources */,
|
| 247 |
+
A10000000000000000000005 /* HiFixture.swift in Sources */,
|
| 248 |
+
A1000000000000000000000D /* AudioPlayback.swift in Sources */,
|
| 249 |
+
A1000000000000000000002A /* LongDemoFixture.swift in Sources */,
|
| 250 |
+
A1000000000000000000002C /* DynamicTextPipeline.swift in Sources */,
|
| 251 |
+
A10000000000000000000035 /* PlainTextView.swift in Sources */,
|
| 252 |
+
A10000000000000000000039 /* UnifiedTTSLabRunner.swift in Sources */,
|
| 253 |
+
A10000000000000000000080 /* SupertonicTTSService.swift in Sources */,
|
| 254 |
+
A10000000000000000000090 /* KokoroCoreMLService.swift in Sources */,
|
| 255 |
+
);
|
| 256 |
+
runOnlyForDeploymentPostprocessing = 0;
|
| 257 |
+
};
|
| 258 |
+
/* End PBXSourcesBuildPhase section */
|
| 259 |
+
|
| 260 |
+
/* Begin XCBuildConfiguration section */
|
| 261 |
+
A10000000000000000000062 /* Debug */ = {
|
| 262 |
+
isa = XCBuildConfiguration;
|
| 263 |
+
buildSettings = {
|
| 264 |
+
ALWAYS_SEARCH_USER_PATHS = NO;
|
| 265 |
+
CLANG_ANALYZER_NONNULL = YES;
|
| 266 |
+
CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
|
| 267 |
+
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
|
| 268 |
+
CLANG_ENABLE_MODULES = YES;
|
| 269 |
+
CLANG_ENABLE_OBJC_ARC = YES;
|
| 270 |
+
CLANG_ENABLE_OBJC_WEAK = YES;
|
| 271 |
+
CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES;
|
| 272 |
+
CLANG_WARN_BOOL_CONVERSION = YES;
|
| 273 |
+
CLANG_WARN_COMMA = YES;
|
| 274 |
+
CLANG_WARN_CONSTANT_CONVERSION = YES;
|
| 275 |
+
CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES;
|
| 276 |
+
CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
|
| 277 |
+
CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
|
| 278 |
+
CLANG_WARN_EMPTY_BODY = YES;
|
| 279 |
+
CLANG_WARN_ENUM_CONVERSION = YES;
|
| 280 |
+
CLANG_WARN_INFINITE_RECURSION = YES;
|
| 281 |
+
CLANG_WARN_INT_CONVERSION = YES;
|
| 282 |
+
CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES;
|
| 283 |
+
CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES;
|
| 284 |
+
CLANG_WARN_OBJC_LITERAL_CONVERSION = YES;
|
| 285 |
+
CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
|
| 286 |
+
CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES;
|
| 287 |
+
CLANG_WARN_RANGE_LOOP_ANALYSIS = YES;
|
| 288 |
+
CLANG_WARN_STRICT_PROTOTYPES = YES;
|
| 289 |
+
CLANG_WARN_SUSPICIOUS_MOVE = YES;
|
| 290 |
+
CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
|
| 291 |
+
CLANG_WARN_UNREACHABLE_CODE = YES;
|
| 292 |
+
CLANG_WARN__DUPLICATE_METHOD_MATCH = YES;
|
| 293 |
+
COPY_PHASE_STRIP = NO;
|
| 294 |
+
DEBUG_INFORMATION_FORMAT = dwarf;
|
| 295 |
+
ENABLE_STRICT_OBJC_MSGSEND = YES;
|
| 296 |
+
ENABLE_TESTABILITY = YES;
|
| 297 |
+
ENABLE_USER_SCRIPT_SANDBOXING = YES;
|
| 298 |
+
GCC_C_LANGUAGE_STANDARD = gnu17;
|
| 299 |
+
GCC_DYNAMIC_NO_PIC = NO;
|
| 300 |
+
GCC_NO_COMMON_BLOCKS = YES;
|
| 301 |
+
GCC_OPTIMIZATION_LEVEL = 0;
|
| 302 |
+
GCC_PREPROCESSOR_DEFINITIONS = (
|
| 303 |
+
"DEBUG=1",
|
| 304 |
+
"$(inherited)",
|
| 305 |
+
);
|
| 306 |
+
GCC_WARN_64_TO_32_BIT_CONVERSION = YES;
|
| 307 |
+
GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
|
| 308 |
+
GCC_WARN_UNDECLARED_SELECTOR = YES;
|
| 309 |
+
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
|
| 310 |
+
GCC_WARN_UNUSED_FUNCTION = YES;
|
| 311 |
+
GCC_WARN_UNUSED_VARIABLE = YES;
|
| 312 |
+
IPHONEOS_DEPLOYMENT_TARGET = 27.0;
|
| 313 |
+
MTL_ENABLE_DEBUG_INFO = INCLUDE_SOURCE;
|
| 314 |
+
MTL_FAST_MATH = YES;
|
| 315 |
+
ONLY_ACTIVE_ARCH = YES;
|
| 316 |
+
SDKROOT = iphoneos;
|
| 317 |
+
STRING_CATALOG_GENERATE_SYMBOLS = YES;
|
| 318 |
+
SWIFT_ACTIVE_COMPILATION_CONDITIONS = "DEBUG $(inherited)";
|
| 319 |
+
SWIFT_OPTIMIZATION_LEVEL = "-Onone";
|
| 320 |
+
};
|
| 321 |
+
name = Debug;
|
| 322 |
+
};
|
| 323 |
+
A10000000000000000000063 /* Release */ = {
|
| 324 |
+
isa = XCBuildConfiguration;
|
| 325 |
+
buildSettings = {
|
| 326 |
+
ALWAYS_SEARCH_USER_PATHS = NO;
|
| 327 |
+
CLANG_ANALYZER_NONNULL = YES;
|
| 328 |
+
CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
|
| 329 |
+
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
|
| 330 |
+
CLANG_ENABLE_MODULES = YES;
|
| 331 |
+
CLANG_ENABLE_OBJC_ARC = YES;
|
| 332 |
+
CLANG_ENABLE_OBJC_WEAK = YES;
|
| 333 |
+
CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES;
|
| 334 |
+
CLANG_WARN_BOOL_CONVERSION = YES;
|
| 335 |
+
CLANG_WARN_COMMA = YES;
|
| 336 |
+
CLANG_WARN_CONSTANT_CONVERSION = YES;
|
| 337 |
+
CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES;
|
| 338 |
+
CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
|
| 339 |
+
CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
|
| 340 |
+
CLANG_WARN_EMPTY_BODY = YES;
|
| 341 |
+
CLANG_WARN_ENUM_CONVERSION = YES;
|
| 342 |
+
CLANG_WARN_INFINITE_RECURSION = YES;
|
| 343 |
+
CLANG_WARN_INT_CONVERSION = YES;
|
| 344 |
+
CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES;
|
| 345 |
+
CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES;
|
| 346 |
+
CLANG_WARN_OBJC_LITERAL_CONVERSION = YES;
|
| 347 |
+
CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
|
| 348 |
+
CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES;
|
| 349 |
+
CLANG_WARN_RANGE_LOOP_ANALYSIS = YES;
|
| 350 |
+
CLANG_WARN_STRICT_PROTOTYPES = YES;
|
| 351 |
+
CLANG_WARN_SUSPICIOUS_MOVE = YES;
|
| 352 |
+
CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
|
| 353 |
+
CLANG_WARN_UNREACHABLE_CODE = YES;
|
| 354 |
+
CLANG_WARN__DUPLICATE_METHOD_MATCH = YES;
|
| 355 |
+
COPY_PHASE_STRIP = NO;
|
| 356 |
+
DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym";
|
| 357 |
+
ENABLE_NS_ASSERTIONS = NO;
|
| 358 |
+
ENABLE_STRICT_OBJC_MSGSEND = YES;
|
| 359 |
+
ENABLE_USER_SCRIPT_SANDBOXING = YES;
|
| 360 |
+
GCC_C_LANGUAGE_STANDARD = gnu17;
|
| 361 |
+
GCC_NO_COMMON_BLOCKS = YES;
|
| 362 |
+
GCC_WARN_64_TO_32_BIT_CONVERSION = YES;
|
| 363 |
+
GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
|
| 364 |
+
GCC_WARN_UNDECLARED_SELECTOR = YES;
|
| 365 |
+
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
|
| 366 |
+
GCC_WARN_UNUSED_FUNCTION = YES;
|
| 367 |
+
GCC_WARN_UNUSED_VARIABLE = YES;
|
| 368 |
+
IPHONEOS_DEPLOYMENT_TARGET = 27.0;
|
| 369 |
+
MTL_ENABLE_DEBUG_INFO = NO;
|
| 370 |
+
MTL_FAST_MATH = YES;
|
| 371 |
+
SDKROOT = iphoneos;
|
| 372 |
+
STRING_CATALOG_GENERATE_SYMBOLS = YES;
|
| 373 |
+
SWIFT_COMPILATION_MODE = wholemodule;
|
| 374 |
+
VALIDATE_PRODUCT = YES;
|
| 375 |
+
};
|
| 376 |
+
name = Release;
|
| 377 |
+
};
|
| 378 |
+
A10000000000000000000071 /* Debug */ = {
|
| 379 |
+
isa = XCBuildConfiguration;
|
| 380 |
+
buildSettings = {
|
| 381 |
+
ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
|
| 382 |
+
CODE_SIGN_STYLE = Automatic;
|
| 383 |
+
CURRENT_PROJECT_VERSION = 1;
|
| 384 |
+
GENERATE_INFOPLIST_FILE = NO;
|
| 385 |
+
INFOPLIST_FILE = TTSCoreAILab/Info.plist;
|
| 386 |
+
LD_RUNPATH_SEARCH_PATHS = (
|
| 387 |
+
"$(inherited)",
|
| 388 |
+
"@executable_path/Frameworks",
|
| 389 |
+
);
|
| 390 |
+
MARKETING_VERSION = 1.0;
|
| 391 |
+
MTL_ENABLE_DEBUG_INFO = NO;
|
| 392 |
+
PRODUCT_BUNDLE_IDENTIFIER = com.coreai.TTSCoreAILab;
|
| 393 |
+
PRODUCT_NAME = "$(TARGET_NAME)";
|
| 394 |
+
SUPPORTED_PLATFORMS = "iphoneos iphonesimulator";
|
| 395 |
+
SWIFT_EMIT_LOC_STRINGS = YES;
|
| 396 |
+
SWIFT_VERSION = 6.0;
|
| 397 |
+
TARGETED_DEVICE_FAMILY = 1;
|
| 398 |
+
};
|
| 399 |
+
name = Debug;
|
| 400 |
+
};
|
| 401 |
+
A10000000000000000000072 /* Release */ = {
|
| 402 |
+
isa = XCBuildConfiguration;
|
| 403 |
+
buildSettings = {
|
| 404 |
+
ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
|
| 405 |
+
CODE_SIGN_STYLE = Automatic;
|
| 406 |
+
CURRENT_PROJECT_VERSION = 1;
|
| 407 |
+
GENERATE_INFOPLIST_FILE = NO;
|
| 408 |
+
INFOPLIST_FILE = TTSCoreAILab/Info.plist;
|
| 409 |
+
LD_RUNPATH_SEARCH_PATHS = (
|
| 410 |
+
"$(inherited)",
|
| 411 |
+
"@executable_path/Frameworks",
|
| 412 |
+
);
|
| 413 |
+
MARKETING_VERSION = 1.0;
|
| 414 |
+
PRODUCT_BUNDLE_IDENTIFIER = com.coreai.TTSCoreAILab;
|
| 415 |
+
PRODUCT_NAME = "$(TARGET_NAME)";
|
| 416 |
+
SUPPORTED_PLATFORMS = "iphoneos iphonesimulator";
|
| 417 |
+
SWIFT_EMIT_LOC_STRINGS = YES;
|
| 418 |
+
SWIFT_VERSION = 6.0;
|
| 419 |
+
TARGETED_DEVICE_FAMILY = 1;
|
| 420 |
+
};
|
| 421 |
+
name = Release;
|
| 422 |
+
};
|
| 423 |
+
/* End XCBuildConfiguration section */
|
| 424 |
+
|
| 425 |
+
/* Begin XCConfigurationList section */
|
| 426 |
+
A10000000000000000000061 /* Build configuration list for PBXProject "TTSCoreAILab" */ = {
|
| 427 |
+
isa = XCConfigurationList;
|
| 428 |
+
buildConfigurations = (
|
| 429 |
+
A10000000000000000000062 /* Debug */,
|
| 430 |
+
A10000000000000000000063 /* Release */,
|
| 431 |
+
);
|
| 432 |
+
defaultConfigurationIsVisible = 0;
|
| 433 |
+
defaultConfigurationName = Release;
|
| 434 |
+
};
|
| 435 |
+
A10000000000000000000070 /* Build configuration list for PBXNativeTarget "TTSCoreAILab" */ = {
|
| 436 |
+
isa = XCConfigurationList;
|
| 437 |
+
buildConfigurations = (
|
| 438 |
+
A10000000000000000000071 /* Debug */,
|
| 439 |
+
A10000000000000000000072 /* Release */,
|
| 440 |
+
);
|
| 441 |
+
defaultConfigurationIsVisible = 0;
|
| 442 |
+
defaultConfigurationName = Release;
|
| 443 |
+
};
|
| 444 |
+
/* End XCConfigurationList section */
|
| 445 |
+
|
| 446 |
+
/* Begin XCRemoteSwiftPackageReference section */
|
| 447 |
+
A1000000000000000000003C /* XCRemoteSwiftPackageReference "KittenTTS-swift" */ = {
|
| 448 |
+
isa = XCRemoteSwiftPackageReference;
|
| 449 |
+
repositoryURL = "https://github.com/KittenML/KittenTTS-swift.git";
|
| 450 |
+
requirement = {
|
| 451 |
+
kind = revision;
|
| 452 |
+
revision = 20cd4d8784c1bad348326955709892c8dfe7226b;
|
| 453 |
+
};
|
| 454 |
+
};
|
| 455 |
+
/* End XCRemoteSwiftPackageReference section */
|
| 456 |
+
|
| 457 |
+
/* Begin XCSwiftPackageProductDependency section */
|
| 458 |
+
A1000000000000000000003D /* KittenTTS */ = {
|
| 459 |
+
isa = XCSwiftPackageProductDependency;
|
| 460 |
+
package = A1000000000000000000003C /* XCRemoteSwiftPackageReference "KittenTTS-swift" */;
|
| 461 |
+
productName = KittenTTS;
|
| 462 |
+
};
|
| 463 |
+
/* End XCSwiftPackageProductDependency section */
|
| 464 |
+
};
|
| 465 |
+
rootObject = A10000000000000000000060 /* Project object */;
|
| 466 |
+
}
|
TTSCoreAILab/TTSCoreAILab/CoreAIModelSmokeRunner.swift
ADDED
|
@@ -0,0 +1,1137 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import Foundation
|
| 2 |
+
|
| 3 |
+
#if canImport(CoreAI)
|
| 4 |
+
import CoreAI
|
| 5 |
+
#endif
|
| 6 |
+
|
| 7 |
+
struct LabStageTiming: Identifiable, Sendable {
|
| 8 |
+
let name: String
|
| 9 |
+
let seconds: Double
|
| 10 |
+
var id: String { name }
|
| 11 |
+
}
|
| 12 |
+
|
| 13 |
+
struct LabRunReport: Sendable {
|
| 14 |
+
let totalSeconds: Double
|
| 15 |
+
let coreAISeconds: Double
|
| 16 |
+
let audioSeconds: Double
|
| 17 |
+
let segmentCount: Int
|
| 18 |
+
let stages: [LabStageTiming]
|
| 19 |
+
let summary: String
|
| 20 |
+
let samples: [Float]
|
| 21 |
+
let sampleRate: Int
|
| 22 |
+
|
| 23 |
+
var realTimeFactor: Double {
|
| 24 |
+
audioSeconds > 0 ? totalSeconds / audioSeconds : 0
|
| 25 |
+
}
|
| 26 |
+
}
|
| 27 |
+
|
| 28 |
+
struct CoreAIProfile: Sendable {
|
| 29 |
+
static let dynamicID = "dynamic"
|
| 30 |
+
|
| 31 |
+
let id: String
|
| 32 |
+
let tokenBudget: Int
|
| 33 |
+
let frameBudget: Int
|
| 34 |
+
let assetDirectory: String?
|
| 35 |
+
let tokenStageProfileID: String?
|
| 36 |
+
let frameStageProfileID: String?
|
| 37 |
+
let names: [String: String]
|
| 38 |
+
|
| 39 |
+
static let compact = CoreAIProfile(
|
| 40 |
+
id: "64x64",
|
| 41 |
+
tokenBudget: 64,
|
| 42 |
+
frameBudget: 64,
|
| 43 |
+
assetDirectory: nil,
|
| 44 |
+
tokenStageProfileID: nil,
|
| 45 |
+
frameStageProfileID: nil,
|
| 46 |
+
names: [
|
| 47 |
+
"bert": "kokoro_bert_eager_64",
|
| 48 |
+
"projection": "kokoro_bert_projection",
|
| 49 |
+
"durationEncoder": "kokoro_duration_encoder_no_pack_64_intmask",
|
| 50 |
+
"durationHead": "kokoro_duration_head",
|
| 51 |
+
"f0Shared": "kokoro_f0n_shared_lstm_64",
|
| 52 |
+
"f0Blocks": "kokoro_f0n_blocks_64",
|
| 53 |
+
"textConv": "kokoro_text_encoder_conv_64_intmask",
|
| 54 |
+
"textLSTM": "kokoro_text_encoder_lstm",
|
| 55 |
+
"decoder": "kokoro_decoder_pre",
|
| 56 |
+
"generator": "kokoro_generator_core",
|
| 57 |
+
"istft": "kokoro_istft",
|
| 58 |
+
]
|
| 59 |
+
)
|
| 60 |
+
|
| 61 |
+
static let large = CoreAIProfile(
|
| 62 |
+
id: "128x256",
|
| 63 |
+
tokenBudget: 128,
|
| 64 |
+
frameBudget: 256,
|
| 65 |
+
assetDirectory: "128x256",
|
| 66 |
+
tokenStageProfileID: nil,
|
| 67 |
+
frameStageProfileID: nil,
|
| 68 |
+
names: [
|
| 69 |
+
"bert": "kokoro_bert_eager",
|
| 70 |
+
"projection": "kokoro_bert_projection",
|
| 71 |
+
"durationEncoder": "kokoro_duration_encoder",
|
| 72 |
+
"durationHead": "kokoro_duration_head",
|
| 73 |
+
"f0Shared": "kokoro_f0n_shared_lstm",
|
| 74 |
+
"f0Blocks": "kokoro_f0n_blocks",
|
| 75 |
+
"textConv": "kokoro_text_encoder_conv",
|
| 76 |
+
"textLSTM": "kokoro_text_encoder_lstm",
|
| 77 |
+
"decoder": "kokoro_decoder_pre",
|
| 78 |
+
"generator": "kokoro_generator_core",
|
| 79 |
+
"istft": "kokoro_istft",
|
| 80 |
+
]
|
| 81 |
+
)
|
| 82 |
+
|
| 83 |
+
static let frame512 = CoreAIProfile(
|
| 84 |
+
id: "256x512",
|
| 85 |
+
tokenBudget: 256,
|
| 86 |
+
frameBudget: 512,
|
| 87 |
+
assetDirectory: "256x512",
|
| 88 |
+
tokenStageProfileID: nil,
|
| 89 |
+
frameStageProfileID: nil,
|
| 90 |
+
names: [
|
| 91 |
+
"bert": "kokoro_bert_eager",
|
| 92 |
+
"projection": "kokoro_bert_projection",
|
| 93 |
+
"durationEncoder": "kokoro_duration_encoder",
|
| 94 |
+
"durationHead": "kokoro_duration_head",
|
| 95 |
+
"f0Shared": "kokoro_f0n_shared_lstm",
|
| 96 |
+
"f0Blocks": "kokoro_f0n_blocks",
|
| 97 |
+
"textConv": "kokoro_text_encoder_conv",
|
| 98 |
+
"textLSTM": "kokoro_text_encoder_lstm",
|
| 99 |
+
"decoder": "kokoro_decoder_pre",
|
| 100 |
+
"generator": "kokoro_generator_core",
|
| 101 |
+
"istft": "kokoro_istft",
|
| 102 |
+
]
|
| 103 |
+
)
|
| 104 |
+
|
| 105 |
+
static let hybrid128x512 = CoreAIProfile(
|
| 106 |
+
id: "128x512-hybrid",
|
| 107 |
+
tokenBudget: 128,
|
| 108 |
+
frameBudget: 512,
|
| 109 |
+
assetDirectory: nil,
|
| 110 |
+
tokenStageProfileID: "128x256",
|
| 111 |
+
frameStageProfileID: "256x512",
|
| 112 |
+
names: [
|
| 113 |
+
"bert": "kokoro_bert_eager",
|
| 114 |
+
"projection": "kokoro_bert_projection",
|
| 115 |
+
"durationEncoder": "kokoro_duration_encoder",
|
| 116 |
+
"durationHead": "kokoro_duration_head",
|
| 117 |
+
"f0Shared": "kokoro_f0n_shared_lstm",
|
| 118 |
+
"f0Blocks": "kokoro_f0n_blocks",
|
| 119 |
+
"textConv": "kokoro_text_encoder_conv",
|
| 120 |
+
"textLSTM": "kokoro_text_encoder_lstm",
|
| 121 |
+
"decoder": "kokoro_decoder_pre",
|
| 122 |
+
"generator": "kokoro_generator_core",
|
| 123 |
+
"istft": "kokoro_istft",
|
| 124 |
+
]
|
| 125 |
+
)
|
| 126 |
+
|
| 127 |
+
static func profile(id: String) -> CoreAIProfile {
|
| 128 |
+
switch id {
|
| 129 |
+
case large.id:
|
| 130 |
+
large
|
| 131 |
+
case frame512.id:
|
| 132 |
+
frame512
|
| 133 |
+
case hybrid128x512.id:
|
| 134 |
+
hybrid128x512
|
| 135 |
+
default:
|
| 136 |
+
compact
|
| 137 |
+
}
|
| 138 |
+
}
|
| 139 |
+
|
| 140 |
+
static func autoCandidates(for validTokenLength: Int) -> [CoreAIProfile] {
|
| 141 |
+
if validTokenLength <= compact.tokenBudget {
|
| 142 |
+
return [.compact, .large, .frame512]
|
| 143 |
+
}
|
| 144 |
+
if validTokenLength <= large.tokenBudget {
|
| 145 |
+
return [.large, .frame512]
|
| 146 |
+
}
|
| 147 |
+
if validTokenLength <= frame512.tokenBudget {
|
| 148 |
+
return [.frame512]
|
| 149 |
+
}
|
| 150 |
+
return []
|
| 151 |
+
}
|
| 152 |
+
}
|
| 153 |
+
|
| 154 |
+
enum CoreAIModelSmokeError: Error, CustomStringConvertible {
|
| 155 |
+
case assetMissing(String)
|
| 156 |
+
case functionMissing(String)
|
| 157 |
+
case inputMissing(String)
|
| 158 |
+
case outputMissing(String)
|
| 159 |
+
case unsupportedScalarType(String)
|
| 160 |
+
case invalidFixture(String)
|
| 161 |
+
case unsupportedText(String)
|
| 162 |
+
case segmentTooLong(text: String, frames: Int)
|
| 163 |
+
|
| 164 |
+
var description: String {
|
| 165 |
+
switch self {
|
| 166 |
+
case .assetMissing(let name):
|
| 167 |
+
return "Bundled asset missing: \(name)"
|
| 168 |
+
case .functionMissing(let asset):
|
| 169 |
+
return "Cannot load Core AI function main from \(asset)"
|
| 170 |
+
case .inputMissing(let name):
|
| 171 |
+
return "Input descriptor missing: \(name)"
|
| 172 |
+
case .outputMissing(let name):
|
| 173 |
+
return "Output missing: \(name)"
|
| 174 |
+
case .unsupportedScalarType(let message):
|
| 175 |
+
return message
|
| 176 |
+
case .invalidFixture(let message):
|
| 177 |
+
return message
|
| 178 |
+
case .unsupportedText(let text):
|
| 179 |
+
return "Text is not prepared yet: \(text)"
|
| 180 |
+
case .segmentTooLong(let text, let frames):
|
| 181 |
+
return "Segment is too long (\(frames) frames): \(text)"
|
| 182 |
+
}
|
| 183 |
+
}
|
| 184 |
+
}
|
| 185 |
+
|
| 186 |
+
actor CoreAIModelSmokeRunner {
|
| 187 |
+
static let shared = CoreAIModelSmokeRunner()
|
| 188 |
+
|
| 189 |
+
#if canImport(CoreAI)
|
| 190 |
+
private var stageSeconds: [String: Double] = [:]
|
| 191 |
+
private var loadedFunctions: [String: LoadedFunction] = [:]
|
| 192 |
+
|
| 193 |
+
private struct LoadedFunction {
|
| 194 |
+
let model: AIModel
|
| 195 |
+
let function: InferenceFunction
|
| 196 |
+
}
|
| 197 |
+
|
| 198 |
+
private struct SegmentResult {
|
| 199 |
+
let text: String
|
| 200 |
+
let phonemes: String
|
| 201 |
+
let validTokenLength: Int
|
| 202 |
+
let durations: [Int]
|
| 203 |
+
let rawFrameCount: Int
|
| 204 |
+
let samples: [Float]
|
| 205 |
+
let meanAbs: Float
|
| 206 |
+
}
|
| 207 |
+
#endif
|
| 208 |
+
|
| 209 |
+
func run() async throws -> String {
|
| 210 |
+
#if canImport(CoreAI)
|
| 211 |
+
loadedFunctions.removeAll(keepingCapacity: false)
|
| 212 |
+
defer {
|
| 213 |
+
loadedFunctions.removeAll(keepingCapacity: false)
|
| 214 |
+
}
|
| 215 |
+
let decoderStyle = try loadFloatFixture(name: "hi_decoder_style", count: 128)
|
| 216 |
+
let result = try await synthesize(HiFixture().segment(decoderStyle: decoderStyle))
|
| 217 |
+
try await AudioPlayback.shared.play(samples: result.samples, sampleRate: 24_000)
|
| 218 |
+
return """
|
| 219 |
+
Core AI TTS passed and played.
|
| 220 |
+
|
| 221 |
+
text: \(result.text)
|
| 222 |
+
phonemes: \(result.phonemes)
|
| 223 |
+
valid tokens: \(result.validTokenLength)/64
|
| 224 |
+
durations: \(result.durations)
|
| 225 |
+
predicted frames: \(result.rawFrameCount)
|
| 226 |
+
audio samples: \(result.samples.count)
|
| 227 |
+
audio meanAbs: \(result.meanAbs)
|
| 228 |
+
"""
|
| 229 |
+
#else
|
| 230 |
+
return "This SDK does not expose the CoreAI Swift module."
|
| 231 |
+
#endif
|
| 232 |
+
}
|
| 233 |
+
|
| 234 |
+
func runText(
|
| 235 |
+
_ text: String,
|
| 236 |
+
progress: @escaping @MainActor @Sendable (String) -> Void
|
| 237 |
+
) async throws -> String {
|
| 238 |
+
(try await runLab(text, progress: progress)).summary
|
| 239 |
+
}
|
| 240 |
+
|
| 241 |
+
func runLab(
|
| 242 |
+
_ text: String,
|
| 243 |
+
profileID: String = "64x64",
|
| 244 |
+
autoSplit: Bool = true,
|
| 245 |
+
playAudio: Bool = true,
|
| 246 |
+
progress: @escaping @MainActor @Sendable (String) -> Void
|
| 247 |
+
) async throws -> LabRunReport {
|
| 248 |
+
#if canImport(CoreAI)
|
| 249 |
+
stageSeconds = [:]
|
| 250 |
+
loadedFunctions.removeAll(keepingCapacity: false)
|
| 251 |
+
defer {
|
| 252 |
+
loadedFunctions.removeAll(keepingCapacity: false)
|
| 253 |
+
}
|
| 254 |
+
let totalStart = ContinuousClock.now
|
| 255 |
+
let isDynamic = profileID == CoreAIProfile.dynamicID
|
| 256 |
+
let requestedProfile = CoreAIProfile.profile(id: profileID)
|
| 257 |
+
let planningProfile = isDynamic ? CoreAIProfile.frame512 : requestedProfile
|
| 258 |
+
let pipeline = try DynamicTextPipeline()
|
| 259 |
+
let selectedSegments = try pipeline.makeSegments(
|
| 260 |
+
text: text,
|
| 261 |
+
tokenBudget: planningProfile.tokenBudget,
|
| 262 |
+
segmentTokenLimit: isDynamic
|
| 263 |
+
? planningProfile.tokenBudget - 2
|
| 264 |
+
: (planningProfile.frameBudget == 64 ? 20 : planningProfile.tokenBudget - 2),
|
| 265 |
+
autoSplit: autoSplit
|
| 266 |
+
)
|
| 267 |
+
var pendingSegments = selectedSegments
|
| 268 |
+
var combined: [Float] = []
|
| 269 |
+
var summaries: [String] = []
|
| 270 |
+
var completedTexts: [String] = []
|
| 271 |
+
var index = 0
|
| 272 |
+
|
| 273 |
+
while index < pendingSegments.count {
|
| 274 |
+
let segment = pendingSegments[index]
|
| 275 |
+
let candidateProfiles: [CoreAIProfile]
|
| 276 |
+
if isDynamic {
|
| 277 |
+
candidateProfiles = CoreAIProfile.autoCandidates(for: segment.validTokenLength)
|
| 278 |
+
} else {
|
| 279 |
+
candidateProfiles = [requestedProfile]
|
| 280 |
+
}
|
| 281 |
+
|
| 282 |
+
var selectedResult: SegmentResult?
|
| 283 |
+
var selectedProfile: CoreAIProfile?
|
| 284 |
+
var finalOverflow: CoreAIModelSmokeError?
|
| 285 |
+
|
| 286 |
+
for (candidateIndex, candidate) in candidateProfiles.enumerated() {
|
| 287 |
+
let candidateSegment: TTSegmentFixture
|
| 288 |
+
if segment.inputIds.count == candidate.tokenBudget {
|
| 289 |
+
candidateSegment = segment
|
| 290 |
+
} else {
|
| 291 |
+
guard let rebuilt = try pipeline.makeSegments(
|
| 292 |
+
text: segment.text,
|
| 293 |
+
tokenBudget: candidate.tokenBudget,
|
| 294 |
+
autoSplit: false
|
| 295 |
+
).first else {
|
| 296 |
+
throw CoreAIModelSmokeError.unsupportedText(segment.text)
|
| 297 |
+
}
|
| 298 |
+
candidateSegment = rebuilt
|
| 299 |
+
}
|
| 300 |
+
|
| 301 |
+
await progress(
|
| 302 |
+
"Generating \(index + 1) of \(pendingSegments.count) with \(candidate.id)\n"
|
| 303 |
+
+ "\(candidateSegment.validTokenLength) tokens · \(candidateSegment.text)"
|
| 304 |
+
)
|
| 305 |
+
do {
|
| 306 |
+
selectedResult = try await synthesize(
|
| 307 |
+
candidateSegment,
|
| 308 |
+
profile: candidate,
|
| 309 |
+
progress: progress
|
| 310 |
+
)
|
| 311 |
+
selectedProfile = candidate
|
| 312 |
+
break
|
| 313 |
+
} catch let error as CoreAIModelSmokeError {
|
| 314 |
+
guard case .segmentTooLong(_, let frames) = error else {
|
| 315 |
+
throw error
|
| 316 |
+
}
|
| 317 |
+
finalOverflow = error
|
| 318 |
+
if candidateIndex + 1 < candidateProfiles.count {
|
| 319 |
+
let next = candidateProfiles[candidateIndex + 1]
|
| 320 |
+
await progress(
|
| 321 |
+
"\(candidate.id) predicted \(frames) frames.\n"
|
| 322 |
+
+ "Upgrading this segment to \(next.id)…"
|
| 323 |
+
)
|
| 324 |
+
}
|
| 325 |
+
}
|
| 326 |
+
}
|
| 327 |
+
|
| 328 |
+
guard let result = selectedResult, let profile = selectedProfile else {
|
| 329 |
+
guard autoSplit else {
|
| 330 |
+
throw finalOverflow ?? CoreAIModelSmokeError.unsupportedText(segment.text)
|
| 331 |
+
}
|
| 332 |
+
let split = try pipeline.splitForFrameOverflow(segment.text)
|
| 333 |
+
let replacements = try split.flatMap {
|
| 334 |
+
try pipeline.makeSegments(
|
| 335 |
+
text: $0,
|
| 336 |
+
tokenBudget: planningProfile.tokenBudget,
|
| 337 |
+
autoSplit: false
|
| 338 |
+
)
|
| 339 |
+
}
|
| 340 |
+
pendingSegments.replaceSubrange(index...index, with: replacements)
|
| 341 |
+
await progress(
|
| 342 |
+
"Predicted duration exceeded \(planningProfile.frameBudget) frames.\n"
|
| 343 |
+
+ "Split into \(pendingSegments.count) segments and retrying…"
|
| 344 |
+
)
|
| 345 |
+
continue
|
| 346 |
+
}
|
| 347 |
+
let samplesPerFrame = 600
|
| 348 |
+
let predictedSamples = max(1, result.rawFrameCount) * samplesPerFrame
|
| 349 |
+
let audibleSamples = min(result.samples.count, predictedSamples)
|
| 350 |
+
let trimmedTailSamples = max(0, result.samples.count - audibleSamples)
|
| 351 |
+
var audible = Array(result.samples.prefix(audibleSamples))
|
| 352 |
+
applyFadeOut(to: &audible, count: min(240, audible.count))
|
| 353 |
+
appendSmoothly(
|
| 354 |
+
audible,
|
| 355 |
+
after: completedTexts.last,
|
| 356 |
+
to: &combined,
|
| 357 |
+
sampleRate: 24_000
|
| 358 |
+
)
|
| 359 |
+
summaries.append(
|
| 360 |
+
"\(index + 1). \(result.text)\n"
|
| 361 |
+
+ " \(result.validTokenLength)/\(profile.tokenBudget) tokens · "
|
| 362 |
+
+ "\(result.rawFrameCount)/\(profile.frameBudget) frames · \(profile.id)\n"
|
| 363 |
+
+ " audio \(String(format: "%.2f", Double(audibleSamples) / 24_000))s · "
|
| 364 |
+
+ "trimmed tail \(String(format: "%.2f", Double(trimmedTailSamples) / 24_000))s"
|
| 365 |
+
)
|
| 366 |
+
completedTexts.append(result.text)
|
| 367 |
+
loadedFunctions.removeAll(keepingCapacity: false)
|
| 368 |
+
index += 1
|
| 369 |
+
}
|
| 370 |
+
|
| 371 |
+
if playAudio {
|
| 372 |
+
await progress("Playing \(pendingSegments.count) generated segments...")
|
| 373 |
+
try await AudioPlayback.shared.play(samples: combined, sampleRate: 24_000)
|
| 374 |
+
}
|
| 375 |
+
let totalSeconds = durationSeconds(since: totalStart)
|
| 376 |
+
let coreAISeconds = stageSeconds.values.reduce(0, +)
|
| 377 |
+
let audioSeconds = Double(combined.count) / 24_000
|
| 378 |
+
let summary = """
|
| 379 |
+
Long Core AI TTS passed and played.
|
| 380 |
+
|
| 381 |
+
\(text)
|
| 382 |
+
|
| 383 |
+
\(summaries.joined(separator: "\n"))
|
| 384 |
+
|
| 385 |
+
total audio: \(String(format: "%.1f", Double(combined.count) / 24_000)) seconds
|
| 386 |
+
samples: \(combined.count)
|
| 387 |
+
"""
|
| 388 |
+
let stages = stageSeconds
|
| 389 |
+
.map { LabStageTiming(name: $0.key, seconds: $0.value) }
|
| 390 |
+
.sorted { $0.seconds > $1.seconds }
|
| 391 |
+
return LabRunReport(
|
| 392 |
+
totalSeconds: totalSeconds,
|
| 393 |
+
coreAISeconds: coreAISeconds,
|
| 394 |
+
audioSeconds: audioSeconds,
|
| 395 |
+
segmentCount: pendingSegments.count,
|
| 396 |
+
stages: stages,
|
| 397 |
+
summary: summary,
|
| 398 |
+
samples: combined,
|
| 399 |
+
sampleRate: 24_000
|
| 400 |
+
)
|
| 401 |
+
#else
|
| 402 |
+
return LabRunReport(
|
| 403 |
+
totalSeconds: 0,
|
| 404 |
+
coreAISeconds: 0,
|
| 405 |
+
audioSeconds: 0,
|
| 406 |
+
segmentCount: 0,
|
| 407 |
+
stages: [],
|
| 408 |
+
summary: "This SDK does not expose the CoreAI Swift module.",
|
| 409 |
+
samples: [],
|
| 410 |
+
sampleRate: 24_000
|
| 411 |
+
)
|
| 412 |
+
#endif
|
| 413 |
+
}
|
| 414 |
+
|
| 415 |
+
#if canImport(CoreAI)
|
| 416 |
+
private func stageProfile(for stage: String, profile: CoreAIProfile) -> CoreAIProfile {
|
| 417 |
+
guard let tokenProfileID = profile.tokenStageProfileID,
|
| 418 |
+
let frameProfileID = profile.frameStageProfileID
|
| 419 |
+
else {
|
| 420 |
+
return profile
|
| 421 |
+
}
|
| 422 |
+
let tokenStages: Set<String> = [
|
| 423 |
+
"bert",
|
| 424 |
+
"projection",
|
| 425 |
+
"durationEncoder",
|
| 426 |
+
"durationHead",
|
| 427 |
+
"textConv",
|
| 428 |
+
"textLSTM",
|
| 429 |
+
]
|
| 430 |
+
return CoreAIProfile.profile(id: tokenStages.contains(stage) ? tokenProfileID : frameProfileID)
|
| 431 |
+
}
|
| 432 |
+
|
| 433 |
+
private func synthesize(
|
| 434 |
+
_ fixture: TTSegmentFixture,
|
| 435 |
+
profile: CoreAIProfile = .compact,
|
| 436 |
+
progress: @escaping @MainActor @Sendable (String) -> Void = { _ in }
|
| 437 |
+
) async throws -> SegmentResult {
|
| 438 |
+
let tokenCount = profile.tokenBudget
|
| 439 |
+
let frameCount = profile.frameBudget
|
| 440 |
+
guard fixture.inputIds.count == tokenCount,
|
| 441 |
+
fixture.attentionMask.count == tokenCount,
|
| 442 |
+
fixture.textMask.count == tokenCount,
|
| 443 |
+
fixture.prosodyStyle.count == 128
|
| 444 |
+
else {
|
| 445 |
+
throw CoreAIModelSmokeError.invalidFixture("Hi fixture has invalid tensor sizes")
|
| 446 |
+
}
|
| 447 |
+
|
| 448 |
+
await progress("Loading and running BERT…")
|
| 449 |
+
let bertHidden = try await withFunction(asset: profile.names["bert"]!, profile: stageProfile(for: "bert", profile: profile)) { function in
|
| 450 |
+
let inputIds = try makeInt32Input(
|
| 451 |
+
descriptor: function.descriptor,
|
| 452 |
+
name: "input_ids",
|
| 453 |
+
values: fixture.inputIds
|
| 454 |
+
)
|
| 455 |
+
let attentionMask = try makeInt32Input(
|
| 456 |
+
descriptor: function.descriptor,
|
| 457 |
+
name: "attention_mask",
|
| 458 |
+
values: fixture.attentionMask
|
| 459 |
+
)
|
| 460 |
+
return try await run(
|
| 461 |
+
function,
|
| 462 |
+
inputs: ["input_ids": inputIds, "attention_mask": attentionMask],
|
| 463 |
+
output: "bert_hidden",
|
| 464 |
+
stage: "kokoro_bert_eager"
|
| 465 |
+
)
|
| 466 |
+
}
|
| 467 |
+
|
| 468 |
+
let dEn = try await withFunction(asset: profile.names["projection"]!, profile: stageProfile(for: "projection", profile: profile)) { function in
|
| 469 |
+
try await run(
|
| 470 |
+
function,
|
| 471 |
+
inputs: ["bert_hidden": bertHidden],
|
| 472 |
+
output: "d_en",
|
| 473 |
+
stage: "kokoro_bert_projection"
|
| 474 |
+
)
|
| 475 |
+
}
|
| 476 |
+
|
| 477 |
+
await progress("Predicting phoneme durations…")
|
| 478 |
+
let durationFeatures = try await withFunction(
|
| 479 |
+
asset: profile.names["durationEncoder"]!,
|
| 480 |
+
profile: stageProfile(for: "durationEncoder", profile: profile)
|
| 481 |
+
) { function in
|
| 482 |
+
let style = try makeFloat32Input(
|
| 483 |
+
descriptor: function.descriptor,
|
| 484 |
+
name: "style",
|
| 485 |
+
values: fixture.prosodyStyle
|
| 486 |
+
)
|
| 487 |
+
let textMask = try makeInt32Input(
|
| 488 |
+
descriptor: function.descriptor,
|
| 489 |
+
name: "text_mask",
|
| 490 |
+
values: fixture.textMask
|
| 491 |
+
)
|
| 492 |
+
return try await run(
|
| 493 |
+
function,
|
| 494 |
+
inputs: ["d_en": dEn, "style": style, "text_mask": textMask],
|
| 495 |
+
output: "d",
|
| 496 |
+
stage: "kokoro_duration_encoder"
|
| 497 |
+
)
|
| 498 |
+
}
|
| 499 |
+
|
| 500 |
+
let durationLogits = try await withFunction(
|
| 501 |
+
asset: profile.names["durationHead"]!,
|
| 502 |
+
profile: stageProfile(for: "durationHead", profile: profile)
|
| 503 |
+
) { function in
|
| 504 |
+
try await run(
|
| 505 |
+
function,
|
| 506 |
+
inputs: ["d": durationFeatures],
|
| 507 |
+
output: "duration_logits",
|
| 508 |
+
stage: "kokoro_duration_head"
|
| 509 |
+
)
|
| 510 |
+
}
|
| 511 |
+
|
| 512 |
+
let durations = predictedDurations(
|
| 513 |
+
logits: durationLogits,
|
| 514 |
+
tokenCount: tokenCount,
|
| 515 |
+
binsPerToken: 50,
|
| 516 |
+
validTokenLength: fixture.validTokenLength
|
| 517 |
+
)
|
| 518 |
+
let rawFrameCount = durations.reduce(0, +)
|
| 519 |
+
guard rawFrameCount <= frameCount else {
|
| 520 |
+
throw CoreAIModelSmokeError.segmentTooLong(
|
| 521 |
+
text: fixture.text,
|
| 522 |
+
frames: rawFrameCount
|
| 523 |
+
)
|
| 524 |
+
}
|
| 525 |
+
let alignment = fixedAlignment(
|
| 526 |
+
durations: durations,
|
| 527 |
+
tokenCount: tokenCount,
|
| 528 |
+
frameCount: frameCount
|
| 529 |
+
)
|
| 530 |
+
|
| 531 |
+
let durationFeatureValues = flattenFloat32(durationFeatures)
|
| 532 |
+
let enValues = projectTokenFeaturesToFrames(
|
| 533 |
+
tokenFeatures: durationFeatureValues,
|
| 534 |
+
tokenCount: tokenCount,
|
| 535 |
+
featureCount: 640,
|
| 536 |
+
alignment: alignment.values,
|
| 537 |
+
frameCount: frameCount
|
| 538 |
+
)
|
| 539 |
+
|
| 540 |
+
await progress("Generating pitch and noise curves…")
|
| 541 |
+
let sharedFeatures = try await withFunction(
|
| 542 |
+
asset: profile.names["f0Shared"]!,
|
| 543 |
+
profile: stageProfile(for: "f0Shared", profile: profile)
|
| 544 |
+
) { function in
|
| 545 |
+
let en = try makeFloat32Input(
|
| 546 |
+
descriptor: function.descriptor,
|
| 547 |
+
name: "en",
|
| 548 |
+
values: enValues
|
| 549 |
+
)
|
| 550 |
+
return try await run(
|
| 551 |
+
function,
|
| 552 |
+
inputs: ["en": en],
|
| 553 |
+
output: "shared_features",
|
| 554 |
+
stage: "kokoro_f0n_shared_lstm"
|
| 555 |
+
)
|
| 556 |
+
}
|
| 557 |
+
|
| 558 |
+
let f0Noise = try await withFunction(
|
| 559 |
+
asset: profile.names["f0Blocks"]!,
|
| 560 |
+
profile: stageProfile(for: "f0Blocks", profile: profile)
|
| 561 |
+
) { function in
|
| 562 |
+
let f0Style = try makeFloat32Input(
|
| 563 |
+
descriptor: function.descriptor,
|
| 564 |
+
name: "style",
|
| 565 |
+
values: fixture.prosodyStyle
|
| 566 |
+
)
|
| 567 |
+
return try await runAll(
|
| 568 |
+
function,
|
| 569 |
+
inputs: ["shared_features": sharedFeatures, "style": f0Style],
|
| 570 |
+
outputs: ["f0", "noise"],
|
| 571 |
+
stage: "kokoro_f0n_blocks"
|
| 572 |
+
)
|
| 573 |
+
}
|
| 574 |
+
guard let f0 = f0Noise["f0"], let noise = f0Noise["noise"] else {
|
| 575 |
+
throw CoreAIModelSmokeError.outputMissing("f0/noise")
|
| 576 |
+
}
|
| 577 |
+
|
| 578 |
+
await progress("Encoding text features…")
|
| 579 |
+
let convFeatures = try await withFunction(
|
| 580 |
+
asset: profile.names["textConv"]!,
|
| 581 |
+
profile: stageProfile(for: "textConv", profile: profile)
|
| 582 |
+
) { function in
|
| 583 |
+
let textInputIds = try makeInt32Input(
|
| 584 |
+
descriptor: function.descriptor,
|
| 585 |
+
name: "input_ids",
|
| 586 |
+
values: fixture.inputIds
|
| 587 |
+
)
|
| 588 |
+
let textInputMask = try makeInt32Input(
|
| 589 |
+
descriptor: function.descriptor,
|
| 590 |
+
name: "text_mask",
|
| 591 |
+
values: fixture.textMask
|
| 592 |
+
)
|
| 593 |
+
return try await run(
|
| 594 |
+
function,
|
| 595 |
+
inputs: ["input_ids": textInputIds, "text_mask": textInputMask],
|
| 596 |
+
output: "conv_features",
|
| 597 |
+
stage: "kokoro_text_encoder_conv"
|
| 598 |
+
)
|
| 599 |
+
}
|
| 600 |
+
|
| 601 |
+
let textHidden = try await withFunction(
|
| 602 |
+
asset: profile.names["textLSTM"]!,
|
| 603 |
+
profile: stageProfile(for: "textLSTM", profile: profile)
|
| 604 |
+
) { function in
|
| 605 |
+
try await run(
|
| 606 |
+
function,
|
| 607 |
+
inputs: ["conv_features": convFeatures],
|
| 608 |
+
output: "asr",
|
| 609 |
+
stage: "kokoro_text_encoder_lstm"
|
| 610 |
+
)
|
| 611 |
+
}
|
| 612 |
+
let asrValues = projectChannelTokensToFrames(
|
| 613 |
+
channelTokens: flattenFloat32(textHidden),
|
| 614 |
+
channelCount: 512,
|
| 615 |
+
tokenCount: tokenCount,
|
| 616 |
+
alignment: alignment.values,
|
| 617 |
+
frameCount: frameCount
|
| 618 |
+
)
|
| 619 |
+
|
| 620 |
+
let f0Stats = summarize(f0)
|
| 621 |
+
let noiseStats = summarize(noise)
|
| 622 |
+
let asrStats = summarize(asrValues)
|
| 623 |
+
let decoderStyle = fixture.decoderStyle
|
| 624 |
+
let harmonicFeatures: [Float]
|
| 625 |
+
if let harmonicResource = fixture.harmonicResource {
|
| 626 |
+
harmonicFeatures = try loadFloatFixture(
|
| 627 |
+
name: harmonicResource,
|
| 628 |
+
count: 22 * 7681
|
| 629 |
+
)
|
| 630 |
+
} else {
|
| 631 |
+
await progress("Building harmonic source features…")
|
| 632 |
+
harmonicFeatures = try HarmonicSourceGenerator().makeFeatures(
|
| 633 |
+
f0: flattenFloat32(f0),
|
| 634 |
+
frameBudget: frameCount
|
| 635 |
+
)
|
| 636 |
+
}
|
| 637 |
+
|
| 638 |
+
await progress("Running decoder pre-stage…")
|
| 639 |
+
let generatorInput = try await withFunction(
|
| 640 |
+
asset: profile.names["decoder"]!,
|
| 641 |
+
profile: stageProfile(for: "decoder", profile: profile)
|
| 642 |
+
) { function in
|
| 643 |
+
let decoderASR = try makeFloat32Input(
|
| 644 |
+
descriptor: function.descriptor,
|
| 645 |
+
name: "asr",
|
| 646 |
+
values: asrValues
|
| 647 |
+
)
|
| 648 |
+
let decoderStyleInput = try makeFloat32Input(
|
| 649 |
+
descriptor: function.descriptor,
|
| 650 |
+
name: "style",
|
| 651 |
+
values: decoderStyle
|
| 652 |
+
)
|
| 653 |
+
return try await run(
|
| 654 |
+
function,
|
| 655 |
+
inputs: [
|
| 656 |
+
"asr": decoderASR,
|
| 657 |
+
"f0_curve": f0,
|
| 658 |
+
"noise": noise,
|
| 659 |
+
"style": decoderStyleInput,
|
| 660 |
+
],
|
| 661 |
+
output: "generator_input",
|
| 662 |
+
stage: "kokoro_decoder_pre"
|
| 663 |
+
)
|
| 664 |
+
}
|
| 665 |
+
|
| 666 |
+
await progress("Running generator core… This is the slowest stage.")
|
| 667 |
+
let specPhase = try await withFunction(
|
| 668 |
+
asset: profile.names["generator"]!,
|
| 669 |
+
profile: stageProfile(for: "generator", profile: profile)
|
| 670 |
+
) { function in
|
| 671 |
+
let generatorStyle = try makeFloat32Input(
|
| 672 |
+
descriptor: function.descriptor,
|
| 673 |
+
name: "style",
|
| 674 |
+
values: decoderStyle
|
| 675 |
+
)
|
| 676 |
+
let harmonicInput = try makeFloat32Input(
|
| 677 |
+
descriptor: function.descriptor,
|
| 678 |
+
name: "harmonic_features",
|
| 679 |
+
values: harmonicFeatures
|
| 680 |
+
)
|
| 681 |
+
return try await run(
|
| 682 |
+
function,
|
| 683 |
+
inputs: [
|
| 684 |
+
"x": generatorInput,
|
| 685 |
+
"style": generatorStyle,
|
| 686 |
+
"harmonic_features": harmonicInput,
|
| 687 |
+
],
|
| 688 |
+
output: "spec_phase",
|
| 689 |
+
stage: "kokoro_generator_core"
|
| 690 |
+
)
|
| 691 |
+
}
|
| 692 |
+
|
| 693 |
+
await progress("Reconstructing waveform…")
|
| 694 |
+
let audio = try await withFunction(asset: profile.names["istft"]!, profile: stageProfile(for: "istft", profile: profile)) { function in
|
| 695 |
+
try await run(
|
| 696 |
+
function,
|
| 697 |
+
inputs: ["spec_phase": specPhase],
|
| 698 |
+
output: "audio",
|
| 699 |
+
stage: "kokoro_istft"
|
| 700 |
+
)
|
| 701 |
+
}
|
| 702 |
+
let audioValues = flattenFloat32(audio)
|
| 703 |
+
let audioStats = summarize(audioValues)
|
| 704 |
+
_ = (f0Stats, noiseStats, asrStats, generatorInput, specPhase, alignment)
|
| 705 |
+
|
| 706 |
+
return SegmentResult(
|
| 707 |
+
text: fixture.text,
|
| 708 |
+
phonemes: fixture.phonemes,
|
| 709 |
+
validTokenLength: fixture.validTokenLength,
|
| 710 |
+
durations: durations,
|
| 711 |
+
rawFrameCount: rawFrameCount,
|
| 712 |
+
samples: audioValues,
|
| 713 |
+
meanAbs: audioStats.meanAbs
|
| 714 |
+
)
|
| 715 |
+
}
|
| 716 |
+
|
| 717 |
+
private func appendSmoothly(
|
| 718 |
+
_ segment: [Float],
|
| 719 |
+
after previousText: String?,
|
| 720 |
+
to combined: inout [Float],
|
| 721 |
+
sampleRate: Int
|
| 722 |
+
) {
|
| 723 |
+
guard !combined.isEmpty, let previousText else {
|
| 724 |
+
combined.append(contentsOf: segment)
|
| 725 |
+
return
|
| 726 |
+
}
|
| 727 |
+
|
| 728 |
+
let trimmed = segment
|
| 729 |
+
let punctuation = previousText.trimmingCharacters(in: .whitespacesAndNewlines).last
|
| 730 |
+
let pauseSeconds: Double
|
| 731 |
+
switch punctuation {
|
| 732 |
+
case ".", "!", "?":
|
| 733 |
+
pauseSeconds = 0.16
|
| 734 |
+
case ",", ";", ":":
|
| 735 |
+
pauseSeconds = 0.08
|
| 736 |
+
default:
|
| 737 |
+
pauseSeconds = 0
|
| 738 |
+
}
|
| 739 |
+
|
| 740 |
+
if pauseSeconds > 0 {
|
| 741 |
+
applyFadeOut(to: &combined, count: min(240, combined.count))
|
| 742 |
+
combined.append(contentsOf: repeatElement(0, count: Int(Double(sampleRate) * pauseSeconds)))
|
| 743 |
+
var faded = trimmed
|
| 744 |
+
applyFadeIn(to: &faded, count: min(240, faded.count))
|
| 745 |
+
combined.append(contentsOf: faded)
|
| 746 |
+
return
|
| 747 |
+
}
|
| 748 |
+
|
| 749 |
+
let overlap = min(Int(Double(sampleRate) * 0.035), combined.count, trimmed.count)
|
| 750 |
+
guard overlap > 0 else {
|
| 751 |
+
combined.append(contentsOf: trimmed)
|
| 752 |
+
return
|
| 753 |
+
}
|
| 754 |
+
let start = combined.count - overlap
|
| 755 |
+
for index in 0..<overlap {
|
| 756 |
+
let incoming = Float(index + 1) / Float(overlap + 1)
|
| 757 |
+
let outgoing = 1 - incoming
|
| 758 |
+
combined[start + index] =
|
| 759 |
+
combined[start + index] * outgoing + trimmed[index] * incoming
|
| 760 |
+
}
|
| 761 |
+
combined.append(contentsOf: trimmed.dropFirst(overlap))
|
| 762 |
+
}
|
| 763 |
+
|
| 764 |
+
private func trimQuietEdges(
|
| 765 |
+
_ samples: [Float],
|
| 766 |
+
threshold: Float,
|
| 767 |
+
guardSamples: Int
|
| 768 |
+
) -> [Float] {
|
| 769 |
+
guard let first = samples.firstIndex(where: { abs($0) >= threshold }),
|
| 770 |
+
let last = samples.lastIndex(where: { abs($0) >= threshold })
|
| 771 |
+
else {
|
| 772 |
+
return samples
|
| 773 |
+
}
|
| 774 |
+
let start = max(0, first - guardSamples)
|
| 775 |
+
let end = min(samples.count - 1, last + guardSamples)
|
| 776 |
+
return Array(samples[start...end])
|
| 777 |
+
}
|
| 778 |
+
|
| 779 |
+
private func applyFadeIn(to samples: inout [Float], count: Int) {
|
| 780 |
+
guard count > 0 else { return }
|
| 781 |
+
for index in 0..<count {
|
| 782 |
+
samples[index] *= Float(index + 1) / Float(count)
|
| 783 |
+
}
|
| 784 |
+
}
|
| 785 |
+
|
| 786 |
+
private func applyFadeOut(to samples: inout [Float], count: Int) {
|
| 787 |
+
guard count > 0 else { return }
|
| 788 |
+
let start = samples.count - count
|
| 789 |
+
for index in 0..<count {
|
| 790 |
+
samples[start + index] *= Float(count - index) / Float(count)
|
| 791 |
+
}
|
| 792 |
+
}
|
| 793 |
+
|
| 794 |
+
private func loadFunction(
|
| 795 |
+
asset: String,
|
| 796 |
+
profile: CoreAIProfile = .compact
|
| 797 |
+
) async throws -> LoadedFunction {
|
| 798 |
+
let cacheKey = "\(profile.id)/\(asset)"
|
| 799 |
+
if let loaded = loadedFunctions[cacheKey] {
|
| 800 |
+
return loaded
|
| 801 |
+
}
|
| 802 |
+
|
| 803 |
+
let modelURL = modelURL(for: asset, profile: profile)
|
| 804 |
+
guard let modelURL else {
|
| 805 |
+
throw CoreAIModelSmokeError.assetMissing("\(asset).aimodel")
|
| 806 |
+
}
|
| 807 |
+
let options = SpecializationOptions(preferredComputeUnitKind: .gpu)
|
| 808 |
+
let model = try await AIModel(contentsOf: modelURL, options: options)
|
| 809 |
+
guard let function = try model.loadFunction(named: "main") else {
|
| 810 |
+
throw CoreAIModelSmokeError.functionMissing(asset)
|
| 811 |
+
}
|
| 812 |
+
let loaded = LoadedFunction(model: model, function: function)
|
| 813 |
+
loadedFunctions[cacheKey] = loaded
|
| 814 |
+
return loaded
|
| 815 |
+
}
|
| 816 |
+
|
| 817 |
+
private func modelURL(for asset: String, profile: CoreAIProfile) -> URL? {
|
| 818 |
+
var subdirectories: [String?] = [nil, "Models"]
|
| 819 |
+
if let assetDirectory = profile.assetDirectory {
|
| 820 |
+
subdirectories = [
|
| 821 |
+
assetDirectory,
|
| 822 |
+
"Profiles/\(assetDirectory)",
|
| 823 |
+
"Models/Profiles/\(assetDirectory)",
|
| 824 |
+
nil,
|
| 825 |
+
"Models",
|
| 826 |
+
]
|
| 827 |
+
}
|
| 828 |
+
|
| 829 |
+
for subdirectory in subdirectories {
|
| 830 |
+
if let url = Bundle.main.url(
|
| 831 |
+
forResource: asset,
|
| 832 |
+
withExtension: "aimodel",
|
| 833 |
+
subdirectory: subdirectory
|
| 834 |
+
) {
|
| 835 |
+
return url
|
| 836 |
+
}
|
| 837 |
+
}
|
| 838 |
+
return recursivelyFindModel(
|
| 839 |
+
named: "\(asset).aimodel",
|
| 840 |
+
preferredDirectory: profile.assetDirectory
|
| 841 |
+
)
|
| 842 |
+
}
|
| 843 |
+
|
| 844 |
+
private func recursivelyFindModel(
|
| 845 |
+
named modelDirectory: String,
|
| 846 |
+
preferredDirectory: String?
|
| 847 |
+
) -> URL? {
|
| 848 |
+
guard let resourceURL = Bundle.main.resourceURL,
|
| 849 |
+
let enumerator = FileManager.default.enumerator(
|
| 850 |
+
at: resourceURL,
|
| 851 |
+
includingPropertiesForKeys: [.isDirectoryKey],
|
| 852 |
+
options: [.skipsHiddenFiles]
|
| 853 |
+
)
|
| 854 |
+
else {
|
| 855 |
+
return nil
|
| 856 |
+
}
|
| 857 |
+
|
| 858 |
+
var fallbackURL: URL?
|
| 859 |
+
for case let url as URL in enumerator {
|
| 860 |
+
guard url.lastPathComponent == modelDirectory else {
|
| 861 |
+
continue
|
| 862 |
+
}
|
| 863 |
+
let resourceValues = try? url.resourceValues(forKeys: [.isDirectoryKey])
|
| 864 |
+
guard resourceValues?.isDirectory == true else {
|
| 865 |
+
continue
|
| 866 |
+
}
|
| 867 |
+
if let preferredDirectory {
|
| 868 |
+
let components = Set(url.pathComponents)
|
| 869 |
+
if components.contains(preferredDirectory) {
|
| 870 |
+
return url
|
| 871 |
+
}
|
| 872 |
+
continue
|
| 873 |
+
} else if fallbackURL == nil {
|
| 874 |
+
fallbackURL = url
|
| 875 |
+
}
|
| 876 |
+
}
|
| 877 |
+
return fallbackURL
|
| 878 |
+
}
|
| 879 |
+
|
| 880 |
+
private func withFunction<T>(
|
| 881 |
+
asset: String,
|
| 882 |
+
profile: CoreAIProfile = .compact,
|
| 883 |
+
operation: (InferenceFunction) async throws -> T
|
| 884 |
+
) async throws -> T {
|
| 885 |
+
let loaded = try await loadFunction(asset: asset, profile: profile)
|
| 886 |
+
defer {
|
| 887 |
+
loadedFunctions.removeValue(forKey: "\(profile.id)/\(asset)")
|
| 888 |
+
}
|
| 889 |
+
return try await operation(loaded.function)
|
| 890 |
+
}
|
| 891 |
+
|
| 892 |
+
private func run(
|
| 893 |
+
_ function: InferenceFunction,
|
| 894 |
+
inputs: [String: NDArray],
|
| 895 |
+
output outputName: String,
|
| 896 |
+
stage: String? = nil
|
| 897 |
+
) async throws -> NDArray {
|
| 898 |
+
let start = ContinuousClock.now
|
| 899 |
+
var outputs = try await function.run(inputs: inputs)
|
| 900 |
+
if let stage {
|
| 901 |
+
stageSeconds[stage, default: 0] += durationSeconds(since: start)
|
| 902 |
+
}
|
| 903 |
+
guard let output = outputs.remove(outputName)?.ndArray else {
|
| 904 |
+
throw CoreAIModelSmokeError.outputMissing(outputName)
|
| 905 |
+
}
|
| 906 |
+
return output
|
| 907 |
+
}
|
| 908 |
+
|
| 909 |
+
private func runAll(
|
| 910 |
+
_ function: InferenceFunction,
|
| 911 |
+
inputs: [String: NDArray],
|
| 912 |
+
outputs outputNames: [String],
|
| 913 |
+
stage: String? = nil
|
| 914 |
+
) async throws -> [String: NDArray] {
|
| 915 |
+
let start = ContinuousClock.now
|
| 916 |
+
var rawOutputs = try await function.run(inputs: inputs)
|
| 917 |
+
if let stage {
|
| 918 |
+
stageSeconds[stage, default: 0] += durationSeconds(since: start)
|
| 919 |
+
}
|
| 920 |
+
var result: [String: NDArray] = [:]
|
| 921 |
+
for name in outputNames {
|
| 922 |
+
guard let output = rawOutputs.remove(name)?.ndArray else {
|
| 923 |
+
throw CoreAIModelSmokeError.outputMissing(name)
|
| 924 |
+
}
|
| 925 |
+
result[name] = output
|
| 926 |
+
}
|
| 927 |
+
return result
|
| 928 |
+
}
|
| 929 |
+
|
| 930 |
+
private func makeInt32Input(
|
| 931 |
+
descriptor: InferenceFunctionDescriptor,
|
| 932 |
+
name: String,
|
| 933 |
+
values: [Int32]
|
| 934 |
+
) throws -> NDArray {
|
| 935 |
+
guard case .ndArray(let inputDescriptor) = descriptor.inputDescriptor(of: name) else {
|
| 936 |
+
throw CoreAIModelSmokeError.inputMissing(name)
|
| 937 |
+
}
|
| 938 |
+
guard inputDescriptor.scalarType == .int32 else {
|
| 939 |
+
throw CoreAIModelSmokeError.unsupportedScalarType(
|
| 940 |
+
"Expected int32 input for \(name), got \(inputDescriptor.scalarType)"
|
| 941 |
+
)
|
| 942 |
+
}
|
| 943 |
+
var array = NDArray(descriptor: inputDescriptor)
|
| 944 |
+
var view = array.mutableView(as: Int32.self)
|
| 945 |
+
view.copyElements(fromContentsOf: values)
|
| 946 |
+
return array
|
| 947 |
+
}
|
| 948 |
+
|
| 949 |
+
private func makeFloat32Input(
|
| 950 |
+
descriptor: InferenceFunctionDescriptor,
|
| 951 |
+
name: String,
|
| 952 |
+
values: [Float]
|
| 953 |
+
) throws -> NDArray {
|
| 954 |
+
guard case .ndArray(let inputDescriptor) = descriptor.inputDescriptor(of: name) else {
|
| 955 |
+
throw CoreAIModelSmokeError.inputMissing(name)
|
| 956 |
+
}
|
| 957 |
+
guard inputDescriptor.scalarType == .float32 else {
|
| 958 |
+
throw CoreAIModelSmokeError.unsupportedScalarType(
|
| 959 |
+
"Expected float32 input for \(name), got \(inputDescriptor.scalarType)"
|
| 960 |
+
)
|
| 961 |
+
}
|
| 962 |
+
var array = NDArray(descriptor: inputDescriptor)
|
| 963 |
+
var view = array.mutableView(as: Float.self)
|
| 964 |
+
view.copyElements(fromContentsOf: values)
|
| 965 |
+
return array
|
| 966 |
+
}
|
| 967 |
+
|
| 968 |
+
private func loadFloatFixture(name: String, count: Int) throws -> [Float] {
|
| 969 |
+
guard let url = Bundle.main.url(forResource: name, withExtension: "f32")
|
| 970 |
+
?? Bundle.main.url(
|
| 971 |
+
forResource: name,
|
| 972 |
+
withExtension: "f32",
|
| 973 |
+
subdirectory: "LongDemo"
|
| 974 |
+
)
|
| 975 |
+
else {
|
| 976 |
+
throw CoreAIModelSmokeError.assetMissing("\(name).f32")
|
| 977 |
+
}
|
| 978 |
+
let data = try Data(contentsOf: url)
|
| 979 |
+
guard data.count == count * MemoryLayout<Float>.size else {
|
| 980 |
+
throw CoreAIModelSmokeError.invalidFixture(
|
| 981 |
+
"\(name).f32 has \(data.count) bytes; expected \(count * MemoryLayout<Float>.size)"
|
| 982 |
+
)
|
| 983 |
+
}
|
| 984 |
+
return data.withUnsafeBytes { rawBuffer in
|
| 985 |
+
Array(rawBuffer.bindMemory(to: Float.self))
|
| 986 |
+
}
|
| 987 |
+
}
|
| 988 |
+
|
| 989 |
+
private func predictedDurations(
|
| 990 |
+
logits: NDArray,
|
| 991 |
+
tokenCount: Int,
|
| 992 |
+
binsPerToken: Int,
|
| 993 |
+
validTokenLength: Int
|
| 994 |
+
) -> [Int] {
|
| 995 |
+
precondition(logits.scalarType == .float32)
|
| 996 |
+
var durations = [Int]()
|
| 997 |
+
durations.reserveCapacity(validTokenLength)
|
| 998 |
+
|
| 999 |
+
logits.view(as: Float.self).withUnsafePointer { pointer, shape, strides in
|
| 1000 |
+
precondition(shape.count == 3)
|
| 1001 |
+
precondition(shape[0] == 1)
|
| 1002 |
+
precondition(shape[1] == tokenCount)
|
| 1003 |
+
precondition(shape[2] == binsPerToken)
|
| 1004 |
+
for token in 0..<validTokenLength {
|
| 1005 |
+
var sum: Float = 0
|
| 1006 |
+
for bin in 0..<binsPerToken {
|
| 1007 |
+
let offset = token * strides[1] + bin * strides[2]
|
| 1008 |
+
let value = pointer[offset]
|
| 1009 |
+
sum += 1 / (1 + exp(-value))
|
| 1010 |
+
}
|
| 1011 |
+
durations.append(max(1, Int(sum.rounded())))
|
| 1012 |
+
}
|
| 1013 |
+
}
|
| 1014 |
+
return durations
|
| 1015 |
+
}
|
| 1016 |
+
|
| 1017 |
+
private func fixedAlignment(
|
| 1018 |
+
durations: [Int],
|
| 1019 |
+
tokenCount: Int,
|
| 1020 |
+
frameCount: Int
|
| 1021 |
+
) -> (values: [Float], frameCount: Int, paddedFrames: Int, truncatedFrames: Int) {
|
| 1022 |
+
let predictedFrames = durations.reduce(0, +)
|
| 1023 |
+
var tokenForFrame: [Int] = []
|
| 1024 |
+
for (token, duration) in durations.enumerated() {
|
| 1025 |
+
tokenForFrame.append(contentsOf: repeatElement(token, count: duration))
|
| 1026 |
+
}
|
| 1027 |
+
if tokenForFrame.isEmpty {
|
| 1028 |
+
tokenForFrame = [0]
|
| 1029 |
+
}
|
| 1030 |
+
tokenForFrame = Array(tokenForFrame.prefix(frameCount))
|
| 1031 |
+
|
| 1032 |
+
var values = [Float](repeating: 0, count: tokenCount * frameCount)
|
| 1033 |
+
for frame in 0..<tokenForFrame.count {
|
| 1034 |
+
values[tokenForFrame[frame] * frameCount + frame] = 1
|
| 1035 |
+
}
|
| 1036 |
+
return (
|
| 1037 |
+
values,
|
| 1038 |
+
frameCount,
|
| 1039 |
+
max(0, frameCount - predictedFrames),
|
| 1040 |
+
max(0, predictedFrames - frameCount)
|
| 1041 |
+
)
|
| 1042 |
+
}
|
| 1043 |
+
|
| 1044 |
+
private func projectTokenFeaturesToFrames(
|
| 1045 |
+
tokenFeatures: [Float],
|
| 1046 |
+
tokenCount: Int,
|
| 1047 |
+
featureCount: Int,
|
| 1048 |
+
alignment: [Float],
|
| 1049 |
+
frameCount: Int
|
| 1050 |
+
) -> [Float] {
|
| 1051 |
+
var output = [Float](repeating: 0, count: featureCount * frameCount)
|
| 1052 |
+
for token in 0..<tokenCount {
|
| 1053 |
+
for frame in 0..<frameCount where alignment[token * frameCount + frame] != 0 {
|
| 1054 |
+
for feature in 0..<featureCount {
|
| 1055 |
+
output[feature * frameCount + frame] =
|
| 1056 |
+
tokenFeatures[token * featureCount + feature]
|
| 1057 |
+
}
|
| 1058 |
+
}
|
| 1059 |
+
}
|
| 1060 |
+
return output
|
| 1061 |
+
}
|
| 1062 |
+
|
| 1063 |
+
private func projectChannelTokensToFrames(
|
| 1064 |
+
channelTokens: [Float],
|
| 1065 |
+
channelCount: Int,
|
| 1066 |
+
tokenCount: Int,
|
| 1067 |
+
alignment: [Float],
|
| 1068 |
+
frameCount: Int
|
| 1069 |
+
) -> [Float] {
|
| 1070 |
+
var output = [Float](repeating: 0, count: channelCount * frameCount)
|
| 1071 |
+
for channel in 0..<channelCount {
|
| 1072 |
+
for token in 0..<tokenCount {
|
| 1073 |
+
let value = channelTokens[channel * tokenCount + token]
|
| 1074 |
+
for frame in 0..<frameCount where alignment[token * frameCount + frame] != 0 {
|
| 1075 |
+
output[channel * frameCount + frame] = value
|
| 1076 |
+
}
|
| 1077 |
+
}
|
| 1078 |
+
}
|
| 1079 |
+
return output
|
| 1080 |
+
}
|
| 1081 |
+
|
| 1082 |
+
private func flattenFloat32(_ array: NDArray) -> [Float] {
|
| 1083 |
+
precondition(array.scalarType == .float32)
|
| 1084 |
+
let outerShape = array.shape
|
| 1085 |
+
let rank = outerShape.count
|
| 1086 |
+
let total = outerShape.reduce(1, *)
|
| 1087 |
+
var result = [Float](repeating: 0, count: total)
|
| 1088 |
+
|
| 1089 |
+
array.view(as: Float.self).withUnsafePointer { pointer, shape, strides in
|
| 1090 |
+
var indices = [Int](repeating: 0, count: rank)
|
| 1091 |
+
for outputIndex in 0..<total {
|
| 1092 |
+
var offset = 0
|
| 1093 |
+
for dimension in 0..<rank {
|
| 1094 |
+
offset += indices[dimension] * strides[dimension]
|
| 1095 |
+
}
|
| 1096 |
+
result[outputIndex] = pointer[offset]
|
| 1097 |
+
|
| 1098 |
+
var dimension = rank - 1
|
| 1099 |
+
while dimension >= 0 {
|
| 1100 |
+
indices[dimension] += 1
|
| 1101 |
+
if indices[dimension] < shape[dimension] {
|
| 1102 |
+
break
|
| 1103 |
+
}
|
| 1104 |
+
indices[dimension] = 0
|
| 1105 |
+
dimension -= 1
|
| 1106 |
+
}
|
| 1107 |
+
}
|
| 1108 |
+
}
|
| 1109 |
+
return result
|
| 1110 |
+
}
|
| 1111 |
+
|
| 1112 |
+
private func summarize(_ array: NDArray) -> (min: Float, max: Float, meanAbs: Float) {
|
| 1113 |
+
summarize(flattenFloat32(array))
|
| 1114 |
+
}
|
| 1115 |
+
|
| 1116 |
+
private func summarize(_ values: [Float]) -> (min: Float, max: Float, meanAbs: Float) {
|
| 1117 |
+
guard let first = values.first else {
|
| 1118 |
+
return (0, 0, 0)
|
| 1119 |
+
}
|
| 1120 |
+
var minValue = first
|
| 1121 |
+
var maxValue = first
|
| 1122 |
+
var sumAbs: Float = 0
|
| 1123 |
+
for value in values {
|
| 1124 |
+
minValue = min(minValue, value)
|
| 1125 |
+
maxValue = max(maxValue, value)
|
| 1126 |
+
sumAbs += abs(value)
|
| 1127 |
+
}
|
| 1128 |
+
return (minValue, maxValue, sumAbs / Float(values.count))
|
| 1129 |
+
}
|
| 1130 |
+
|
| 1131 |
+
private func durationSeconds(since start: ContinuousClock.Instant) -> Double {
|
| 1132 |
+
let duration = start.duration(to: .now)
|
| 1133 |
+
return Double(duration.components.seconds)
|
| 1134 |
+
+ Double(duration.components.attoseconds) / 1_000_000_000_000_000_000
|
| 1135 |
+
}
|
| 1136 |
+
#endif
|
| 1137 |
+
}
|
TTSCoreAILab/TTSCoreAILab/SupertonicTTSService.swift
ADDED
|
@@ -0,0 +1,873 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import Foundation
|
| 2 |
+
|
| 3 |
+
#if canImport(CoreAI)
|
| 4 |
+
import CoreAI
|
| 5 |
+
#endif
|
| 6 |
+
|
| 7 |
+
struct SupertonicOutput: Sendable {
|
| 8 |
+
let samples: [Float]
|
| 9 |
+
let sampleRate: Int
|
| 10 |
+
let inferenceSeconds: Double
|
| 11 |
+
let profileID: String
|
| 12 |
+
}
|
| 13 |
+
|
| 14 |
+
actor SupertonicTTSService {
|
| 15 |
+
enum Voice: String, CaseIterable, Identifiable, Sendable {
|
| 16 |
+
case m1 = "M1", m2 = "M2", m3 = "M3", m4 = "M4", m5 = "M5"
|
| 17 |
+
case f1 = "F1", f2 = "F2", f3 = "F3", f4 = "F4", f5 = "F5"
|
| 18 |
+
|
| 19 |
+
var id: String { rawValue }
|
| 20 |
+
var title: String { rawValue }
|
| 21 |
+
}
|
| 22 |
+
|
| 23 |
+
private struct VoiceStyleData: Decodable {
|
| 24 |
+
struct Component: Decodable {
|
| 25 |
+
let data: [[[Float]]]
|
| 26 |
+
}
|
| 27 |
+
|
| 28 |
+
let style_ttl: Component
|
| 29 |
+
let style_dp: Component
|
| 30 |
+
}
|
| 31 |
+
|
| 32 |
+
private struct TextSegment {
|
| 33 |
+
let text: String
|
| 34 |
+
let pauseAfter: Float
|
| 35 |
+
let syntheticEnding: Character?
|
| 36 |
+
}
|
| 37 |
+
|
| 38 |
+
private struct Profile: Sendable {
|
| 39 |
+
static let dynamicID = "dynamic"
|
| 40 |
+
|
| 41 |
+
let id: String
|
| 42 |
+
let textBudget: Int
|
| 43 |
+
let latentBudget: Int
|
| 44 |
+
|
| 45 |
+
var audioWindowSeconds: Float {
|
| 46 |
+
Float(latentBudget * 512 * 6) / 44_100
|
| 47 |
+
}
|
| 48 |
+
|
| 49 |
+
func asset(_ name: String) -> String {
|
| 50 |
+
"supertonic_\(name)_\(id)"
|
| 51 |
+
}
|
| 52 |
+
|
| 53 |
+
static let compact = Profile(id: "64x32", textBudget: 64, latentBudget: 32)
|
| 54 |
+
static let large = Profile(id: "128x64", textBudget: 128, latentBudget: 64)
|
| 55 |
+
static let wide = Profile(id: "128x128", textBudget: 128, latentBudget: 128)
|
| 56 |
+
static let kokoroLarge = Profile(id: "128x256", textBudget: 128, latentBudget: 256)
|
| 57 |
+
}
|
| 58 |
+
|
| 59 |
+
private struct SegmentRender {
|
| 60 |
+
let samples: [Float]
|
| 61 |
+
let pauseAfter: Float
|
| 62 |
+
let profileID: String
|
| 63 |
+
}
|
| 64 |
+
|
| 65 |
+
#if canImport(CoreAI)
|
| 66 |
+
private struct DurationPrediction {
|
| 67 |
+
let duration: Float
|
| 68 |
+
let activeLatents: Int
|
| 69 |
+
}
|
| 70 |
+
#endif
|
| 71 |
+
|
| 72 |
+
private let unicodeIndexer: [Int32]
|
| 73 |
+
private let sampleRate = 44_100
|
| 74 |
+
private let latentChannels = 144
|
| 75 |
+
private let latentChunkSamples = 512 * 6
|
| 76 |
+
#if canImport(CoreAI)
|
| 77 |
+
private var loadedModels: [String: LoadedModel] = [:]
|
| 78 |
+
|
| 79 |
+
private struct LoadedModel {
|
| 80 |
+
let model: AIModel
|
| 81 |
+
let function: InferenceFunction
|
| 82 |
+
}
|
| 83 |
+
#endif
|
| 84 |
+
|
| 85 |
+
init() throws {
|
| 86 |
+
guard let url = Bundle.main.url(
|
| 87 |
+
forResource: "unicode_indexer",
|
| 88 |
+
withExtension: "json"
|
| 89 |
+
) else {
|
| 90 |
+
throw NSError(
|
| 91 |
+
domain: "Supertonic",
|
| 92 |
+
code: 1,
|
| 93 |
+
userInfo: [NSLocalizedDescriptionKey: "Missing unicode_indexer.json."]
|
| 94 |
+
)
|
| 95 |
+
}
|
| 96 |
+
unicodeIndexer = try JSONDecoder().decode(
|
| 97 |
+
[Int32].self,
|
| 98 |
+
from: Data(contentsOf: url)
|
| 99 |
+
)
|
| 100 |
+
}
|
| 101 |
+
|
| 102 |
+
private func profile(id: String, availableProfiles: [Profile]) -> Profile {
|
| 103 |
+
availableProfiles.first { $0.id == id } ?? .compact
|
| 104 |
+
}
|
| 105 |
+
|
| 106 |
+
private func availableProfiles() -> [Profile] {
|
| 107 |
+
var profiles: [Profile] = [.compact]
|
| 108 |
+
for profile in [Profile.large, .wide, .kokoroLarge] {
|
| 109 |
+
if hasAllAssets(for: profile) {
|
| 110 |
+
profiles.append(profile)
|
| 111 |
+
}
|
| 112 |
+
}
|
| 113 |
+
return profiles
|
| 114 |
+
}
|
| 115 |
+
|
| 116 |
+
private func hasAllAssets(for profile: Profile) -> Bool {
|
| 117 |
+
[
|
| 118 |
+
"duration_predictor",
|
| 119 |
+
"text_encoder",
|
| 120 |
+
"vector_estimator",
|
| 121 |
+
"vocoder",
|
| 122 |
+
].allSatisfy { hasAsset(profile.asset($0)) }
|
| 123 |
+
}
|
| 124 |
+
|
| 125 |
+
private func hasAsset(_ asset: String) -> Bool {
|
| 126 |
+
let resourceURL = Bundle.main.resourceURL
|
| 127 |
+
let candidates = [
|
| 128 |
+
Bundle.main.url(
|
| 129 |
+
forResource: asset,
|
| 130 |
+
withExtension: "aimodel",
|
| 131 |
+
subdirectory: "coreai-assets"
|
| 132 |
+
),
|
| 133 |
+
Bundle.main.url(
|
| 134 |
+
forResource: asset,
|
| 135 |
+
withExtension: "aimodel",
|
| 136 |
+
subdirectory: "coreai-assets-baseline"
|
| 137 |
+
),
|
| 138 |
+
Bundle.main.url(forResource: asset, withExtension: "aimodel"),
|
| 139 |
+
resourceURL?
|
| 140 |
+
.appendingPathComponent("coreai-assets", isDirectory: true)
|
| 141 |
+
.appendingPathComponent("\(asset).aimodel", isDirectory: true),
|
| 142 |
+
resourceURL?
|
| 143 |
+
.appendingPathComponent("coreai-assets-baseline", isDirectory: true)
|
| 144 |
+
.appendingPathComponent("\(asset).aimodel", isDirectory: true),
|
| 145 |
+
resourceURL?
|
| 146 |
+
.appendingPathComponent("\(asset).aimodel", isDirectory: true),
|
| 147 |
+
]
|
| 148 |
+
return candidates.compactMap { $0 }.contains {
|
| 149 |
+
FileManager.default.fileExists(atPath: $0.path)
|
| 150 |
+
}
|
| 151 |
+
}
|
| 152 |
+
|
| 153 |
+
func generate(
|
| 154 |
+
_ text: String,
|
| 155 |
+
voice: Voice,
|
| 156 |
+
steps: Int,
|
| 157 |
+
speed: Float,
|
| 158 |
+
profileID: String = Profile.dynamicID
|
| 159 |
+
) async throws -> SupertonicOutput {
|
| 160 |
+
#if canImport(CoreAI)
|
| 161 |
+
let clock = ContinuousClock()
|
| 162 |
+
let start = clock.now
|
| 163 |
+
let style = try loadStyle(voice)
|
| 164 |
+
let availableProfiles = availableProfiles()
|
| 165 |
+
let requestedProfile = profile(
|
| 166 |
+
id: profileID,
|
| 167 |
+
availableProfiles: availableProfiles
|
| 168 |
+
)
|
| 169 |
+
let planningProfile = profileID == Profile.dynamicID
|
| 170 |
+
? (availableProfiles.last ?? .compact)
|
| 171 |
+
: requestedProfile
|
| 172 |
+
var pendingSegments = makeTextSegments(text, profile: planningProfile)
|
| 173 |
+
guard !pendingSegments.isEmpty else {
|
| 174 |
+
throw NSError(
|
| 175 |
+
domain: "Supertonic",
|
| 176 |
+
code: 11,
|
| 177 |
+
userInfo: [NSLocalizedDescriptionKey: "Enter text to synthesize."]
|
| 178 |
+
)
|
| 179 |
+
}
|
| 180 |
+
var renderedSegments: [SegmentRender] = []
|
| 181 |
+
|
| 182 |
+
while !pendingSegments.isEmpty {
|
| 183 |
+
let segment = pendingSegments.removeFirst()
|
| 184 |
+
let textScalarCount = processedScalarCount(segment)
|
| 185 |
+
let durationProfile: Profile?
|
| 186 |
+
if profileID == Profile.dynamicID {
|
| 187 |
+
durationProfile = smallestTextProfile(
|
| 188 |
+
for: textScalarCount,
|
| 189 |
+
availableProfiles: availableProfiles
|
| 190 |
+
)
|
| 191 |
+
} else if textScalarCount <= requestedProfile.textBudget {
|
| 192 |
+
durationProfile = requestedProfile
|
| 193 |
+
} else {
|
| 194 |
+
durationProfile = nil
|
| 195 |
+
}
|
| 196 |
+
|
| 197 |
+
if let durationProfile {
|
| 198 |
+
let prediction = try await predictDuration(
|
| 199 |
+
segment,
|
| 200 |
+
style: style,
|
| 201 |
+
speed: speed,
|
| 202 |
+
profile: durationProfile
|
| 203 |
+
)
|
| 204 |
+
let renderProfile: Profile?
|
| 205 |
+
if profileID == Profile.dynamicID {
|
| 206 |
+
renderProfile = smallestRenderProfile(
|
| 207 |
+
textScalarCount: textScalarCount,
|
| 208 |
+
activeLatents: prediction.activeLatents,
|
| 209 |
+
availableProfiles: availableProfiles
|
| 210 |
+
)
|
| 211 |
+
} else if prediction.activeLatents <= requestedProfile.latentBudget {
|
| 212 |
+
renderProfile = requestedProfile
|
| 213 |
+
} else {
|
| 214 |
+
renderProfile = nil
|
| 215 |
+
}
|
| 216 |
+
|
| 217 |
+
if let renderProfile {
|
| 218 |
+
let samples = try await renderSegment(
|
| 219 |
+
segment,
|
| 220 |
+
style: style,
|
| 221 |
+
steps: steps,
|
| 222 |
+
profile: renderProfile,
|
| 223 |
+
duration: prediction.duration,
|
| 224 |
+
activeLatents: prediction.activeLatents
|
| 225 |
+
)
|
| 226 |
+
renderedSegments.append(
|
| 227 |
+
SegmentRender(
|
| 228 |
+
samples: samples,
|
| 229 |
+
pauseAfter: segment.pauseAfter,
|
| 230 |
+
profileID: profileSummary(
|
| 231 |
+
durationProfile: durationProfile,
|
| 232 |
+
renderProfile: renderProfile
|
| 233 |
+
)
|
| 234 |
+
)
|
| 235 |
+
)
|
| 236 |
+
continue
|
| 237 |
+
}
|
| 238 |
+
}
|
| 239 |
+
|
| 240 |
+
guard let split = splitSegment(segment) else {
|
| 241 |
+
let maxProfile = profileID == Profile.dynamicID
|
| 242 |
+
? (availableProfiles.last ?? requestedProfile)
|
| 243 |
+
: requestedProfile
|
| 244 |
+
throw NSError(
|
| 245 |
+
domain: "Supertonic",
|
| 246 |
+
code: 2,
|
| 247 |
+
userInfo: [
|
| 248 |
+
NSLocalizedDescriptionKey:
|
| 249 |
+
"A single segment exceeds "
|
| 250 |
+
+ "\(maxProfile.textBudget) text units or "
|
| 251 |
+
+ "\(String(format: "%.2f", maxProfile.audioWindowSeconds))s "
|
| 252 |
+
+ "Core AI segment profile."
|
| 253 |
+
]
|
| 254 |
+
)
|
| 255 |
+
}
|
| 256 |
+
pendingSegments.insert(split.right, at: 0)
|
| 257 |
+
pendingSegments.insert(split.left, at: 0)
|
| 258 |
+
}
|
| 259 |
+
|
| 260 |
+
var samples: [Float] = []
|
| 261 |
+
for (index, segment) in renderedSegments.enumerated() {
|
| 262 |
+
samples.append(contentsOf: segment.samples)
|
| 263 |
+
if index < renderedSegments.count - 1 {
|
| 264 |
+
samples.append(
|
| 265 |
+
contentsOf: repeatElement(
|
| 266 |
+
0,
|
| 267 |
+
count: Int(segment.pauseAfter * Float(sampleRate))
|
| 268 |
+
)
|
| 269 |
+
)
|
| 270 |
+
}
|
| 271 |
+
}
|
| 272 |
+
return SupertonicOutput(
|
| 273 |
+
samples: samples,
|
| 274 |
+
sampleRate: sampleRate,
|
| 275 |
+
inferenceSeconds: Self.seconds(from: start, to: clock.now),
|
| 276 |
+
profileID: usedProfileSummary(renderedSegments)
|
| 277 |
+
)
|
| 278 |
+
#else
|
| 279 |
+
throw NSError(
|
| 280 |
+
domain: "Supertonic",
|
| 281 |
+
code: 3,
|
| 282 |
+
userInfo: [NSLocalizedDescriptionKey: "Core AI is unavailable."]
|
| 283 |
+
)
|
| 284 |
+
#endif
|
| 285 |
+
}
|
| 286 |
+
|
| 287 |
+
private func loadStyle(_ voice: Voice) throws -> (ttl: [Float], dp: [Float]) {
|
| 288 |
+
let candidates = [
|
| 289 |
+
Bundle.main.url(
|
| 290 |
+
forResource: voice.rawValue,
|
| 291 |
+
withExtension: "json",
|
| 292 |
+
subdirectory: "voice_styles"
|
| 293 |
+
),
|
| 294 |
+
Bundle.main.url(forResource: voice.rawValue, withExtension: "json"),
|
| 295 |
+
]
|
| 296 |
+
guard let url = candidates.compactMap({ $0 }).first else {
|
| 297 |
+
throw NSError(
|
| 298 |
+
domain: "Supertonic",
|
| 299 |
+
code: 4,
|
| 300 |
+
userInfo: [NSLocalizedDescriptionKey: "Missing \(voice.rawValue).json."]
|
| 301 |
+
)
|
| 302 |
+
}
|
| 303 |
+
let decoded = try JSONDecoder().decode(
|
| 304 |
+
VoiceStyleData.self,
|
| 305 |
+
from: Data(contentsOf: url)
|
| 306 |
+
)
|
| 307 |
+
return (
|
| 308 |
+
decoded.style_ttl.data.flatMap { $0.flatMap { $0 } },
|
| 309 |
+
decoded.style_dp.data.flatMap { $0.flatMap { $0 } }
|
| 310 |
+
)
|
| 311 |
+
}
|
| 312 |
+
|
| 313 |
+
#if canImport(CoreAI)
|
| 314 |
+
private func predictDuration(
|
| 315 |
+
_ segment: TextSegment,
|
| 316 |
+
style: (ttl: [Float], dp: [Float]),
|
| 317 |
+
speed: Float,
|
| 318 |
+
profile: Profile
|
| 319 |
+
) async throws -> DurationPrediction {
|
| 320 |
+
let (textIDs, textMask) = try makeTextInputs(segment, profile: profile)
|
| 321 |
+
let durationArray = try await runModel(
|
| 322 |
+
asset: profile.asset("duration_predictor"),
|
| 323 |
+
inputs: [
|
| 324 |
+
"text_ids": .int32(textIDs),
|
| 325 |
+
"style_dp": .float32(style.dp),
|
| 326 |
+
"text_mask": .float32(textMask),
|
| 327 |
+
],
|
| 328 |
+
output: "duration"
|
| 329 |
+
)
|
| 330 |
+
let duration = flattenFloat32(durationArray)[0]
|
| 331 |
+
/ max(0.5, min(2, speed))
|
| 332 |
+
let activeLatents = Int(
|
| 333 |
+
ceil(duration * Float(sampleRate) / Float(latentChunkSamples))
|
| 334 |
+
)
|
| 335 |
+
return DurationPrediction(duration: duration, activeLatents: activeLatents)
|
| 336 |
+
}
|
| 337 |
+
|
| 338 |
+
private func renderSegment(
|
| 339 |
+
_ segment: TextSegment,
|
| 340 |
+
style: (ttl: [Float], dp: [Float]),
|
| 341 |
+
steps: Int,
|
| 342 |
+
profile: Profile,
|
| 343 |
+
duration: Float,
|
| 344 |
+
activeLatents: Int
|
| 345 |
+
) async throws -> [Float] {
|
| 346 |
+
let (textIDs, textMask) = try makeTextInputs(segment, profile: profile)
|
| 347 |
+
let textEmbedding = try await runModel(
|
| 348 |
+
asset: profile.asset("text_encoder"),
|
| 349 |
+
inputs: [
|
| 350 |
+
"text_ids": .int32(textIDs),
|
| 351 |
+
"style_ttl": .float32(style.ttl),
|
| 352 |
+
"text_mask": .float32(textMask),
|
| 353 |
+
],
|
| 354 |
+
output: "text_emb"
|
| 355 |
+
)
|
| 356 |
+
|
| 357 |
+
var latentMask = [Float](repeating: 0, count: profile.latentBudget)
|
| 358 |
+
for index in 0..<activeLatents {
|
| 359 |
+
latentMask[index] = 1
|
| 360 |
+
}
|
| 361 |
+
var latent = gaussianValues(count: latentChannels * profile.latentBudget)
|
| 362 |
+
for channel in 0..<latentChannels {
|
| 363 |
+
for frame in activeLatents..<profile.latentBudget {
|
| 364 |
+
latent[channel * profile.latentBudget + frame] = 0
|
| 365 |
+
}
|
| 366 |
+
}
|
| 367 |
+
|
| 368 |
+
let vectorModel = try await loadModel(
|
| 369 |
+
asset: profile.asset("vector_estimator")
|
| 370 |
+
)
|
| 371 |
+
let clampedSteps = max(2, min(15, steps))
|
| 372 |
+
for step in 0..<clampedSteps {
|
| 373 |
+
latent = flattenFloat32(
|
| 374 |
+
try await run(
|
| 375 |
+
vectorModel.function,
|
| 376 |
+
inputs: try makeInputs(
|
| 377 |
+
descriptor: vectorModel.function.descriptor,
|
| 378 |
+
values: [
|
| 379 |
+
"noisy_latent": .float32(latent),
|
| 380 |
+
"text_emb": .array(textEmbedding),
|
| 381 |
+
"style_ttl": .float32(style.ttl),
|
| 382 |
+
"latent_mask": .float32(latentMask),
|
| 383 |
+
"text_mask": .float32(textMask),
|
| 384 |
+
"current_step": .float32([Float(step)]),
|
| 385 |
+
"total_step": .float32([Float(clampedSteps)]),
|
| 386 |
+
]
|
| 387 |
+
),
|
| 388 |
+
output: "denoised_latent"
|
| 389 |
+
)
|
| 390 |
+
)
|
| 391 |
+
}
|
| 392 |
+
|
| 393 |
+
let waveform = flattenFloat32(
|
| 394 |
+
try await runModel(
|
| 395 |
+
asset: profile.asset("vocoder"),
|
| 396 |
+
inputs: ["latent": .float32(latent)],
|
| 397 |
+
output: "wav_tts"
|
| 398 |
+
)
|
| 399 |
+
)
|
| 400 |
+
let sampleCount = min(
|
| 401 |
+
waveform.count,
|
| 402 |
+
Int(duration * Float(sampleRate))
|
| 403 |
+
)
|
| 404 |
+
return Array(waveform.prefix(sampleCount))
|
| 405 |
+
}
|
| 406 |
+
#endif
|
| 407 |
+
|
| 408 |
+
private func makeTextInputs(_ segment: TextSegment, profile: Profile) throws -> ([Int32], [Float]) {
|
| 409 |
+
var text = normalizeText(segment.text)
|
| 410 |
+
if let last = text.last, !".!?;:,'\")]}…".contains(last) {
|
| 411 |
+
text.append(segment.syntheticEnding ?? ".")
|
| 412 |
+
}
|
| 413 |
+
text = "<en>\(text)</en>"
|
| 414 |
+
let scalars = Array(text.unicodeScalars)
|
| 415 |
+
guard scalars.count <= profile.textBudget else {
|
| 416 |
+
throw NSError(
|
| 417 |
+
domain: "Supertonic",
|
| 418 |
+
code: 5,
|
| 419 |
+
userInfo: [
|
| 420 |
+
NSLocalizedDescriptionKey:
|
| 421 |
+
"Processed segment uses \(scalars.count) characters; "
|
| 422 |
+
+ "the current Core AI profile supports \(profile.textBudget)."
|
| 423 |
+
]
|
| 424 |
+
)
|
| 425 |
+
}
|
| 426 |
+
var ids = [Int32](repeating: 0, count: profile.textBudget)
|
| 427 |
+
var mask = [Float](repeating: 0, count: profile.textBudget)
|
| 428 |
+
for (index, scalar) in scalars.enumerated() {
|
| 429 |
+
let value = Int(scalar.value)
|
| 430 |
+
ids[index] = value < unicodeIndexer.count ? unicodeIndexer[value] : -1
|
| 431 |
+
mask[index] = 1
|
| 432 |
+
}
|
| 433 |
+
return (ids, mask)
|
| 434 |
+
}
|
| 435 |
+
|
| 436 |
+
private func processedScalarCount(_ segment: TextSegment) -> Int {
|
| 437 |
+
var text = normalizeText(segment.text)
|
| 438 |
+
if let last = text.last, !".!?;:,'\")]}…".contains(last) {
|
| 439 |
+
text.append(segment.syntheticEnding ?? ".")
|
| 440 |
+
}
|
| 441 |
+
return "<en>\(text)</en>".unicodeScalars.count
|
| 442 |
+
}
|
| 443 |
+
|
| 444 |
+
private func smallestTextProfile(
|
| 445 |
+
for textScalarCount: Int,
|
| 446 |
+
availableProfiles: [Profile]
|
| 447 |
+
) -> Profile? {
|
| 448 |
+
availableProfiles
|
| 449 |
+
.sorted { $0.textBudget < $1.textBudget }
|
| 450 |
+
.first { textScalarCount <= $0.textBudget }
|
| 451 |
+
}
|
| 452 |
+
|
| 453 |
+
private func smallestRenderProfile(
|
| 454 |
+
textScalarCount: Int,
|
| 455 |
+
activeLatents: Int,
|
| 456 |
+
availableProfiles: [Profile]
|
| 457 |
+
) -> Profile? {
|
| 458 |
+
availableProfiles
|
| 459 |
+
.sorted {
|
| 460 |
+
if $0.latentBudget == $1.latentBudget {
|
| 461 |
+
return $0.textBudget < $1.textBudget
|
| 462 |
+
}
|
| 463 |
+
return $0.latentBudget < $1.latentBudget
|
| 464 |
+
}
|
| 465 |
+
.first {
|
| 466 |
+
textScalarCount <= $0.textBudget && activeLatents <= $0.latentBudget
|
| 467 |
+
}
|
| 468 |
+
}
|
| 469 |
+
|
| 470 |
+
private func profileSummary(durationProfile: Profile, renderProfile: Profile) -> String {
|
| 471 |
+
durationProfile.id == renderProfile.id
|
| 472 |
+
? renderProfile.id
|
| 473 |
+
: "\(durationProfile.id)->\(renderProfile.id)"
|
| 474 |
+
}
|
| 475 |
+
|
| 476 |
+
private func usedProfileSummary(_ segments: [SegmentRender]) -> String {
|
| 477 |
+
let ids = segments.map(\.profileID)
|
| 478 |
+
let uniqueIDs = Array(Set(ids)).sorted()
|
| 479 |
+
guard !uniqueIDs.isEmpty else { return Profile.compact.id }
|
| 480 |
+
if uniqueIDs.count == 1 {
|
| 481 |
+
return uniqueIDs[0]
|
| 482 |
+
}
|
| 483 |
+
return uniqueIDs.joined(separator: "+")
|
| 484 |
+
}
|
| 485 |
+
|
| 486 |
+
private func normalizeText(_ source: String) -> String {
|
| 487 |
+
var text = source.decomposedStringWithCompatibilityMapping
|
| 488 |
+
let replacements: [String: String] = [
|
| 489 |
+
"–": "-", "‑": "-", "—": "-", "_": " ",
|
| 490 |
+
"“": "\"", "”": "\"", "‘": "'", "’": "'",
|
| 491 |
+
"[": " ", "]": " ", "|": " ", "/": " ", "#": " ",
|
| 492 |
+
"@": " at ",
|
| 493 |
+
]
|
| 494 |
+
for (old, new) in replacements {
|
| 495 |
+
text = text.replacingOccurrences(of: old, with: new)
|
| 496 |
+
}
|
| 497 |
+
text = text
|
| 498 |
+
.split(whereSeparator: { $0.isWhitespace })
|
| 499 |
+
.joined(separator: " ")
|
| 500 |
+
.trimmingCharacters(in: .whitespacesAndNewlines)
|
| 501 |
+
return text
|
| 502 |
+
}
|
| 503 |
+
|
| 504 |
+
private func makeTextSegments(_ source: String, profile: Profile) -> [TextSegment] {
|
| 505 |
+
let text = normalizeText(source)
|
| 506 |
+
guard !text.isEmpty else { return [] }
|
| 507 |
+
|
| 508 |
+
// The language tags occupy 9 of the fixed Unicode-scalar slots.
|
| 509 |
+
let contentBudget = profile.textBudget - 9
|
| 510 |
+
var segments: [TextSegment] = []
|
| 511 |
+
var current = ""
|
| 512 |
+
|
| 513 |
+
for word in sentenceAwareWords(text) {
|
| 514 |
+
let candidate = current.isEmpty ? word : "\(current) \(word)"
|
| 515 |
+
if candidate.unicodeScalars.count <= contentBudget {
|
| 516 |
+
current = candidate
|
| 517 |
+
if let last = word.last, ".!?".contains(last) {
|
| 518 |
+
segments.append(
|
| 519 |
+
TextSegment(
|
| 520 |
+
text: current,
|
| 521 |
+
pauseAfter: pauseDuration(after: last),
|
| 522 |
+
syntheticEnding: nil
|
| 523 |
+
)
|
| 524 |
+
)
|
| 525 |
+
current = ""
|
| 526 |
+
} else if let last = word.last, ",;:".contains(last),
|
| 527 |
+
current.unicodeScalars.count >= 18 {
|
| 528 |
+
segments.append(
|
| 529 |
+
TextSegment(
|
| 530 |
+
text: current,
|
| 531 |
+
pauseAfter: pauseDuration(after: last),
|
| 532 |
+
syntheticEnding: nil
|
| 533 |
+
)
|
| 534 |
+
)
|
| 535 |
+
current = ""
|
| 536 |
+
}
|
| 537 |
+
continue
|
| 538 |
+
}
|
| 539 |
+
|
| 540 |
+
if !current.isEmpty {
|
| 541 |
+
segments.append(
|
| 542 |
+
TextSegment(
|
| 543 |
+
text: current,
|
| 544 |
+
pauseAfter: 0.06,
|
| 545 |
+
syntheticEnding: ","
|
| 546 |
+
)
|
| 547 |
+
)
|
| 548 |
+
current = ""
|
| 549 |
+
}
|
| 550 |
+
var remaining = word
|
| 551 |
+
while remaining.unicodeScalars.count > contentBudget {
|
| 552 |
+
let splitIndex = remaining.unicodeScalars.index(
|
| 553 |
+
remaining.unicodeScalars.startIndex,
|
| 554 |
+
offsetBy: contentBudget
|
| 555 |
+
)
|
| 556 |
+
segments.append(
|
| 557 |
+
TextSegment(
|
| 558 |
+
text: String(remaining.unicodeScalars[..<splitIndex]),
|
| 559 |
+
pauseAfter: 0.06,
|
| 560 |
+
syntheticEnding: ","
|
| 561 |
+
)
|
| 562 |
+
)
|
| 563 |
+
remaining = String(remaining.unicodeScalars[splitIndex...])
|
| 564 |
+
}
|
| 565 |
+
current = remaining
|
| 566 |
+
if let last = current.last, ".!?;:,".contains(last) {
|
| 567 |
+
segments.append(
|
| 568 |
+
TextSegment(
|
| 569 |
+
text: current,
|
| 570 |
+
pauseAfter: pauseDuration(after: last),
|
| 571 |
+
syntheticEnding: nil
|
| 572 |
+
)
|
| 573 |
+
)
|
| 574 |
+
current = ""
|
| 575 |
+
}
|
| 576 |
+
}
|
| 577 |
+
if !current.isEmpty {
|
| 578 |
+
segments.append(
|
| 579 |
+
TextSegment(
|
| 580 |
+
text: current,
|
| 581 |
+
pauseAfter: 0.12,
|
| 582 |
+
syntheticEnding: "."
|
| 583 |
+
)
|
| 584 |
+
)
|
| 585 |
+
}
|
| 586 |
+
return segments
|
| 587 |
+
}
|
| 588 |
+
|
| 589 |
+
private func splitSegment(
|
| 590 |
+
_ segment: TextSegment
|
| 591 |
+
) -> (left: TextSegment, right: TextSegment)? {
|
| 592 |
+
let words = segment.text.split(separator: " ").map(String.init)
|
| 593 |
+
guard words.count > 1 else { return nil }
|
| 594 |
+
|
| 595 |
+
let splitIndex = naturalSplitIndex(words)
|
| 596 |
+
let leftText = words[..<splitIndex].joined(separator: " ")
|
| 597 |
+
let rightText = words[splitIndex...].joined(separator: " ")
|
| 598 |
+
return (
|
| 599 |
+
TextSegment(
|
| 600 |
+
text: leftText,
|
| 601 |
+
pauseAfter: pauseAfterSplit(leftText),
|
| 602 |
+
syntheticEnding: syntheticEnding(for: leftText, fallback: ",")
|
| 603 |
+
),
|
| 604 |
+
TextSegment(
|
| 605 |
+
text: rightText,
|
| 606 |
+
pauseAfter: segment.pauseAfter,
|
| 607 |
+
syntheticEnding: segment.syntheticEnding
|
| 608 |
+
)
|
| 609 |
+
)
|
| 610 |
+
}
|
| 611 |
+
|
| 612 |
+
private func naturalSplitIndex(_ words: [String]) -> Int {
|
| 613 |
+
let target = words.joined(separator: " ").unicodeScalars.count / 2
|
| 614 |
+
var bestIndex: Int?
|
| 615 |
+
var bestScore = Int.max
|
| 616 |
+
var runningCount = 0
|
| 617 |
+
|
| 618 |
+
for index in 0..<(words.count - 1) {
|
| 619 |
+
let word = words[index]
|
| 620 |
+
runningCount += word.unicodeScalars.count + (index == 0 ? 0 : 1)
|
| 621 |
+
guard let last = word.last else { continue }
|
| 622 |
+
|
| 623 |
+
let priority: Int
|
| 624 |
+
if ".!?".contains(last) {
|
| 625 |
+
priority = 0
|
| 626 |
+
} else if ";:".contains(last) {
|
| 627 |
+
priority = 1
|
| 628 |
+
} else if ",".contains(last) {
|
| 629 |
+
priority = 2
|
| 630 |
+
} else {
|
| 631 |
+
continue
|
| 632 |
+
}
|
| 633 |
+
|
| 634 |
+
let distance = abs(runningCount - target)
|
| 635 |
+
let score = distance * 4 + priority
|
| 636 |
+
if score < bestScore {
|
| 637 |
+
bestScore = score
|
| 638 |
+
bestIndex = index + 1
|
| 639 |
+
}
|
| 640 |
+
}
|
| 641 |
+
|
| 642 |
+
if let bestIndex {
|
| 643 |
+
return bestIndex
|
| 644 |
+
}
|
| 645 |
+
|
| 646 |
+
runningCount = 0
|
| 647 |
+
var leftWords = 0
|
| 648 |
+
for word in words.dropLast() {
|
| 649 |
+
let added = word.unicodeScalars.count + (leftWords == 0 ? 0 : 1)
|
| 650 |
+
if leftWords > 0, runningCount + added > target {
|
| 651 |
+
break
|
| 652 |
+
}
|
| 653 |
+
runningCount += added
|
| 654 |
+
leftWords += 1
|
| 655 |
+
}
|
| 656 |
+
return max(1, leftWords)
|
| 657 |
+
}
|
| 658 |
+
|
| 659 |
+
private func syntheticEnding(for text: String, fallback: Character) -> Character? {
|
| 660 |
+
guard let last = text.last else { return fallback }
|
| 661 |
+
return ".!?;:,'\")]}…".contains(last) ? nil : fallback
|
| 662 |
+
}
|
| 663 |
+
|
| 664 |
+
private func pauseAfterSplit(_ text: String) -> Float {
|
| 665 |
+
guard let last = text.last else { return 0.06 }
|
| 666 |
+
return ".!?;:,".contains(last) ? pauseDuration(after: last) : 0.06
|
| 667 |
+
}
|
| 668 |
+
|
| 669 |
+
private func sentenceAwareWords(_ text: String) -> [String] {
|
| 670 |
+
var spaced = ""
|
| 671 |
+
let characters = Array(text)
|
| 672 |
+
for index in characters.indices {
|
| 673 |
+
let character = characters[index]
|
| 674 |
+
spaced.append(character)
|
| 675 |
+
guard ".!?;:,".contains(character),
|
| 676 |
+
index + 1 < characters.count,
|
| 677 |
+
!characters[index + 1].isWhitespace else {
|
| 678 |
+
continue
|
| 679 |
+
}
|
| 680 |
+
let isDecimalPoint = character == "."
|
| 681 |
+
&& index > 0
|
| 682 |
+
&& characters[index - 1].isNumber
|
| 683 |
+
&& characters[index + 1].isNumber
|
| 684 |
+
if !isDecimalPoint {
|
| 685 |
+
spaced.append(" ")
|
| 686 |
+
}
|
| 687 |
+
}
|
| 688 |
+
return spaced.split(whereSeparator: { $0.isWhitespace }).map(String.init)
|
| 689 |
+
}
|
| 690 |
+
|
| 691 |
+
private func pauseDuration(after punctuation: Character) -> Float {
|
| 692 |
+
switch punctuation {
|
| 693 |
+
case ".", "!", "?":
|
| 694 |
+
return 0.38
|
| 695 |
+
case ";", ":":
|
| 696 |
+
return 0.24
|
| 697 |
+
case ",":
|
| 698 |
+
return 0.16
|
| 699 |
+
default:
|
| 700 |
+
return 0.10
|
| 701 |
+
}
|
| 702 |
+
}
|
| 703 |
+
|
| 704 |
+
private func gaussianValues(count: Int) -> [Float] {
|
| 705 |
+
var values = [Float]()
|
| 706 |
+
values.reserveCapacity(count)
|
| 707 |
+
while values.count < count {
|
| 708 |
+
let u1 = Float.random(in: 0.0001...1)
|
| 709 |
+
let u2 = Float.random(in: 0...1)
|
| 710 |
+
let radius = sqrt(-2 * log(u1))
|
| 711 |
+
values.append(radius * cos(2 * .pi * u2))
|
| 712 |
+
if values.count < count {
|
| 713 |
+
values.append(radius * sin(2 * .pi * u2))
|
| 714 |
+
}
|
| 715 |
+
}
|
| 716 |
+
return values
|
| 717 |
+
}
|
| 718 |
+
|
| 719 |
+
#if canImport(CoreAI)
|
| 720 |
+
private enum InputValue {
|
| 721 |
+
case int32([Int32])
|
| 722 |
+
case float32([Float])
|
| 723 |
+
case array(NDArray)
|
| 724 |
+
}
|
| 725 |
+
|
| 726 |
+
private func loadModel(
|
| 727 |
+
asset: String
|
| 728 |
+
) async throws -> LoadedModel {
|
| 729 |
+
if let loaded = loadedModels[asset] {
|
| 730 |
+
return loaded
|
| 731 |
+
}
|
| 732 |
+
|
| 733 |
+
let resourceURL = Bundle.main.resourceURL
|
| 734 |
+
let candidates = [
|
| 735 |
+
Bundle.main.url(
|
| 736 |
+
forResource: asset,
|
| 737 |
+
withExtension: "aimodel",
|
| 738 |
+
subdirectory: "coreai-assets"
|
| 739 |
+
),
|
| 740 |
+
Bundle.main.url(
|
| 741 |
+
forResource: asset,
|
| 742 |
+
withExtension: "aimodel",
|
| 743 |
+
subdirectory: "coreai-assets-baseline"
|
| 744 |
+
),
|
| 745 |
+
Bundle.main.url(forResource: asset, withExtension: "aimodel"),
|
| 746 |
+
resourceURL?
|
| 747 |
+
.appendingPathComponent("coreai-assets", isDirectory: true)
|
| 748 |
+
.appendingPathComponent("\(asset).aimodel", isDirectory: true),
|
| 749 |
+
resourceURL?
|
| 750 |
+
.appendingPathComponent("coreai-assets-baseline", isDirectory: true)
|
| 751 |
+
.appendingPathComponent("\(asset).aimodel", isDirectory: true),
|
| 752 |
+
resourceURL?
|
| 753 |
+
.appendingPathComponent("\(asset).aimodel", isDirectory: true),
|
| 754 |
+
]
|
| 755 |
+
guard let url = candidates.compactMap({ $0 }).first(where: {
|
| 756 |
+
FileManager.default.fileExists(atPath: $0.path)
|
| 757 |
+
}) else {
|
| 758 |
+
throw NSError(
|
| 759 |
+
domain: "Supertonic",
|
| 760 |
+
code: 6,
|
| 761 |
+
userInfo: [NSLocalizedDescriptionKey: "Missing \(asset).aimodel."]
|
| 762 |
+
)
|
| 763 |
+
}
|
| 764 |
+
let options = SpecializationOptions(preferredComputeUnitKind: .gpu)
|
| 765 |
+
let model = try await AIModel(contentsOf: url, options: options)
|
| 766 |
+
guard let function = try model.loadFunction(named: "main") else {
|
| 767 |
+
throw NSError(
|
| 768 |
+
domain: "Supertonic",
|
| 769 |
+
code: 7,
|
| 770 |
+
userInfo: [NSLocalizedDescriptionKey: "Cannot load \(asset)."]
|
| 771 |
+
)
|
| 772 |
+
}
|
| 773 |
+
let loaded = LoadedModel(model: model, function: function)
|
| 774 |
+
loadedModels[asset] = loaded
|
| 775 |
+
return loaded
|
| 776 |
+
}
|
| 777 |
+
|
| 778 |
+
private func runModel(
|
| 779 |
+
asset: String,
|
| 780 |
+
inputs: [String: InputValue],
|
| 781 |
+
output: String
|
| 782 |
+
) async throws -> NDArray {
|
| 783 |
+
let loaded = try await loadModel(asset: asset)
|
| 784 |
+
return try await run(
|
| 785 |
+
loaded.function,
|
| 786 |
+
inputs: try makeInputs(
|
| 787 |
+
descriptor: loaded.function.descriptor,
|
| 788 |
+
values: inputs
|
| 789 |
+
),
|
| 790 |
+
output: output
|
| 791 |
+
)
|
| 792 |
+
}
|
| 793 |
+
|
| 794 |
+
private func makeInputs(
|
| 795 |
+
descriptor: InferenceFunctionDescriptor,
|
| 796 |
+
values: [String: InputValue]
|
| 797 |
+
) throws -> [String: NDArray] {
|
| 798 |
+
var result: [String: NDArray] = [:]
|
| 799 |
+
for (name, value) in values {
|
| 800 |
+
switch value {
|
| 801 |
+
case .array(let array):
|
| 802 |
+
result[name] = array
|
| 803 |
+
case .int32(let contents):
|
| 804 |
+
guard case .ndArray(let input) = descriptor.inputDescriptor(of: name) else {
|
| 805 |
+
throw NSError(domain: "Supertonic", code: 8)
|
| 806 |
+
}
|
| 807 |
+
var array = NDArray(descriptor: input)
|
| 808 |
+
var view = array.mutableView(as: Int32.self)
|
| 809 |
+
view.copyElements(fromContentsOf: contents)
|
| 810 |
+
result[name] = array
|
| 811 |
+
case .float32(let contents):
|
| 812 |
+
guard case .ndArray(let input) = descriptor.inputDescriptor(of: name) else {
|
| 813 |
+
throw NSError(domain: "Supertonic", code: 9)
|
| 814 |
+
}
|
| 815 |
+
var array = NDArray(descriptor: input)
|
| 816 |
+
var view = array.mutableView(as: Float.self)
|
| 817 |
+
view.copyElements(fromContentsOf: contents)
|
| 818 |
+
result[name] = array
|
| 819 |
+
}
|
| 820 |
+
}
|
| 821 |
+
return result
|
| 822 |
+
}
|
| 823 |
+
|
| 824 |
+
private func run(
|
| 825 |
+
_ function: InferenceFunction,
|
| 826 |
+
inputs: [String: NDArray],
|
| 827 |
+
output: String
|
| 828 |
+
) async throws -> NDArray {
|
| 829 |
+
var outputs = try await function.run(inputs: inputs)
|
| 830 |
+
guard let array = outputs.remove(output)?.ndArray else {
|
| 831 |
+
throw NSError(
|
| 832 |
+
domain: "Supertonic",
|
| 833 |
+
code: 10,
|
| 834 |
+
userInfo: [NSLocalizedDescriptionKey: "Missing output \(output)."]
|
| 835 |
+
)
|
| 836 |
+
}
|
| 837 |
+
return array
|
| 838 |
+
}
|
| 839 |
+
|
| 840 |
+
private func flattenFloat32(_ array: NDArray) -> [Float] {
|
| 841 |
+
let shape = array.shape
|
| 842 |
+
let total = shape.reduce(1, *)
|
| 843 |
+
var result = [Float](repeating: 0, count: total)
|
| 844 |
+
array.view(as: Float.self).withUnsafePointer { pointer, actualShape, strides in
|
| 845 |
+
var indices = [Int](repeating: 0, count: actualShape.count)
|
| 846 |
+
for outputIndex in 0..<total {
|
| 847 |
+
var offset = 0
|
| 848 |
+
for dimension in actualShape.indices {
|
| 849 |
+
offset += indices[dimension] * strides[dimension]
|
| 850 |
+
}
|
| 851 |
+
result[outputIndex] = pointer[offset]
|
| 852 |
+
for dimension in actualShape.indices.reversed() {
|
| 853 |
+
indices[dimension] += 1
|
| 854 |
+
if indices[dimension] < actualShape[dimension] {
|
| 855 |
+
break
|
| 856 |
+
}
|
| 857 |
+
indices[dimension] = 0
|
| 858 |
+
}
|
| 859 |
+
}
|
| 860 |
+
}
|
| 861 |
+
return result
|
| 862 |
+
}
|
| 863 |
+
#endif
|
| 864 |
+
|
| 865 |
+
private static func seconds(
|
| 866 |
+
from start: ContinuousClock.Instant,
|
| 867 |
+
to end: ContinuousClock.Instant
|
| 868 |
+
) -> Double {
|
| 869 |
+
let duration = start.duration(to: end)
|
| 870 |
+
return Double(duration.components.seconds)
|
| 871 |
+
+ Double(duration.components.attoseconds) / 1_000_000_000_000_000_000
|
| 872 |
+
}
|
| 873 |
+
}
|