teddybear082 commited on
Commit
47a0ea8
·
1 Parent(s): 9bfa614

Eliminate unnecessary dev files

Browse files
pocket-tts-generate-docs.md DELETED
@@ -1,83 +0,0 @@
1
- # Generate Command Documentation
2
-
3
- The `generate` command allows you to generate speech from text directly from the command line using Kyutai Pocket TTS.
4
-
5
- ## Basic Usage
6
-
7
- ```bash
8
- uvx pocket-tts generate
9
- # or if installed manually:
10
- pocket-tts generate
11
- ```
12
-
13
- This will generate a WAV file `./tts_output.wav` with the default text and voice.
14
-
15
- ## Command Options
16
-
17
- ### Core Options
18
-
19
- - `--text TEXT`: Text to generate (default: "Hello world! I am Kyutai Pocket TTS. I'm fast enough to run on small CPUs. I hope you'll like me.")
20
- - `--voice VOICE`: Path to audio conditioning file (voice to clone) (default: "hf://kyutai/tts-voices/alba-mackenna/casual.wav"). Urls and local paths are supported.
21
- - `--output-path OUTPUT_PATH`: Output path for generated audio (default: "./tts_output.wav")
22
-
23
- ### Generation Parameters
24
-
25
- - `--variant VARIANT`: Model signature (default: "b6369a24")
26
- - `--lsd-decode-steps LSD_DECODE_STEPS`: Number of generation steps (default: 1)
27
- - `--temperature TEMPERATURE`: Temperature for generation (default: 0.7)
28
- - `--noise-clamp NOISE_CLAMP`: Noise clamp value (default: None)
29
- - `--eos-threshold EOS_THRESHOLD`: EOS threshold (default: -4.0)
30
- - `--frames-after-eos FRAMES_AFTER_EOS`: Number of frames to generate after EOS (default: None, auto-calculated based on the text length). Each frame is 80ms.
31
-
32
- ### Performance Options
33
-
34
- - `--device DEVICE`: Device to use (default: "cpu", you may not get a speedup by using a gpu since it's a small model)
35
- - `--quiet`, `-q`: Disable logging output
36
-
37
- ## Examples
38
-
39
- ### Basic Generation
40
-
41
- ```bash
42
- # Generate with default settings
43
- pocket-tts generate
44
-
45
- # Custom text
46
- pocket-tts generate --text "Hello, this is a custom message."
47
-
48
- # Custom output path
49
- pocket-tts generate --output-path "./my_audio.wav"
50
- ```
51
-
52
- ### Voice Selection
53
-
54
- ```bash
55
- # Use different voice from HuggingFace
56
- pocket-tts generate --voice "hf://kyutai/tts-voices/jessica-jian/casual.wav"
57
-
58
- # Use local voice file
59
- pocket-tts generate --voice "./my_voice.wav"
60
- ```
61
-
62
- ### Quality Tuning
63
-
64
- ```bash
65
- # Higher quality (more steps)
66
- pocket-tts generate --lsd-decode-steps 5 --temperature 0.5
67
-
68
- # More expressive (higher temperature)
69
- pocket-tts generate --temperature 1.0
70
-
71
- # Adjust EOS threshold, smaller means finishing earlier.
72
- pocket-tts generate --eos-threshold -3.0
73
- ```
74
-
75
- ## Output Format
76
-
77
- The generate command always outputs WAV files in the following format:
78
- - **Sample Rate**: 24kHz
79
- - **Channels**: Mono
80
- - **Bit Depth**: 16-bit PCM
81
- - **Format**: Standard WAV file
82
-
83
- For more advanced usage, see the [Python API documentation](python-api.md) or consider using the [serve command](serve.md) for web-based generation and quick iteration.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
pocket-tts-python-api.md DELETED
@@ -1,223 +0,0 @@
1
- # Python API Documentation
2
-
3
- Kyutai Pocket TTS provides a Python API for integrating text-to-speech capabilities into your applications.
4
-
5
- ## Installation
6
-
7
- ```bash
8
- pip install pocket-tts
9
- ```
10
-
11
- ## Quick Start
12
-
13
- ```python
14
- from pocket_tts import TTSModel
15
- import scipy.io.wavfile
16
-
17
- # Load the model
18
- tts_model = TTSModel.load_model()
19
-
20
- # Get voice state from an audio file
21
- voice_state = tts_model.get_state_for_audio_prompt(
22
- "hf://kyutai/tts-voices/alba-mackenna/casual.wav"
23
- )
24
-
25
- # Generate audio
26
- audio = tts_model.generate_audio(voice_state, "Hello world, this is a test.")
27
-
28
- # Save to file
29
- scipy.io.wavfile.write("output.wav", tts_model.sample_rate, audio.numpy())
30
- ```
31
-
32
- ## Core Classes
33
-
34
- ### TTSModel
35
-
36
- The main class for text-to-speech generation.
37
-
38
- #### Class Methods
39
-
40
- ##### `load_model(variant="b6369a24", temp=0.7, lsd_decode_steps=1, noise_clamp=None, eos_threshold=-4.0)`
41
-
42
- Load and return a TTSModel instance with pre-trained weights.
43
-
44
- **Parameters:**
45
- - `variant` (str): Model variant identifier (default: "b6369a24")
46
- - `temp` (float): Sampling temperature for generation (default: 0.7)
47
- - `lsd_decode_steps` (int): Number of generation steps (default: 1)
48
- - `noise_clamp` (float | None): Maximum value for noise sampling (default: None)
49
- - `eos_threshold` (float): Threshold for end-of-sequence detection (default: -4.0)
50
-
51
- **Returns:**
52
- - `TTSModel`: Loaded model instance on CPU
53
-
54
- **Example:**
55
- ```python
56
- from pocket_tts import TTSModel
57
-
58
- # Load with default settings
59
- model = TTSModel.load_model()
60
-
61
- # Load with custom parameters
62
- model = TTSModel.load_model(variant="b6369a24", temp=0.5, lsd_decode_steps=5, eos_threshold=-3.0)
63
- ```
64
-
65
- #### Properties
66
-
67
- ##### `device` (str)
68
-
69
- Returns the device type where the model is running ("cpu" or "cuda").
70
- By default, the model runs on CPU.
71
-
72
- ```python
73
- from pocket_tts import TTSModel
74
-
75
- model = TTSModel.load_model()
76
- print(f"Model running on: {model.device}")
77
- ```
78
-
79
- ##### `sample_rate` (int)
80
-
81
- Returns the generated audio sample rate (typically 24000 Hz).
82
-
83
- ```python
84
- from pocket_tts import TTSModel
85
-
86
- model = TTSModel.load_model()
87
- print(f"Sample rate: {model.sample_rate} Hz")
88
- ```
89
-
90
- #### Methods
91
-
92
- ##### `get_state_for_audio_prompt(audio_conditioning, truncate=False)`
93
-
94
- Extract model state for a given audio file or URL (voice cloning).
95
-
96
- **Parameters:**
97
- - `audio_conditioning` (Path | str | torch.Tensor): Audio file path, URL, or tensor
98
- - `truncate` (bool): Whether to truncate the audio (default: False)
99
-
100
- **Returns:**
101
- - `dict`: Model state dictionary containing hidden states and positional information
102
-
103
- **Example:**
104
- ```python
105
- from pocket_tts import TTSModel
106
-
107
- model = TTSModel.load_model()
108
- # From HuggingFace URL
109
- voice_state = model.get_state_for_audio_prompt("hf://kyutai/tts-voices/alba-mackenna/casual.wav")
110
-
111
- # From local file
112
- voice_state = model.get_state_for_audio_prompt("./my_voice.wav")
113
-
114
- # From HTTP URL
115
- voice_state = model.get_state_for_audio_prompt(
116
- "https://huggingface.co/kyutai/tts-voices/resolve"
117
- "/main/expresso/ex01-ex02_default_001_channel1_168s.wav"
118
- )
119
- ```
120
-
121
- ##### `generate_audio(model_state, text_to_generate, frames_after_eos=None, copy_state=True)`
122
-
123
- Generate complete audio tensor from text input.
124
-
125
- **Parameters:**
126
- - `model_state` (dict): Model state from `get_state_for_audio_prompt()`
127
- - `text_to_generate` (str): Text to convert to speech
128
- - `frames_after_eos` (int | None): Frames to generate after EOS detection (default: None)
129
- - `copy_state` (bool): Whether to copy the state (default: True)
130
-
131
- **Returns:**
132
- - `torch.Tensor`: Audio 1D tensor with shape [samples]
133
-
134
- **Example:**
135
- ```python
136
- from pocket_tts import TTSModel
137
-
138
- model = TTSModel.load_model()
139
-
140
- voice_state = model.get_state_for_audio_prompt("hf://kyutai/tts-voices/alba-mackenna/casual.wav")
141
-
142
- # Generate audio
143
- audio = model.generate_audio(voice_state, "Hello world!", frames_after_eos=2, copy_state=True)
144
-
145
- print(f"Generated audio shape: {audio.shape}")
146
- print(f"Audio duration: {audio.shape[-1] / model.sample_rate:.2f} seconds")
147
- ```
148
-
149
- ##### `generate_audio_stream(model_state, text_to_generate, frames_after_eos=None, copy_state=True)`
150
-
151
- Generate audio streaming chunks from text input.
152
-
153
- **Parameters:** Same as `generate_audio()`
154
-
155
- **Yields:**
156
- - `torch.Tensor`: Audio chunks with shape [samples]
157
-
158
- **Example:**
159
- ```python
160
- from pocket_tts import TTSModel
161
-
162
- model = TTSModel.load_model()
163
-
164
- voice_state = model.get_state_for_audio_prompt("hf://kyutai/tts-voices/alba-mackenna/casual.wav")
165
- # Stream generation
166
- for chunk in model.generate_audio_stream(voice_state, "Long text content..."):
167
- # Process each chunk as it's generated
168
- print(f"Generated chunk: {chunk.shape[0]} samples")
169
- # Could save chunks to file or play in real-time
170
- ```
171
-
172
- ## Advanced Usage
173
-
174
- ### Voice Management
175
-
176
- ```python
177
- from pocket_tts import TTSModel
178
-
179
- model = TTSModel.load_model()
180
- # Preload multiple voices
181
- voices = {
182
- "casual": model.get_state_for_audio_prompt("hf://kyutai/tts-voices/alba-mackenna/casual.wav"),
183
- "funny": model.get_state_for_audio_prompt(
184
- "https://huggingface.co/kyutai/tts-voices/resolve/main/expresso/ex01-ex02_default_001_channel1_168s.wav"
185
- ),
186
- }
187
-
188
- # Generate with different voices
189
- casual_audio = model.generate_audio(voices["casual"], "Hey there!")
190
- funny_audio = model.generate_audio(voices["funny"], "Good morning.")
191
- ```
192
-
193
- ### Batch Processing
194
-
195
- ```python
196
- from pocket_tts import TTSModel
197
- import scipy.io.wavfile
198
- import torch
199
-
200
- model = TTSModel.load_model()
201
-
202
- voice_state = model.get_state_for_audio_prompt("hf://kyutai/tts-voices/alba-mackenna/casual.wav")
203
- # Process multiple texts efficiently by re-using the same voice state
204
- texts = [
205
- "First sentence to generate.",
206
- "Second sentence to generate.",
207
- "Third sentence to generate.",
208
- ]
209
-
210
- audios = []
211
- for text in texts:
212
- audio = model.generate_audio(voice_state, text)
213
- audios.append(audio)
214
-
215
- # Concatenate all audio
216
- full_audio = torch.cat(audios, dim=0)
217
- scipy.io.wavfile.write("batch_output.wav", model.sample_rate, full_audio.numpy())
218
- ```
219
-
220
- ### Streaming to File
221
- You can refer to our CLI implementation which can stream audio to a wav file.
222
-
223
- For more information about the command-line interface, see the [Generate Documentation](generate.md) or [Serve Documentation](serve.md).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
project specifications.txt DELETED
@@ -1,62 +0,0 @@
1
- Project Specification: Universal TTS API Wrapper
2
- 1. Project Objective
3
- Develop a production-ready Flask wrapper for pocket-tts that supports synthesis, voice cloning, and streaming audio, compatible with standard industry API TTS schemas (OpenAI) and includes a simple but elegant preview website that runs on the same host/post as the TTS server endpoint .
4
-
5
- 2. Phase 1: Infrastructure & Configuration
6
- Implement a robust CLI-driven configuration layer to handle model loading and hardware orchestration.
7
-
8
- CLI Argument Parser: Support arguments for --model_path, --host (host to run server on), --port (post to run server on), and --stresm (if added, use streaming TTS)
9
-
10
- 3. Phase 2: Core Synthesis Logic
11
- Develop necessary functions to bridge the API and the model to allow for streaming and non-streaming voice TTS generation.
12
-
13
- Text Pre-processing: Integrate a text cleaner/tokenizer to handle language-specific normalization.
14
-
15
- Audio Post-processing:
16
-
17
- Function to convert raw model output (tensors) into a BytesIO buffer.
18
-
19
- Use torchaudio to convert raw audio into various formats: mp3, wav, opus, aac, and flac.
20
-
21
- Voice Cloning/Conditioning: Support accepting a path to a reference audio file for zero-shot cloning. If supported by model, cache latents to avoid having to reprocess wav files for cloning each generarion.
22
-
23
- 4. Phase 3: API Surface Implementation
24
- Expose the model to the user in two ways:
25
-
26
- A. Simple web page at root of host/port used for server
27
- Use pocket-tts-logo.png and develop a simple localhost html and or css web page that is run automatically when running the server and allows the user to test and play output from the built in voices or select a wav file from their file system to clone and provides from default text prompts as well as a free text input field the user can utilize to test the model, and download output wavs through a file selection dislog to a location of their choice.
28
-
29
- B. OpenAI Compatibility Layer endpoint (/v1/audio/speech)
30
- Endpoint: Follow OpenAI’s JSON payload schema: { "model": "...", "input": "...", "voice": "...", "response_format": "mp3", "speed": 1.0 }.
31
- The response format should support the formats noted above
32
- Speed should be passed to the model if supported by pocket-tts to control how fast the TTS plays
33
- The "voice" parameter should accept one of the model's built in voice names or the absolute path to a .wav file specified by the user
34
- Model should always be defaulted to "pocket-tts" if not specified and should have no i,pact on the actual voice generation as this server is only for use with the pocket-tts model
35
- Logic: Map "voice" strings to internal speaker IDs (for voices supported natively by the model) or paths to cloning .wav files.
36
- Non-streaming response: If not using streaming via command line argument when running server, generate a non-streaming file response supporting all the audio formats specified above
37
- Streaming: Implement steaming audio response option at user's discretion using Response(generate_chunks(), mimetype=...).
38
-
39
- Voices endooint: also include a voices endpoint that will return built in voices and the wav files in a directory for cloning wav files specified by the user as a command line argument when running the server:
40
- Voices endpoint sample to adapt:
41
-
42
- @app.route("/v1/voices", methods=["GET"])
43
- def openai_list_voices():
44
- """JSON list of available built-in voice names.
45
- Not an official OpenAI endpoint; provided for convenience for clients.
46
- """
47
- voices = _get_builtin_voice_names()
48
- return jsonify(
49
- {
50
- "object": "list",
51
- "data": [{"id": v, "object": "voice"} for v in voices],
52
- }
53
- )
54
-
55
- 5. Phase 4: Error Handling: Implement try-except blocks around relevant logic to return 500-level JSON errors instead of crashing the Flask thread.
56
-
57
- 6. Technical Requirements
58
- Language: Python 3.11
59
-
60
- Framework: Flask
61
-
62
- Core Libraries: torch, torchaudio, numpy, whatever else needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
test_server.py DELETED
@@ -1,79 +0,0 @@
1
- import subprocess
2
- import time
3
- import requests
4
- import sys
5
- import os
6
-
7
- def test_server():
8
- print("Starting server for testing...")
9
- # Start server as a subprocess
10
- process = subprocess.Popen(
11
- [sys.executable, "pocket_tts_openai_server.py", "--port", "5001"],
12
- stdout=subprocess.PIPE,
13
- stderr=subprocess.PIPE,
14
- text=True
15
- )
16
-
17
- try:
18
- # Wait for valid startup
19
- print("Waiting for server to start...")
20
- time.sleep(15) # Give it time to load model
21
-
22
- base_url = "http://localhost:5001"
23
-
24
- # 1. Test Home
25
- try:
26
- r = requests.get(base_url + "/")
27
- print(f"GET /: {r.status_code}")
28
- if r.status_code == 200 and "Pocket TTS" in r.text:
29
- print("SUCCESS: Home page loaded.")
30
- else:
31
- print("FAILURE: Home page check failed.")
32
- except Exception as e:
33
- print(f"FAILURE: Could not connect to home page: {e}")
34
-
35
- # 2. Test Voices List
36
- try:
37
- r = requests.get(base_url + "/v1/voices")
38
- print(f"GET /v1/voices: {r.status_code}")
39
- if r.status_code == 200:
40
- data = r.json()
41
- if data.get("object") == "list" and len(data.get("data", [])) > 0:
42
- print(f"SUCCESS: Voices list returned {len(data['data'])} voices.")
43
- else:
44
- print("FAILURE: Voices list format incorrect.")
45
- else:
46
- print("FAILURE: Voices endpoint returned error.")
47
- except Exception as e:
48
- print(f"FAILURE: Voices test failed: {e}")
49
-
50
- # 3. Test Generation (Mock or Real)
51
- payload = {
52
- "model": "pocket-tts",
53
- "input": "Hi",
54
- "voice": "hf://kyutai/tts-voices/alba-mackenna/casual.wav",
55
- "response_format": "wav"
56
- }
57
- try:
58
- t0 = time.time()
59
- r = requests.post(base_url + "/v1/audio/speech", json=payload, timeout=60)
60
- print(f"POST /v1/audio/speech: {r.status_code} (took {time.time()-t0:.2f}s)")
61
- if r.status_code == 200:
62
- print(f"SUCCESS: Audio generated ({len(r.content)} bytes).")
63
- else:
64
- print(f"FAILURE: Generation failed: {r.text}")
65
- except Exception as e:
66
- print(f"FAILURE: Generation test failed: {e}")
67
-
68
- finally:
69
- print("Terminating server...")
70
- process.terminate()
71
- try:
72
- outs, errs = process.communicate(timeout=5)
73
- # print("Server Output:", outs)
74
- # print("Server Errors:", errs)
75
- except:
76
- process.kill()
77
-
78
- if __name__ == "__main__":
79
- test_server()