update readme

README.md

@@ -16,9 +16,10 @@ Open-weights Dutch TTS based on the [Parakeet](https://jordandarefsky.com/blog/2
 
 | Text | File |
 |---|---|
-| [S1] denk je dat je een open source model kan trainen met weinig geld en middelen? [S2] ja ik denk het wel. [S1] oh ja, hoe dan? [S2] nou kijk maar in de repo (laughs). | [intro](https://github.com/pevers/parkiet/raw/refs/heads/main/samples/intro.mp3)
+| [S1] denk je dat je een open source model kan trainen met weinig geld en middelen? [S2] ja ik denk het wel. [S1] oh ja, hoe dan? [S2] nou kijk maar in de repo op Git Hub of Hugging Face. | [intro](https://github.com/pevers/parkiet/raw/refs/heads/main/samples/intro.mp3)
 | [S1] hoeveel stemmen worden er ondersteund? [S2] nou, uhm, ik denk toch wel meer dan twee. [S3] ja, ja, d dat is het mooie aan dit model. [S4] ja klopt, het ondersteund tot vier verschillende stemmen per prompt. | [multi](https://github.com/pevers/parkiet/raw/refs/heads/main/samples/multi.mp3) 
 | [S1] h h et is dus ook mogelijk, om eh ... uhm, heel veel t te st stotteren in een prompt. | [stutter](https://github.com/pevers/parkiet/raw/refs/heads/main/samples/stutter.mp3) |
+| [S1] (laughs) luister, ik heb een mop, wat uhm, drinkt een webdesigner het liefst? [S2] nou ... ? [S1] Earl Grey (laughs) . [S2] (laughs) heel goed. | [laughs](https://github.com/pevers/parkiet/raw/refs/heads/main/samples/laughs.mp3)
 | [S1] je hebt maar weinig audio nodig om een stem te clonen de rest van deze tekst is uitgesproken door een computer. [S2] wauw, dat klinkt wel erg goed. [S1] ja, ik hoop dat je er wat aan hebt. | [clone](https://github.com/pevers/parkiet/raw/refs/heads/main/samples/voice_out.mp3)
 
 
@@ -27,11 +28,14 @@ Open-weights Dutch TTS based on the [Parakeet](https://jordandarefsky.com/blog/2
 * Use `[S1]`, `[S2]`, `[S3]`, `[S4]` to indicate the different speakers. Always start with `[S1]` and always alternate between [S1] and [S2] (i.e. [S1]... [S1]... is not good).
 * Prefer lower capital text prompts with punctuation. Write out digits as words. Even though the model should be able to handle some variety, it is better to stick close to the output of [WhisperD-NL](https://huggingface.co/pevers/whisperd-nl).
 * Slowing down can be encouraged by using `...` in the prompt.
-* Stuttering and disfluencies can be encouraged by using `eh`, `ehm`, `uh` or `uhm`.
+* Stuttering and disfluencies can be encouraged by using `uh`, `uhm`, `mmm`.
 * Laughter can be added with the `(laughs)` tag. However, use it sparingly because the model quickly derails for too many events.
+* Reduce hallucination by tuning the text prompts. The model can be brittle for unexpected events or tokens. Take a look at the example sentences and mimick the style.
 
 ## Quickstart
 
+The JAX model has the best performance in terms of quality, but requires a bit more setup, and is (for the moment) a little bit slower. The model is also ported back to PyTorch. However, I suspect that due to small differences in the attention kernel between PyTorch and JAX, the PyTorch model hallucinates more and generates strange artifacts more than the JAX model.
+
 ```bash
 uv sync # For CPU
 uv sync --extra tpu # For TPU
@@ -49,8 +53,46 @@ unzip weights/dia-nl-v1.zip -d weights
 uv run python src/parkiet/jax/inference.py
 ```
 
-Notes:
-- We use the JAX model by default. The model is also ported back to PyTorch. However, I suspect that due to small differences in the attention kernel between PyTorch and JAX, the PyTorch model hallucinates more and generates strange artifacts more than the JAX model. You can download the PyTorch model from [HuggingFace](https://huggingface.co/pevers/parkiet/blob/main/dia-nl-v1.pth) and use it in the [Dia](https://github.com/nari-labs/dia) pipeline.
+<details>
+
+<summary>PyTorch</summary>
+
+```bash
+uv sync # For CPU
+uv sync --extra cuda # For CUDA
+
+wget https://huggingface.co/pevers/parkiet/blob/main/dia-nl-v1.pth -O weights/dia-nl-v1.pth
+uv run python src/parkiet/dia/inference.py
+```
+
+</details>
+
+<details>
+
+<summary>Dia Plug-and-Play Transformers</summary>
+
+NOTE: Tune the `cfg_scale` option and temperature to reduce hallucinations.
+
+```python
+from transformers import AutoProcessor, DiaForConditionalGeneration
+
+torch_device = "cuda"
+model_checkpoint = "pevers/parkiet/v1/"
+
+text = [
+    "[S1] denk je dat je een open source model kan trainen met weinig geld en middelen? [S2] ja ik denk het wel. [S1] oh ja, hoe dan? [S2] nou kijk maar in de repo op Git Hub of Hugging Face."
+]
+processor = AutoProcessor.from_pretrained(model_checkpoint)
+inputs = processor(text=text, padding=True, return_tensors="pt").to(torch_device)
+
+model = DiaForConditionalGeneration.from_pretrained(model_checkpoint).to(torch_device)
+outputs = model.generate(**inputs, max_new_tokens=3072, guidance_scale=3.0, temperature=1.8, top_p=0.90, top_k=50)
+
+outputs = processor.batch_decode(outputs)
+processor.save_audio(outputs, "example.mp3")
+```
+
+</details>
 
 ## ⚠️ Disclaimer
 This project offers a high-fidelity speech generation model intended for research and educational use. The following uses are strictly forbidden:
@@ -62,7 +104,7 @@ By using this model, you agree to uphold relevant legal standards and ethical re
 
 ## Training
 
-For a full guide on data preparation, model conversion and the TPU setup to train this model for any language, see [TRAINING.md](https://github.com/pevers/parkiet/blob/main/TRAINING.md).
+For a full guide on data preparation, model conversion and the TPU setup to train this model for any language, see [TRAINING.md](TRAINING.md).
 
 ## Acknowledgements
 
@@ -73,4 +115,4 @@ For a full guide on data preparation, model conversion and the TPU setup to trai
 
 ## License
 
-Repository code is licensed under the [MIT License](LICENSE). The TTS model itself is licensed as [RAIL-M](MODEL_LICENSE).
+Repository code is licensed under the [MIT License](LICENSE). The TTS model itself is licensed as [RAIL-M](MODEL_LICENSE).