Release Jaiden Voice4 0.6B direct fine-tune

README.md

@@ -1,103 +1,107 @@
 ---
 license: apache-2.0
+base_model: Qwen/Qwen3-TTS-12Hz-0.6B-Base
 pipeline_tag: text-to-speech
 language:
-- zh
 - en
-- ja
-- ko
-- de
-- fr
-- ru
-- pt
-- es
-- it
 tags:
 - audio
 - tts
-- voice-clone
+- qwen3-tts
+- custom-voice
+- fine-tuned
 ---
 
-# Qwen3-TTS-12Hz-0.6B-Base
+# E3 Voice4 / Jaiden — Qwen3-TTS 0.6B direct fine-tune
 
-[**Qwen3-TTS Technical Report**](https://huggingface.co/papers/2601.15621) | [**GitHub Repository**](https://github.com/QwenLM/Qwen3-TTS) | [**Hugging Face Demo**](https://huggingface.co/spaces/Qwen/Qwen3-TTS)
+This is the ear-selected **epoch-0 full fine-tune** of
+[`Qwen/Qwen3-TTS-12Hz-0.6B-Base`](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base)
+for the E3 Voice4 single-speaker voice. The embedded speaker is exposed as
+`speaker_test`; inference does not require reference audio.
 
-Qwen3-TTS is a family of advanced multilingual, controllable, robust, and streaming text-to-speech models. Trained on over 5 million hours of speech data spanning 10 languages, Qwen3-TTS supports state-of-the-art 3-second voice cloning and description-based control.
-
-This specific checkpoint is the **0.6B Base model**, which is capable of rapid voice cloning from a user-provided audio input.
+The checkpoint was selected from five fixed-seed draws. The preferred output
+was draw 1 (seed 43), chosen for its balance of speaker identity and expressive
+range. This replaces the older Vivian/Fish-distilled weights previously on the
+repository's `main` branch. Historical models remain available through the
+repository's `v1`–`v5` tags and commit history.
 
 ## Quickstart
 
-### Installation
-
 ```bash
-pip install -U qwen-tts
-# Optional: for optimized performance
-pip install -U flash-attn --no-build-isolation
+pip install -U qwen-tts soundfile
 ```
 
-### Sample Usage (Voice Clone)
-
-To clone a voice and synthesize new content using the Base model, you can use the following code snippet:
-
 ```python
-import torch
 import soundfile as sf
+import torch
 from qwen_tts import Qwen3TTSModel
 
-# Load the model
 model = Qwen3TTSModel.from_pretrained(
-    "Qwen/Qwen3-TTS-12Hz-0.6B-Base",
+    "e3group-research/enderv1.0",
     device_map="cuda:0",
     dtype=torch.bfloat16,
-    attn_implementation="flash_attention_2",
 )
 
-# Reference audio for cloning
-ref_audio = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/clone.wav"
-ref_text  = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
-
-# Generate speech
-wavs, sr = model.generate_voice_clone(
-    text="I am solving the equation: x = [-b ± √(b²-4ac)] / 2a? Nobody can — it's a disaster (◍•͈⌔•͈◍), very sad!",
-    language="English",
-    ref_audio=ref_audio,
-    ref_text=ref_text,
+torch.manual_seed(43)
+wavs, sample_rate = model.generate_custom_voice(
+    text="This is a test of the E3 Voice4 checkpoint.",
+    speaker="speaker_test",
+    language="english",
 )
-
-# Save the resulting audio
-sf.write("output_voice_clone.wav", wavs[0], sr)
+sf.write("voice4.wav", wavs[0], sample_rate)
 ```
 
-## Overview
-### Introduction
+The only configured speaker key is `speaker_test` (speaker slot 3000).
+
+## Training recipe
+
+- Base model: Qwen3-TTS 12 Hz 0.6B Base
+- Method: single-stage full fine-tune, not LoRA or teacher distillation
+- Data: 130 single-speaker speech clips, 16.56 minutes total
+- Retained checkpoint: epoch 0 (65 optimizer steps)
+- Learning rate: `2e-5`, with 50 warmup steps
+- Objective: codec-0 cross entropy + `0.3 ×` sub-talker loss
+- Embedded speaker: slot 3000, populated from the training reference voice
+
+`history.json` records the complete eight-epoch training run. Only epoch 0 was
+retained for this release; later epochs were rejected because they overfit and
+could produce severe duration runaway.
+
+## Selection measurements
+
+For the selected seed-43 evaluation draw:
+
+| Measurement | Value |
+|---|---:|
+| Speaker similarity vs. internal reference | 0.9261 |
+| F0 p10–p90 range | 12.45 semitones |
+| F0 median | 246.83 Hz |
+| Duration | 22.78 seconds |
+| Speaking rate | 187 WPM |
+
+These are measurements for the selected generation, not aggregate benchmark
+results. Generation is stochastic and results vary with text and seed.
 
-<p align="center">
-    <img src="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/qwen3_tts_introduction.png" width="90%"/>
-<p>
+## Integrity
 
-Qwen3-TTS covers 10 major languages (Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, and Italian) as well as multiple dialectal voice profiles to meet global application needs. Key features:
+- `model.safetensors` SHA-256: `54f399102bdfc0b55e5c2ffa7fd1d3eb3d68cae4a516e151cbe6df8d01e96b7a`
+- `speech_tokenizer/model.safetensors` SHA-256: `836b7b357f5ea43e889936a3709af68dfe3751881acefe4ecf0dbd30ba571258`
 
-* **Powerful Speech Representation**: Powered by the self-developed Qwen3-TTS-Tokenizer-12Hz, it achieves efficient acoustic compression and high-dimensional semantic modeling.
-* **Universal End-to-End Architecture**: Utilizing a discrete multi-codebook LM architecture, it realizes full-information end-to-end speech modeling.
-* **Extreme Low-Latency Streaming Generation**: End-to-end synthesis latency as low as 97ms, meeting the rigorous demands of real-time interactive scenarios.
-* **Intelligent Text Understanding and Voice Control**: Supports speech generation driven by natural language instructions, allowing for flexible control over multi-dimensional acoustic attributes.
+## Limitations and responsible use
 
-### Model Architecture
+The checkpoint was selected primarily on English narration and has not been
+validated equally across all languages or production workloads. It may inherit
+pronunciation, hallucination, pacing, and robustness limitations from the base
+model.
 
-<p align="center">
-    <img src="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-TTS-Repo/overview.png" width="80%"/>
-<p>
+This is a research voice checkpoint. It is not affiliated with or endorsed by
+the source speaker. Do not use it for deceptive impersonation, fraud,
+harassment, or to imply endorsement. Users are responsible for obtaining any
+required consent and complying with applicable privacy, publicity, copyright,
+and disclosure requirements.
 
-## Citation
+## Base model
 
-If you find this work useful, please consider citing the technical report:
+- [Qwen3-TTS technical report](https://huggingface.co/papers/2601.15621)
+- [Qwen3-TTS source repository](https://github.com/QwenLM/Qwen3-TTS)
 
-```BibTeX
-@article{Qwen3-TTS,
-  title={Qwen3-TTS Technical Report},
-  author={Hangrui Hu and Xinfa Zhu and Ting He and Dake Guo and Bin Zhang and Xiong Wang and Zhifang Guo and Ziyue Jiang and Hongkun Hao and Zishan Guo and Xinyu Zhang and Pei Zhang and Baosong Yang and Jin Xu and Jingren Zhou and Junyang Lin},
-  journal={arXiv preprint arXiv:2601.15621},
-  year={2026}
-}
-```

config.json

@@ -166,12 +166,16 @@
     "vocab_size": 3072
   },
   "transformers_version": "4.57.3",
-  "two_stage_metadata": {
-    "stage1_epochs": 15,
-    "stage1_lr": 1e-05,
-    "stage2_epochs": 20,
-    "stage2_lr": 0.0005,
-    "eos_weight": 200.0,
-    "train_samples": 250
+  "direct_ft_metadata": {
+    "script": "sft_direct.py",
+    "epoch": 0,
+    "lr": 2e-05,
+    "warmup_steps": 50,
+    "alpha_sub_talker": 0.3,
+    "epochs_total_planned": 8,
+    "train_samples": 130,
+    "loss_avg": 4.452753011996929,
+    "ce_avg": 1.1196514423076922,
+    "sub_avg": 11.110338123028095
   }
 }

history.json

@@ -0,0 +1,74 @@
+[
+  {
+    "epoch": 0,
+    "loss": 4.452753011996929,
+    "ce": 1.1196514423076922,
+    "sub": 11.110338123028095,
+    "lr": 6.9320000000000016e-06,
+    "steps": 65,
+    "eval": null
+  },
+  {
+    "epoch": 1,
+    "loss": 3.636187718464778,
+    "ce": 1.010096153846154,
+    "sub": 8.753638230837309,
+    "lr": 1.3664000000000005e-05,
+    "steps": 65,
+    "eval": null
+  },
+  {
+    "epoch": 2,
+    "loss": 3.1426233181586634,
+    "ce": 0.8228064903846154,
+    "sub": 7.7327224364647495,
+    "lr": 2e-05,
+    "steps": 65,
+    "eval": null
+  },
+  {
+    "epoch": 3,
+    "loss": 2.8571965621067927,
+    "ce": 0.5620342548076923,
+    "sub": 7.6505407039935776,
+    "lr": 2e-05,
+    "steps": 65,
+    "eval": null
+  },
+  {
+    "epoch": 4,
+    "loss": 2.6110953367673435,
+    "ce": 0.3226111778846154,
+    "sub": 7.62828022149893,
+    "lr": 2e-05,
+    "steps": 65,
+    "eval": null
+  },
+  {
+    "epoch": 5,
+    "loss": 2.4492226417248064,
+    "ce": 0.16533203125,
+    "sub": 7.6129683494567875,
+    "lr": 2e-05,
+    "steps": 65,
+    "eval": null
+  },
+  {
+    "epoch": 6,
+    "loss": 2.3791526060837964,
+    "ce": 0.09971829927884615,
+    "sub": 7.598114013671875,
+    "lr": 2e-05,
+    "steps": 65,
+    "eval": null
+  },
+  {
+    "epoch": 7,
+    "loss": 2.329445670201228,
+    "ce": 0.05500300480769231,
+    "sub": 7.581475243201623,
+    "lr": 2e-05,
+    "steps": 65,
+    "eval": null
+  }
+]

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0962d2b722b44d67fcb4595609f704514200dd3f6afa377c0a3b83a8495f4dfd
+oid sha256:54f399102bdfc0b55e5c2ffa7fd1d3eb3d68cae4a516e151cbe6df8d01e96b7a
 size 1811626544

v6_seg5.tar.gz

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:9902b4411f10a705550fd085ddbf1625d16bf61eb068e87ab8dc870957c520b7
-size 2105439959

v6_transfer.tar.gz

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:169d526ba3ed8bf309d4fc91ee8543f431e2c52967f7c84c8a9561178168af8b
-size 8503623358

v_old_transfer.tar.gz

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:134f4c09c671aaae266e154dd611a99bf9e9c322fc2defa0c2a340ad94dba2be
-size 8472423614