QwenTTS-he-1.7B

A Hebrew LoRA adapter for Qwen3-TTS-12Hz-1.7B-Base, trained on ~131k Hebrew utterances with stressed-IPA text conditioning.

The adapter adds Hebrew to the base model without touching a single base weight. Load it and you get Hebrew; disable it and you get the original model, bit-for-bit, with all ten of its native languages intact.

Trained by Max Melichov.

What this is

Base model	Qwen/Qwen3-TTS-12Hz-1.7B-Base (voice-clone variant)
Adapter type	LoRA, r=32, α=64, dropout=0.05
Adapter size	246 MB (bf16)
Language added	Hebrew
Text input	Stressed IPA (not Hebrew orthography — see below)
Voice control	Reference-audio speaker cloning, same as the base model
Best eval loss	2.1339 @ step 6000

Trained parameters

LoRA is applied to q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj across both the talker backbone and the MTP code_predictor.

These modules are additionally trained in full and shipped inside the adapter (modules_to_save), because Hebrew phonotactics need the output distribution itself to move, not just the attention/MLP deltas:

• codec_head
• text_projection
• lm_head.0 … lm_head.14 (the 15 residual-codebook heads)

The text embedding table is not resized or retrained, so the tokenizer and vocabulary are unchanged from the base model.

Important: the text field takes IPA, not Hebrew script

This adapter was trained with stressed IPA in the text field. Hebrew orthography is famously under-specified for vowels, so feeding raw Hebrew script will not work well. Convert first with a Hebrew G2P (the training data used RenikudPlus for diacritization followed by IPA conversion).

Hebrew:  עוֹד רֶגַע אֲנִי אֶתְיַיחֵס לִיהוּדָה וְשׁוֹמְרוֹן.
IPA:     ʔˈod ʁˈeɡa ʔanˈi ʔetjaχˈes lihudˈa veʃomʁˈon.

The ˈ (U+02C8) primary-stress marker precedes the stressed vowel and matters — the model learned prosody from it.

Usage

Requires the Qwen3-TTS inference code plus peft. The adapter wraps the model's talker submodule, not the whole model.

import torch, soundfile as sf
from peft import PeftModel
from qwen_tts import Qwen3TTSModel

tts = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="sdpa",
)

# Attach the Hebrew adapter to the talker.
tts.model.talker = PeftModel.from_pretrained(
    tts.model.talker, "notmax123/QwenTTS-he-1.7B"
)
tts.model.eval()

wavs, sr = tts.generate_voice_clone(
    text="ʃalˈom, mˈa ʃlomχˈa hajˈom?",   # stressed IPA
    language="Auto",
    ref_audio="my_reference_voice.wav",
    x_vector_only_mode=True,
    non_streaming_mode=True,
    do_sample=False,
    subtalker_dosample=False,
    repetition_penalty=1.0,
    max_new_tokens=2048,
)
sf.write("out.wav", wavs[0], sr)

language="Auto" is what the model was trained and sampled with — Hebrew is not in the base model's language table, and the IPA text carries the phonetics.

Getting the base model's other languages back

Do not run Chinese / English / French / German / Italian / Japanese / Korean / Portuguese / Russian / Spanish with the adapter active — it shifts the output distribution toward Hebrew. Wrap those calls instead:

with tts.model.talker.disable_adapter():
    wavs, sr = tts.generate_voice_clone(
        text="Hello, how are you today?",
        language="English",
        ref_audio=ref,
        ref_text=ref_text,
    )

With the adapter disabled the forward pass is bit-identical to the unmodified base model (verified: all 404 base tensors unchanged, and generated audio matches the pre-adapter base at max|diff| = 0 for English and Japanese).

Do not merge

merge_and_unload() bakes the Hebrew deltas into the base weights permanently and destroys the guarantee above. Keep the adapter separate and toggle it.

Training

Train set	131,569 Hebrew utterances
Eval set	600 held-out utterances
Steps	6,000 (≈1.46 epochs)
Batch	1 × 32 gradient accumulation = 32 effective
LR	5e-5, cosine schedule, 3% warmup
Precision	bf16 mixed
Attention	sdpa
Seed	0
Hardware	single 32 GB GPU (~16 GB peak)

Eval loss decreased monotonically at every 500-step checkpoint, from 2.2999 (step 500) to 2.1339 (step 6000) — the run had not yet plateaued, so more steps would likely still help.

Eval loss by step

step	eval loss
500	2.2999
1000	2.2435
1500	2.2096
2000	2.1902
2500	2.1776
3000	2.1619
3500	2.1505
4000	2.1431
4500	2.1401
5000	2.1358
5500	2.1340
6000	2.1339

Files

File	What it is
adapter_model.safetensors	The LoRA weights + saved output heads
adapter_config.json	PEFT config
training_state.json	Full step/loss history from the run
samples/	Generated audio at step 6000 (2 Hebrew, 2 English reference voices)

Limitations

• IPA input required. Raw Hebrew text needs a G2P pass first.
• Trained on read/narrated speech; expressive or conversational Hebrew is out of distribution.
• Speaker coverage comes from the training corpus's voices; cloning to a very different voice may carry over training-speaker prosody.
• Hebrew only. Yiddish was deliberately excluded from this run.

License

Apache-2.0, matching the base model. See the Qwen3-TTS model card for base-model terms.

Citation

@misc{melichov2026qwentts-he,
  author = {Max Melichov},
  title  = {QwenTTS-he-1.7B: A Hebrew LoRA adapter for Qwen3-TTS-12Hz-1.7B-Base},
  year   = {2026},
  url    = {https://huggingface.co/notmax123/QwenTTS-he-1.7B}
}