Chatterbox fine-tuned model + logs

training.log

@@ -1,7 +1,7 @@
 
 /usr/local/lib/python3.13/site-packages/perth/perth_net/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
   from pkg_resources import resource_filename
-02/06/2026 05:10:16 - INFO - __main__ - Training/evaluation parameters CustomTrainingArguments(
+02/06/2026 05:27:31 - INFO - __main__ - Training/evaluation parameters CustomTrainingArguments(
 accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None, 'use_configured_state': False},
 adam_beta1=0.9,
 adam_beta2=0.999,
@@ -113,58 +113,110 @@ warmup_ratio=None,
 warmup_steps=1.0,
 weight_decay=0.0,
 )
-02/06/2026 05:10:16 - INFO - __main__ - Model parameters ModelArguments(model_name_or_path='ResembleAI/chatterbox', local_model_dir=None, cache_dir=None, freeze_voice_encoder=True, freeze_s3gen=True)
-02/06/2026 05:10:16 - INFO - __main__ - Data parameters DataArguments(language='hi', dataset_dir=None, metadata_file=None, dataset_name='maddi99/bengali-banspeech', dataset_config_name=None, train_split_name='train', eval_split_name='validation', text_column_name='text_scribe', audio_column_name='audio', max_text_len=256, max_speech_len=800, audio_prompt_duration_s=3.0, eval_split_size=0.0002, preprocessing_num_workers=None, ignore_verifications=False)
-02/06/2026 05:10:16 - INFO - __main__ - Loading ChatterboxTTS model...
-02/06/2026 05:10:16 - INFO - __main__ - Loading model from Hugging Face Hub: ResembleAI/chatterbox
+02/06/2026 05:27:31 - INFO - __main__ - Model parameters ModelArguments(model_name_or_path='ResembleAI/chatterbox', local_model_dir=None, cache_dir=None, freeze_voice_encoder=True, freeze_s3gen=True)
+02/06/2026 05:27:31 - INFO - __main__ - Data parameters DataArguments(language='bn', dataset_dir=None, metadata_file=None, dataset_name=' maddi99/bengali-banspeech', dataset_config_name=None, train_split_name='train', eval_split_name='validation', text_column_name='text_scribe', audio_column_name='audio', max_text_len=256, max_speech_len=800, audio_prompt_duration_s=3.0, eval_split_size=0.0002, preprocessing_num_workers=None, ignore_verifications=False)
+02/06/2026 05:27:31 - INFO - __main__ - Loading ChatterboxTTS model...
+02/06/2026 05:27:31 - INFO - __main__ - Loading model from Hugging Face Hub: ResembleAI/chatterbox
 /usr/local/lib/python3.13/site-packages/huggingface_hub/utils/_validators.py:202: UserWarning: The `local_dir_use_symlinks` argument is deprecated and ignored in `hf_hub_download`. Downloading to a local directory does not use symlinks anymore.
   warnings.warn(
-02/06/2026 05:10:16 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/main/ve.safetensors "HTTP/1.1 302 Found"
-02/06/2026 05:10:16 - INFO - httpx - HTTP Request: GET https://huggingface.co/api/models/ResembleAI/chatterbox/xet-read-token/05e904af2b5c7f8e482687a9d7336c5c824467d9 "HTTP/1.1 200 OK"
+02/06/2026 05:27:31 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/main/ve.safetensors "HTTP/1.1 302 Found"
+02/06/2026 05:27:31 - INFO - httpx - HTTP Request: GET https://huggingface.co/api/models/ResembleAI/chatterbox/xet-read-token/05e904af2b5c7f8e482687a9d7336c5c824467d9 "HTTP/1.1 200 OK"
 
 
 ve.safetensors:   0%|          | 0.00/5.70M [00:00<?, ?B/s]
-ve.safetensors: 100%|██████████| 5.70M/5.70M [00:00<00:00, 38.2MB/s]
-02/06/2026 05:10:16 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/main/t3_mtl23ls_v2.safetensors "HTTP/1.1 302 Found"
+ve.safetensors: 100%|██████████| 5.70M/5.70M [00:00<00:00, 21.1MB/s]
+02/06/2026 05:27:32 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/main/t3_mtl23ls_v2.safetensors "HTTP/1.1 302 Found"
 
 
 t3_mtl23ls_v2.safetensors:   0%|          | 0.00/2.14G [00:00<?, ?B/s]
 
-t3_mtl23ls_v2.safetensors:   0%|          | 7.60M/2.14G [00:01<07:58, 4.47MB/s]
+t3_mtl23ls_v2.safetensors:   0%|          | 7.60M/2.14G [00:01<07:04, 5.04MB/s]
 
-t3_mtl23ls_v2.safetensors:   4%|▎         | 78.7M/2.14G [00:02<00:59, 34.6MB/s]
+t3_mtl23ls_v2.safetensors:   4%|▍         | 80.9M/2.14G [00:05<02:07, 16.2MB/s]
 
-t3_mtl23ls_v2.safetensors:  10%|█         | 221M/2.14G [00:03<00:27, 70.5MB/s] 
+t3_mtl23ls_v2.safetensors:  27%|██▋       | 579M/2.14G [00:06<00:13, 118MB/s]  
 
-t3_mtl23ls_v2.safetensors:  24%|██▎       | 505M/2.14G [00:06<00:16, 101MB/s] 
+t3_mtl23ls_v2.safetensors:  37%|███▋      | 792M/2.14G [00:07<00:10, 128MB/s]
 
-t3_mtl23ls_v2.safetensors:  30%|███       | 647M/2.14G [00:07<00:15, 97.3MB/s]
-t3_mtl23ls_v2.safetensors: 100%|██████████| 2.14G/2.14G [00:08<00:00, 246MB/s]
-02/06/2026 05:10:25 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/main/s3gen.safetensors "HTTP/1.1 302 Found"
+t3_mtl23ls_v2.safetensors:  60%|██████    | 1.29G/2.14G [00:09<00:04, 211MB/s]
+t3_mtl23ls_v2.safetensors: 100%|██████████| 2.14G/2.14G [00:09<00:00, 221MB/s]
+02/06/2026 05:27:41 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/main/s3gen.safetensors "HTTP/1.1 302 Found"
 
 
 s3gen.safetensors:   0%|          | 0.00/1.06G [00:00<?, ?B/s]
 
-s3gen.safetensors:   5%|▍         | 50.7M/1.06G [00:01<00:27, 36.9MB/s]
+s3gen.safetensors:   6%|▋         | 67.0M/1.06G [00:01<00:29, 33.9MB/s]
 
-s3gen.safetensors:  62%|██████▏   | 654M/1.06G [00:02<00:01, 311MB/s]  
-s3gen.safetensors: 100%|██████████| 1.06G/1.06G [00:02<00:00, 391MB/s]
-02/06/2026 05:10:28 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/main/mtl_tokenizer.json "HTTP/1.1 307 Temporary Redirect"
-02/06/2026 05:10:28 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/ResembleAI/chatterbox/05e904af2b5c7f8e482687a9d7336c5c824467d9/mtl_tokenizer.json "HTTP/1.1 200 OK"
-02/06/2026 05:10:28 - INFO - httpx - HTTP Request: GET https://huggingface.co/api/resolve-cache/models/ResembleAI/chatterbox/05e904af2b5c7f8e482687a9d7336c5c824467d9/mtl_tokenizer.json "HTTP/1.1 200 OK"
+s3gen.safetensors:  56%|█████▌    | 587M/1.06G [00:03<00:02, 229MB/s]  
+s3gen.safetensors: 100%|██████████| 1.06G/1.06G [00:03<00:00, 322MB/s]
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/main/mtl_tokenizer.json "HTTP/1.1 307 Temporary Redirect"
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/ResembleAI/chatterbox/05e904af2b5c7f8e482687a9d7336c5c824467d9/mtl_tokenizer.json "HTTP/1.1 200 OK"
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: GET https://huggingface.co/api/resolve-cache/models/ResembleAI/chatterbox/05e904af2b5c7f8e482687a9d7336c5c824467d9/mtl_tokenizer.json "HTTP/1.1 200 OK"
 
 
 mtl_tokenizer.json:   0%|          | 0.00/68.1k [00:00<?, ?B/s]
-mtl_tokenizer.json: 100%|██████████| 68.1k/68.1k [00:00<00:00, 145MB/s]
-02/06/2026 05:10:28 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/main/conds.pt "HTTP/1.1 302 Found"
+mtl_tokenizer.json: 100%|██████████| 68.1k/68.1k [00:00<00:00, 134MB/s]
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/main/conds.pt "HTTP/1.1 302 Found"
 
 
 conds.pt:   0%|          | 0.00/107k [00:00<?, ?B/s]
-conds.pt: 100%|██████████| 107k/107k [00:00<00:00, 1.30MB/s]
-Traceback (most recent call last):
+conds.pt: 100%|██████████| 107k/107k [00:00<00:00, 1.31MB/s]
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: GET https://huggingface.co/api/models/ResembleAI/chatterbox/revision/main "HTTP/1.1 200 OK"
+
+
+Downloading (incomplete total...): 0.00B [00:00, ?B/s]
+
+Fetching 6 files:   0%|          | 0/6 [00:00<?, ?it/s]02/06/2026 05:27:45 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/05e904af2b5c7f8e482687a9d7336c5c824467d9/Cangjie5_TC.json "HTTP/1.1 307 Temporary Redirect"
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/05e904af2b5c7f8e482687a9d7336c5c824467d9/conds.pt "HTTP/1.1 302 Found"
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/05e904af2b5c7f8e482687a9d7336c5c824467d9/s3gen.pt "HTTP/1.1 302 Found"
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/05e904af2b5c7f8e482687a9d7336c5c824467d9/ve.pt "HTTP/1.1 302 Found"
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/05e904af2b5c7f8e482687a9d7336c5c824467d9/t3_mtl23ls_v2.safetensors "HTTP/1.1 302 Found"
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/ResembleAI/chatterbox/resolve/05e904af2b5c7f8e482687a9d7336c5c824467d9/grapheme_mtl_merged_expanded_v1.json "HTTP/1.1 307 Temporary Redirect"
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/ResembleAI/chatterbox/05e904af2b5c7f8e482687a9d7336c5c824467d9/Cangjie5_TC.json "HTTP/1.1 200 OK"
+
+
+Downloading (incomplete total...):   0%|          | 0.00/2.14G [00:00<?, ?B/s]
+
+Downloading (incomplete total...):   0%|          | 0.00/3.20G [00:00<?, ?B/s]
+
+Downloading (incomplete total...):   0%|          | 0.00/3.21G [00:00<?, ?B/s]
+
+Downloading (incomplete total...):   0%|          | 0.00/3.21G [00:00<?, ?B/s]02/06/2026 05:27:45 - INFO - httpx - HTTP Request: GET https://huggingface.co/api/resolve-cache/models/ResembleAI/chatterbox/05e904af2b5c7f8e482687a9d7336c5c824467d9/Cangjie5_TC.json "HTTP/1.1 200 OK"
+
+
+Downloading (incomplete total...):   0%|          | 0.00/3.21G [00:00<?, ?B/s]02/06/2026 05:27:45 - INFO - httpx - HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/ResembleAI/chatterbox/05e904af2b5c7f8e482687a9d7336c5c824467d9/grapheme_mtl_merged_expanded_v1.json "HTTP/1.1 200 OK"
+02/06/2026 05:27:45 - INFO - httpx - HTTP Request: GET https://huggingface.co/api/resolve-cache/models/ResembleAI/chatterbox/05e904af2b5c7f8e482687a9d7336c5c824467d9/grapheme_mtl_merged_expanded_v1.json "HTTP/1.1 200 OK"
+
+
+Downloading (incomplete total...):   0%|          | 1.92M/3.21G [00:00<02:36, 20.5MB/s]
+
+Downloading (incomplete total...):   0%|          | 15.4M/3.21G [00:01<05:28, 9.71MB/s]
+
+Downloading (incomplete total...):   3%|▎         | 86.5M/3.21G [00:06<03:42, 14.0MB/s]
+
+Downloading (incomplete total...):   5%|▍         | 158M/3.21G [00:07<02:13, 22.9MB/s] 
+
+Downloading (incomplete total...):   9%|▊         | 280M/3.21G [00:09<01:17, 37.6MB/s]
+
+Downloading (incomplete total...):  28%|██▊       | 908M/3.21G [00:10<00:14, 155MB/s] 
+
+Downloading (incomplete total...):  44%|████▎     | 1.40G/3.21G [00:11<00:08, 223MB/s]
+
+Fetching 6 files:  67%|██████▋   | 4/6 [00:12<00:06,  3.09s/it]
+
+Downloading (incomplete total...):  87%|████████▋ | 2.78G/3.21G [00:12<00:00, 493MB/s]
+Fetching 6 files: 100%|██████████| 6/6 [00:13<00:00,  2.20s/it]
+
+
+Download complete: 100%|██████████| 3.21G/3.21G [00:13<00:00, 493MB/s]                Traceback (most recent call last):
   File "/app/chatterbox-multilingual-finetuning/src/finetune_t3.py", line 848, in <module>
     main()
     ~~~~^^
   File "/app/chatterbox-multilingual-finetuning/src/finetune_t3.py", line 616, in main
     chatterbox_model = ChatterboxMultilingualTTS.from_pretrained(device="cpu")
-TypeError: ChatterboxMultilingualTTS.from_pretrained() got an unexpected keyword argument 'device'
+  File "/app/chatterbox-multilingual-finetuning/src/chatterbox/mtl_tts.py", line 188, in from_pretrained
+    return cls.from_local(ckpt_dir, device)
+           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
+TypeError: ChatterboxMultilingualTTS.from_local() takes 2 positional arguments but 3 were given
+
+Download complete: 100%|██████████| 3.21G/3.21G [00:13<00:00, 239MB/s]