Add detailed results and logs.

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/lid_inference_test.log

@@ -0,0 +1,300 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Mon Jun  2 02:37:15 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue04] 2025-06-02 02:37:35,038 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue04] 2025-06-02 02:37:46,607 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamConditionModel(
+  (frontend): S3prlFrontendCondition(
+    (upstream): S3PRLUpstreamCondition(
+      (upstream): UpstreamExpertCondition(
+        (model): Wav2Vec2ModelCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+            (aamsoftmax_weight): ParameterDict()
+            (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True)
+            (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec(
+              (ce): CrossEntropyLoss()
+              (lang2vec_head): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (lang2vec_loss): MSELoss()
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamConditionModel
+    Total Number of model parameters: 977.14 M
+    Number of trainable parameters: 977.14 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
+  warnings.warn(_create_warning_msg(
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue04] 2025-06-02 02:37:47,156 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/lids0
+[gpue04] 2025-06-02 02:38:41,828 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue04] 2025-06-02 02:39:27,483 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1
+[gpue04] 2025-06-02 02:40:15,909 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2
+[gpue04] 2025-06-02 02:41:08,571 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3
+[gpue04] 2025-06-02 02:41:56,182 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4
+[gpue04] 2025-06-02 02:42:40,736 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5
+[gpue04] 2025-06-02 02:43:27,814 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6
+[gpue04] 2025-06-02 02:44:10,740 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7
+[gpue04] 2025-06-02 02:44:52,065 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8
+[gpue04] 2025-06-02 02:45:40,635 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9
+[gpue04] 2025-06-02 02:46:28,394 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10
+[gpue04] 2025-06-02 02:47:09,502 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11
+[gpue04] 2025-06-02 02:47:59,978 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12
+[gpue04] 2025-06-02 02:48:52,866 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13
+[gpue04] 2025-06-02 02:49:41,279 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14
+[gpue04] 2025-06-02 02:50:32,817 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15
+[gpue04] 2025-06-02 02:51:20,444 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 16
+[gpue04] 2025-06-02 02:52:09,714 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 17
+[gpue04] 2025-06-02 02:52:55,108 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 18
+[gpue04] 2025-06-02 02:53:50,212 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 19
+[gpue04] 2025-06-02 02:54:31,533 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 20
+[gpue04] 2025-06-02 02:55:19,223 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True
+[gpue04] 2025-06-02 02:55:19,224 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=1085 threads=1
+# Ended (code 0) at Mon Jun  2 02:55:20 CDT 2025, elapsed time 1085 seconds

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/results

@@ -0,0 +1,1039 @@
+Accuracy: 95.38%
+Macro Accuracy: 95.44%
+Accuracy per Language:
+amh: 95.81%
+tam: 96.86%
+luo: 95.22%
+tpi: 88.69%
+ibo: 96.11%
+ben: 92.18%
+lao: 97.22%
+swa: 90.31%
+asm: 97.21%
+gug: 95.29%
+kaz: 94.59%
+pus: 98.80%
+tgl: 95.10%
+hat: 97.81%
+jav: 91.76%
+zul: 97.44%
+vie: 96.39%
+kmr: 94.60%
+kat: 96.10%
+tur: 98.39%
+ceb: 91.89%
+yue: 100.00%
+khk: 94.71%
+lit: 98.77%
+tel: 94.88%
+Key: amh_18766_A_20140725_193025_031291, Target: amh, Predicted: tpi
+Key: amh_16601_A_20140616_191918_057010, Target: amh, Predicted: tel
+Key: amh_19621_B_20140517_232031_046443, Target: amh, Predicted: pus
+Key: amh_19782_A_20140702_230513_056385, Target: amh, Predicted: tel
+Key: amh_41741_A_20140422_000845_000000, Target: amh, Predicted: kat
+Key: amh_46625_B_20140414_224528_000000, Target: amh, Predicted: tgl
+Key: amh_46625_B_20140414_224528_011634, Target: amh, Predicted: tel
+Key: amh_47799_A_20140902_200301_004751, Target: amh, Predicted: khk
+Key: amh_41741_A_20140422_000845_021144, Target: amh, Predicted: ben
+Key: amh_44961_A_20140421_215913_034469, Target: amh, Predicted: tel
+Key: amh_42883_A_20140823_230118_001930, Target: amh, Predicted: asm
+Key: amh_44961_A_20140421_215913_040626, Target: amh, Predicted: tur
+Key: amh_44961_A_20140421_215913_048405, Target: amh, Predicted: tel
+Key: amh_47799_A_20140902_200301_028605, Target: amh, Predicted: ibo
+Key: amh_61011_B_20140415_180846_022820, Target: amh, Predicted: tpi
+Key: amh_60498_A_20140823_192847_039762, Target: amh, Predicted: kaz
+Key: amh_69633_A_20140607_233440_058823, Target: amh, Predicted: tgl
+Key: amh_69633_A_20140607_233440_001199, Target: amh, Predicted: tgl
+Key: amh_64870_A_20140518_011602_000000, Target: amh, Predicted: pus
+Key: amh_69633_A_20140607_233440_027832, Target: amh, Predicted: kaz
+Key: amh_69633_A_20140607_233440_052352, Target: amh, Predicted: ceb
+Key: amh_69633_A_20140607_233440_054267, Target: amh, Predicted: tel
+Key: amh_85439_A_20140814_215435_004561, Target: amh, Predicted: kmr
+Key: amh_81553_A_20140707_003952_001198, Target: amh, Predicted: kmr
+Key: amh_73757_A_20140512_231155_058241, Target: amh, Predicted: ibo
+Key: amh_89888_B_20140520_191659_037281, Target: amh, Predicted: ben
+Key: amh_93320_A_20140823_214255_040946, Target: amh, Predicted: swa
+Key: amh_93320_A_20140823_214255_050913, Target: amh, Predicted: tel
+Key: amh_89888_B_20140520_191659_020529, Target: amh, Predicted: kaz
+Key: amh_89888_B_20140520_191659_058819, Target: amh, Predicted: zul
+Key: amh_95124_A_20140828_224047_058345, Target: amh, Predicted: khk
+Key: amh_95124_A_20140828_224047_059534, Target: amh, Predicted: gug
+Key: amh_95124_A_20140828_224047_022900, Target: amh, Predicted: kat
+Key: amh_95124_A_20140828_224047_034153, Target: amh, Predicted: kat
+Key: amh_94002_A_20140511_172143_000793, Target: amh, Predicted: tgl
+Key: amh_95124_A_20140828_224047_038927, Target: amh, Predicted: lit
+Key: amh_96940_B_20140901_181148_007703, Target: amh, Predicted: kaz
+Key: amh_94237_A_20140814_181922_050462, Target: amh, Predicted: hat
+Key: amh_94237_A_20140814_181922_051539, Target: amh, Predicted: tam
+Key: amh_94237_A_20140814_181922_058366, Target: amh, Predicted: asm
+Key: amh_94002_A_20140511_172143_023305, Target: amh, Predicted: tel
+Key: amh_95124_A_20140828_224047_004869, Target: amh, Predicted: kat
+Key: amh_96940_B_20140901_181148_039855, Target: amh, Predicted: kaz
+Key: amh_98506_A_20140807_170934_060854, Target: amh, Predicted: kaz
+Key: asm_34446_B_20120426_195519_020649, Target: asm, Predicted: ben
+Key: asm_33969_B_20130123_165132_045069, Target: asm, Predicted: tam
+Key: asm_33704_A_20130204_172729_034778, Target: asm, Predicted: tam
+Key: asm_33704_A_20130204_172729_049460, Target: asm, Predicted: ben
+Key: asm_43587_A_20120607_204145_034715, Target: asm, Predicted: tel
+Key: asm_40385_A_20121224_164959_041220, Target: asm, Predicted: tel
+Key: asm_40385_B_20121224_164959_020689, Target: asm, Predicted: tgl
+Key: asm_46593_B_20121010_023019_043252, Target: asm, Predicted: ben
+Key: asm_46593_B_20121010_023019_046408, Target: asm, Predicted: ben
+Key: asm_47429_A_20130121_172000_012339, Target: asm, Predicted: vie
+Key: asm_59544_B_20120401_222134_013082, Target: asm, Predicted: ben
+Key: asm_80856_A_20120423_184225_031581, Target: asm, Predicted: ben
+Key: asm_79519_B_20121008_214502_049044, Target: asm, Predicted: tgl
+Key: asm_66668_B_20120409_185702_056020, Target: asm, Predicted: tel
+Key: asm_80856_A_20120423_184225_058863, Target: asm, Predicted: tgl
+Key: asm_87885_A_20121113_193407_023378, Target: asm, Predicted: ben
+Key: asm_87885_A_20121113_193407_024567, Target: asm, Predicted: tel
+Key: asm_87671_B_20120401_172420_054685, Target: asm, Predicted: ben
+Key: asm_87885_A_20121113_193407_044881, Target: asm, Predicted: ben
+Key: asm_87885_A_20121113_193407_007808, Target: asm, Predicted: kmr
+Key: asm_87885_A_20121113_193407_014210, Target: asm, Predicted: kmr
+Key: ben_10576_A_20111221_214850_004672, Target: ben, Predicted: asm
+Key: ben_10576_A_20111221_214850_016340, Target: ben, Predicted: asm
+Key: ben_10576_A_20111221_214850_030232, Target: ben, Predicted: tel
+Key: ben_10576_A_20111221_214850_036179, Target: ben, Predicted: asm
+Key: ben_10569_B_20111221_201913_002481, Target: ben, Predicted: asm
+Key: ben_10576_A_20111221_214850_050139, Target: ben, Predicted: asm
+Key: ben_24810_B_20120114_225518_016801, Target: ben, Predicted: yue
+Key: ben_21203_A_20120523_225358_000338, Target: ben, Predicted: asm
+Key: ben_21203_A_20120523_225358_012402, Target: ben, Predicted: asm
+Key: ben_27912_B_20120123_185402_005366, Target: ben, Predicted: asm
+Key: ben_27912_B_20120123_185402_013907, Target: ben, Predicted: lao
+Key: ben_27912_B_20120123_185402_040188, Target: ben, Predicted: asm
+Key: ben_38382_B_20120110_013824_008463, Target: ben, Predicted: tel
+Key: ben_38382_B_20120110_013824_009617, Target: ben, Predicted: tel
+Key: ben_38382_B_20120110_013824_015051, Target: ben, Predicted: asm
+Key: ben_40114_A_20120122_183602_035788, Target: ben, Predicted: asm
+Key: ben_40114_A_20120122_183602_049079, Target: ben, Predicted: asm
+Key: ben_44799_B_20120131_222925_044707, Target: ben, Predicted: asm
+Key: ben_40114_B_20120122_183602_002987, Target: ben, Predicted: tel
+Key: ben_40114_B_20120122_183602_025113, Target: ben, Predicted: asm
+Key: ben_50583_B_20120114_233345_025147, Target: ben, Predicted: tel
+Key: ben_50910_B_20120122_001708_020387, Target: ben, Predicted: asm
+Key: ben_50910_B_20120122_001708_050086, Target: ben, Predicted: gug
+Key: ben_44799_A_20120131_222925_022756, Target: ben, Predicted: asm
+Key: ben_44799_A_20120131_222925_025503, Target: ben, Predicted: asm
+Key: ben_44799_A_20120131_222925_031017, Target: ben, Predicted: asm
+Key: ben_53805_B_20120126_211949_044950, Target: ben, Predicted: tel
+Key: ben_62169_A_20120304_153842_051418, Target: ben, Predicted: asm
+Key: ben_53805_B_20120126_211949_048578, Target: ben, Predicted: asm
+Key: ben_53805_B_20120126_211949_054532, Target: ben, Predicted: tel
+Key: ben_57721_A_20120531_194610_023753, Target: ben, Predicted: asm
+Key: ben_52845_B_20120126_200807_030406, Target: ben, Predicted: asm
+Key: ben_52845_B_20120126_200807_034210, Target: ben, Predicted: asm
+Key: ben_62038_B_20111230_004215_016225, Target: ben, Predicted: yue
+Key: ben_53805_A_20120126_211949_037154, Target: ben, Predicted: yue
+Key: ben_62169_A_20120304_153842_019993, Target: ben, Predicted: asm
+Key: ben_63220_A_20120514_232049_025353, Target: ben, Predicted: asm
+Key: ben_62169_A_20120304_153842_024495, Target: ben, Predicted: asm
+Key: ben_62169_A_20120304_153842_027469, Target: ben, Predicted: asm
+Key: ben_62169_A_20120304_153842_039193, Target: ben, Predicted: asm
+Key: ben_62169_A_20120304_153842_041729, Target: ben, Predicted: asm
+Key: ben_63220_B_20120514_232049_020791, Target: ben, Predicted: asm
+Key: ben_63220_B_20120514_232049_021921, Target: ben, Predicted: asm
+Key: ben_65895_A_20120229_202918_036080, Target: ben, Predicted: tel
+Key: ben_65895_A_20120229_202918_046912, Target: ben, Predicted: asm
+Key: ben_66313_B_20120229_230907_037485, Target: ben, Predicted: tel
+Key: ben_65895_A_20120229_202918_011382, Target: ben, Predicted: asm
+Key: ben_80875_A_20120522_224314_028055, Target: ben, Predicted: asm
+Key: ben_86207_B_20120127_145936_022109, Target: ben, Predicted: asm
+Key: ben_80875_A_20120522_224314_031448, Target: ben, Predicted: asm
+Key: ben_80875_A_20120522_224314_033805, Target: ben, Predicted: asm
+Key: ben_80875_A_20120522_224314_037820, Target: ben, Predicted: asm
+Key: ben_80875_A_20120522_224314_044450, Target: ben, Predicted: tel
+Key: ben_80875_A_20120522_224314_050018, Target: ben, Predicted: asm
+Key: ben_81773_B_20120101_024120_043949, Target: ben, Predicted: asm
+Key: ben_80875_A_20120522_224314_011161, Target: ben, Predicted: tel
+Key: ben_91275_A_20120529_195749_013758, Target: ben, Predicted: asm
+Key: ben_91275_A_20120529_195749_014937, Target: ben, Predicted: asm
+Key: ben_91275_A_20120529_195749_018687, Target: ben, Predicted: asm
+Key: ben_91275_A_20120529_195749_023140, Target: ben, Predicted: tel
+Key: ceb_15638_B_20131210_131327_018092, Target: ceb, Predicted: tgl
+Key: ben_91275_A_20120529_195749_025289, Target: ben, Predicted: asm
+Key: ben_93273_B_20120123_022109_041146, Target: ben, Predicted: tel
+Key: ben_91275_A_20120529_195749_043069, Target: ben, Predicted: asm
+Key: ben_95826_A_20120201_001701_020909, Target: ben, Predicted: yue
+Key: ben_95826_B_20120201_001701_006650, Target: ben, Predicted: asm
+Key: ceb_14141_B_20140118_202248_001941, Target: ceb, Predicted: lao
+Key: ceb_14141_B_20140118_202248_008492, Target: ceb, Predicted: jav
+Key: ceb_14141_B_20140118_202248_015284, Target: ceb, Predicted: amh
+Key: ceb_14141_B_20140118_202248_016622, Target: ceb, Predicted: tgl
+Key: ceb_15262_A_20131105_213812_038869, Target: ceb, Predicted: asm
+Key: ceb_21109_A_20140102_180619_050237, Target: ceb, Predicted: tgl
+Key: ceb_17881_B_20140122_201653_009579, Target: ceb, Predicted: tel
+Key: ceb_22466_A_20131015_174457_021603, Target: ceb, Predicted: tgl
+Key: ceb_17881_B_20140122_201653_034245, Target: ceb, Predicted: asm
+Key: ceb_21109_A_20140102_180619_017721, Target: ceb, Predicted: jav
+Key: ceb_22466_A_20131015_174457_022828, Target: ceb, Predicted: tgl
+Key: ceb_21109_A_20140102_180619_018853, Target: ceb, Predicted: jav
+Key: ceb_22466_A_20131015_174457_025528, Target: ceb, Predicted: tgl
+Key: ceb_21109_A_20140102_180619_019994, Target: ceb, Predicted: jav
+Key: ceb_22466_A_20131015_174457_031272, Target: ceb, Predicted: tgl
+Key: ceb_22466_A_20131015_174457_033555, Target: ceb, Predicted: tgl
+Key: ceb_21109_A_20140102_180619_024469, Target: ceb, Predicted: jav
+Key: ceb_22466_A_20131015_174457_052722, Target: ceb, Predicted: tgl
+Key: ceb_21109_A_20140102_180619_025655, Target: ceb, Predicted: jav
+Key: ceb_22466_B_20131015_174457_001633, Target: ceb, Predicted: kmr
+Key: ceb_21109_A_20140102_180619_031392, Target: ceb, Predicted: tgl
+Key: ceb_22466_B_20131015_174457_045431, Target: ceb, Predicted: tgl
+Key: ceb_22466_B_20131015_174457_051524, Target: ceb, Predicted: tgl
+Key: ceb_21109_A_20140102_180619_040169, Target: ceb, Predicted: jav
+Key: ceb_21109_A_20140102_180619_041294, Target: ceb, Predicted: jav
+Key: ceb_21109_A_20140102_180619_042402, Target: ceb, Predicted: jav
+Key: ceb_38340_B_20131128_145618_035396, Target: ceb, Predicted: asm
+Key: ceb_38340_B_20131128_145618_044704, Target: ceb, Predicted: tgl
+Key: ceb_36059_B_20140118_204512_003449, Target: ceb, Predicted: tgl
+Key: ceb_38340_B_20131128_145618_050471, Target: ceb, Predicted: tgl
+Key: ceb_38340_B_20131128_145618_001728, Target: ceb, Predicted: tgl
+Key: ceb_38340_B_20131128_145618_028374, Target: ceb, Predicted: tgl
+Key: ceb_43646_A_20131019_165638_004395, Target: ceb, Predicted: tgl
+Key: ceb_50565_B_20131025_202729_012748, Target: ceb, Predicted: asm
+Key: ceb_43646_A_20131019_165638_019162, Target: ceb, Predicted: tgl
+Key: ceb_43646_A_20131019_165638_027625, Target: ceb, Predicted: asm
+Key: ceb_51530_B_20140125_195307_042590, Target: ceb, Predicted: tgl
+Key: ceb_51530_B_20140125_195307_043726, Target: ceb, Predicted: tgl
+Key: ceb_51530_B_20140125_195307_055117, Target: ceb, Predicted: hat
+Key: ceb_56370_A_20131101_175739_018773, Target: ceb, Predicted: lao
+Key: ceb_56370_B_20131101_175739_043790, Target: ceb, Predicted: tgl
+Key: ceb_54744_B_20131202_184432_002469, Target: ceb, Predicted: asm
+Key: ceb_54744_B_20131202_184432_003641, Target: ceb, Predicted: asm
+Key: ceb_60299_A_20140202_130806_026932, Target: ceb, Predicted: lao
+Key: ceb_60299_A_20140202_130806_030919, Target: ceb, Predicted: tam
+Key: ceb_54744_B_20131202_184432_014262, Target: ceb, Predicted: tgl
+Key: ceb_60299_A_20140202_130806_047310, Target: ceb, Predicted: tgl
+Key: ceb_54744_B_20131202_184432_036018, Target: ceb, Predicted: asm
+Key: ceb_60299_A_20140202_130806_053018, Target: ceb, Predicted: asm
+Key: ceb_54744_B_20131202_184432_044887, Target: ceb, Predicted: asm
+Key: ceb_56370_A_20131101_175739_004673, Target: ceb, Predicted: tgl
+Key: ceb_81427_A_20131126_151401_058032, Target: ceb, Predicted: tel
+Key: ceb_84611_A_20131125_193454_001166, Target: ceb, Predicted: tgl
+Key: ceb_79660_A_20140201_160331_000129, Target: ceb, Predicted: tgl
+Key: ceb_74455_A_20140115_152935_051492, Target: ceb, Predicted: ben
+Key: ceb_74455_B_20140115_152935_007394, Target: ceb, Predicted: tgl
+Key: ceb_74455_B_20140115_152935_015341, Target: ceb, Predicted: tgl
+Key: ceb_79660_A_20140201_160331_046537, Target: ceb, Predicted: vie
+Key: ceb_86467_A_20131112_182159_030337, Target: ceb, Predicted: tgl
+Key: ceb_86467_B_20131112_193636_008827, Target: ceb, Predicted: tgl
+Key: ceb_86467_B_20131112_193636_017008, Target: ceb, Predicted: tgl
+Key: ceb_85179_A_20131227_172225_003961, Target: ceb, Predicted: tgl
+Key: ceb_96985_A_20131021_164130_003454, Target: ceb, Predicted: tgl
+Key: ceb_96985_A_20131021_164130_042953, Target: ceb, Predicted: tam
+Key: ceb_98489_A_20131123_233440_004829, Target: ceb, Predicted: tgl
+Key: ceb_85179_A_20131227_172225_021268, Target: ceb, Predicted: lao
+Key: gug_21624_A_20150222_054542_006999, Target: gug, Predicted: tel
+Key: gug_21624_A_20150222_054542_008195, Target: gug, Predicted: tel
+Key: gug_21624_A_20150222_054542_021054, Target: gug, Predicted: tel
+Key: gug_21624_A_20150222_054542_023373, Target: gug, Predicted: tel
+Key: gug_21004_B_20150217_083755_046019, Target: gug, Predicted: tpi
+Key: gug_21004_B_20150217_083755_048475, Target: gug, Predicted: ibo
+Key: gug_23006_A_20140807_062702_004252, Target: gug, Predicted: luo
+Key: gug_39555_A_20141023_010258_027629, Target: gug, Predicted: lao
+Key: gug_41685_A_20150320_083024_019491, Target: gug, Predicted: tel
+Key: gug_41685_A_20150320_083024_050188, Target: gug, Predicted: tel
+Key: gug_23006_B_20140807_062702_021561, Target: gug, Predicted: zul
+Key: gug_43395_B_20150303_092614_017102, Target: gug, Predicted: vie
+Key: gug_43395_B_20150303_092614_043917, Target: gug, Predicted: lao
+Key: gug_50810_B_20140619_063147_011354, Target: gug, Predicted: lao
+Key: gug_50810_B_20140619_063147_023949, Target: gug, Predicted: jav
+Key: gug_44619_B_20140621_050143_005200, Target: gug, Predicted: lao
+Key: gug_50810_B_20140619_063147_034848, Target: gug, Predicted: tur
+Key: gug_50090_A_20150206_002321_001260, Target: gug, Predicted: tam
+Key: gug_50090_B_20150206_002321_026694, Target: gug, Predicted: tur
+Key: gug_56019_A_20150221_084856_048093, Target: gug, Predicted: tel
+Key: gug_56019_A_20150221_084856_054819, Target: gug, Predicted: tel
+Key: gug_56019_A_20150221_084856_012820, Target: gug, Predicted: tel
+Key: gug_56019_A_20150221_084856_014014, Target: gug, Predicted: tel
+Key: gug_53441_A_20140612_055846_030438, Target: gug, Predicted: tpi
+Key: gug_56019_A_20150221_084856_016253, Target: gug, Predicted: tel
+Key: gug_58717_A_20150201_022141_058962, Target: gug, Predicted: tur
+Key: gug_56019_A_20150221_084856_036105, Target: gug, Predicted: tel
+Key: gug_56019_A_20150221_084856_040658, Target: gug, Predicted: tel
+Key: gug_78161_A_20150312_093559_034226, Target: gug, Predicted: khk
+Key: gug_78161_A_20150312_093559_042472, Target: gug, Predicted: tel
+Key: gug_78161_A_20150312_093559_047262, Target: gug, Predicted: tel
+Key: gug_97911_A_20150304_082443_021658, Target: gug, Predicted: tel
+Key: gug_97911_A_20150304_082443_026325, Target: gug, Predicted: khk
+Key: gug_97911_A_20150304_082443_058612, Target: gug, Predicted: kmr
+Key: gug_97911_A_20150304_082443_060424, Target: gug, Predicted: tel
+Key: hat_14440_B_20130302_012105_008041, Target: hat, Predicted: ibo
+Key: hat_14440_B_20130302_012105_047037, Target: hat, Predicted: kat
+Key: hat_23983_B_20130503_023139_038952, Target: hat, Predicted: tpi
+Key: hat_32832_A_20130430_060411_001029, Target: hat, Predicted: amh
+Key: hat_49197_B_20130529_061436_045077, Target: hat, Predicted: nor
+Key: hat_61357_B_20130602_030259_014385, Target: hat, Predicted: ibo
+Key: hat_61357_B_20130602_030259_016440, Target: hat, Predicted: jav
+Key: hat_61357_B_20130602_030259_019295, Target: hat, Predicted: lao
+Key: hat_61357_B_20130602_030259_038622, Target: hat, Predicted: jav
+Key: hat_61357_B_20130602_030259_044885, Target: hat, Predicted: lao
+Key: hat_61357_B_20130602_030259_052961, Target: hat, Predicted: ibo
+Key: hat_65640_B_20130429_103434_018865, Target: hat, Predicted: tpi
+Key: hat_65640_B_20130429_103434_040586, Target: hat, Predicted: gug
+Key: hat_71263_A_20130602_021725_030898, Target: hat, Predicted: ibo
+Key: hat_77112_B_20130528_050544_000322, Target: hat, Predicted: swa
+Key: hat_74226_B_20130303_125222_045352, Target: hat, Predicted: tpi
+Key: hat_80881_A_20130220_022131_028792, Target: hat, Predicted: ibo
+Key: hat_78360_B_20130430_101414_041610, Target: hat, Predicted: vie
+Key: hat_80881_A_20130220_022131_034410, Target: hat, Predicted: gug
+Key: hat_80881_A_20130220_022131_012911, Target: hat, Predicted: yue
+Key: hat_79571_A_20130302_074959_009017, Target: hat, Predicted: amh
+Key: hat_80881_A_20130220_022131_016364, Target: hat, Predicted: gug
+Key: hat_81553_A_20130430_095301_044907, Target: hat, Predicted: gug
+Key: ibo_13427_B_20140810_232413_045755, Target: ibo, Predicted: tpi
+Key: ibo_19818_A_20140801_211130_040524, Target: ibo, Predicted: spa
+Key: ibo_13427_A_20140810_232413_021572, Target: ibo, Predicted: hat
+Key: ibo_33497_B_20140730_031414_000072, Target: ibo, Predicted: hat
+Key: ibo_28419_B_20140606_201307_010615, Target: ibo, Predicted: luo
+Key: ibo_35420_A_20140527_001314_003007, Target: ibo, Predicted: amh
+Key: ibo_34197_A_20140520_215059_023638, Target: ibo, Predicted: tel
+Key: ibo_35420_B_20140527_001314_032983, Target: ibo, Predicted: luo
+Key: ibo_50726_A_20140521_235356_009208, Target: ibo, Predicted: tel
+Key: ibo_50726_A_20140521_235356_011538, Target: ibo, Predicted: khk
+Key: ibo_50726_A_20140521_235356_019433, Target: ibo, Predicted: luo
+Key: ibo_50726_A_20140521_235356_022903, Target: ibo, Predicted: tel
+Key: ibo_50726_A_20140521_235356_024051, Target: ibo, Predicted: kat
+Key: ibo_50726_A_20140521_235356_031085, Target: ibo, Predicted: amh
+Key: ibo_53842_A_20140905_005627_005670, Target: ibo, Predicted: swa
+Key: ibo_50726_A_20140521_235356_032272, Target: ibo, Predicted: kaz
+Key: ibo_50726_A_20140521_235356_033466, Target: ibo, Predicted: tam
+Key: ibo_50726_A_20140521_235356_034611, Target: ibo, Predicted: kaz
+Key: ibo_53842_A_20140905_005627_027190, Target: ibo, Predicted: tpi
+Key: ibo_50726_A_20140521_235356_044616, Target: ibo, Predicted: kaz
+Key: ibo_53842_A_20140905_005627_028328, Target: ibo, Predicted: zul
+Key: ibo_52301_A_20140607_003158_025482, Target: ibo, Predicted: tel
+Key: ibo_52301_A_20140607_003158_039732, Target: ibo, Predicted: lit
+Key: ibo_50726_A_20140521_235356_053784, Target: ibo, Predicted: kat
+Key: ibo_63334_A_20150216_005033_042571, Target: ibo, Predicted: hat
+Key: ibo_63334_B_20150216_005033_011676, Target: ibo, Predicted: tpi
+Key: ibo_63334_B_20150216_005033_016403, Target: ibo, Predicted: tpi
+Key: ibo_58107_B_20140805_204322_048668, Target: ibo, Predicted: yue
+Key: ibo_63334_B_20150216_005033_027618, Target: ibo, Predicted: tpi
+Key: ibo_63334_B_20150216_005033_042556, Target: ibo, Predicted: tur
+Key: ibo_60508_A_20140521_055301_003833, Target: ibo, Predicted: kat
+Key: ibo_77112_B_20140609_224704_017697, Target: ibo, Predicted: swa
+Key: ibo_77803_A_20140517_202422_000000, Target: ibo, Predicted: amh
+Key: ibo_77803_A_20140517_202422_004727, Target: ibo, Predicted: luo
+Key: ibo_66959_B_20141031_215547_046888, Target: ibo, Predicted: tpi
+Key: ibo_79723_A_20150331_184104_029068, Target: ibo, Predicted: tpi
+Key: ibo_79723_A_20150331_184104_039764, Target: ibo, Predicted: zul
+Key: ibo_87280_A_20141026_002639_013843, Target: ibo, Predicted: lao
+Key: ibo_87313_B_20140802_002411_026424, Target: ibo, Predicted: tpi
+Key: ibo_94212_B_20140525_012758_040617, Target: ibo, Predicted: tgl
+Key: jav_10184_A_20141119_194233_051384, Target: jav, Predicted: lao
+Key: jav_10184_A_20141119_194233_017863, Target: jav, Predicted: ceb
+Key: jav_10184_A_20141119_194233_059426, Target: jav, Predicted: lao
+Key: jav_10184_A_20141119_194233_064088, Target: jav, Predicted: lao
+Key: jav_15535_B_20150104_232347_044037, Target: jav, Predicted: lao
+Key: jav_10184_A_20141119_194233_025595, Target: jav, Predicted: ceb
+Key: jav_10184_A_20141119_194233_029118, Target: jav, Predicted: ceb
+Key: jav_20133_B_20140911_170812_017218, Target: jav, Predicted: mlt
+Key: jav_21581_A_20141107_151147_007012, Target: jav, Predicted: ceb
+Key: jav_21393_B_20150304_163256_011005, Target: jav, Predicted: lao
+Key: jav_23046_A_20141103_212247_000903, Target: jav, Predicted: msa
+Key: jav_23505_A_20141029_003347_043611, Target: jav, Predicted: lao
+Key: jav_23046_A_20141103_212247_032712, Target: jav, Predicted: vie
+Key: jav_23046_A_20141103_212247_037678, Target: jav, Predicted: asm
+Key: jav_23505_B_20141029_003347_024606, Target: jav, Predicted: luo
+Key: jav_21807_A_20141125_194924_048994, Target: jav, Predicted: tur
+Key: jav_27590_A_20141227_191710_047520, Target: jav, Predicted: tgl
+Key: jav_27590_A_20141227_191710_050692, Target: jav, Predicted: cym
+Key: jav_27590_A_20141227_191710_055289, Target: jav, Predicted: hat
+Key: jav_36293_A_20141001_145552_001194, Target: jav, Predicted: ceb
+Key: jav_36293_A_20141001_145552_002391, Target: jav, Predicted: ceb
+Key: jav_36293_A_20141001_145552_003577, Target: jav, Predicted: ceb
+Key: jav_36293_A_20141001_145552_004772, Target: jav, Predicted: ceb
+Key: jav_36293_A_20141001_145552_005969, Target: jav, Predicted: ceb
+Key: jav_27590_B_20141227_191710_048052, Target: jav, Predicted: tgl
+Key: jav_36293_A_20141001_145552_012962, Target: jav, Predicted: ceb
+Key: jav_36293_A_20141001_145552_016469, Target: jav, Predicted: lao
+Key: jav_36293_A_20141001_145552_017668, Target: jav, Predicted: ceb
+Key: jav_36293_A_20141001_145552_020039, Target: jav, Predicted: ceb
+Key: jav_36505_A_20150106_201700_045871, Target: jav, Predicted: swa
+Key: jav_36293_A_20141001_145552_022419, Target: jav, Predicted: tgl
+Key: jav_41598_B_20150201_142509_000238, Target: jav, Predicted: tgl
+Key: jav_36293_A_20141001_145552_023618, Target: jav, Predicted: ceb
+Key: jav_36505_A_20150106_201700_053028, Target: jav, Predicted: lao
+Key: jav_36293_B_20141001_145552_013357, Target: jav, Predicted: lao
+Key: jav_36894_A_20140919_222930_000092, Target: jav, Predicted: ceb
+Key: jav_36293_A_20141001_145552_028259, Target: jav, Predicted: ceb
+Key: jav_36293_A_20141001_145552_030651, Target: jav, Predicted: ceb
+Key: jav_41745_B_20141108_162338_035175, Target: jav, Predicted: sun
+Key: jav_41745_B_20141108_162338_053557, Target: jav, Predicted: sun
+Key: jav_41745_B_20141108_162338_055340, Target: jav, Predicted: ind
+Key: jav_36293_A_20141001_145552_038873, Target: jav, Predicted: ceb
+Key: jav_36505_A_20150106_201700_014759, Target: jav, Predicted: luo
+Key: jav_36293_A_20141001_145552_043584, Target: jav, Predicted: ceb
+Key: jav_36293_A_20141001_145552_045967, Target: jav, Predicted: ceb
+Key: jav_36505_A_20150106_201700_033633, Target: jav, Predicted: ceb
+Key: jav_49118_B_20150201_023112_044097, Target: jav, Predicted: tgl
+Key: jav_52490_A_20140916_192446_040486, Target: jav, Predicted: gug
+Key: jav_49437_B_20150112_204645_005926, Target: jav, Predicted: cym
+Key: jav_52490_A_20140916_192446_052161, Target: jav, Predicted: kat
+Key: jav_49437_B_20150112_204645_038219, Target: jav, Predicted: hat
+Key: jav_52717_A_20140923_130849_023513, Target: jav, Predicted: khk
+Key: jav_56306_A_20150103_203751_000250, Target: jav, Predicted: lao
+Key: jav_52717_B_20140923_130849_020418, Target: jav, Predicted: lao
+Key: jav_52717_B_20140923_130849_028193, Target: jav, Predicted: lao
+Key: jav_65882_B_20141102_005627_039399, Target: jav, Predicted: tpi
+Key: jav_65882_B_20141102_005627_041556, Target: jav, Predicted: tpi
+Key: jav_64494_A_20141012_193548_027781, Target: jav, Predicted: asm
+Key: jav_70386_B_20141116_170547_042186, Target: jav, Predicted: pus
+Key: jav_73837_A_20141101_183259_039061, Target: jav, Predicted: khk
+Key: jav_68289_B_20150216_010725_004241, Target: jav, Predicted: swa
+Key: jav_68289_B_20150216_010725_012391, Target: jav, Predicted: gug
+Key: jav_68289_B_20150216_010725_030183, Target: jav, Predicted: tgl
+Key: jav_68289_B_20150216_010725_034736, Target: jav, Predicted: tgl
+Key: jav_73511_A_20141226_133330_013131, Target: jav, Predicted: lao
+Key: jav_70343_B_20150212_004248_014681, Target: jav, Predicted: khk
+Key: jav_70386_B_20141116_170547_013684, Target: jav, Predicted: hat
+Key: jav_78454_A_20141128_203259_000000, Target: jav, Predicted: amh
+Key: jav_68068_B_20150119_135822_043484, Target: jav, Predicted: asm
+Key: jav_70386_B_20141116_170547_028312, Target: jav, Predicted: asm
+Key: jav_73837_A_20141101_183259_026069, Target: jav, Predicted: tgl
+Key: jav_70386_B_20141116_170547_039873, Target: jav, Predicted: nep
+Key: jav_68182_A_20150111_002528_041112, Target: jav, Predicted: vie
+Key: jav_73837_A_20141101_183259_032808, Target: jav, Predicted: luo
+Key: jav_86467_B_20140920_125939_040288, Target: jav, Predicted: kaz
+Key: jav_88445_B_20141205_204305_027285, Target: jav, Predicted: tgl
+Key: jav_82935_A_20150104_005835_023512, Target: jav, Predicted: asm
+Key: jav_87921_B_20141225_203350_058462, Target: jav, Predicted: ces
+Key: jav_89457_B_20141117_212710_047919, Target: jav, Predicted: tgl
+Key: jav_89457_B_20141117_212710_051520, Target: jav, Predicted: lao
+Key: jav_78604_A_20141031_181612_041553, Target: jav, Predicted: ceb
+Key: jav_92176_A_20141222_021733_023517, Target: jav, Predicted: ceb
+Key: jav_82935_B_20150104_005835_046292, Target: jav, Predicted: ceb
+Key: jav_92176_A_20141222_021733_038411, Target: jav, Predicted: vie
+Key: jav_92176_B_20141222_021733_005786, Target: jav, Predicted: tgl
+Key: jav_92176_B_20141222_021733_007770, Target: jav, Predicted: tel
+Key: jav_92176_B_20141222_021733_012229, Target: jav, Predicted: luo
+Key: jav_92176_B_20141222_021733_035480, Target: jav, Predicted: swa
+Key: jav_92176_B_20141222_021733_043027, Target: jav, Predicted: asm
+Key: jav_92176_B_20141222_021733_046066, Target: jav, Predicted: sun
+Key: jav_93632_B_20150119_150118_019742, Target: jav, Predicted: msa
+Key: jav_93632_B_20150119_150118_049835, Target: jav, Predicted: tel
+Key: jav_93632_B_20150119_150118_052713, Target: jav, Predicted: tgl
+Key: kat_10184_A_20141107_212406_000114, Target: kat, Predicted: lit
+Key: kat_17165_A_20141117_063008_033016, Target: kat, Predicted: tur
+Key: kat_10184_A_20141107_212406_043262, Target: kat, Predicted: jav
+Key: kat_16184_A_20141020_233508_031838, Target: kat, Predicted: hat
+Key: kat_17472_A_20141201_023731_021216, Target: kat, Predicted: ceb
+Key: kat_17472_A_20141201_023731_026158, Target: kat, Predicted: tur
+Key: kat_17472_A_20141201_023731_033322, Target: kat, Predicted: sin
+Key: kat_17472_A_20141201_023731_036303, Target: kat, Predicted: yue
+Key: kat_17472_A_20141201_023731_038320, Target: kat, Predicted: tam
+Key: kat_17472_A_20141201_023731_040410, Target: kat, Predicted: luo
+Key: kat_18380_A_20141118_001754_037874, Target: kat, Predicted: tel
+Key: kat_18380_A_20141118_001754_050009, Target: kat, Predicted: kaz
+Key: kat_23239_A_20141127_054155_000001, Target: kat, Predicted: khk
+Key: kat_35467_A_20141020_054030_002174, Target: kat, Predicted: ibo
+Key: kat_38431_B_20141130_190122_043698, Target: kat, Predicted: khk
+Key: kat_38431_B_20141130_190122_053025, Target: kat, Predicted: kaz
+Key: kat_41592_A_20141117_033328_012603, Target: kat, Predicted: tpi
+Key: kat_41592_A_20141117_033328_017235, Target: kat, Predicted: hat
+Key: kat_41592_A_20141117_033328_024808, Target: kat, Predicted: hat
+Key: kat_41592_A_20141117_033328_028265, Target: kat, Predicted: hat
+Key: kat_41592_A_20141117_033328_030608, Target: kat, Predicted: vie
+Key: kat_42600_A_20141029_174857_000524, Target: kat, Predicted: gug
+Key: kat_41592_A_20141117_033328_037242, Target: kat, Predicted: swa
+Key: kat_41592_B_20141117_033328_045799, Target: kat, Predicted: amh
+Key: kat_41592_A_20141117_033328_041983, Target: kat, Predicted: pus
+Key: kat_44619_A_20141028_234639_041015, Target: kat, Predicted: khk
+Key: kat_44619_A_20141028_234639_055151, Target: kat, Predicted: khk
+Key: kat_44619_A_20141028_234639_019716, Target: kat, Predicted: kaz
+Key: kat_44619_A_20141028_234639_024432, Target: kat, Predicted: kmr
+Key: kat_44619_A_20141028_234639_031498, Target: kat, Predicted: bre
+Key: kat_47959_B_20141026_214447_024462, Target: kat, Predicted: ben
+Key: kat_51955_A_20141024_012212_000000, Target: kat, Predicted: amh
+Key: kat_56826_B_20141201_042429_027677, Target: kat, Predicted: kmr
+Key: kat_61190_A_20141029_013447_018598, Target: kat, Predicted: kaz
+Key: kat_61190_A_20141029_013447_034683, Target: kat, Predicted: khk
+Key: kat_73757_A_20141117_025704_005504, Target: kat, Predicted: tur
+Key: kat_73757_A_20141117_025704_008069, Target: kat, Predicted: tur
+Key: kat_73757_A_20141117_025704_010415, Target: kat, Predicted: tur
+Key: kat_73757_A_20141117_025704_019493, Target: kat, Predicted: kaz
+Key: kat_73757_A_20141117_025704_020655, Target: kat, Predicted: tur
+Key: kat_73757_A_20141117_025704_021743, Target: kat, Predicted: tur
+Key: kat_73757_A_20141117_025704_028261, Target: kat, Predicted: tur
+Key: kat_73757_A_20141117_025704_029368, Target: kat, Predicted: tur
+Key: kat_73757_A_20141117_025704_030527, Target: kat, Predicted: tur
+Key: kat_73757_A_20141117_025704_031627, Target: kat, Predicted: lit
+Key: kat_74121_A_20141120_020705_056254, Target: kat, Predicted: tur
+Key: kat_73757_A_20141117_025704_036336, Target: kat, Predicted: khk
+Key: kat_73757_A_20141117_025704_042111, Target: kat, Predicted: kaz
+Key: kat_73757_A_20141117_025704_050281, Target: kat, Predicted: tur
+Key: kat_73757_A_20141117_025704_053596, Target: kat, Predicted: zul
+Key: kat_81424_B_20141123_000421_002417, Target: kat, Predicted: ibo
+Key: kat_80781_A_20141104_212234_029772, Target: kat, Predicted: khk
+Key: kat_87298_A_20141025_213601_000000, Target: kat, Predicted: kaz
+Key: kat_87313_A_20141119_014632_003672, Target: kat, Predicted: kaz
+Key: kat_87313_A_20141119_014632_008923, Target: kat, Predicted: ceb
+Key: kat_87298_A_20141025_213601_050430, Target: kat, Predicted: khk
+Key: kat_87298_A_20141025_213601_053980, Target: kat, Predicted: tam
+Key: kat_87313_A_20141119_014632_061503, Target: kat, Predicted: khk
+Key: kat_87298_A_20141025_213601_055170, Target: kat, Predicted: kaz
+Key: kat_87298_A_20141025_213601_056354, Target: kat, Predicted: jav
+Key: kat_87313_B_20141119_014632_047555, Target: kat, Predicted: gug
+Key: kat_87298_A_20141025_213601_034714, Target: kat, Predicted: khk
+Key: kat_87298_A_20141025_213601_035904, Target: kat, Predicted: khk
+Key: kat_88776_A_20141006_193621_056655, Target: kat, Predicted: zul
+Key: kaz_20768_A_20140203_190423_002434, Target: kaz, Predicted: tur
+Key: kaz_20768_A_20140203_190423_019244, Target: kaz, Predicted: tur
+Key: kaz_17573_A_20140312_030325_027741, Target: kaz, Predicted: tpi
+Key: kaz_20768_A_20140203_190423_026665, Target: kaz, Predicted: amh
+Key: kaz_20768_A_20140203_190423_033715, Target: kaz, Predicted: tur
+Key: kaz_20682_A_20140114_221052_048257, Target: kaz, Predicted: gug
+Key: kaz_20768_A_20140203_190423_034895, Target: kaz, Predicted: tur
+Key: kaz_20768_A_20140203_190423_035970, Target: kaz, Predicted: tur
+Key: kaz_17914_A_20140126_234956_004076, Target: kaz, Predicted: tam
+Key: kaz_20768_A_20140203_185125_012980, Target: kaz, Predicted: tur
+Key: kaz_36669_B_20131206_164229_046083, Target: kaz, Predicted: tur
+Key: kaz_33175_B_20131105_201906_003032, Target: kaz, Predicted: kmr
+Key: kaz_44868_B_20131217_205716_001796, Target: kaz, Predicted: ibo
+Key: kaz_44868_B_20131217_205716_004022, Target: kaz, Predicted: kmr
+Key: kaz_23355_B_20140317_191841_029508, Target: kaz, Predicted: khk
+Key: kaz_41174_B_20131212_200450_053004, Target: kaz, Predicted: khk
+Key: kaz_23355_B_20140317_191841_053397, Target: kaz, Predicted: khk
+Key: kaz_24589_A_20131129_215929_000010, Target: kaz, Predicted: lit
+Key: kaz_70110_A_20131109_190313_007919, Target: kaz, Predicted: tur
+Key: kaz_47156_A_20140313_011009_046718, Target: kaz, Predicted: vie
+Key: kaz_72654_A_20131207_162604_000403, Target: kaz, Predicted: kmr
+Key: kaz_50726_A_20131118_025621_023121, Target: kaz, Predicted: tur
+Key: kaz_72654_A_20131207_162604_044645, Target: kaz, Predicted: kmr
+Key: kaz_72654_B_20131207_162604_031261, Target: kaz, Predicted: asm
+Key: kaz_77730_B_20131114_230511_028376, Target: kaz, Predicted: amh
+Key: kaz_77730_B_20131114_230511_029459, Target: kaz, Predicted: tpi
+Key: kaz_93320_B_20140218_173001_042863, Target: kaz, Predicted: tur
+Key: kaz_96842_A_20140131_154710_036513, Target: kaz, Predicted: khk
+Key: kaz_96842_A_20140131_154710_048679, Target: kaz, Predicted: tpi
+Key: khk_12916_B_20140930_182205_051257, Target: khk, Predicted: hat
+Key: khk_12916_B_20140930_182205_052432, Target: khk, Predicted: tam
+Key: kaz_96842_B_20140131_154710_013611, Target: kaz, Predicted: khk
+Key: kaz_96842_A_20140131_154710_015068, Target: kaz, Predicted: tur
+Key: kaz_96842_B_20140131_154710_028483, Target: kaz, Predicted: khk
+Key: kaz_96842_A_20140131_154710_030373, Target: kaz, Predicted: tur
+Key: khk_15163_A_20141020_201846_022885, Target: khk, Predicted: spa
+Key: khk_15324_A_20141031_194259_031379, Target: khk, Predicted: tam
+Key: khk_15324_A_20141031_194259_037917, Target: khk, Predicted: kat
+Key: khk_29208_B_20141018_152040_004635, Target: khk, Predicted: kaz
+Key: khk_29208_B_20141018_152040_010368, Target: khk, Predicted: amh
+Key: khk_29208_B_20141018_152040_013675, Target: khk, Predicted: zul
+Key: khk_29208_B_20141018_152040_019952, Target: khk, Predicted: som
+Key: khk_29208_B_20141018_152040_021125, Target: khk, Predicted: swa
+Key: khk_29208_B_20141018_152040_024507, Target: khk, Predicted: ibo
+Key: khk_29208_B_20141018_152040_026705, Target: khk, Predicted: hat
+Key: khk_32861_B_20141112_183418_031908, Target: khk, Predicted: jav
+Key: khk_32914_B_20141101_192546_000024, Target: khk, Predicted: tam
+Key: khk_32914_B_20141101_192546_001223, Target: khk, Predicted: kaz
+Key: khk_29208_B_20141018_152040_036119, Target: khk, Predicted: pus
+Key: khk_32301_A_20140927_150237_007302, Target: khk, Predicted: ibo
+Key: khk_32301_A_20140927_150237_036562, Target: khk, Predicted: nno
+Key: khk_29208_B_20141018_152040_056155, Target: khk, Predicted: ibo
+Key: khk_32914_B_20141101_192546_054968, Target: khk, Predicted: swa
+Key: khk_41741_A_20141002_230232_018106, Target: khk, Predicted: hat
+Key: khk_42243_B_20140924_154551_016645, Target: khk, Predicted: hat
+Key: khk_42243_B_20140924_154551_019014, Target: khk, Predicted: hat
+Key: khk_42243_B_20140924_154551_023792, Target: khk, Predicted: kmr
+Key: khk_38554_A_20140917_124843_000359, Target: khk, Predicted: hat
+Key: khk_42243_B_20140924_154551_028728, Target: khk, Predicted: luo
+Key: khk_42243_B_20140924_154551_031039, Target: khk, Predicted: kaz
+Key: khk_42243_B_20140924_154551_032234, Target: khk, Predicted: hat
+Key: khk_42243_B_20140924_154551_033396, Target: khk, Predicted: hat
+Key: khk_42243_B_20140924_154551_038963, Target: khk, Predicted: kaz
+Key: khk_42243_B_20140924_154551_041321, Target: khk, Predicted: hat
+Key: khk_42243_B_20140924_154551_043694, Target: khk, Predicted: ibo
+Key: khk_42243_B_20140924_154551_044893, Target: khk, Predicted: swa
+Key: khk_42243_B_20140924_154551_046082, Target: khk, Predicted: kat
+Key: khk_41741_A_20141002_230232_000647, Target: khk, Predicted: lao
+Key: khk_42243_B_20140924_154551_048430, Target: khk, Predicted: kat
+Key: khk_42243_B_20140924_154551_008138, Target: khk, Predicted: amh
+Key: khk_42243_B_20140924_154551_053068, Target: khk, Predicted: swa
+Key: khk_42243_B_20140924_154551_055972, Target: khk, Predicted: kmr
+Key: khk_43789_A_20141020_153059_005612, Target: khk, Predicted: amh
+Key: khk_43789_A_20141020_153059_058315, Target: khk, Predicted: luo
+Key: khk_44347_B_20141103_201828_003178, Target: khk, Predicted: tpi
+Key: khk_61678_A_20140919_183209_007194, Target: khk, Predicted: kaz
+Key: khk_48200_B_20141104_174608_001562, Target: khk, Predicted: tam
+Key: khk_48200_B_20141104_174608_011814, Target: khk, Predicted: kaz
+Key: khk_61678_A_20140919_183209_015134, Target: khk, Predicted: yue
+Key: khk_56090_A_20140917_155639_034508, Target: khk, Predicted: yue
+Key: khk_61678_A_20140919_183209_047196, Target: khk, Predicted: asm
+Key: khk_61678_A_20140919_183209_053397, Target: khk, Predicted: kaz
+Key: khk_61011_A_20140919_134829_037385, Target: khk, Predicted: yue
+Key: khk_61678_A_20140919_183209_054582, Target: khk, Predicted: tpi
+Key: khk_61678_A_20140919_183209_057627, Target: khk, Predicted: kaz
+Key: khk_78544_A_20140924_155131_014754, Target: khk, Predicted: kaz
+Key: kmr_14229_B_20130325_212616_027274, Target: kmr, Predicted: tur
+Key: khk_87884_B_20141014_190149_034467, Target: khk, Predicted: luo
+Key: kmr_16787_B_20130323_072114_020661, Target: kmr, Predicted: tur
+Key: kmr_16787_B_20130323_072114_053722, Target: kmr, Predicted: tur
+Key: kmr_15638_B_20130331_200208_030577, Target: kmr, Predicted: amh
+Key: kmr_16056_A_20130323_010902_056031, Target: kmr, Predicted: tur
+Key: kmr_22288_A_20131228_021559_008179, Target: kmr, Predicted: ceb
+Key: kmr_22288_A_20131228_021559_014122, Target: kmr, Predicted: tur
+Key: kmr_26206_A_20130507_004626_009278, Target: kmr, Predicted: gug
+Key: kmr_20454_B_20140125_002855_001925, Target: kmr, Predicted: kaz
+Key: kmr_20454_B_20140125_002855_003117, Target: kmr, Predicted: pus
+Key: kmr_22288_A_20131228_021559_000000, Target: kmr, Predicted: ceb
+Key: kmr_26999_A_20130414_220838_035286, Target: kmr, Predicted: urd
+Key: kmr_22288_A_20131228_021559_004683, Target: kmr, Predicted: ceb
+Key: kmr_26999_A_20130414_220838_053277, Target: kmr, Predicted: pus
+Key: kmr_34336_A_20130325_005404_056036, Target: kmr, Predicted: tur
+Key: kmr_29039_A_20130401_012825_032029, Target: kmr, Predicted: amh
+Key: kmr_35069_A_20130407_023338_000875, Target: kmr, Predicted: tur
+Key: kmr_31919_A_20130413_172911_000011, Target: kmr, Predicted: tur
+Key: kmr_29135_B_20130303_025305_050023, Target: kmr, Predicted: tur
+Key: kmr_29039_A_20130401_012825_004886, Target: kmr, Predicted: pus
+Key: kmr_46535_A_20140108_201338_006589, Target: kmr, Predicted: lit
+Key: kmr_35788_A_20131231_021724_026943, Target: kmr, Predicted: kaz
+Key: kmr_35788_A_20131231_021724_028134, Target: kmr, Predicted: kaz
+Key: kmr_46535_A_20140108_201338_011066, Target: kmr, Predicted: khk
+Key: kmr_35788_A_20131231_021724_029322, Target: kmr, Predicted: kaz
+Key: kmr_35788_A_20131231_021724_037053, Target: kmr, Predicted: lit
+Key: kmr_35788_A_20131231_021724_057350, Target: kmr, Predicted: khk
+Key: kmr_46535_A_20140108_201338_038179, Target: kmr, Predicted: kaz
+Key: kmr_35788_A_20131231_021724_064099, Target: kmr, Predicted: kaz
+Key: kmr_60830_A_20131223_005744_047132, Target: kmr, Predicted: khk
+Key: kmr_60830_A_20131223_005744_017741, Target: kmr, Predicted: kat
+Key: kmr_54735_A_20131228_012336_006164, Target: kmr, Predicted: khk
+Key: kmr_54735_A_20131228_012336_067811, Target: kmr, Predicted: lit
+Key: kmr_79139_A_20130621_004019_041961, Target: kmr, Predicted: gug
+Key: kmr_72903_A_20131225_002056_059562, Target: kmr, Predicted: tur
+Key: kmr_77225_A_20140106_235541_046086, Target: kmr, Predicted: khk
+Key: kmr_77225_A_20140106_235541_047285, Target: kmr, Predicted: khk
+Key: kmr_77225_A_20140106_235541_002601, Target: kmr, Predicted: khk
+Key: kmr_77225_A_20140106_235541_052011, Target: kmr, Predicted: tpi
+Key: kmr_77225_A_20140106_235541_054379, Target: kmr, Predicted: lit
+Key: kmr_77225_A_20140106_235541_013543, Target: kmr, Predicted: khk
+Key: kmr_86830_B_20130413_225657_001588, Target: kmr, Predicted: tgl
+Key: kmr_77225_B_20140106_235541_031416, Target: kmr, Predicted: tpi
+Key: kmr_77225_A_20140106_235541_027581, Target: kmr, Predicted: khk
+Key: kmr_78360_A_20140123_011434_006644, Target: kmr, Predicted: tur
+Key: lao_15042_A_20130727_173946_049342, Target: lao, Predicted: asm
+Key: lao_14158_A_20130409_182411_002652, Target: lao, Predicted: yue
+Key: lao_15042_A_20130727_173946_056098, Target: lao, Predicted: kaz
+Key: lao_15042_A_20130727_173946_062068, Target: lao, Predicted: swe
+Key: lao_14228_B_20130405_163836_016251, Target: lao, Predicted: vie
+Key: lao_22466_B_20130218_191925_033283, Target: lao, Predicted: kaz
+Key: lao_23681_A_20130730_162132_027917, Target: lao, Predicted: ceb
+Key: lao_23681_A_20130730_162132_034911, Target: lao, Predicted: ceb
+Key: lao_23995_A_20130731_195202_051372, Target: lao, Predicted: jav
+Key: lao_23681_B_20130730_162132_043721, Target: lao, Predicted: tam
+Key: lao_25012_A_20130814_141020_041372, Target: lao, Predicted: jav
+Key: lao_23995_A_20130731_195202_055484, Target: lao, Predicted: jav
+Key: lao_23995_A_20130731_195202_058875, Target: lao, Predicted: kaz
+Key: lao_23995_B_20130731_195202_000006, Target: lao, Predicted: tgl
+Key: lao_29765_B_20130426_185032_006590, Target: lao, Predicted: hat
+Key: lao_23995_A_20130731_195202_041755, Target: lao, Predicted: kaz
+Key: lao_41920_B_20130310_185621_038917, Target: lao, Predicted: luo
+Key: lao_29765_B_20130426_185032_046533, Target: lao, Predicted: tam
+Key: lao_41400_A_20130728_194416_033862, Target: lao, Predicted: jav
+Key: lao_52025_A_20130306_143713_025120, Target: lao, Predicted: tur
+Key: lao_60836_A_20130314_211014_025046, Target: lao, Predicted: gug
+Key: lao_72733_A_20130731_235502_038441, Target: lao, Predicted: ceb
+Key: lao_79190_A_20130714_135011_021885, Target: lao, Predicted: asm
+Key: lao_84370_B_20130506_190748_025300, Target: lao, Predicted: luo
+Key: lit_21581_A_20131216_220706_014319, Target: lit, Predicted: tpi
+Key: lit_21581_A_20131216_220706_018756, Target: lit, Predicted: pus
+Key: lit_21581_A_20131216_220706_019954, Target: lit, Predicted: lao
+Key: lit_21581_A_20131216_220706_040302, Target: lit, Predicted: ibo
+Key: lit_37064_A_20131129_035959_013167, Target: lit, Predicted: spa
+Key: lit_46702_A_20131115_213311_054246, Target: lit, Predicted: tam
+Key: lit_70110_B_20131118_222225_012085, Target: lit, Predicted: kat
+Key: lit_76837_A_20131020_200525_061435, Target: lit, Predicted: kaz
+Key: lit_70110_B_20131118_222225_028655, Target: lit, Predicted: kaz
+Key: lit_70110_B_20131118_222225_031106, Target: lit, Predicted: luo
+Key: lit_86878_B_20131129_043842_052347, Target: lit, Predicted: kat
+Key: lit_86878_B_20131129_043842_056777, Target: lit, Predicted: luo
+Key: lit_96934_A_20131207_231603_029039, Target: lit, Predicted: kmr
+Key: luo_12220_A_20141026_204025_053435, Target: luo, Predicted: hat
+Key: luo_14440_B_20141129_004855_047075, Target: luo, Predicted: swa
+Key: luo_43388_A_20141028_212938_020510, Target: luo, Predicted: swa
+Key: luo_25012_A_20150201_000040_003577, Target: luo, Predicted: amh
+Key: luo_56090_B_20141001_220534_001800, Target: luo, Predicted: swa
+Key: luo_56090_B_20141001_220534_035748, Target: luo, Predicted: tam
+Key: luo_47882_B_20150131_215134_013596, Target: luo, Predicted: ibo
+Key: luo_50726_B_20141015_222945_042179, Target: luo, Predicted: swa
+Key: luo_45560_B_20141012_204242_000000, Target: luo, Predicted: lao
+Key: luo_45697_A_20150211_181356_003721, Target: luo, Predicted: tam
+Key: luo_61225_B_20141014_225524_022997, Target: luo, Predicted: hat
+Key: luo_66026_A_20141207_212517_024451, Target: luo, Predicted: hat
+Key: luo_66026_A_20141207_212517_026866, Target: luo, Predicted: hat
+Key: luo_61225_B_20141014_225524_003458, Target: luo, Predicted: hat
+Key: luo_61225_B_20141014_225524_004632, Target: luo, Predicted: swa
+Key: luo_66026_A_20141207_212517_002144, Target: luo, Predicted: swa
+Key: luo_72349_A_20150313_194307_008763, Target: luo, Predicted: swa
+Key: luo_79820_A_20141005_212016_000020, Target: luo, Predicted: lao
+Key: luo_72349_A_20150313_194307_011149, Target: luo, Predicted: lao
+Key: luo_72349_A_20150313_194307_015010, Target: luo, Predicted: swa
+Key: luo_79820_A_20141005_212016_025297, Target: luo, Predicted: ibo
+Key: luo_72349_A_20150313_194307_043718, Target: luo, Predicted: gug
+Key: luo_97264_A_20141220_220653_028177, Target: luo, Predicted: hat
+Key: luo_97264_B_20141220_220653_009736, Target: luo, Predicted: swa
+Key: luo_97264_B_20141220_220653_027161, Target: luo, Predicted: pus
+Key: luo_99813_A_20141106_211637_001355, Target: luo, Predicted: hat
+Key: pus_28102_B_20120326_171523_031756, Target: pus, Predicted: ibo
+Key: pus_28102_A_20120326_171523_016544, Target: pus, Predicted: asm
+Key: pus_29368_A_20120321_233801_022408, Target: pus, Predicted: kaz
+Key: pus_29368_A_20120321_235133_019809, Target: pus, Predicted: amh
+Key: pus_29368_A_20120321_235133_025809, Target: pus, Predicted: tpi
+Key: pus_56226_B_20120205_235429_051101, Target: pus, Predicted: kmr
+Key: pus_61592_B_20120126_181735_055683, Target: pus, Predicted: tam
+Key: pus_82160_B_20120126_022907_039657, Target: pus, Predicted: lit
+Key: pus_76812_B_20120320_180439_024771, Target: pus, Predicted: kmr
+Key: pus_86680_B_20120309_181746_007085, Target: pus, Predicted: tur
+Key: pus_89308_B_20120131_214111_006761, Target: pus, Predicted: kaz
+Key: pus_89308_B_20120131_214111_013816, Target: pus, Predicted: khk
+Key: swa_17115_A_20140218_210921_045736, Target: swa, Predicted: khk
+Key: swa_17115_A_20140218_210921_046909, Target: swa, Predicted: tam
+Key: swa_17115_A_20140218_210921_053530, Target: swa, Predicted: khk
+Key: swa_17115_A_20140218_210921_055633, Target: swa, Predicted: lit
+Key: swa_17115_A_20140218_210921_056773, Target: swa, Predicted: khk
+Key: swa_17115_A_20140218_210921_057897, Target: swa, Predicted: tam
+Key: swa_17115_A_20140218_210921_059104, Target: swa, Predicted: khk
+Key: swa_16249_B_20131202_232723_000000, Target: swa, Predicted: amh
+Key: swa_14814_A_20140205_210842_036227, Target: swa, Predicted: luo
+Key: swa_24290_B_20140219_000423_029635, Target: swa, Predicted: ibo
+Key: swa_17115_A_20140218_210921_008274, Target: swa, Predicted: yor
+Key: swa_24290_B_20140219_000423_038218, Target: swa, Predicted: ibo
+Key: swa_24290_B_20140219_000423_043494, Target: swa, Predicted: ibo
+Key: swa_24239_A_20140206_191516_047532, Target: swa, Predicted: kmr
+Key: swa_15420_A_20140210_010333_056109, Target: swa, Predicted: tam
+Key: swa_24290_B_20140219_000423_046817, Target: swa, Predicted: luo
+Key: swa_24290_B_20140219_000423_054951, Target: swa, Predicted: hat
+Key: swa_34197_B_20121228_201800_025473, Target: swa, Predicted: amh
+Key: swa_38588_A_20130228_211322_002708, Target: swa, Predicted: kmr
+Key: swa_39893_B_20140115_023429_035762, Target: swa, Predicted: luo
+Key: swa_45459_A_20131012_022245_022507, Target: swa, Predicted: amh
+Key: swa_45459_A_20131012_022245_042952, Target: swa, Predicted: gug
+Key: swa_45459_B_20131012_022245_051341, Target: swa, Predicted: hat
+Key: swa_63084_B_20130801_015957_000093, Target: swa, Predicted: luo
+Key: swa_63084_B_20130801_015957_020419, Target: swa, Predicted: luo
+Key: swa_63084_B_20130801_015957_034096, Target: swa, Predicted: luo
+Key: swa_59549_B_20131003_203701_010964, Target: swa, Predicted: amh
+Key: swa_59549_B_20131003_203701_021584, Target: swa, Predicted: gug
+Key: swa_63084_A_20130801_014407_000990, Target: swa, Predicted: luo
+Key: swa_63084_A_20130801_014407_002124, Target: swa, Predicted: jav
+Key: swa_63084_A_20130801_014407_003284, Target: swa, Predicted: luo
+Key: swa_55042_B_20131217_033729_038274, Target: swa, Predicted: gug
+Key: swa_55106_A_20131215_030617_020580, Target: swa, Predicted: luo
+Key: swa_63084_A_20130801_015957_043913, Target: swa, Predicted: khk
+Key: swa_55106_A_20131215_030617_036846, Target: swa, Predicted: luo
+Key: swa_73819_B_20130911_163458_041264, Target: swa, Predicted: luo
+Key: swa_73819_B_20130911_163458_042943, Target: swa, Predicted: hat
+Key: swa_73819_B_20130927_003321_003623, Target: swa, Predicted: ibo
+Key: swa_73301_A_20140226_185528_044387, Target: swa, Predicted: kaz
+Key: swa_73301_A_20140226_185528_046773, Target: swa, Predicted: kaz
+Key: swa_72040_B_20131002_213605_049382, Target: swa, Predicted: hat
+Key: swa_73301_A_20140226_185528_048995, Target: swa, Predicted: lao
+Key: swa_66822_B_20130219_222318_006840, Target: swa, Predicted: zul
+Key: swa_73301_A_20140226_185528_050193, Target: swa, Predicted: tam
+Key: swa_73301_A_20140226_185528_054191, Target: swa, Predicted: tam
+Key: swa_73301_A_20140226_185528_058750, Target: swa, Predicted: kaz
+Key: swa_73301_B_20140226_185528_034627, Target: swa, Predicted: tpi
+Key: swa_76756_A_20130417_210400_018795, Target: swa, Predicted: kaz
+Key: swa_77990_A_20131007_063102_055659, Target: swa, Predicted: ibo
+Key: swa_90080_A_20140319_222809_027316, Target: swa, Predicted: tpi
+Key: swa_88661_A_20130801_192922_004595, Target: swa, Predicted: luo
+Key: swa_90080_A_20140319_222809_043606, Target: swa, Predicted: ibo
+Key: swa_77990_B_20131007_063102_030607, Target: swa, Predicted: hat
+Key: swa_77990_B_20131007_063102_031743, Target: swa, Predicted: gug
+Key: swa_88661_A_20130801_192922_015175, Target: swa, Predicted: khk
+Key: swa_90080_A_20140319_222809_052888, Target: swa, Predicted: kmr
+Key: swa_77990_A_20131007_063102_018142, Target: swa, Predicted: hat
+Key: swa_77990_A_20131007_063102_019854, Target: swa, Predicted: hat
+Key: swa_77990_A_20131007_063102_020926, Target: swa, Predicted: luo
+Key: swa_88661_A_20130801_192922_039985, Target: swa, Predicted: kat
+Key: swa_77990_A_20131007_063102_022115, Target: swa, Predicted: ibo
+Key: swa_77990_A_20131007_063102_026741, Target: swa, Predicted: lao
+Key: swa_77990_A_20131007_063102_027903, Target: swa, Predicted: hat
+Key: swa_88661_B_20130801_192922_026940, Target: swa, Predicted: luo
+Key: swa_77990_A_20131007_063102_038377, Target: swa, Predicted: gug
+Key: swa_90080_B_20140319_222809_051417, Target: swa, Predicted: zul
+Key: swa_92740_A_20130923_235638_046188, Target: swa, Predicted: luo
+Key: swa_84177_A_20131208_021104_017433, Target: swa, Predicted: luo
+Key: swa_84177_A_20131208_021104_023749, Target: swa, Predicted: hau
+Key: swa_92740_A_20130923_235638_051565, Target: swa, Predicted: hat
+Key: swa_92740_A_20130923_235638_056635, Target: swa, Predicted: amh
+Key: swa_98311_A_20130109_195922_006959, Target: swa, Predicted: zul
+Key: swa_98311_B_20130109_191639_008512, Target: swa, Predicted: zul
+Key: swa_98311_B_20130109_191639_019516, Target: swa, Predicted: tpi
+Key: swa_98311_B_20130109_191639_020611, Target: swa, Predicted: tgl
+Key: tam_20682_B_20130209_174057_019472, Target: tam, Predicted: tel
+Key: swa_98311_B_20130109_195922_013350, Target: swa, Predicted: luo
+Key: tam_18924_A_20130224_150538_016506, Target: tam, Predicted: ben
+Key: swa_98311_B_20130109_195922_029196, Target: swa, Predicted: ibo
+Key: tam_26602_A_20130215_003413_056511, Target: tam, Predicted: tel
+Key: tam_28606_A_20130126_221856_016645, Target: tam, Predicted: tel
+Key: tam_32287_A_20130902_231135_036249, Target: tam, Predicted: ben
+Key: tam_31624_A_20130107_221428_051356, Target: tam, Predicted: ben
+Key: tam_31624_B_20130107_221428_000000, Target: tam, Predicted: tel
+Key: tam_32287_A_20130902_231135_045702, Target: tam, Predicted: tur
+Key: tam_28606_A_20130126_221856_035560, Target: tam, Predicted: asm
+Key: tam_51701_A_20130312_022556_031090, Target: tam, Predicted: tel
+Key: tam_55136_A_20130705_164312_053934, Target: tam, Predicted: tel
+Key: tam_47451_A_20130210_010011_028292, Target: tam, Predicted: tel
+Key: tam_57935_A_20130126_234131_007506, Target: tam, Predicted: tel
+Key: tam_55136_A_20130705_164312_000000, Target: tam, Predicted: asm
+Key: tam_55136_A_20130705_164312_007891, Target: tam, Predicted: kat
+Key: tam_55136_A_20130705_164312_026691, Target: tam, Predicted: gug
+Key: tam_59747_B_20121222_160946_052528, Target: tam, Predicted: tgl
+Key: tam_63484_A_20130821_005511_000000, Target: tam, Predicted: tel
+Key: tam_63484_A_20130821_005511_007534, Target: tam, Predicted: tel
+Key: tam_59747_B_20121222_160946_003000, Target: tam, Predicted: tel
+Key: tam_64902_B_20130215_191500_019463, Target: tam, Predicted: tel
+Key: tam_78161_B_20130521_152635_032561, Target: tam, Predicted: mal
+Key: tam_87074_A_20130107_181209_056155, Target: tam, Predicted: tel
+Key: tam_91808_A_20130603_193623_033822, Target: tam, Predicted: spa
+Key: tam_91808_A_20130603_193623_035009, Target: tam, Predicted: spa
+Key: tam_91808_A_20130603_193623_036170, Target: tam, Predicted: spa
+Key: tam_90937_B_20130516_224543_057733, Target: tam, Predicted: tur
+Key: tam_91808_A_20130603_193623_038526, Target: tam, Predicted: spa
+Key: tam_91808_A_20130603_193623_040739, Target: tam, Predicted: spa
+Key: tam_91808_A_20130603_193623_044312, Target: tam, Predicted: spa
+Key: tam_91808_A_20130603_193623_046627, Target: tam, Predicted: spa
+Key: tam_91808_A_20130603_193623_047810, Target: tam, Predicted: luo
+Key: tam_91808_A_20130603_193623_050178, Target: tam, Predicted: spa
+Key: tam_90937_B_20130516_224543_000698, Target: tam, Predicted: tel
+Key: tam_91808_A_20130603_193623_029099, Target: tam, Predicted: nno
+Key: tel_22965_A_20131114_213605_007220, Target: tel, Predicted: mal
+Key: tel_22965_A_20131114_213605_021162, Target: tel, Predicted: tur
+Key: tel_21029_A_20131112_180205_050248, Target: tel, Predicted: tur
+Key: tel_19703_A_20131114_213952_013187, Target: tel, Predicted: pan
+Key: tel_19703_A_20131114_213952_025814, Target: tel, Predicted: gug
+Key: tel_34336_B_20131114_162157_016722, Target: tel, Predicted: yue
+Key: tel_46333_A_20131102_160049_011357, Target: tel, Predicted: tam
+Key: tel_46702_A_20131023_225137_036937, Target: tel, Predicted: lao
+Key: tel_46333_A_20131102_160049_044218, Target: tel, Predicted: tam
+Key: tel_39848_A_20131113_195552_021978, Target: tel, Predicted: tam
+Key: tel_46333_A_20131102_160049_050427, Target: tel, Predicted: tam
+Key: tel_46333_A_20131102_160049_058406, Target: tel, Predicted: asm
+Key: tel_49287_A_20131115_193114_004879, Target: tel, Predicted: tam
+Key: tel_56720_B_20131122_215343_034540, Target: tel, Predicted: tam
+Key: tel_61167_A_20131104_210455_048458, Target: tel, Predicted: yue
+Key: tel_52854_A_20131105_013802_050825, Target: tel, Predicted: tam
+Key: tel_58734_A_20131109_181122_003170, Target: tel, Predicted: khk
+Key: tel_64759_A_20131104_195356_000000, Target: tel, Predicted: asm
+Key: tel_65370_A_20140222_225324_021275, Target: tel, Predicted: tam
+Key: tel_52854_A_20131105_013802_010802, Target: tel, Predicted: ben
+Key: tel_86472_B_20131204_195705_020665, Target: tel, Predicted: asm
+Key: tel_86472_B_20131204_195705_038616, Target: tel, Predicted: tam
+Key: tel_74280_A_20131025_160420_021789, Target: tel, Predicted: tpi
+Key: tel_75064_A_20131114_174949_038514, Target: tel, Predicted: gug
+Key: tel_99487_A_20131027_195100_033800, Target: tel, Predicted: tam
+Key: tel_99487_A_20131027_195100_039891, Target: tel, Predicted: ben
+Key: tel_99487_A_20131027_195100_041660, Target: tel, Predicted: tam
+Key: tel_99487_A_20131027_195100_050799, Target: tel, Predicted: asm
+Key: tel_99487_A_20131027_195100_057242, Target: tel, Predicted: asm
+Key: tel_99487_A_20131027_195100_002251, Target: tel, Predicted: asm
+Key: tgl_16883_A_20120219_191154_047091, Target: tgl, Predicted: hat
+Key: tel_99487_A_20131027_195100_017295, Target: tel, Predicted: tam
+Key: tgl_25035_A_20120213_014750_039539, Target: tgl, Predicted: tur
+Key: tgl_24379_A_20120303_015051_058579, Target: tgl, Predicted: tam
+Key: tgl_42766_A_20120217_003639_055563, Target: tgl, Predicted: jav
+Key: tgl_47845_A_20120405_122139_002633, Target: tgl, Predicted: tur
+Key: tgl_35896_A_20120302_123550_002677, Target: tgl, Predicted: asm
+Key: tgl_35896_A_20120302_123550_037263, Target: tgl, Predicted: jav
+Key: tgl_42766_A_20120217_003639_003845, Target: tgl, Predicted: pus
+Key: tgl_42766_A_20120217_003639_013190, Target: tgl, Predicted: ceb
+Key: tgl_42766_A_20120217_003639_015529, Target: tgl, Predicted: jav
+Key: tgl_42766_A_20120217_003639_018270, Target: tgl, Predicted: asm
+Key: tgl_42766_A_20120217_003639_035803, Target: tgl, Predicted: hat
+Key: tgl_42766_A_20120217_003639_037776, Target: tgl, Predicted: ceb
+Key: tgl_42766_A_20120217_003639_040807, Target: tgl, Predicted: tur
+Key: tgl_42766_A_20120217_003639_041997, Target: tgl, Predicted: ceb
+Key: tgl_53982_A_20120224_233136_000808, Target: tgl, Predicted: hat
+Key: tgl_53982_A_20120224_233136_057579, Target: tgl, Predicted: ceb
+Key: tgl_53982_A_20120224_233136_004130, Target: tgl, Predicted: vie
+Key: tgl_53982_A_20120224_233136_058755, Target: tgl, Predicted: ceb
+Key: tgl_53982_A_20120224_233136_010994, Target: tgl, Predicted: ceb
+Key: tgl_57422_B_20120227_015422_058809, Target: tgl, Predicted: ceb
+Key: tgl_53982_B_20120224_233136_034869, Target: tgl, Predicted: kmr
+Key: tgl_53982_A_20120224_233136_038175, Target: tgl, Predicted: swa
+Key: tgl_53982_A_20120224_233136_042693, Target: tgl, Predicted: ceb
+Key: tgl_53982_B_20120224_233136_050145, Target: tgl, Predicted: asm
+Key: tgl_53982_A_20120224_233136_047272, Target: tgl, Predicted: hau
+Key: tgl_53982_A_20120224_233136_050701, Target: tgl, Predicted: ceb
+Key: tgl_57422_B_20120227_015422_005263, Target: tgl, Predicted: asm
+Key: tgl_65580_B_20120221_210222_019328, Target: tgl, Predicted: tam
+Key: tgl_66026_A_20120511_112437_000000, Target: tgl, Predicted: amh
+Key: tgl_69050_B_20120203_173053_035312, Target: tgl, Predicted: kaz
+Key: tgl_69050_B_20120203_173053_037579, Target: tgl, Predicted: ceb
+Key: tgl_69050_B_20120203_173053_038725, Target: tgl, Predicted: tel
+Key: tgl_81587_B_20120309_163209_015741, Target: tgl, Predicted: pus
+Key: tgl_83891_A_20120327_163405_052916, Target: tgl, Predicted: zul
+Key: tgl_83255_A_20120530_214353_011677, Target: tgl, Predicted: ceb
+Key: tgl_79698_A_20120315_223952_001345, Target: tgl, Predicted: kaz
+Key: tgl_85617_A_20120225_212818_053793, Target: tgl, Predicted: ceb
+Key: tgl_83891_A_20120327_163405_025856, Target: tgl, Predicted: jav
+Key: tgl_95589_B_20120225_032340_018516, Target: tgl, Predicted: ceb
+Key: tgl_93000_B_20120227_164805_038142, Target: tgl, Predicted: jav
+Key: tgl_93000_B_20120227_164805_050923, Target: tgl, Predicted: tpi
+Key: tgl_93000_B_20120227_164805_054742, Target: tgl, Predicted: hat
+Key: tpi_14440_A_20130824_153139_000000, Target: tpi, Predicted: yue
+Key: tpi_14440_A_20130824_153139_003448, Target: tpi, Predicted: spa
+Key: tpi_14440_A_20130824_153139_011131, Target: tpi, Predicted: luo
+Key: tpi_14440_B_20130824_152406_002756, Target: tpi, Predicted: hat
+Key: tpi_14440_B_20130824_153643_009173, Target: tpi, Predicted: tur
+Key: tpi_14875_A_20130731_170626_024438, Target: tpi, Predicted: lao
+Key: tpi_21244_A_20131010_122553_035642, Target: tpi, Predicted: kaz
+Key: tpi_21244_A_20131010_122553_000000, Target: tpi, Predicted: tel
+Key: tpi_21244_A_20131010_122553_004612, Target: tpi, Predicted: lit
+Key: tpi_29911_A_20131212_174224_044795, Target: tpi, Predicted: luo
+Key: tpi_32708_A_20130730_130556_000000, Target: tpi, Predicted: ben
+Key: tpi_32708_B_20130730_130556_018569, Target: tpi, Predicted: tur
+Key: tpi_32708_B_20130730_130556_040242, Target: tpi, Predicted: tel
+Key: tpi_32708_B_20130730_130556_044557, Target: tpi, Predicted: ibo
+Key: tpi_32708_B_20130730_130556_056565, Target: tpi, Predicted: asm
+Key: tpi_46535_A_20131219_223648_000000, Target: tpi, Predicted: ceb
+Key: tpi_46535_A_20131219_223648_002331, Target: tpi, Predicted: ceb
+Key: tpi_33175_B_20130621_162225_012734, Target: tpi, Predicted: kmr
+Key: tpi_46535_A_20131219_223648_004702, Target: tpi, Predicted: ceb
+Key: tpi_46535_A_20131219_223648_009377, Target: tpi, Predicted: ceb
+Key: tpi_46535_A_20131219_223648_011760, Target: tpi, Predicted: kaz
+Key: tpi_46535_A_20131219_223648_019923, Target: tpi, Predicted: kaz
+Key: tpi_46535_A_20131219_223648_021109, Target: tpi, Predicted: kaz
+Key: tpi_46535_A_20131219_223648_023410, Target: tpi, Predicted: lit
+Key: tpi_46535_A_20131219_223648_034620, Target: tpi, Predicted: kaz
+Key: tpi_46535_A_20131219_223648_035772, Target: tpi, Predicted: kaz
+Key: tpi_46535_A_20131219_223648_036968, Target: tpi, Predicted: lit
+Key: tpi_46535_A_20131219_223648_040468, Target: tpi, Predicted: kaz
+Key: tpi_46535_A_20131219_223648_043973, Target: tpi, Predicted: kaz
+Key: tpi_46535_A_20131219_223648_045145, Target: tpi, Predicted: lit
+Key: tpi_46535_A_20131219_223648_046327, Target: tpi, Predicted: kaz
+Key: tpi_46535_A_20131219_223648_048678, Target: tpi, Predicted: kaz
+Key: tpi_46535_A_20131219_223648_055715, Target: tpi, Predicted: khk
+Key: tpi_61963_A_20130830_141616_048803, Target: tpi, Predicted: ceb
+Key: tpi_67213_A_20131218_185924_000007, Target: tpi, Predicted: tam
+Key: tpi_65252_A_20131008_183014_027781, Target: tpi, Predicted: khk
+Key: tpi_61963_A_20130830_141616_014582, Target: tpi, Predicted: tgl
+Key: tpi_61963_A_20130830_141616_016809, Target: tpi, Predicted: ceb
+Key: tpi_61963_A_20130830_141616_056934, Target: tpi, Predicted: yue
+Key: tpi_67213_A_20131218_185924_009958, Target: tpi, Predicted: lao
+Key: tpi_67213_A_20131218_185924_013351, Target: tpi, Predicted: kaz
+Key: tpi_61963_A_20130830_141616_021279, Target: tpi, Predicted: ceb
+Key: tpi_61963_A_20130830_141616_023535, Target: tpi, Predicted: kaz
+Key: tpi_61963_A_20130830_141616_024661, Target: tpi, Predicted: kaz
+Key: tpi_67213_A_20131218_185924_024779, Target: tpi, Predicted: kaz
+Key: tpi_67213_A_20131218_185924_027069, Target: tpi, Predicted: kaz
+Key: tpi_61963_A_20130830_141616_029312, Target: tpi, Predicted: tel
+Key: tpi_65252_A_20131008_183014_049192, Target: tpi, Predicted: kaz
+Key: tpi_65252_A_20131008_183014_005780, Target: tpi, Predicted: ceb
+Key: tpi_67213_A_20131218_185924_030537, Target: tpi, Predicted: kaz
+Key: tpi_67213_A_20131218_185924_031728, Target: tpi, Predicted: lao
+Key: tpi_65252_A_20131008_183014_009092, Target: tpi, Predicted: kaz
+Key: tpi_65252_A_20131008_183014_052661, Target: tpi, Predicted: kaz
+Key: tpi_67213_A_20131218_185924_032851, Target: tpi, Predicted: kaz
+Key: tpi_65252_A_20131008_183014_055433, Target: tpi, Predicted: kaz
+Key: tpi_67213_A_20131218_185924_035174, Target: tpi, Predicted: lao
+Key: tpi_65252_A_20131008_183014_057351, Target: tpi, Predicted: kaz
+Key: tpi_61963_A_20130830_141616_040718, Target: tpi, Predicted: kaz
+Key: tpi_65252_A_20131008_183014_016004, Target: tpi, Predicted: kaz
+Key: tpi_67213_A_20131218_185924_040830, Target: tpi, Predicted: kaz
+Key: tpi_61963_A_20130830_141616_043068, Target: tpi, Predicted: kaz
+Key: tpi_67213_A_20131218_185924_042022, Target: tpi, Predicted: lit
+Key: tpi_65252_A_20131008_183014_019897, Target: tpi, Predicted: lit
+Key: tpi_67213_A_20131218_185924_043218, Target: tpi, Predicted: kaz
+Key: tpi_65252_A_20131008_183014_021073, Target: tpi, Predicted: ceb
+Key: tpi_67213_A_20131218_185924_044365, Target: tpi, Predicted: lao
+Key: tpi_74226_B_20130828_115915_013376, Target: tpi, Predicted: ces
+Key: tpi_67213_A_20131218_185924_046992, Target: tpi, Predicted: kaz
+Key: tpi_67213_A_20131218_185924_048146, Target: tpi, Predicted: tel
+Key: tpi_74226_B_20130828_115915_021609, Target: tpi, Predicted: tam
+Key: tpi_67213_A_20131218_185924_051355, Target: tpi, Predicted: asm
+Key: tpi_74226_B_20130828_115915_022808, Target: tpi, Predicted: tel
+Key: tpi_67213_A_20131218_185924_052511, Target: tpi, Predicted: kaz
+Key: tpi_67213_A_20131218_185924_054773, Target: tpi, Predicted: ceb
+Key: tpi_74226_B_20130828_115915_028173, Target: tpi, Predicted: luo
+Key: tpi_67213_B_20131218_185924_004053, Target: tpi, Predicted: kat
+Key: tpi_74226_B_20130828_115915_032801, Target: tpi, Predicted: luo
+Key: tpi_67213_B_20131218_185924_022830, Target: tpi, Predicted: tam
+Key: tpi_67213_B_20131218_185924_048209, Target: tpi, Predicted: khk
+Key: tpi_67213_B_20131218_185924_057759, Target: tpi, Predicted: ben
+Key: tpi_70726_A_20131222_161540_019387, Target: tpi, Predicted: khk
+Key: tpi_70726_A_20131222_161540_023625, Target: tpi, Predicted: lao
+Key: tpi_74226_B_20130828_115915_005432, Target: tpi, Predicted: asm
+Key: tpi_70726_A_20131222_161540_024767, Target: tpi, Predicted: lit
+Key: tpi_74226_B_20130828_115915_006573, Target: tpi, Predicted: tur
+Key: tpi_74226_B_20130828_115915_010205, Target: tpi, Predicted: tur
+Key: tpi_76837_A_20131207_184347_043285, Target: tpi, Predicted: lit
+Key: tpi_85179_B_20130920_130213_039435, Target: tpi, Predicted: hat
+Key: tpi_90777_B_20130725_111134_034620, Target: tpi, Predicted: zul
+Key: tpi_80577_B_20130930_204532_000000, Target: tpi, Predicted: kaz
+Key: tpi_80577_B_20130930_204532_004421, Target: tpi, Predicted: kaz
+Key: tpi_80577_B_20130930_204532_010349, Target: tpi, Predicted: khk
+Key: tpi_80577_B_20130930_204532_023950, Target: tpi, Predicted: tel
+Key: tpi_80577_B_20130930_204532_037734, Target: tpi, Predicted: kat
+Key: tpi_92886_B_20130711_144627_042334, Target: tpi, Predicted: tel
+Key: tur_21541_A_20120518_012528_006278, Target: tur, Predicted: kmr
+Key: tur_21541_A_20120518_012528_014473, Target: tur, Predicted: kmr
+Key: tur_21541_A_20120518_012528_018664, Target: tur, Predicted: kmr
+Key: tur_21541_A_20120518_012528_031124, Target: tur, Predicted: kmr
+Key: tur_11521_A_20120602_034839_041086, Target: tur, Predicted: ceb
+Key: tur_11521_A_20120602_034839_049327, Target: tur, Predicted: kmr
+Key: tur_32236_A_20120516_221311_019954, Target: tur, Predicted: zul
+Key: tur_39963_A_20120209_083935_000000, Target: tur, Predicted: tgl
+Key: tur_31256_A_20120531_015506_021282, Target: tur, Predicted: kmr
+Key: tur_44023_A_20120530_220359_022785, Target: tur, Predicted: kmr
+Key: tur_76372_B_20120709_015738_018584, Target: tur, Predicted: kmr
+Key: vie_12963_B_20120509_003852_003025, Target: vie, Predicted: lao
+Key: vie_11031_B_20120617_182613_013283, Target: vie, Predicted: gug
+Key: vie_14769_B_20120420_013147_027012, Target: vie, Predicted: hat
+Key: vie_14769_A_20120420_013147_000233, Target: vie, Predicted: asm
+Key: vie_32236_B_20120505_195420_013203, Target: vie, Predicted: lao
+Key: vie_32236_B_20120505_195420_021428, Target: vie, Predicted: tel
+Key: vie_31538_A_20120320_202748_018919, Target: vie, Predicted: gug
+Key: vie_31538_A_20120320_202748_020100, Target: vie, Predicted: tpi
+Key: vie_31538_A_20120320_202748_021281, Target: vie, Predicted: hat
+Key: vie_32236_B_20120505_195420_035087, Target: vie, Predicted: lao
+Key: vie_32236_B_20120505_195420_039604, Target: vie, Predicted: lao
+Key: vie_31538_A_20120320_202748_028672, Target: vie, Predicted: jav
+Key: vie_31538_A_20120320_202748_031014, Target: vie, Predicted: hat
+Key: vie_32236_B_20120505_195420_053886, Target: vie, Predicted: lao
+Key: vie_32236_B_20120505_195420_058774, Target: vie, Predicted: lao
+Key: vie_31538_A_20120320_202748_036046, Target: vie, Predicted: ibo
+Key: vie_31538_A_20120320_202748_038410, Target: vie, Predicted: tam
+Key: vie_31538_A_20120320_202748_040232, Target: vie, Predicted: gug
+Key: vie_31538_A_20120320_202748_003435, Target: vie, Predicted: por
+Key: vie_31538_A_20120320_202748_051509, Target: vie, Predicted: gug
+Key: vie_31538_A_20120320_202748_008124, Target: vie, Predicted: ibo
+Key: vie_31538_A_20120320_202748_010955, Target: vie, Predicted: gug
+Key: vie_31538_A_20120320_202748_053819, Target: vie, Predicted: hat
+Key: vie_31538_A_20120320_202748_012151, Target: vie, Predicted: hat
+Key: vie_35391_A_20120416_192241_046900, Target: vie, Predicted: gug
+Key: vie_35391_A_20120416_192241_054357, Target: vie, Predicted: jav
+Key: vie_45512_A_20120505_135144_053538, Target: vie, Predicted: hat
+Key: vie_45512_A_20120505_135144_004505, Target: vie, Predicted: ceb
+Key: vie_45512_A_20120505_135144_011913, Target: vie, Predicted: ibo
+Key: vie_63459_B_20120415_003841_021302, Target: vie, Predicted: kaz
+Key: vie_63459_B_20120415_003841_029357, Target: vie, Predicted: ceb
+Key: vie_79526_A_20120420_150504_017293, Target: vie, Predicted: gug
+Key: vie_85204_A_20120212_190017_002132, Target: vie, Predicted: gug
+Key: vie_77771_B_20120421_231323_012583, Target: vie, Predicted: tur
+Key: vie_85204_A_20120212_190017_028994, Target: vie, Predicted: tgl
+Key: vie_90202_A_20120502_194459_035522, Target: vie, Predicted: hat
+Key: vie_90202_A_20120502_194459_004459, Target: vie, Predicted: swa
+Key: vie_90202_A_20120502_194459_045611, Target: vie, Predicted: hat
+Key: vie_90202_A_20120502_194459_050828, Target: vie, Predicted: lao
+Key: vie_90202_A_20120502_194459_024874, Target: vie, Predicted: swa
+Key: vie_92386_A_20120322_195456_024899, Target: vie, Predicted: lao
+Key: vie_92386_A_20120322_195456_031300, Target: vie, Predicted: lao
+Key: zul_22466_B_20121130_231814_007273, Target: zul, Predicted: tgl
+Key: zul_28190_A_20121213_031401_032444, Target: zul, Predicted: asm
+Key: zul_35583_B_20130529_005600_015367, Target: zul, Predicted: ibo
+Key: zul_35583_B_20130529_005600_036873, Target: zul, Predicted: ibo
+Key: zul_35583_B_20130529_005600_053939, Target: zul, Predicted: amh
+Key: zul_43646_B_20121206_213819_000000, Target: zul, Predicted: asm
+Key: zul_42600_A_20121206_212006_003728, Target: zul, Predicted: tgl
+Key: zul_41100_A_20121129_003855_000513, Target: zul, Predicted: swa
+Key: zul_56198_A_20121128_190457_008384, Target: zul, Predicted: swa
+Key: zul_56198_A_20121128_190457_010740, Target: zul, Predicted: amh
+Key: zul_56198_A_20121128_190457_055361, Target: zul, Predicted: hat
+Key: zul_79858_B_20121126_013705_033065, Target: zul, Predicted: tgl
+Key: zul_82224_A_20130602_234038_044542, Target: zul, Predicted: ibo
+Key: zul_82224_A_20130602_234038_048706, Target: zul, Predicted: ssw
+Key: zul_84838_B_20121210_051040_025030, Target: zul, Predicted: jav
+Key: zul_93007_A_20130528_211314_002393, Target: zul, Predicted: luo
+Key: zul_93007_A_20130528_211314_017504, Target: zul, Predicted: sna
+Key: zul_93007_A_20130528_211314_057839, Target: zul, Predicted: sna

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/lid_inference_test.log

@@ -0,0 +1,286 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Mon Jun  2 02:33:14 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue04] 2025-06-02 02:33:33,758 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue04] 2025-06-02 02:33:45,800 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamConditionModel(
+  (frontend): S3prlFrontendCondition(
+    (upstream): S3PRLUpstreamCondition(
+      (upstream): UpstreamExpertCondition(
+        (model): Wav2Vec2ModelCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+            (aamsoftmax_weight): ParameterDict()
+            (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True)
+            (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec(
+              (ce): CrossEntropyLoss()
+              (lang2vec_head): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (lang2vec_loss): MSELoss()
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamConditionModel
+    Total Number of model parameters: 977.14 M
+    Number of trainable parameters: 977.14 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
+  warnings.warn(_create_warning_msg(
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue04] 2025-06-02 02:33:46,351 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/lids0
+[gpue04] 2025-06-02 02:34:18,099 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue04] 2025-06-02 02:34:42,593 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1
+[gpue04] 2025-06-02 02:35:05,422 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2
+[gpue04] 2025-06-02 02:35:28,092 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3
+[gpue04] 2025-06-02 02:35:52,743 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4
+[gpue04] 2025-06-02 02:36:39,290 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5
+[gpue04] 2025-06-02 02:37:10,073 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6
+[gpue04] 2025-06-02 02:37:13,207 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True
+[gpue04] 2025-06-02 02:37:13,208 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=240 threads=1
+# Ended (code 0) at Mon Jun  2 02:37:14 CDT 2025, elapsed time 240 seconds

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/results

@@ -0,0 +1,946 @@
+Accuracy: 86.82%
+Macro Accuracy: 86.92%
+Accuracy per Language:
+tam: 100.00%
+guj: 97.89%
+ell: 71.37%
+eng: 93.57%
+deu: 74.00%
+tel: 99.01%
+spa: 95.19%
+ara: 64.31%
+Key: ara_sada_acw_000000, Target: ara, Predicted: heb
+Key: ara_sada_acw_000032, Target: ara, Predicted: mon
+Key: ara_sada_acw_000064, Target: ara, Predicted: slv
+Key: ara_sada_acw_000096, Target: ara, Predicted: fao
+Key: ara_sada_acw_000001, Target: ara, Predicted: sot
+Key: ara_sada_acw_000033, Target: ara, Predicted: pus
+Key: ara_sada_acw_000065, Target: ara, Predicted: fin
+Key: ara_sada_acw_000067, Target: ara, Predicted: sqi
+Key: ara_sada_acw_000099, Target: ara, Predicted: amh
+Key: ara_sada_acw_000004, Target: ara, Predicted: tuk
+Key: ara_sada_acw_000005, Target: ara, Predicted: mon
+Key: ara_sada_acw_000069, Target: ara, Predicted: slv
+Key: ara_sada_acw_000101, Target: ara, Predicted: cmn
+Key: ara_sada_acw_000006, Target: ara, Predicted: eng
+Key: ara_sada_acw_000038, Target: ara, Predicted: sqi
+Key: ara_sada_acw_000071, Target: ara, Predicted: khm
+Key: ara_sada_acw_000103, Target: ara, Predicted: sqi
+Key: ara_sada_acw_000009, Target: ara, Predicted: som
+Key: ara_sada_acw_000041, Target: ara, Predicted: aze
+Key: ara_sada_acw_000042, Target: ara, Predicted: aze
+Key: ara_sada_acw_000106, Target: ara, Predicted: kaz
+Key: ara_sada_acw_000011, Target: ara, Predicted: ces
+Key: ara_sada_acw_000043, Target: ara, Predicted: pus
+Key: ara_sada_acw_000075, Target: ara, Predicted: cmn
+Key: ara_sada_acw_000044, Target: ara, Predicted: kaz
+Key: ara_sada_acw_000045, Target: ara, Predicted: fas
+Key: ara_sada_acw_000077, Target: ara, Predicted: deu
+Key: ara_sada_acw_000014, Target: ara, Predicted: hat
+Key: ara_sada_acw_000046, Target: ara, Predicted: som
+Key: ara_sada_acw_000078, Target: ara, Predicted: spa
+Key: ara_sada_acw_000110, Target: ara, Predicted: lin
+Key: ara_sada_acw_000079, Target: ara, Predicted: aze
+Key: ara_sada_acw_000111, Target: ara, Predicted: cym
+Key: ara_sada_acw_000016, Target: ara, Predicted: hrv
+Key: ara_sada_acw_000048, Target: ara, Predicted: eng
+Key: ara_sada_acw_000080, Target: ara, Predicted: ces
+Key: ara_sada_acw_000083, Target: ara, Predicted: jav
+Key: ara_sada_acw_000052, Target: ara, Predicted: mlt
+Key: ara_sada_acw_000116, Target: ara, Predicted: cym
+Key: ara_sada_acw_000053, Target: ara, Predicted: heb
+Key: ara_sada_acw_000117, Target: ara, Predicted: ben
+Key: ara_sada_acw_000118, Target: ara, Predicted: slv
+Key: ara_sada_acw_000023, Target: ara, Predicted: mon
+Key: ara_sada_acw_000056, Target: ara, Predicted: heb
+Key: ara_sada_acw_000120, Target: ara, Predicted: yid
+Key: ara_sada_acw_000025, Target: ara, Predicted: snd
+Key: ara_sada_acw_000057, Target: ara, Predicted: fra
+Key: ara_sada_acw_000124, Target: ara, Predicted: lit
+Key: ara_sada_acw_000029, Target: ara, Predicted: sin
+Key: ara_sada_acw_000125, Target: ara, Predicted: yid
+Key: ara_sada_acw_000094, Target: ara, Predicted: tat
+Key: ara_sada_acw_000063, Target: ara, Predicted: amh
+Key: ara_sada_acw_000127, Target: ara, Predicted: xty
+Key: ara_sada_acw_000129, Target: ara, Predicted: pol
+Key: ara_sada_afb_000018, Target: ara, Predicted: afr
+Key: ara_sada_acw_000130, Target: ara, Predicted: war
+Key: ara_sada_acw_000162, Target: ara, Predicted: ell
+Key: ara_sada_acw_000131, Target: ara, Predicted: guj
+Key: ara_sada_acw_000163, Target: ara, Predicted: sna
+Key: ara_sada_acw_000132, Target: ara, Predicted: bre
+Key: ara_sada_acw_000164, Target: ara, Predicted: mlt
+Key: ara_sada_afb_000021, Target: ara, Predicted: ell
+Key: ara_sada_afb_000054, Target: ara, Predicted: nld
+Key: ara_sada_afb_000023, Target: ara, Predicted: heb
+Key: ara_sada_acw_000135, Target: ara, Predicted: som
+Key: ara_sada_acw_000167, Target: ara, Predicted: isl
+Key: ara_sada_afb_000025, Target: ara, Predicted: mon
+Key: ara_sada_afb_000057, Target: ara, Predicted: sqi
+Key: ara_sada_acw_000169, Target: ara, Predicted: deu
+Key: ara_sada_afb_000058, Target: ara, Predicted: hau
+Key: ara_sada_acw_000170, Target: ara, Predicted: mon
+Key: ara_sada_acw_000139, Target: ara, Predicted: azz
+Key: ara_sada_acw_000171, Target: ara, Predicted: heb
+Key: ara_sada_afb_000028, Target: ara, Predicted: amh
+Key: ara_sada_acw_000140, Target: ara, Predicted: mon
+Key: ara_sada_acw_000173, Target: ara, Predicted: tat
+Key: ara_sada_afb_000032, Target: ara, Predicted: tuk
+Key: ara_sada_afb_000002, Target: ara, Predicted: rus
+Key: ara_sada_afb_000034, Target: ara, Predicted: ltz
+Key: ara_sada_afb_000003, Target: ara, Predicted: hrv
+Key: ara_sada_afb_000035, Target: ara, Predicted: som
+Key: ara_sada_acw_000147, Target: ara, Predicted: mlt
+Key: ara_sada_afb_000068, Target: ara, Predicted: fas
+Key: ara_sada_afb_000037, Target: ara, Predicted: abk
+Key: ara_sada_afb_000069, Target: ara, Predicted: hau
+Key: ara_sada_acw_000149, Target: ara, Predicted: aze
+Key: ara_sada_afb_000070, Target: ara, Predicted: pus
+Key: ara_sada_afb_000007, Target: ara, Predicted: pus
+Key: ara_sada_afb_000039, Target: ara, Predicted: heb
+Key: ara_sada_afb_000071, Target: ara, Predicted: heb
+Key: ara_sada_acw_000151, Target: ara, Predicted: heb
+Key: ara_sada_afb_000008, Target: ara, Predicted: heb
+Key: ara_sada_afb_000072, Target: ara, Predicted: cym
+Key: ara_sada_afb_000009, Target: ara, Predicted: tat
+Key: ara_sada_afb_000073, Target: ara, Predicted: mya
+Key: ara_sada_acw_000153, Target: ara, Predicted: fra
+Key: ara_sada_afb_000010, Target: ara, Predicted: heb
+Key: ara_sada_afb_000042, Target: ara, Predicted: yid
+Key: ara_sada_acw_000154, Target: ara, Predicted: som
+Key: ara_sada_afb_000011, Target: ara, Predicted: nep
+Key: ara_sada_afb_000043, Target: ara, Predicted: mlt
+Key: ara_sada_afb_000075, Target: ara, Predicted: abk
+Key: ara_sada_afb_000046, Target: ara, Predicted: nep
+Key: ara_sada_afb_000016, Target: ara, Predicted: amh
+Key: ara_sada_afb_000080, Target: ara, Predicted: nep
+Key: ara_sada_afb_000178, Target: ara, Predicted: som
+Key: ara_sada_afb_000083, Target: ara, Predicted: aze
+Key: ara_sada_afb_000115, Target: ara, Predicted: som
+Key: ara_sada_afb_000147, Target: ara, Predicted: fas
+Key: ara_sada_afb_000084, Target: ara, Predicted: fra
+Key: ara_sada_afb_000148, Target: ara, Predicted: fra
+Key: ara_sada_afb_000085, Target: ara, Predicted: eng
+Key: ara_sada_afb_000117, Target: ara, Predicted: glv
+Key: ara_sada_afb_000181, Target: ara, Predicted: spa
+Key: ara_sada_afb_000118, Target: ara, Predicted: guj
+Key: ara_sada_afb_000119, Target: ara, Predicted: hun
+Key: ara_sada_afb_000151, Target: ara, Predicted: fas
+Key: ara_sada_afb_000183, Target: ara, Predicted: deu
+Key: ara_sada_afb_000088, Target: ara, Predicted: swa
+Key: ara_sada_afb_000184, Target: ara, Predicted: som
+Key: ara_sada_afb_000121, Target: ara, Predicted: slv
+Key: ara_sada_afb_000091, Target: ara, Predicted: tat
+Key: ara_sada_afb_000155, Target: ara, Predicted: mya
+Key: ara_sada_afb_000124, Target: ara, Predicted: mya
+Key: ara_sada_afb_000156, Target: ara, Predicted: amh
+Key: ara_sada_afb_000188, Target: ara, Predicted: mon
+Key: ara_sada_afb_000094, Target: ara, Predicted: fra
+Key: ara_sada_afb_000126, Target: ara, Predicted: heb
+Key: ara_sada_afb_000190, Target: ara, Predicted: yor
+Key: ara_sada_afb_000159, Target: ara, Predicted: heb
+Key: ara_sada_afb_000160, Target: ara, Predicted: heb
+Key: ara_sada_afb_000192, Target: ara, Predicted: tat
+Key: ara_sada_afb_000129, Target: ara, Predicted: hau
+Key: ara_sada_afb_000161, Target: ara, Predicted: som
+Key: ara_sada_afb_000193, Target: ara, Predicted: hrv
+Key: ara_sada_afb_000130, Target: ara, Predicted: deu
+Key: ara_sada_afb_000162, Target: ara, Predicted: bre
+Key: ara_sada_afb_000099, Target: ara, Predicted: afr
+Key: ara_sada_afb_000131, Target: ara, Predicted: nld
+Key: ara_sada_afb_000163, Target: ara, Predicted: sna
+Key: ara_sada_afb_000100, Target: ara, Predicted: nep
+Key: ara_sada_afb_000132, Target: ara, Predicted: bre
+Key: ara_sada_afb_000164, Target: ara, Predicted: bod
+Key: ara_sada_afb_000196, Target: ara, Predicted: uzb
+Key: ara_sada_afb_000165, Target: ara, Predicted: heb
+Key: ara_sada_afb_000197, Target: ara, Predicted: ben
+Key: ara_sada_afb_000134, Target: ara, Predicted: eng
+Key: ara_sada_afb_000166, Target: ara, Predicted: tgk
+Key: ara_sada_afb_000167, Target: ara, Predicted: nso
+Key: ara_sada_afb_000104, Target: ara, Predicted: heb
+Key: ara_sada_afb_000136, Target: ara, Predicted: slv
+Key: ara_sada_afb_000168, Target: ara, Predicted: swa
+Key: ara_sada_afb_000137, Target: ara, Predicted: bod
+Key: ara_sada_afb_000106, Target: ara, Predicted: eng
+Key: ara_sada_afb_000138, Target: ara, Predicted: tat
+Key: ara_sada_afb_000108, Target: ara, Predicted: eus
+Key: ara_sada_afb_000140, Target: ara, Predicted: tat
+Key: ara_sada_afb_000141, Target: ara, Predicted: eng
+Key: ara_sada_afb_000173, Target: ara, Predicted: kat
+Key: ara_sada_afb_000142, Target: ara, Predicted: nld
+Key: ara_sada_afb_000208, Target: ara, Predicted: kan
+Key: ara_sada_ars_000007, Target: ara, Predicted: amh
+Key: ara_sada_ars_000039, Target: ara, Predicted: fra
+Key: ara_sada_ars_000008, Target: ara, Predicted: som
+Key: ara_sada_ars_000042, Target: ara, Predicted: ces
+Key: ara_sada_ars_000043, Target: ara, Predicted: lit
+Key: ara_sada_ars_000044, Target: ara, Predicted: hrv
+Key: ara_sada_afb_000216, Target: ara, Predicted: heb
+Key: ara_sada_ars_000016, Target: ara, Predicted: grn
+Key: ara_sada_ars_000048, Target: ara, Predicted: som
+Key: ara_sada_ars_000049, Target: ara, Predicted: heb
+Key: ara_sada_afb_000220, Target: ara, Predicted: hye
+Key: ara_sada_ars_000018, Target: ara, Predicted: bre
+Key: ara_sada_afb_000221, Target: ara, Predicted: azz
+Key: ara_sada_arb_000031, Target: ara, Predicted: yor
+Key: ara_sada_ars_000019, Target: ara, Predicted: nld
+Key: ara_sada_ars_000053, Target: ara, Predicted: cym
+Key: ara_sada_ars_000022, Target: ara, Predicted: mlg
+Key: ara_sada_ars_000055, Target: ara, Predicted: eng
+Key: ara_sada_ars_000024, Target: ara, Predicted: som
+Key: ara_sada_arb_000005, Target: ara, Predicted: heb
+Key: ara_sada_ars_000058, Target: ara, Predicted: mya
+Key: ara_sada_arb_000041, Target: ara, Predicted: mlg
+Key: ara_sada_ars_000061, Target: ara, Predicted: isl
+Key: ara_sada_ars_000030, Target: ara, Predicted: amh
+Key: ara_sada_ars_000031, Target: ara, Predicted: snd
+Key: ara_sada_ars_000032, Target: ara, Predicted: deu
+Key: ara_sada_ars_000033, Target: ara, Predicted: bos
+Key: ara_sada_ars_000037, Target: ara, Predicted: pus
+Key: ara_sada_ars_000069, Target: ara, Predicted: bre
+Key: ara_sada_ars_000038, Target: ara, Predicted: pol
+Key: deu_swissdial_ag_000014, Target: deu, Predicted: afr
+Key: ara_sada_ars_000072, Target: ara, Predicted: slk
+Key: ara_sada_ars_000136, Target: ara, Predicted: nno
+Key: deu_swissdial_ag_000015, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000016, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000017, Target: deu, Predicted: nld
+Key: ara_sada_ars_000075, Target: ara, Predicted: deu
+Key: deu_swissdial_ag_000019, Target: deu, Predicted: yid
+Key: ara_sada_ars_000110, Target: ara, Predicted: pan
+Key: ara_sada_ars_000111, Target: ara, Predicted: sqi
+Key: deu_swissdial_ag_000023, Target: deu, Predicted: afr
+Key: ara_sada_ars_000082, Target: ara, Predicted: fra
+Key: ara_sada_ars_000146, Target: ara, Predicted: tuk
+Key: deu_swissdial_ag_000025, Target: deu, Predicted: afr
+Key: ara_sada_ars_000083, Target: ara, Predicted: pus
+Key: ara_sada_ars_000115, Target: ara, Predicted: tat
+Key: deu_swissdial_ag_000026, Target: deu, Predicted: cym
+Key: deu_swissdial_ag_000027, Target: deu, Predicted: yid
+Key: deu_swissdial_ag_000028, Target: deu, Predicted: nld
+Key: ara_sada_ars_000150, Target: ara, Predicted: isl
+Key: deu_swissdial_ag_000029, Target: deu, Predicted: ces
+Key: ara_sada_ars_000151, Target: ara, Predicted: heb
+Key: ara_sada_ars_000152, Target: ara, Predicted: amh
+Key: deu_swissdial_ag_000031, Target: deu, Predicted: nld
+Key: ara_sada_ars_000089, Target: ara, Predicted: bod
+Key: ara_sada_ars_000121, Target: ara, Predicted: fra
+Key: deu_swissdial_ag_000032, Target: deu, Predicted: afr
+Key: ara_sada_ars_000090, Target: ara, Predicted: mar
+Key: ara_sada_ars_000091, Target: ara, Predicted: fra
+Key: ara_sada_ars_000123, Target: ara, Predicted: isl
+Key: ara_sada_ars_000092, Target: ara, Predicted: cym
+Key: deu_swissdial_ag_000035, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000004, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000037, Target: deu, Predicted: gle
+Key: deu_swissdial_ag_000006, Target: deu, Predicted: cym
+Key: deu_swissdial_ag_000038, Target: deu, Predicted: afr
+Key: ara_sada_ars_000129, Target: ara, Predicted: heb
+Key: deu_swissdial_ag_000008, Target: deu, Predicted: tat
+Key: deu_swissdial_ag_000040, Target: deu, Predicted: afr
+Key: ara_sada_ars_000098, Target: ara, Predicted: khm
+Key: ara_sada_ars_000131, Target: ara, Predicted: heb
+Key: deu_swissdial_ag_000042, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000013, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000110, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000047, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000113, Target: deu, Predicted: gle
+Key: deu_swissdial_ag_000114, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000115, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000084, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000148, Target: deu, Predicted: gle
+Key: deu_swissdial_ag_000085, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000117, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000149, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000086, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000118, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000150, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000055, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000151, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000088, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000120, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000089, Target: deu, Predicted: gle
+Key: deu_swissdial_ag_000121, Target: deu, Predicted: cym
+Key: deu_swissdial_ag_000058, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000122, Target: deu, Predicted: gle
+Key: deu_swissdial_ag_000059, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000091, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000092, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000124, Target: deu, Predicted: ces
+Key: deu_swissdial_ag_000093, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000125, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000157, Target: deu, Predicted: cym
+Key: deu_swissdial_ag_000126, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000095, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000127, Target: deu, Predicted: slv
+Key: deu_swissdial_ag_000159, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000064, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000096, Target: deu, Predicted: ces
+Key: deu_swissdial_ag_000128, Target: deu, Predicted: gle
+Key: deu_swissdial_ag_000097, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000099, Target: deu, Predicted: cym
+Key: deu_swissdial_ag_000163, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000100, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000134, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000103, Target: deu, Predicted: gle
+Key: deu_swissdial_ag_000135, Target: deu, Predicted: cym
+Key: deu_swissdial_ag_000072, Target: deu, Predicted: gle
+Key: deu_swissdial_be_000004, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000138, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000107, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000139, Target: deu, Predicted: gle
+Key: deu_swissdial_ag_000076, Target: deu, Predicted: afr
+Key: deu_swissdial_ag_000108, Target: deu, Predicted: ltz
+Key: deu_swissdial_ag_000140, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000008, Target: deu, Predicted: cym
+Key: deu_swissdial_ag_000077, Target: deu, Predicted: nld
+Key: deu_swissdial_ag_000109, Target: deu, Predicted: gle
+Key: deu_swissdial_ag_000141, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000042, Target: deu, Predicted: isl
+Key: deu_swissdial_be_000043, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000075, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000107, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000044, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000076, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000108, Target: deu, Predicted: ltz
+Key: deu_swissdial_be_000013, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000110, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000016, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000048, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000112, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000049, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000113, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000018, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000082, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000114, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000115, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000084, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000021, Target: deu, Predicted: ltz
+Key: deu_swissdial_be_000022, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000054, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000023, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000087, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000024, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000120, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000025, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000089, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000121, Target: deu, Predicted: cym
+Key: deu_swissdial_be_000059, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000091, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000123, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000060, Target: deu, Predicted: isl
+Key: deu_swissdial_be_000124, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000093, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000125, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000030, Target: deu, Predicted: nld
+Key: deu_swissdial_be_000062, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000031, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000095, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000096, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000033, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000129, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000034, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000066, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000035, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000036, Target: deu, Predicted: slv
+Key: deu_swissdial_be_000039, Target: deu, Predicted: afr
+Key: deu_swissdial_be_000103, Target: deu, Predicted: est
+Key: deu_swissdial_be_000040, Target: deu, Predicted: afr
+Key: deu_swissdial_bs_000005, Target: deu, Predicted: gle
+Key: deu_swissdial_be_000105, Target: deu, Predicted: afr
+Key: deu_swissdial_bs_000103, Target: deu, Predicted: ltz
+Key: deu_swissdial_bs_000082, Target: deu, Predicted: nld
+Key: deu_swissdial_bs_000114, Target: deu, Predicted: cym
+Key: deu_swissdial_bs_000088, Target: deu, Predicted: ltz
+Key: deu_swissdial_bs_000093, Target: deu, Predicted: gle
+Key: deu_swissdial_bs_000031, Target: deu, Predicted: cym
+Key: deu_swissdial_bs_000036, Target: deu, Predicted: ltz
+Key: deu_swissdial_bs_000133, Target: deu, Predicted: nld
+Key: deu_swissdial_gr_000088, Target: deu, Predicted: nld
+Key: deu_swissdial_bs_000139, Target: deu, Predicted: afr
+Key: deu_swissdial_bs_000144, Target: deu, Predicted: ltz
+Key: deu_swissdial_gr_000064, Target: deu, Predicted: slv
+Key: deu_swissdial_gr_000040, Target: deu, Predicted: afr
+Key: deu_swissdial_gr_000105, Target: deu, Predicted: afr
+Key: deu_swissdial_gr_000010, Target: deu, Predicted: afr
+Key: deu_swissdial_gr_000114, Target: deu, Predicted: slv
+Key: deu_swissdial_gr_000116, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000006, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000038, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000007, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000071, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000040, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000072, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000042, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000011, Target: deu, Predicted: nno
+Key: deu_swissdial_lu_000043, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000044, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000077, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000014, Target: deu, Predicted: yid
+Key: deu_swissdial_lu_000047, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000079, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000016, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000017, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000019, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000051, Target: deu, Predicted: cym
+Key: deu_swissdial_lu_000052, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000053, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000085, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000054, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000023, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000055, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000087, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000024, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000056, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000026, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000058, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000090, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000027, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000059, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000092, Target: deu, Predicted: glv
+Key: deu_swissdial_lu_000029, Target: deu, Predicted: gle
+Key: deu_swissdial_lu_000093, Target: deu, Predicted: cym
+Key: deu_swissdial_lu_000031, Target: deu, Predicted: cym
+Key: deu_swissdial_lu_000095, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000096, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000033, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000065, Target: deu, Predicted: hun
+Key: deu_swissdial_lu_000097, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000002, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000034, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000066, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000067, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000099, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000004, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000068, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000069, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000102, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000134, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000166, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000103, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000135, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000104, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000136, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000168, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000137, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000106, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000170, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000108, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000140, Target: deu, Predicted: gle
+Key: deu_swissdial_lu_000112, Target: deu, Predicted: yid
+Key: deu_swissdial_lu_000144, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000113, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000116, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000150, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000151, Target: deu, Predicted: nld
+Key: deu_swissdial_sg_000010, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000121, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000123, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000155, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000124, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000157, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000126, Target: deu, Predicted: afr
+Key: deu_swissdial_lu_000158, Target: deu, Predicted: ltz
+Key: deu_swissdial_lu_000159, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000130, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000162, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000163, Target: deu, Predicted: nld
+Key: deu_swissdial_sg_000021, Target: deu, Predicted: gle
+Key: deu_swissdial_lu_000132, Target: deu, Predicted: nld
+Key: deu_swissdial_lu_000164, Target: deu, Predicted: ltz
+Key: deu_swissdial_sg_000022, Target: deu, Predicted: gle
+Key: deu_swissdial_lu_000133, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000006, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000038, Target: deu, Predicted: cym
+Key: deu_swissdial_vs_000042, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000045, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000047, Target: deu, Predicted: slv
+Key: deu_swissdial_vs_000016, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000048, Target: deu, Predicted: afr
+Key: deu_swissdial_sg_000067, Target: deu, Predicted: cym
+Key: deu_swissdial_vs_000053, Target: deu, Predicted: gle
+Key: deu_swissdial_sg_000104, Target: deu, Predicted: est
+Key: deu_swissdial_vs_000059, Target: deu, Predicted: nld
+Key: deu_swissdial_sg_000078, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000060, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000030, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000032, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000034, Target: deu, Predicted: gle
+Key: deu_swissdial_vs_000003, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000004, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000005, Target: deu, Predicted: cym
+Key: deu_swissdial_zh_000026, Target: deu, Predicted: gle
+Key: deu_swissdial_vs_000136, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000073, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000137, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000029, Target: deu, Predicted: gle
+Key: deu_swissdial_zh_000031, Target: deu, Predicted: yid
+Key: deu_swissdial_vs_000076, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000000, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000032, Target: deu, Predicted: nld
+Key: deu_swissdial_vs_000078, Target: deu, Predicted: gle
+Key: deu_swissdial_vs_000082, Target: deu, Predicted: dan
+Key: deu_swissdial_vs_000114, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000006, Target: deu, Predicted: ltz
+Key: deu_swissdial_vs_000083, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000007, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000008, Target: deu, Predicted: ltz
+Key: deu_swissdial_zh_000011, Target: deu, Predicted: cym
+Key: deu_swissdial_vs_000088, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000120, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000045, Target: deu, Predicted: est
+Key: deu_swissdial_vs_000090, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000124, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000048, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000017, Target: deu, Predicted: afr
+Key: deu_swissdial_vs_000127, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000020, Target: deu, Predicted: gle
+Key: deu_swissdial_vs_000129, Target: deu, Predicted: gle
+Key: deu_swissdial_zh_000025, Target: deu, Predicted: ltz
+Key: deu_swissdial_zh_000122, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000059, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000091, Target: deu, Predicted: ltz
+Key: ell_cretan_cre_000013, Target: ell, Predicted: ukr
+Key: deu_swissdial_zh_000060, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000124, Target: deu, Predicted: afr
+Key: ell_cretan_cre_000014, Target: ell, Predicted: pus
+Key: deu_swissdial_zh_000094, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000127, Target: deu, Predicted: nld
+Key: ell_cretan_cre_000017, Target: ell, Predicted: swa
+Key: deu_swissdial_zh_000129, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000066, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000098, Target: deu, Predicted: cym
+Key: ell_cretan_cre_000020, Target: ell, Predicted: sqi
+Key: deu_swissdial_zh_000133, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000134, Target: deu, Predicted: gle
+Key: deu_swissdial_zh_000103, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000072, Target: deu, Predicted: nld
+Key: deu_swissdial_zh_000105, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000141, Target: deu, Predicted: gle
+Key: deu_swissdial_zh_000078, Target: deu, Predicted: afr
+Key: deu_swissdial_zh_000080, Target: deu, Predicted: gle
+Key: deu_swissdial_zh_000112, Target: deu, Predicted: ltz
+Key: ell_cretan_cre_000002, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000034, Target: ell, Predicted: bel
+Key: deu_swissdial_zh_000081, Target: deu, Predicted: afr
+Key: ell_cretan_cre_000003, Target: ell, Predicted: mlg
+Key: ell_cretan_cre_000035, Target: ell, Predicted: hrv
+Key: deu_swissdial_zh_000082, Target: deu, Predicted: nld
+Key: ell_cretan_cre_000004, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000036, Target: ell, Predicted: rus
+Key: ell_cretan_cre_000037, Target: ell, Predicted: bel
+Key: deu_swissdial_zh_000117, Target: deu, Predicted: afr
+Key: ell_cretan_cre_000007, Target: ell, Predicted: srp
+Key: ell_cretan_cre_000039, Target: ell, Predicted: tam
+Key: ell_cretan_cre_000040, Target: ell, Predicted: sqi
+Key: ell_cretan_cre_000041, Target: ell, Predicted: hrv
+Key: deu_swissdial_zh_000120, Target: deu, Predicted: afr
+Key: ell_cretan_cre_000011, Target: ell, Predicted: rus
+Key: ell_cretan_cre_000076, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000108, Target: ell, Predicted: ron
+Key: ell_cretan_cre_000140, Target: ell, Predicted: hrv
+Key: ell_cretan_cre_000109, Target: ell, Predicted: sqi
+Key: ell_cretan_cre_000110, Target: ell, Predicted: bul
+Key: ell_cretan_cre_000048, Target: ell, Predicted: rus
+Key: ell_cretan_cre_000112, Target: ell, Predicted: lav
+Key: ell_cretan_cre_000081, Target: ell, Predicted: slv
+Key: ell_cretan_cre_000050, Target: ell, Predicted: por
+Key: ell_cretan_cre_000115, Target: ell, Predicted: ita
+Key: ell_cretan_cre_000147, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000052, Target: ell, Predicted: azz
+Key: ell_cretan_cre_000084, Target: ell, Predicted: rus
+Key: ell_cretan_cre_000053, Target: ell, Predicted: luo
+Key: ell_cretan_cre_000117, Target: ell, Predicted: bel
+Key: ell_cretan_cre_000149, Target: ell, Predicted: ita
+Key: ell_cretan_cre_000054, Target: ell, Predicted: ita
+Key: ell_cretan_cre_000150, Target: ell, Predicted: sqi
+Key: ell_cretan_cre_000055, Target: ell, Predicted: ron
+Key: ell_cretan_cre_000088, Target: ell, Predicted: srp
+Key: ell_cretan_cre_000120, Target: ell, Predicted: por
+Key: ell_cretan_cre_000153, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000090, Target: ell, Predicted: ita
+Key: ell_cretan_cre_000059, Target: ell, Predicted: mlg
+Key: ell_cretan_cre_000091, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000155, Target: ell, Predicted: hrv
+Key: ell_cretan_cre_000092, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000156, Target: ell, Predicted: lao
+Key: ell_cretan_cre_000157, Target: ell, Predicted: ita
+Key: ell_cretan_cre_000094, Target: ell, Predicted: grn
+Key: ell_cretan_cre_000126, Target: ell, Predicted: azz
+Key: ell_cretan_cre_000064, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000128, Target: ell, Predicted: ita
+Key: ell_cretan_cre_000160, Target: ell, Predicted: hrv
+Key: ell_cretan_cre_000065, Target: ell, Predicted: hrv
+Key: ell_cretan_cre_000097, Target: ell, Predicted: rus
+Key: ell_cretan_cre_000161, Target: ell, Predicted: sqi
+Key: ell_cretan_cre_000066, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000098, Target: ell, Predicted: pus
+Key: ell_cretan_cre_000067, Target: ell, Predicted: pus
+Key: ell_cretan_cre_000099, Target: ell, Predicted: xty
+Key: ell_cretan_cre_000163, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000101, Target: ell, Predicted: grn
+Key: ell_cretan_cre_000071, Target: ell, Predicted: guj
+Key: ell_cretan_cre_000103, Target: ell, Predicted: ina
+Key: ell_cretan_cre_000167, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000168, Target: ell, Predicted: ita
+Key: ell_cretan_cre_000073, Target: ell, Predicted: sqi
+Key: ell_cretan_cre_000170, Target: ell, Predicted: sot
+Key: ell_cretan_cre_000107, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000139, Target: ell, Predicted: rus
+Key: ell_cretan_cre_000171, Target: ell, Predicted: hrv
+Key: ell_cretan_cre_000172, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000268, Target: ell, Predicted: pol
+Key: ell_cretan_cre_000237, Target: ell, Predicted: sqi
+Key: ell_cretan_cre_000174, Target: ell, Predicted: bel
+Key: ell_cretan_cre_000206, Target: ell, Predicted: aze
+Key: ell_cretan_cre_000270, Target: ell, Predicted: ron
+Key: ell_cretan_cre_000239, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000240, Target: ell, Predicted: ron
+Key: ell_cretan_cre_000177, Target: ell, Predicted: lit
+Key: ell_cretan_cre_000209, Target: ell, Predicted: azz
+Key: ell_cretan_cre_000241, Target: ell, Predicted: lit
+Key: ell_cretan_cre_000242, Target: ell, Predicted: abk
+Key: ell_cretan_cre_000274, Target: ell, Predicted: slv
+Key: ell_cretan_cre_000275, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000181, Target: ell, Predicted: azz
+Key: ell_cretan_cre_000245, Target: ell, Predicted: por
+Key: ell_cretan_cre_000248, Target: ell, Predicted: tuk
+Key: ell_cretan_cre_000185, Target: ell, Predicted: guj
+Key: ell_cretan_cre_000249, Target: ell, Predicted: guj
+Key: ell_cretan_cre_000250, Target: ell, Predicted: ron
+Key: ell_cretan_cre_000282, Target: ell, Predicted: ron
+Key: ell_cretan_cre_000187, Target: ell, Predicted: por
+Key: ell_cretan_cre_000219, Target: ell, Predicted: nep
+Key: ell_cretan_cre_000251, Target: ell, Predicted: sqi
+Key: ell_cretan_cre_000252, Target: ell, Predicted: pol
+Key: ell_cretan_cre_000189, Target: ell, Predicted: rus
+Key: ell_cretan_cre_000285, Target: ell, Predicted: bel
+Key: ell_cretan_cre_000286, Target: ell, Predicted: grn
+Key: ell_cretan_cre_000288, Target: ell, Predicted: mlg
+Key: ell_cretan_cre_000225, Target: ell, Predicted: sot
+Key: ell_cretan_cre_000226, Target: ell, Predicted: ita
+Key: ell_cretan_cre_000258, Target: ell, Predicted: lit
+Key: ell_cretan_cre_000195, Target: ell, Predicted: por
+Key: ell_cretan_cre_000227, Target: ell, Predicted: xho
+Key: ell_cretan_cre_000259, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000228, Target: ell, Predicted: bel
+Key: ell_cretan_cre_000197, Target: ell, Predicted: por
+Key: ell_cretan_cre_000261, Target: ell, Predicted: ben
+Key: ell_cretan_cre_000198, Target: ell, Predicted: hrv
+Key: ell_cretan_cre_000230, Target: ell, Predicted: sna
+Key: ell_messenian_mes_000005, Target: ell, Predicted: heb
+Key: ell_cretan_cre_000200, Target: ell, Predicted: swa
+Key: ell_cretan_cre_000232, Target: ell, Predicted: hrv
+Key: ell_cretan_cre_000264, Target: ell, Predicted: sun
+Key: ell_cretan_cre_000265, Target: ell, Predicted: ces
+Key: ell_cretan_cre_000202, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000266, Target: ell, Predicted: ukr
+Key: ell_cretan_cre_000203, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000235, Target: ell, Predicted: mkd
+Key: ell_cretan_cre_000267, Target: ell, Predicted: azz
+Key: ell_messenian_mes_000009, Target: ell, Predicted: cym
+Key: ell_messenian_mes_000011, Target: ell, Predicted: cat
+Key: ell_messenian_mes_000043, Target: ell, Predicted: hrv
+Key: ell_messenian_mes_000077, Target: ell, Predicted: ces
+Key: ell_messenian_mes_000079, Target: ell, Predicted: cym
+Key: ell_messenian_mes_000112, Target: ell, Predicted: mkd
+Key: ell_messenian_mes_000085, Target: ell, Predicted: cym
+Key: ell_messenian_mes_000087, Target: ell, Predicted: heb
+Key: ell_messenian_mes_000056, Target: ell, Predicted: nno
+Key: ell_messenian_mes_000089, Target: ell, Predicted: sqi
+Key: ell_messenian_mes_000062, Target: ell, Predicted: cym
+Key: ell_messenian_mes_000099, Target: ell, Predicted: bul
+Key: ell_messenian_mes_000136, Target: ell, Predicted: cym
+Key: ell_messenian_mes_000139, Target: ell, Predicted: hrv
+Key: ell_messenian_mes_000141, Target: ell, Predicted: mkd
+Key: ell_messenian_mes_000143, Target: ell, Predicted: slv
+Key: eng_globe_aus_000018, Target: eng, Predicted: sqi
+Key: ell_messenian_mes_000155, Target: ell, Predicted: nno
+Key: ell_messenian_mes_000156, Target: ell, Predicted: heb
+Key: eng_globe_aus_000000, Target: eng, Predicted: gle
+Key: ell_messenian_mes_000161, Target: ell, Predicted: ces
+Key: ell_messenian_mes_000164, Target: ell, Predicted: cym
+Key: eng_globe_aus_000143, Target: eng, Predicted: tam
+Key: eng_globe_aus_000082, Target: eng, Predicted: sqi
+Key: eng_globe_aus_000118, Target: eng, Predicted: tgl
+Key: eng_globe_bre_000034, Target: eng, Predicted: hun
+Key: eng_globe_bre_000100, Target: eng, Predicted: cym
+Key: eng_globe_bre_000133, Target: eng, Predicted: nor
+Key: eng_globe_bre_000116, Target: eng, Predicted: gle
+Key: eng_globe_bre_000124, Target: eng, Predicted: azz
+Key: eng_globe_bre_000130, Target: eng, Predicted: cym
+Key: eng_globe_bre_000099, Target: eng, Predicted: deu
+Key: eng_globe_can_000087, Target: eng, Predicted: deu
+Key: eng_globe_can_000063, Target: eng, Predicted: kor
+Key: eng_globe_can_000098, Target: eng, Predicted: glv
+Key: eng_globe_fil_000016, Target: eng, Predicted: ces
+Key: eng_globe_fil_000000, Target: eng, Predicted: gle
+Key: eng_globe_fil_000070, Target: eng, Predicted: nld
+Key: eng_globe_fil_000008, Target: eng, Predicted: glv
+Key: eng_globe_gle_000007, Target: eng, Predicted: cym
+Key: eng_globe_fil_000146, Target: eng, Predicted: tgl
+Key: eng_globe_gle_000030, Target: eng, Predicted: snd
+Key: eng_globe_gle_000069, Target: eng, Predicted: gle
+Key: eng_globe_gle_000104, Target: eng, Predicted: gle
+Key: eng_globe_gle_000137, Target: eng, Predicted: gle
+Key: eng_globe_gle_000055, Target: eng, Predicted: gle
+Key: eng_globe_gle_000087, Target: eng, Predicted: cym
+Key: eng_globe_gle_000154, Target: eng, Predicted: tel
+Key: eng_globe_gle_000126, Target: eng, Predicted: gle
+Key: eng_globe_gle_000167, Target: eng, Predicted: hin
+Key: eng_globe_nze_000066, Target: eng, Predicted: mri
+Key: eng_globe_nze_000108, Target: eng, Predicted: urd
+Key: eng_globe_sae_000008, Target: eng, Predicted: ben
+Key: eng_globe_sae_000011, Target: eng, Predicted: tam
+Key: eng_globe_sae_000015, Target: eng, Predicted: urd
+Key: eng_globe_sae_000062, Target: eng, Predicted: ben
+Key: eng_globe_sae_000063, Target: eng, Predicted: xho
+Key: eng_globe_sae_000033, Target: eng, Predicted: tel
+Key: eng_globe_sae_000067, Target: eng, Predicted: tgl
+Key: eng_globe_sae_000168, Target: eng, Predicted: cym
+Key: eng_globe_sae_000169, Target: eng, Predicted: tam
+Key: eng_globe_sae_000140, Target: eng, Predicted: nep
+Key: eng_globe_sae_000109, Target: eng, Predicted: glv
+Key: eng_globe_sae_000143, Target: eng, Predicted: gle
+Key: eng_globe_sae_000115, Target: eng, Predicted: tel
+Key: eng_globe_sco_000009, Target: eng, Predicted: cym
+Key: eng_globe_sae_000160, Target: eng, Predicted: ltz
+Key: eng_globe_sae_000161, Target: eng, Predicted: tam
+Key: eng_globe_sae_000165, Target: eng, Predicted: deu
+Key: eng_globe_sco_000093, Target: eng, Predicted: glv
+Key: eng_globe_sco_000062, Target: eng, Predicted: cym
+Key: eng_globe_sco_000140, Target: eng, Predicted: nld
+Key: eng_globe_sco_000078, Target: eng, Predicted: cym
+Key: eng_globe_sco_000110, Target: eng, Predicted: gle
+Key: eng_globe_sco_000047, Target: eng, Predicted: cym
+Key: eng_globe_sco_000144, Target: eng, Predicted: gle
+Key: eng_globe_sco_000053, Target: eng, Predicted: gle
+Key: eng_globe_use_000098, Target: eng, Predicted: nep
+Key: eng_globe_use_000038, Target: eng, Predicted: mri
+Key: eng_globe_use_000102, Target: eng, Predicted: deu
+Key: eng_globe_use_000044, Target: eng, Predicted: ltz
+Key: eng_globe_use_000076, Target: eng, Predicted: sot
+Key: eng_globe_use_000113, Target: eng, Predicted: glv
+Key: eng_globe_use_000115, Target: eng, Predicted: oci
+Key: eng_globe_use_000020, Target: eng, Predicted: cym
+Key: eng_globe_use_000023, Target: eng, Predicted: msa
+Key: eng_globe_use_000090, Target: eng, Predicted: cym
+Key: eng_globe_use_000059, Target: eng, Predicted: deu
+Key: eng_l2arctic_ara_000029, Target: eng, Predicted: sqi
+Key: eng_globe_use_000176, Target: eng, Predicted: cym
+Key: eng_l2arctic_ara_000002, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000003, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000069, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000038, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000070, Target: eng, Predicted: fao
+Key: eng_l2arctic_ara_000040, Target: eng, Predicted: glv
+Key: eng_l2arctic_ara_000077, Target: eng, Predicted: ces
+Key: eng_l2arctic_ara_000046, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000079, Target: eng, Predicted: fra
+Key: eng_l2arctic_ara_000080, Target: eng, Predicted: nld
+Key: eng_l2arctic_ara_000146, Target: eng, Predicted: pus
+Key: eng_l2arctic_cmn_000011, Target: eng, Predicted: cmn
+Key: eng_l2arctic_ara_000122, Target: eng, Predicted: hye
+Key: eng_l2arctic_ara_000123, Target: eng, Predicted: kat
+Key: eng_l2arctic_ara_000127, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000128, Target: eng, Predicted: som
+Key: eng_l2arctic_ara_000160, Target: eng, Predicted: deu
+Key: eng_l2arctic_ara_000129, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000130, Target: eng, Predicted: tgk
+Key: eng_l2arctic_ara_000164, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000133, Target: eng, Predicted: nld
+Key: eng_l2arctic_ara_000135, Target: eng, Predicted: hun
+Key: eng_l2arctic_ara_000169, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000108, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000140, Target: eng, Predicted: ara
+Key: eng_l2arctic_ara_000109, Target: eng, Predicted: ckb
+Key: eng_l2arctic_cmn_000135, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000138, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000107, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000139, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000108, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000141, Target: eng, Predicted: lao
+Key: eng_l2arctic_cmn_000078, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000047, Target: eng, Predicted: ron
+Key: eng_l2arctic_cmn_000113, Target: eng, Predicted: cmn
+Key: eng_l2arctic_hin_000001, Target: eng, Predicted: hin
+Key: eng_l2arctic_cmn_000118, Target: eng, Predicted: por
+Key: eng_l2arctic_cmn_000125, Target: eng, Predicted: mya
+Key: eng_l2arctic_hin_000010, Target: eng, Predicted: hin
+Key: eng_l2arctic_cmn_000094, Target: eng, Predicted: hun
+Key: eng_l2arctic_cmn_000126, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000097, Target: eng, Predicted: bod
+Key: eng_l2arctic_cmn_000100, Target: eng, Predicted: cmn
+Key: eng_l2arctic_cmn_000133, Target: eng, Predicted: fas
+Key: eng_l2arctic_hin_000119, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000025, Target: eng, Predicted: urd
+Key: eng_l2arctic_hin_000122, Target: eng, Predicted: pus
+Key: eng_l2arctic_hin_000123, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000061, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000126, Target: eng, Predicted: ben
+Key: eng_l2arctic_hin_000127, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000096, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000128, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000131, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000132, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000069, Target: eng, Predicted: deu
+Key: eng_l2arctic_hin_000101, Target: eng, Predicted: hin
+Key: eng_l2arctic_hin_000133, Target: eng, Predicted: kan
+Key: eng_l2arctic_hin_000102, Target: eng, Predicted: pan
+Key: eng_l2arctic_hin_000104, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000106, Target: eng, Predicted: mal
+Key: eng_l2arctic_hin_000108, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000140, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000141, Target: eng, Predicted: mar
+Key: eng_l2arctic_hin_000110, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000144, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000147, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000149, Target: eng, Predicted: guj
+Key: eng_l2arctic_hin_000152, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000154, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000188, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000157, Target: eng, Predicted: mal
+Key: eng_l2arctic_hin_000159, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000191, Target: eng, Predicted: ben
+Key: eng_l2arctic_hin_000193, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000163, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000195, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000164, Target: eng, Predicted: nep
+Key: eng_l2arctic_hin_000196, Target: eng, Predicted: guj
+Key: eng_l2arctic_hin_000166, Target: eng, Predicted: slv
+Key: eng_l2arctic_kor_000022, Target: eng, Predicted: ltz
+Key: eng_l2arctic_hin_000167, Target: eng, Predicted: tam
+Key: eng_l2arctic_hin_000199, Target: eng, Predicted: urd
+Key: eng_l2arctic_hin_000169, Target: eng, Predicted: cym
+Key: eng_l2arctic_hin_000203, Target: eng, Predicted: pan
+Key: eng_l2arctic_hin_000205, Target: eng, Predicted: tel
+Key: eng_l2arctic_hin_000175, Target: eng, Predicted: pan
+Key: eng_l2arctic_kor_000069, Target: eng, Predicted: dan
+Key: eng_l2arctic_kor_000165, Target: eng, Predicted: slv
+Key: eng_l2arctic_kor_000142, Target: eng, Predicted: xho
+Key: eng_l2arctic_kor_000118, Target: eng, Predicted: bod
+Key: eng_l2arctic_kor_000157, Target: eng, Predicted: hun
+Key: eng_l2arctic_spa_000014, Target: eng, Predicted: heb
+Key: eng_l2arctic_spa_000067, Target: eng, Predicted: ita
+Key: eng_l2arctic_spa_000036, Target: eng, Predicted: azz
+Key: eng_l2arctic_spa_000102, Target: eng, Predicted: ron
+Key: eng_l2arctic_spa_000149, Target: eng, Predicted: fin
+Key: eng_l2arctic_vie_000034, Target: eng, Predicted: deu
+Key: eng_l2arctic_vie_000004, Target: eng, Predicted: tgl
+Key: eng_l2arctic_vie_000069, Target: eng, Predicted: mri
+Key: eng_l2arctic_vie_000106, Target: eng, Predicted: lao
+Key: eng_l2arctic_vie_000044, Target: eng, Predicted: xho
+Key: eng_l2arctic_vie_000109, Target: eng, Predicted: cym
+Key: eng_l2arctic_vie_000047, Target: eng, Predicted: xho
+Key: eng_l2arctic_vie_000016, Target: eng, Predicted: lat
+Key: eng_l2arctic_vie_000022, Target: eng, Predicted: pus
+Key: eng_openslr83_nor_000017, Target: eng, Predicted: gle
+Key: eng_openslr83_mid_000076, Target: eng, Predicted: glv
+Key: eng_openslr83_mid_000078, Target: eng, Predicted: cym
+Key: eng_openslr83_nor_000031, Target: eng, Predicted: cym
+Key: eng_openslr83_nor_000069, Target: eng, Predicted: cym
+Key: eng_openslr83_nor_000070, Target: eng, Predicted: cym
+Key: eng_openslr83_nor_000074, Target: eng, Predicted: cym
+Key: eng_openslr83_nor_000050, Target: eng, Predicted: cym
+Key: eng_openslr83_nor_000083, Target: eng, Predicted: gle
+Key: eng_openslr83_nor_000092, Target: eng, Predicted: cym
+Key: eng_openslr83_sou_000065, Target: eng, Predicted: cym
+Key: eng_openslr83_sco_000086, Target: eng, Predicted: gle
+Key: eng_openslr83_sco_000091, Target: eng, Predicted: gle
+Key: eng_openslr83_wel_000002, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000003, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000035, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000067, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000068, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000037, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000069, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000006, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000038, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000040, Target: eng, Predicted: dan
+Key: eng_openslr83_wel_000072, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000009, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000073, Target: eng, Predicted: gle
+Key: eng_openslr83_wel_000010, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000042, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000074, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000011, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000075, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000012, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000076, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000013, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000077, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000014, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000046, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000015, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000079, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000080, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000081, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000082, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000052, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000084, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000053, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000085, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000022, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000086, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000087, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000056, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000088, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000057, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000026, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000058, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000090, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000060, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000092, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000031, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000063, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000032, Target: eng, Predicted: glv
+Key: eng_openslr83_wel_000065, Target: eng, Predicted: cym
+Key: eng_openslr83_wel_000034, Target: eng, Predicted: gle
+Key: eng_openslr83_wel_000066, Target: eng, Predicted: cym
+Key: eng_voxpopuli_est_000017, Target: eng, Predicted: deu
+Key: eng_voxpopuli_est_000008, Target: eng, Predicted: est
+Key: eng_voxpopuli_hun_000015, Target: eng, Predicted: hun
+Key: eng_voxpopuli_hun_000057, Target: eng, Predicted: hun
+Key: eng_voxpopuli_pol_000021, Target: eng, Predicted: pol
+Key: eng_voxpopuli_pol_000024, Target: eng, Predicted: pol
+Key: eng_voxpopuli_nld_000034, Target: eng, Predicted: nld
+Key: eng_voxpopuli_pol_000038, Target: eng, Predicted: pol
+Key: eng_voxpopuli_ron_000015, Target: eng, Predicted: ron
+Key: eng_voxpopuli_ron_000017, Target: eng, Predicted: ron
+Key: guj_ms_speech_guj_000038, Target: guj, Predicted: mar
+Key: spa_openslr_spa_arg_000015, Target: spa, Predicted: por
+Key: spa_openslr_spa_arg_000018, Target: spa, Predicted: mlt
+Key: guj_ms_speech_guj_000083, Target: guj, Predicted: hin
+Key: spa_openslr_spa_arg_000022, Target: spa, Predicted: est
+Key: spa_openslr_spa_arg_000062, Target: spa, Predicted: hau
+Key: spa_openslr_spa_arg_000031, Target: spa, Predicted: isl
+Key: spa_openslr_spa_arg_000003, Target: spa, Predicted: sot
+Key: spa_openslr_spa_arg_000006, Target: spa, Predicted: ita
+Key: spa_openslr_spa_arg_000079, Target: spa, Predicted: cym
+Key: spa_openslr_spa_chi_000049, Target: spa, Predicted: ell
+Key: spa_openslr_spa_arg_000114, Target: spa, Predicted: eus
+Key: spa_openslr_spa_chi_000022, Target: spa, Predicted: grn
+Key: spa_openslr_spa_chi_000037, Target: spa, Predicted: ita
+Key: spa_openslr_spa_chi_000076, Target: spa, Predicted: sot
+Key: spa_openslr_spa_col_000001, Target: spa, Predicted: por
+Key: spa_openslr_spa_col_000007, Target: spa, Predicted: ita
+Key: spa_openslr_spa_col_000013, Target: spa, Predicted: ita
+Key: spa_openslr_spa_col_000081, Target: spa, Predicted: ron
+Key: spa_openslr_spa_col_000056, Target: spa, Predicted: ita
+Key: spa_openslr_spa_col_000099, Target: spa, Predicted: epo
+Key: spa_openslr_spa_per_000018, Target: spa, Predicted: glg
+Key: spa_openslr_spa_per_000063, Target: spa, Predicted: ina
+Key: spa_openslr_spa_per_000033, Target: spa, Predicted: ina
+Key: spa_openslr_spa_per_000065, Target: spa, Predicted: glg
+Key: spa_openslr_spa_per_000035, Target: spa, Predicted: ita
+Key: spa_openslr_spa_pue_000060, Target: spa, Predicted: ita
+Key: spa_openslr_spa_pue_000094, Target: spa, Predicted: eus
+Key: spa_openslr_spa_pue_000003, Target: spa, Predicted: ita
+Key: spa_openslr_spa_pue_000071, Target: spa, Predicted: ita
+Key: spa_openslr_spa_pue_000014, Target: spa, Predicted: gug
+Key: spa_openslr_spa_pue_000051, Target: spa, Predicted: ron
+Key: spa_openslr_spa_ven_000112, Target: spa, Predicted: eus
+Key: spa_openslr_spa_ven_000116, Target: spa, Predicted: isl
+Key: spa_openslr_spa_ven_000118, Target: spa, Predicted: ina
+Key: tel_ms_speech_tel_000014, Target: tel, Predicted: mal

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang/lid_inference_test.log

@@ -0,0 +1,302 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Mon Jun  2 02:17:42 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue04] 2025-06-02 02:18:18,542 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue04] 2025-06-02 02:18:35,714 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamConditionModel(
+  (frontend): S3prlFrontendCondition(
+    (upstream): S3PRLUpstreamCondition(
+      (upstream): UpstreamExpertCondition(
+        (model): Wav2Vec2ModelCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+            (aamsoftmax_weight): ParameterDict()
+            (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True)
+            (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec(
+              (ce): CrossEntropyLoss()
+              (lang2vec_head): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (lang2vec_loss): MSELoss()
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamConditionModel
+    Total Number of model parameters: 977.14 M
+    Number of trainable parameters: 977.14 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
+  warnings.warn(_create_warning_msg(
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue04] 2025-06-02 02:18:36,278 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang/lids0
+[gpue04] 2025-06-02 02:19:16,352 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue04] 2025-06-02 02:19:50,349 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1
+[gpue04] 2025-06-02 02:20:29,160 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2
+[gpue04] 2025-06-02 02:21:04,869 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3
+[gpue04] 2025-06-02 02:21:41,255 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4
+[gpue04] 2025-06-02 02:22:12,467 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5
+[gpue04] 2025-06-02 02:22:52,319 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6
+[gpue04] 2025-06-02 02:23:31,893 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7
+[gpue04] 2025-06-02 02:24:09,283 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8
+[gpue04] 2025-06-02 02:24:49,938 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9
+[gpue04] 2025-06-02 02:25:28,601 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10
+[gpue04] 2025-06-02 02:26:08,097 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11
+[gpue04] 2025-06-02 02:26:47,335 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12
+[gpue04] 2025-06-02 02:27:25,789 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13
+[gpue04] 2025-06-02 02:27:56,679 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14
+[gpue04] 2025-06-02 02:28:40,096 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15
+[gpue04] 2025-06-02 02:29:19,221 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 16
+[gpue04] 2025-06-02 02:29:55,785 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 17
+[gpue04] 2025-06-02 02:30:34,585 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 18
+[gpue04] 2025-06-02 02:31:08,302 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 19
+[gpue04] 2025-06-02 02:31:41,628 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 20
+[gpue04] 2025-06-02 02:32:15,682 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 21
+[gpue04] 2025-06-02 02:32:50,538 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 22
+[gpue04] 2025-06-02 02:33:12,317 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True
+[gpue04] 2025-06-02 02:33:12,318 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=931 threads=1
+# Ended (code 0) at Mon Jun  2 02:33:13 CDT 2025, elapsed time 931 seconds

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang/lid_inference_test.log

@@ -0,0 +1,280 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Mon Jun  2 00:34:26 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue04] 2025-06-02 00:34:58,673 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue04] 2025-06-02 00:35:15,310 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamConditionModel(
+  (frontend): S3prlFrontendCondition(
+    (upstream): S3PRLUpstreamCondition(
+      (upstream): UpstreamExpertCondition(
+        (model): Wav2Vec2ModelCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+            (aamsoftmax_weight): ParameterDict()
+            (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True)
+            (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec(
+              (ce): CrossEntropyLoss()
+              (lang2vec_head): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (lang2vec_loss): MSELoss()
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamConditionModel
+    Total Number of model parameters: 977.14 M
+    Number of trainable parameters: 977.14 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
+  warnings.warn(_create_warning_msg(
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue04] 2025-06-02 00:35:15,911 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang/lids0
+[gpue04] 2025-06-02 00:36:12,151 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue04] 2025-06-02 00:36:44,813 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True
+[gpue04] 2025-06-02 00:36:44,813 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=139 threads=1
+# Ended (code 0) at Mon Jun  2 00:36:45 CDT 2025, elapsed time 139 seconds

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang/results

@@ -0,0 +1,126 @@
+Accuracy: 94.41%
+Macro Accuracy: 93.10%
+Accuracy per Language:
+slv: 100.00%
+aze: 91.18%
+cmn: 95.65%
+srp: 71.43%
+hun: 100.00%
+fas: 89.00%
+hye: 96.00%
+urd: 80.77%
+spa: 92.73%
+est: 94.12%
+tur: 98.61%
+rus: 100.00%
+ita: 100.00%
+ara: 95.00%
+nor: 54.17%
+lav: 98.95%
+ukr: 100.00%
+swe: 96.00%
+deu: 93.90%
+ell: 80.00%
+isl: 100.00%
+nno: 100.00%
+pol: 100.00%
+hrv: 75.00%
+dan: 93.94%
+fra: 97.00%
+nld: 95.00%
+mkd: 100.00%
+jpn: 97.62%
+por: 100.00%
+fin: 97.85%
+eng: 96.25%
+lit: 92.31%
+Key: ara_cwvnYGInNNg__U__S229---0661.830-0670.560.wav, Target: ara, Predicted: heb
+Key: aze_3UUShvAQxQY__U__S199---1315.800-1322.250.wav, Target: aze, Predicted: tur
+Key: aze_3qOGhbHQuAc__U__S157---1061.380-1066.120.wav, Target: aze, Predicted: tur
+Key: ara_AfS6C1PXAdQ__U__S20---0104.730-0111.410.wav, Target: ara, Predicted: hau
+Key: ara_TPWwuy20K_c__U__S70---0466.380-0472.600.wav, Target: ara, Predicted: hau
+Key: ara_XplxxijLuFI__U__S0---0372.560-0375.230.wav, Target: ara, Predicted: heb
+Key: aze_bYKK1m78ecE__U__S91---0592.500-0596.130.wav, Target: aze, Predicted: fao
+Key: ara_tl39W93P0r4__U__S32---0282.970-0286.530.wav, Target: ara, Predicted: dan
+Key: dan_Nyl6CuW6Qfk__U__S26---0557.690-0560.120.wav, Target: dan, Predicted: nor
+Key: dan_ONZC1wL5hBw__U__S100---1407.470-1417.260.wav, Target: dan, Predicted: nno
+Key: cmn_ZUzq_TIfYL4__U__S39---0442.690-0454.380.wav, Target: cmn, Predicted: yue
+Key: dan_SbE2FKexCW4__U__S62---0546.280-0551.260.wav, Target: dan, Predicted: isl
+Key: dan_E3vuA0Mqk1Q__U__S13---0072.140-0083.530.wav, Target: dan, Predicted: nno
+Key: dan_ZZD1qu4ScPg__U__S14---0166.700-0176.010.wav, Target: dan, Predicted: hat
+Key: dan_yEEcGssW0Qg__U__S112---1016.050-1020.110.wav, Target: dan, Predicted: deu
+Key: eng_4y7p9R2No-4__U__S12---0266.390-0268.460.wav, Target: eng, Predicted: gle
+Key: deu_4zCzyVjLkcc__U__S0---0123.750-0127.540.wav, Target: deu, Predicted: ltz
+Key: deu_8L3k8XNTtNA__U__S100---2689.380-2692.180.wav, Target: deu, Predicted: fin
+Key: deu_9O2haSYzftE__U__S0---0000.000-0004.200.wav, Target: deu, Predicted: yid
+Key: deu_cMZO2zXTBv8__U__S100---0341.910-0344.350.wav, Target: deu, Predicted: yid
+Key: ell_bw_mDLVdgtY__U__S18---0119.750-0127.200.wav, Target: ell, Predicted: isl
+Key: deu_eyZqRcgGkiY__U__S126---1155.890-1162.390.wav, Target: deu, Predicted: nor
+Key: eng_K977aQQpAVk__U__S106---0393.230-0397.100.wav, Target: eng, Predicted: cym
+Key: est_EtWRBtavckY__U__S116---1906.220-1908.810.wav, Target: est, Predicted: fin
+Key: eng_eQXHc-tJMXM__U__S11---1066.230-1077.360.wav, Target: eng, Predicted: cym
+Key: est_gTl2GSJBxNw__U__S0---0000.000-0008.420.wav, Target: est, Predicted: tur
+Key: est_5gWpxiFOouQ__U__S2---1635.950-1646.620.wav, Target: est, Predicted: tel
+Key: est_7vZIuc9qumg__U__S21---0145.690-0153.320.wav, Target: est, Predicted: dan
+Key: est_E05LlgvSMg0__U__S156---1171.030-1172.780.wav, Target: est, Predicted: fin
+Key: fas_SMcjja_krx4__U__S2---0012.190-0021.730.wav, Target: fas, Predicted: tgk
+Key: fas_4sboRMmC2TM__U__S212---1293.790-1307.370.wav, Target: fas, Predicted: tgk
+Key: fas_XUGZwtXgvRA__U__S154---0993.540-0997.340.wav, Target: fas, Predicted: san
+Key: fas_9k1oVW4Ynyw__U__S15---0097.430-0101.630.wav, Target: fas, Predicted: sqi
+Key: fas_nPts67VQKRQ__U__S250---1629.010-1632.750.wav, Target: fas, Predicted: hat
+Key: fas_EjSRRddYuc4__U__S58---0355.980-0359.590.wav, Target: fas, Predicted: lat
+Key: fas_pt166R7v8kU__U__S13---0267.910-0272.370.wav, Target: fas, Predicted: tgk
+Key: fin_C4H2GlJRkNU__U__S100---1604.910-1610.210.wav, Target: fin, Predicted: est
+Key: fas_x_Di4cq4ixM__U__S100---1353.580-1358.390.wav, Target: fas, Predicted: pus
+Key: fas_gLoBPMrad3E__U__S14---0097.650-0102.010.wav, Target: fas, Predicted: yid
+Key: fas_zZCjOs-WwKo__U__S195---1357.430-1377.010.wav, Target: fas, Predicted: aze
+Key: fas_QYwCDYVxjpo__U__S68---0428.220-0432.740.wav, Target: fas, Predicted: pus
+Key: fin_S_VWbBtBey4__U__S0---0308.380-0310.650.wav, Target: fin, Predicted: glv
+Key: fra_SLfpp704KI8__U__S57---0368.470-0372.910.wav, Target: fra, Predicted: rus
+Key: fra_Lo_JX-8KHEw__U__S151---0284.430-0299.020.wav, Target: fra, Predicted: lin
+Key: hrv_Jntmbw5_vOI__U__S291---0379.300-0383.970.wav, Target: hrv, Predicted: srp
+Key: fra_jjEvNgbuptE__U__S103---0990.080-0997.340.wav, Target: fra, Predicted: hat
+Key: hye_PcLE4N63O9M__U__S352---2333.340-2337.540.wav, Target: hye, Predicted: yid
+Key: hye_Qmo3P38Ytek__U__S32---0245.460-0249.320.wav, Target: hye, Predicted: jav
+Key: hye_qkMM0rYsa0c__U__S276---1611.690-1615.350.wav, Target: hye, Predicted: sqi
+Key: hye_um6xT5Gjgus__U__S194---1224.460-1234.130.wav, Target: hye, Predicted: lat
+Key: jpn_rQPhM6wNQwc__U__S47---0317.270-0323.120.wav, Target: jpn, Predicted: est
+Key: lit_3svAywrL0_I__U__S149---0461.370-0464.980.wav, Target: lit, Predicted: por
+Key: nld_0LhAXOxz-JU__U__S32---0243.280-0247.880.wav, Target: nld, Predicted: afr
+Key: nld_0LhAXOxz-JU__U__S396---2475.670-2488.950.wav, Target: nld, Predicted: afr
+Key: nld_2C5HehL-Fx0__U__S101---1125.890-1131.720.wav, Target: nld, Predicted: ltz
+Key: lav_DWPBBIdz0Mo__U__S52---0339.380-0356.600.wav, Target: lav, Predicted: ukr
+Key: nld_QflBX7-rF9c__U__S106---0919.840-0926.610.wav, Target: nld, Predicted: afr
+Key: nor_HW_49WuFloM__U__S106---0621.590-0626.130.wav, Target: nor, Predicted: nno
+Key: nor_I1vUI8va8Yc__U__S49---0294.940-0302.560.wav, Target: nor, Predicted: nno
+Key: nld_7AZTxaq_37U__U__S29---0226.530-0237.250.wav, Target: nld, Predicted: afr
+Key: nor_UxHL_uql05E__U__S118---0587.340-0598.790.wav, Target: nor, Predicted: nno
+Key: nor_XC4Ffj9XDls__U__S105---0636.770-0655.460.wav, Target: nor, Predicted: nno
+Key: nor_tV3Le8SUz_0__U__S276---1831.870-1841.550.wav, Target: nor, Predicted: nno
+Key: nor_xVNA15ifyIw__U__S494---0311.220-0317.160.wav, Target: nor, Predicted: nno
+Key: nor_ySVkmT8SgNM__U__S345---2245.790-2255.930.wav, Target: nor, Predicted: nno
+Key: nor_0eQvHBz2Zb0__U__S0---0000.000-0018.430.wav, Target: nor, Predicted: nno
+Key: nor_1KFP5wVtthQ__U__S130---0511.650-0521.160.wav, Target: nor, Predicted: nno
+Key: nor_41P9Uue3YbQ__U__S38---0255.810-0264.280.wav, Target: nor, Predicted: nno
+Key: nor_97e9pEtHAxg__U__S32---0201.830-0210.250.wav, Target: nor, Predicted: nno
+Key: spa_BApoyHcbdls__U__S286---1705.860-1722.550.wav, Target: spa, Predicted: ast
+Key: swe_CizHFWTDSnU__U__S113---0867.420-0878.560.wav, Target: swe, Predicted: nor
+Key: spa_z5b-CjOOhK8__U__S251---1701.420-1708.280.wav, Target: spa, Predicted: glg
+Key: srp_8dvIaAOLlGA__U__S216---1326.410-1335.490.wav, Target: srp, Predicted: hrv
+Key: spa_UYBcNrx8kvQ__U__S186---2292.670-2299.590.wav, Target: spa, Predicted: kor
+Key: srp_rkQhxxO5Qt4__U__S109---0820.610-0826.190.wav, Target: srp, Predicted: bos
+Key: spa_Y0mzNQqBR3A__U__S151---1160.320-1166.480.wav, Target: spa, Predicted: glg
+Key: swe_0x4xb4AaTy0__U__S0---0301.450-0319.780.wav, Target: swe, Predicted: nno
+Key: tur_4C-efpD-DlM__U__S7---0050.890-0055.080.wav, Target: tur, Predicted: war
+Key: swe_ilhngbAuxvs__U__S14---2441.740-2445.720.wav, Target: swe, Predicted: nno
+Key: swe_wMAAiJhj0VA__U__S100---0564.840-0568.420.wav, Target: swe, Predicted: nno
+Key: urd_o3awRytwrUY__U__S1---0290.850-0306.430.wav, Target: urd, Predicted: fas
+Key: urd_pYId2x4cutY__U__S151---1539.990-1542.820.wav, Target: urd, Predicted: hin
+Key: urd_J7RizO2mvm4__U__S3---0042.600-0051.180.wav, Target: urd, Predicted: san
+Key: urd_ySjOb5uaA-U__U__S107---0336.690-0353.110.wav, Target: urd, Predicted: cym
+Key: urd_N59t4A1mxfA__U__S101---0715.390-0720.460.wav, Target: urd, Predicted: snd
+Key: urd_Tj2pngm_vuA__U__S1---0070.400-0089.660.wav, Target: urd, Predicted: hin
+Key: urd_U_h8Bgywxrc__U__S0---0222.420-0227.610.wav, Target: urd, Predicted: hin
+Key: urd_eTfyAm6CFB0__U__S25---0439.860-0446.680.wav, Target: urd, Predicted: hin
+Key: urd_8VDrhDx37OA__U__S12---0094.320-0106.080.wav, Target: urd, Predicted: hin
+Key: urd_n3l7PavcOFk__U__S0---0379.380-0397.930.wav, Target: urd, Predicted: hin

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang/lid_inference_test.log

@@ -0,0 +1,356 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Mon Jun  2 00:54:21 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue04] 2025-06-02 00:54:40,331 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue04] 2025-06-02 00:54:51,981 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamConditionModel(
+  (frontend): S3prlFrontendCondition(
+    (upstream): S3PRLUpstreamCondition(
+      (upstream): UpstreamExpertCondition(
+        (model): Wav2Vec2ModelCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+            (aamsoftmax_weight): ParameterDict()
+            (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True)
+            (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec(
+              (ce): CrossEntropyLoss()
+              (lang2vec_head): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (lang2vec_loss): MSELoss()
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamConditionModel
+    Total Number of model parameters: 977.14 M
+    Number of trainable parameters: 977.14 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
+  warnings.warn(_create_warning_msg(
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue04] 2025-06-02 00:54:52,516 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang/lids0
+[gpue04] 2025-06-02 00:55:50,135 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue04] 2025-06-02 00:56:51,199 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1
+[gpue04] 2025-06-02 00:57:39,967 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2
+[gpue04] 2025-06-02 00:58:38,305 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3
+[gpue04] 2025-06-02 00:59:50,779 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4
+[gpue04] 2025-06-02 01:00:54,111 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5
+[gpue04] 2025-06-02 01:01:48,017 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6
+[gpue04] 2025-06-02 01:02:41,062 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7
+[gpue04] 2025-06-02 01:03:46,378 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8
+[gpue04] 2025-06-02 01:04:44,682 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9
+[gpue04] 2025-06-02 01:05:39,283 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10
+[gpue04] 2025-06-02 01:06:43,893 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11
+[gpue04] 2025-06-02 01:07:51,015 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12
+[gpue04] 2025-06-02 01:08:54,020 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13
+[gpue04] 2025-06-02 01:09:52,678 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14
+[gpue04] 2025-06-02 01:10:43,497 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15
+[gpue04] 2025-06-02 01:11:46,070 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 16
+[gpue04] 2025-06-02 01:13:04,592 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 17
+[gpue04] 2025-06-02 01:14:19,368 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 18
+[gpue04] 2025-06-02 01:15:13,278 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 19
+[gpue04] 2025-06-02 01:16:21,130 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 20
+[gpue04] 2025-06-02 01:17:22,122 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 21
+[gpue04] 2025-06-02 01:18:11,762 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 22
+[gpue04] 2025-06-02 01:19:30,749 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 23
+[gpue04] 2025-06-02 01:20:24,887 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 24
+[gpue04] 2025-06-02 01:21:15,131 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 25
+[gpue04] 2025-06-02 01:22:09,711 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 26
+[gpue04] 2025-06-02 01:23:05,444 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 27
+[gpue04] 2025-06-02 01:24:41,530 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 28
+[gpue04] 2025-06-02 01:25:45,696 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 29
+[gpue04] 2025-06-02 01:26:55,280 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 30
+[gpue04] 2025-06-02 01:27:56,405 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 31
+[gpue04] 2025-06-02 01:29:15,643 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 32
+[gpue04] 2025-06-02 01:30:14,980 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 33
+[gpue04] 2025-06-02 01:31:18,218 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 34
+[gpue04] 2025-06-02 01:32:29,009 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 35
+[gpue04] 2025-06-02 01:33:37,784 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 36
+[gpue04] 2025-06-02 01:34:36,080 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 37
+[gpue04] 2025-06-02 01:35:38,726 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 38
+[gpue04] 2025-06-02 01:36:47,523 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 39
+[gpue04] 2025-06-02 01:37:52,504 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 40
+[gpue04] 2025-06-02 01:38:45,251 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 41
+[gpue04] 2025-06-02 01:40:00,452 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 42
+[gpue04] 2025-06-02 01:41:11,257 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 43
+[gpue04] 2025-06-02 01:42:19,878 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 44
+[gpue04] 2025-06-02 01:43:18,828 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 45
+[gpue04] 2025-06-02 01:44:21,603 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 46
+[gpue04] 2025-06-02 01:45:16,599 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 47
+[gpue04] 2025-06-02 01:47:00,696 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 48
+[gpue04] 2025-06-02 01:48:00,085 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 49
+[gpue04] 2025-06-02 01:49:05,199 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 50
+[gpue04] 2025-06-02 01:49:59,112 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 51
+[gpue04] 2025-06-02 01:51:30,912 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 52
+[gpue04] 2025-06-02 01:52:53,232 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 53
+[gpue04] 2025-06-02 01:54:03,035 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 54
+[gpue04] 2025-06-02 01:55:01,946 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 55
+[gpue04] 2025-06-02 01:55:51,439 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 56
+[gpue04] 2025-06-02 01:56:50,653 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 57
+[gpue04] 2025-06-02 01:57:42,745 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 58
+[gpue04] 2025-06-02 01:58:36,260 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 59
+[gpue04] 2025-06-02 01:59:29,171 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 60
+[gpue04] 2025-06-02 02:00:27,515 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 61
+[gpue04] 2025-06-02 02:01:31,325 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 62
+[gpue04] 2025-06-02 02:02:34,747 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 63
+[gpue04] 2025-06-02 02:03:36,231 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 64
+[gpue04] 2025-06-02 02:04:29,492 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 65
+[gpue04] 2025-06-02 02:05:28,729 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 66
+[gpue04] 2025-06-02 02:06:31,995 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 67
+[gpue04] 2025-06-02 02:07:34,329 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 68
+[gpue04] 2025-06-02 02:08:36,453 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 69
+[gpue04] 2025-06-02 02:09:32,760 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 70
+[gpue04] 2025-06-02 02:11:06,537 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 71
+[gpue04] 2025-06-02 02:12:03,962 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 72
+[gpue04] 2025-06-02 02:13:08,461 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 73
+[gpue04] 2025-06-02 02:14:18,704 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 74
+[gpue04] 2025-06-02 02:15:32,453 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 75
+[gpue04] 2025-06-02 02:16:33,681 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 76
+[gpue04] 2025-06-02 02:17:40,005 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True
+[gpue04] 2025-06-02 02:17:40,006 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=5000 threads=1
+# Ended (code 0) at Mon Jun  2 02:17:41 CDT 2025, elapsed time 5000 seconds

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang/lid_inference_test.log

@@ -0,0 +1,295 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Mon Jun  2 00:36:46 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue04] 2025-06-02 00:37:06,533 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue04] 2025-06-02 00:37:18,559 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamConditionModel(
+  (frontend): S3prlFrontendCondition(
+    (upstream): S3PRLUpstreamCondition(
+      (upstream): UpstreamExpertCondition(
+        (model): Wav2Vec2ModelCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+            (aamsoftmax_weight): ParameterDict()
+            (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True)
+            (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec(
+              (ce): CrossEntropyLoss()
+              (lang2vec_head): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (lang2vec_loss): MSELoss()
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamConditionModel
+    Total Number of model parameters: 977.14 M
+    Number of trainable parameters: 977.14 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
+  warnings.warn(_create_warning_msg(
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue04] 2025-06-02 00:37:19,091 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang/lids0
+[gpue04] 2025-06-02 00:38:18,371 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue04] 2025-06-02 00:39:13,446 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1
+[gpue04] 2025-06-02 00:40:09,054 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2
+[gpue04] 2025-06-02 00:41:05,743 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3
+[gpue04] 2025-06-02 00:42:03,918 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4
+[gpue04] 2025-06-02 00:43:01,470 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5
+[gpue04] 2025-06-02 00:44:00,210 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6
+[gpue04] 2025-06-02 00:45:05,238 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7
+[gpue04] 2025-06-02 00:46:04,557 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8
+[gpue04] 2025-06-02 00:47:20,175 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9
+[gpue04] 2025-06-02 00:48:16,561 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10
+[gpue04] 2025-06-02 00:49:08,874 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11
+[gpue04] 2025-06-02 00:50:08,094 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12
+[gpue04] 2025-06-02 00:51:11,746 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13
+[gpue04] 2025-06-02 00:52:10,238 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14
+[gpue04] 2025-06-02 00:53:15,344 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15
+[gpue04] 2025-06-02 00:54:19,104 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True
+[gpue04] 2025-06-02 00:54:19,105 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=1054 threads=1
+# Ended (code 0) at Mon Jun  2 00:54:20 CDT 2025, elapsed time 1054 seconds

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang/results

@@ -0,0 +1,197 @@
+Accuracy: 98.95%
+Macro Accuracy: 99.09%
+Accuracy per Language:
+hrv: 99.40%
+ces: 98.75%
+spa: 97.64%
+hun: 98.83%
+pol: 98.31%
+slk: 97.35%
+nld: 99.12%
+eng: 99.51%
+est: 100.00%
+ron: 99.49%
+slv: 99.35%
+ita: 99.24%
+lit: 100.00%
+fra: 99.31%
+deu: 99.29%
+fin: 99.79%
+Key: ces_20110609-0900-PLENARY-4-cs_20110609-11:20:13_0, Target: ces, Predicted: cym
+Key: ces_20130610-0900-PLENARY-15-cs_20130610-20:51:10_5, Target: ces, Predicted: slk
+Key: ces_20141126-0900-PLENARY-14-cs_20141126-18:28:01_1, Target: ces, Predicted: slk
+Key: ces_20150209-0900-PLENARY-11-cs_20150209-21:09:35_2, Target: ces, Predicted: hun
+Key: ces_20170403-0900-PLENARY-17-cs_20170403-20:24:45_0, Target: ces, Predicted: deu
+Key: ces_20180614-0900-PLENARY-5-cs_20180614-11:09:11_0, Target: ces, Predicted: pol
+Key: ces_20180614-0900-PLENARY-cs_20180614-11:09:11_0, Target: ces, Predicted: pol
+Key: ces_20180612-0900-PLENARY-14-cs_20180612-17:15:35_13, Target: ces, Predicted: pol
+Key: deu_20090202-0900-PLENARY-13-de_20090202-22:12:47_16, Target: deu, Predicted: ces
+Key: ces_20180912-0900-PLENARY-widetrim-cs_20180912-16:32:19_1, Target: ces, Predicted: fra
+Key: ces_20180912-0900-PLENARY-widetrim-cs_20180912-16:32:19_2, Target: ces, Predicted: fra
+Key: ces_20180912-0900-PLENARY-widetrim-cs_20180912-16:32:19_3, Target: ces, Predicted: fra
+Key: ces_20180912-0900-PLENARY-widetrim-cs_20180912-16:32:19_5, Target: ces, Predicted: fra
+Key: ces_20180912-0900-PLENARY-widetrim-cs_20180912-16:32:19_6, Target: ces, Predicted: fra
+Key: ces_20180612-0900-PLENARY-cs_20180612-17:15:35_14, Target: ces, Predicted: pol
+Key: deu_20111116-0900-PLENARY-3-de_20111116-11:38:53_0, Target: deu, Predicted: ltz
+Key: deu_20111024-0900-PLENARY-10-de_20111024-17:46:08_0, Target: deu, Predicted: eng
+Key: deu_20131021-0900-PLENARY-10-de_20131021-19:11:07_0, Target: deu, Predicted: ell
+Key: deu_20131022-0900-PLENARY-4-de_20131022-09:24:30_14, Target: deu, Predicted: hrv
+Key: deu_20131022-0900-PLENARY-4-de_20131022-08:42:26_10, Target: deu, Predicted: nld
+Key: deu_20160511-0900-PLENARY-14-de_20160511-15:48:52_1, Target: deu, Predicted: fra
+Key: deu_20170314-0900-PLENARY-13-de_20170314-20:56:04_2, Target: deu, Predicted: nld
+Key: deu_20170613-0900-PLENARY-20-de_20170613-22:55:01_13, Target: deu, Predicted: slv
+Key: deu_20171025-0900-PLENARY-21-de_20171025-19:19:40_0, Target: deu, Predicted: ina
+Key: deu_20180611-0900-PLENARY-11-de_20180611-18:10:02_0, Target: deu, Predicted: ron
+Key: deu_20180912-0900-PLENARY-widetrim-de_20180912-16:34:37_1, Target: deu, Predicted: ell
+Key: deu_20180912-0900-PLENARY-widetrim-de_20180912-19:37:22_2, Target: deu, Predicted: fra
+Key: deu_20180912-0900-PLENARY-widetrim-de_20180912-20:47:17_1, Target: deu, Predicted: slk
+Key: eng_20110310-0900-PLENARY-5-en_20110310-10:53:26_3, Target: eng, Predicted: hun
+Key: eng_20120912-0900-PLENARY-9-en_20120912-16:27:37_7, Target: eng, Predicted: slv
+Key: eng_20131022-0900-PLENARY-20-en_20131022-22:05:54_6, Target: eng, Predicted: nld
+Key: eng_20131023-0900-PLENARY-11-en_20131023-17:16:39_6, Target: eng, Predicted: ces
+Key: eng_20171114-0900-PLENARY-14-en_20171114-15:46:05_9, Target: eng, Predicted: ron
+Key: eng_20180911-0900-PLENARY-witholdRO-en_20180911-18:37:21_2, Target: eng, Predicted: deu
+Key: eng_20180613-0900-PLENARY-15-en_20180613-15:21:04_16, Target: eng, Predicted: deu
+Key: eng_20180613-0900-PLENARY-15-en_20180613-15:21:04_6, Target: eng, Predicted: deu
+Key: eng_20200914-0900-PLENARY-en_20200914-21:39:43_1, Target: eng, Predicted: slv
+Key: fin_20140313-0900-PLENARY-14-fi_20140313-13:36:53_0, Target: fin, Predicted: ell
+Key: fra_20111130-0900-PLENARY-11-fr_20111130-16:35:45_18, Target: fra, Predicted: nld
+Key: fra_20111130-0900-PLENARY-11-fr_20111130-16:35:45_19, Target: fra, Predicted: pol
+Key: fra_20131022-0900-PLENARY-14-fr_20131022-16:32:57_5, Target: fra, Predicted: ron
+Key: fra_20140225-0900-PLENARY-11-fr_20140225-15:56:55_0, Target: fra, Predicted: deu
+Key: fra_20140312-0900-PLENARY-15-fr_20140312-20:54:27_9, Target: fra, Predicted: ell
+Key: fra_20160704-0900-PLENARY-13-fr_20160704-20:03:29_0, Target: fra, Predicted: nno
+Key: fra_20170912-0900-PLENARY-21-fr_20170912-20:09:57_0, Target: fra, Predicted: deu
+Key: fra_20180530-0900-PLENARY-3-fr_20180530-11:02:02_4, Target: fra, Predicted: ron
+Key: fra_20180912-0900-PLENARY-widetrim-fr_20180912-19:32:09_2, Target: fra, Predicted: slk
+Key: fra_20180912-0900-PLENARY-widetrim-fr_20180912-19:32:09_3, Target: fra, Predicted: fin
+Key: fra_20180912-0900-PLENARY-widetrim-fr_20180912-19:32:09_5, Target: fra, Predicted: fin
+Key: fra_20201019-0900-PLENARY-fr_20201019-19:35:21_8, Target: fra, Predicted: ita
+Key: hrv_20140114-0900-PLENARY-6-hr_20140114-13:40:47_0, Target: hrv, Predicted: eng
+Key: hrv_20151216-0900-PLENARY-16-hr_20151216-20:01:08_3, Target: hrv, Predicted: ita
+Key: hrv_20170213-0900-PLENARY-18-hr_20170213-22:14:46_3, Target: hrv, Predicted: nld
+Key: hun_20090203-0900-PLENARY-13-hu_20090203-21:55:15_8, Target: hun, Predicted: slv
+Key: hun_20090204-0900-PLENARY-3-hu_20090204-10:53:37_0, Target: hun, Predicted: fra
+Key: hrv_20181022-0900-PLENARY-hr_20181022-22:55:28_7, Target: hrv, Predicted: bos
+Key: hun_20110117-0900-PLENARY-14-hu_20110117-21:41:35_0, Target: hun, Predicted: ita
+Key: hun_20120313-0900-PLENARY-10-hu_20120313-17:33:54_0, Target: hun, Predicted: isl
+Key: hun_20120313-0900-PLENARY-6-hu_20120313-12:45:53_0, Target: hun, Predicted: ita
+Key: hun_20160414-0900-PLENARY-10-hu_20160414-13:26:41_0, Target: hun, Predicted: fin
+Key: hun_20171212-0900-PLENARY-15-hu_20171212-16:18:30_0, Target: hun, Predicted: ron
+Key: hun_20170705-0900-PLENARY-8-hu_20170705-12:19:50_0, Target: hun, Predicted: fra
+Key: ita_20130521-0900-PLENARY-10-it_20130521-17:59:38_11, Target: ita, Predicted: spa
+Key: hun_20180912-0900-PLENARY-widetrim-hu_20180912-22:41:34_2, Target: hun, Predicted: eng
+Key: hun_20180912-0900-PLENARY-widetrim-hu_20180912-22:41:34_3, Target: hun, Predicted: eng
+Key: hun_20180912-0900-PLENARY-widetrim-hu_20180912-22:41:34_4, Target: hun, Predicted: eng
+Key: hun_20180912-0900-PLENARY-widetrim-hu_20180912-22:41:34_5, Target: hun, Predicted: eng
+Key: hun_20180912-0900-PLENARY-widetrim-hu_20180912-22:41:34_6, Target: hun, Predicted: eng
+Key: ita_20140702-0900-PLENARY-12-it_20140702-16:59:58_1, Target: ita, Predicted: spa
+Key: ita_20140225-0900-PLENARY-6-it_20140225-13:50:00_0, Target: ita, Predicted: fra
+Key: ita_20140226-0900-PLENARY-3-it_20140226-09:26:22_1, Target: ita, Predicted: fra
+Key: ita_20140226-0900-PLENARY-3-it_20140226-09:26:22_7, Target: ita, Predicted: spa
+Key: ita_20141021-0900-PLENARY-4-it_20141021-09:37:26_4, Target: ita, Predicted: ces
+Key: ita_20151125-0900-PLENARY-7-it_20151125-12:02:07_15, Target: ita, Predicted: deu
+Key: ita_20151125-0900-PLENARY-7-it_20151125-12:02:07_12, Target: ita, Predicted: ces
+Key: ita_20180613-0900-PLENARY-17-it_20180613-17:05:13_2, Target: ita, Predicted: spa
+Key: nld_20090311-0900-PLENARY-20-nl_20090311-21:13:31_16, Target: nld, Predicted: ces
+Key: nld_20101019-0900-PLENARY-11-nl_20101019-18:14:25_20, Target: nld, Predicted: azz
+Key: nld_20100120-0900-PLENARY-13-nl_20100120-21:54:24_0, Target: nld, Predicted: ell
+Key: nld_20140116-0900-PLENARY-7-nl_20140116-12:49:10_0, Target: nld, Predicted: hun
+Key: nld_20141021-0900-PLENARY-11-nl_20141021-17:19:09_13, Target: nld, Predicted: fra
+Key: nld_20141021-0900-PLENARY-16-nl_20141021-22:35:16_6, Target: nld, Predicted: swe
+Key: nld_20170613-0900-PLENARY-13-nl_20170613-16:02:56_0, Target: nld, Predicted: fra
+Key: nld_20180315-0900-PLENARY-3-nl_20180315-09:28:48_0, Target: nld, Predicted: hun
+Key: nld_20180912-0900-PLENARY-widetrim-nl_20180912-16:04:33_1, Target: nld, Predicted: deu
+Key: nld_20180912-0900-PLENARY-widetrim-nl_20180912-16:04:33_3, Target: nld, Predicted: spa
+Key: pol_20090324-0900-PLENARY-3-pl_20090324-09:54:57_10, Target: pol, Predicted: slk
+Key: pol_20091124-0900-PLENARY-19-pl_20091124-23:31:52_0, Target: pol, Predicted: ukr
+Key: pol_20091124-0900-PLENARY-19-pl_20091124-23:31:52_1, Target: pol, Predicted: ukr
+Key: pol_20091124-0900-PLENARY-19-pl_20091124-23:31:52_2, Target: pol, Predicted: ukr
+Key: pol_20091124-0900-PLENARY-19-pl_20091124-23:31:52_4, Target: pol, Predicted: ukr
+Key: pol_20110512-0900-PLENARY-3-pl_20110512-11:04:51_2, Target: pol, Predicted: bel
+Key: pol_20110512-0900-PLENARY-3-pl_20110512-11:04:51_3, Target: pol, Predicted: bel
+Key: pol_20110705-0900-PLENARY-5-pl_20110705-12:17:25_0, Target: pol, Predicted: ita
+Key: pol_20110706-0900-PLENARY-4-pl_20110706-13:12:02_0, Target: pol, Predicted: ita
+Key: pol_20110915-0900-PLENARY-3-pl_20110915-09:28:56_3, Target: pol, Predicted: ukr
+Key: pol_20111116-0900-PLENARY-9-pl_20111116-17:01:22_0, Target: pol, Predicted: eng
+Key: pol_20111026-0900-PLENARY-15-pl_20111026-18:51:55_8, Target: pol, Predicted: ces
+Key: pol_20111213-0900-PLENARY-10-pl_20111213-16:58:12_1, Target: pol, Predicted: bel
+Key: pol_20111213-0900-PLENARY-5-pl_20111213-11:07:05_3, Target: pol, Predicted: ukr
+Key: pol_20111213-0900-PLENARY-5-pl_20111213-11:07:05_4, Target: pol, Predicted: ukr
+Key: pol_20120201-0900-PLENARY-13-pl_20120201-20:56:53_1, Target: pol, Predicted: bel
+Key: pol_20120201-0900-PLENARY-13-pl_20120201-20:56:53_2, Target: pol, Predicted: bel
+Key: pol_20120201-0900-PLENARY-13-pl_20120201-20:56:53_3, Target: pol, Predicted: bel
+Key: pol_20120201-0900-PLENARY-13-pl_20120201-20:56:53_5, Target: pol, Predicted: bel
+Key: pol_20120201-0900-PLENARY-13-pl_20120201-20:56:53_6, Target: pol, Predicted: ukr
+Key: pol_20120611-0900-PLENARY-13-pl_20120611-17:23:16_0, Target: pol, Predicted: aze
+Key: pol_20120614-0900-PLENARY-5-pl_20120614-11:12:48_3, Target: pol, Predicted: ukr
+Key: pol_20131008-0900-PLENARY-3-pl_20131008-10:35:17_0, Target: pol, Predicted: nld
+Key: pol_20131021-0900-PLENARY-12-pl_20131021-20:34:35_0, Target: pol, Predicted: eng
+Key: pol_20151125-0900-PLENARY-16-pl_20151125-16:59:57_0, Target: pol, Predicted: hun
+Key: pol_20170313-0900-PLENARY-11-pl_20170313-19:41:51_0, Target: pol, Predicted: deu
+Key: pol_20180115-0900-PLENARY-11-pl_20180115-19:09:08_0, Target: pol, Predicted: ell
+Key: pol_20180207-0900-PLENARY-9-pl_20180207-13:13:22_0, Target: pol, Predicted: deu
+Key: pol_20171115-0900-PLENARY-4-pl_20171115-09:24:40_56, Target: pol, Predicted: fra
+Key: pol_20180313-0900-PLENARY-18-pl_20180313-21:30:24_0, Target: pol, Predicted: spa
+Key: pol_20180704-0900-PLENARY-pl_20180704-11:34:24_0, Target: pol, Predicted: sna
+Key: ron_20090309-0900-PLENARY-14-ro_20090309-21:35:00_0, Target: ron, Predicted: deu
+Key: ron_20090310-0900-PLENARY-19-ro_20090310-21:35:23_0, Target: ron, Predicted: eng
+Key: ron_20130312-0900-PLENARY-5-ro_20130312-10:44:27_4, Target: ron, Predicted: slv
+Key: ron_20140416-0900-PLENARY-4-ro_20140416-11:26:09_0, Target: ron, Predicted: hun
+Key: ron_20180207-0900-PLENARY-17-ro_20180207-18:27:19_0, Target: ron, Predicted: ell
+Key: ron_20180207-0900-PLENARY-17-ro_20180207-17:49:18_13, Target: ron, Predicted: ell
+Key: ron_20180613-0900-PLENARY-6-ro_20180613-12:38:55_0, Target: ron, Predicted: ita
+Key: slk_20090310-0900-PLENARY-9-sk_20090310-13:41:29_0, Target: slk, Predicted: eng
+Key: slk_20091124-0900-PLENARY-19-sk_20091124-23:19:02_13, Target: slk, Predicted: ces
+Key: slk_20091124-0900-PLENARY-19-sk_20091124-23:19:02_14, Target: slk, Predicted: ces
+Key: slk_20130312-0900-PLENARY-11-sk_20130312-14:03:05_5, Target: slk, Predicted: ces
+Key: slk_20091124-0900-PLENARY-19-sk_20091124-23:19:02_2, Target: slk, Predicted: ces
+Key: slk_20090421-0900-PLENARY-23-sk_20090421-23:31:18_16, Target: slk, Predicted: ces
+Key: slk_20091124-0900-PLENARY-19-sk_20091124-23:19:02_9, Target: slk, Predicted: ces
+Key: slk_20131210-0900-PLENARY-11-sk_20131210-14:35:10_1, Target: slk, Predicted: ces
+Key: slk_20131120-0900-PLENARY-12-sk_20131120-14:39:04_0, Target: slk, Predicted: ita
+Key: slk_20150908-0900-PLENARY-12-sk_20150908-16:31:31_0, Target: slk, Predicted: nno
+Key: slk_20150211-0900-PLENARY-10-sk_20150211-16:15:34_12, Target: slk, Predicted: ces
+Key: slk_20151124-0900-PLENARY-13-sk_20151124-20:29:15_5, Target: slk, Predicted: ces
+Key: slk_20180312-0900-PLENARY-20-sk_20180312-22:35:26_1, Target: slk, Predicted: ces
+Key: slk_20180612-0900-PLENARY-8-sk_20180612-13:22:33_3, Target: slk, Predicted: ces
+Key: slk_20180612-0900-PLENARY-sk_20180612-13:22:33_4, Target: slk, Predicted: ces
+Key: slk_20201021-0900-PLENARY-sk_20201021-16:00:55_13, Target: slk, Predicted: ces
+Key: slv_20171114-0900-PLENARY-14-sl_20171114-16:22:58_11, Target: slv, Predicted: hrv
+Key: slv_20170704-0900-PLENARY-22-sl_20170704-23:03:46_0, Target: slv, Predicted: deu
+Key: spa_20090203-0900-PLENARY-14-es_20090203-22:21:14_10, Target: spa, Predicted: ron
+Key: spa_20090505-0900-PLENARY-3-es_20090505-09:56:00_7, Target: spa, Predicted: slv
+Key: spa_20091215-0900-PLENARY-14-es_20091215-22:05:17_2, Target: spa, Predicted: ita
+Key: spa_20100120-0900-PLENARY-5-es_20100120-12:42:38_2, Target: spa, Predicted: lit
+Key: spa_20100615-0900-PLENARY-14-es_20100615-21:30:17_28, Target: spa, Predicted: ita
+Key: spa_20140114-0900-PLENARY-6-es_20140114-13:41:54_0, Target: spa, Predicted: ita
+Key: spa_20140114-0900-PLENARY-6-es_20140114-13:41:54_2, Target: spa, Predicted: ita
+Key: spa_20141126-0900-PLENARY-13-es_20141126-16:38:25_2, Target: spa, Predicted: eng
+Key: spa_20141126-0900-PLENARY-13-es_20141126-16:38:25_3, Target: spa, Predicted: eng
+Key: spa_20141126-0900-PLENARY-13-es_20141126-16:38:25_4, Target: spa, Predicted: eng
+Key: spa_20151007-0900-PLENARY-6-es_20151007-12:04:24_3, Target: spa, Predicted: fra
+Key: spa_20151007-0900-PLENARY-6-es_20151007-12:04:24_5, Target: spa, Predicted: deu
+Key: spa_20151007-0900-PLENARY-7-es_20151007-12:04:24_3, Target: spa, Predicted: deu
+Key: spa_20151007-0900-PLENARY-7-es_20151007-12:04:24_5, Target: spa, Predicted: deu
+Key: spa_20170216-0900-PLENARY-3-es_20170216-09:41:17_6, Target: spa, Predicted: ces
+Key: spa_20170216-0900-PLENARY-5-es_20170216-10:42:11_1, Target: spa, Predicted: ces
+Key: spa_20170404-0900-PLENARY-18-es_20170404-18:36:59_2, Target: spa, Predicted: deu
+Key: spa_20170704-0900-PLENARY-21-es_20170704-21:50:24_2, Target: spa, Predicted: ita
+Key: spa_20170704-0900-PLENARY-21-es_20170704-21:50:24_3, Target: spa, Predicted: ita
+Key: spa_20170704-0900-PLENARY-21-es_20170704-21:50:24_4, Target: spa, Predicted: ita
+Key: spa_20170704-0900-PLENARY-21-es_20170704-21:50:24_5, Target: spa, Predicted: ita
+Key: spa_20171004-0900-PLENARY-3-es_20171004-10:28:48_0, Target: spa, Predicted: deu
+Key: spa_20180529-0900-PLENARY-18-es_20180529-16:25:55_8, Target: spa, Predicted: ces
+Key: spa_20180912-0900-PLENARY-widetrim-es_20180912-22:22:18_2, Target: spa, Predicted: deu
+Key: spa_20180912-0900-PLENARY-widetrim-es_20180912-22:22:18_3, Target: spa, Predicted: deu
+Key: spa_20180912-0900-PLENARY-widetrim-es_20180912-22:22:18_4, Target: spa, Predicted: deu
+Key: spa_20180612-0900-PLENARY-14-es_20180612-17:34:51_0, Target: spa, Predicted: ita
+Key: spa_20180912-0900-PLENARY-widetrim-es_20180912-22:22:18_5, Target: spa, Predicted: deu
+Key: spa_20180612-0900-PLENARY-14-es_20180612-17:34:51_3, Target: spa, Predicted: ita
+Key: spa_20180612-0900-PLENARY-14-es_20180612-17:34:51_4, Target: spa, Predicted: ita
+Key: spa_20180612-0900-PLENARY-14-es_20180612-17:34:51_5, Target: spa, Predicted: ita
+Key: spa_20180612-0900-PLENARY-es_20180612-17:34:51_0, Target: spa, Predicted: ita
+Key: spa_20180612-0900-PLENARY-es_20180612-17:34:51_3, Target: spa, Predicted: ita
+Key: spa_20180612-0900-PLENARY-es_20180612-17:34:51_4, Target: spa, Predicted: ita
+Key: spa_20180612-0900-PLENARY-es_20180612-17:34:51_5, Target: spa, Predicted: ita
+Key: spa_20180912-0900-PLENARY-widetrim-es_20180912-22:22:18_1, Target: spa, Predicted: deu

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/lid_inference_test.log

@@ -0,0 +1,300 @@
+# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True 
+# Started at Mon Jun  2 02:37:15 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True
+[gpue04] 2025-06-02 02:37:35,038 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  torch.load(model_file, map_location=device),
+[gpue04] 2025-06-02 02:37:46,607 (lid_inference_dist:86) INFO: Model structure:
+ESPnetLIDUpstreamConditionModel(
+  (frontend): S3prlFrontendCondition(
+    (upstream): S3PRLUpstreamCondition(
+      (upstream): UpstreamExpertCondition(
+        (model): Wav2Vec2ModelCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+            (aamsoftmax_weight): ParameterDict()
+            (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True)
+            (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec(
+              (ce): CrossEntropyLoss()
+              (lang2vec_head): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (lang2vec_loss): MSELoss()
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamConditionModel
+    Total Number of model parameters: 977.14 M
+    Number of trainable parameters: 977.14 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
+  warnings.warn(_create_warning_msg(
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist.
+  warnings.warn(
+[gpue04] 2025-06-02 02:37:47,156 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/lids0
+[gpue04] 2025-06-02 02:38:41,828 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0
+[gpue04] 2025-06-02 02:39:27,483 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1
+[gpue04] 2025-06-02 02:40:15,909 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2
+[gpue04] 2025-06-02 02:41:08,571 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3
+[gpue04] 2025-06-02 02:41:56,182 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4
+[gpue04] 2025-06-02 02:42:40,736 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5
+[gpue04] 2025-06-02 02:43:27,814 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6
+[gpue04] 2025-06-02 02:44:10,740 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7
+[gpue04] 2025-06-02 02:44:52,065 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8
+[gpue04] 2025-06-02 02:45:40,635 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9
+[gpue04] 2025-06-02 02:46:28,394 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10
+[gpue04] 2025-06-02 02:47:09,502 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11
+[gpue04] 2025-06-02 02:47:59,978 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12
+[gpue04] 2025-06-02 02:48:52,866 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13
+[gpue04] 2025-06-02 02:49:41,279 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14
+[gpue04] 2025-06-02 02:50:32,817 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15
+[gpue04] 2025-06-02 02:51:20,444 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 16
+[gpue04] 2025-06-02 02:52:09,714 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 17
+[gpue04] 2025-06-02 02:52:55,108 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 18
+[gpue04] 2025-06-02 02:53:50,212 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 19
+[gpue04] 2025-06-02 02:54:31,533 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 20
+[gpue04] 2025-06-02 02:55:19,223 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True
+[gpue04] 2025-06-02 02:55:19,224 (lid_inference_dist:215) INFO: args.save_tsne_plot: False
+# Accounting: time=1085 threads=1
+# Ended (code 0) at Mon Jun  2 02:55:20 CDT 2025, elapsed time 1085 seconds

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.1.log

@@ -0,0 +1,390 @@
+# python3 -m espnet2.bin.lid_train --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True 
+# Started at Wed Jun  4 20:24:52 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_train.py --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+[gpue08] 2025-06-04 20:25:25,391 (abs_task:1420) INFO: pytorch.version=2.4.0+cu118, cuda.available=True, cudnn.version=90100, cudnn.benchmark=True, cudnn.deterministic=False
+[gpue08] 2025-06-04 20:25:25,398 (abs_task:1421) INFO: Model structure:
+ESPnetLIDUpstreamConditionModel(
+  (frontend): S3prlFrontendCondition(
+    (upstream): S3PRLUpstreamCondition(
+      (upstream): UpstreamExpertCondition(
+        (model): Wav2Vec2ModelCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+            (aamsoftmax_weight): ParameterDict()
+            (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True)
+            (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec(
+              (ce): CrossEntropyLoss()
+              (lang2vec_head): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (lang2vec_loss): MSELoss()
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamConditionModel
+    Total Number of model parameters: 977.14 M
+    Number of trainable parameters: 977.14 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+[gpue08] 2025-06-04 20:25:25,398 (abs_task:1424) INFO: Optimizer:
+Adam (
+Parameter Group 0
+    amsgrad: False
+    betas: [0.9, 0.98]
+    capturable: False
+    differentiable: False
+    eps: 1e-08
+    foreach: None
+    fused: None
+    initial_lr: 1e-05
+    lr: 6.0032e-06
+    maximize: False
+    weight_decay: 0
+)
+[gpue08] 2025-06-04 20:25:25,398 (abs_task:1425) INFO: Scheduler: TristageLR(warmup_steps=1250)(hold_steps=5000)(decay_steps=6250)(init_lr_scale=0.6)(final_lr_scale=0.1)(decay_factor=0.00036841361487904725)
+[gpue08] 2025-06-04 20:25:25,404 (abs_task:1434) INFO: Saving the configuration in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/config.yaml
+[gpue08] 2025-06-04 20:25:25,693 (preprocessor:2245) INFO: Using lang2vec geo
+# Accounting: time=218 threads=1
+# Ended (code 0) at Wed Jun  4 20:25:32 CDT 2025, elapsed time 218 seconds
+[gpue08] 2025-06-04 20:25:41,611 (abs_task:1899) WARNING: Reading dump/raw/train_all_no_filter_lang/category2utt
+[gpue08] 2025-06-04 20:25:41,660 (abs_task:1946) WARNING: Reading dump/raw/train_all_no_filter_lang/dataset2utt
+[gpue08] 2025-06-04 20:25:41,663 (abs_task:1962) WARNING: Reading dump/raw/train_all_no_filter_lang/utt2dataset
+[gpue08] 2025-06-04 20:27:58,237 (abs_task:1997) INFO: [train] dataset:
+ESPnetDataset(
+  speech: {"path": "dump/raw/train_all_no_filter_lang/wav.scp", "type": "sound"}
+  lid_labels: {"path": "dump/raw/train_all_no_filter_lang/utt2spk", "type": "text"}
+  preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False))
+[gpue08] 2025-06-04 20:27:58,256 (abs_task:1998) INFO: [train] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)
+[gpue08] 2025-06-04 20:27:58,256 (abs_task:1999) INFO: [train] collate_fn: <class 'espnet2.train.collate_fn.CommonCollateFn'>(float_pad_value=0.0, int_pad_value=0.0)
+[gpue08] 2025-06-04 20:27:58,256 (abs_task:2000) INFO: [train] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=727476, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3)
+[gpue08] 2025-06-04 20:27:58,323 (abs_task:2001) INFO: [train] mini-batch sizes summary: N-batch=727476, mean=6.0, min=1, max=6
+[gpue08] 2025-06-04 20:27:58,742 (preprocessor:2245) INFO: Using lang2vec geo
+[gpue08] 2025-06-04 20:28:11,299 (abs_task:1899) WARNING: Reading dump/raw/dev_ml_superb2_lang/category2utt
+[gpue08] 2025-06-04 20:28:11,301 (abs_task:1946) WARNING: Reading dump/raw/dev_ml_superb2_lang/dataset2utt
+[gpue08] 2025-06-04 20:28:11,302 (abs_task:1962) WARNING: Reading dump/raw/dev_ml_superb2_lang/utt2dataset
+[gpue08] 2025-06-04 20:28:12,337 (abs_task:1997) INFO: [valid] dataset:
+ESPnetDataset(
+  speech: {"path": "dump/raw/dev_ml_superb2_lang/wav.scp", "type": "sound"}
+  lid_labels: {"path": "dump/raw/dev_ml_superb2_lang/utt2spk", "type": "text"}
+  preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False))
+[gpue08] 2025-06-04 20:28:12,337 (abs_task:1998) INFO: [valid] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)
+[gpue08] 2025-06-04 20:28:12,338 (abs_task:1999) INFO: [valid] collate_fn: <class 'espnet2.train.collate_fn.CommonCollateFn'>(float_pad_value=0.0, int_pad_value=0.0)
+[gpue08] 2025-06-04 20:28:12,338 (abs_task:2000) INFO: [valid] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=4722, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3)
+[gpue08] 2025-06-04 20:28:12,338 (abs_task:2001) INFO: [valid] mini-batch sizes summary: N-batch=4722, mean=6.0, min=4, max=6
+wandb: Currently logged in as: qingzhew (qingzhew-carnegie-mellon-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
+wandb: Tracking run with wandb version 0.19.10
+wandb: Run data is saved locally in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_202812-6dkg2ayp
+wandb: Run `wandb offline` to turn off syncing.
+wandb: Syncing run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch
+wandb: ⭐️ View project at https://wandb.ai/qingzhew-carnegie-mellon-university/lid
+wandb: 🚀 View run at https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/6dkg2ayp
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:218: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  scaler = GradScaler()
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:159: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  states = torch.load(
+[gpue08] 2025-06-04 20:28:22,171 (trainer:176) INFO: The training was resumed using exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/checkpoint.pth
+[gpue08] 2025-06-04 20:28:22,239 (trainer:251) INFO: Frontend featurizer weights for each layer: 
+Parameter containing:
+tensor([-0.0056, -0.0141, -0.0168, -0.0187, -0.0203, -0.0225, -0.0231, -0.0246,
+        -0.0253, -0.0252, -0.0254, -0.0241, -0.0226, -0.0200, -0.0162, -0.0120,
+        -0.0095, -0.0059, -0.0017,  0.0058,  0.0097,  0.0142,  0.0175,  0.0196,
+         0.0211,  0.0224,  0.0228,  0.0230,  0.0226,  0.0224,  0.0215,  0.0210,
+         0.0196,  0.0176,  0.0157,  0.0126,  0.0095,  0.0070,  0.0051,  0.0037,
+         0.0020, -0.0003, -0.0030, -0.0056, -0.0076, -0.0090, -0.0096, -0.0102,
+        -0.0102], device='cuda:0', requires_grad=True)
+[gpue08] 2025-06-04 20:28:22,239 (trainer:267) INFO: Error: 'Linear' object is not subscriptable
+[gpue08] 2025-06-04 20:28:22,240 (trainer:272) INFO: cos_mp: 1.0
+[gpue08] 2025-06-04 20:28:22,240 (trainer:273) INFO: easy_margin: False
+[gpue08] 2025-06-04 20:28:22,240 (trainer:281) WARNING: The training has already reached at max_epoch: 34
+[gpue08] 2025-06-04 20:28:22,253 (trainer:541) INFO: The training was finished at 33 epochs 
+[gpue08] 2025-06-04 20:28:22,253 (average_nbest_models:69) INFO: Averaging 2best models: criterion="valid.accuracy": exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/valid.accuracy.ave_2best.pth
+/work/nvme/bbjs/qwang20/espnet/espnet2/main_funcs/average_nbest_models.py:77: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  _loaded[e] = torch.load(
+[gpue08] 2025-06-04 20:28:27,695 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.32.attention.2.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,695 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.36.attention.2.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,696 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.40.attention.2.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,697 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.44.attention.2.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,697 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.32.bn.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,698 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.36.bn.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,698 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.40.bn.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,699 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.44.bn.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,701 (average_nbest_models:96) INFO: Accumulating encoder.bn.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,701 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn1.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,701 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.0.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.1.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.2.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.3.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.4.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.5.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.6.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn3.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.se.se.3.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn1.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.0.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.1.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.2.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.3.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.4.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.5.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,704 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.6.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,704 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn3.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,704 (average_nbest_models:96) INFO: Accumulating encoder.layer2.se.se.3.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,704 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn1.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,704 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.0.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.1.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.2.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.3.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.4.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.5.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.6.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn3.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.se.se.3.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,707 (average_nbest_models:96) INFO: Accumulating pooling.attention.2.num_batches_tracked instead of averaging
+[gpue08] 2025-06-04 20:28:27,707 (average_nbest_models:96) INFO: Accumulating projector.bn.num_batches_tracked instead of averaging
+wandb:                                                                                
+wandb: 🚀 View run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/6dkg2ayp
+wandb: ⭐️ View project at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid
+wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
+wandb: Find logs at: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_202812-6dkg2ayp/logs
+# Accounting: time=221 threads=1
+# Ended (code 0) at Wed Jun  4 20:28:33 CDT 2025, elapsed time 221 seconds

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.2.log

@@ -0,0 +1,441 @@
+# python3 -m espnet2.bin.lid_train --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True 
+# Started at Wed Jun  4 20:21:54 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_train.py --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+[gpue06] 2025-06-04 20:22:27,532 (abs_task:1420) INFO: pytorch.version=2.4.0+cu118, cuda.available=True, cudnn.version=90100, cudnn.benchmark=True, cudnn.deterministic=False
+[gpue06] 2025-06-04 20:22:27,538 (abs_task:1421) INFO: Model structure:
+ESPnetLIDUpstreamConditionModel(
+  (frontend): S3prlFrontendCondition(
+    (upstream): S3PRLUpstreamCondition(
+      (upstream): UpstreamExpertCondition(
+        (model): Wav2Vec2ModelCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+            (aamsoftmax_weight): ParameterDict()
+            (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True)
+            (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec(
+              (ce): CrossEntropyLoss()
+              (lang2vec_head): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (lang2vec_loss): MSELoss()
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamConditionModel
+    Total Number of model parameters: 977.14 M
+    Number of trainable parameters: 977.14 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+[gpue06] 2025-06-04 20:22:27,538 (abs_task:1424) INFO: Optimizer:
+Adam (
+Parameter Group 0
+    amsgrad: False
+    betas: [0.9, 0.98]
+    capturable: False
+    differentiable: False
+    eps: 1e-08
+    foreach: None
+    fused: None
+    initial_lr: 1e-05
+    lr: 6.0032e-06
+    maximize: False
+    weight_decay: 0
+)
+[gpue06] 2025-06-04 20:22:27,538 (abs_task:1425) INFO: Scheduler: TristageLR(warmup_steps=1250)(hold_steps=5000)(decay_steps=6250)(init_lr_scale=0.6)(final_lr_scale=0.1)(decay_factor=0.00036841361487904725)
+[gpue06] 2025-06-04 20:22:27,544 (abs_task:1434) INFO: Saving the configuration in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/config.yaml
+[gpue06] 2025-06-04 20:22:27,823 (preprocessor:2245) INFO: Using lang2vec geo
+[gpue06] 2025-06-04 20:22:43,726 (abs_task:1899) WARNING: Reading dump/raw/train_all_no_filter_lang/category2utt
+[gpue06] 2025-06-04 20:22:43,727 (abs_task:1946) WARNING: Reading dump/raw/train_all_no_filter_lang/dataset2utt
+[gpue06] 2025-06-04 20:22:43,729 (abs_task:1962) WARNING: Reading dump/raw/train_all_no_filter_lang/utt2dataset
+[gpue06] 2025-06-04 20:24:57,630 (abs_task:1997) INFO: [train] dataset:
+ESPnetDataset(
+  speech: {"path": "dump/raw/train_all_no_filter_lang/wav.scp", "type": "sound"}
+  lid_labels: {"path": "dump/raw/train_all_no_filter_lang/utt2spk", "type": "text"}
+  preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False))
+[gpue06] 2025-06-04 20:24:57,648 (abs_task:1998) INFO: [train] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)
+[gpue06] 2025-06-04 20:24:57,648 (abs_task:1999) INFO: [train] collate_fn: <class 'espnet2.train.collate_fn.CommonCollateFn'>(float_pad_value=0.0, int_pad_value=0.0)
+[gpue06] 2025-06-04 20:24:57,649 (abs_task:2000) INFO: [train] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=727472, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3)
+[gpue06] 2025-06-04 20:24:57,715 (abs_task:2001) INFO: [train] mini-batch sizes summary: N-batch=727472, mean=6.0, min=1, max=6
+[gpue06] 2025-06-04 20:24:58,116 (preprocessor:2245) INFO: Using lang2vec geo
+[gpue06] 2025-06-04 20:25:10,662 (abs_task:1899) WARNING: Reading dump/raw/dev_ml_superb2_lang/category2utt
+[gpue06] 2025-06-04 20:25:10,664 (abs_task:1946) WARNING: Reading dump/raw/dev_ml_superb2_lang/dataset2utt
+[gpue06] 2025-06-04 20:25:10,666 (abs_task:1962) WARNING: Reading dump/raw/dev_ml_superb2_lang/utt2dataset
+[gpue06] 2025-06-04 20:25:11,695 (abs_task:1997) INFO: [valid] dataset:
+ESPnetDataset(
+  speech: {"path": "dump/raw/dev_ml_superb2_lang/wav.scp", "type": "sound"}
+  lid_labels: {"path": "dump/raw/dev_ml_superb2_lang/utt2spk", "type": "text"}
+  preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False))
+[gpue06] 2025-06-04 20:25:11,696 (abs_task:1998) INFO: [valid] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)
+[gpue06] 2025-06-04 20:25:11,696 (abs_task:1999) INFO: [valid] collate_fn: <class 'espnet2.train.collate_fn.CommonCollateFn'>(float_pad_value=0.0, int_pad_value=0.0)
+[gpue06] 2025-06-04 20:25:11,696 (abs_task:2000) INFO: [valid] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=4722, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3)
+[gpue06] 2025-06-04 20:25:11,696 (abs_task:2001) INFO: [valid] mini-batch sizes summary: N-batch=4722, mean=6.0, min=4, max=6
+wandb: Currently logged in as: qingzhew (qingzhew-carnegie-mellon-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
+wandb: Tracking run with wandb version 0.19.10
+wandb: Run data is saved locally in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_202512-0zfdmaq1
+wandb: Run `wandb offline` to turn off syncing.
+wandb: Resuming run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch
+wandb: ⭐️ View project at https://wandb.ai/qingzhew-carnegie-mellon-university/lid
+wandb: 🚀 View run at https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/0zfdmaq1
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:218: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  scaler = GradScaler()
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:159: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  states = torch.load(
+[gpue06] 2025-06-04 20:25:22,045 (trainer:176) INFO: The training was resumed using exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/checkpoint.pth
+[gpue06] 2025-06-04 20:25:22,123 (trainer:251) INFO: Frontend featurizer weights for each layer: 
+Parameter containing:
+tensor([-0.0056, -0.0141, -0.0168, -0.0187, -0.0203, -0.0225, -0.0231, -0.0246,
+        -0.0253, -0.0252, -0.0254, -0.0241, -0.0226, -0.0200, -0.0162, -0.0120,
+        -0.0095, -0.0059, -0.0017,  0.0058,  0.0097,  0.0142,  0.0175,  0.0196,
+         0.0211,  0.0224,  0.0228,  0.0230,  0.0226,  0.0224,  0.0215,  0.0210,
+         0.0196,  0.0176,  0.0157,  0.0126,  0.0095,  0.0070,  0.0051,  0.0037,
+         0.0020, -0.0003, -0.0030, -0.0056, -0.0076, -0.0090, -0.0096, -0.0102,
+        -0.0102], device='cuda:0', requires_grad=True)
+[gpue06] 2025-06-04 20:25:22,124 (trainer:267) INFO: Error: 'Linear' object is not subscriptable
+[gpue06] 2025-06-04 20:25:22,124 (trainer:272) INFO: cos_mp: 1.0
+[gpue06] 2025-06-04 20:25:22,124 (trainer:273) INFO: easy_margin: False
+[gpue06] 2025-06-04 20:25:22,124 (trainer:281) WARNING: The training has already reached at max_epoch: 34
+[gpue06] 2025-06-04 20:25:22,135 (trainer:541) INFO: The training was finished at 33 epochs 
+[gpue06] 2025-06-04 20:25:22,136 (average_nbest_models:69) INFO: Averaging 2best models: criterion="valid.accuracy": exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/valid.accuracy.ave_2best.pth
+/work/nvme/bbjs/qwang20/espnet/espnet2/main_funcs/average_nbest_models.py:77: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  _loaded[e] = torch.load(
+[gpue06] 2025-06-04 20:25:27,720 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.32.attention.2.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,721 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.36.attention.2.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,722 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.40.attention.2.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,723 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.44.attention.2.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,723 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.32.bn.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,723 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.36.bn.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,724 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.40.bn.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,724 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.44.bn.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,727 (average_nbest_models:96) INFO: Accumulating encoder.bn.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,727 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn1.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,727 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.0.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.1.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.2.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.3.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.4.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.5.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.6.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn3.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.se.se.3.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn1.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.0.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.1.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.2.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.3.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.4.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,730 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.5.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,730 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.6.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,730 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn3.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,730 (average_nbest_models:96) INFO: Accumulating encoder.layer2.se.se.3.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,730 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn1.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.0.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.1.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.2.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.3.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.4.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.5.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.6.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn3.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,732 (average_nbest_models:96) INFO: Accumulating encoder.layer3.se.se.3.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,733 (average_nbest_models:96) INFO: Accumulating pooling.attention.2.num_batches_tracked instead of averaging
+[gpue06] 2025-06-04 20:25:27,733 (average_nbest_models:96) INFO: Accumulating projector.bn.num_batches_tracked instead of averaging
+wandb:                                                                                
+wandb: 
+wandb: Run summary:
+wandb:                                    epoch 50
+wandb:                                iteration 100000
+wandb:                         metrics/accuracy 0.95477
+wandb:                    metrics/backward_time 0.96399
+wandb:                       metrics/class_loss 1.09721
+wandb:                             metrics/clip 0
+wandb:                     metrics/forward_time 0.28474
+wandb:                     metrics/geo_loss_all 0.10049
+wandb:              metrics/geo_loss_downstream 0.15867
+wandb:                        metrics/grad_norm 59.37694
+wandb:           metrics/inter_geo_loss_layer32 0.01366
+wandb:           metrics/inter_geo_loss_layer36 0.01344
+wandb:           metrics/inter_geo_loss_layer40 0.01298
+wandb:           metrics/inter_geo_loss_layer44 0.01279
+wandb:              metrics/inter_geo_loss_mean 0.01322
+wandb:                        metrics/iter_time 0.00022
+wandb:                             metrics/loss 0.22447
+wandb:                       metrics/loss_scale 268435456
+wandb:                       metrics/optim0_lr0 0.0
+wandb:                  metrics/optim_step_time 0.03651
+wandb:               train/train_accuracy_epoch 0.95477
+wandb:          train/train_backward_time_epoch 0.96399
+wandb:             train/train_class_loss_epoch 1.09721
+wandb:                   train/train_clip_epoch 0
+wandb:           train/train_forward_time_epoch 0.28474
+wandb:           train/train_geo_loss_all_epoch 0.10049
+wandb:    train/train_geo_loss_downstream_epoch 0.15867
+wandb:  train/train_gpu_max_cached_mem_GB_epoch 130.68359
+wandb:              train/train_grad_norm_epoch 59.37694
+wandb: train/train_inter_geo_loss_layer32_epoch 0.01366
+wandb: train/train_inter_geo_loss_layer36_epoch 0.01344
+wandb: train/train_inter_geo_loss_layer40_epoch 0.01298
+wandb: train/train_inter_geo_loss_layer44_epoch 0.01279
+wandb:    train/train_inter_geo_loss_mean_epoch 0.01322
+wandb:              train/train_iter_time_epoch 0.00022
+wandb:                   train/train_loss_epoch 0.22447
+wandb:             train/train_loss_scale_epoch 268435456
+wandb:             train/train_optim0_lr0_epoch 0.0
+wandb:        train/train_optim_step_time_epoch 0.03651
+wandb:                         train/train_time 5.07792
+wandb:             train/train_train_time_epoch 5.07792
+wandb:               valid/valid_accuracy_epoch 0.89594
+wandb:             valid/valid_class_loss_epoch 2.57223
+wandb:           valid/valid_geo_loss_all_epoch 0.13273
+wandb:    valid/valid_geo_loss_downstream_epoch 0.20731
+wandb:  valid/valid_gpu_max_cached_mem_GB_epoch 130.68359
+wandb: valid/valid_inter_geo_loss_layer32_epoch 0.01976
+wandb: valid/valid_inter_geo_loss_layer36_epoch 0.02211
+wandb: valid/valid_inter_geo_loss_layer40_epoch 0.02097
+wandb: valid/valid_inter_geo_loss_layer44_epoch 0.02058
+wandb:    valid/valid_inter_geo_loss_mean_epoch 0.02086
+wandb:                   valid/valid_loss_epoch 2.08433
+wandb: 
+wandb: 🚀 View run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/0zfdmaq1
+wandb: ⭐️ View project at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid
+wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
+wandb: Find logs at: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_202512-0zfdmaq1/logs

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.3.log

@@ -0,0 +1,460 @@
+# python3 -m espnet2.bin.lid_train --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True 
+# Started at Mon Jun  2 08:00:04 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_train.py --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+[gpue01] 2025-06-02 08:00:37,184 (abs_task:1420) INFO: pytorch.version=2.4.0+cu118, cuda.available=True, cudnn.version=90100, cudnn.benchmark=True, cudnn.deterministic=False
+[gpue01] 2025-06-02 08:00:37,190 (abs_task:1421) INFO: Model structure:
+ESPnetLIDUpstreamConditionModel(
+  (frontend): S3prlFrontendCondition(
+    (upstream): S3PRLUpstreamCondition(
+      (upstream): UpstreamExpertCondition(
+        (model): Wav2Vec2ModelCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+            (aamsoftmax_weight): ParameterDict()
+            (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True)
+            (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec(
+              (ce): CrossEntropyLoss()
+              (lang2vec_head): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (lang2vec_loss): MSELoss()
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamConditionModel
+    Total Number of model parameters: 977.14 M
+    Number of trainable parameters: 977.14 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+[gpue01] 2025-06-02 08:00:37,190 (abs_task:1424) INFO: Optimizer:
+Adam (
+Parameter Group 0
+    amsgrad: False
+    betas: [0.9, 0.98]
+    capturable: False
+    differentiable: False
+    eps: 1e-08
+    foreach: None
+    fused: None
+    initial_lr: 1e-05
+    lr: 6.0032e-06
+    maximize: False
+    weight_decay: 0
+)
+[gpue01] 2025-06-02 08:00:37,190 (abs_task:1425) INFO: Scheduler: TristageLR(warmup_steps=1250)(hold_steps=5000)(decay_steps=6250)(init_lr_scale=0.6)(final_lr_scale=0.1)(decay_factor=0.00036841361487904725)
+[gpue01] 2025-06-02 08:00:37,195 (abs_task:1434) INFO: Saving the configuration in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml
+[gpue01] 2025-06-02 08:00:37,476 (preprocessor:2245) INFO: Using lang2vec geo
+[gpue01] 2025-06-02 08:00:53,379 (abs_task:1899) WARNING: Reading dump/raw/train_all_no_filter_lang/category2utt
+[gpue01] 2025-06-02 08:00:53,380 (abs_task:1946) WARNING: Reading dump/raw/train_all_no_filter_lang/dataset2utt
+[gpue01] 2025-06-02 08:00:53,382 (abs_task:1962) WARNING: Reading dump/raw/train_all_no_filter_lang/utt2dataset
+[gpue01] 2025-06-02 08:03:08,576 (abs_task:1997) INFO: [train] dataset:
+ESPnetDataset(
+  speech: {"path": "dump/raw/train_all_no_filter_lang/wav.scp", "type": "sound"}
+  lid_labels: {"path": "dump/raw/train_all_no_filter_lang/utt2spk", "type": "text"}
+  preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False))
+[gpue01] 2025-06-02 08:03:08,577 (abs_task:1998) INFO: [train] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)
+[gpue01] 2025-06-02 08:03:08,577 (abs_task:1999) INFO: [train] collate_fn: <class 'espnet2.train.collate_fn.CommonCollateFn'>(float_pad_value=0.0, int_pad_value=0.0)
+[gpue01] 2025-06-02 08:03:08,577 (abs_task:2000) INFO: [train] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=727460, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3)
+[gpue01] 2025-06-02 08:03:08,642 (abs_task:2001) INFO: [train] mini-batch sizes summary: N-batch=727460, mean=6.0, min=1, max=6
+[gpue01] 2025-06-02 08:03:09,071 (preprocessor:2245) INFO: Using lang2vec geo
+[gpue01] 2025-06-02 08:03:21,631 (abs_task:1899) WARNING: Reading dump/raw/dev_ml_superb2_lang/category2utt
+[gpue01] 2025-06-02 08:03:21,632 (abs_task:1946) WARNING: Reading dump/raw/dev_ml_superb2_lang/dataset2utt
+[gpue01] 2025-06-02 08:03:21,633 (abs_task:1962) WARNING: Reading dump/raw/dev_ml_superb2_lang/utt2dataset
+[gpue01] 2025-06-02 08:03:22,657 (abs_task:1997) INFO: [valid] dataset:
+ESPnetDataset(
+  speech: {"path": "dump/raw/dev_ml_superb2_lang/wav.scp", "type": "sound"}
+  lid_labels: {"path": "dump/raw/dev_ml_superb2_lang/utt2spk", "type": "text"}
+  preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False))
+[gpue01] 2025-06-02 08:03:22,657 (abs_task:1998) INFO: [valid] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)
+[gpue01] 2025-06-02 08:03:22,657 (abs_task:1999) INFO: [valid] collate_fn: <class 'espnet2.train.collate_fn.CommonCollateFn'>(float_pad_value=0.0, int_pad_value=0.0)
+[gpue01] 2025-06-02 08:03:22,657 (abs_task:2000) INFO: [valid] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=4722, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3)
+[gpue01] 2025-06-02 08:03:22,658 (abs_task:2001) INFO: [valid] mini-batch sizes summary: N-batch=4722, mean=6.0, min=4, max=6
+wandb: Currently logged in as: qingzhew (qingzhew-carnegie-mellon-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
+wandb: Tracking run with wandb version 0.19.10
+wandb: Run data is saved locally in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/wandb/run-20250602_080323-0zfdmaq1
+wandb: Run `wandb offline` to turn off syncing.
+wandb: Resuming run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3
+wandb: ⭐️ View project at https://wandb.ai/qingzhew-carnegie-mellon-university/lid
+wandb: 🚀 View run at https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/0zfdmaq1
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:218: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  scaler = GradScaler()
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:159: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  states = torch.load(
+[gpue01] 2025-06-02 08:03:32,100 (trainer:176) INFO: The training was resumed using exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/checkpoint.pth
+[gpue01] 2025-06-02 08:03:32,199 (trainer:251) INFO: Frontend featurizer weights for each layer: 
+Parameter containing:
+tensor([-0.0056, -0.0140, -0.0167, -0.0186, -0.0202, -0.0224, -0.0230, -0.0245,
+        -0.0252, -0.0250, -0.0253, -0.0240, -0.0225, -0.0199, -0.0161, -0.0120,
+        -0.0094, -0.0058, -0.0017,  0.0059,  0.0098,  0.0142,  0.0175,  0.0197,
+         0.0211,  0.0224,  0.0228,  0.0230,  0.0225,  0.0223,  0.0215,  0.0209,
+         0.0195,  0.0176,  0.0156,  0.0126,  0.0094,  0.0070,  0.0050,  0.0036,
+         0.0019, -0.0004, -0.0031, -0.0057, -0.0077, -0.0090, -0.0097, -0.0103,
+        -0.0103], device='cuda:0', requires_grad=True)
+[gpue01] 2025-06-02 08:03:32,200 (trainer:267) INFO: Error: 'Linear' object is not subscriptable
+[gpue01] 2025-06-02 08:03:32,200 (trainer:272) INFO: cos_mp: 1.0
+[gpue01] 2025-06-02 08:03:32,200 (trainer:273) INFO: easy_margin: False
+[gpue01] 2025-06-02 08:03:32,211 (trainer:347) INFO: 29/50epoch started
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:645: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with autocast(
+[gpue01] 2025-06-02 08:08:32,675 (trainer:816) INFO: 29epoch:train:1-100batch: iter_time=0.003, forward_time=0.394, class_loss=1.038, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.213, accuracy=0.960, backward_time=1.214, grad_norm=40.433, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=5.725e-07, train_time=6.534
+[gpue01] 2025-06-02 08:10:53,126 (trainer:816) INFO: 29epoch:train:101-200batch: iter_time=8.669e-05, forward_time=0.352, class_loss=1.234, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.106, loss=0.252, accuracy=0.947, backward_time=1.033, grad_norm=67.550, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=5.672e-07, train_time=5.618
+[gpue01] 2025-06-02 08:13:06,132 (trainer:816) INFO: 29epoch:train:201-300batch: iter_time=8.747e-05, forward_time=0.336, class_loss=1.151, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.012, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.235, accuracy=0.948, backward_time=0.973, grad_norm=55.134, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.620e-07, train_time=5.320
+[gpue01] 2025-06-02 08:15:18,146 (trainer:816) INFO: 29epoch:train:301-400batch: iter_time=9.459e-05, forward_time=0.338, class_loss=1.302, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.266, accuracy=0.950, backward_time=0.962, grad_norm=56.978, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=5.569e-07, train_time=5.280
+[gpue01] 2025-06-02 08:17:18,134 (trainer:816) INFO: 29epoch:train:401-500batch: iter_time=9.884e-05, forward_time=0.300, class_loss=1.251, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.255, accuracy=0.945, backward_time=0.879, grad_norm=50.561, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.518e-07, train_time=4.799
+[gpue01] 2025-06-02 08:19:20,112 (trainer:816) INFO: 29epoch:train:501-600batch: iter_time=9.952e-05, forward_time=0.288, class_loss=0.865, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.178, accuracy=0.968, backward_time=0.911, grad_norm=51.561, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=5.467e-07, train_time=4.879
+[gpue01] 2025-06-02 08:21:24,824 (trainer:816) INFO: 29epoch:train:601-700batch: iter_time=8.929e-05, forward_time=0.286, class_loss=0.922, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.190, accuracy=0.962, backward_time=0.941, grad_norm=48.480, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.417e-07, train_time=4.988
+[gpue01] 2025-06-02 08:23:15,407 (trainer:816) INFO: 29epoch:train:701-800batch: iter_time=1.002e-04, forward_time=0.255, class_loss=1.501, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.305, accuracy=0.943, backward_time=0.830, grad_norm=74.153, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.367e-07, train_time=4.423
+[gpue01] 2025-06-02 08:25:12,844 (trainer:816) INFO: 29epoch:train:801-900batch: iter_time=9.754e-05, forward_time=0.252, class_loss=1.060, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.217, accuracy=0.957, backward_time=0.902, grad_norm=54.887, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=5.318e-07, train_time=4.697
+[gpue01] 2025-06-02 08:27:12,890 (trainer:816) INFO: 29epoch:train:901-1000batch: iter_time=9.747e-05, forward_time=0.244, class_loss=1.193, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.244, accuracy=0.952, backward_time=0.937, grad_norm=42.200, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.269e-07, train_time=4.801
+[gpue01] 2025-06-02 08:29:03,462 (trainer:816) INFO: 29epoch:train:1001-1100batch: iter_time=9.858e-05, forward_time=0.249, class_loss=1.149, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.235, accuracy=0.953, backward_time=0.835, grad_norm=58.891, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.221e-07, train_time=4.422
+[gpue01] 2025-06-02 08:31:01,649 (trainer:816) INFO: 29epoch:train:1101-1200batch: iter_time=1.007e-04, forward_time=0.238, class_loss=1.181, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.241, accuracy=0.949, backward_time=0.922, grad_norm=100.141, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.173e-07, train_time=4.727
+[gpue01] 2025-06-02 08:32:49,871 (trainer:816) INFO: 29epoch:train:1201-1300batch: iter_time=9.155e-05, forward_time=0.239, class_loss=1.129, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.231, accuracy=0.948, backward_time=0.822, grad_norm=46.727, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.126e-07, train_time=4.328
+[gpue01] 2025-06-02 08:34:45,411 (trainer:816) INFO: 29epoch:train:1301-1400batch: iter_time=9.736e-05, forward_time=0.264, class_loss=1.095, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.224, accuracy=0.952, backward_time=0.870, grad_norm=40.216, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=5.079e-07, train_time=4.621
+[gpue01] 2025-06-02 08:36:35,310 (trainer:816) INFO: 29epoch:train:1401-1500batch: iter_time=9.289e-05, forward_time=0.240, class_loss=1.256, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.256, accuracy=0.948, backward_time=0.837, grad_norm=65.161, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.032e-07, train_time=4.395
+[gpue01] 2025-06-02 08:38:33,818 (trainer:816) INFO: 29epoch:train:1501-1600batch: iter_time=9.491e-05, forward_time=0.265, class_loss=1.038, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.213, accuracy=0.965, backward_time=0.900, grad_norm=51.130, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.986e-07, train_time=4.740
+[gpue01] 2025-06-02 08:40:35,271 (trainer:816) INFO: 29epoch:train:1601-1700batch: iter_time=9.165e-05, forward_time=0.272, class_loss=0.837, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.173, accuracy=0.965, backward_time=0.921, grad_norm=52.871, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.940e-07, train_time=4.857
+[gpue01] 2025-06-02 08:42:35,412 (trainer:816) INFO: 29epoch:train:1701-1800batch: iter_time=8.960e-05, forward_time=0.266, class_loss=1.142, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.234, accuracy=0.953, backward_time=0.915, grad_norm=50.981, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=4.895e-07, train_time=4.805
+[gpue01] 2025-06-02 08:44:31,106 (trainer:816) INFO: 29epoch:train:1801-1900batch: iter_time=1.021e-04, forward_time=0.230, class_loss=1.361, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.277, accuracy=0.945, backward_time=0.906, grad_norm=76.043, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.038, optim0_lr0=4.850e-07, train_time=4.627
+[gpue01] 2025-06-02 08:46:36,834 (trainer:816) INFO: 29epoch:train:1901-2000batch: iter_time=8.900e-05, forward_time=0.264, class_loss=1.395, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.284, accuracy=0.945, backward_time=0.973, grad_norm=59.458, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.806e-07, train_time=5.028
+[gpue01] 2025-06-02 09:09:51,218 (trainer:401) INFO: 29epoch results: [train] iter_time=2.479e-04, forward_time=0.279, class_loss=1.155, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.236, accuracy=0.953, backward_time=0.924, grad_norm=57.178, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.253e-07, train_time=4.894, time=43 minutes and 4.77 seconds, total_count=58000, gpu_max_cached_mem_GB=79.320, [valid] class_loss=2.591, geo_loss_downstream=0.207, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.133, loss=2.099, accuracy=0.894, time=23 minutes and 14.24 seconds, total_count=136938, gpu_max_cached_mem_GB=79.320
+[gpue01] 2025-06-02 09:10:04,925 (trainer:467) INFO: There are no improvements in this epoch
+[gpue01] 2025-06-02 09:10:04,945 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/27epoch.pth, exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/28epoch.pth
+[gpue01] 2025-06-02 09:10:04,945 (trainer:335) INFO: 30/50epoch started. Estimated time to finish: 23 hours, 17 minutes and 27.41 seconds
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:645: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
+  with autocast(
+[gpue01] 2025-06-02 09:14:40,846 (trainer:816) INFO: 30epoch:train:1-100batch: iter_time=0.008, forward_time=0.415, class_loss=1.226, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.250, accuracy=0.952, backward_time=0.915, grad_norm=53.097, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=4.762e-07, train_time=5.456
+[gpue01] 2025-06-02 09:16:39,711 (trainer:816) INFO: 30epoch:train:101-200batch: iter_time=9.637e-05, forward_time=0.353, class_loss=1.158, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.237, accuracy=0.953, backward_time=0.815, grad_norm=71.507, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.718e-07, train_time=4.754
+[gpue01] 2025-06-02 09:19:02,535 (trainer:816) INFO: 30epoch:train:201-300batch: iter_time=9.507e-05, forward_time=0.368, class_loss=1.156, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.236, accuracy=0.952, backward_time=1.041, grad_norm=46.785, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.675e-07, train_time=5.712
+[gpue01] 2025-06-02 09:21:05,464 (trainer:816) INFO: 30epoch:train:301-400batch: iter_time=1.003e-04, forward_time=0.318, class_loss=1.121, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.229, accuracy=0.951, backward_time=0.891, grad_norm=48.383, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.632e-07, train_time=4.916
+[gpue01] 2025-06-02 09:22:55,038 (trainer:816) INFO: 30epoch:train:401-500batch: iter_time=9.499e-05, forward_time=0.284, class_loss=1.061, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.217, accuracy=0.962, backward_time=0.792, grad_norm=77.788, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.589e-07, train_time=4.382
+[gpue01] 2025-06-02 09:24:57,833 (trainer:816) INFO: 30epoch:train:501-600batch: iter_time=1.014e-04, forward_time=0.290, class_loss=1.085, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.222, accuracy=0.963, backward_time=0.918, grad_norm=59.019, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.547e-07, train_time=4.911
+[gpue01] 2025-06-02 09:26:53,859 (trainer:816) INFO: 30epoch:train:601-700batch: iter_time=9.895e-05, forward_time=0.280, class_loss=1.015, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.208, accuracy=0.965, backward_time=0.858, grad_norm=55.588, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=4.506e-07, train_time=4.640
+[gpue01] 2025-06-02 09:29:00,707 (trainer:816) INFO: 30epoch:train:701-800batch: iter_time=1.006e-04, forward_time=0.274, class_loss=1.328, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.271, accuracy=0.943, backward_time=0.974, grad_norm=65.465, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.464e-07, train_time=5.073
+[gpue01] 2025-06-02 09:31:13,341 (trainer:816) INFO: 30epoch:train:801-900batch: iter_time=1.231e-04, forward_time=0.272, class_loss=1.336, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.272, accuracy=0.947, backward_time=1.035, grad_norm=50.503, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.423e-07, train_time=5.305
+[gpue01] 2025-06-02 09:33:12,121 (trainer:816) INFO: 30epoch:train:901-1000batch: iter_time=1.060e-04, forward_time=0.241, class_loss=1.184, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.242, accuracy=0.955, backward_time=0.926, grad_norm=48.061, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.383e-07, train_time=4.750
+[gpue01] 2025-06-02 09:35:05,485 (trainer:816) INFO: 30epoch:train:1001-1100batch: iter_time=1.136e-04, forward_time=0.232, class_loss=1.135, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.232, accuracy=0.948, backward_time=0.881, grad_norm=41.919, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=4.343e-07, train_time=4.534
+[gpue01] 2025-06-02 09:37:02,151 (trainer:816) INFO: 30epoch:train:1101-1200batch: iter_time=1.137e-04, forward_time=0.245, class_loss=1.438, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.293, accuracy=0.942, backward_time=0.902, grad_norm=75.206, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=4.303e-07, train_time=4.666
+[gpue01] 2025-06-02 09:38:59,107 (trainer:816) INFO: 30epoch:train:1201-1300batch: iter_time=1.021e-04, forward_time=0.252, class_loss=1.081, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.221, accuracy=0.962, backward_time=0.897, grad_norm=52.787, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.263e-07, train_time=4.677
+[gpue01] 2025-06-02 09:40:58,086 (trainer:816) INFO: 30epoch:train:1301-1400batch: iter_time=1.154e-04, forward_time=0.263, class_loss=1.120, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.229, accuracy=0.955, backward_time=0.906, grad_norm=47.309, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.224e-07, train_time=4.758
+[gpue01] 2025-06-02 09:42:58,353 (trainer:816) INFO: 30epoch:train:1401-1500batch: iter_time=1.042e-04, forward_time=0.265, class_loss=1.275, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.260, accuracy=0.942, backward_time=0.918, grad_norm=74.393, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=4.186e-07, train_time=4.810
+[gpue01] 2025-06-02 09:44:43,549 (trainer:816) INFO: 30epoch:train:1501-1600batch: iter_time=1.006e-04, forward_time=0.231, class_loss=1.188, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.243, accuracy=0.950, backward_time=0.799, grad_norm=68.925, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=4.147e-07, train_time=4.207
+[gpue01] 2025-06-02 09:46:43,647 (trainer:816) INFO: 30epoch:train:1601-1700batch: iter_time=9.242e-05, forward_time=0.278, class_loss=1.219, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.249, accuracy=0.947, backward_time=0.903, grad_norm=38.583, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.109e-07, train_time=4.803
+[gpue01] 2025-06-02 09:48:51,075 (trainer:816) INFO: 30epoch:train:1701-1800batch: iter_time=1.070e-04, forward_time=0.275, class_loss=1.158, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.237, accuracy=0.957, backward_time=0.979, grad_norm=72.308, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.072e-07, train_time=5.096
+[gpue01] 2025-06-02 09:50:42,998 (trainer:816) INFO: 30epoch:train:1801-1900batch: iter_time=9.723e-05, forward_time=0.249, class_loss=1.209, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.247, accuracy=0.948, backward_time=0.849, grad_norm=54.531, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.034e-07, train_time=4.476
+[gpue01] 2025-06-02 09:52:36,303 (trainer:816) INFO: 30epoch:train:1901-2000batch: iter_time=9.175e-05, forward_time=0.226, class_loss=1.016, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.208, accuracy=0.953, backward_time=0.887, grad_norm=55.463, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.997e-07, train_time=4.531
+[gpue01] 2025-06-02 10:15:46,744 (trainer:401) INFO: 30epoch results: [train] iter_time=4.751e-04, forward_time=0.281, class_loss=1.175, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.240, accuracy=0.952, backward_time=0.904, grad_norm=57.881, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.369e-07, train_time=4.823, time=42 minutes and 31.56 seconds, total_count=60000, gpu_max_cached_mem_GB=81.803, [valid] class_loss=2.597, geo_loss_downstream=0.220, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.141, loss=2.106, accuracy=0.895, time=23 minutes and 10.24 seconds, total_count=141660, gpu_max_cached_mem_GB=81.803
+[gpue01] 2025-06-02 10:15:59,998 (trainer:467) INFO: There are no improvements in this epoch
+[gpue01] 2025-06-02 10:16:00,022 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/29epoch.pth
+[gpue01] 2025-06-02 10:16:00,023 (trainer:335) INFO: 31/50epoch started. Estimated time to finish: 22 hours, 4 minutes and 38.11 seconds
+[gpue01] 2025-06-02 10:20:30,892 (trainer:816) INFO: 31epoch:train:1-100batch: iter_time=0.002, forward_time=0.401, class_loss=1.160, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.237, accuracy=0.952, backward_time=0.896, grad_norm=73.411, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=3.961e-07, train_time=5.300
+[gpue01] 2025-06-02 10:22:38,629 (trainer:816) INFO: 31epoch:train:101-200batch: iter_time=9.710e-05, forward_time=0.364, class_loss=1.277, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.261, accuracy=0.948, backward_time=0.893, grad_norm=65.669, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.924e-07, train_time=5.109
+[gpue01] 2025-06-02 10:24:35,032 (trainer:816) INFO: 31epoch:train:201-300batch: iter_time=9.069e-05, forward_time=0.330, class_loss=1.280, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.261, accuracy=0.948, backward_time=0.813, grad_norm=67.795, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.888e-07, train_time=4.655
+[gpue01] 2025-06-02 10:26:50,531 (trainer:816) INFO: 31epoch:train:301-400batch: iter_time=9.965e-05, forward_time=0.337, class_loss=1.468, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.299, accuracy=0.942, backward_time=0.999, grad_norm=90.267, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=3.853e-07, train_time=5.419
+[gpue01] 2025-06-02 10:28:43,674 (trainer:816) INFO: 31epoch:train:401-500batch: iter_time=9.749e-05, forward_time=0.288, class_loss=1.018, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.209, accuracy=0.960, backward_time=0.823, grad_norm=47.089, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.817e-07, train_time=4.525
+[gpue01] 2025-06-02 10:30:42,003 (trainer:816) INFO: 31epoch:train:501-600batch: iter_time=1.059e-04, forward_time=0.283, class_loss=1.170, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.239, accuracy=0.953, backward_time=0.879, grad_norm=56.840, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.782e-07, train_time=4.732
+[gpue01] 2025-06-02 10:32:49,253 (trainer:816) INFO: 31epoch:train:601-700batch: iter_time=9.643e-05, forward_time=0.288, class_loss=1.159, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.237, accuracy=0.955, backward_time=0.965, grad_norm=71.334, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.748e-07, train_time=5.089
+[gpue01] 2025-06-02 10:34:41,795 (trainer:816) INFO: 31epoch:train:701-800batch: iter_time=9.690e-05, forward_time=0.263, class_loss=1.211, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.247, accuracy=0.952, backward_time=0.841, grad_norm=58.619, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.713e-07, train_time=4.501
+[gpue01] 2025-06-02 10:36:43,614 (trainer:816) INFO: 31epoch:train:801-900batch: iter_time=9.865e-05, forward_time=0.262, class_loss=1.068, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.219, accuracy=0.958, backward_time=0.938, grad_norm=79.938, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.679e-07, train_time=4.872
+[gpue01] 2025-06-02 10:38:47,447 (trainer:816) INFO: 31epoch:train:901-1000batch: iter_time=1.059e-04, forward_time=0.263, class_loss=1.064, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.218, accuracy=0.955, backward_time=0.955, grad_norm=56.857, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.646e-07, train_time=4.953
+[gpue01] 2025-06-02 10:40:54,941 (trainer:816) INFO: 31epoch:train:1001-1100batch: iter_time=1.020e-04, forward_time=0.273, class_loss=1.107, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.227, accuracy=0.957, backward_time=0.981, grad_norm=74.898, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.612e-07, train_time=5.099
+[gpue01] 2025-06-02 10:42:58,685 (trainer:816) INFO: 31epoch:train:1101-1200batch: iter_time=9.935e-05, forward_time=0.250, class_loss=1.028, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.211, accuracy=0.960, backward_time=0.967, grad_norm=52.780, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.579e-07, train_time=4.949
+[gpue01] 2025-06-02 10:45:01,783 (trainer:816) INFO: 31epoch:train:1201-1300batch: iter_time=9.316e-05, forward_time=0.254, class_loss=1.400, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.285, accuracy=0.943, backward_time=0.958, grad_norm=58.518, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.546e-07, train_time=4.923
+[gpue01] 2025-06-02 10:47:06,260 (trainer:816) INFO: 31epoch:train:1301-1400batch: iter_time=1.076e-04, forward_time=0.263, class_loss=0.926, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.190, accuracy=0.961, backward_time=0.962, grad_norm=50.230, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.514e-07, train_time=4.978
+[gpue01] 2025-06-02 10:48:58,412 (trainer:816) INFO: 31epoch:train:1401-1500batch: iter_time=9.517e-05, forward_time=0.235, class_loss=1.449, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.295, accuracy=0.940, backward_time=0.867, grad_norm=53.965, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.481e-07, train_time=4.485
+[gpue01] 2025-06-02 10:50:59,774 (trainer:816) INFO: 31epoch:train:1501-1600batch: iter_time=1.044e-04, forward_time=0.242, class_loss=0.961, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.197, accuracy=0.960, backward_time=0.951, grad_norm=72.314, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.450e-07, train_time=4.854
+[gpue01] 2025-06-02 10:53:04,143 (trainer:816) INFO: 31epoch:train:1601-1700batch: iter_time=9.928e-05, forward_time=0.267, class_loss=1.200, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.245, accuracy=0.953, backward_time=0.957, grad_norm=59.742, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.418e-07, train_time=4.974
+[gpue01] 2025-06-02 10:55:04,940 (trainer:816) INFO: 31epoch:train:1701-1800batch: iter_time=9.423e-05, forward_time=0.243, class_loss=0.887, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.183, accuracy=0.965, backward_time=0.944, grad_norm=30.131, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.387e-07, train_time=4.831
+[gpue01] 2025-06-02 10:57:19,283 (trainer:816) INFO: 31epoch:train:1801-1900batch: iter_time=1.037e-04, forward_time=0.271, class_loss=1.390, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.283, accuracy=0.942, backward_time=1.053, grad_norm=51.783, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.356e-07, train_time=5.373
+[gpue01] 2025-06-02 10:59:05,692 (trainer:816) INFO: 31epoch:train:1901-2000batch: iter_time=9.012e-05, forward_time=0.236, class_loss=1.287, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.263, accuracy=0.950, backward_time=0.808, grad_norm=56.740, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.325e-07, train_time=4.256
+[gpue01] 2025-06-02 11:22:28,488 (trainer:401) INFO: 31epoch results: [train] iter_time=1.963e-04, forward_time=0.281, class_loss=1.176, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.240, accuracy=0.953, backward_time=0.922, grad_norm=61.446, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.634e-07, train_time=4.894, time=43 minutes and 5.9 seconds, total_count=62000, gpu_max_cached_mem_GB=81.803, [valid] class_loss=2.593, geo_loss_downstream=0.212, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.021, geo_loss_all=0.135, loss=2.102, accuracy=0.895, time=23 minutes and 22.56 seconds, total_count=146382, gpu_max_cached_mem_GB=81.803
+[gpue01] 2025-06-02 11:22:41,952 (trainer:469) INFO: The best model has been updated: valid.accuracy
+[gpue01] 2025-06-02 11:22:41,971 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/30epoch.pth
+[gpue01] 2025-06-02 11:22:41,972 (trainer:335) INFO: 32/50epoch started. Estimated time to finish: 21 hours, 1 minute and 21.81 seconds
+[gpue01] 2025-06-02 11:27:07,026 (trainer:816) INFO: 32epoch:train:1-100batch: iter_time=0.003, forward_time=0.400, class_loss=1.536, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.312, accuracy=0.938, backward_time=0.832, grad_norm=74.839, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=3.294e-07, train_time=5.039
+[gpue01] 2025-06-02 11:29:21,087 (trainer:816) INFO: 32epoch:train:101-200batch: iter_time=9.411e-05, forward_time=0.386, class_loss=1.352, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.276, accuracy=0.945, backward_time=0.935, grad_norm=43.988, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.264e-07, train_time=5.362
+[gpue01] 2025-06-02 11:31:30,188 (trainer:816) INFO: 32epoch:train:201-300batch: iter_time=9.077e-05, forward_time=0.347, class_loss=1.172, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.240, accuracy=0.957, backward_time=0.924, grad_norm=62.186, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.234e-07, train_time=5.163
+[gpue01] 2025-06-02 11:33:47,390 (trainer:816) INFO: 32epoch:train:301-400batch: iter_time=1.018e-04, forward_time=0.344, class_loss=1.022, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.210, accuracy=0.957, backward_time=1.009, grad_norm=44.390, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.205e-07, train_time=5.487
+[gpue01] 2025-06-02 11:35:49,789 (trainer:816) INFO: 32epoch:train:401-500batch: iter_time=1.054e-04, forward_time=0.297, class_loss=1.172, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.240, accuracy=0.952, backward_time=0.908, grad_norm=75.410, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.175e-07, train_time=4.895
+[gpue01] 2025-06-02 11:37:41,933 (trainer:816) INFO: 32epoch:train:501-600batch: iter_time=9.698e-05, forward_time=0.280, class_loss=1.211, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.247, accuracy=0.953, backward_time=0.819, grad_norm=79.169, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.146e-07, train_time=4.485
+[gpue01] 2025-06-02 11:39:46,309 (trainer:816) INFO: 32epoch:train:601-700batch: iter_time=9.339e-05, forward_time=0.293, class_loss=1.218, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.249, accuracy=0.953, backward_time=0.930, grad_norm=58.856, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.117e-07, train_time=4.974
+[gpue01] 2025-06-02 11:41:54,707 (trainer:816) INFO: 32epoch:train:701-800batch: iter_time=1.009e-04, forward_time=0.292, class_loss=0.833, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.172, accuracy=0.968, backward_time=0.972, grad_norm=58.946, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=3.089e-07, train_time=5.135
+[gpue01] 2025-06-02 11:44:02,832 (trainer:816) INFO: 32epoch:train:801-900batch: iter_time=9.829e-05, forward_time=0.264, class_loss=1.373, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.280, accuracy=0.942, backward_time=0.998, grad_norm=62.452, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.060e-07, train_time=5.124
+[gpue01] 2025-06-02 11:45:51,409 (trainer:816) INFO: 32epoch:train:901-1000batch: iter_time=9.919e-05, forward_time=0.229, class_loss=0.924, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.012, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.190, accuracy=0.963, backward_time=0.834, grad_norm=42.125, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.032e-07, train_time=4.342
+[gpue01] 2025-06-02 11:47:40,644 (trainer:816) INFO: 32epoch:train:1001-1100batch: iter_time=1.009e-04, forward_time=0.222, class_loss=0.990, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.203, accuracy=0.957, backward_time=0.848, grad_norm=42.052, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.004e-07, train_time=4.369
+[gpue01] 2025-06-02 11:49:25,715 (trainer:816) INFO: 32epoch:train:1101-1200batch: iter_time=9.471e-05, forward_time=0.231, class_loss=1.161, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.237, accuracy=0.953, backward_time=0.797, grad_norm=49.309, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=2.977e-07, train_time=4.202
+[gpue01] 2025-06-02 11:51:08,460 (trainer:816) INFO: 32epoch:train:1201-1300batch: iter_time=9.436e-05, forward_time=0.238, class_loss=0.849, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.012, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.175, accuracy=0.970, backward_time=0.768, grad_norm=53.095, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=2.950e-07, train_time=4.109
+[gpue01] 2025-06-02 11:53:07,433 (trainer:816) INFO: 32epoch:train:1301-1400batch: iter_time=1.007e-04, forward_time=0.257, class_loss=1.168, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.239, accuracy=0.952, backward_time=0.912, grad_norm=45.273, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=2.923e-07, train_time=4.758
+[gpue01] 2025-06-02 11:55:01,420 (trainer:816) INFO: 32epoch:train:1401-1500batch: iter_time=9.979e-05, forward_time=0.244, class_loss=1.106, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.226, accuracy=0.957, backward_time=0.875, grad_norm=36.384, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=2.896e-07, train_time=4.559
+[gpue01] 2025-06-02 11:56:54,340 (trainer:816) INFO: 32epoch:train:1501-1600batch: iter_time=9.456e-05, forward_time=0.253, class_loss=0.688, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.012, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.012, geo_loss_all=0.102, loss=0.143, accuracy=0.973, backward_time=0.855, grad_norm=53.800, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=2.869e-07, train_time=4.516
+[gpue01] 2025-06-02 11:58:55,292 (trainer:816) INFO: 32epoch:train:1601-1700batch: iter_time=1.007e-04, forward_time=0.249, class_loss=1.404, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.286, accuracy=0.938, backward_time=0.940, grad_norm=57.055, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=2.843e-07, train_time=4.837
+[gpue01] 2025-06-02 12:00:48,312 (trainer:816) INFO: 32epoch:train:1701-1800batch: iter_time=1.005e-04, forward_time=0.230, class_loss=1.274, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.260, accuracy=0.943, backward_time=0.879, grad_norm=44.005, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=2.817e-07, train_time=4.520
+[gpue01] 2025-06-02 12:02:52,056 (trainer:816) INFO: 32epoch:train:1801-1900batch: iter_time=1.037e-04, forward_time=0.257, class_loss=0.952, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.195, accuracy=0.962, backward_time=0.960, grad_norm=41.989, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=2.791e-07, train_time=4.949
+[gpue01] 2025-06-02 12:04:49,603 (trainer:816) INFO: 32epoch:train:1901-2000batch: iter_time=1.031e-04, forward_time=0.260, class_loss=0.932, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.192, accuracy=0.965, backward_time=0.894, grad_norm=54.873, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=2.765e-07, train_time=4.701
+[gpue01] 2025-06-02 12:28:09,414 (trainer:401) INFO: 32epoch results: [train] iter_time=2.404e-04, forward_time=0.279, class_loss=1.117, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.229, accuracy=0.955, backward_time=0.894, grad_norm=54.010, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.023e-07, train_time=4.776, time=42 minutes and 7.83 seconds, total_count=64000, gpu_max_cached_mem_GB=82.354, [valid] class_loss=2.619, geo_loss_downstream=0.221, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.141, loss=2.123, accuracy=0.895, time=23 minutes and 19.61 seconds, total_count=151104, gpu_max_cached_mem_GB=82.354
+[gpue01] 2025-06-02 12:28:22,860 (trainer:467) INFO: There are no improvements in this epoch
+[gpue01] 2025-06-02 12:28:22,878 (trainer:335) INFO: 33/50epoch started. Estimated time to finish: 19 hours, 51 minutes and 48 seconds
+[gpue01] 2025-06-02 12:32:47,324 (trainer:816) INFO: 33epoch:train:1-100batch: iter_time=0.002, forward_time=0.393, class_loss=1.030, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.211, accuracy=0.962, backward_time=0.823, grad_norm=47.793, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.740e-07, train_time=4.976
+[gpue01] 2025-06-02 12:35:01,310 (trainer:816) INFO: 33epoch:train:101-200batch: iter_time=9.706e-05, forward_time=0.376, class_loss=1.497, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.305, accuracy=0.933, backward_time=0.945, grad_norm=58.238, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.715e-07, train_time=5.359
+[gpue01] 2025-06-02 12:37:11,584 (trainer:816) INFO: 33epoch:train:201-300batch: iter_time=1.017e-04, forward_time=0.353, class_loss=0.605, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.126, accuracy=0.980, backward_time=0.929, grad_norm=49.261, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.690e-07, train_time=5.210
+[gpue01] 2025-06-02 12:39:22,119 (trainer:816) INFO: 33epoch:train:301-400batch: iter_time=1.060e-04, forward_time=0.338, class_loss=1.064, geo_loss_downstream=0.160, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.218, accuracy=0.960, backward_time=0.948, grad_norm=72.623, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.036, optim0_lr0=2.665e-07, train_time=5.221
+[gpue01] 2025-06-02 12:41:30,505 (trainer:816) INFO: 33epoch:train:401-500batch: iter_time=1.061e-04, forward_time=0.305, class_loss=1.219, geo_loss_downstream=0.160, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.249, accuracy=0.950, backward_time=0.960, grad_norm=49.901, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.036, optim0_lr0=2.641e-07, train_time=5.135
+[gpue01] 2025-06-02 12:43:31,704 (trainer:816) INFO: 33epoch:train:501-600batch: iter_time=1.046e-04, forward_time=0.294, class_loss=1.601, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.325, accuracy=0.937, backward_time=0.898, grad_norm=77.095, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.617e-07, train_time=4.847
+[gpue01] 2025-06-02 12:45:34,686 (trainer:816) INFO: 33epoch:train:601-700batch: iter_time=9.555e-05, forward_time=0.290, class_loss=1.288, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.263, accuracy=0.950, backward_time=0.920, grad_norm=59.251, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.593e-07, train_time=4.918
+[gpue01] 2025-06-02 12:47:38,091 (trainer:816) INFO: 33epoch:train:701-800batch: iter_time=9.700e-05, forward_time=0.266, class_loss=1.299, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.265, accuracy=0.953, backward_time=0.948, grad_norm=62.744, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.569e-07, train_time=4.935
+[gpue01] 2025-06-02 12:49:30,107 (trainer:816) INFO: 33epoch:train:801-900batch: iter_time=1.092e-04, forward_time=0.250, class_loss=1.356, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.276, accuracy=0.945, backward_time=0.850, grad_norm=72.159, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.545e-07, train_time=4.480
+[gpue01] 2025-06-02 12:51:37,122 (trainer:816) INFO: 33epoch:train:901-1000batch: iter_time=1.187e-04, forward_time=0.278, class_loss=1.025, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.210, accuracy=0.962, backward_time=0.972, grad_norm=63.217, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.522e-07, train_time=5.080
+[gpue01] 2025-06-02 12:53:44,686 (trainer:816) INFO: 33epoch:train:1001-1100batch: iter_time=1.035e-04, forward_time=0.249, class_loss=1.190, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.243, accuracy=0.955, backward_time=1.007, grad_norm=72.056, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.034, optim0_lr0=2.499e-07, train_time=5.102
+[gpue01] 2025-06-02 12:55:38,967 (trainer:816) INFO: 33epoch:train:1101-1200batch: iter_time=1.035e-04, forward_time=0.249, class_loss=0.918, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.189, accuracy=0.968, backward_time=0.873, grad_norm=51.495, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.034, optim0_lr0=2.476e-07, train_time=4.570
+[gpue01] 2025-06-02 12:57:49,673 (trainer:816) INFO: 33epoch:train:1201-1300batch: iter_time=1.087e-04, forward_time=0.276, class_loss=0.972, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.200, accuracy=0.958, backward_time=1.010, grad_norm=58.794, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.453e-07, train_time=5.227
+[gpue01] 2025-06-02 12:59:38,197 (trainer:816) INFO: 33epoch:train:1301-1400batch: iter_time=1.066e-04, forward_time=0.231, class_loss=1.716, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.348, accuracy=0.918, backward_time=0.833, grad_norm=63.048, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.034, optim0_lr0=2.431e-07, train_time=4.340
+[gpue01] 2025-06-02 13:01:29,801 (trainer:816) INFO: 33epoch:train:1401-1500batch: iter_time=1.127e-04, forward_time=0.251, class_loss=1.037, geo_loss_downstream=0.160, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.101, loss=0.213, accuracy=0.957, backward_time=0.845, grad_norm=49.476, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.409e-07, train_time=4.463
+[gpue01] 2025-06-02 13:03:25,989 (trainer:816) INFO: 33epoch:train:1501-1600batch: iter_time=1.010e-04, forward_time=0.253, class_loss=1.036, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.212, accuracy=0.957, backward_time=0.887, grad_norm=52.597, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.034, optim0_lr0=2.387e-07, train_time=4.647
+[gpue01] 2025-06-02 13:05:27,481 (trainer:816) INFO: 33epoch:train:1601-1700batch: iter_time=9.690e-05, forward_time=0.251, class_loss=1.181, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.241, accuracy=0.950, backward_time=0.944, grad_norm=62.438, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.365e-07, train_time=4.859
+[gpue01] 2025-06-02 13:07:19,919 (trainer:816) INFO: 33epoch:train:1701-1800batch: iter_time=9.964e-05, forward_time=0.227, class_loss=0.954, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.196, accuracy=0.963, backward_time=0.877, grad_norm=71.114, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.343e-07, train_time=4.497
+[gpue01] 2025-06-02 13:09:14,103 (trainer:816) INFO: 33epoch:train:1801-1900batch: iter_time=9.797e-05, forward_time=0.250, class_loss=1.066, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.218, accuracy=0.957, backward_time=0.871, grad_norm=49.760, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.321e-07, train_time=4.567
+[gpue01] 2025-06-02 13:11:13,214 (trainer:816) INFO: 33epoch:train:1901-2000batch: iter_time=9.810e-05, forward_time=0.248, class_loss=0.987, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.203, accuracy=0.958, backward_time=0.923, grad_norm=45.787, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.034, optim0_lr0=2.300e-07, train_time=4.764
+[gpue01] 2025-06-02 13:34:34,616 (trainer:401) INFO: 33epoch results: [train] iter_time=2.171e-04, forward_time=0.281, class_loss=1.152, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.236, accuracy=0.954, backward_time=0.913, grad_norm=59.442, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.514e-07, train_time=4.860, time=42 minutes and 50.54 seconds, total_count=66000, gpu_max_cached_mem_GB=82.354, [valid] class_loss=2.631, geo_loss_downstream=0.208, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.133, loss=2.131, accuracy=0.893, time=23 minutes and 21.19 seconds, total_count=155826, gpu_max_cached_mem_GB=82.354
+[gpue01] 2025-06-02 13:34:48,118 (trainer:467) INFO: There are no improvements in this epoch
+[gpue01] 2025-06-02 13:34:48,138 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/32epoch.pth
+[gpue01] 2025-06-02 13:34:48,138 (trainer:335) INFO: 34/50epoch started. Estimated time to finish: 18 hours, 46 minutes and 18.15 seconds
+[gpue01] 2025-06-02 13:39:13,564 (trainer:816) INFO: 34epoch:train:1-100batch: iter_time=0.003, forward_time=0.403, class_loss=1.001, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.205, accuracy=0.962, backward_time=0.814, grad_norm=34.245, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.036, optim0_lr0=2.279e-07, train_time=4.988

exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.log

@@ -0,0 +1,388 @@
+# python3 -m espnet2.bin.lid_train --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True 
+# Started at Wed Jun  4 20:37:36 CDT 2025
+#
+/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_train.py --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True
+/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
+  torchaudio.set_audio_backend("sox_io")
+[gpue03] 2025-06-04 20:38:25,336 (abs_task:1420) INFO: pytorch.version=2.4.0+cu118, cuda.available=True, cudnn.version=90100, cudnn.benchmark=True, cudnn.deterministic=False
+[gpue03] 2025-06-04 20:38:25,343 (abs_task:1421) INFO: Model structure:
+ESPnetLIDUpstreamConditionModel(
+  (frontend): S3prlFrontendCondition(
+    (upstream): S3PRLUpstreamCondition(
+      (upstream): UpstreamExpertCondition(
+        (model): Wav2Vec2ModelCondition(
+          (feature_extractor): Wav2Vec2FeatureEncoder(
+            (conv_layers): ModuleList(
+              (0): Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (1-4): 4 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+              (5-6): 2 x Wav2Vec2LayerNormConvLayer(
+                (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
+                (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+                (activation): GELUActivation()
+              )
+            )
+          )
+          (feature_projection): Wav2Vec2FeatureProjection(
+            (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
+            (projection): Linear(in_features=512, out_features=1280, bias=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+          )
+          (encoder): Wav2Vec2EncoderCondition(
+            (pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
+              (conv): ParametrizedConv1d(
+                1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
+                (parametrizations): ModuleDict(
+                  (weight): ParametrizationList(
+                    (0): _WeightNorm()
+                  )
+                )
+              )
+              (padding): Wav2Vec2SamePadLayer()
+              (activation): GELUActivation()
+            )
+            (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+            (dropout): Dropout(p=0.1, inplace=False)
+            (layers): ModuleList(
+              (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm(
+                (attention): Wav2Vec2SdpaAttention(
+                  (k_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (v_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (q_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                  (out_proj): Linear(in_features=1280, out_features=1280, bias=True)
+                )
+                (dropout): Dropout(p=0.1, inplace=False)
+                (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+                (feed_forward): Wav2Vec2FeedForward(
+                  (intermediate_dropout): Dropout(p=0.0, inplace=False)
+                  (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
+                  (intermediate_act_fn): GELUActivation()
+                  (output_dense): Linear(in_features=5120, out_features=1280, bias=True)
+                  (output_dropout): Dropout(p=0.1, inplace=False)
+                )
+                (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
+              )
+            )
+            (ecapa_encoder): ModuleDict(
+              (32): IdentityEncoder()
+              (36): IdentityEncoder()
+              (40): IdentityEncoder()
+              (44): IdentityEncoder()
+            )
+            (pooling): ModuleDict(
+              (32): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (36): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (40): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+              (44): ChnAttnStatPooling(
+                (attention): Sequential(
+                  (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
+                  (1): ReLU()
+                  (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                  (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
+                )
+                (softmax): Softmax(dim=2)
+              )
+            )
+            (projector): ModuleDict(
+              (32): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (36): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (40): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+              (44): RawNet3Projector(
+                (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+                (fc): Linear(in_features=2560, out_features=192, bias=True)
+              )
+            )
+            (lang2vec_head): ModuleDict(
+              (32): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (36): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (40): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (44): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+            )
+            (aamsoftmax_weight): ParameterDict()
+            (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True)
+            (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec(
+              (ce): CrossEntropyLoss()
+              (lang2vec_head): Sequential(
+                (0): Linear(in_features=192, out_features=299, bias=True)
+              )
+              (lang2vec_loss): MSELoss()
+            )
+          )
+        )
+      )
+    )
+    (featurizer): Featurizer()
+  )
+  (normalize): UtteranceMVN(norm_means=True, norm_vars=False)
+  (encoder): EcapaTdnnEncoder(
+    (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
+    (relu): ReLU()
+    (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (layer1): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer2): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer3): EcapaBlock(
+      (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (convs): ModuleList(
+        (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
+      )
+      (bns): ModuleList(
+        (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      )
+      (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
+      (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (relu): ReLU()
+      (se): SEModule(
+        (se): Sequential(
+          (0): AdaptiveAvgPool1d(output_size=1)
+          (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
+          (2): ReLU()
+          (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+          (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
+          (5): Sigmoid()
+        )
+      )
+    )
+    (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
+    (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
+  )
+  (pooling): ChnAttnStatPooling(
+    (attention): Sequential(
+      (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
+      (1): ReLU()
+      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+      (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
+    )
+    (softmax): Softmax(dim=2)
+  )
+  (projector): RawNet3Projector(
+    (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+    (fc): Linear(in_features=3072, out_features=192, bias=True)
+  )
+  (loss): AAMSoftmaxSCTopKLang2Vec(
+    (ce): CrossEntropyLoss()
+    (lang2vec_head): Sequential(
+      (0): Linear(in_features=192, out_features=299, bias=True)
+    )
+    (lang2vec_loss): MSELoss()
+  )
+)
+
+Model summary:
+    Class Name: ESPnetLIDUpstreamConditionModel
+    Total Number of model parameters: 977.14 M
+    Number of trainable parameters: 977.14 M (100.0%)
+    Size: 3.91 GB
+    Type: torch.float32
+[gpue03] 2025-06-04 20:38:25,343 (abs_task:1424) INFO: Optimizer:
+Adam (
+Parameter Group 0
+    amsgrad: False
+    betas: [0.9, 0.98]
+    capturable: False
+    differentiable: False
+    eps: 1e-08
+    foreach: None
+    fused: None
+    initial_lr: 1e-05
+    lr: 6.0032e-06
+    maximize: False
+    weight_decay: 0
+)
+[gpue03] 2025-06-04 20:38:25,343 (abs_task:1425) INFO: Scheduler: TristageLR(warmup_steps=1250)(hold_steps=5000)(decay_steps=6250)(init_lr_scale=0.6)(final_lr_scale=0.1)(decay_factor=0.00036841361487904725)
+[gpue03] 2025-06-04 20:38:25,349 (abs_task:1434) INFO: Saving the configuration in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/config.yaml
+[gpue03] 2025-06-04 20:38:25,625 (preprocessor:2245) INFO: Using lang2vec geo
+[gpue03] 2025-06-04 20:38:41,537 (abs_task:1899) WARNING: Reading dump/raw/train_all_no_filter_lang/category2utt
+[gpue03] 2025-06-04 20:38:41,539 (abs_task:1946) WARNING: Reading dump/raw/train_all_no_filter_lang/dataset2utt
+[gpue03] 2025-06-04 20:38:41,540 (abs_task:1962) WARNING: Reading dump/raw/train_all_no_filter_lang/utt2dataset
+[gpue03] 2025-06-04 20:40:59,199 (abs_task:1997) INFO: [train] dataset:
+ESPnetDataset(
+  speech: {"path": "dump/raw/train_all_no_filter_lang/wav.scp", "type": "sound"}
+  lid_labels: {"path": "dump/raw/train_all_no_filter_lang/utt2spk", "type": "text"}
+  preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False))
+[gpue03] 2025-06-04 20:40:59,199 (abs_task:1998) INFO: [train] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)
+[gpue03] 2025-06-04 20:40:59,200 (abs_task:1999) INFO: [train] collate_fn: <class 'espnet2.train.collate_fn.CommonCollateFn'>(float_pad_value=0.0, int_pad_value=0.0)
+[gpue03] 2025-06-04 20:40:59,200 (abs_task:2000) INFO: [train] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=727460, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3)
+[gpue03] 2025-06-04 20:40:59,266 (abs_task:2001) INFO: [train] mini-batch sizes summary: N-batch=727460, mean=6.0, min=1, max=6
+[gpue03] 2025-06-04 20:40:59,684 (preprocessor:2245) INFO: Using lang2vec geo
+[gpue03] 2025-06-04 20:41:12,217 (abs_task:1899) WARNING: Reading dump/raw/dev_ml_superb2_lang/category2utt
+[gpue03] 2025-06-04 20:41:12,219 (abs_task:1946) WARNING: Reading dump/raw/dev_ml_superb2_lang/dataset2utt
+[gpue03] 2025-06-04 20:41:12,221 (abs_task:1962) WARNING: Reading dump/raw/dev_ml_superb2_lang/utt2dataset
+[gpue03] 2025-06-04 20:41:13,249 (abs_task:1997) INFO: [valid] dataset:
+ESPnetDataset(
+  speech: {"path": "dump/raw/dev_ml_superb2_lang/wav.scp", "type": "sound"}
+  lid_labels: {"path": "dump/raw/dev_ml_superb2_lang/utt2spk", "type": "text"}
+  preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False))
+[gpue03] 2025-06-04 20:41:13,249 (abs_task:1998) INFO: [valid] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)
+[gpue03] 2025-06-04 20:41:13,249 (abs_task:1999) INFO: [valid] collate_fn: <class 'espnet2.train.collate_fn.CommonCollateFn'>(float_pad_value=0.0, int_pad_value=0.0)
+[gpue03] 2025-06-04 20:41:13,249 (abs_task:2000) INFO: [valid] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=4722, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3)
+[gpue03] 2025-06-04 20:41:13,250 (abs_task:2001) INFO: [valid] mini-batch sizes summary: N-batch=4722, mean=6.0, min=4, max=6
+wandb: Currently logged in as: qingzhew (qingzhew-carnegie-mellon-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
+wandb: Tracking run with wandb version 0.19.10
+wandb: Run data is saved locally in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_204114-htm68ys8
+wandb: Run `wandb offline` to turn off syncing.
+wandb: Syncing run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch
+wandb: ⭐️ View project at https://wandb.ai/qingzhew-carnegie-mellon-university/lid
+wandb: 🚀 View run at https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/htm68ys8
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:218: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  scaler = GradScaler()
+/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:159: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  states = torch.load(
+[gpue03] 2025-06-04 20:41:24,303 (trainer:176) INFO: The training was resumed using exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/checkpoint.pth
+[gpue03] 2025-06-04 20:41:24,647 (trainer:251) INFO: Frontend featurizer weights for each layer: 
+Parameter containing:
+tensor([-0.0056, -0.0141, -0.0168, -0.0187, -0.0203, -0.0225, -0.0231, -0.0246,
+        -0.0253, -0.0252, -0.0254, -0.0241, -0.0226, -0.0200, -0.0162, -0.0120,
+        -0.0095, -0.0059, -0.0017,  0.0058,  0.0097,  0.0142,  0.0175,  0.0196,
+         0.0211,  0.0224,  0.0228,  0.0230,  0.0226,  0.0224,  0.0215,  0.0210,
+         0.0196,  0.0176,  0.0157,  0.0126,  0.0095,  0.0070,  0.0051,  0.0037,
+         0.0020, -0.0003, -0.0030, -0.0056, -0.0076, -0.0090, -0.0096, -0.0102,
+        -0.0102], device='cuda:0', requires_grad=True)
+[gpue03] 2025-06-04 20:41:24,648 (trainer:267) INFO: Error: 'Linear' object is not subscriptable
+[gpue03] 2025-06-04 20:41:24,648 (trainer:272) INFO: cos_mp: 1.0
+[gpue03] 2025-06-04 20:41:24,648 (trainer:273) INFO: easy_margin: False
+[gpue03] 2025-06-04 20:41:24,648 (trainer:281) WARNING: The training has already reached at max_epoch: 34
+[gpue03] 2025-06-04 20:41:24,659 (trainer:541) INFO: The training was finished at 33 epochs 
+[gpue03] 2025-06-04 20:41:24,660 (average_nbest_models:69) INFO: Averaging 2best models: criterion="valid.accuracy": exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/valid.accuracy.ave_2best.pth
+/work/nvme/bbjs/qwang20/espnet/espnet2/main_funcs/average_nbest_models.py:77: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  _loaded[e] = torch.load(
+[gpue03] 2025-06-04 20:41:30,224 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.32.attention.2.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,225 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.36.attention.2.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,225 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.40.attention.2.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,226 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.44.attention.2.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,227 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.32.bn.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,227 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.36.bn.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,227 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.40.bn.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,228 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.44.bn.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,230 (average_nbest_models:96) INFO: Accumulating encoder.bn.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn1.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.0.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.1.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.2.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.3.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.4.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.5.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,232 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.6.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,232 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn3.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,232 (average_nbest_models:96) INFO: Accumulating encoder.layer1.se.se.3.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,232 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn1.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.0.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.1.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.2.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.3.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.4.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.5.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.6.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn3.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.se.se.3.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn1.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.0.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.1.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.2.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.3.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.4.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.5.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.6.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,235 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn3.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,235 (average_nbest_models:96) INFO: Accumulating encoder.layer3.se.se.3.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,237 (average_nbest_models:96) INFO: Accumulating pooling.attention.2.num_batches_tracked instead of averaging
+[gpue03] 2025-06-04 20:41:30,237 (average_nbest_models:96) INFO: Accumulating projector.bn.num_batches_tracked instead of averaging
+wandb:                                                                                
+wandb: 🚀 View run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/htm68ys8
+wandb: ⭐️ View project at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid
+wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
+wandb: Find logs at: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_204114-htm68ys8/logs
+# Accounting: time=240 threads=1
+# Ended (code 0) at Wed Jun  4 20:41:36 CDT 2025, elapsed time 240 seconds