diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/lid_inference_test.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/lid_inference_test.log new file mode 100644 index 0000000000000000000000000000000000000000..bd0aa4d0899bd08f066567e940b6644d0705dc5a --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/lid_inference_test.log @@ -0,0 +1,300 @@ +# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +# Started at Mon Jun 2 02:37:15 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +[gpue04] 2025-06-02 02:37:35,038 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + torch.load(model_file, map_location=device), +[gpue04] 2025-06-02 02:37:46,607 (lid_inference_dist:86) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary. + warnings.warn(_create_warning_msg( +/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist. + warnings.warn( +[gpue04] 2025-06-02 02:37:47,156 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/lids0 +[gpue04] 2025-06-02 02:38:41,828 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0 +[gpue04] 2025-06-02 02:39:27,483 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1 +[gpue04] 2025-06-02 02:40:15,909 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2 +[gpue04] 2025-06-02 02:41:08,571 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3 +[gpue04] 2025-06-02 02:41:56,182 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4 +[gpue04] 2025-06-02 02:42:40,736 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5 +[gpue04] 2025-06-02 02:43:27,814 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6 +[gpue04] 2025-06-02 02:44:10,740 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7 +[gpue04] 2025-06-02 02:44:52,065 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8 +[gpue04] 2025-06-02 02:45:40,635 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9 +[gpue04] 2025-06-02 02:46:28,394 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10 +[gpue04] 2025-06-02 02:47:09,502 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11 +[gpue04] 2025-06-02 02:47:59,978 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12 +[gpue04] 2025-06-02 02:48:52,866 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13 +[gpue04] 2025-06-02 02:49:41,279 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14 +[gpue04] 2025-06-02 02:50:32,817 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15 +[gpue04] 2025-06-02 02:51:20,444 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 16 +[gpue04] 2025-06-02 02:52:09,714 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 17 +[gpue04] 2025-06-02 02:52:55,108 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 18 +[gpue04] 2025-06-02 02:53:50,212 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 19 +[gpue04] 2025-06-02 02:54:31,533 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 20 +[gpue04] 2025-06-02 02:55:19,223 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True +[gpue04] 2025-06-02 02:55:19,224 (lid_inference_dist:215) INFO: args.save_tsne_plot: False +# Accounting: time=1085 threads=1 +# Ended (code 0) at Mon Jun 2 02:55:20 CDT 2025, elapsed time 1085 seconds diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/results b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/results new file mode 100644 index 0000000000000000000000000000000000000000..856da80d836e9556be06cec0df190dbdc67a5912 --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/results @@ -0,0 +1,1039 @@ +Accuracy: 95.38% +Macro Accuracy: 95.44% +Accuracy per Language: +amh: 95.81% +tam: 96.86% +luo: 95.22% +tpi: 88.69% +ibo: 96.11% +ben: 92.18% +lao: 97.22% +swa: 90.31% +asm: 97.21% +gug: 95.29% +kaz: 94.59% +pus: 98.80% +tgl: 95.10% +hat: 97.81% +jav: 91.76% +zul: 97.44% +vie: 96.39% +kmr: 94.60% +kat: 96.10% +tur: 98.39% +ceb: 91.89% +yue: 100.00% +khk: 94.71% +lit: 98.77% +tel: 94.88% +Key: amh_18766_A_20140725_193025_031291, Target: amh, Predicted: tpi +Key: amh_16601_A_20140616_191918_057010, Target: amh, Predicted: tel +Key: amh_19621_B_20140517_232031_046443, Target: amh, Predicted: pus +Key: amh_19782_A_20140702_230513_056385, Target: amh, Predicted: tel +Key: amh_41741_A_20140422_000845_000000, Target: amh, Predicted: kat +Key: amh_46625_B_20140414_224528_000000, Target: amh, Predicted: tgl +Key: amh_46625_B_20140414_224528_011634, Target: amh, Predicted: tel +Key: amh_47799_A_20140902_200301_004751, Target: amh, Predicted: khk +Key: amh_41741_A_20140422_000845_021144, Target: amh, Predicted: ben +Key: amh_44961_A_20140421_215913_034469, Target: amh, Predicted: tel +Key: amh_42883_A_20140823_230118_001930, Target: amh, Predicted: asm +Key: amh_44961_A_20140421_215913_040626, Target: amh, Predicted: tur +Key: amh_44961_A_20140421_215913_048405, Target: amh, Predicted: tel +Key: amh_47799_A_20140902_200301_028605, Target: amh, Predicted: ibo +Key: amh_61011_B_20140415_180846_022820, Target: amh, Predicted: tpi +Key: amh_60498_A_20140823_192847_039762, Target: amh, Predicted: kaz +Key: amh_69633_A_20140607_233440_058823, Target: amh, Predicted: tgl +Key: amh_69633_A_20140607_233440_001199, Target: amh, Predicted: tgl +Key: amh_64870_A_20140518_011602_000000, Target: amh, Predicted: pus +Key: amh_69633_A_20140607_233440_027832, Target: amh, Predicted: kaz +Key: amh_69633_A_20140607_233440_052352, Target: amh, Predicted: ceb +Key: amh_69633_A_20140607_233440_054267, Target: amh, Predicted: tel +Key: amh_85439_A_20140814_215435_004561, Target: amh, Predicted: kmr +Key: amh_81553_A_20140707_003952_001198, Target: amh, Predicted: kmr +Key: amh_73757_A_20140512_231155_058241, Target: amh, Predicted: ibo +Key: amh_89888_B_20140520_191659_037281, Target: amh, Predicted: ben +Key: amh_93320_A_20140823_214255_040946, Target: amh, Predicted: swa +Key: amh_93320_A_20140823_214255_050913, Target: amh, Predicted: tel +Key: amh_89888_B_20140520_191659_020529, Target: amh, Predicted: kaz +Key: amh_89888_B_20140520_191659_058819, Target: amh, Predicted: zul +Key: amh_95124_A_20140828_224047_058345, Target: amh, Predicted: khk +Key: amh_95124_A_20140828_224047_059534, Target: amh, Predicted: gug +Key: amh_95124_A_20140828_224047_022900, Target: amh, Predicted: kat +Key: amh_95124_A_20140828_224047_034153, Target: amh, Predicted: kat +Key: amh_94002_A_20140511_172143_000793, Target: amh, Predicted: tgl +Key: amh_95124_A_20140828_224047_038927, Target: amh, Predicted: lit +Key: amh_96940_B_20140901_181148_007703, Target: amh, Predicted: kaz +Key: amh_94237_A_20140814_181922_050462, Target: amh, Predicted: hat +Key: amh_94237_A_20140814_181922_051539, Target: amh, Predicted: tam +Key: amh_94237_A_20140814_181922_058366, Target: amh, Predicted: asm +Key: amh_94002_A_20140511_172143_023305, Target: amh, Predicted: tel +Key: amh_95124_A_20140828_224047_004869, Target: amh, Predicted: kat +Key: amh_96940_B_20140901_181148_039855, Target: amh, Predicted: kaz +Key: amh_98506_A_20140807_170934_060854, Target: amh, Predicted: kaz +Key: asm_34446_B_20120426_195519_020649, Target: asm, Predicted: ben +Key: asm_33969_B_20130123_165132_045069, Target: asm, Predicted: tam +Key: asm_33704_A_20130204_172729_034778, Target: asm, Predicted: tam +Key: asm_33704_A_20130204_172729_049460, Target: asm, Predicted: ben +Key: asm_43587_A_20120607_204145_034715, Target: asm, Predicted: tel +Key: asm_40385_A_20121224_164959_041220, Target: asm, Predicted: tel +Key: asm_40385_B_20121224_164959_020689, Target: asm, Predicted: tgl +Key: asm_46593_B_20121010_023019_043252, Target: asm, Predicted: ben +Key: asm_46593_B_20121010_023019_046408, Target: asm, Predicted: ben +Key: asm_47429_A_20130121_172000_012339, Target: asm, Predicted: vie +Key: asm_59544_B_20120401_222134_013082, Target: asm, Predicted: ben +Key: asm_80856_A_20120423_184225_031581, Target: asm, Predicted: ben +Key: asm_79519_B_20121008_214502_049044, Target: asm, Predicted: tgl +Key: asm_66668_B_20120409_185702_056020, Target: asm, Predicted: tel +Key: asm_80856_A_20120423_184225_058863, Target: asm, Predicted: tgl +Key: asm_87885_A_20121113_193407_023378, Target: asm, Predicted: ben +Key: asm_87885_A_20121113_193407_024567, Target: asm, Predicted: tel +Key: asm_87671_B_20120401_172420_054685, Target: asm, Predicted: ben +Key: asm_87885_A_20121113_193407_044881, Target: asm, Predicted: ben +Key: asm_87885_A_20121113_193407_007808, Target: asm, Predicted: kmr +Key: asm_87885_A_20121113_193407_014210, Target: asm, Predicted: kmr +Key: ben_10576_A_20111221_214850_004672, Target: ben, Predicted: asm +Key: ben_10576_A_20111221_214850_016340, Target: ben, Predicted: asm +Key: ben_10576_A_20111221_214850_030232, Target: ben, Predicted: tel +Key: ben_10576_A_20111221_214850_036179, Target: ben, Predicted: asm +Key: ben_10569_B_20111221_201913_002481, Target: ben, Predicted: asm +Key: ben_10576_A_20111221_214850_050139, Target: ben, Predicted: asm +Key: ben_24810_B_20120114_225518_016801, Target: ben, Predicted: yue +Key: ben_21203_A_20120523_225358_000338, Target: ben, Predicted: asm +Key: ben_21203_A_20120523_225358_012402, Target: ben, Predicted: asm +Key: ben_27912_B_20120123_185402_005366, Target: ben, Predicted: asm +Key: ben_27912_B_20120123_185402_013907, Target: ben, Predicted: lao +Key: ben_27912_B_20120123_185402_040188, Target: ben, Predicted: asm +Key: ben_38382_B_20120110_013824_008463, Target: ben, Predicted: tel +Key: ben_38382_B_20120110_013824_009617, Target: ben, Predicted: tel +Key: ben_38382_B_20120110_013824_015051, Target: ben, Predicted: asm +Key: ben_40114_A_20120122_183602_035788, Target: ben, Predicted: asm +Key: ben_40114_A_20120122_183602_049079, Target: ben, Predicted: asm +Key: ben_44799_B_20120131_222925_044707, Target: ben, Predicted: asm +Key: ben_40114_B_20120122_183602_002987, Target: ben, Predicted: tel +Key: ben_40114_B_20120122_183602_025113, Target: ben, Predicted: asm +Key: ben_50583_B_20120114_233345_025147, Target: ben, Predicted: tel +Key: ben_50910_B_20120122_001708_020387, Target: ben, Predicted: asm +Key: ben_50910_B_20120122_001708_050086, Target: ben, Predicted: gug +Key: ben_44799_A_20120131_222925_022756, Target: ben, Predicted: asm +Key: ben_44799_A_20120131_222925_025503, Target: ben, Predicted: asm +Key: ben_44799_A_20120131_222925_031017, Target: ben, Predicted: asm +Key: ben_53805_B_20120126_211949_044950, Target: ben, Predicted: tel +Key: ben_62169_A_20120304_153842_051418, Target: ben, Predicted: asm +Key: ben_53805_B_20120126_211949_048578, Target: ben, Predicted: asm +Key: ben_53805_B_20120126_211949_054532, Target: ben, Predicted: tel +Key: ben_57721_A_20120531_194610_023753, Target: ben, Predicted: asm +Key: ben_52845_B_20120126_200807_030406, Target: ben, Predicted: asm +Key: ben_52845_B_20120126_200807_034210, Target: ben, Predicted: asm +Key: ben_62038_B_20111230_004215_016225, Target: ben, Predicted: yue +Key: ben_53805_A_20120126_211949_037154, Target: ben, Predicted: yue +Key: ben_62169_A_20120304_153842_019993, Target: ben, Predicted: asm +Key: ben_63220_A_20120514_232049_025353, Target: ben, Predicted: asm +Key: ben_62169_A_20120304_153842_024495, Target: ben, Predicted: asm +Key: ben_62169_A_20120304_153842_027469, Target: ben, Predicted: asm +Key: ben_62169_A_20120304_153842_039193, Target: ben, Predicted: asm +Key: ben_62169_A_20120304_153842_041729, Target: ben, Predicted: asm +Key: ben_63220_B_20120514_232049_020791, Target: ben, Predicted: asm +Key: ben_63220_B_20120514_232049_021921, Target: ben, Predicted: asm +Key: ben_65895_A_20120229_202918_036080, Target: ben, Predicted: tel +Key: ben_65895_A_20120229_202918_046912, Target: ben, Predicted: asm +Key: ben_66313_B_20120229_230907_037485, Target: ben, Predicted: tel +Key: ben_65895_A_20120229_202918_011382, Target: ben, Predicted: asm +Key: ben_80875_A_20120522_224314_028055, Target: ben, Predicted: asm +Key: ben_86207_B_20120127_145936_022109, Target: ben, Predicted: asm +Key: ben_80875_A_20120522_224314_031448, Target: ben, Predicted: asm +Key: ben_80875_A_20120522_224314_033805, Target: ben, Predicted: asm +Key: ben_80875_A_20120522_224314_037820, Target: ben, Predicted: asm +Key: ben_80875_A_20120522_224314_044450, Target: ben, Predicted: tel +Key: ben_80875_A_20120522_224314_050018, Target: ben, Predicted: asm +Key: ben_81773_B_20120101_024120_043949, Target: ben, Predicted: asm +Key: ben_80875_A_20120522_224314_011161, Target: ben, Predicted: tel +Key: ben_91275_A_20120529_195749_013758, Target: ben, Predicted: asm +Key: ben_91275_A_20120529_195749_014937, Target: ben, Predicted: asm +Key: ben_91275_A_20120529_195749_018687, Target: ben, Predicted: asm +Key: ben_91275_A_20120529_195749_023140, Target: ben, Predicted: tel +Key: ceb_15638_B_20131210_131327_018092, Target: ceb, Predicted: tgl +Key: ben_91275_A_20120529_195749_025289, Target: ben, Predicted: asm +Key: ben_93273_B_20120123_022109_041146, Target: ben, Predicted: tel +Key: ben_91275_A_20120529_195749_043069, Target: ben, Predicted: asm +Key: ben_95826_A_20120201_001701_020909, Target: ben, Predicted: yue +Key: ben_95826_B_20120201_001701_006650, Target: ben, Predicted: asm +Key: ceb_14141_B_20140118_202248_001941, Target: ceb, Predicted: lao +Key: ceb_14141_B_20140118_202248_008492, Target: ceb, Predicted: jav +Key: ceb_14141_B_20140118_202248_015284, Target: ceb, Predicted: amh +Key: ceb_14141_B_20140118_202248_016622, Target: ceb, Predicted: tgl +Key: ceb_15262_A_20131105_213812_038869, Target: ceb, Predicted: asm +Key: ceb_21109_A_20140102_180619_050237, Target: ceb, Predicted: tgl +Key: ceb_17881_B_20140122_201653_009579, Target: ceb, Predicted: tel +Key: ceb_22466_A_20131015_174457_021603, Target: ceb, Predicted: tgl +Key: ceb_17881_B_20140122_201653_034245, Target: ceb, Predicted: asm +Key: ceb_21109_A_20140102_180619_017721, Target: ceb, Predicted: jav +Key: ceb_22466_A_20131015_174457_022828, Target: ceb, Predicted: tgl +Key: ceb_21109_A_20140102_180619_018853, Target: ceb, Predicted: jav +Key: ceb_22466_A_20131015_174457_025528, Target: ceb, Predicted: tgl +Key: ceb_21109_A_20140102_180619_019994, Target: ceb, Predicted: jav +Key: ceb_22466_A_20131015_174457_031272, Target: ceb, Predicted: tgl +Key: ceb_22466_A_20131015_174457_033555, Target: ceb, Predicted: tgl +Key: ceb_21109_A_20140102_180619_024469, Target: ceb, Predicted: jav +Key: ceb_22466_A_20131015_174457_052722, Target: ceb, Predicted: tgl +Key: ceb_21109_A_20140102_180619_025655, Target: ceb, Predicted: jav +Key: ceb_22466_B_20131015_174457_001633, Target: ceb, Predicted: kmr +Key: ceb_21109_A_20140102_180619_031392, Target: ceb, Predicted: tgl +Key: ceb_22466_B_20131015_174457_045431, Target: ceb, Predicted: tgl +Key: ceb_22466_B_20131015_174457_051524, Target: ceb, Predicted: tgl +Key: ceb_21109_A_20140102_180619_040169, Target: ceb, Predicted: jav +Key: ceb_21109_A_20140102_180619_041294, Target: ceb, Predicted: jav +Key: ceb_21109_A_20140102_180619_042402, Target: ceb, Predicted: jav +Key: ceb_38340_B_20131128_145618_035396, Target: ceb, Predicted: asm +Key: ceb_38340_B_20131128_145618_044704, Target: ceb, Predicted: tgl +Key: ceb_36059_B_20140118_204512_003449, Target: ceb, Predicted: tgl +Key: ceb_38340_B_20131128_145618_050471, Target: ceb, Predicted: tgl +Key: ceb_38340_B_20131128_145618_001728, Target: ceb, Predicted: tgl +Key: ceb_38340_B_20131128_145618_028374, Target: ceb, Predicted: tgl +Key: ceb_43646_A_20131019_165638_004395, Target: ceb, Predicted: tgl +Key: ceb_50565_B_20131025_202729_012748, Target: ceb, Predicted: asm +Key: ceb_43646_A_20131019_165638_019162, Target: ceb, Predicted: tgl +Key: ceb_43646_A_20131019_165638_027625, Target: ceb, Predicted: asm +Key: ceb_51530_B_20140125_195307_042590, Target: ceb, Predicted: tgl +Key: ceb_51530_B_20140125_195307_043726, Target: ceb, Predicted: tgl +Key: ceb_51530_B_20140125_195307_055117, Target: ceb, Predicted: hat +Key: ceb_56370_A_20131101_175739_018773, Target: ceb, Predicted: lao +Key: ceb_56370_B_20131101_175739_043790, Target: ceb, Predicted: tgl +Key: ceb_54744_B_20131202_184432_002469, Target: ceb, Predicted: asm +Key: ceb_54744_B_20131202_184432_003641, Target: ceb, Predicted: asm +Key: ceb_60299_A_20140202_130806_026932, Target: ceb, Predicted: lao +Key: ceb_60299_A_20140202_130806_030919, Target: ceb, Predicted: tam +Key: ceb_54744_B_20131202_184432_014262, Target: ceb, Predicted: tgl +Key: ceb_60299_A_20140202_130806_047310, Target: ceb, Predicted: tgl +Key: ceb_54744_B_20131202_184432_036018, Target: ceb, Predicted: asm +Key: ceb_60299_A_20140202_130806_053018, Target: ceb, Predicted: asm +Key: ceb_54744_B_20131202_184432_044887, Target: ceb, Predicted: asm +Key: ceb_56370_A_20131101_175739_004673, Target: ceb, Predicted: tgl +Key: ceb_81427_A_20131126_151401_058032, Target: ceb, Predicted: tel +Key: ceb_84611_A_20131125_193454_001166, Target: ceb, Predicted: tgl +Key: ceb_79660_A_20140201_160331_000129, Target: ceb, Predicted: tgl +Key: ceb_74455_A_20140115_152935_051492, Target: ceb, Predicted: ben +Key: ceb_74455_B_20140115_152935_007394, Target: ceb, Predicted: tgl +Key: ceb_74455_B_20140115_152935_015341, Target: ceb, Predicted: tgl +Key: ceb_79660_A_20140201_160331_046537, Target: ceb, Predicted: vie +Key: ceb_86467_A_20131112_182159_030337, Target: ceb, Predicted: tgl +Key: ceb_86467_B_20131112_193636_008827, Target: ceb, Predicted: tgl +Key: ceb_86467_B_20131112_193636_017008, Target: ceb, Predicted: tgl +Key: ceb_85179_A_20131227_172225_003961, Target: ceb, Predicted: tgl +Key: ceb_96985_A_20131021_164130_003454, Target: ceb, Predicted: tgl +Key: ceb_96985_A_20131021_164130_042953, Target: ceb, Predicted: tam +Key: ceb_98489_A_20131123_233440_004829, Target: ceb, Predicted: tgl +Key: ceb_85179_A_20131227_172225_021268, Target: ceb, Predicted: lao +Key: gug_21624_A_20150222_054542_006999, Target: gug, Predicted: tel +Key: gug_21624_A_20150222_054542_008195, Target: gug, Predicted: tel +Key: gug_21624_A_20150222_054542_021054, Target: gug, Predicted: tel +Key: gug_21624_A_20150222_054542_023373, Target: gug, Predicted: tel +Key: gug_21004_B_20150217_083755_046019, Target: gug, Predicted: tpi +Key: gug_21004_B_20150217_083755_048475, Target: gug, Predicted: ibo +Key: gug_23006_A_20140807_062702_004252, Target: gug, Predicted: luo +Key: gug_39555_A_20141023_010258_027629, Target: gug, Predicted: lao +Key: gug_41685_A_20150320_083024_019491, Target: gug, Predicted: tel +Key: gug_41685_A_20150320_083024_050188, Target: gug, Predicted: tel +Key: gug_23006_B_20140807_062702_021561, Target: gug, Predicted: zul +Key: gug_43395_B_20150303_092614_017102, Target: gug, Predicted: vie +Key: gug_43395_B_20150303_092614_043917, Target: gug, Predicted: lao +Key: gug_50810_B_20140619_063147_011354, Target: gug, Predicted: lao +Key: gug_50810_B_20140619_063147_023949, Target: gug, Predicted: jav +Key: gug_44619_B_20140621_050143_005200, Target: gug, Predicted: lao +Key: gug_50810_B_20140619_063147_034848, Target: gug, Predicted: tur +Key: gug_50090_A_20150206_002321_001260, Target: gug, Predicted: tam +Key: gug_50090_B_20150206_002321_026694, Target: gug, Predicted: tur +Key: gug_56019_A_20150221_084856_048093, Target: gug, Predicted: tel +Key: gug_56019_A_20150221_084856_054819, Target: gug, Predicted: tel +Key: gug_56019_A_20150221_084856_012820, Target: gug, Predicted: tel +Key: gug_56019_A_20150221_084856_014014, Target: gug, Predicted: tel +Key: gug_53441_A_20140612_055846_030438, Target: gug, Predicted: tpi +Key: gug_56019_A_20150221_084856_016253, Target: gug, Predicted: tel +Key: gug_58717_A_20150201_022141_058962, Target: gug, Predicted: tur +Key: gug_56019_A_20150221_084856_036105, Target: gug, Predicted: tel +Key: gug_56019_A_20150221_084856_040658, Target: gug, Predicted: tel +Key: gug_78161_A_20150312_093559_034226, Target: gug, Predicted: khk +Key: gug_78161_A_20150312_093559_042472, Target: gug, Predicted: tel +Key: gug_78161_A_20150312_093559_047262, Target: gug, Predicted: tel +Key: gug_97911_A_20150304_082443_021658, Target: gug, Predicted: tel +Key: gug_97911_A_20150304_082443_026325, Target: gug, Predicted: khk +Key: gug_97911_A_20150304_082443_058612, Target: gug, Predicted: kmr +Key: gug_97911_A_20150304_082443_060424, Target: gug, Predicted: tel +Key: hat_14440_B_20130302_012105_008041, Target: hat, Predicted: ibo +Key: hat_14440_B_20130302_012105_047037, Target: hat, Predicted: kat +Key: hat_23983_B_20130503_023139_038952, Target: hat, Predicted: tpi +Key: hat_32832_A_20130430_060411_001029, Target: hat, Predicted: amh +Key: hat_49197_B_20130529_061436_045077, Target: hat, Predicted: nor +Key: hat_61357_B_20130602_030259_014385, Target: hat, Predicted: ibo +Key: hat_61357_B_20130602_030259_016440, Target: hat, Predicted: jav +Key: hat_61357_B_20130602_030259_019295, Target: hat, Predicted: lao +Key: hat_61357_B_20130602_030259_038622, Target: hat, Predicted: jav +Key: hat_61357_B_20130602_030259_044885, Target: hat, Predicted: lao +Key: hat_61357_B_20130602_030259_052961, Target: hat, Predicted: ibo +Key: hat_65640_B_20130429_103434_018865, Target: hat, Predicted: tpi +Key: hat_65640_B_20130429_103434_040586, Target: hat, Predicted: gug +Key: hat_71263_A_20130602_021725_030898, Target: hat, Predicted: ibo +Key: hat_77112_B_20130528_050544_000322, Target: hat, Predicted: swa +Key: hat_74226_B_20130303_125222_045352, Target: hat, Predicted: tpi +Key: hat_80881_A_20130220_022131_028792, Target: hat, Predicted: ibo +Key: hat_78360_B_20130430_101414_041610, Target: hat, Predicted: vie +Key: hat_80881_A_20130220_022131_034410, Target: hat, Predicted: gug +Key: hat_80881_A_20130220_022131_012911, Target: hat, Predicted: yue +Key: hat_79571_A_20130302_074959_009017, Target: hat, Predicted: amh +Key: hat_80881_A_20130220_022131_016364, Target: hat, Predicted: gug +Key: hat_81553_A_20130430_095301_044907, Target: hat, Predicted: gug +Key: ibo_13427_B_20140810_232413_045755, Target: ibo, Predicted: tpi +Key: ibo_19818_A_20140801_211130_040524, Target: ibo, Predicted: spa +Key: ibo_13427_A_20140810_232413_021572, Target: ibo, Predicted: hat +Key: ibo_33497_B_20140730_031414_000072, Target: ibo, Predicted: hat +Key: ibo_28419_B_20140606_201307_010615, Target: ibo, Predicted: luo +Key: ibo_35420_A_20140527_001314_003007, Target: ibo, Predicted: amh +Key: ibo_34197_A_20140520_215059_023638, Target: ibo, Predicted: tel +Key: ibo_35420_B_20140527_001314_032983, Target: ibo, Predicted: luo +Key: ibo_50726_A_20140521_235356_009208, Target: ibo, Predicted: tel +Key: ibo_50726_A_20140521_235356_011538, Target: ibo, Predicted: khk +Key: ibo_50726_A_20140521_235356_019433, Target: ibo, Predicted: luo +Key: ibo_50726_A_20140521_235356_022903, Target: ibo, Predicted: tel +Key: ibo_50726_A_20140521_235356_024051, Target: ibo, Predicted: kat +Key: ibo_50726_A_20140521_235356_031085, Target: ibo, Predicted: amh +Key: ibo_53842_A_20140905_005627_005670, Target: ibo, Predicted: swa +Key: ibo_50726_A_20140521_235356_032272, Target: ibo, Predicted: kaz +Key: ibo_50726_A_20140521_235356_033466, Target: ibo, Predicted: tam +Key: ibo_50726_A_20140521_235356_034611, Target: ibo, Predicted: kaz +Key: ibo_53842_A_20140905_005627_027190, Target: ibo, Predicted: tpi +Key: ibo_50726_A_20140521_235356_044616, Target: ibo, Predicted: kaz +Key: ibo_53842_A_20140905_005627_028328, Target: ibo, Predicted: zul +Key: ibo_52301_A_20140607_003158_025482, Target: ibo, Predicted: tel +Key: ibo_52301_A_20140607_003158_039732, Target: ibo, Predicted: lit +Key: ibo_50726_A_20140521_235356_053784, Target: ibo, Predicted: kat +Key: ibo_63334_A_20150216_005033_042571, Target: ibo, Predicted: hat +Key: ibo_63334_B_20150216_005033_011676, Target: ibo, Predicted: tpi +Key: ibo_63334_B_20150216_005033_016403, Target: ibo, Predicted: tpi +Key: ibo_58107_B_20140805_204322_048668, Target: ibo, Predicted: yue +Key: ibo_63334_B_20150216_005033_027618, Target: ibo, Predicted: tpi +Key: ibo_63334_B_20150216_005033_042556, Target: ibo, Predicted: tur +Key: ibo_60508_A_20140521_055301_003833, Target: ibo, Predicted: kat +Key: ibo_77112_B_20140609_224704_017697, Target: ibo, Predicted: swa +Key: ibo_77803_A_20140517_202422_000000, Target: ibo, Predicted: amh +Key: ibo_77803_A_20140517_202422_004727, Target: ibo, Predicted: luo +Key: ibo_66959_B_20141031_215547_046888, Target: ibo, Predicted: tpi +Key: ibo_79723_A_20150331_184104_029068, Target: ibo, Predicted: tpi +Key: ibo_79723_A_20150331_184104_039764, Target: ibo, Predicted: zul +Key: ibo_87280_A_20141026_002639_013843, Target: ibo, Predicted: lao +Key: ibo_87313_B_20140802_002411_026424, Target: ibo, Predicted: tpi +Key: ibo_94212_B_20140525_012758_040617, Target: ibo, Predicted: tgl +Key: jav_10184_A_20141119_194233_051384, Target: jav, Predicted: lao +Key: jav_10184_A_20141119_194233_017863, Target: jav, Predicted: ceb +Key: jav_10184_A_20141119_194233_059426, Target: jav, Predicted: lao +Key: jav_10184_A_20141119_194233_064088, Target: jav, Predicted: lao +Key: jav_15535_B_20150104_232347_044037, Target: jav, Predicted: lao +Key: jav_10184_A_20141119_194233_025595, Target: jav, Predicted: ceb +Key: jav_10184_A_20141119_194233_029118, Target: jav, Predicted: ceb +Key: jav_20133_B_20140911_170812_017218, Target: jav, Predicted: mlt +Key: jav_21581_A_20141107_151147_007012, Target: jav, Predicted: ceb +Key: jav_21393_B_20150304_163256_011005, Target: jav, Predicted: lao +Key: jav_23046_A_20141103_212247_000903, Target: jav, Predicted: msa +Key: jav_23505_A_20141029_003347_043611, Target: jav, Predicted: lao +Key: jav_23046_A_20141103_212247_032712, Target: jav, Predicted: vie +Key: jav_23046_A_20141103_212247_037678, Target: jav, Predicted: asm +Key: jav_23505_B_20141029_003347_024606, Target: jav, Predicted: luo +Key: jav_21807_A_20141125_194924_048994, Target: jav, Predicted: tur +Key: jav_27590_A_20141227_191710_047520, Target: jav, Predicted: tgl +Key: jav_27590_A_20141227_191710_050692, Target: jav, Predicted: cym +Key: jav_27590_A_20141227_191710_055289, Target: jav, Predicted: hat +Key: jav_36293_A_20141001_145552_001194, Target: jav, Predicted: ceb +Key: jav_36293_A_20141001_145552_002391, Target: jav, Predicted: ceb +Key: jav_36293_A_20141001_145552_003577, Target: jav, Predicted: ceb +Key: jav_36293_A_20141001_145552_004772, Target: jav, Predicted: ceb +Key: jav_36293_A_20141001_145552_005969, Target: jav, Predicted: ceb +Key: jav_27590_B_20141227_191710_048052, Target: jav, Predicted: tgl +Key: jav_36293_A_20141001_145552_012962, Target: jav, Predicted: ceb +Key: jav_36293_A_20141001_145552_016469, Target: jav, Predicted: lao +Key: jav_36293_A_20141001_145552_017668, Target: jav, Predicted: ceb +Key: jav_36293_A_20141001_145552_020039, Target: jav, Predicted: ceb +Key: jav_36505_A_20150106_201700_045871, Target: jav, Predicted: swa +Key: jav_36293_A_20141001_145552_022419, Target: jav, Predicted: tgl +Key: jav_41598_B_20150201_142509_000238, Target: jav, Predicted: tgl +Key: jav_36293_A_20141001_145552_023618, Target: jav, Predicted: ceb +Key: jav_36505_A_20150106_201700_053028, Target: jav, Predicted: lao +Key: jav_36293_B_20141001_145552_013357, Target: jav, Predicted: lao +Key: jav_36894_A_20140919_222930_000092, Target: jav, Predicted: ceb +Key: jav_36293_A_20141001_145552_028259, Target: jav, Predicted: ceb +Key: jav_36293_A_20141001_145552_030651, Target: jav, Predicted: ceb +Key: jav_41745_B_20141108_162338_035175, Target: jav, Predicted: sun +Key: jav_41745_B_20141108_162338_053557, Target: jav, Predicted: sun +Key: jav_41745_B_20141108_162338_055340, Target: jav, Predicted: ind +Key: jav_36293_A_20141001_145552_038873, Target: jav, Predicted: ceb +Key: jav_36505_A_20150106_201700_014759, Target: jav, Predicted: luo +Key: jav_36293_A_20141001_145552_043584, Target: jav, Predicted: ceb +Key: jav_36293_A_20141001_145552_045967, Target: jav, Predicted: ceb +Key: jav_36505_A_20150106_201700_033633, Target: jav, Predicted: ceb +Key: jav_49118_B_20150201_023112_044097, Target: jav, Predicted: tgl +Key: jav_52490_A_20140916_192446_040486, Target: jav, Predicted: gug +Key: jav_49437_B_20150112_204645_005926, Target: jav, Predicted: cym +Key: jav_52490_A_20140916_192446_052161, Target: jav, Predicted: kat +Key: jav_49437_B_20150112_204645_038219, Target: jav, Predicted: hat +Key: jav_52717_A_20140923_130849_023513, Target: jav, Predicted: khk +Key: jav_56306_A_20150103_203751_000250, Target: jav, Predicted: lao +Key: jav_52717_B_20140923_130849_020418, Target: jav, Predicted: lao +Key: jav_52717_B_20140923_130849_028193, Target: jav, Predicted: lao +Key: jav_65882_B_20141102_005627_039399, Target: jav, Predicted: tpi +Key: jav_65882_B_20141102_005627_041556, Target: jav, Predicted: tpi +Key: jav_64494_A_20141012_193548_027781, Target: jav, Predicted: asm +Key: jav_70386_B_20141116_170547_042186, Target: jav, Predicted: pus +Key: jav_73837_A_20141101_183259_039061, Target: jav, Predicted: khk +Key: jav_68289_B_20150216_010725_004241, Target: jav, Predicted: swa +Key: jav_68289_B_20150216_010725_012391, Target: jav, Predicted: gug +Key: jav_68289_B_20150216_010725_030183, Target: jav, Predicted: tgl +Key: jav_68289_B_20150216_010725_034736, Target: jav, Predicted: tgl +Key: jav_73511_A_20141226_133330_013131, Target: jav, Predicted: lao +Key: jav_70343_B_20150212_004248_014681, Target: jav, Predicted: khk +Key: jav_70386_B_20141116_170547_013684, Target: jav, Predicted: hat +Key: jav_78454_A_20141128_203259_000000, Target: jav, Predicted: amh +Key: jav_68068_B_20150119_135822_043484, Target: jav, Predicted: asm +Key: jav_70386_B_20141116_170547_028312, Target: jav, Predicted: asm +Key: jav_73837_A_20141101_183259_026069, Target: jav, Predicted: tgl +Key: jav_70386_B_20141116_170547_039873, Target: jav, Predicted: nep +Key: jav_68182_A_20150111_002528_041112, Target: jav, Predicted: vie +Key: jav_73837_A_20141101_183259_032808, Target: jav, Predicted: luo +Key: jav_86467_B_20140920_125939_040288, Target: jav, Predicted: kaz +Key: jav_88445_B_20141205_204305_027285, Target: jav, Predicted: tgl +Key: jav_82935_A_20150104_005835_023512, Target: jav, Predicted: asm +Key: jav_87921_B_20141225_203350_058462, Target: jav, Predicted: ces +Key: jav_89457_B_20141117_212710_047919, Target: jav, Predicted: tgl +Key: jav_89457_B_20141117_212710_051520, Target: jav, Predicted: lao +Key: jav_78604_A_20141031_181612_041553, Target: jav, Predicted: ceb +Key: jav_92176_A_20141222_021733_023517, Target: jav, Predicted: ceb +Key: jav_82935_B_20150104_005835_046292, Target: jav, Predicted: ceb +Key: jav_92176_A_20141222_021733_038411, Target: jav, Predicted: vie +Key: jav_92176_B_20141222_021733_005786, Target: jav, Predicted: tgl +Key: jav_92176_B_20141222_021733_007770, Target: jav, Predicted: tel +Key: jav_92176_B_20141222_021733_012229, Target: jav, Predicted: luo +Key: jav_92176_B_20141222_021733_035480, Target: jav, Predicted: swa +Key: jav_92176_B_20141222_021733_043027, Target: jav, Predicted: asm +Key: jav_92176_B_20141222_021733_046066, Target: jav, Predicted: sun +Key: jav_93632_B_20150119_150118_019742, Target: jav, Predicted: msa +Key: jav_93632_B_20150119_150118_049835, Target: jav, Predicted: tel +Key: jav_93632_B_20150119_150118_052713, Target: jav, Predicted: tgl +Key: kat_10184_A_20141107_212406_000114, Target: kat, Predicted: lit +Key: kat_17165_A_20141117_063008_033016, Target: kat, Predicted: tur +Key: kat_10184_A_20141107_212406_043262, Target: kat, Predicted: jav +Key: kat_16184_A_20141020_233508_031838, Target: kat, Predicted: hat +Key: kat_17472_A_20141201_023731_021216, Target: kat, Predicted: ceb +Key: kat_17472_A_20141201_023731_026158, Target: kat, Predicted: tur +Key: kat_17472_A_20141201_023731_033322, Target: kat, Predicted: sin +Key: kat_17472_A_20141201_023731_036303, Target: kat, Predicted: yue +Key: kat_17472_A_20141201_023731_038320, Target: kat, Predicted: tam +Key: kat_17472_A_20141201_023731_040410, Target: kat, Predicted: luo +Key: kat_18380_A_20141118_001754_037874, Target: kat, Predicted: tel +Key: kat_18380_A_20141118_001754_050009, Target: kat, Predicted: kaz +Key: kat_23239_A_20141127_054155_000001, Target: kat, Predicted: khk +Key: kat_35467_A_20141020_054030_002174, Target: kat, Predicted: ibo +Key: kat_38431_B_20141130_190122_043698, Target: kat, Predicted: khk +Key: kat_38431_B_20141130_190122_053025, Target: kat, Predicted: kaz +Key: kat_41592_A_20141117_033328_012603, Target: kat, Predicted: tpi +Key: kat_41592_A_20141117_033328_017235, Target: kat, Predicted: hat +Key: kat_41592_A_20141117_033328_024808, Target: kat, Predicted: hat +Key: kat_41592_A_20141117_033328_028265, Target: kat, Predicted: hat +Key: kat_41592_A_20141117_033328_030608, Target: kat, Predicted: vie +Key: kat_42600_A_20141029_174857_000524, Target: kat, Predicted: gug +Key: kat_41592_A_20141117_033328_037242, Target: kat, Predicted: swa +Key: kat_41592_B_20141117_033328_045799, Target: kat, Predicted: amh +Key: kat_41592_A_20141117_033328_041983, Target: kat, Predicted: pus +Key: kat_44619_A_20141028_234639_041015, Target: kat, Predicted: khk +Key: kat_44619_A_20141028_234639_055151, Target: kat, Predicted: khk +Key: kat_44619_A_20141028_234639_019716, Target: kat, Predicted: kaz +Key: kat_44619_A_20141028_234639_024432, Target: kat, Predicted: kmr +Key: kat_44619_A_20141028_234639_031498, Target: kat, Predicted: bre +Key: kat_47959_B_20141026_214447_024462, Target: kat, Predicted: ben +Key: kat_51955_A_20141024_012212_000000, Target: kat, Predicted: amh +Key: kat_56826_B_20141201_042429_027677, Target: kat, Predicted: kmr +Key: kat_61190_A_20141029_013447_018598, Target: kat, Predicted: kaz +Key: kat_61190_A_20141029_013447_034683, Target: kat, Predicted: khk +Key: kat_73757_A_20141117_025704_005504, Target: kat, Predicted: tur +Key: kat_73757_A_20141117_025704_008069, Target: kat, Predicted: tur +Key: kat_73757_A_20141117_025704_010415, Target: kat, Predicted: tur +Key: kat_73757_A_20141117_025704_019493, Target: kat, Predicted: kaz +Key: kat_73757_A_20141117_025704_020655, Target: kat, Predicted: tur +Key: kat_73757_A_20141117_025704_021743, Target: kat, Predicted: tur +Key: kat_73757_A_20141117_025704_028261, Target: kat, Predicted: tur +Key: kat_73757_A_20141117_025704_029368, Target: kat, Predicted: tur +Key: kat_73757_A_20141117_025704_030527, Target: kat, Predicted: tur +Key: kat_73757_A_20141117_025704_031627, Target: kat, Predicted: lit +Key: kat_74121_A_20141120_020705_056254, Target: kat, Predicted: tur +Key: kat_73757_A_20141117_025704_036336, Target: kat, Predicted: khk +Key: kat_73757_A_20141117_025704_042111, Target: kat, Predicted: kaz +Key: kat_73757_A_20141117_025704_050281, Target: kat, Predicted: tur +Key: kat_73757_A_20141117_025704_053596, Target: kat, Predicted: zul +Key: kat_81424_B_20141123_000421_002417, Target: kat, Predicted: ibo +Key: kat_80781_A_20141104_212234_029772, Target: kat, Predicted: khk +Key: kat_87298_A_20141025_213601_000000, Target: kat, Predicted: kaz +Key: kat_87313_A_20141119_014632_003672, Target: kat, Predicted: kaz +Key: kat_87313_A_20141119_014632_008923, Target: kat, Predicted: ceb +Key: kat_87298_A_20141025_213601_050430, Target: kat, Predicted: khk +Key: kat_87298_A_20141025_213601_053980, Target: kat, Predicted: tam +Key: kat_87313_A_20141119_014632_061503, Target: kat, Predicted: khk +Key: kat_87298_A_20141025_213601_055170, Target: kat, Predicted: kaz +Key: kat_87298_A_20141025_213601_056354, Target: kat, Predicted: jav +Key: kat_87313_B_20141119_014632_047555, Target: kat, Predicted: gug +Key: kat_87298_A_20141025_213601_034714, Target: kat, Predicted: khk +Key: kat_87298_A_20141025_213601_035904, Target: kat, Predicted: khk +Key: kat_88776_A_20141006_193621_056655, Target: kat, Predicted: zul +Key: kaz_20768_A_20140203_190423_002434, Target: kaz, Predicted: tur +Key: kaz_20768_A_20140203_190423_019244, Target: kaz, Predicted: tur +Key: kaz_17573_A_20140312_030325_027741, Target: kaz, Predicted: tpi +Key: kaz_20768_A_20140203_190423_026665, Target: kaz, Predicted: amh +Key: kaz_20768_A_20140203_190423_033715, Target: kaz, Predicted: tur +Key: kaz_20682_A_20140114_221052_048257, Target: kaz, Predicted: gug +Key: kaz_20768_A_20140203_190423_034895, Target: kaz, Predicted: tur +Key: kaz_20768_A_20140203_190423_035970, Target: kaz, Predicted: tur +Key: kaz_17914_A_20140126_234956_004076, Target: kaz, Predicted: tam +Key: kaz_20768_A_20140203_185125_012980, Target: kaz, Predicted: tur +Key: kaz_36669_B_20131206_164229_046083, Target: kaz, Predicted: tur +Key: kaz_33175_B_20131105_201906_003032, Target: kaz, Predicted: kmr +Key: kaz_44868_B_20131217_205716_001796, Target: kaz, Predicted: ibo +Key: kaz_44868_B_20131217_205716_004022, Target: kaz, Predicted: kmr +Key: kaz_23355_B_20140317_191841_029508, Target: kaz, Predicted: khk +Key: kaz_41174_B_20131212_200450_053004, Target: kaz, Predicted: khk +Key: kaz_23355_B_20140317_191841_053397, Target: kaz, Predicted: khk +Key: kaz_24589_A_20131129_215929_000010, Target: kaz, Predicted: lit +Key: kaz_70110_A_20131109_190313_007919, Target: kaz, Predicted: tur +Key: kaz_47156_A_20140313_011009_046718, Target: kaz, Predicted: vie +Key: kaz_72654_A_20131207_162604_000403, Target: kaz, Predicted: kmr +Key: kaz_50726_A_20131118_025621_023121, Target: kaz, Predicted: tur +Key: kaz_72654_A_20131207_162604_044645, Target: kaz, Predicted: kmr +Key: kaz_72654_B_20131207_162604_031261, Target: kaz, Predicted: asm +Key: kaz_77730_B_20131114_230511_028376, Target: kaz, Predicted: amh +Key: kaz_77730_B_20131114_230511_029459, Target: kaz, Predicted: tpi +Key: kaz_93320_B_20140218_173001_042863, Target: kaz, Predicted: tur +Key: kaz_96842_A_20140131_154710_036513, Target: kaz, Predicted: khk +Key: kaz_96842_A_20140131_154710_048679, Target: kaz, Predicted: tpi +Key: khk_12916_B_20140930_182205_051257, Target: khk, Predicted: hat +Key: khk_12916_B_20140930_182205_052432, Target: khk, Predicted: tam +Key: kaz_96842_B_20140131_154710_013611, Target: kaz, Predicted: khk +Key: kaz_96842_A_20140131_154710_015068, Target: kaz, Predicted: tur +Key: kaz_96842_B_20140131_154710_028483, Target: kaz, Predicted: khk +Key: kaz_96842_A_20140131_154710_030373, Target: kaz, Predicted: tur +Key: khk_15163_A_20141020_201846_022885, Target: khk, Predicted: spa +Key: khk_15324_A_20141031_194259_031379, Target: khk, Predicted: tam +Key: khk_15324_A_20141031_194259_037917, Target: khk, Predicted: kat +Key: khk_29208_B_20141018_152040_004635, Target: khk, Predicted: kaz +Key: khk_29208_B_20141018_152040_010368, Target: khk, Predicted: amh +Key: khk_29208_B_20141018_152040_013675, Target: khk, Predicted: zul +Key: khk_29208_B_20141018_152040_019952, Target: khk, Predicted: som +Key: khk_29208_B_20141018_152040_021125, Target: khk, Predicted: swa +Key: khk_29208_B_20141018_152040_024507, Target: khk, Predicted: ibo +Key: khk_29208_B_20141018_152040_026705, Target: khk, Predicted: hat +Key: khk_32861_B_20141112_183418_031908, Target: khk, Predicted: jav +Key: khk_32914_B_20141101_192546_000024, Target: khk, Predicted: tam +Key: khk_32914_B_20141101_192546_001223, Target: khk, Predicted: kaz +Key: khk_29208_B_20141018_152040_036119, Target: khk, Predicted: pus +Key: khk_32301_A_20140927_150237_007302, Target: khk, Predicted: ibo +Key: khk_32301_A_20140927_150237_036562, Target: khk, Predicted: nno +Key: khk_29208_B_20141018_152040_056155, Target: khk, Predicted: ibo +Key: khk_32914_B_20141101_192546_054968, Target: khk, Predicted: swa +Key: khk_41741_A_20141002_230232_018106, Target: khk, Predicted: hat +Key: khk_42243_B_20140924_154551_016645, Target: khk, Predicted: hat +Key: khk_42243_B_20140924_154551_019014, Target: khk, Predicted: hat +Key: khk_42243_B_20140924_154551_023792, Target: khk, Predicted: kmr +Key: khk_38554_A_20140917_124843_000359, Target: khk, Predicted: hat +Key: khk_42243_B_20140924_154551_028728, Target: khk, Predicted: luo +Key: khk_42243_B_20140924_154551_031039, Target: khk, Predicted: kaz +Key: khk_42243_B_20140924_154551_032234, Target: khk, Predicted: hat +Key: khk_42243_B_20140924_154551_033396, Target: khk, Predicted: hat +Key: khk_42243_B_20140924_154551_038963, Target: khk, Predicted: kaz +Key: khk_42243_B_20140924_154551_041321, Target: khk, Predicted: hat +Key: khk_42243_B_20140924_154551_043694, Target: khk, Predicted: ibo +Key: khk_42243_B_20140924_154551_044893, Target: khk, Predicted: swa +Key: khk_42243_B_20140924_154551_046082, Target: khk, Predicted: kat +Key: khk_41741_A_20141002_230232_000647, Target: khk, Predicted: lao +Key: khk_42243_B_20140924_154551_048430, Target: khk, Predicted: kat +Key: khk_42243_B_20140924_154551_008138, Target: khk, Predicted: amh +Key: khk_42243_B_20140924_154551_053068, Target: khk, Predicted: swa +Key: khk_42243_B_20140924_154551_055972, Target: khk, Predicted: kmr +Key: khk_43789_A_20141020_153059_005612, Target: khk, Predicted: amh +Key: khk_43789_A_20141020_153059_058315, Target: khk, Predicted: luo +Key: khk_44347_B_20141103_201828_003178, Target: khk, Predicted: tpi +Key: khk_61678_A_20140919_183209_007194, Target: khk, Predicted: kaz +Key: khk_48200_B_20141104_174608_001562, Target: khk, Predicted: tam +Key: khk_48200_B_20141104_174608_011814, Target: khk, Predicted: kaz +Key: khk_61678_A_20140919_183209_015134, Target: khk, Predicted: yue +Key: khk_56090_A_20140917_155639_034508, Target: khk, Predicted: yue +Key: khk_61678_A_20140919_183209_047196, Target: khk, Predicted: asm +Key: khk_61678_A_20140919_183209_053397, Target: khk, Predicted: kaz +Key: khk_61011_A_20140919_134829_037385, Target: khk, Predicted: yue +Key: khk_61678_A_20140919_183209_054582, Target: khk, Predicted: tpi +Key: khk_61678_A_20140919_183209_057627, Target: khk, Predicted: kaz +Key: khk_78544_A_20140924_155131_014754, Target: khk, Predicted: kaz +Key: kmr_14229_B_20130325_212616_027274, Target: kmr, Predicted: tur +Key: khk_87884_B_20141014_190149_034467, Target: khk, Predicted: luo +Key: kmr_16787_B_20130323_072114_020661, Target: kmr, Predicted: tur +Key: kmr_16787_B_20130323_072114_053722, Target: kmr, Predicted: tur +Key: kmr_15638_B_20130331_200208_030577, Target: kmr, Predicted: amh +Key: kmr_16056_A_20130323_010902_056031, Target: kmr, Predicted: tur +Key: kmr_22288_A_20131228_021559_008179, Target: kmr, Predicted: ceb +Key: kmr_22288_A_20131228_021559_014122, Target: kmr, Predicted: tur +Key: kmr_26206_A_20130507_004626_009278, Target: kmr, Predicted: gug +Key: kmr_20454_B_20140125_002855_001925, Target: kmr, Predicted: kaz +Key: kmr_20454_B_20140125_002855_003117, Target: kmr, Predicted: pus +Key: kmr_22288_A_20131228_021559_000000, Target: kmr, Predicted: ceb +Key: kmr_26999_A_20130414_220838_035286, Target: kmr, Predicted: urd +Key: kmr_22288_A_20131228_021559_004683, Target: kmr, Predicted: ceb +Key: kmr_26999_A_20130414_220838_053277, Target: kmr, Predicted: pus +Key: kmr_34336_A_20130325_005404_056036, Target: kmr, Predicted: tur +Key: kmr_29039_A_20130401_012825_032029, Target: kmr, Predicted: amh +Key: kmr_35069_A_20130407_023338_000875, Target: kmr, Predicted: tur +Key: kmr_31919_A_20130413_172911_000011, Target: kmr, Predicted: tur +Key: kmr_29135_B_20130303_025305_050023, Target: kmr, Predicted: tur +Key: kmr_29039_A_20130401_012825_004886, Target: kmr, Predicted: pus +Key: kmr_46535_A_20140108_201338_006589, Target: kmr, Predicted: lit +Key: kmr_35788_A_20131231_021724_026943, Target: kmr, Predicted: kaz +Key: kmr_35788_A_20131231_021724_028134, Target: kmr, Predicted: kaz +Key: kmr_46535_A_20140108_201338_011066, Target: kmr, Predicted: khk +Key: kmr_35788_A_20131231_021724_029322, Target: kmr, Predicted: kaz +Key: kmr_35788_A_20131231_021724_037053, Target: kmr, Predicted: lit +Key: kmr_35788_A_20131231_021724_057350, Target: kmr, Predicted: khk +Key: kmr_46535_A_20140108_201338_038179, Target: kmr, Predicted: kaz +Key: kmr_35788_A_20131231_021724_064099, Target: kmr, Predicted: kaz +Key: kmr_60830_A_20131223_005744_047132, Target: kmr, Predicted: khk +Key: kmr_60830_A_20131223_005744_017741, Target: kmr, Predicted: kat +Key: kmr_54735_A_20131228_012336_006164, Target: kmr, Predicted: khk +Key: kmr_54735_A_20131228_012336_067811, Target: kmr, Predicted: lit +Key: kmr_79139_A_20130621_004019_041961, Target: kmr, Predicted: gug +Key: kmr_72903_A_20131225_002056_059562, Target: kmr, Predicted: tur +Key: kmr_77225_A_20140106_235541_046086, Target: kmr, Predicted: khk +Key: kmr_77225_A_20140106_235541_047285, Target: kmr, Predicted: khk +Key: kmr_77225_A_20140106_235541_002601, Target: kmr, Predicted: khk +Key: kmr_77225_A_20140106_235541_052011, Target: kmr, Predicted: tpi +Key: kmr_77225_A_20140106_235541_054379, Target: kmr, Predicted: lit +Key: kmr_77225_A_20140106_235541_013543, Target: kmr, Predicted: khk +Key: kmr_86830_B_20130413_225657_001588, Target: kmr, Predicted: tgl +Key: kmr_77225_B_20140106_235541_031416, Target: kmr, Predicted: tpi +Key: kmr_77225_A_20140106_235541_027581, Target: kmr, Predicted: khk +Key: kmr_78360_A_20140123_011434_006644, Target: kmr, Predicted: tur +Key: lao_15042_A_20130727_173946_049342, Target: lao, Predicted: asm +Key: lao_14158_A_20130409_182411_002652, Target: lao, Predicted: yue +Key: lao_15042_A_20130727_173946_056098, Target: lao, Predicted: kaz +Key: lao_15042_A_20130727_173946_062068, Target: lao, Predicted: swe +Key: lao_14228_B_20130405_163836_016251, Target: lao, Predicted: vie +Key: lao_22466_B_20130218_191925_033283, Target: lao, Predicted: kaz +Key: lao_23681_A_20130730_162132_027917, Target: lao, Predicted: ceb +Key: lao_23681_A_20130730_162132_034911, Target: lao, Predicted: ceb +Key: lao_23995_A_20130731_195202_051372, Target: lao, Predicted: jav +Key: lao_23681_B_20130730_162132_043721, Target: lao, Predicted: tam +Key: lao_25012_A_20130814_141020_041372, Target: lao, Predicted: jav +Key: lao_23995_A_20130731_195202_055484, Target: lao, Predicted: jav +Key: lao_23995_A_20130731_195202_058875, Target: lao, Predicted: kaz +Key: lao_23995_B_20130731_195202_000006, Target: lao, Predicted: tgl +Key: lao_29765_B_20130426_185032_006590, Target: lao, Predicted: hat +Key: lao_23995_A_20130731_195202_041755, Target: lao, Predicted: kaz +Key: lao_41920_B_20130310_185621_038917, Target: lao, Predicted: luo +Key: lao_29765_B_20130426_185032_046533, Target: lao, Predicted: tam +Key: lao_41400_A_20130728_194416_033862, Target: lao, Predicted: jav +Key: lao_52025_A_20130306_143713_025120, Target: lao, Predicted: tur +Key: lao_60836_A_20130314_211014_025046, Target: lao, Predicted: gug +Key: lao_72733_A_20130731_235502_038441, Target: lao, Predicted: ceb +Key: lao_79190_A_20130714_135011_021885, Target: lao, Predicted: asm +Key: lao_84370_B_20130506_190748_025300, Target: lao, Predicted: luo +Key: lit_21581_A_20131216_220706_014319, Target: lit, Predicted: tpi +Key: lit_21581_A_20131216_220706_018756, Target: lit, Predicted: pus +Key: lit_21581_A_20131216_220706_019954, Target: lit, Predicted: lao +Key: lit_21581_A_20131216_220706_040302, Target: lit, Predicted: ibo +Key: lit_37064_A_20131129_035959_013167, Target: lit, Predicted: spa +Key: lit_46702_A_20131115_213311_054246, Target: lit, Predicted: tam +Key: lit_70110_B_20131118_222225_012085, Target: lit, Predicted: kat +Key: lit_76837_A_20131020_200525_061435, Target: lit, Predicted: kaz +Key: lit_70110_B_20131118_222225_028655, Target: lit, Predicted: kaz +Key: lit_70110_B_20131118_222225_031106, Target: lit, Predicted: luo +Key: lit_86878_B_20131129_043842_052347, Target: lit, Predicted: kat +Key: lit_86878_B_20131129_043842_056777, Target: lit, Predicted: luo +Key: lit_96934_A_20131207_231603_029039, Target: lit, Predicted: kmr +Key: luo_12220_A_20141026_204025_053435, Target: luo, Predicted: hat +Key: luo_14440_B_20141129_004855_047075, Target: luo, Predicted: swa +Key: luo_43388_A_20141028_212938_020510, Target: luo, Predicted: swa +Key: luo_25012_A_20150201_000040_003577, Target: luo, Predicted: amh +Key: luo_56090_B_20141001_220534_001800, Target: luo, Predicted: swa +Key: luo_56090_B_20141001_220534_035748, Target: luo, Predicted: tam +Key: luo_47882_B_20150131_215134_013596, Target: luo, Predicted: ibo +Key: luo_50726_B_20141015_222945_042179, Target: luo, Predicted: swa +Key: luo_45560_B_20141012_204242_000000, Target: luo, Predicted: lao +Key: luo_45697_A_20150211_181356_003721, Target: luo, Predicted: tam +Key: luo_61225_B_20141014_225524_022997, Target: luo, Predicted: hat +Key: luo_66026_A_20141207_212517_024451, Target: luo, Predicted: hat +Key: luo_66026_A_20141207_212517_026866, Target: luo, Predicted: hat +Key: luo_61225_B_20141014_225524_003458, Target: luo, Predicted: hat +Key: luo_61225_B_20141014_225524_004632, Target: luo, Predicted: swa +Key: luo_66026_A_20141207_212517_002144, Target: luo, Predicted: swa +Key: luo_72349_A_20150313_194307_008763, Target: luo, Predicted: swa +Key: luo_79820_A_20141005_212016_000020, Target: luo, Predicted: lao +Key: luo_72349_A_20150313_194307_011149, Target: luo, Predicted: lao +Key: luo_72349_A_20150313_194307_015010, Target: luo, Predicted: swa +Key: luo_79820_A_20141005_212016_025297, Target: luo, Predicted: ibo +Key: luo_72349_A_20150313_194307_043718, Target: luo, Predicted: gug +Key: luo_97264_A_20141220_220653_028177, Target: luo, Predicted: hat +Key: luo_97264_B_20141220_220653_009736, Target: luo, Predicted: swa +Key: luo_97264_B_20141220_220653_027161, Target: luo, Predicted: pus +Key: luo_99813_A_20141106_211637_001355, Target: luo, Predicted: hat +Key: pus_28102_B_20120326_171523_031756, Target: pus, Predicted: ibo +Key: pus_28102_A_20120326_171523_016544, Target: pus, Predicted: asm +Key: pus_29368_A_20120321_233801_022408, Target: pus, Predicted: kaz +Key: pus_29368_A_20120321_235133_019809, Target: pus, Predicted: amh +Key: pus_29368_A_20120321_235133_025809, Target: pus, Predicted: tpi +Key: pus_56226_B_20120205_235429_051101, Target: pus, Predicted: kmr +Key: pus_61592_B_20120126_181735_055683, Target: pus, Predicted: tam +Key: pus_82160_B_20120126_022907_039657, Target: pus, Predicted: lit +Key: pus_76812_B_20120320_180439_024771, Target: pus, Predicted: kmr +Key: pus_86680_B_20120309_181746_007085, Target: pus, Predicted: tur +Key: pus_89308_B_20120131_214111_006761, Target: pus, Predicted: kaz +Key: pus_89308_B_20120131_214111_013816, Target: pus, Predicted: khk +Key: swa_17115_A_20140218_210921_045736, Target: swa, Predicted: khk +Key: swa_17115_A_20140218_210921_046909, Target: swa, Predicted: tam +Key: swa_17115_A_20140218_210921_053530, Target: swa, Predicted: khk +Key: swa_17115_A_20140218_210921_055633, Target: swa, Predicted: lit +Key: swa_17115_A_20140218_210921_056773, Target: swa, Predicted: khk +Key: swa_17115_A_20140218_210921_057897, Target: swa, Predicted: tam +Key: swa_17115_A_20140218_210921_059104, Target: swa, Predicted: khk +Key: swa_16249_B_20131202_232723_000000, Target: swa, Predicted: amh +Key: swa_14814_A_20140205_210842_036227, Target: swa, Predicted: luo +Key: swa_24290_B_20140219_000423_029635, Target: swa, Predicted: ibo +Key: swa_17115_A_20140218_210921_008274, Target: swa, Predicted: yor +Key: swa_24290_B_20140219_000423_038218, Target: swa, Predicted: ibo +Key: swa_24290_B_20140219_000423_043494, Target: swa, Predicted: ibo +Key: swa_24239_A_20140206_191516_047532, Target: swa, Predicted: kmr +Key: swa_15420_A_20140210_010333_056109, Target: swa, Predicted: tam +Key: swa_24290_B_20140219_000423_046817, Target: swa, Predicted: luo +Key: swa_24290_B_20140219_000423_054951, Target: swa, Predicted: hat +Key: swa_34197_B_20121228_201800_025473, Target: swa, Predicted: amh +Key: swa_38588_A_20130228_211322_002708, Target: swa, Predicted: kmr +Key: swa_39893_B_20140115_023429_035762, Target: swa, Predicted: luo +Key: swa_45459_A_20131012_022245_022507, Target: swa, Predicted: amh +Key: swa_45459_A_20131012_022245_042952, Target: swa, Predicted: gug +Key: swa_45459_B_20131012_022245_051341, Target: swa, Predicted: hat +Key: swa_63084_B_20130801_015957_000093, Target: swa, Predicted: luo +Key: swa_63084_B_20130801_015957_020419, Target: swa, Predicted: luo +Key: swa_63084_B_20130801_015957_034096, Target: swa, Predicted: luo +Key: swa_59549_B_20131003_203701_010964, Target: swa, Predicted: amh +Key: swa_59549_B_20131003_203701_021584, Target: swa, Predicted: gug +Key: swa_63084_A_20130801_014407_000990, Target: swa, Predicted: luo +Key: swa_63084_A_20130801_014407_002124, Target: swa, Predicted: jav +Key: swa_63084_A_20130801_014407_003284, Target: swa, Predicted: luo +Key: swa_55042_B_20131217_033729_038274, Target: swa, Predicted: gug +Key: swa_55106_A_20131215_030617_020580, Target: swa, Predicted: luo +Key: swa_63084_A_20130801_015957_043913, Target: swa, Predicted: khk +Key: swa_55106_A_20131215_030617_036846, Target: swa, Predicted: luo +Key: swa_73819_B_20130911_163458_041264, Target: swa, Predicted: luo +Key: swa_73819_B_20130911_163458_042943, Target: swa, Predicted: hat +Key: swa_73819_B_20130927_003321_003623, Target: swa, Predicted: ibo +Key: swa_73301_A_20140226_185528_044387, Target: swa, Predicted: kaz +Key: swa_73301_A_20140226_185528_046773, Target: swa, Predicted: kaz +Key: swa_72040_B_20131002_213605_049382, Target: swa, Predicted: hat +Key: swa_73301_A_20140226_185528_048995, Target: swa, Predicted: lao +Key: swa_66822_B_20130219_222318_006840, Target: swa, Predicted: zul +Key: swa_73301_A_20140226_185528_050193, Target: swa, Predicted: tam +Key: swa_73301_A_20140226_185528_054191, Target: swa, Predicted: tam +Key: swa_73301_A_20140226_185528_058750, Target: swa, Predicted: kaz +Key: swa_73301_B_20140226_185528_034627, Target: swa, Predicted: tpi +Key: swa_76756_A_20130417_210400_018795, Target: swa, Predicted: kaz +Key: swa_77990_A_20131007_063102_055659, Target: swa, Predicted: ibo +Key: swa_90080_A_20140319_222809_027316, Target: swa, Predicted: tpi +Key: swa_88661_A_20130801_192922_004595, Target: swa, Predicted: luo +Key: swa_90080_A_20140319_222809_043606, Target: swa, Predicted: ibo +Key: swa_77990_B_20131007_063102_030607, Target: swa, Predicted: hat +Key: swa_77990_B_20131007_063102_031743, Target: swa, Predicted: gug +Key: swa_88661_A_20130801_192922_015175, Target: swa, Predicted: khk +Key: swa_90080_A_20140319_222809_052888, Target: swa, Predicted: kmr +Key: swa_77990_A_20131007_063102_018142, Target: swa, Predicted: hat +Key: swa_77990_A_20131007_063102_019854, Target: swa, Predicted: hat +Key: swa_77990_A_20131007_063102_020926, Target: swa, Predicted: luo +Key: swa_88661_A_20130801_192922_039985, Target: swa, Predicted: kat +Key: swa_77990_A_20131007_063102_022115, Target: swa, Predicted: ibo +Key: swa_77990_A_20131007_063102_026741, Target: swa, Predicted: lao +Key: swa_77990_A_20131007_063102_027903, Target: swa, Predicted: hat +Key: swa_88661_B_20130801_192922_026940, Target: swa, Predicted: luo +Key: swa_77990_A_20131007_063102_038377, Target: swa, Predicted: gug +Key: swa_90080_B_20140319_222809_051417, Target: swa, Predicted: zul +Key: swa_92740_A_20130923_235638_046188, Target: swa, Predicted: luo +Key: swa_84177_A_20131208_021104_017433, Target: swa, Predicted: luo +Key: swa_84177_A_20131208_021104_023749, Target: swa, Predicted: hau +Key: swa_92740_A_20130923_235638_051565, Target: swa, Predicted: hat +Key: swa_92740_A_20130923_235638_056635, Target: swa, Predicted: amh +Key: swa_98311_A_20130109_195922_006959, Target: swa, Predicted: zul +Key: swa_98311_B_20130109_191639_008512, Target: swa, Predicted: zul +Key: swa_98311_B_20130109_191639_019516, Target: swa, Predicted: tpi +Key: swa_98311_B_20130109_191639_020611, Target: swa, Predicted: tgl +Key: tam_20682_B_20130209_174057_019472, Target: tam, Predicted: tel +Key: swa_98311_B_20130109_195922_013350, Target: swa, Predicted: luo +Key: tam_18924_A_20130224_150538_016506, Target: tam, Predicted: ben +Key: swa_98311_B_20130109_195922_029196, Target: swa, Predicted: ibo +Key: tam_26602_A_20130215_003413_056511, Target: tam, Predicted: tel +Key: tam_28606_A_20130126_221856_016645, Target: tam, Predicted: tel +Key: tam_32287_A_20130902_231135_036249, Target: tam, Predicted: ben +Key: tam_31624_A_20130107_221428_051356, Target: tam, Predicted: ben +Key: tam_31624_B_20130107_221428_000000, Target: tam, Predicted: tel +Key: tam_32287_A_20130902_231135_045702, Target: tam, Predicted: tur +Key: tam_28606_A_20130126_221856_035560, Target: tam, Predicted: asm +Key: tam_51701_A_20130312_022556_031090, Target: tam, Predicted: tel +Key: tam_55136_A_20130705_164312_053934, Target: tam, Predicted: tel +Key: tam_47451_A_20130210_010011_028292, Target: tam, Predicted: tel +Key: tam_57935_A_20130126_234131_007506, Target: tam, Predicted: tel +Key: tam_55136_A_20130705_164312_000000, Target: tam, Predicted: asm +Key: tam_55136_A_20130705_164312_007891, Target: tam, Predicted: kat +Key: tam_55136_A_20130705_164312_026691, Target: tam, Predicted: gug +Key: tam_59747_B_20121222_160946_052528, Target: tam, Predicted: tgl +Key: tam_63484_A_20130821_005511_000000, Target: tam, Predicted: tel +Key: tam_63484_A_20130821_005511_007534, Target: tam, Predicted: tel +Key: tam_59747_B_20121222_160946_003000, Target: tam, Predicted: tel +Key: tam_64902_B_20130215_191500_019463, Target: tam, Predicted: tel +Key: tam_78161_B_20130521_152635_032561, Target: tam, Predicted: mal +Key: tam_87074_A_20130107_181209_056155, Target: tam, Predicted: tel +Key: tam_91808_A_20130603_193623_033822, Target: tam, Predicted: spa +Key: tam_91808_A_20130603_193623_035009, Target: tam, Predicted: spa +Key: tam_91808_A_20130603_193623_036170, Target: tam, Predicted: spa +Key: tam_90937_B_20130516_224543_057733, Target: tam, Predicted: tur +Key: tam_91808_A_20130603_193623_038526, Target: tam, Predicted: spa +Key: tam_91808_A_20130603_193623_040739, Target: tam, Predicted: spa +Key: tam_91808_A_20130603_193623_044312, Target: tam, Predicted: spa +Key: tam_91808_A_20130603_193623_046627, Target: tam, Predicted: spa +Key: tam_91808_A_20130603_193623_047810, Target: tam, Predicted: luo +Key: tam_91808_A_20130603_193623_050178, Target: tam, Predicted: spa +Key: tam_90937_B_20130516_224543_000698, Target: tam, Predicted: tel +Key: tam_91808_A_20130603_193623_029099, Target: tam, Predicted: nno +Key: tel_22965_A_20131114_213605_007220, Target: tel, Predicted: mal +Key: tel_22965_A_20131114_213605_021162, Target: tel, Predicted: tur +Key: tel_21029_A_20131112_180205_050248, Target: tel, Predicted: tur +Key: tel_19703_A_20131114_213952_013187, Target: tel, Predicted: pan +Key: tel_19703_A_20131114_213952_025814, Target: tel, Predicted: gug +Key: tel_34336_B_20131114_162157_016722, Target: tel, Predicted: yue +Key: tel_46333_A_20131102_160049_011357, Target: tel, Predicted: tam +Key: tel_46702_A_20131023_225137_036937, Target: tel, Predicted: lao +Key: tel_46333_A_20131102_160049_044218, Target: tel, Predicted: tam +Key: tel_39848_A_20131113_195552_021978, Target: tel, Predicted: tam +Key: tel_46333_A_20131102_160049_050427, Target: tel, Predicted: tam +Key: tel_46333_A_20131102_160049_058406, Target: tel, Predicted: asm +Key: tel_49287_A_20131115_193114_004879, Target: tel, Predicted: tam +Key: tel_56720_B_20131122_215343_034540, Target: tel, Predicted: tam +Key: tel_61167_A_20131104_210455_048458, Target: tel, Predicted: yue +Key: tel_52854_A_20131105_013802_050825, Target: tel, Predicted: tam +Key: tel_58734_A_20131109_181122_003170, Target: tel, Predicted: khk +Key: tel_64759_A_20131104_195356_000000, Target: tel, Predicted: asm +Key: tel_65370_A_20140222_225324_021275, Target: tel, Predicted: tam +Key: tel_52854_A_20131105_013802_010802, Target: tel, Predicted: ben +Key: tel_86472_B_20131204_195705_020665, Target: tel, Predicted: asm +Key: tel_86472_B_20131204_195705_038616, Target: tel, Predicted: tam +Key: tel_74280_A_20131025_160420_021789, Target: tel, Predicted: tpi +Key: tel_75064_A_20131114_174949_038514, Target: tel, Predicted: gug +Key: tel_99487_A_20131027_195100_033800, Target: tel, Predicted: tam +Key: tel_99487_A_20131027_195100_039891, Target: tel, Predicted: ben +Key: tel_99487_A_20131027_195100_041660, Target: tel, Predicted: tam +Key: tel_99487_A_20131027_195100_050799, Target: tel, Predicted: asm +Key: tel_99487_A_20131027_195100_057242, Target: tel, Predicted: asm +Key: tel_99487_A_20131027_195100_002251, Target: tel, Predicted: asm +Key: tgl_16883_A_20120219_191154_047091, Target: tgl, Predicted: hat +Key: tel_99487_A_20131027_195100_017295, Target: tel, Predicted: tam +Key: tgl_25035_A_20120213_014750_039539, Target: tgl, Predicted: tur +Key: tgl_24379_A_20120303_015051_058579, Target: tgl, Predicted: tam +Key: tgl_42766_A_20120217_003639_055563, Target: tgl, Predicted: jav +Key: tgl_47845_A_20120405_122139_002633, Target: tgl, Predicted: tur +Key: tgl_35896_A_20120302_123550_002677, Target: tgl, Predicted: asm +Key: tgl_35896_A_20120302_123550_037263, Target: tgl, Predicted: jav +Key: tgl_42766_A_20120217_003639_003845, Target: tgl, Predicted: pus +Key: tgl_42766_A_20120217_003639_013190, Target: tgl, Predicted: ceb +Key: tgl_42766_A_20120217_003639_015529, Target: tgl, Predicted: jav +Key: tgl_42766_A_20120217_003639_018270, Target: tgl, Predicted: asm +Key: tgl_42766_A_20120217_003639_035803, Target: tgl, Predicted: hat +Key: tgl_42766_A_20120217_003639_037776, Target: tgl, Predicted: ceb +Key: tgl_42766_A_20120217_003639_040807, Target: tgl, Predicted: tur +Key: tgl_42766_A_20120217_003639_041997, Target: tgl, Predicted: ceb +Key: tgl_53982_A_20120224_233136_000808, Target: tgl, Predicted: hat +Key: tgl_53982_A_20120224_233136_057579, Target: tgl, Predicted: ceb +Key: tgl_53982_A_20120224_233136_004130, Target: tgl, Predicted: vie +Key: tgl_53982_A_20120224_233136_058755, Target: tgl, Predicted: ceb +Key: tgl_53982_A_20120224_233136_010994, Target: tgl, Predicted: ceb +Key: tgl_57422_B_20120227_015422_058809, Target: tgl, Predicted: ceb +Key: tgl_53982_B_20120224_233136_034869, Target: tgl, Predicted: kmr +Key: tgl_53982_A_20120224_233136_038175, Target: tgl, Predicted: swa +Key: tgl_53982_A_20120224_233136_042693, Target: tgl, Predicted: ceb +Key: tgl_53982_B_20120224_233136_050145, Target: tgl, Predicted: asm +Key: tgl_53982_A_20120224_233136_047272, Target: tgl, Predicted: hau +Key: tgl_53982_A_20120224_233136_050701, Target: tgl, Predicted: ceb +Key: tgl_57422_B_20120227_015422_005263, Target: tgl, Predicted: asm +Key: tgl_65580_B_20120221_210222_019328, Target: tgl, Predicted: tam +Key: tgl_66026_A_20120511_112437_000000, Target: tgl, Predicted: amh +Key: tgl_69050_B_20120203_173053_035312, Target: tgl, Predicted: kaz +Key: tgl_69050_B_20120203_173053_037579, Target: tgl, Predicted: ceb +Key: tgl_69050_B_20120203_173053_038725, Target: tgl, Predicted: tel +Key: tgl_81587_B_20120309_163209_015741, Target: tgl, Predicted: pus +Key: tgl_83891_A_20120327_163405_052916, Target: tgl, Predicted: zul +Key: tgl_83255_A_20120530_214353_011677, Target: tgl, Predicted: ceb +Key: tgl_79698_A_20120315_223952_001345, Target: tgl, Predicted: kaz +Key: tgl_85617_A_20120225_212818_053793, Target: tgl, Predicted: ceb +Key: tgl_83891_A_20120327_163405_025856, Target: tgl, Predicted: jav +Key: tgl_95589_B_20120225_032340_018516, Target: tgl, Predicted: ceb +Key: tgl_93000_B_20120227_164805_038142, Target: tgl, Predicted: jav +Key: tgl_93000_B_20120227_164805_050923, Target: tgl, Predicted: tpi +Key: tgl_93000_B_20120227_164805_054742, Target: tgl, Predicted: hat +Key: tpi_14440_A_20130824_153139_000000, Target: tpi, Predicted: yue +Key: tpi_14440_A_20130824_153139_003448, Target: tpi, Predicted: spa +Key: tpi_14440_A_20130824_153139_011131, Target: tpi, Predicted: luo +Key: tpi_14440_B_20130824_152406_002756, Target: tpi, Predicted: hat +Key: tpi_14440_B_20130824_153643_009173, Target: tpi, Predicted: tur +Key: tpi_14875_A_20130731_170626_024438, Target: tpi, Predicted: lao +Key: tpi_21244_A_20131010_122553_035642, Target: tpi, Predicted: kaz +Key: tpi_21244_A_20131010_122553_000000, Target: tpi, Predicted: tel +Key: tpi_21244_A_20131010_122553_004612, Target: tpi, Predicted: lit +Key: tpi_29911_A_20131212_174224_044795, Target: tpi, Predicted: luo +Key: tpi_32708_A_20130730_130556_000000, Target: tpi, Predicted: ben +Key: tpi_32708_B_20130730_130556_018569, Target: tpi, Predicted: tur +Key: tpi_32708_B_20130730_130556_040242, Target: tpi, Predicted: tel +Key: tpi_32708_B_20130730_130556_044557, Target: tpi, Predicted: ibo +Key: tpi_32708_B_20130730_130556_056565, Target: tpi, Predicted: asm +Key: tpi_46535_A_20131219_223648_000000, Target: tpi, Predicted: ceb +Key: tpi_46535_A_20131219_223648_002331, Target: tpi, Predicted: ceb +Key: tpi_33175_B_20130621_162225_012734, Target: tpi, Predicted: kmr +Key: tpi_46535_A_20131219_223648_004702, Target: tpi, Predicted: ceb +Key: tpi_46535_A_20131219_223648_009377, Target: tpi, Predicted: ceb +Key: tpi_46535_A_20131219_223648_011760, Target: tpi, Predicted: kaz +Key: tpi_46535_A_20131219_223648_019923, Target: tpi, Predicted: kaz +Key: tpi_46535_A_20131219_223648_021109, Target: tpi, Predicted: kaz +Key: tpi_46535_A_20131219_223648_023410, Target: tpi, Predicted: lit +Key: tpi_46535_A_20131219_223648_034620, Target: tpi, Predicted: kaz +Key: tpi_46535_A_20131219_223648_035772, Target: tpi, Predicted: kaz +Key: tpi_46535_A_20131219_223648_036968, Target: tpi, Predicted: lit +Key: tpi_46535_A_20131219_223648_040468, Target: tpi, Predicted: kaz +Key: tpi_46535_A_20131219_223648_043973, Target: tpi, Predicted: kaz +Key: tpi_46535_A_20131219_223648_045145, Target: tpi, Predicted: lit +Key: tpi_46535_A_20131219_223648_046327, Target: tpi, Predicted: kaz +Key: tpi_46535_A_20131219_223648_048678, Target: tpi, Predicted: kaz +Key: tpi_46535_A_20131219_223648_055715, Target: tpi, Predicted: khk +Key: tpi_61963_A_20130830_141616_048803, Target: tpi, Predicted: ceb +Key: tpi_67213_A_20131218_185924_000007, Target: tpi, Predicted: tam +Key: tpi_65252_A_20131008_183014_027781, Target: tpi, Predicted: khk +Key: tpi_61963_A_20130830_141616_014582, Target: tpi, Predicted: tgl +Key: tpi_61963_A_20130830_141616_016809, Target: tpi, Predicted: ceb +Key: tpi_61963_A_20130830_141616_056934, Target: tpi, Predicted: yue +Key: tpi_67213_A_20131218_185924_009958, Target: tpi, Predicted: lao +Key: tpi_67213_A_20131218_185924_013351, Target: tpi, Predicted: kaz +Key: tpi_61963_A_20130830_141616_021279, Target: tpi, Predicted: ceb +Key: tpi_61963_A_20130830_141616_023535, Target: tpi, Predicted: kaz +Key: tpi_61963_A_20130830_141616_024661, Target: tpi, Predicted: kaz +Key: tpi_67213_A_20131218_185924_024779, Target: tpi, Predicted: kaz +Key: tpi_67213_A_20131218_185924_027069, Target: tpi, Predicted: kaz +Key: tpi_61963_A_20130830_141616_029312, Target: tpi, Predicted: tel +Key: tpi_65252_A_20131008_183014_049192, Target: tpi, Predicted: kaz +Key: tpi_65252_A_20131008_183014_005780, Target: tpi, Predicted: ceb +Key: tpi_67213_A_20131218_185924_030537, Target: tpi, Predicted: kaz +Key: tpi_67213_A_20131218_185924_031728, Target: tpi, Predicted: lao +Key: tpi_65252_A_20131008_183014_009092, Target: tpi, Predicted: kaz +Key: tpi_65252_A_20131008_183014_052661, Target: tpi, Predicted: kaz +Key: tpi_67213_A_20131218_185924_032851, Target: tpi, Predicted: kaz +Key: tpi_65252_A_20131008_183014_055433, Target: tpi, Predicted: kaz +Key: tpi_67213_A_20131218_185924_035174, Target: tpi, Predicted: lao +Key: tpi_65252_A_20131008_183014_057351, Target: tpi, Predicted: kaz +Key: tpi_61963_A_20130830_141616_040718, Target: tpi, Predicted: kaz +Key: tpi_65252_A_20131008_183014_016004, Target: tpi, Predicted: kaz +Key: tpi_67213_A_20131218_185924_040830, Target: tpi, Predicted: kaz +Key: tpi_61963_A_20130830_141616_043068, Target: tpi, Predicted: kaz +Key: tpi_67213_A_20131218_185924_042022, Target: tpi, Predicted: lit +Key: tpi_65252_A_20131008_183014_019897, Target: tpi, Predicted: lit +Key: tpi_67213_A_20131218_185924_043218, Target: tpi, Predicted: kaz +Key: tpi_65252_A_20131008_183014_021073, Target: tpi, Predicted: ceb +Key: tpi_67213_A_20131218_185924_044365, Target: tpi, Predicted: lao +Key: tpi_74226_B_20130828_115915_013376, Target: tpi, Predicted: ces +Key: tpi_67213_A_20131218_185924_046992, Target: tpi, Predicted: kaz +Key: tpi_67213_A_20131218_185924_048146, Target: tpi, Predicted: tel +Key: tpi_74226_B_20130828_115915_021609, Target: tpi, Predicted: tam +Key: tpi_67213_A_20131218_185924_051355, Target: tpi, Predicted: asm +Key: tpi_74226_B_20130828_115915_022808, Target: tpi, Predicted: tel +Key: tpi_67213_A_20131218_185924_052511, Target: tpi, Predicted: kaz +Key: tpi_67213_A_20131218_185924_054773, Target: tpi, Predicted: ceb +Key: tpi_74226_B_20130828_115915_028173, Target: tpi, Predicted: luo +Key: tpi_67213_B_20131218_185924_004053, Target: tpi, Predicted: kat +Key: tpi_74226_B_20130828_115915_032801, Target: tpi, Predicted: luo +Key: tpi_67213_B_20131218_185924_022830, Target: tpi, Predicted: tam +Key: tpi_67213_B_20131218_185924_048209, Target: tpi, Predicted: khk +Key: tpi_67213_B_20131218_185924_057759, Target: tpi, Predicted: ben +Key: tpi_70726_A_20131222_161540_019387, Target: tpi, Predicted: khk +Key: tpi_70726_A_20131222_161540_023625, Target: tpi, Predicted: lao +Key: tpi_74226_B_20130828_115915_005432, Target: tpi, Predicted: asm +Key: tpi_70726_A_20131222_161540_024767, Target: tpi, Predicted: lit +Key: tpi_74226_B_20130828_115915_006573, Target: tpi, Predicted: tur +Key: tpi_74226_B_20130828_115915_010205, Target: tpi, Predicted: tur +Key: tpi_76837_A_20131207_184347_043285, Target: tpi, Predicted: lit +Key: tpi_85179_B_20130920_130213_039435, Target: tpi, Predicted: hat +Key: tpi_90777_B_20130725_111134_034620, Target: tpi, Predicted: zul +Key: tpi_80577_B_20130930_204532_000000, Target: tpi, Predicted: kaz +Key: tpi_80577_B_20130930_204532_004421, Target: tpi, Predicted: kaz +Key: tpi_80577_B_20130930_204532_010349, Target: tpi, Predicted: khk +Key: tpi_80577_B_20130930_204532_023950, Target: tpi, Predicted: tel +Key: tpi_80577_B_20130930_204532_037734, Target: tpi, Predicted: kat +Key: tpi_92886_B_20130711_144627_042334, Target: tpi, Predicted: tel +Key: tur_21541_A_20120518_012528_006278, Target: tur, Predicted: kmr +Key: tur_21541_A_20120518_012528_014473, Target: tur, Predicted: kmr +Key: tur_21541_A_20120518_012528_018664, Target: tur, Predicted: kmr +Key: tur_21541_A_20120518_012528_031124, Target: tur, Predicted: kmr +Key: tur_11521_A_20120602_034839_041086, Target: tur, Predicted: ceb +Key: tur_11521_A_20120602_034839_049327, Target: tur, Predicted: kmr +Key: tur_32236_A_20120516_221311_019954, Target: tur, Predicted: zul +Key: tur_39963_A_20120209_083935_000000, Target: tur, Predicted: tgl +Key: tur_31256_A_20120531_015506_021282, Target: tur, Predicted: kmr +Key: tur_44023_A_20120530_220359_022785, Target: tur, Predicted: kmr +Key: tur_76372_B_20120709_015738_018584, Target: tur, Predicted: kmr +Key: vie_12963_B_20120509_003852_003025, Target: vie, Predicted: lao +Key: vie_11031_B_20120617_182613_013283, Target: vie, Predicted: gug +Key: vie_14769_B_20120420_013147_027012, Target: vie, Predicted: hat +Key: vie_14769_A_20120420_013147_000233, Target: vie, Predicted: asm +Key: vie_32236_B_20120505_195420_013203, Target: vie, Predicted: lao +Key: vie_32236_B_20120505_195420_021428, Target: vie, Predicted: tel +Key: vie_31538_A_20120320_202748_018919, Target: vie, Predicted: gug +Key: vie_31538_A_20120320_202748_020100, Target: vie, Predicted: tpi +Key: vie_31538_A_20120320_202748_021281, Target: vie, Predicted: hat +Key: vie_32236_B_20120505_195420_035087, Target: vie, Predicted: lao +Key: vie_32236_B_20120505_195420_039604, Target: vie, Predicted: lao +Key: vie_31538_A_20120320_202748_028672, Target: vie, Predicted: jav +Key: vie_31538_A_20120320_202748_031014, Target: vie, Predicted: hat +Key: vie_32236_B_20120505_195420_053886, Target: vie, Predicted: lao +Key: vie_32236_B_20120505_195420_058774, Target: vie, Predicted: lao +Key: vie_31538_A_20120320_202748_036046, Target: vie, Predicted: ibo +Key: vie_31538_A_20120320_202748_038410, Target: vie, Predicted: tam +Key: vie_31538_A_20120320_202748_040232, Target: vie, Predicted: gug +Key: vie_31538_A_20120320_202748_003435, Target: vie, Predicted: por +Key: vie_31538_A_20120320_202748_051509, Target: vie, Predicted: gug +Key: vie_31538_A_20120320_202748_008124, Target: vie, Predicted: ibo +Key: vie_31538_A_20120320_202748_010955, Target: vie, Predicted: gug +Key: vie_31538_A_20120320_202748_053819, Target: vie, Predicted: hat +Key: vie_31538_A_20120320_202748_012151, Target: vie, Predicted: hat +Key: vie_35391_A_20120416_192241_046900, Target: vie, Predicted: gug +Key: vie_35391_A_20120416_192241_054357, Target: vie, Predicted: jav +Key: vie_45512_A_20120505_135144_053538, Target: vie, Predicted: hat +Key: vie_45512_A_20120505_135144_004505, Target: vie, Predicted: ceb +Key: vie_45512_A_20120505_135144_011913, Target: vie, Predicted: ibo +Key: vie_63459_B_20120415_003841_021302, Target: vie, Predicted: kaz +Key: vie_63459_B_20120415_003841_029357, Target: vie, Predicted: ceb +Key: vie_79526_A_20120420_150504_017293, Target: vie, Predicted: gug +Key: vie_85204_A_20120212_190017_002132, Target: vie, Predicted: gug +Key: vie_77771_B_20120421_231323_012583, Target: vie, Predicted: tur +Key: vie_85204_A_20120212_190017_028994, Target: vie, Predicted: tgl +Key: vie_90202_A_20120502_194459_035522, Target: vie, Predicted: hat +Key: vie_90202_A_20120502_194459_004459, Target: vie, Predicted: swa +Key: vie_90202_A_20120502_194459_045611, Target: vie, Predicted: hat +Key: vie_90202_A_20120502_194459_050828, Target: vie, Predicted: lao +Key: vie_90202_A_20120502_194459_024874, Target: vie, Predicted: swa +Key: vie_92386_A_20120322_195456_024899, Target: vie, Predicted: lao +Key: vie_92386_A_20120322_195456_031300, Target: vie, Predicted: lao +Key: zul_22466_B_20121130_231814_007273, Target: zul, Predicted: tgl +Key: zul_28190_A_20121213_031401_032444, Target: zul, Predicted: asm +Key: zul_35583_B_20130529_005600_015367, Target: zul, Predicted: ibo +Key: zul_35583_B_20130529_005600_036873, Target: zul, Predicted: ibo +Key: zul_35583_B_20130529_005600_053939, Target: zul, Predicted: amh +Key: zul_43646_B_20121206_213819_000000, Target: zul, Predicted: asm +Key: zul_42600_A_20121206_212006_003728, Target: zul, Predicted: tgl +Key: zul_41100_A_20121129_003855_000513, Target: zul, Predicted: swa +Key: zul_56198_A_20121128_190457_008384, Target: zul, Predicted: swa +Key: zul_56198_A_20121128_190457_010740, Target: zul, Predicted: amh +Key: zul_56198_A_20121128_190457_055361, Target: zul, Predicted: hat +Key: zul_79858_B_20121126_013705_033065, Target: zul, Predicted: tgl +Key: zul_82224_A_20130602_234038_044542, Target: zul, Predicted: ibo +Key: zul_82224_A_20130602_234038_048706, Target: zul, Predicted: ssw +Key: zul_84838_B_20121210_051040_025030, Target: zul, Predicted: jav +Key: zul_93007_A_20130528_211314_002393, Target: zul, Predicted: luo +Key: zul_93007_A_20130528_211314_017504, Target: zul, Predicted: sna +Key: zul_93007_A_20130528_211314_057839, Target: zul, Predicted: sna diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/lid_inference_test.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/lid_inference_test.log new file mode 100644 index 0000000000000000000000000000000000000000..602cc9b18527f32a3e714e5cd63a19ce05d5d2bd --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/lid_inference_test.log @@ -0,0 +1,286 @@ +# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +# Started at Mon Jun 2 02:33:14 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +[gpue04] 2025-06-02 02:33:33,758 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + torch.load(model_file, map_location=device), +[gpue04] 2025-06-02 02:33:45,800 (lid_inference_dist:86) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary. + warnings.warn(_create_warning_msg( +/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist. + warnings.warn( +[gpue04] 2025-06-02 02:33:46,351 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/lids0 +[gpue04] 2025-06-02 02:34:18,099 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0 +[gpue04] 2025-06-02 02:34:42,593 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1 +[gpue04] 2025-06-02 02:35:05,422 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2 +[gpue04] 2025-06-02 02:35:28,092 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3 +[gpue04] 2025-06-02 02:35:52,743 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4 +[gpue04] 2025-06-02 02:36:39,290 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5 +[gpue04] 2025-06-02 02:37:10,073 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6 +[gpue04] 2025-06-02 02:37:13,207 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True +[gpue04] 2025-06-02 02:37:13,208 (lid_inference_dist:215) INFO: args.save_tsne_plot: False +# Accounting: time=240 threads=1 +# Ended (code 0) at Mon Jun 2 02:37:14 CDT 2025, elapsed time 240 seconds diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/results b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/results new file mode 100644 index 0000000000000000000000000000000000000000..2a34650a392d5ec0cc251c7cd4d2cebe18fb16f8 --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_dialect_ml_superb2_lang_cross_train_all_no_filter_lang/results @@ -0,0 +1,946 @@ +Accuracy: 86.82% +Macro Accuracy: 86.92% +Accuracy per Language: +tam: 100.00% +guj: 97.89% +ell: 71.37% +eng: 93.57% +deu: 74.00% +tel: 99.01% +spa: 95.19% +ara: 64.31% +Key: ara_sada_acw_000000, Target: ara, Predicted: heb +Key: ara_sada_acw_000032, Target: ara, Predicted: mon +Key: ara_sada_acw_000064, Target: ara, Predicted: slv +Key: ara_sada_acw_000096, Target: ara, Predicted: fao +Key: ara_sada_acw_000001, Target: ara, Predicted: sot +Key: ara_sada_acw_000033, Target: ara, Predicted: pus +Key: ara_sada_acw_000065, Target: ara, Predicted: fin +Key: ara_sada_acw_000067, Target: ara, Predicted: sqi +Key: ara_sada_acw_000099, Target: ara, Predicted: amh +Key: ara_sada_acw_000004, Target: ara, Predicted: tuk +Key: ara_sada_acw_000005, Target: ara, Predicted: mon +Key: ara_sada_acw_000069, Target: ara, Predicted: slv +Key: ara_sada_acw_000101, Target: ara, Predicted: cmn +Key: ara_sada_acw_000006, Target: ara, Predicted: eng +Key: ara_sada_acw_000038, Target: ara, Predicted: sqi +Key: ara_sada_acw_000071, Target: ara, Predicted: khm +Key: ara_sada_acw_000103, Target: ara, Predicted: sqi +Key: ara_sada_acw_000009, Target: ara, Predicted: som +Key: ara_sada_acw_000041, Target: ara, Predicted: aze +Key: ara_sada_acw_000042, Target: ara, Predicted: aze +Key: ara_sada_acw_000106, Target: ara, Predicted: kaz +Key: ara_sada_acw_000011, Target: ara, Predicted: ces +Key: ara_sada_acw_000043, Target: ara, Predicted: pus +Key: ara_sada_acw_000075, Target: ara, Predicted: cmn +Key: ara_sada_acw_000044, Target: ara, Predicted: kaz +Key: ara_sada_acw_000045, Target: ara, Predicted: fas +Key: ara_sada_acw_000077, Target: ara, Predicted: deu +Key: ara_sada_acw_000014, Target: ara, Predicted: hat +Key: ara_sada_acw_000046, Target: ara, Predicted: som +Key: ara_sada_acw_000078, Target: ara, Predicted: spa +Key: ara_sada_acw_000110, Target: ara, Predicted: lin +Key: ara_sada_acw_000079, Target: ara, Predicted: aze +Key: ara_sada_acw_000111, Target: ara, Predicted: cym +Key: ara_sada_acw_000016, Target: ara, Predicted: hrv +Key: ara_sada_acw_000048, Target: ara, Predicted: eng +Key: ara_sada_acw_000080, Target: ara, Predicted: ces +Key: ara_sada_acw_000083, Target: ara, Predicted: jav +Key: ara_sada_acw_000052, Target: ara, Predicted: mlt +Key: ara_sada_acw_000116, Target: ara, Predicted: cym +Key: ara_sada_acw_000053, Target: ara, Predicted: heb +Key: ara_sada_acw_000117, Target: ara, Predicted: ben +Key: ara_sada_acw_000118, Target: ara, Predicted: slv +Key: ara_sada_acw_000023, Target: ara, Predicted: mon +Key: ara_sada_acw_000056, Target: ara, Predicted: heb +Key: ara_sada_acw_000120, Target: ara, Predicted: yid +Key: ara_sada_acw_000025, Target: ara, Predicted: snd +Key: ara_sada_acw_000057, Target: ara, Predicted: fra +Key: ara_sada_acw_000124, Target: ara, Predicted: lit +Key: ara_sada_acw_000029, Target: ara, Predicted: sin +Key: ara_sada_acw_000125, Target: ara, Predicted: yid +Key: ara_sada_acw_000094, Target: ara, Predicted: tat +Key: ara_sada_acw_000063, Target: ara, Predicted: amh +Key: ara_sada_acw_000127, Target: ara, Predicted: xty +Key: ara_sada_acw_000129, Target: ara, Predicted: pol +Key: ara_sada_afb_000018, Target: ara, Predicted: afr +Key: ara_sada_acw_000130, Target: ara, Predicted: war +Key: ara_sada_acw_000162, Target: ara, Predicted: ell +Key: ara_sada_acw_000131, Target: ara, Predicted: guj +Key: ara_sada_acw_000163, Target: ara, Predicted: sna +Key: ara_sada_acw_000132, Target: ara, Predicted: bre +Key: ara_sada_acw_000164, Target: ara, Predicted: mlt +Key: ara_sada_afb_000021, Target: ara, Predicted: ell +Key: ara_sada_afb_000054, Target: ara, Predicted: nld +Key: ara_sada_afb_000023, Target: ara, Predicted: heb +Key: ara_sada_acw_000135, Target: ara, Predicted: som +Key: ara_sada_acw_000167, Target: ara, Predicted: isl +Key: ara_sada_afb_000025, Target: ara, Predicted: mon +Key: ara_sada_afb_000057, Target: ara, Predicted: sqi +Key: ara_sada_acw_000169, Target: ara, Predicted: deu +Key: ara_sada_afb_000058, Target: ara, Predicted: hau +Key: ara_sada_acw_000170, Target: ara, Predicted: mon +Key: ara_sada_acw_000139, Target: ara, Predicted: azz +Key: ara_sada_acw_000171, Target: ara, Predicted: heb +Key: ara_sada_afb_000028, Target: ara, Predicted: amh +Key: ara_sada_acw_000140, Target: ara, Predicted: mon +Key: ara_sada_acw_000173, Target: ara, Predicted: tat +Key: ara_sada_afb_000032, Target: ara, Predicted: tuk +Key: ara_sada_afb_000002, Target: ara, Predicted: rus +Key: ara_sada_afb_000034, Target: ara, Predicted: ltz +Key: ara_sada_afb_000003, Target: ara, Predicted: hrv +Key: ara_sada_afb_000035, Target: ara, Predicted: som +Key: ara_sada_acw_000147, Target: ara, Predicted: mlt +Key: ara_sada_afb_000068, Target: ara, Predicted: fas +Key: ara_sada_afb_000037, Target: ara, Predicted: abk +Key: ara_sada_afb_000069, Target: ara, Predicted: hau +Key: ara_sada_acw_000149, Target: ara, Predicted: aze +Key: ara_sada_afb_000070, Target: ara, Predicted: pus +Key: ara_sada_afb_000007, Target: ara, Predicted: pus +Key: ara_sada_afb_000039, Target: ara, Predicted: heb +Key: ara_sada_afb_000071, Target: ara, Predicted: heb +Key: ara_sada_acw_000151, Target: ara, Predicted: heb +Key: ara_sada_afb_000008, Target: ara, Predicted: heb +Key: ara_sada_afb_000072, Target: ara, Predicted: cym +Key: ara_sada_afb_000009, Target: ara, Predicted: tat +Key: ara_sada_afb_000073, Target: ara, Predicted: mya +Key: ara_sada_acw_000153, Target: ara, Predicted: fra +Key: ara_sada_afb_000010, Target: ara, Predicted: heb +Key: ara_sada_afb_000042, Target: ara, Predicted: yid +Key: ara_sada_acw_000154, Target: ara, Predicted: som +Key: ara_sada_afb_000011, Target: ara, Predicted: nep +Key: ara_sada_afb_000043, Target: ara, Predicted: mlt +Key: ara_sada_afb_000075, Target: ara, Predicted: abk +Key: ara_sada_afb_000046, Target: ara, Predicted: nep +Key: ara_sada_afb_000016, Target: ara, Predicted: amh +Key: ara_sada_afb_000080, Target: ara, Predicted: nep +Key: ara_sada_afb_000178, Target: ara, Predicted: som +Key: ara_sada_afb_000083, Target: ara, Predicted: aze +Key: ara_sada_afb_000115, Target: ara, Predicted: som +Key: ara_sada_afb_000147, Target: ara, Predicted: fas +Key: ara_sada_afb_000084, Target: ara, Predicted: fra +Key: ara_sada_afb_000148, Target: ara, Predicted: fra +Key: ara_sada_afb_000085, Target: ara, Predicted: eng +Key: ara_sada_afb_000117, Target: ara, Predicted: glv +Key: ara_sada_afb_000181, Target: ara, Predicted: spa +Key: ara_sada_afb_000118, Target: ara, Predicted: guj +Key: ara_sada_afb_000119, Target: ara, Predicted: hun +Key: ara_sada_afb_000151, Target: ara, Predicted: fas +Key: ara_sada_afb_000183, Target: ara, Predicted: deu +Key: ara_sada_afb_000088, Target: ara, Predicted: swa +Key: ara_sada_afb_000184, Target: ara, Predicted: som +Key: ara_sada_afb_000121, Target: ara, Predicted: slv +Key: ara_sada_afb_000091, Target: ara, Predicted: tat +Key: ara_sada_afb_000155, Target: ara, Predicted: mya +Key: ara_sada_afb_000124, Target: ara, Predicted: mya +Key: ara_sada_afb_000156, Target: ara, Predicted: amh +Key: ara_sada_afb_000188, Target: ara, Predicted: mon +Key: ara_sada_afb_000094, Target: ara, Predicted: fra +Key: ara_sada_afb_000126, Target: ara, Predicted: heb +Key: ara_sada_afb_000190, Target: ara, Predicted: yor +Key: ara_sada_afb_000159, Target: ara, Predicted: heb +Key: ara_sada_afb_000160, Target: ara, Predicted: heb +Key: ara_sada_afb_000192, Target: ara, Predicted: tat +Key: ara_sada_afb_000129, Target: ara, Predicted: hau +Key: ara_sada_afb_000161, Target: ara, Predicted: som +Key: ara_sada_afb_000193, Target: ara, Predicted: hrv +Key: ara_sada_afb_000130, Target: ara, Predicted: deu +Key: ara_sada_afb_000162, Target: ara, Predicted: bre +Key: ara_sada_afb_000099, Target: ara, Predicted: afr +Key: ara_sada_afb_000131, Target: ara, Predicted: nld +Key: ara_sada_afb_000163, Target: ara, Predicted: sna +Key: ara_sada_afb_000100, Target: ara, Predicted: nep +Key: ara_sada_afb_000132, Target: ara, Predicted: bre +Key: ara_sada_afb_000164, Target: ara, Predicted: bod +Key: ara_sada_afb_000196, Target: ara, Predicted: uzb +Key: ara_sada_afb_000165, Target: ara, Predicted: heb +Key: ara_sada_afb_000197, Target: ara, Predicted: ben +Key: ara_sada_afb_000134, Target: ara, Predicted: eng +Key: ara_sada_afb_000166, Target: ara, Predicted: tgk +Key: ara_sada_afb_000167, Target: ara, Predicted: nso +Key: ara_sada_afb_000104, Target: ara, Predicted: heb +Key: ara_sada_afb_000136, Target: ara, Predicted: slv +Key: ara_sada_afb_000168, Target: ara, Predicted: swa +Key: ara_sada_afb_000137, Target: ara, Predicted: bod +Key: ara_sada_afb_000106, Target: ara, Predicted: eng +Key: ara_sada_afb_000138, Target: ara, Predicted: tat +Key: ara_sada_afb_000108, Target: ara, Predicted: eus +Key: ara_sada_afb_000140, Target: ara, Predicted: tat +Key: ara_sada_afb_000141, Target: ara, Predicted: eng +Key: ara_sada_afb_000173, Target: ara, Predicted: kat +Key: ara_sada_afb_000142, Target: ara, Predicted: nld +Key: ara_sada_afb_000208, Target: ara, Predicted: kan +Key: ara_sada_ars_000007, Target: ara, Predicted: amh +Key: ara_sada_ars_000039, Target: ara, Predicted: fra +Key: ara_sada_ars_000008, Target: ara, Predicted: som +Key: ara_sada_ars_000042, Target: ara, Predicted: ces +Key: ara_sada_ars_000043, Target: ara, Predicted: lit +Key: ara_sada_ars_000044, Target: ara, Predicted: hrv +Key: ara_sada_afb_000216, Target: ara, Predicted: heb +Key: ara_sada_ars_000016, Target: ara, Predicted: grn +Key: ara_sada_ars_000048, Target: ara, Predicted: som +Key: ara_sada_ars_000049, Target: ara, Predicted: heb +Key: ara_sada_afb_000220, Target: ara, Predicted: hye +Key: ara_sada_ars_000018, Target: ara, Predicted: bre +Key: ara_sada_afb_000221, Target: ara, Predicted: azz +Key: ara_sada_arb_000031, Target: ara, Predicted: yor +Key: ara_sada_ars_000019, Target: ara, Predicted: nld +Key: ara_sada_ars_000053, Target: ara, Predicted: cym +Key: ara_sada_ars_000022, Target: ara, Predicted: mlg +Key: ara_sada_ars_000055, Target: ara, Predicted: eng +Key: ara_sada_ars_000024, Target: ara, Predicted: som +Key: ara_sada_arb_000005, Target: ara, Predicted: heb +Key: ara_sada_ars_000058, Target: ara, Predicted: mya +Key: ara_sada_arb_000041, Target: ara, Predicted: mlg +Key: ara_sada_ars_000061, Target: ara, Predicted: isl +Key: ara_sada_ars_000030, Target: ara, Predicted: amh +Key: ara_sada_ars_000031, Target: ara, Predicted: snd +Key: ara_sada_ars_000032, Target: ara, Predicted: deu +Key: ara_sada_ars_000033, Target: ara, Predicted: bos +Key: ara_sada_ars_000037, Target: ara, Predicted: pus +Key: ara_sada_ars_000069, Target: ara, Predicted: bre +Key: ara_sada_ars_000038, Target: ara, Predicted: pol +Key: deu_swissdial_ag_000014, Target: deu, Predicted: afr +Key: ara_sada_ars_000072, Target: ara, Predicted: slk +Key: ara_sada_ars_000136, Target: ara, Predicted: nno +Key: deu_swissdial_ag_000015, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000016, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000017, Target: deu, Predicted: nld +Key: ara_sada_ars_000075, Target: ara, Predicted: deu +Key: deu_swissdial_ag_000019, Target: deu, Predicted: yid +Key: ara_sada_ars_000110, Target: ara, Predicted: pan +Key: ara_sada_ars_000111, Target: ara, Predicted: sqi +Key: deu_swissdial_ag_000023, Target: deu, Predicted: afr +Key: ara_sada_ars_000082, Target: ara, Predicted: fra +Key: ara_sada_ars_000146, Target: ara, Predicted: tuk +Key: deu_swissdial_ag_000025, Target: deu, Predicted: afr +Key: ara_sada_ars_000083, Target: ara, Predicted: pus +Key: ara_sada_ars_000115, Target: ara, Predicted: tat +Key: deu_swissdial_ag_000026, Target: deu, Predicted: cym +Key: deu_swissdial_ag_000027, Target: deu, Predicted: yid +Key: deu_swissdial_ag_000028, Target: deu, Predicted: nld +Key: ara_sada_ars_000150, Target: ara, Predicted: isl +Key: deu_swissdial_ag_000029, Target: deu, Predicted: ces +Key: ara_sada_ars_000151, Target: ara, Predicted: heb +Key: ara_sada_ars_000152, Target: ara, Predicted: amh +Key: deu_swissdial_ag_000031, Target: deu, Predicted: nld +Key: ara_sada_ars_000089, Target: ara, Predicted: bod +Key: ara_sada_ars_000121, Target: ara, Predicted: fra +Key: deu_swissdial_ag_000032, Target: deu, Predicted: afr +Key: ara_sada_ars_000090, Target: ara, Predicted: mar +Key: ara_sada_ars_000091, Target: ara, Predicted: fra +Key: ara_sada_ars_000123, Target: ara, Predicted: isl +Key: ara_sada_ars_000092, Target: ara, Predicted: cym +Key: deu_swissdial_ag_000035, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000004, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000037, Target: deu, Predicted: gle +Key: deu_swissdial_ag_000006, Target: deu, Predicted: cym +Key: deu_swissdial_ag_000038, Target: deu, Predicted: afr +Key: ara_sada_ars_000129, Target: ara, Predicted: heb +Key: deu_swissdial_ag_000008, Target: deu, Predicted: tat +Key: deu_swissdial_ag_000040, Target: deu, Predicted: afr +Key: ara_sada_ars_000098, Target: ara, Predicted: khm +Key: ara_sada_ars_000131, Target: ara, Predicted: heb +Key: deu_swissdial_ag_000042, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000013, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000110, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000047, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000113, Target: deu, Predicted: gle +Key: deu_swissdial_ag_000114, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000115, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000084, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000148, Target: deu, Predicted: gle +Key: deu_swissdial_ag_000085, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000117, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000149, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000086, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000118, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000150, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000055, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000151, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000088, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000120, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000089, Target: deu, Predicted: gle +Key: deu_swissdial_ag_000121, Target: deu, Predicted: cym +Key: deu_swissdial_ag_000058, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000122, Target: deu, Predicted: gle +Key: deu_swissdial_ag_000059, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000091, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000092, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000124, Target: deu, Predicted: ces +Key: deu_swissdial_ag_000093, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000125, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000157, Target: deu, Predicted: cym +Key: deu_swissdial_ag_000126, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000095, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000127, Target: deu, Predicted: slv +Key: deu_swissdial_ag_000159, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000064, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000096, Target: deu, Predicted: ces +Key: deu_swissdial_ag_000128, Target: deu, Predicted: gle +Key: deu_swissdial_ag_000097, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000099, Target: deu, Predicted: cym +Key: deu_swissdial_ag_000163, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000100, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000134, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000103, Target: deu, Predicted: gle +Key: deu_swissdial_ag_000135, Target: deu, Predicted: cym +Key: deu_swissdial_ag_000072, Target: deu, Predicted: gle +Key: deu_swissdial_be_000004, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000138, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000107, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000139, Target: deu, Predicted: gle +Key: deu_swissdial_ag_000076, Target: deu, Predicted: afr +Key: deu_swissdial_ag_000108, Target: deu, Predicted: ltz +Key: deu_swissdial_ag_000140, Target: deu, Predicted: nld +Key: deu_swissdial_be_000008, Target: deu, Predicted: cym +Key: deu_swissdial_ag_000077, Target: deu, Predicted: nld +Key: deu_swissdial_ag_000109, Target: deu, Predicted: gle +Key: deu_swissdial_ag_000141, Target: deu, Predicted: afr +Key: deu_swissdial_be_000042, Target: deu, Predicted: isl +Key: deu_swissdial_be_000043, Target: deu, Predicted: afr +Key: deu_swissdial_be_000075, Target: deu, Predicted: afr +Key: deu_swissdial_be_000107, Target: deu, Predicted: afr +Key: deu_swissdial_be_000044, Target: deu, Predicted: afr +Key: deu_swissdial_be_000076, Target: deu, Predicted: nld +Key: deu_swissdial_be_000108, Target: deu, Predicted: ltz +Key: deu_swissdial_be_000013, Target: deu, Predicted: afr +Key: deu_swissdial_be_000110, Target: deu, Predicted: afr +Key: deu_swissdial_be_000016, Target: deu, Predicted: nld +Key: deu_swissdial_be_000048, Target: deu, Predicted: afr +Key: deu_swissdial_be_000112, Target: deu, Predicted: afr +Key: deu_swissdial_be_000049, Target: deu, Predicted: afr +Key: deu_swissdial_be_000113, Target: deu, Predicted: afr +Key: deu_swissdial_be_000018, Target: deu, Predicted: afr +Key: deu_swissdial_be_000082, Target: deu, Predicted: afr +Key: deu_swissdial_be_000114, Target: deu, Predicted: nld +Key: deu_swissdial_be_000115, Target: deu, Predicted: afr +Key: deu_swissdial_be_000084, Target: deu, Predicted: nld +Key: deu_swissdial_be_000021, Target: deu, Predicted: ltz +Key: deu_swissdial_be_000022, Target: deu, Predicted: afr +Key: deu_swissdial_be_000054, Target: deu, Predicted: afr +Key: deu_swissdial_be_000023, Target: deu, Predicted: afr +Key: deu_swissdial_be_000087, Target: deu, Predicted: afr +Key: deu_swissdial_be_000024, Target: deu, Predicted: afr +Key: deu_swissdial_be_000120, Target: deu, Predicted: afr +Key: deu_swissdial_be_000025, Target: deu, Predicted: afr +Key: deu_swissdial_be_000089, Target: deu, Predicted: nld +Key: deu_swissdial_be_000121, Target: deu, Predicted: cym +Key: deu_swissdial_be_000059, Target: deu, Predicted: afr +Key: deu_swissdial_be_000091, Target: deu, Predicted: afr +Key: deu_swissdial_be_000123, Target: deu, Predicted: afr +Key: deu_swissdial_be_000060, Target: deu, Predicted: isl +Key: deu_swissdial_be_000124, Target: deu, Predicted: nld +Key: deu_swissdial_be_000093, Target: deu, Predicted: afr +Key: deu_swissdial_be_000125, Target: deu, Predicted: nld +Key: deu_swissdial_be_000030, Target: deu, Predicted: nld +Key: deu_swissdial_be_000062, Target: deu, Predicted: afr +Key: deu_swissdial_be_000031, Target: deu, Predicted: afr +Key: deu_swissdial_be_000095, Target: deu, Predicted: afr +Key: deu_swissdial_be_000096, Target: deu, Predicted: afr +Key: deu_swissdial_be_000033, Target: deu, Predicted: afr +Key: deu_swissdial_be_000129, Target: deu, Predicted: afr +Key: deu_swissdial_be_000034, Target: deu, Predicted: afr +Key: deu_swissdial_be_000066, Target: deu, Predicted: afr +Key: deu_swissdial_be_000035, Target: deu, Predicted: afr +Key: deu_swissdial_be_000036, Target: deu, Predicted: slv +Key: deu_swissdial_be_000039, Target: deu, Predicted: afr +Key: deu_swissdial_be_000103, Target: deu, Predicted: est +Key: deu_swissdial_be_000040, Target: deu, Predicted: afr +Key: deu_swissdial_bs_000005, Target: deu, Predicted: gle +Key: deu_swissdial_be_000105, Target: deu, Predicted: afr +Key: deu_swissdial_bs_000103, Target: deu, Predicted: ltz +Key: deu_swissdial_bs_000082, Target: deu, Predicted: nld +Key: deu_swissdial_bs_000114, Target: deu, Predicted: cym +Key: deu_swissdial_bs_000088, Target: deu, Predicted: ltz +Key: deu_swissdial_bs_000093, Target: deu, Predicted: gle +Key: deu_swissdial_bs_000031, Target: deu, Predicted: cym +Key: deu_swissdial_bs_000036, Target: deu, Predicted: ltz +Key: deu_swissdial_bs_000133, Target: deu, Predicted: nld +Key: deu_swissdial_gr_000088, Target: deu, Predicted: nld +Key: deu_swissdial_bs_000139, Target: deu, Predicted: afr +Key: deu_swissdial_bs_000144, Target: deu, Predicted: ltz +Key: deu_swissdial_gr_000064, Target: deu, Predicted: slv +Key: deu_swissdial_gr_000040, Target: deu, Predicted: afr +Key: deu_swissdial_gr_000105, Target: deu, Predicted: afr +Key: deu_swissdial_gr_000010, Target: deu, Predicted: afr +Key: deu_swissdial_gr_000114, Target: deu, Predicted: slv +Key: deu_swissdial_gr_000116, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000006, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000038, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000007, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000071, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000040, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000072, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000042, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000011, Target: deu, Predicted: nno +Key: deu_swissdial_lu_000043, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000044, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000077, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000014, Target: deu, Predicted: yid +Key: deu_swissdial_lu_000047, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000079, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000016, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000017, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000019, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000051, Target: deu, Predicted: cym +Key: deu_swissdial_lu_000052, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000053, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000085, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000054, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000023, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000055, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000087, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000024, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000056, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000026, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000058, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000090, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000027, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000059, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000092, Target: deu, Predicted: glv +Key: deu_swissdial_lu_000029, Target: deu, Predicted: gle +Key: deu_swissdial_lu_000093, Target: deu, Predicted: cym +Key: deu_swissdial_lu_000031, Target: deu, Predicted: cym +Key: deu_swissdial_lu_000095, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000096, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000033, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000065, Target: deu, Predicted: hun +Key: deu_swissdial_lu_000097, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000002, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000034, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000066, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000067, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000099, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000004, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000068, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000069, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000102, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000134, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000166, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000103, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000135, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000104, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000136, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000168, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000137, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000106, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000170, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000108, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000140, Target: deu, Predicted: gle +Key: deu_swissdial_lu_000112, Target: deu, Predicted: yid +Key: deu_swissdial_lu_000144, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000113, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000116, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000150, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000151, Target: deu, Predicted: nld +Key: deu_swissdial_sg_000010, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000121, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000123, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000155, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000124, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000157, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000126, Target: deu, Predicted: afr +Key: deu_swissdial_lu_000158, Target: deu, Predicted: ltz +Key: deu_swissdial_lu_000159, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000130, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000162, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000163, Target: deu, Predicted: nld +Key: deu_swissdial_sg_000021, Target: deu, Predicted: gle +Key: deu_swissdial_lu_000132, Target: deu, Predicted: nld +Key: deu_swissdial_lu_000164, Target: deu, Predicted: ltz +Key: deu_swissdial_sg_000022, Target: deu, Predicted: gle +Key: deu_swissdial_lu_000133, Target: deu, Predicted: nld +Key: deu_swissdial_vs_000006, Target: deu, Predicted: afr +Key: deu_swissdial_vs_000038, Target: deu, Predicted: cym +Key: deu_swissdial_vs_000042, Target: deu, Predicted: nld +Key: deu_swissdial_vs_000045, Target: deu, Predicted: nld +Key: deu_swissdial_vs_000047, Target: deu, Predicted: slv +Key: deu_swissdial_vs_000016, Target: deu, Predicted: nld +Key: deu_swissdial_vs_000048, Target: deu, Predicted: afr +Key: deu_swissdial_sg_000067, Target: deu, Predicted: cym +Key: deu_swissdial_vs_000053, Target: deu, Predicted: gle +Key: deu_swissdial_sg_000104, Target: deu, Predicted: est +Key: deu_swissdial_vs_000059, Target: deu, Predicted: nld +Key: deu_swissdial_sg_000078, Target: deu, Predicted: nld +Key: deu_swissdial_vs_000060, Target: deu, Predicted: afr +Key: deu_swissdial_vs_000030, Target: deu, Predicted: nld +Key: deu_swissdial_vs_000032, Target: deu, Predicted: afr +Key: deu_swissdial_vs_000034, Target: deu, Predicted: gle +Key: deu_swissdial_vs_000003, Target: deu, Predicted: afr +Key: deu_swissdial_vs_000004, Target: deu, Predicted: afr +Key: deu_swissdial_vs_000005, Target: deu, Predicted: cym +Key: deu_swissdial_zh_000026, Target: deu, Predicted: gle +Key: deu_swissdial_vs_000136, Target: deu, Predicted: afr +Key: deu_swissdial_vs_000073, Target: deu, Predicted: nld +Key: deu_swissdial_vs_000137, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000029, Target: deu, Predicted: gle +Key: deu_swissdial_zh_000031, Target: deu, Predicted: yid +Key: deu_swissdial_vs_000076, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000000, Target: deu, Predicted: nld +Key: deu_swissdial_zh_000032, Target: deu, Predicted: nld +Key: deu_swissdial_vs_000078, Target: deu, Predicted: gle +Key: deu_swissdial_vs_000082, Target: deu, Predicted: dan +Key: deu_swissdial_vs_000114, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000006, Target: deu, Predicted: ltz +Key: deu_swissdial_vs_000083, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000007, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000008, Target: deu, Predicted: ltz +Key: deu_swissdial_zh_000011, Target: deu, Predicted: cym +Key: deu_swissdial_vs_000088, Target: deu, Predicted: afr +Key: deu_swissdial_vs_000120, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000045, Target: deu, Predicted: est +Key: deu_swissdial_vs_000090, Target: deu, Predicted: afr +Key: deu_swissdial_vs_000124, Target: deu, Predicted: nld +Key: deu_swissdial_zh_000048, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000017, Target: deu, Predicted: afr +Key: deu_swissdial_vs_000127, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000020, Target: deu, Predicted: gle +Key: deu_swissdial_vs_000129, Target: deu, Predicted: gle +Key: deu_swissdial_zh_000025, Target: deu, Predicted: ltz +Key: deu_swissdial_zh_000122, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000059, Target: deu, Predicted: nld +Key: deu_swissdial_zh_000091, Target: deu, Predicted: ltz +Key: ell_cretan_cre_000013, Target: ell, Predicted: ukr +Key: deu_swissdial_zh_000060, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000124, Target: deu, Predicted: afr +Key: ell_cretan_cre_000014, Target: ell, Predicted: pus +Key: deu_swissdial_zh_000094, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000127, Target: deu, Predicted: nld +Key: ell_cretan_cre_000017, Target: ell, Predicted: swa +Key: deu_swissdial_zh_000129, Target: deu, Predicted: nld +Key: deu_swissdial_zh_000066, Target: deu, Predicted: nld +Key: deu_swissdial_zh_000098, Target: deu, Predicted: cym +Key: ell_cretan_cre_000020, Target: ell, Predicted: sqi +Key: deu_swissdial_zh_000133, Target: deu, Predicted: nld +Key: deu_swissdial_zh_000134, Target: deu, Predicted: gle +Key: deu_swissdial_zh_000103, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000072, Target: deu, Predicted: nld +Key: deu_swissdial_zh_000105, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000141, Target: deu, Predicted: gle +Key: deu_swissdial_zh_000078, Target: deu, Predicted: afr +Key: deu_swissdial_zh_000080, Target: deu, Predicted: gle +Key: deu_swissdial_zh_000112, Target: deu, Predicted: ltz +Key: ell_cretan_cre_000002, Target: ell, Predicted: swa +Key: ell_cretan_cre_000034, Target: ell, Predicted: bel +Key: deu_swissdial_zh_000081, Target: deu, Predicted: afr +Key: ell_cretan_cre_000003, Target: ell, Predicted: mlg +Key: ell_cretan_cre_000035, Target: ell, Predicted: hrv +Key: deu_swissdial_zh_000082, Target: deu, Predicted: nld +Key: ell_cretan_cre_000004, Target: ell, Predicted: mkd +Key: ell_cretan_cre_000036, Target: ell, Predicted: rus +Key: ell_cretan_cre_000037, Target: ell, Predicted: bel +Key: deu_swissdial_zh_000117, Target: deu, Predicted: afr +Key: ell_cretan_cre_000007, Target: ell, Predicted: srp +Key: ell_cretan_cre_000039, Target: ell, Predicted: tam +Key: ell_cretan_cre_000040, Target: ell, Predicted: sqi +Key: ell_cretan_cre_000041, Target: ell, Predicted: hrv +Key: deu_swissdial_zh_000120, Target: deu, Predicted: afr +Key: ell_cretan_cre_000011, Target: ell, Predicted: rus +Key: ell_cretan_cre_000076, Target: ell, Predicted: mkd +Key: ell_cretan_cre_000108, Target: ell, Predicted: ron +Key: ell_cretan_cre_000140, Target: ell, Predicted: hrv +Key: ell_cretan_cre_000109, Target: ell, Predicted: sqi +Key: ell_cretan_cre_000110, Target: ell, Predicted: bul +Key: ell_cretan_cre_000048, Target: ell, Predicted: rus +Key: ell_cretan_cre_000112, Target: ell, Predicted: lav +Key: ell_cretan_cre_000081, Target: ell, Predicted: slv +Key: ell_cretan_cre_000050, Target: ell, Predicted: por +Key: ell_cretan_cre_000115, Target: ell, Predicted: ita +Key: ell_cretan_cre_000147, Target: ell, Predicted: swa +Key: ell_cretan_cre_000052, Target: ell, Predicted: azz +Key: ell_cretan_cre_000084, Target: ell, Predicted: rus +Key: ell_cretan_cre_000053, Target: ell, Predicted: luo +Key: ell_cretan_cre_000117, Target: ell, Predicted: bel +Key: ell_cretan_cre_000149, Target: ell, Predicted: ita +Key: ell_cretan_cre_000054, Target: ell, Predicted: ita +Key: ell_cretan_cre_000150, Target: ell, Predicted: sqi +Key: ell_cretan_cre_000055, Target: ell, Predicted: ron +Key: ell_cretan_cre_000088, Target: ell, Predicted: srp +Key: ell_cretan_cre_000120, Target: ell, Predicted: por +Key: ell_cretan_cre_000153, Target: ell, Predicted: mkd +Key: ell_cretan_cre_000090, Target: ell, Predicted: ita +Key: ell_cretan_cre_000059, Target: ell, Predicted: mlg +Key: ell_cretan_cre_000091, Target: ell, Predicted: swa +Key: ell_cretan_cre_000155, Target: ell, Predicted: hrv +Key: ell_cretan_cre_000092, Target: ell, Predicted: ukr +Key: ell_cretan_cre_000156, Target: ell, Predicted: lao +Key: ell_cretan_cre_000157, Target: ell, Predicted: ita +Key: ell_cretan_cre_000094, Target: ell, Predicted: grn +Key: ell_cretan_cre_000126, Target: ell, Predicted: azz +Key: ell_cretan_cre_000064, Target: ell, Predicted: swa +Key: ell_cretan_cre_000128, Target: ell, Predicted: ita +Key: ell_cretan_cre_000160, Target: ell, Predicted: hrv +Key: ell_cretan_cre_000065, Target: ell, Predicted: hrv +Key: ell_cretan_cre_000097, Target: ell, Predicted: rus +Key: ell_cretan_cre_000161, Target: ell, Predicted: sqi +Key: ell_cretan_cre_000066, Target: ell, Predicted: ukr +Key: ell_cretan_cre_000098, Target: ell, Predicted: pus +Key: ell_cretan_cre_000067, Target: ell, Predicted: pus +Key: ell_cretan_cre_000099, Target: ell, Predicted: xty +Key: ell_cretan_cre_000163, Target: ell, Predicted: swa +Key: ell_cretan_cre_000101, Target: ell, Predicted: grn +Key: ell_cretan_cre_000071, Target: ell, Predicted: guj +Key: ell_cretan_cre_000103, Target: ell, Predicted: ina +Key: ell_cretan_cre_000167, Target: ell, Predicted: mkd +Key: ell_cretan_cre_000168, Target: ell, Predicted: ita +Key: ell_cretan_cre_000073, Target: ell, Predicted: sqi +Key: ell_cretan_cre_000170, Target: ell, Predicted: sot +Key: ell_cretan_cre_000107, Target: ell, Predicted: ukr +Key: ell_cretan_cre_000139, Target: ell, Predicted: rus +Key: ell_cretan_cre_000171, Target: ell, Predicted: hrv +Key: ell_cretan_cre_000172, Target: ell, Predicted: mkd +Key: ell_cretan_cre_000268, Target: ell, Predicted: pol +Key: ell_cretan_cre_000237, Target: ell, Predicted: sqi +Key: ell_cretan_cre_000174, Target: ell, Predicted: bel +Key: ell_cretan_cre_000206, Target: ell, Predicted: aze +Key: ell_cretan_cre_000270, Target: ell, Predicted: ron +Key: ell_cretan_cre_000239, Target: ell, Predicted: ukr +Key: ell_cretan_cre_000240, Target: ell, Predicted: ron +Key: ell_cretan_cre_000177, Target: ell, Predicted: lit +Key: ell_cretan_cre_000209, Target: ell, Predicted: azz +Key: ell_cretan_cre_000241, Target: ell, Predicted: lit +Key: ell_cretan_cre_000242, Target: ell, Predicted: abk +Key: ell_cretan_cre_000274, Target: ell, Predicted: slv +Key: ell_cretan_cre_000275, Target: ell, Predicted: ukr +Key: ell_cretan_cre_000181, Target: ell, Predicted: azz +Key: ell_cretan_cre_000245, Target: ell, Predicted: por +Key: ell_cretan_cre_000248, Target: ell, Predicted: tuk +Key: ell_cretan_cre_000185, Target: ell, Predicted: guj +Key: ell_cretan_cre_000249, Target: ell, Predicted: guj +Key: ell_cretan_cre_000250, Target: ell, Predicted: ron +Key: ell_cretan_cre_000282, Target: ell, Predicted: ron +Key: ell_cretan_cre_000187, Target: ell, Predicted: por +Key: ell_cretan_cre_000219, Target: ell, Predicted: nep +Key: ell_cretan_cre_000251, Target: ell, Predicted: sqi +Key: ell_cretan_cre_000252, Target: ell, Predicted: pol +Key: ell_cretan_cre_000189, Target: ell, Predicted: rus +Key: ell_cretan_cre_000285, Target: ell, Predicted: bel +Key: ell_cretan_cre_000286, Target: ell, Predicted: grn +Key: ell_cretan_cre_000288, Target: ell, Predicted: mlg +Key: ell_cretan_cre_000225, Target: ell, Predicted: sot +Key: ell_cretan_cre_000226, Target: ell, Predicted: ita +Key: ell_cretan_cre_000258, Target: ell, Predicted: lit +Key: ell_cretan_cre_000195, Target: ell, Predicted: por +Key: ell_cretan_cre_000227, Target: ell, Predicted: xho +Key: ell_cretan_cre_000259, Target: ell, Predicted: swa +Key: ell_cretan_cre_000228, Target: ell, Predicted: bel +Key: ell_cretan_cre_000197, Target: ell, Predicted: por +Key: ell_cretan_cre_000261, Target: ell, Predicted: ben +Key: ell_cretan_cre_000198, Target: ell, Predicted: hrv +Key: ell_cretan_cre_000230, Target: ell, Predicted: sna +Key: ell_messenian_mes_000005, Target: ell, Predicted: heb +Key: ell_cretan_cre_000200, Target: ell, Predicted: swa +Key: ell_cretan_cre_000232, Target: ell, Predicted: hrv +Key: ell_cretan_cre_000264, Target: ell, Predicted: sun +Key: ell_cretan_cre_000265, Target: ell, Predicted: ces +Key: ell_cretan_cre_000202, Target: ell, Predicted: mkd +Key: ell_cretan_cre_000266, Target: ell, Predicted: ukr +Key: ell_cretan_cre_000203, Target: ell, Predicted: mkd +Key: ell_cretan_cre_000235, Target: ell, Predicted: mkd +Key: ell_cretan_cre_000267, Target: ell, Predicted: azz +Key: ell_messenian_mes_000009, Target: ell, Predicted: cym +Key: ell_messenian_mes_000011, Target: ell, Predicted: cat +Key: ell_messenian_mes_000043, Target: ell, Predicted: hrv +Key: ell_messenian_mes_000077, Target: ell, Predicted: ces +Key: ell_messenian_mes_000079, Target: ell, Predicted: cym +Key: ell_messenian_mes_000112, Target: ell, Predicted: mkd +Key: ell_messenian_mes_000085, Target: ell, Predicted: cym +Key: ell_messenian_mes_000087, Target: ell, Predicted: heb +Key: ell_messenian_mes_000056, Target: ell, Predicted: nno +Key: ell_messenian_mes_000089, Target: ell, Predicted: sqi +Key: ell_messenian_mes_000062, Target: ell, Predicted: cym +Key: ell_messenian_mes_000099, Target: ell, Predicted: bul +Key: ell_messenian_mes_000136, Target: ell, Predicted: cym +Key: ell_messenian_mes_000139, Target: ell, Predicted: hrv +Key: ell_messenian_mes_000141, Target: ell, Predicted: mkd +Key: ell_messenian_mes_000143, Target: ell, Predicted: slv +Key: eng_globe_aus_000018, Target: eng, Predicted: sqi +Key: ell_messenian_mes_000155, Target: ell, Predicted: nno +Key: ell_messenian_mes_000156, Target: ell, Predicted: heb +Key: eng_globe_aus_000000, Target: eng, Predicted: gle +Key: ell_messenian_mes_000161, Target: ell, Predicted: ces +Key: ell_messenian_mes_000164, Target: ell, Predicted: cym +Key: eng_globe_aus_000143, Target: eng, Predicted: tam +Key: eng_globe_aus_000082, Target: eng, Predicted: sqi +Key: eng_globe_aus_000118, Target: eng, Predicted: tgl +Key: eng_globe_bre_000034, Target: eng, Predicted: hun +Key: eng_globe_bre_000100, Target: eng, Predicted: cym +Key: eng_globe_bre_000133, Target: eng, Predicted: nor +Key: eng_globe_bre_000116, Target: eng, Predicted: gle +Key: eng_globe_bre_000124, Target: eng, Predicted: azz +Key: eng_globe_bre_000130, Target: eng, Predicted: cym +Key: eng_globe_bre_000099, Target: eng, Predicted: deu +Key: eng_globe_can_000087, Target: eng, Predicted: deu +Key: eng_globe_can_000063, Target: eng, Predicted: kor +Key: eng_globe_can_000098, Target: eng, Predicted: glv +Key: eng_globe_fil_000016, Target: eng, Predicted: ces +Key: eng_globe_fil_000000, Target: eng, Predicted: gle +Key: eng_globe_fil_000070, Target: eng, Predicted: nld +Key: eng_globe_fil_000008, Target: eng, Predicted: glv +Key: eng_globe_gle_000007, Target: eng, Predicted: cym +Key: eng_globe_fil_000146, Target: eng, Predicted: tgl +Key: eng_globe_gle_000030, Target: eng, Predicted: snd +Key: eng_globe_gle_000069, Target: eng, Predicted: gle +Key: eng_globe_gle_000104, Target: eng, Predicted: gle +Key: eng_globe_gle_000137, Target: eng, Predicted: gle +Key: eng_globe_gle_000055, Target: eng, Predicted: gle +Key: eng_globe_gle_000087, Target: eng, Predicted: cym +Key: eng_globe_gle_000154, Target: eng, Predicted: tel +Key: eng_globe_gle_000126, Target: eng, Predicted: gle +Key: eng_globe_gle_000167, Target: eng, Predicted: hin +Key: eng_globe_nze_000066, Target: eng, Predicted: mri +Key: eng_globe_nze_000108, Target: eng, Predicted: urd +Key: eng_globe_sae_000008, Target: eng, Predicted: ben +Key: eng_globe_sae_000011, Target: eng, Predicted: tam +Key: eng_globe_sae_000015, Target: eng, Predicted: urd +Key: eng_globe_sae_000062, Target: eng, Predicted: ben +Key: eng_globe_sae_000063, Target: eng, Predicted: xho +Key: eng_globe_sae_000033, Target: eng, Predicted: tel +Key: eng_globe_sae_000067, Target: eng, Predicted: tgl +Key: eng_globe_sae_000168, Target: eng, Predicted: cym +Key: eng_globe_sae_000169, Target: eng, Predicted: tam +Key: eng_globe_sae_000140, Target: eng, Predicted: nep +Key: eng_globe_sae_000109, Target: eng, Predicted: glv +Key: eng_globe_sae_000143, Target: eng, Predicted: gle +Key: eng_globe_sae_000115, Target: eng, Predicted: tel +Key: eng_globe_sco_000009, Target: eng, Predicted: cym +Key: eng_globe_sae_000160, Target: eng, Predicted: ltz +Key: eng_globe_sae_000161, Target: eng, Predicted: tam +Key: eng_globe_sae_000165, Target: eng, Predicted: deu +Key: eng_globe_sco_000093, Target: eng, Predicted: glv +Key: eng_globe_sco_000062, Target: eng, Predicted: cym +Key: eng_globe_sco_000140, Target: eng, Predicted: nld +Key: eng_globe_sco_000078, Target: eng, Predicted: cym +Key: eng_globe_sco_000110, Target: eng, Predicted: gle +Key: eng_globe_sco_000047, Target: eng, Predicted: cym +Key: eng_globe_sco_000144, Target: eng, Predicted: gle +Key: eng_globe_sco_000053, Target: eng, Predicted: gle +Key: eng_globe_use_000098, Target: eng, Predicted: nep +Key: eng_globe_use_000038, Target: eng, Predicted: mri +Key: eng_globe_use_000102, Target: eng, Predicted: deu +Key: eng_globe_use_000044, Target: eng, Predicted: ltz +Key: eng_globe_use_000076, Target: eng, Predicted: sot +Key: eng_globe_use_000113, Target: eng, Predicted: glv +Key: eng_globe_use_000115, Target: eng, Predicted: oci +Key: eng_globe_use_000020, Target: eng, Predicted: cym +Key: eng_globe_use_000023, Target: eng, Predicted: msa +Key: eng_globe_use_000090, Target: eng, Predicted: cym +Key: eng_globe_use_000059, Target: eng, Predicted: deu +Key: eng_l2arctic_ara_000029, Target: eng, Predicted: sqi +Key: eng_globe_use_000176, Target: eng, Predicted: cym +Key: eng_l2arctic_ara_000002, Target: eng, Predicted: ara +Key: eng_l2arctic_ara_000003, Target: eng, Predicted: ara +Key: eng_l2arctic_ara_000069, Target: eng, Predicted: ara +Key: eng_l2arctic_ara_000038, Target: eng, Predicted: ara +Key: eng_l2arctic_ara_000070, Target: eng, Predicted: fao +Key: eng_l2arctic_ara_000040, Target: eng, Predicted: glv +Key: eng_l2arctic_ara_000077, Target: eng, Predicted: ces +Key: eng_l2arctic_ara_000046, Target: eng, Predicted: ara +Key: eng_l2arctic_ara_000079, Target: eng, Predicted: fra +Key: eng_l2arctic_ara_000080, Target: eng, Predicted: nld +Key: eng_l2arctic_ara_000146, Target: eng, Predicted: pus +Key: eng_l2arctic_cmn_000011, Target: eng, Predicted: cmn +Key: eng_l2arctic_ara_000122, Target: eng, Predicted: hye +Key: eng_l2arctic_ara_000123, Target: eng, Predicted: kat +Key: eng_l2arctic_ara_000127, Target: eng, Predicted: ara +Key: eng_l2arctic_ara_000128, Target: eng, Predicted: som +Key: eng_l2arctic_ara_000160, Target: eng, Predicted: deu +Key: eng_l2arctic_ara_000129, Target: eng, Predicted: ara +Key: eng_l2arctic_ara_000130, Target: eng, Predicted: tgk +Key: eng_l2arctic_ara_000164, Target: eng, Predicted: ara +Key: eng_l2arctic_ara_000133, Target: eng, Predicted: nld +Key: eng_l2arctic_ara_000135, Target: eng, Predicted: hun +Key: eng_l2arctic_ara_000169, Target: eng, Predicted: ara +Key: eng_l2arctic_ara_000108, Target: eng, Predicted: ara +Key: eng_l2arctic_ara_000140, Target: eng, Predicted: ara +Key: eng_l2arctic_ara_000109, Target: eng, Predicted: ckb +Key: eng_l2arctic_cmn_000135, Target: eng, Predicted: cmn +Key: eng_l2arctic_cmn_000138, Target: eng, Predicted: cmn +Key: eng_l2arctic_cmn_000107, Target: eng, Predicted: cmn +Key: eng_l2arctic_cmn_000139, Target: eng, Predicted: cmn +Key: eng_l2arctic_cmn_000108, Target: eng, Predicted: cmn +Key: eng_l2arctic_cmn_000141, Target: eng, Predicted: lao +Key: eng_l2arctic_cmn_000078, Target: eng, Predicted: cmn +Key: eng_l2arctic_cmn_000047, Target: eng, Predicted: ron +Key: eng_l2arctic_cmn_000113, Target: eng, Predicted: cmn +Key: eng_l2arctic_hin_000001, Target: eng, Predicted: hin +Key: eng_l2arctic_cmn_000118, Target: eng, Predicted: por +Key: eng_l2arctic_cmn_000125, Target: eng, Predicted: mya +Key: eng_l2arctic_hin_000010, Target: eng, Predicted: hin +Key: eng_l2arctic_cmn_000094, Target: eng, Predicted: hun +Key: eng_l2arctic_cmn_000126, Target: eng, Predicted: cmn +Key: eng_l2arctic_cmn_000097, Target: eng, Predicted: bod +Key: eng_l2arctic_cmn_000100, Target: eng, Predicted: cmn +Key: eng_l2arctic_cmn_000133, Target: eng, Predicted: fas +Key: eng_l2arctic_hin_000119, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000025, Target: eng, Predicted: urd +Key: eng_l2arctic_hin_000122, Target: eng, Predicted: pus +Key: eng_l2arctic_hin_000123, Target: eng, Predicted: kan +Key: eng_l2arctic_hin_000061, Target: eng, Predicted: mar +Key: eng_l2arctic_hin_000126, Target: eng, Predicted: ben +Key: eng_l2arctic_hin_000127, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000096, Target: eng, Predicted: hin +Key: eng_l2arctic_hin_000128, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000131, Target: eng, Predicted: tel +Key: eng_l2arctic_hin_000132, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000069, Target: eng, Predicted: deu +Key: eng_l2arctic_hin_000101, Target: eng, Predicted: hin +Key: eng_l2arctic_hin_000133, Target: eng, Predicted: kan +Key: eng_l2arctic_hin_000102, Target: eng, Predicted: pan +Key: eng_l2arctic_hin_000104, Target: eng, Predicted: tel +Key: eng_l2arctic_hin_000106, Target: eng, Predicted: mal +Key: eng_l2arctic_hin_000108, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000140, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000141, Target: eng, Predicted: mar +Key: eng_l2arctic_hin_000110, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000144, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000147, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000149, Target: eng, Predicted: guj +Key: eng_l2arctic_hin_000152, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000154, Target: eng, Predicted: tel +Key: eng_l2arctic_hin_000188, Target: eng, Predicted: tel +Key: eng_l2arctic_hin_000157, Target: eng, Predicted: mal +Key: eng_l2arctic_hin_000159, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000191, Target: eng, Predicted: ben +Key: eng_l2arctic_hin_000193, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000163, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000195, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000164, Target: eng, Predicted: nep +Key: eng_l2arctic_hin_000196, Target: eng, Predicted: guj +Key: eng_l2arctic_hin_000166, Target: eng, Predicted: slv +Key: eng_l2arctic_kor_000022, Target: eng, Predicted: ltz +Key: eng_l2arctic_hin_000167, Target: eng, Predicted: tam +Key: eng_l2arctic_hin_000199, Target: eng, Predicted: urd +Key: eng_l2arctic_hin_000169, Target: eng, Predicted: cym +Key: eng_l2arctic_hin_000203, Target: eng, Predicted: pan +Key: eng_l2arctic_hin_000205, Target: eng, Predicted: tel +Key: eng_l2arctic_hin_000175, Target: eng, Predicted: pan +Key: eng_l2arctic_kor_000069, Target: eng, Predicted: dan +Key: eng_l2arctic_kor_000165, Target: eng, Predicted: slv +Key: eng_l2arctic_kor_000142, Target: eng, Predicted: xho +Key: eng_l2arctic_kor_000118, Target: eng, Predicted: bod +Key: eng_l2arctic_kor_000157, Target: eng, Predicted: hun +Key: eng_l2arctic_spa_000014, Target: eng, Predicted: heb +Key: eng_l2arctic_spa_000067, Target: eng, Predicted: ita +Key: eng_l2arctic_spa_000036, Target: eng, Predicted: azz +Key: eng_l2arctic_spa_000102, Target: eng, Predicted: ron +Key: eng_l2arctic_spa_000149, Target: eng, Predicted: fin +Key: eng_l2arctic_vie_000034, Target: eng, Predicted: deu +Key: eng_l2arctic_vie_000004, Target: eng, Predicted: tgl +Key: eng_l2arctic_vie_000069, Target: eng, Predicted: mri +Key: eng_l2arctic_vie_000106, Target: eng, Predicted: lao +Key: eng_l2arctic_vie_000044, Target: eng, Predicted: xho +Key: eng_l2arctic_vie_000109, Target: eng, Predicted: cym +Key: eng_l2arctic_vie_000047, Target: eng, Predicted: xho +Key: eng_l2arctic_vie_000016, Target: eng, Predicted: lat +Key: eng_l2arctic_vie_000022, Target: eng, Predicted: pus +Key: eng_openslr83_nor_000017, Target: eng, Predicted: gle +Key: eng_openslr83_mid_000076, Target: eng, Predicted: glv +Key: eng_openslr83_mid_000078, Target: eng, Predicted: cym +Key: eng_openslr83_nor_000031, Target: eng, Predicted: cym +Key: eng_openslr83_nor_000069, Target: eng, Predicted: cym +Key: eng_openslr83_nor_000070, Target: eng, Predicted: cym +Key: eng_openslr83_nor_000074, Target: eng, Predicted: cym +Key: eng_openslr83_nor_000050, Target: eng, Predicted: cym +Key: eng_openslr83_nor_000083, Target: eng, Predicted: gle +Key: eng_openslr83_nor_000092, Target: eng, Predicted: cym +Key: eng_openslr83_sou_000065, Target: eng, Predicted: cym +Key: eng_openslr83_sco_000086, Target: eng, Predicted: gle +Key: eng_openslr83_sco_000091, Target: eng, Predicted: gle +Key: eng_openslr83_wel_000002, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000003, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000035, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000067, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000068, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000037, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000069, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000006, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000038, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000040, Target: eng, Predicted: dan +Key: eng_openslr83_wel_000072, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000009, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000073, Target: eng, Predicted: gle +Key: eng_openslr83_wel_000010, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000042, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000074, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000011, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000075, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000012, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000076, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000013, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000077, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000014, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000046, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000015, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000079, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000080, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000081, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000082, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000052, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000084, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000053, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000085, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000022, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000086, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000087, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000056, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000088, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000057, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000026, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000058, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000090, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000060, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000092, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000031, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000063, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000032, Target: eng, Predicted: glv +Key: eng_openslr83_wel_000065, Target: eng, Predicted: cym +Key: eng_openslr83_wel_000034, Target: eng, Predicted: gle +Key: eng_openslr83_wel_000066, Target: eng, Predicted: cym +Key: eng_voxpopuli_est_000017, Target: eng, Predicted: deu +Key: eng_voxpopuli_est_000008, Target: eng, Predicted: est +Key: eng_voxpopuli_hun_000015, Target: eng, Predicted: hun +Key: eng_voxpopuli_hun_000057, Target: eng, Predicted: hun +Key: eng_voxpopuli_pol_000021, Target: eng, Predicted: pol +Key: eng_voxpopuli_pol_000024, Target: eng, Predicted: pol +Key: eng_voxpopuli_nld_000034, Target: eng, Predicted: nld +Key: eng_voxpopuli_pol_000038, Target: eng, Predicted: pol +Key: eng_voxpopuli_ron_000015, Target: eng, Predicted: ron +Key: eng_voxpopuli_ron_000017, Target: eng, Predicted: ron +Key: guj_ms_speech_guj_000038, Target: guj, Predicted: mar +Key: spa_openslr_spa_arg_000015, Target: spa, Predicted: por +Key: spa_openslr_spa_arg_000018, Target: spa, Predicted: mlt +Key: guj_ms_speech_guj_000083, Target: guj, Predicted: hin +Key: spa_openslr_spa_arg_000022, Target: spa, Predicted: est +Key: spa_openslr_spa_arg_000062, Target: spa, Predicted: hau +Key: spa_openslr_spa_arg_000031, Target: spa, Predicted: isl +Key: spa_openslr_spa_arg_000003, Target: spa, Predicted: sot +Key: spa_openslr_spa_arg_000006, Target: spa, Predicted: ita +Key: spa_openslr_spa_arg_000079, Target: spa, Predicted: cym +Key: spa_openslr_spa_chi_000049, Target: spa, Predicted: ell +Key: spa_openslr_spa_arg_000114, Target: spa, Predicted: eus +Key: spa_openslr_spa_chi_000022, Target: spa, Predicted: grn +Key: spa_openslr_spa_chi_000037, Target: spa, Predicted: ita +Key: spa_openslr_spa_chi_000076, Target: spa, Predicted: sot +Key: spa_openslr_spa_col_000001, Target: spa, Predicted: por +Key: spa_openslr_spa_col_000007, Target: spa, Predicted: ita +Key: spa_openslr_spa_col_000013, Target: spa, Predicted: ita +Key: spa_openslr_spa_col_000081, Target: spa, Predicted: ron +Key: spa_openslr_spa_col_000056, Target: spa, Predicted: ita +Key: spa_openslr_spa_col_000099, Target: spa, Predicted: epo +Key: spa_openslr_spa_per_000018, Target: spa, Predicted: glg +Key: spa_openslr_spa_per_000063, Target: spa, Predicted: ina +Key: spa_openslr_spa_per_000033, Target: spa, Predicted: ina +Key: spa_openslr_spa_per_000065, Target: spa, Predicted: glg +Key: spa_openslr_spa_per_000035, Target: spa, Predicted: ita +Key: spa_openslr_spa_pue_000060, Target: spa, Predicted: ita +Key: spa_openslr_spa_pue_000094, Target: spa, Predicted: eus +Key: spa_openslr_spa_pue_000003, Target: spa, Predicted: ita +Key: spa_openslr_spa_pue_000071, Target: spa, Predicted: ita +Key: spa_openslr_spa_pue_000014, Target: spa, Predicted: gug +Key: spa_openslr_spa_pue_000051, Target: spa, Predicted: ron +Key: spa_openslr_spa_ven_000112, Target: spa, Predicted: eus +Key: spa_openslr_spa_ven_000116, Target: spa, Predicted: isl +Key: spa_openslr_spa_ven_000118, Target: spa, Predicted: ina +Key: tel_ms_speech_tel_000014, Target: tel, Predicted: mal diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang/lid_inference_test.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang/lid_inference_test.log new file mode 100644 index 0000000000000000000000000000000000000000..365c343c7d0c73358412ea1cc451fff27f70e2fa --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang/lid_inference_test.log @@ -0,0 +1,302 @@ +# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +# Started at Mon Jun 2 02:17:42 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_ml_superb2_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +[gpue04] 2025-06-02 02:18:18,542 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + torch.load(model_file, map_location=device), +[gpue04] 2025-06-02 02:18:35,714 (lid_inference_dist:86) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary. + warnings.warn(_create_warning_msg( +/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist. + warnings.warn( +[gpue04] 2025-06-02 02:18:36,278 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang/lids0 +[gpue04] 2025-06-02 02:19:16,352 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0 +[gpue04] 2025-06-02 02:19:50,349 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1 +[gpue04] 2025-06-02 02:20:29,160 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2 +[gpue04] 2025-06-02 02:21:04,869 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3 +[gpue04] 2025-06-02 02:21:41,255 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4 +[gpue04] 2025-06-02 02:22:12,467 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5 +[gpue04] 2025-06-02 02:22:52,319 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6 +[gpue04] 2025-06-02 02:23:31,893 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7 +[gpue04] 2025-06-02 02:24:09,283 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8 +[gpue04] 2025-06-02 02:24:49,938 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9 +[gpue04] 2025-06-02 02:25:28,601 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10 +[gpue04] 2025-06-02 02:26:08,097 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11 +[gpue04] 2025-06-02 02:26:47,335 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12 +[gpue04] 2025-06-02 02:27:25,789 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13 +[gpue04] 2025-06-02 02:27:56,679 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14 +[gpue04] 2025-06-02 02:28:40,096 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15 +[gpue04] 2025-06-02 02:29:19,221 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 16 +[gpue04] 2025-06-02 02:29:55,785 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 17 +[gpue04] 2025-06-02 02:30:34,585 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 18 +[gpue04] 2025-06-02 02:31:08,302 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 19 +[gpue04] 2025-06-02 02:31:41,628 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 20 +[gpue04] 2025-06-02 02:32:15,682 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 21 +[gpue04] 2025-06-02 02:32:50,538 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 22 +[gpue04] 2025-06-02 02:33:12,317 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True +[gpue04] 2025-06-02 02:33:12,318 (lid_inference_dist:215) INFO: args.save_tsne_plot: False +# Accounting: time=931 threads=1 +# Ended (code 0) at Mon Jun 2 02:33:13 CDT 2025, elapsed time 931 seconds diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang/results b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang/results new file mode 100644 index 0000000000000000000000000000000000000000..ba41876ced4bea5f1c5b13396e548b2092131a75 --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_ml_superb2_lang_cross_train_all_no_filter_lang/results @@ -0,0 +1,2830 @@ +Accuracy: 88.61% +Macro Accuracy: 88.89% +Accuracy per Language: +kir: 84.70% +chv: 100.00% +sun: 80.00% +frr: 51.40% +jpn: 93.01% +tok: 99.38% +uzb: 95.03% +oci: 88.89% +eng: 97.70% +swa: 97.72% +azz: 92.04% +jav: 85.19% +skr: 95.97% +ces: 86.55% +kat: 69.91% +tam: 94.98% +tsn: 0.00% +ori: 98.14% +bas: 100.00% +lug: 100.00% +ron: 98.35% +nld: 97.59% +vie: 94.58% +nbl: 38.28% +por: 93.83% +ara: 95.38% +mhr: 90.00% +mar: 99.13% +lit: 99.40% +wol: 100.00% +pus: 100.00% +lao: 94.74% +kin: 89.19% +hye: 99.33% +kam: 96.88% +spa: 86.42% +ven: 86.36% +kab: 84.46% +xty: 98.99% +eus: 97.34% +fil: 100.00% +abk: 99.26% +ukr: 94.83% +cat: 98.21% +aze: 100.00% +ful: 68.18% +tgk: 97.44% +mal: 91.56% +hrv: 90.00% +tat: 72.31% +mri: 100.00% +ibo: 90.91% +ell: 99.49% +slk: 87.19% +pan: 98.15% +asm: 100.00% +isl: 100.00% +ben: 93.53% +fin: 99.12% +myv: 97.94% +ina: 81.15% +mon: 99.35% +slv: 93.12% +srp: 52.32% +uig: 100.00% +sna: 100.00% +mlt: 98.82% +luo: 100.00% +epo: 79.46% +gle: 98.33% +msa: 95.00% +umb: 80.00% +rus: 95.45% +fas: 84.03% +yue: 97.21% +yor: 100.00% +bre: 97.21% +heb: 90.00% +kan: 95.45% +nep: 94.42% +ssw: 66.02% +sot: 39.35% +cym: 89.21% +tel: 100.00% +ltz: 98.04% +dan: 73.94% +khm: 96.97% +hsb: 100.00% +orm: 93.75% +mya: 100.00% +lin: 100.00% +ckb: 94.97% +guj: 98.69% +cmn: 83.85% +zul: 23.27% +urd: 78.68% +tha: 94.19% +sah: 100.00% +mkd: 100.00% +hau: 100.00% +bos: 100.00% +kmr: 82.31% +cnh: 100.00% +kea: 86.36% +ceb: 96.88% +tur: 79.43% +glg: 98.09% +hun: 99.07% +deu: 97.52% +lav: 98.68% +amh: 99.32% +som: 100.00% +mrj: 19.30% +swe: 93.05% +tos: 23.39% +hin: 64.24% +est: 100.00% +tso: 0.00% +fra: 94.65% +ast: 67.16% +div: 98.17% +ita: 98.56% +nso: 79.81% +kaz: 99.25% +bak: 98.21% +snd: 97.87% +grn: 98.52% +afr: 95.83% +kor: 100.00% +sin: 91.24% +nan: 89.66% +nya: 91.18% +bel: 96.64% +bul: 100.00% +ind: 81.77% +xho: 74.18% +pol: 96.10% +Key: abk_cv_abk_001026, Target: abk, Predicted: kin +Key: afr_nchlt_afr_001180, Target: afr, Predicted: slv +Key: afr_fleurs_afr_000032, Target: afr, Predicted: deu +Key: afr_nchlt_afr_001219, Target: afr, Predicted: por +Key: afr_nchlt_afr_001222, Target: afr, Predicted: nld +Key: afr_nchlt_afr_001230, Target: afr, Predicted: eng +Key: afr_nchlt_afr_001169, Target: afr, Predicted: eng +Key: afr_nchlt_afr_001254, Target: afr, Predicted: slv +Key: afr_nchlt_afr_001291, Target: afr, Predicted: srp +Key: afr_nchlt_afr_001328, Target: afr, Predicted: slv +Key: afr_nchlt_afr_001267, Target: afr, Predicted: swe +Key: amh_ALFFA_amh_000173, Target: amh, Predicted: orm +Key: ara_cv_ara_000831, Target: ara, Predicted: ces +Key: ara_cv_ara_000863, Target: ara, Predicted: fas +Key: ara_cv_ara_000832, Target: ara, Predicted: bre +Key: ara_cv_ara_000833, Target: ara, Predicted: ces +Key: ara_cv_ara_000873, Target: ara, Predicted: kab +Key: ara_cv_ara_000843, Target: ara, Predicted: kmr +Key: ara_cv_ara_000816, Target: ara, Predicted: ckb +Key: ara_cv_ara_000888, Target: ara, Predicted: ckb +Key: ara_cv_ara_000937, Target: ara, Predicted: ces +Key: ast_fleurs_ast_000001, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000005, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000037, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000042, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000043, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000012, Target: ast, Predicted: ita +Key: ast_fleurs_ast_000015, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000017, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000049, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000018, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000051, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000052, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000021, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000053, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000023, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000024, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000056, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000026, Target: ast, Predicted: spa +Key: azz_mexico-el_azz_000773, Target: azz, Predicted: xty +Key: azz_mexico-el_azz_000776, Target: azz, Predicted: xty +Key: ast_fleurs_ast_000031, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000032, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000064, Target: ast, Predicted: spa +Key: ast_fleurs_ast_000033, Target: ast, Predicted: spa +Key: azz_mexico-el_azz_000781, Target: azz, Predicted: mon +Key: azz_mexico-el_azz_000783, Target: azz, Predicted: xty +Key: azz_mexico-el_azz_000785, Target: azz, Predicted: xty +Key: azz_mexico-el_azz_000786, Target: azz, Predicted: bul +Key: bak_cv_bak_000675, Target: bak, Predicted: tat +Key: azz_mexico-el_azz_000861, Target: azz, Predicted: xty +Key: azz_mexico-el_azz_000798, Target: azz, Predicted: xty +Key: azz_mexico-el_azz_000802, Target: azz, Predicted: xty +Key: bak_cv_bak_000701, Target: bak, Predicted: mrj +Key: bel_cv_bel_000644, Target: bel, Predicted: ukr +Key: bel_cv_bel_000661, Target: bel, Predicted: rus +Key: bel_cv_bel_000722, Target: bel, Predicted: rus +Key: bel_cv_bel_000692, Target: bel, Predicted: ukr +Key: bel_cv_bel_000699, Target: bel, Predicted: rus +Key: ben_googlei18n-asr_ben_001082, Target: ben, Predicted: nep +Key: ben_googlei18n-asr_ben_001086, Target: ben, Predicted: nep +Key: ben_googlei18n-asr_ben_001118, Target: ben, Predicted: ssw +Key: ben_googlei18n-asr_ben_001023, Target: ben, Predicted: tel +Key: ben_googlei18n-asr_ben_001121, Target: ben, Predicted: ori +Key: ben_googlei18n-asr_ben_001091, Target: ben, Predicted: sot +Key: ben_googlei18n-asr_ben_001030, Target: ben, Predicted: asm +Key: ben_googlei18n-asr_ben_001033, Target: ben, Predicted: asm +Key: ben_googlei18n-asr_ben_001035, Target: ben, Predicted: nep +Key: ben_googlei18n-asr_ben_001067, Target: ben, Predicted: nep +Key: ben_googlei18n-asr_ben_001036, Target: ben, Predicted: asm +Key: ben_googlei18n-asr_ben_001040, Target: ben, Predicted: asm +Key: ben_googlei18n-asr_ben_001140, Target: ben, Predicted: asm +Key: ben_googlei18n-asr_ben_001144, Target: ben, Predicted: sin +Key: ben_googlei18n-asr_ben_001147, Target: ben, Predicted: asm +Key: ben_googlei18n-tts_ben_000680, Target: ben, Predicted: asm +Key: ben_googlei18n-tts_ben_000712, Target: ben, Predicted: ces +Key: ben_googlei18n-tts_ben_000683, Target: ben, Predicted: mar +Key: ben_googlei18n-tts_ben_000718, Target: ben, Predicted: mal +Key: ben_googlei18n-asr_ben_001168, Target: ben, Predicted: nep +Key: ben_googlei18n-tts_ben_000759, Target: ben, Predicted: spa +Key: ben_googlei18n-tts_ben_000761, Target: ben, Predicted: asm +Key: ben_googlei18n-tts_ben_000698, Target: ben, Predicted: guj +Key: ben_googlei18n-asr_ben_001173, Target: ben, Predicted: mar +Key: ben_googlei18n-asr_ben_001174, Target: ben, Predicted: nep +Key: ben_googlei18n-tts_ben_000764, Target: ben, Predicted: hin +Key: ben_googlei18n-tts_ben_000777, Target: ben, Predicted: hin +Key: bre_cv_bre_001111, Target: bre, Predicted: ron +Key: bre_cv_bre_001158, Target: bre, Predicted: gle +Key: bre_cv_bre_001231, Target: bre, Predicted: lav +Key: bre_cv_bre_001280, Target: bre, Predicted: fra +Key: bre_cv_bre_001250, Target: bre, Predicted: gle +Key: cat_cv_cat_000629, Target: cat, Predicted: spa +Key: cat_cv_cat_000607, Target: cat, Predicted: skr +Key: cat_cv_cat_000641, Target: cat, Predicted: ast +Key: cat_cv_cat_000591, Target: cat, Predicted: ell +Key: ces_cv_ces_000938, Target: ces, Predicted: rus +Key: ces_cv_ces_000970, Target: ces, Predicted: kat +Key: ces_cv_ces_000972, Target: ces, Predicted: slk +Key: ces_cv_ces_000941, Target: ces, Predicted: slk +Key: ces_cv_ces_000910, Target: ces, Predicted: por +Key: ces_cv_ces_000944, Target: ces, Predicted: slk +Key: ces_cv_ces_000977, Target: ces, Predicted: uig +Key: ces_cv_ces_000978, Target: ces, Predicted: slk +Key: ceb_fleurs_ceb_000010, Target: ceb, Predicted: fil +Key: ces_cv_ces_000981, Target: ces, Predicted: slk +Key: ces_cv_ces_000991, Target: ces, Predicted: slk +Key: ces_cv_ces_000992, Target: ces, Predicted: kat +Key: ces_cv_ces_000931, Target: ces, Predicted: kat +Key: ces_cv_ces_000996, Target: ces, Predicted: slk +Key: ces_cv_ces_000965, Target: ces, Predicted: kat +Key: ces_cv_ces_000934, Target: ces, Predicted: bul +Key: ces_cv_ces_000966, Target: ces, Predicted: kat +Key: ces_cv_ces_000998, Target: ces, Predicted: slk +Key: ces_cv_ces_000967, Target: ces, Predicted: kat +Key: ces_cv_ces_000968, Target: ces, Predicted: rus +Key: ces_cv_ces_000937, Target: ces, Predicted: ukr +Key: ces_cv_ces_000969, Target: ces, Predicted: bos +Key: ces_cv_ces_001066, Target: ces, Predicted: slk +Key: ces_cv_ces_001068, Target: ces, Predicted: slk +Key: ces_cv_ces_001037, Target: ces, Predicted: kab +Key: ces_cv_ces_001044, Target: ces, Predicted: kat +Key: ces_cv_ces_001076, Target: ces, Predicted: slk +Key: ces_cv_ces_001047, Target: ces, Predicted: lav +Key: ces_cv_ces_001081, Target: ces, Predicted: slk +Key: ces_cv_ces_001019, Target: ces, Predicted: slk +Key: ces_cv_ces_001051, Target: ces, Predicted: slk +Key: ces_cv_ces_001084, Target: ces, Predicted: slk +Key: ces_cv_ces_001022, Target: ces, Predicted: slk +Key: ces_cv_ces_001054, Target: ces, Predicted: mhr +Key: ces_cv_ces_001088, Target: ces, Predicted: kat +Key: ces_cv_ces_001026, Target: ces, Predicted: slk +Key: ces_cv_ces_001090, Target: ces, Predicted: kir +Key: ces_cv_ces_001059, Target: ces, Predicted: kat +Key: ces_cv_ces_001093, Target: ces, Predicted: kat +Key: ces_cv_ces_001065, Target: ces, Predicted: slk +Key: ckb_cv_ckb_000795, Target: ckb, Predicted: ara +Key: ckb_cv_ckb_000801, Target: ckb, Predicted: deu +Key: ckb_cv_ckb_000836, Target: ckb, Predicted: rus +Key: ckb_cv_ckb_000846, Target: ckb, Predicted: mrj +Key: ckb_cv_ckb_000851, Target: ckb, Predicted: kmr +Key: ckb_cv_ckb_000896, Target: ckb, Predicted: kmr +Key: ckb_fleurs_ckb_000042, Target: ckb, Predicted: pus +Key: ckb_cv_ckb_000870, Target: ckb, Predicted: hin +Key: cmn_cv_cmn_000630, Target: cmn, Predicted: cnh +Key: cmn_cv_cmn_000633, Target: cmn, Predicted: kat +Key: cmn_cv_cmn_000634, Target: cmn, Predicted: nan +Key: ckb_cv_ckb_000920, Target: ckb, Predicted: abk +Key: cmn_cv_cmn_000640, Target: cmn, Predicted: ori +Key: cmn_cv_cmn_000673, Target: cmn, Predicted: kir +Key: cmn_cv_cmn_000705, Target: cmn, Predicted: yue +Key: cmn_cv_cmn_000674, Target: cmn, Predicted: kir +Key: cmn_cv_cmn_000706, Target: cmn, Predicted: kat +Key: cmn_cv_cmn_000675, Target: cmn, Predicted: abk +Key: cmn_cv_cmn_000676, Target: cmn, Predicted: kat +Key: cmn_cv_cmn_000645, Target: cmn, Predicted: nan +Key: cmn_cv_cmn_000677, Target: cmn, Predicted: abk +Key: cmn_cv_cmn_000678, Target: cmn, Predicted: nan +Key: cmn_cv_cmn_000651, Target: cmn, Predicted: ces +Key: cmn_cv_cmn_000683, Target: cmn, Predicted: skr +Key: cmn_cv_cmn_000652, Target: cmn, Predicted: nan +Key: cmn_cv_cmn_000717, Target: cmn, Predicted: nan +Key: cmn_cv_cmn_000718, Target: cmn, Predicted: abk +Key: cmn_cv_cmn_000655, Target: cmn, Predicted: nan +Key: cmn_cv_cmn_000724, Target: cmn, Predicted: eng +Key: cmn_cv_cmn_000663, Target: cmn, Predicted: nan +Key: cmn_cv_cmn_000695, Target: cmn, Predicted: nan +Key: cmn_cv_cmn_000664, Target: cmn, Predicted: nan +Key: cmn_cv_cmn_000697, Target: cmn, Predicted: ces +Key: cmn_cv_cmn_000698, Target: cmn, Predicted: por +Key: cmn_cv_cmn_000670, Target: cmn, Predicted: nan +Key: cym_cv_cym_000018, Target: cym, Predicted: aze +Key: cym_cv_cym_000019, Target: cym, Predicted: kat +Key: cym_cv_cym_000052, Target: cym, Predicted: ind +Key: cym_cv_cym_000053, Target: cym, Predicted: kat +Key: cym_cv_cym_000033, Target: cym, Predicted: bre +Key: cym_cv_cym_000072, Target: cym, Predicted: hye +Key: dan_NST_dan_000733, Target: dan, Predicted: epo +Key: cym_cv_cym_000076, Target: cym, Predicted: lat +Key: dan_NST_dan_000737, Target: dan, Predicted: nno +Key: cym_cv_cym_000080, Target: cym, Predicted: gle +Key: dan_NST_dan_000738, Target: dan, Predicted: gle +Key: cym_cv_cym_000081, Target: cym, Predicted: kab +Key: dan_NST_dan_000739, Target: dan, Predicted: swe +Key: dan_NST_dan_000714, Target: dan, Predicted: cym +Key: cym_cv_cym_000091, Target: cym, Predicted: gle +Key: cym_cv_cym_000092, Target: cym, Predicted: gle +Key: dan_NST_dan_000722, Target: dan, Predicted: bre +Key: cym_cv_cym_000097, Target: cym, Predicted: ita +Key: dan_NST_dan_000724, Target: dan, Predicted: xty +Key: dan_NST_dan_000756, Target: dan, Predicted: nno +Key: cym_cv_cym_000099, Target: cym, Predicted: gle +Key: dan_NST_dan_000725, Target: dan, Predicted: mri +Key: cym_cv_cym_000101, Target: cym, Predicted: kat +Key: cym_cv_cym_000102, Target: cym, Predicted: kat +Key: dan_NST_dan_000761, Target: dan, Predicted: slv +Key: dan_cv_dan_001294, Target: dan, Predicted: ces +Key: dan_cv_dan_001295, Target: dan, Predicted: ces +Key: dan_cv_dan_001296, Target: dan, Predicted: ces +Key: dan_cv_dan_001265, Target: dan, Predicted: kat +Key: dan_cv_dan_001297, Target: dan, Predicted: ces +Key: dan_NST_dan_000768, Target: dan, Predicted: sot +Key: dan_cv_dan_001266, Target: dan, Predicted: mhr +Key: dan_cv_dan_001298, Target: dan, Predicted: ces +Key: dan_cv_dan_001299, Target: dan, Predicted: eng +Key: dan_cv_dan_001268, Target: dan, Predicted: kat +Key: dan_cv_dan_001300, Target: dan, Predicted: ces +Key: dan_cv_dan_001269, Target: dan, Predicted: bak +Key: dan_cv_dan_001270, Target: dan, Predicted: kir +Key: dan_cv_dan_001271, Target: dan, Predicted: kat +Key: dan_cv_dan_001272, Target: dan, Predicted: bak +Key: dan_cv_dan_001273, Target: dan, Predicted: abk +Key: dan_cv_dan_001274, Target: dan, Predicted: kat +Key: dan_cv_dan_001275, Target: dan, Predicted: kat +Key: dan_NST_dan_000778, Target: dan, Predicted: nso +Key: dan_cv_dan_001276, Target: dan, Predicted: mrj +Key: dan_cv_dan_001314, Target: dan, Predicted: ces +Key: dan_cv_dan_001315, Target: dan, Predicted: ces +Key: dan_cv_dan_001252, Target: dan, Predicted: kat +Key: dan_cv_dan_001316, Target: dan, Predicted: kat +Key: dan_cv_dan_001253, Target: dan, Predicted: bak +Key: dan_cv_dan_001317, Target: dan, Predicted: ces +Key: dan_cv_dan_001318, Target: dan, Predicted: ces +Key: dan_NST_dan_000789, Target: dan, Predicted: fra +Key: dan_cv_dan_001320, Target: dan, Predicted: nan +Key: dan_cv_dan_001289, Target: dan, Predicted: kir +Key: dan_cv_dan_001321, Target: dan, Predicted: ces +Key: dan_cv_dan_001290, Target: dan, Predicted: por +Key: dan_cv_dan_001322, Target: dan, Predicted: bre +Key: dan_cv_dan_001291, Target: dan, Predicted: ces +Key: dan_cv_dan_001323, Target: dan, Predicted: ces +Key: dan_NST_dan_000794, Target: dan, Predicted: deu +Key: dan_cv_dan_001292, Target: dan, Predicted: ces +Key: dan_cv_dan_001324, Target: dan, Predicted: nan +Key: dan_cv_dan_001293, Target: dan, Predicted: kir +Key: dan_cv_dan_001390, Target: dan, Predicted: ces +Key: dan_cv_dan_001391, Target: dan, Predicted: bre +Key: dan_cv_dan_001392, Target: dan, Predicted: bre +Key: dan_cv_dan_001393, Target: dan, Predicted: bre +Key: dan_cv_dan_001395, Target: dan, Predicted: swe +Key: dan_cv_dan_001364, Target: dan, Predicted: nan +Key: dan_cv_dan_001396, Target: dan, Predicted: ces +Key: dan_cv_dan_001365, Target: dan, Predicted: tat +Key: dan_cv_dan_001397, Target: dan, Predicted: bre +Key: dan_cv_dan_001366, Target: dan, Predicted: ces +Key: dan_cv_dan_001398, Target: dan, Predicted: ces +Key: dan_cv_dan_001367, Target: dan, Predicted: kat +Key: dan_cv_dan_001368, Target: dan, Predicted: ces +Key: dan_cv_dan_001400, Target: dan, Predicted: kat +Key: dan_cv_dan_001369, Target: dan, Predicted: bre +Key: dan_cv_dan_001370, Target: dan, Predicted: ell +Key: dan_cv_dan_001339, Target: dan, Predicted: bre +Key: dan_cv_dan_001371, Target: dan, Predicted: por +Key: dan_cv_dan_001340, Target: dan, Predicted: ces +Key: dan_cv_dan_001372, Target: dan, Predicted: ell +Key: dan_cv_dan_001341, Target: dan, Predicted: ces +Key: dan_cv_dan_001373, Target: dan, Predicted: ces +Key: dan_cv_dan_001342, Target: dan, Predicted: eng +Key: dan_cv_dan_001374, Target: dan, Predicted: nan +Key: dan_cv_dan_001343, Target: dan, Predicted: mhr +Key: dan_cv_dan_001407, Target: dan, Predicted: lav +Key: dan_cv_dan_001344, Target: dan, Predicted: kir +Key: dan_cv_dan_001345, Target: dan, Predicted: kat +Key: dan_cv_dan_001346, Target: dan, Predicted: ces +Key: dan_cv_dan_001347, Target: dan, Predicted: myv +Key: dan_cv_dan_001348, Target: dan, Predicted: por +Key: dan_cv_dan_001349, Target: dan, Predicted: bre +Key: dan_cv_dan_001350, Target: dan, Predicted: ces +Key: dan_cv_dan_001414, Target: dan, Predicted: kat +Key: dan_cv_dan_001415, Target: dan, Predicted: cnh +Key: dan_cv_dan_001416, Target: dan, Predicted: bak +Key: dan_cv_dan_001389, Target: dan, Predicted: ces +Key: deu_cv_deu_000620, Target: deu, Predicted: bre +Key: deu_cv_deu_000621, Target: deu, Predicted: bre +Key: deu_cv_deu_000668, Target: deu, Predicted: por +Key: deu_cv_deu_000669, Target: deu, Predicted: ces +Key: deu_cv_deu_000676, Target: deu, Predicted: dan +Key: deu_cv_deu_000677, Target: deu, Predicted: div +Key: deu_cv_deu_000684, Target: deu, Predicted: eng +Key: deu_cv_deu_000685, Target: deu, Predicted: ces +Key: deu_cv_deu_000666, Target: deu, Predicted: ita +Key: deu_swc_deu_001241, Target: deu, Predicted: pol +Key: deu_swc_deu_001216, Target: deu, Predicted: ltz +Key: deu_swc_deu_001351, Target: deu, Predicted: nso +Key: deu_voxforge_deu_000790, Target: deu, Predicted: slv +Key: deu_swc_deu_001341, Target: deu, Predicted: yid +Key: deu_voxpopuli_deu_000267, Target: deu, Predicted: nld +Key: deu_voxforge_deu_000886, Target: deu, Predicted: nso +Key: div_cv_div_000704, Target: div, Predicted: ara +Key: div_cv_div_000712, Target: div, Predicted: jpn +Key: ell_cv_ell_000867, Target: ell, Predicted: ces +Key: eng_cv_eng_000685, Target: eng, Predicted: hin +Key: eng_cv_eng_000655, Target: eng, Predicted: fra +Key: eng_cv_eng_000689, Target: eng, Predicted: kin +Key: eng_cv_eng_000658, Target: eng, Predicted: cnh +Key: eng_cv_eng_000692, Target: eng, Predicted: bre +Key: eng_cv_eng_000663, Target: eng, Predicted: bre +Key: eng_cv_eng_000666, Target: eng, Predicted: dan +Key: eng_cv_eng_000636, Target: eng, Predicted: lav +Key: eng_cv_eng_000669, Target: eng, Predicted: deu +Key: eng_cv_eng_000615, Target: eng, Predicted: mal +Key: eng_cv_eng_000679, Target: eng, Predicted: skr +Key: eng_cv_eng_000680, Target: eng, Predicted: fra +Key: eng_cv_eng_000651, Target: eng, Predicted: bre +Key: eng_nchlt_eng_001402, Target: eng, Predicted: ces +Key: eng_nchlt_eng_001435, Target: eng, Predicted: afr +Key: eng_nchlt_eng_001468, Target: eng, Predicted: afr +Key: eng_nchlt_eng_001469, Target: eng, Predicted: por +Key: eng_nchlt_eng_001480, Target: eng, Predicted: dan +Key: eng_nchlt_eng_001420, Target: eng, Predicted: afr +Key: eng_nchlt_eng_001486, Target: eng, Predicted: cym +Key: eng_nchlt_eng_001535, Target: eng, Predicted: ces +Key: eng_nchlt_eng_001546, Target: eng, Predicted: mri +Key: eng_swc_eng_001633, Target: eng, Predicted: mlt +Key: eng_voxforge_eng_000784, Target: eng, Predicted: cym +Key: epo_cv_epo_000587, Target: epo, Predicted: skr +Key: epo_cv_epo_000591, Target: epo, Predicted: ina +Key: epo_cv_epo_000593, Target: epo, Predicted: ina +Key: epo_cv_epo_000596, Target: epo, Predicted: ina +Key: epo_cv_epo_000597, Target: epo, Predicted: ron +Key: epo_cv_epo_000604, Target: epo, Predicted: por +Key: epo_cv_epo_000573, Target: epo, Predicted: ita +Key: epo_cv_epo_000605, Target: epo, Predicted: ina +Key: epo_cv_epo_000609, Target: epo, Predicted: mhr +Key: epo_cv_epo_000578, Target: epo, Predicted: bre +Key: epo_cv_epo_000611, Target: epo, Predicted: spa +Key: epo_cv_epo_000646, Target: epo, Predicted: ina +Key: epo_cv_epo_000679, Target: epo, Predicted: tat +Key: epo_cv_epo_000649, Target: epo, Predicted: por +Key: epo_cv_epo_000619, Target: epo, Predicted: fas +Key: epo_cv_epo_000631, Target: epo, Predicted: tok +Key: epo_cv_epo_000633, Target: epo, Predicted: por +Key: epo_cv_epo_000634, Target: epo, Predicted: ces +Key: epo_cv_epo_000635, Target: epo, Predicted: ces +Key: epo_cv_epo_000636, Target: epo, Predicted: ces +Key: epo_cv_epo_000637, Target: epo, Predicted: ces +Key: epo_cv_epo_000674, Target: epo, Predicted: ina +Key: epo_cv_epo_000644, Target: epo, Predicted: lat +Key: eus_cv_eus_000016, Target: eus, Predicted: bre +Key: eus_cv_eus_000057, Target: eus, Predicted: abk +Key: eus_cv_eus_000026, Target: eus, Predicted: cym +Key: eus_cv_eus_000058, Target: eus, Predicted: ell +Key: eus_cv_eus_000038, Target: eus, Predicted: kat +Key: fas_cv_fas_000015, Target: fas, Predicted: sna +Key: fas_cv_fas_000048, Target: fas, Predicted: kab +Key: fas_cv_fas_000049, Target: fas, Predicted: cym +Key: fas_cv_fas_000051, Target: fas, Predicted: kab +Key: fas_cv_fas_000055, Target: fas, Predicted: ces +Key: fas_cv_fas_000058, Target: fas, Predicted: yid +Key: fas_cv_fas_000030, Target: fas, Predicted: ckb +Key: fas_cv_fas_000031, Target: fas, Predicted: por +Key: fas_cv_fas_000063, Target: fas, Predicted: xty +Key: fas_cv_fas_000032, Target: fas, Predicted: mon +Key: fas_cv_fas_000004, Target: fas, Predicted: bre +Key: fas_cv_fas_000068, Target: fas, Predicted: chv +Key: fas_cv_fas_000005, Target: fas, Predicted: uzb +Key: fas_cv_fas_000006, Target: fas, Predicted: div +Key: fas_cv_fas_000009, Target: fas, Predicted: tat +Key: fas_cv_fas_000073, Target: fas, Predicted: urd +Key: fas_cv_fas_000010, Target: fas, Predicted: deu +Key: fas_cv_fas_000077, Target: fas, Predicted: ces +Key: fas_cv_fas_000088, Target: fas, Predicted: mlt +Key: fas_cv_fas_000095, Target: fas, Predicted: por +Key: fas_cv_fas_000099, Target: fas, Predicted: cym +Key: fas_cv_fas_000100, Target: fas, Predicted: yid +Key: fas_cv_fas_000106, Target: fas, Predicted: ara +Key: fin_cv_fin_000749, Target: fin, Predicted: est +Key: fin_voxpopuli_fin_000981, Target: fin, Predicted: pol +Key: fra_cv_fra_000669, Target: fra, Predicted: eus +Key: fra_cv_fra_000734, Target: fra, Predicted: bre +Key: fra_cv_fra_000705, Target: fra, Predicted: bre +Key: fra_cv_fra_000706, Target: fra, Predicted: bre +Key: fra_cv_fra_000645, Target: fra, Predicted: bre +Key: fra_cv_fra_000646, Target: fra, Predicted: bre +Key: fra_cv_fra_000711, Target: fra, Predicted: oci +Key: fra_cv_fra_000745, Target: fra, Predicted: bre +Key: fra_cv_fra_000714, Target: fra, Predicted: bre +Key: fra_cv_fra_000746, Target: fra, Predicted: bre +Key: fra_cv_fra_000651, Target: fra, Predicted: oci +Key: fra_cv_fra_000653, Target: fra, Predicted: ces +Key: fra_cv_fra_000717, Target: fra, Predicted: tat +Key: fra_cv_fra_000654, Target: fra, Predicted: ces +Key: fra_cv_fra_000688, Target: fra, Predicted: bre +Key: fra_cv_fra_000657, Target: fra, Predicted: bre +Key: fra_cv_fra_000659, Target: fra, Predicted: bre +Key: fra_cv_fra_000660, Target: fra, Predicted: bre +Key: fra_cv_fra_000661, Target: fra, Predicted: bre +Key: fra_cv_fra_000662, Target: fra, Predicted: bre +Key: fra_cv_fra_000697, Target: fra, Predicted: kir +Key: fra_cv_fra_000698, Target: fra, Predicted: bre +Key: fra_cv_fra_000668, Target: fra, Predicted: ina +Key: frr_cv_frr_000664, Target: frr, Predicted: nld +Key: frr_cv_frr_000665, Target: frr, Predicted: nld +Key: fra_voxforge_fra_000709, Target: fra, Predicted: bre +Key: frr_cv_frr_000669, Target: frr, Predicted: nld +Key: frr_cv_frr_000670, Target: frr, Predicted: nld +Key: frr_cv_frr_000671, Target: frr, Predicted: nld +Key: frr_cv_frr_000672, Target: frr, Predicted: nld +Key: frr_cv_frr_000705, Target: frr, Predicted: nld +Key: frr_cv_frr_000737, Target: frr, Predicted: nld +Key: frr_cv_frr_000674, Target: frr, Predicted: nld +Key: frr_cv_frr_000706, Target: frr, Predicted: nld +Key: frr_cv_frr_000738, Target: frr, Predicted: nld +Key: frr_cv_frr_000770, Target: frr, Predicted: nld +Key: frr_cv_frr_000739, Target: frr, Predicted: nld +Key: frr_cv_frr_000676, Target: frr, Predicted: nld +Key: frr_cv_frr_000740, Target: frr, Predicted: kat +Key: frr_cv_frr_000677, Target: frr, Predicted: nld +Key: frr_cv_frr_000678, Target: frr, Predicted: nld +Key: ful_fleurs_ful_000004, Target: ful, Predicted: swa +Key: frr_cv_frr_000679, Target: frr, Predicted: nld +Key: frr_cv_frr_000743, Target: frr, Predicted: nld +Key: frr_cv_frr_000680, Target: frr, Predicted: nld +Key: frr_cv_frr_000744, Target: frr, Predicted: nld +Key: ful_fleurs_ful_000006, Target: ful, Predicted: swa +Key: frr_cv_frr_000682, Target: frr, Predicted: nld +Key: frr_cv_frr_000683, Target: frr, Predicted: nld +Key: frr_cv_frr_000747, Target: frr, Predicted: nld +Key: frr_cv_frr_000684, Target: frr, Predicted: nld +Key: ful_fleurs_ful_000011, Target: ful, Predicted: swa +Key: frr_cv_frr_000685, Target: frr, Predicted: nld +Key: frr_cv_frr_000749, Target: frr, Predicted: eng +Key: frr_cv_frr_000719, Target: frr, Predicted: nld +Key: ful_fleurs_ful_000015, Target: ful, Predicted: luo +Key: ful_fleurs_ful_000016, Target: ful, Predicted: luo +Key: frr_cv_frr_000754, Target: frr, Predicted: nld +Key: ful_fleurs_ful_000017, Target: ful, Predicted: hau +Key: frr_cv_frr_000691, Target: frr, Predicted: nld +Key: frr_cv_frr_000755, Target: frr, Predicted: nld +Key: frr_cv_frr_000692, Target: frr, Predicted: nld +Key: frr_cv_frr_000724, Target: frr, Predicted: nld +Key: frr_cv_frr_000756, Target: frr, Predicted: nld +Key: ful_fleurs_ful_000019, Target: ful, Predicted: swa +Key: frr_cv_frr_000693, Target: frr, Predicted: nld +Key: frr_cv_frr_000694, Target: frr, Predicted: nld +Key: frr_cv_frr_000726, Target: frr, Predicted: nld +Key: frr_cv_frr_000695, Target: frr, Predicted: nld +Key: frr_cv_frr_000759, Target: frr, Predicted: nld +Key: frr_cv_frr_000729, Target: frr, Predicted: nld +Key: frr_cv_frr_000698, Target: frr, Predicted: nld +Key: frr_cv_frr_000730, Target: frr, Predicted: nld +Key: frr_cv_frr_000699, Target: frr, Predicted: nld +Key: frr_cv_frr_000700, Target: frr, Predicted: nld +Key: frr_cv_frr_000764, Target: frr, Predicted: nld +Key: frr_cv_frr_000701, Target: frr, Predicted: nld +Key: frr_cv_frr_000733, Target: frr, Predicted: nld +Key: frr_cv_frr_000702, Target: frr, Predicted: nld +Key: frr_cv_frr_000766, Target: frr, Predicted: nld +Key: frr_cv_frr_000703, Target: frr, Predicted: nld +Key: frr_cv_frr_000704, Target: frr, Predicted: nld +Key: frr_cv_frr_000768, Target: frr, Predicted: deu +Key: ful_fleurs_ful_000033, Target: ful, Predicted: pus +Key: ful_fleurs_ful_000037, Target: ful, Predicted: wol +Key: ful_fleurs_ful_000038, Target: ful, Predicted: wol +Key: ful_fleurs_ful_000041, Target: ful, Predicted: swa +Key: ful_fleurs_ful_000042, Target: ful, Predicted: hau +Key: ful_fleurs_ful_000043, Target: ful, Predicted: wol +Key: ful_fleurs_ful_000044, Target: ful, Predicted: swa +Key: gle_LAD_gle_000194, Target: gle, Predicted: eng +Key: gle_cv_gle_000876, Target: gle, Predicted: bre +Key: gle_cv_gle_000877, Target: gle, Predicted: ces +Key: gle_cv_gle_000923, Target: gle, Predicted: eng +Key: gle_cv_gle_000993, Target: gle, Predicted: por +Key: gle_cv_gle_000994, Target: gle, Predicted: yue +Key: glg_fleurs_glg_000011, Target: glg, Predicted: ast +Key: glg_cv_glg_000730, Target: glg, Predicted: por +Key: glg_fleurs_glg_000015, Target: glg, Predicted: ast +Key: glg_fleurs_glg_000025, Target: glg, Predicted: ast +Key: glg_fleurs_glg_000039, Target: glg, Predicted: ast +Key: grn_cv_grn_000784, Target: grn, Predicted: bas +Key: grn_cv_grn_000795, Target: grn, Predicted: spa +Key: guj_googlei18n-tts_guj_000593, Target: guj, Predicted: mar +Key: guj_googlei18n-tts_guj_000609, Target: guj, Predicted: hin +Key: heb_fleurs_heb_000062, Target: heb, Predicted: ara +Key: heb_fleurs_heb_000003, Target: heb, Predicted: ara +Key: heb_fleurs_heb_000036, Target: heb, Predicted: ara +Key: heb_fleurs_heb_000006, Target: heb, Predicted: ara +Key: hin_cv_hin_000000, Target: hin, Predicted: urd +Key: heb_fleurs_heb_000008, Target: heb, Predicted: ara +Key: hin_cv_hin_000002, Target: hin, Predicted: urd +Key: hin_cv_hin_000005, Target: hin, Predicted: urd +Key: heb_fleurs_heb_000048, Target: heb, Predicted: ara +Key: hin_cv_hin_000010, Target: hin, Predicted: urd +Key: heb_fleurs_heb_000018, Target: heb, Predicted: ara +Key: hin_cv_hin_000013, Target: hin, Predicted: skr +Key: hin_cv_hin_000047, Target: hin, Predicted: urd +Key: hin_cv_hin_000048, Target: hin, Predicted: urd +Key: hin_cv_hin_000080, Target: hin, Predicted: urd +Key: hin_fleurs_hin_000003, Target: hin, Predicted: urd +Key: hin_cv_hin_000017, Target: hin, Predicted: urd +Key: hin_cv_hin_000049, Target: hin, Predicted: urd +Key: hin_cv_hin_000081, Target: hin, Predicted: urd +Key: hin_cv_hin_000018, Target: hin, Predicted: skr +Key: hin_cv_hin_000050, Target: hin, Predicted: skr +Key: hin_cv_hin_000082, Target: hin, Predicted: urd +Key: hin_fleurs_hin_000005, Target: hin, Predicted: urd +Key: hin_cv_hin_000019, Target: hin, Predicted: urd +Key: hin_cv_hin_000083, Target: hin, Predicted: urd +Key: hin_cv_hin_000020, Target: hin, Predicted: urd +Key: hin_cv_hin_000021, Target: hin, Predicted: skr +Key: hin_cv_hin_000085, Target: hin, Predicted: ces +Key: hin_cv_hin_000022, Target: hin, Predicted: urd +Key: hin_cv_hin_000086, Target: hin, Predicted: skr +Key: hin_cv_hin_000023, Target: hin, Predicted: urd +Key: hin_cv_hin_000024, Target: hin, Predicted: urd +Key: hin_cv_hin_000056, Target: hin, Predicted: urd +Key: hin_fleurs_hin_000011, Target: hin, Predicted: urd +Key: hin_cv_hin_000026, Target: hin, Predicted: urd +Key: hin_fleurs_hin_000015, Target: hin, Predicted: urd +Key: hin_cv_hin_000029, Target: hin, Predicted: urd +Key: hin_cv_hin_000093, Target: hin, Predicted: urd +Key: hin_cv_hin_000094, Target: hin, Predicted: urd +Key: hin_fleurs_hin_000017, Target: hin, Predicted: khm +Key: hin_cv_hin_000063, Target: hin, Predicted: mal +Key: hin_cv_hin_000095, Target: hin, Predicted: urd +Key: hin_cv_hin_000033, Target: hin, Predicted: skr +Key: hin_cv_hin_000065, Target: hin, Predicted: mal +Key: hin_cv_hin_000034, Target: hin, Predicted: skr +Key: hin_cv_hin_000067, Target: hin, Predicted: urd +Key: hin_cv_hin_000068, Target: hin, Predicted: urd +Key: hin_cv_hin_000101, Target: hin, Predicted: urd +Key: hin_cv_hin_000038, Target: hin, Predicted: urd +Key: hin_cv_hin_000102, Target: hin, Predicted: urd +Key: hin_cv_hin_000039, Target: hin, Predicted: urd +Key: hin_cv_hin_000103, Target: hin, Predicted: urd +Key: hin_cv_hin_000040, Target: hin, Predicted: urd +Key: hin_cv_hin_000104, Target: hin, Predicted: urd +Key: hin_fleurs_hin_000028, Target: hin, Predicted: urd +Key: hin_cv_hin_000041, Target: hin, Predicted: urd +Key: hin_cv_hin_000105, Target: hin, Predicted: urd +Key: hin_cv_hin_000106, Target: hin, Predicted: urd +Key: hin_cv_hin_000076, Target: hin, Predicted: div +Key: hin_cv_hin_000108, Target: hin, Predicted: urd +Key: hin_cv_hin_000046, Target: hin, Predicted: urd +Key: hin_fleurs_hin_000034, Target: hin, Predicted: guj +Key: hin_fleurs_hin_000035, Target: hin, Predicted: urd +Key: hin_fleurs_hin_000036, Target: hin, Predicted: guj +Key: hrv_voxpopuli_hrv_001297, Target: hrv, Predicted: ces +Key: hin_fleurs_hin_000041, Target: hin, Predicted: urd +Key: hrv_fleurs_hrv_000016, Target: hrv, Predicted: mkd +Key: hrv_fleurs_hrv_000049, Target: hrv, Predicted: mkd +Key: hrv_fleurs_hrv_000050, Target: hrv, Predicted: mkd +Key: hrv_fleurs_hrv_000052, Target: hrv, Predicted: mkd +Key: hrv_fleurs_hrv_000055, Target: hrv, Predicted: srp +Key: hrv_fleurs_hrv_000024, Target: hrv, Predicted: srp +Key: hrv_fleurs_hrv_000056, Target: hrv, Predicted: mkd +Key: hrv_fleurs_hrv_000058, Target: hrv, Predicted: slv +Key: hrv_fleurs_hrv_000060, Target: hrv, Predicted: slv +Key: hin_fleurs_hin_000056, Target: hin, Predicted: guj +Key: hrv_fleurs_hrv_000034, Target: hrv, Predicted: mkd +Key: hun_cv_hun_000688, Target: hun, Predicted: fin +Key: hye_cv_hye_000621, Target: hye, Predicted: slk +Key: hun_voxpopuli_hun_001476, Target: hun, Predicted: pol +Key: ina_cv_ina_000758, Target: ina, Predicted: kea +Key: ina_cv_ina_000765, Target: ina, Predicted: por +Key: ina_cv_ina_000768, Target: ina, Predicted: por +Key: ibo_fleurs_ibo_000013, Target: ibo, Predicted: aze +Key: ina_cv_ina_000774, Target: ina, Predicted: kea +Key: ina_cv_ina_000775, Target: ina, Predicted: por +Key: ibo_fleurs_ibo_000019, Target: ibo, Predicted: sot +Key: ina_cv_ina_000780, Target: ina, Predicted: kea +Key: ina_cv_ina_000813, Target: ina, Predicted: por +Key: ina_cv_ina_000782, Target: ina, Predicted: por +Key: ind_cv_ind_000800, Target: ind, Predicted: bre +Key: ina_cv_ina_000816, Target: ina, Predicted: por +Key: ind_cv_ind_000801, Target: ind, Predicted: ell +Key: ina_cv_ina_000785, Target: ina, Predicted: kea +Key: ind_cv_ind_000802, Target: ind, Predicted: cnh +Key: ina_cv_ina_000818, Target: ina, Predicted: ita +Key: ind_cv_ind_000803, Target: ind, Predicted: cnh +Key: ina_cv_ina_000820, Target: ina, Predicted: por +Key: ind_cv_ind_000806, Target: ind, Predicted: bre +Key: ina_cv_ina_000823, Target: ina, Predicted: por +Key: ina_cv_ina_000855, Target: ina, Predicted: ita +Key: ind_cv_ind_000808, Target: ind, Predicted: kir +Key: ina_cv_ina_000824, Target: ina, Predicted: kea +Key: ina_cv_ina_000795, Target: ina, Predicted: por +Key: ind_cv_ind_000812, Target: ind, Predicted: cnh +Key: ina_cv_ina_000828, Target: ina, Predicted: por +Key: ina_cv_ina_000797, Target: ina, Predicted: kat +Key: ina_cv_ina_000829, Target: ina, Predicted: por +Key: ina_cv_ina_000802, Target: ina, Predicted: slv +Key: ina_cv_ina_000806, Target: ina, Predicted: por +Key: ina_cv_ina_000807, Target: ina, Predicted: kea +Key: ina_cv_ina_000808, Target: ina, Predicted: por +Key: ind_cv_ind_000828, Target: ind, Predicted: ces +Key: ind_cv_ind_000894, Target: ind, Predicted: eng +Key: ind_cv_ind_000895, Target: ind, Predicted: ckb +Key: ind_cv_ind_000833, Target: ind, Predicted: ina +Key: ind_cv_ind_000929, Target: ind, Predicted: ina +Key: ind_cv_ind_000834, Target: ind, Predicted: tok +Key: ind_cv_ind_000835, Target: ind, Predicted: cym +Key: ind_cv_ind_000837, Target: ind, Predicted: eng +Key: ind_cv_ind_000838, Target: ind, Predicted: hye +Key: ind_cv_ind_000839, Target: ind, Predicted: gle +Key: ind_cv_ind_000840, Target: ind, Predicted: tok +Key: ind_cv_ind_000841, Target: ind, Predicted: gle +Key: ind_cv_ind_000842, Target: ind, Predicted: ina +Key: ind_cv_ind_000843, Target: ind, Predicted: eng +Key: ind_cv_ind_000844, Target: ind, Predicted: tok +Key: ind_cv_ind_000878, Target: ind, Predicted: nan +Key: ind_cv_ind_000847, Target: ind, Predicted: tok +Key: ind_cv_ind_000881, Target: ind, Predicted: cnh +Key: ind_cv_ind_000946, Target: ind, Predicted: ces +Key: ind_cv_ind_000883, Target: ind, Predicted: ces +Key: ind_cv_ind_000884, Target: ind, Predicted: cnh +Key: ind_cv_ind_000885, Target: ind, Predicted: oci +Key: ind_cv_ind_000886, Target: ind, Predicted: div +Key: ind_cv_ind_000887, Target: ind, Predicted: mrj +Key: ind_cv_ind_000919, Target: ind, Predicted: ell +Key: ind_cv_ind_000888, Target: ind, Predicted: cat +Key: ind_cv_ind_000889, Target: ind, Predicted: kea +Key: ind_cv_ind_000890, Target: ind, Predicted: oci +Key: ind_cv_ind_000892, Target: ind, Predicted: kan +Key: ind_fleurs_ind_000002, Target: ind, Predicted: jav +Key: ita_cv_ita_000604, Target: ita, Predicted: spa +Key: ita_cv_ita_000581, Target: ita, Predicted: bre +Key: ita_cv_ita_000587, Target: ita, Predicted: pol +Key: ita_cv_ita_000624, Target: ita, Predicted: glg +Key: ita_cv_ita_000625, Target: ita, Predicted: bre +Key: ita_voxpopuli_ita_001633, Target: ita, Predicted: deu +Key: jav_fleurs_jav_000000, Target: jav, Predicted: ind +Key: jav_fleurs_jav_000004, Target: jav, Predicted: ind +Key: jav_googlei18n-asr_jav_000626, Target: jav, Predicted: ind +Key: jav_googlei18n-asr_jav_000658, Target: jav, Predicted: ind +Key: jav_googlei18n-asr_jav_000659, Target: jav, Predicted: tha +Key: jav_googlei18n-asr_jav_000692, Target: jav, Predicted: mya +Key: jav_googlei18n-asr_jav_000694, Target: jav, Predicted: eng +Key: jav_fleurs_jav_000029, Target: jav, Predicted: ind +Key: jav_googlei18n-asr_jav_000668, Target: jav, Predicted: sun +Key: jav_googlei18n-asr_jav_000669, Target: jav, Predicted: sin +Key: jav_googlei18n-asr_jav_000610, Target: jav, Predicted: eng +Key: jav_googlei18n-asr_jav_000644, Target: jav, Predicted: ind +Key: jav_googlei18n-asr_jav_000709, Target: jav, Predicted: cnh +Key: jav_googlei18n-asr_jav_000646, Target: jav, Predicted: azz +Key: jav_googlei18n-asr_jav_000647, Target: jav, Predicted: ind +Key: jav_googlei18n-asr_jav_000618, Target: jav, Predicted: ind +Key: jav_googlei18n-asr_jav_000619, Target: jav, Predicted: ind +Key: jav_googlei18n-asr_jav_000621, Target: jav, Predicted: ind +Key: jav_googlei18n-asr_jav_000623, Target: jav, Predicted: msa +Key: jav_googlei18n-asr_jav_000657, Target: jav, Predicted: ind +Key: jpn_cv_jpn_000043, Target: jpn, Predicted: est +Key: jpn_cv_jpn_000049, Target: jpn, Predicted: gle +Key: jpn_cv_jpn_000019, Target: jpn, Predicted: hau +Key: jpn_cv_jpn_000022, Target: jpn, Predicted: kat +Key: jpn_cv_jpn_000736, Target: jpn, Predicted: grn +Key: jpn_cv_jpn_000768, Target: jpn, Predicted: cnh +Key: jpn_cv_jpn_000737, Target: jpn, Predicted: kat +Key: jpn_cv_jpn_000769, Target: jpn, Predicted: kat +Key: jpn_cv_jpn_000770, Target: jpn, Predicted: cnh +Key: jpn_cv_jpn_000771, Target: jpn, Predicted: cnh +Key: jpn_cv_jpn_000772, Target: jpn, Predicted: cnh +Key: jpn_cv_jpn_000753, Target: jpn, Predicted: ces +Key: jpn_cv_jpn_000754, Target: jpn, Predicted: ces +Key: jpn_cv_jpn_000755, Target: jpn, Predicted: ces +Key: jpn_cv_jpn_000756, Target: jpn, Predicted: kat +Key: jpn_cv_jpn_000757, Target: jpn, Predicted: ces +Key: jpn_cv_jpn_000733, Target: jpn, Predicted: bak +Key: jpn_cv_jpn_000734, Target: jpn, Predicted: abk +Key: jpn_cv_jpn_000735, Target: jpn, Predicted: mrj +Key: kab_cv_kab_000906, Target: kab, Predicted: kat +Key: kab_cv_kab_000908, Target: kab, Predicted: ces +Key: kab_cv_kab_000909, Target: kab, Predicted: ces +Key: kab_cv_kab_000911, Target: kab, Predicted: mrj +Key: kab_cv_kab_000887, Target: kab, Predicted: ces +Key: kab_cv_kab_000953, Target: kab, Predicted: bre +Key: kab_cv_kab_000955, Target: kab, Predicted: bre +Key: kab_cv_kab_000989, Target: kab, Predicted: bre +Key: kab_cv_kab_000898, Target: kab, Predicted: ell +Key: kab_cv_kab_000900, Target: kab, Predicted: abk +Key: kab_cv_kab_000932, Target: kab, Predicted: ces +Key: kab_cv_kab_000901, Target: kab, Predicted: abk +Key: kab_cv_kab_000933, Target: kab, Predicted: abk +Key: kab_cv_kab_000902, Target: kab, Predicted: ces +Key: kab_cv_kab_000904, Target: kab, Predicted: ces +Key: kab_cv_kab_000936, Target: kab, Predicted: ces +Key: kab_cv_kab_001000, Target: kab, Predicted: jpn +Key: kab_cv_kab_000905, Target: kab, Predicted: ces +Key: kab_cv_kab_001007, Target: kab, Predicted: mrj +Key: kab_cv_kab_001008, Target: kab, Predicted: abk +Key: kab_cv_kab_001009, Target: kab, Predicted: tat +Key: kab_cv_kab_001010, Target: kab, Predicted: mhr +Key: kab_cv_kab_001015, Target: kab, Predicted: kmr +Key: kam_fleurs_kam_000023, Target: kam, Predicted: swa +Key: kan_googlei18n-tts_kan_000525, Target: kan, Predicted: tam +Key: kat_cv_kat_000000, Target: kat, Predicted: fra +Key: kat_cv_kat_000032, Target: kat, Predicted: abk +Key: kan_googlei18n-tts_kan_000572, Target: kan, Predicted: tam +Key: kat_cv_kat_000002, Target: kat, Predicted: bak +Key: kat_cv_kat_000035, Target: kat, Predicted: ckb +Key: kat_cv_kat_000006, Target: kat, Predicted: ces +Key: kat_cv_kat_000038, Target: kat, Predicted: bre +Key: kat_cv_kat_000008, Target: kat, Predicted: deu +Key: kat_cv_kat_000040, Target: kat, Predicted: kab +Key: kat_cv_kat_000010, Target: kat, Predicted: rus +Key: kan_googlei18n-tts_kan_000550, Target: kan, Predicted: tam +Key: kat_cv_kat_000013, Target: kat, Predicted: kir +Key: kat_cv_kat_000014, Target: kat, Predicted: abk +Key: kat_cv_kat_000015, Target: kat, Predicted: mhr +Key: kat_cv_kat_000047, Target: kat, Predicted: abk +Key: kan_googlei18n-tts_kan_000587, Target: kan, Predicted: mar +Key: kat_cv_kat_000048, Target: kat, Predicted: abk +Key: kat_cv_kat_000049, Target: kat, Predicted: ces +Key: kat_cv_kat_000051, Target: kat, Predicted: slk +Key: kan_googlei18n-tts_kan_000561, Target: kan, Predicted: tam +Key: kat_cv_kat_000054, Target: kat, Predicted: ces +Key: kat_cv_kat_000023, Target: kat, Predicted: ces +Key: kat_cv_kat_000055, Target: kat, Predicted: ces +Key: kan_googlei18n-tts_kan_000563, Target: kan, Predicted: tam +Key: kat_cv_kat_000024, Target: kat, Predicted: slk +Key: kat_cv_kat_000056, Target: kat, Predicted: ces +Key: kat_cv_kat_000026, Target: kat, Predicted: kmr +Key: kat_cv_kat_000028, Target: kat, Predicted: slk +Key: kat_cv_kat_000060, Target: kat, Predicted: ces +Key: kat_cv_kat_000029, Target: kat, Predicted: kab +Key: kat_cv_kat_000061, Target: kat, Predicted: tat +Key: kat_cv_kat_000031, Target: kat, Predicted: kir +Key: kat_cv_kat_000128, Target: kat, Predicted: por +Key: kat_cv_kat_000160, Target: kat, Predicted: bak +Key: kat_cv_kat_000098, Target: kat, Predicted: abk +Key: kat_cv_kat_000130, Target: kat, Predicted: mlt +Key: kat_cv_kat_000131, Target: kat, Predicted: ces +Key: kat_cv_kat_000100, Target: kat, Predicted: ori +Key: kat_cv_kat_000132, Target: kat, Predicted: cmn +Key: kat_cv_kat_000133, Target: kat, Predicted: mlt +Key: kat_cv_kat_000102, Target: kat, Predicted: ell +Key: kat_cv_kat_000166, Target: kat, Predicted: kab +Key: kat_cv_kat_000103, Target: kat, Predicted: bre +Key: kat_cv_kat_000167, Target: kat, Predicted: kab +Key: kat_cv_kat_000072, Target: kat, Predicted: abk +Key: kat_cv_kat_000168, Target: kat, Predicted: ces +Key: kat_cv_kat_000073, Target: kat, Predicted: ckb +Key: kat_cv_kat_000105, Target: kat, Predicted: ckb +Key: kat_cv_kat_000074, Target: kat, Predicted: lav +Key: kat_cv_kat_000075, Target: kat, Predicted: kir +Key: kat_cv_kat_000076, Target: kat, Predicted: ell +Key: kat_cv_kat_000108, Target: kat, Predicted: hye +Key: kat_cv_kat_000111, Target: kat, Predicted: bre +Key: kat_cv_kat_000113, Target: kat, Predicted: nan +Key: kat_cv_kat_000082, Target: kat, Predicted: slk +Key: kat_cv_kat_000114, Target: kat, Predicted: abk +Key: kat_cv_kat_000115, Target: kat, Predicted: abk +Key: kat_cv_kat_000117, Target: kat, Predicted: abk +Key: kat_cv_kat_000118, Target: kat, Predicted: slk +Key: kat_cv_kat_000119, Target: kat, Predicted: lug +Key: kat_cv_kat_000120, Target: kat, Predicted: kir +Key: kat_cv_kat_000090, Target: kat, Predicted: abk +Key: kat_cv_kat_000154, Target: kat, Predicted: bre +Key: kat_cv_kat_000091, Target: kat, Predicted: abk +Key: kat_cv_kat_000155, Target: kat, Predicted: bre +Key: kat_cv_kat_000156, Target: kat, Predicted: ces +Key: kat_cv_kat_000093, Target: kat, Predicted: abk +Key: kat_cv_kat_000125, Target: kat, Predicted: bre +Key: kat_cv_kat_000157, Target: kat, Predicted: ces +Key: kat_cv_kat_000094, Target: kat, Predicted: ell +Key: kat_cv_kat_000158, Target: kat, Predicted: bre +Key: kat_cv_kat_000095, Target: kat, Predicted: abk +Key: kat_cv_kat_000127, Target: kat, Predicted: kir +Key: kaz_cv_kaz_000749, Target: kaz, Predicted: kir +Key: kea_fleurs_kea_000029, Target: kea, Predicted: por +Key: kea_fleurs_kea_000004, Target: kea, Predicted: por +Key: khm_fleurs_khm_000031, Target: khm, Predicted: lao +Key: kea_fleurs_kea_000008, Target: kea, Predicted: por +Key: kea_fleurs_kea_000043, Target: kea, Predicted: por +Key: kea_fleurs_kea_000020, Target: kea, Predicted: por +Key: kea_fleurs_kea_000021, Target: kea, Predicted: por +Key: kin_cv_kin_000764, Target: kin, Predicted: mlt +Key: kin_cv_kin_000703, Target: kin, Predicted: rus +Key: kir_cv_kir_000850, Target: kir, Predicted: kat +Key: kin_cv_kin_000704, Target: kin, Predicted: por +Key: kir_cv_kir_000851, Target: kir, Predicted: tat +Key: kir_cv_kir_000852, Target: kir, Predicted: kat +Key: kir_cv_kir_000853, Target: kir, Predicted: kat +Key: kir_cv_kir_000854, Target: kir, Predicted: kat +Key: kin_cv_kin_000708, Target: kin, Predicted: swa +Key: kir_cv_kir_000855, Target: kir, Predicted: mhr +Key: kin_cv_kin_000709, Target: kin, Predicted: eng +Key: kin_cv_kin_000710, Target: kin, Predicted: por +Key: kir_cv_kir_000857, Target: kir, Predicted: kat +Key: kin_cv_kin_000711, Target: kin, Predicted: swa +Key: kir_cv_kir_000858, Target: kir, Predicted: kat +Key: kin_cv_kin_000712, Target: kin, Predicted: eng +Key: kir_cv_kir_000859, Target: kir, Predicted: kmr +Key: kin_cv_kin_000713, Target: kin, Predicted: eng +Key: kir_cv_kir_000860, Target: kir, Predicted: abk +Key: kin_cv_kin_000682, Target: kin, Predicted: eng +Key: kin_cv_kin_000714, Target: kin, Predicted: bre +Key: kin_cv_kin_000715, Target: kin, Predicted: kat +Key: kir_cv_kir_000911, Target: kir, Predicted: kat +Key: kir_cv_kir_000912, Target: kir, Predicted: ces +Key: kir_cv_kir_000913, Target: kir, Predicted: kat +Key: kir_cv_kir_000915, Target: kir, Predicted: kat +Key: kir_cv_kir_000948, Target: kir, Predicted: por +Key: kir_cv_kir_000949, Target: kir, Predicted: ces +Key: kir_cv_kir_000950, Target: kir, Predicted: por +Key: kir_cv_kir_000951, Target: kir, Predicted: por +Key: kir_cv_kir_000954, Target: kir, Predicted: skr +Key: kir_cv_kir_000955, Target: kir, Predicted: bak +Key: kir_cv_kir_000956, Target: kir, Predicted: kat +Key: kir_cv_kir_000894, Target: kir, Predicted: tur +Key: kir_cv_kir_000926, Target: kir, Predicted: rus +Key: kir_cv_kir_000958, Target: kir, Predicted: kab +Key: kir_cv_kir_000908, Target: kir, Predicted: ces +Key: kir_cv_kir_000909, Target: kir, Predicted: por +Key: kir_cv_kir_000941, Target: kir, Predicted: tat +Key: kir_cv_kir_000910, Target: kir, Predicted: por +Key: kmr_cv_kmr_000798, Target: kmr, Predicted: ron +Key: kmr_cv_kmr_000807, Target: kmr, Predicted: kir +Key: kmr_cv_kmr_000777, Target: kmr, Predicted: bul +Key: kmr_cv_kmr_000778, Target: kmr, Predicted: slk +Key: kmr_cv_kmr_000780, Target: kmr, Predicted: kab +Key: kmr_cv_kmr_000812, Target: kmr, Predicted: ckb +Key: kmr_cv_kmr_000781, Target: kmr, Predicted: mrj +Key: kmr_cv_kmr_000813, Target: kmr, Predicted: ckb +Key: kmr_cv_kmr_000782, Target: kmr, Predicted: ckb +Key: kmr_cv_kmr_000783, Target: kmr, Predicted: ckb +Key: kmr_cv_kmr_000815, Target: kmr, Predicted: bak +Key: kmr_cv_kmr_000752, Target: kmr, Predicted: ckb +Key: kmr_cv_kmr_000784, Target: kmr, Predicted: ckb +Key: kmr_cv_kmr_000816, Target: kmr, Predicted: ckb +Key: kmr_cv_kmr_000786, Target: kmr, Predicted: skr +Key: kmr_cv_kmr_000787, Target: kmr, Predicted: ckb +Key: kmr_cv_kmr_000788, Target: kmr, Predicted: ckb +Key: kmr_cv_kmr_000820, Target: kmr, Predicted: abk +Key: kmr_cv_kmr_000789, Target: kmr, Predicted: ckb +Key: kmr_cv_kmr_000790, Target: kmr, Predicted: ckb +Key: kmr_cv_kmr_000791, Target: kmr, Predicted: tok +Key: kmr_cv_kmr_000834, Target: kmr, Predicted: ces +Key: lao_fleurs_lao_000006, Target: lao, Predicted: tha +Key: kmr_cv_kmr_000850, Target: kmr, Predicted: bul +Key: lao_fleurs_lao_000019, Target: lao, Predicted: tha +Key: lao_fleurs_lao_000056, Target: lao, Predicted: tha +Key: lav_cv_lav_001032, Target: lav, Predicted: ukr +Key: lav_cv_lav_000937, Target: lav, Predicted: ukr +Key: lav_cv_lav_001038, Target: lav, Predicted: ukr +Key: lit_cv_lit_000673, Target: lit, Predicted: bul +Key: ltz_fleurs_ltz_000031, Target: ltz, Predicted: dan +Key: mal_cv_mal_000053, Target: mal, Predicted: tam +Key: mal_cv_mal_000056, Target: mal, Predicted: div +Key: mal_cv_mal_000088, Target: mal, Predicted: tam +Key: mal_cv_mal_000032, Target: mal, Predicted: chv +Key: mal_cv_mal_000096, Target: mal, Predicted: ces +Key: mal_cv_mal_000097, Target: mal, Predicted: ces +Key: mal_cv_mal_000004, Target: mal, Predicted: tam +Key: mal_cv_mal_000005, Target: mal, Predicted: skr +Key: mal_cv_mal_000006, Target: mal, Predicted: skr +Key: mal_cv_mal_000007, Target: mal, Predicted: skr +Key: mal_cv_mal_000009, Target: mal, Predicted: skr +Key: mal_cv_mal_000011, Target: mal, Predicted: tam +Key: mal_cv_mal_000015, Target: mal, Predicted: urd +Key: mal_cv_mal_000020, Target: mal, Predicted: nan +Key: mal_googlei18n-tts_mal_000755, Target: mal, Predicted: tam +Key: mal_googlei18n-tts_mal_000756, Target: mal, Predicted: tam +Key: mal_googlei18n-tts_mal_000760, Target: mal, Predicted: tam +Key: mal_googlei18n-tts_mal_000796, Target: mal, Predicted: tam +Key: mal_googlei18n-tts_mal_000765, Target: mal, Predicted: tam +Key: mal_googlei18n-tts_mal_000802, Target: mal, Predicted: tam +Key: mal_cv_mal_000144, Target: mal, Predicted: ces +Key: mal_googlei18n-tts_mal_000847, Target: mal, Predicted: sin +Key: mal_googlei18n-tts_mal_000835, Target: mal, Predicted: tam +Key: mal_googlei18n-tts_mal_000842, Target: mal, Predicted: tam +Key: mal_googlei18n-tts_mal_000874, Target: mal, Predicted: tam +Key: mal_googlei18n-tts_mal_000876, Target: mal, Predicted: tam +Key: mar_fleurs_mar_000030, Target: mar, Predicted: guj +Key: mar_googlei18n-tts_mar_000568, Target: mar, Predicted: hin +Key: mhr_cv_mhr_000067, Target: mhr, Predicted: mrj +Key: mhr_cv_mhr_000005, Target: mhr, Predicted: rus +Key: mhr_cv_mhr_000006, Target: mhr, Predicted: rus +Key: mhr_cv_mhr_000071, Target: mhr, Predicted: hau +Key: mhr_cv_mhr_000042, Target: mhr, Predicted: mrj +Key: mhr_cv_mhr_000079, Target: mhr, Predicted: bel +Key: mhr_cv_mhr_000080, Target: mhr, Predicted: rus +Key: mhr_cv_mhr_000018, Target: mhr, Predicted: myv +Key: mhr_cv_mhr_000052, Target: mhr, Predicted: myv +Key: mhr_cv_mhr_000022, Target: mhr, Predicted: mrj +Key: mhr_cv_mhr_000054, Target: mhr, Predicted: mrj +Key: mlt_cv_mlt_000752, Target: mlt, Predicted: ita +Key: mlt_cv_mlt_000786, Target: mlt, Predicted: ces +Key: mon_cv_mon_000704, Target: mon, Predicted: kir +Key: mrj_cv_mrj_000707, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000739, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000740, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000709, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000741, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000773, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000710, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000742, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000774, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000743, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000775, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000712, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000744, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000713, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000745, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000714, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000746, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000778, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000747, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000779, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000748, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000717, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000749, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000781, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000718, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000750, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000782, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000719, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000751, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000720, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000752, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000784, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000721, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000785, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000722, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000754, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000786, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000723, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000755, Target: mrj, Predicted: myv +Key: mrj_cv_mrj_000787, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000756, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000788, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000725, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000757, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000789, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000726, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000758, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000790, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000727, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000759, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000728, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000760, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000729, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000761, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000793, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000730, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000794, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000763, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000795, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000732, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000764, Target: mrj, Predicted: bel +Key: mrj_cv_mrj_000796, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000733, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000765, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000797, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000766, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000798, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000735, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000767, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000799, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000736, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000768, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000800, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000737, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000769, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000801, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000738, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000770, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000802, Target: mrj, Predicted: mhr +Key: msa_fleurs_msa_000014, Target: msa, Predicted: orm +Key: mrj_cv_mrj_000805, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000806, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000807, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000808, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000810, Target: mrj, Predicted: mhr +Key: msa_fleurs_msa_000054, Target: msa, Predicted: ind +Key: mrj_cv_mrj_000811, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000812, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000814, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000816, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000817, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000818, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000819, Target: mrj, Predicted: mhr +Key: mrj_cv_mrj_000820, Target: mrj, Predicted: mhr +Key: msa_fleurs_msa_000037, Target: msa, Predicted: ind +Key: myv_cv_myv_000643, Target: myv, Predicted: lav +Key: nan_cv_nan_000020, Target: nan, Predicted: yue +Key: nan_cv_nan_000021, Target: nan, Predicted: kat +Key: nan_cv_nan_000032, Target: nan, Predicted: ces +Key: nan_cv_nan_000003, Target: nan, Predicted: yue +Key: myv_cv_myv_000677, Target: myv, Predicted: lav +Key: nan_cv_nan_000041, Target: nan, Predicted: cnh +Key: nan_cv_nan_000043, Target: nan, Predicted: cnh +Key: nan_cv_nan_000075, Target: nan, Predicted: cnh +Key: nan_cv_nan_000110, Target: nan, Predicted: kat +Key: nan_cv_nan_000080, Target: nan, Predicted: abk +Key: nan_cv_nan_000113, Target: nan, Predicted: kir +Key: nan_cv_nan_000084, Target: nan, Predicted: yue +Key: nan_cv_nan_000153, Target: nan, Predicted: cnh +Key: nan_cv_nan_000092, Target: nan, Predicted: yue +Key: nan_cv_nan_000062, Target: nan, Predicted: abk +Key: nan_cv_nan_000128, Target: nan, Predicted: tha +Key: nan_cv_nan_000129, Target: nan, Predicted: yue +Key: nan_cv_nan_000098, Target: nan, Predicted: yue +Key: nan_cv_nan_000135, Target: nan, Predicted: yue +Key: nan_cv_nan_000072, Target: nan, Predicted: cnh +Key: nan_cv_nan_000136, Target: nan, Predicted: ckb +Key: nan_cv_nan_000073, Target: nan, Predicted: yue +Key: nbl_nchlt_nbl_000780, Target: nbl, Predicted: zul +Key: nbl_nchlt_nbl_000812, Target: nbl, Predicted: ven +Key: nbl_nchlt_nbl_000781, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000750, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000814, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000751, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000783, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000752, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000786, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000755, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000787, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000819, Target: nbl, Predicted: ven +Key: nbl_nchlt_nbl_000756, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000788, Target: nbl, Predicted: ven +Key: nbl_nchlt_nbl_000757, Target: nbl, Predicted: zul +Key: nbl_nchlt_nbl_000789, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000821, Target: nbl, Predicted: ven +Key: nbl_nchlt_nbl_000822, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000791, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000760, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000761, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000793, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000825, Target: nbl, Predicted: zul +Key: nbl_nchlt_nbl_000762, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000794, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000826, Target: nbl, Predicted: nso +Key: nbl_nchlt_nbl_000763, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000827, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000764, Target: nbl, Predicted: ven +Key: nbl_nchlt_nbl_000796, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000765, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000797, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000829, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000766, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000798, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000830, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000767, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000831, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000768, Target: nbl, Predicted: ven +Key: nbl_nchlt_nbl_000769, Target: nbl, Predicted: ven +Key: nbl_nchlt_nbl_000833, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000770, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000802, Target: nbl, Predicted: zul +Key: nbl_nchlt_nbl_000834, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000835, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000772, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000836, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000773, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000805, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000837, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000806, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000838, Target: nbl, Predicted: ven +Key: nbl_nchlt_nbl_000775, Target: nbl, Predicted: ven +Key: nbl_nchlt_nbl_000776, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000778, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000842, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000779, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000843, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000844, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000845, Target: nbl, Predicted: ssw +Key: nep_googlei18n-asr_nep_001039, Target: nep, Predicted: sin +Key: nbl_nchlt_nbl_000847, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000848, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000850, Target: nbl, Predicted: ven +Key: nbl_nchlt_nbl_000851, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000852, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000854, Target: nbl, Predicted: nso +Key: nbl_nchlt_nbl_000855, Target: nbl, Predicted: xho +Key: nep_googlei18n-asr_nep_001048, Target: nep, Predicted: nso +Key: nbl_nchlt_nbl_000856, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000858, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000859, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000860, Target: nbl, Predicted: nso +Key: nbl_nchlt_nbl_000861, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000865, Target: nbl, Predicted: ven +Key: nep_googlei18n-asr_nep_001027, Target: nep, Predicted: eng +Key: nbl_nchlt_nbl_000867, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000870, Target: nbl, Predicted: ssw +Key: nbl_nchlt_nbl_000871, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000872, Target: nbl, Predicted: ven +Key: nbl_nchlt_nbl_000873, Target: nbl, Predicted: xho +Key: nbl_nchlt_nbl_000874, Target: nbl, Predicted: ssw +Key: nep_googlei18n-asr_nep_001035, Target: nep, Predicted: tel +Key: nep_googlei18n-asr_nep_001106, Target: nep, Predicted: hin +Key: nep_googlei18n-asr_nep_001139, Target: nep, Predicted: hin +Key: nep_googlei18n-asr_nep_001109, Target: nep, Predicted: ben +Key: nep_googlei18n-asr_nep_001119, Target: nep, Predicted: guj +Key: nep_googlei18n-asr_nep_001121, Target: nep, Predicted: mar +Key: nep_googlei18n-asr_nep_001091, Target: nep, Predicted: wol +Key: nep_googlei18n-asr_nep_001164, Target: nep, Predicted: hin +Key: nep_googlei18n-asr_nep_001201, Target: nep, Predicted: tel +Key: nep_googlei18n-asr_nep_001202, Target: nep, Predicted: wol +Key: nld_cv_nld_000727, Target: nld, Predicted: gle +Key: nld_mls_nld_000279, Target: nld, Predicted: afr +Key: nld_swc_nld_001293, Target: nld, Predicted: fin +Key: nld_swc_nld_001334, Target: nld, Predicted: deu +Key: nld_swc_nld_001307, Target: nld, Predicted: afr +Key: nld_swc_nld_001315, Target: nld, Predicted: bre +Key: nld_swc_nld_001318, Target: nld, Predicted: por +Key: nld_swc_nld_001387, Target: nld, Predicted: ell +Key: nld_swc_nld_001358, Target: nld, Predicted: sot +Key: nld_swc_nld_001423, Target: nld, Predicted: ssw +Key: nld_swc_nld_001459, Target: nld, Predicted: eng +Key: nld_swc_nld_001467, Target: nld, Predicted: ell +Key: nld_swc_nld_001437, Target: nld, Predicted: swe +Key: nld_voxforge_nld_000861, Target: nld, Predicted: afr +Key: nld_voxforge_nld_000831, Target: nld, Predicted: afr +Key: nld_swc_nld_001487, Target: nld, Predicted: deu +Key: nld_voxforge_nld_000905, Target: nld, Predicted: eng +Key: nld_voxforge_nld_000916, Target: nld, Predicted: isl +Key: nso_nchlt_nso_001067, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001099, Target: nso, Predicted: ssw +Key: nso_nchlt_nso_001074, Target: nso, Predicted: ssw +Key: nld_voxpopuli_nld_001931, Target: nld, Predicted: deu +Key: nso_nchlt_nso_001110, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001083, Target: nso, Predicted: sna +Key: nso_nchlt_nso_001055, Target: nso, Predicted: fra +Key: nso_nchlt_nso_001120, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001121, Target: nso, Predicted: nbl +Key: nso_nchlt_nso_001126, Target: nso, Predicted: nbl +Key: nso_nchlt_nso_001128, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001097, Target: nso, Predicted: xho +Key: nso_nchlt_nso_001098, Target: nso, Predicted: nbl +Key: nso_nchlt_nso_001130, Target: nso, Predicted: wol +Key: nso_nchlt_nso_001227, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001132, Target: nso, Predicted: umb +Key: nso_nchlt_nso_001136, Target: nso, Predicted: sot +Key: nso_nchlt_nso_001234, Target: nso, Predicted: sot +Key: nso_nchlt_nso_001236, Target: nso, Predicted: sot +Key: nso_nchlt_nso_001173, Target: nso, Predicted: eng +Key: nso_nchlt_nso_001205, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001237, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001174, Target: nso, Predicted: ssw +Key: nso_nchlt_nso_001238, Target: nso, Predicted: abk +Key: nso_nchlt_nso_001143, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001175, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001207, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001239, Target: nso, Predicted: xho +Key: nso_nchlt_nso_001144, Target: nso, Predicted: ssw +Key: nso_nchlt_nso_001208, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001211, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001180, Target: nso, Predicted: nbl +Key: nso_nchlt_nso_001150, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001183, Target: nso, Predicted: lin +Key: nso_nchlt_nso_001186, Target: nso, Predicted: lug +Key: nso_nchlt_nso_001218, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001188, Target: nso, Predicted: sot +Key: nso_nchlt_nso_001220, Target: nso, Predicted: xty +Key: nso_nchlt_nso_001157, Target: nso, Predicted: nbl +Key: nso_nchlt_nso_001221, Target: nso, Predicted: umb +Key: nya_fleurs_nya_000011, Target: nya, Predicted: spa +Key: nso_nchlt_nso_001222, Target: nso, Predicted: ven +Key: nso_nchlt_nso_001192, Target: nso, Predicted: ven +Key: nya_fleurs_nya_000015, Target: nya, Predicted: spa +Key: nso_nchlt_nso_001162, Target: nso, Predicted: sot +Key: nso_nchlt_nso_001226, Target: nso, Predicted: wol +Key: oci_fleurs_oci_000019, Target: oci, Predicted: epo +Key: nya_fleurs_nya_000025, Target: nya, Predicted: umb +Key: oci_fleurs_oci_000024, Target: oci, Predicted: lin +Key: oci_fleurs_oci_000037, Target: oci, Predicted: fra +Key: oci_fleurs_oci_000012, Target: oci, Predicted: epo +Key: ori_fleurs_ori_000025, Target: ori, Predicted: asm +Key: ori_fleurs_ori_000033, Target: ori, Predicted: pan +Key: ori_fleurs_ori_000009, Target: ori, Predicted: pan +Key: orm_fleurs_orm_000003, Target: orm, Predicted: wol +Key: pan_fleurs_pan_000037, Target: pan, Predicted: mal +Key: pol_cv_pol_000708, Target: pol, Predicted: ces +Key: pol_cv_pol_000745, Target: pol, Predicted: ukr +Key: pol_cv_pol_000810, Target: pol, Predicted: gle +Key: pol_cv_pol_000715, Target: pol, Predicted: bel +Key: pol_cv_pol_000811, Target: pol, Predicted: cym +Key: pol_cv_pol_000716, Target: pol, Predicted: bel +Key: pol_cv_pol_000812, Target: pol, Predicted: ces +Key: pol_cv_pol_000717, Target: pol, Predicted: ces +Key: pol_cv_pol_000781, Target: pol, Predicted: lav +Key: pol_cv_pol_000813, Target: pol, Predicted: kir +Key: pol_cv_pol_000756, Target: pol, Predicted: ces +Key: pol_cv_pol_000761, Target: pol, Predicted: ukr +Key: pol_cv_pol_000796, Target: pol, Predicted: ces +Key: pol_cv_pol_000802, Target: pol, Predicted: slk +Key: por_cv_por_000841, Target: por, Predicted: kat +Key: por_cv_por_000845, Target: por, Predicted: ita +Key: por_cv_por_000783, Target: por, Predicted: glg +Key: por_cv_por_000784, Target: por, Predicted: kat +Key: por_cv_por_000818, Target: por, Predicted: eng +Key: por_cv_por_000787, Target: por, Predicted: grn +Key: por_cv_por_000858, Target: por, Predicted: div +Key: por_cv_por_000828, Target: por, Predicted: abk +Key: por_cv_por_000862, Target: por, Predicted: kab +Key: por_cv_por_000865, Target: por, Predicted: ces +Key: por_voxforge_por_000972, Target: por, Predicted: glg +Key: por_voxforge_por_000989, Target: por, Predicted: nld +Key: por_voxforge_por_000993, Target: por, Predicted: fin +Key: por_voxforge_por_000996, Target: por, Predicted: pol +Key: por_voxforge_por_000964, Target: por, Predicted: grn +Key: por_voxforge_por_000999, Target: por, Predicted: rus +Key: por_voxforge_por_001006, Target: por, Predicted: slv +Key: por_voxforge_por_001038, Target: por, Predicted: rus +Key: por_voxforge_por_001050, Target: por, Predicted: glg +Key: por_voxforge_por_001019, Target: por, Predicted: ron +Key: por_voxforge_por_001063, Target: por, Predicted: glg +Key: por_voxforge_por_001096, Target: por, Predicted: glg +Key: por_voxforge_por_001065, Target: por, Predicted: ibo +Key: ron_cv_ron_000034, Target: ron, Predicted: bul +Key: ron_cv_ron_000004, Target: ron, Predicted: cat +Key: ron_cv_ron_000007, Target: ron, Predicted: rus +Key: ron_cv_ron_000136, Target: ron, Predicted: chv +Key: rus_LAD_rus_000323, Target: rus, Predicted: bel +Key: rus_cv_rus_000752, Target: rus, Predicted: dan +Key: rus_cv_rus_000690, Target: rus, Predicted: ces +Key: rus_cv_rus_000692, Target: rus, Predicted: ces +Key: rus_cv_rus_000727, Target: rus, Predicted: ita +Key: rus_cv_rus_000697, Target: rus, Predicted: abk +Key: rus_cv_rus_000731, Target: rus, Predicted: bel +Key: rus_cv_rus_000735, Target: rus, Predicted: bel +Key: rus_cv_rus_000768, Target: rus, Predicted: ell +Key: rus_cv_rus_000737, Target: rus, Predicted: myv +Key: rus_cv_rus_000770, Target: rus, Predicted: bel +Key: rus_cv_rus_000708, Target: rus, Predicted: bel +Key: rus_cv_rus_000742, Target: rus, Predicted: ces +Key: rus_cv_rus_000777, Target: rus, Predicted: bel +Key: rus_cv_rus_000716, Target: rus, Predicted: bel +Key: rus_cv_rus_000750, Target: rus, Predicted: kin +Key: rus_cv_rus_000687, Target: rus, Predicted: mhr +Key: rus_cv_rus_000751, Target: rus, Predicted: kat +Key: rus_fleurs_rus_000023, Target: rus, Predicted: ukr +Key: rus_voxforge_rus_000456, Target: rus, Predicted: myv +Key: rus_cv_rus_000788, Target: rus, Predicted: mhr +Key: rus_voxforge_rus_000471, Target: rus, Predicted: ukr +Key: sin_googlei18n-asr_sin_000845, Target: sin, Predicted: ben +Key: sin_googlei18n-asr_sin_000949, Target: sin, Predicted: wol +Key: sin_googlei18n-asr_sin_000920, Target: sin, Predicted: gug +Key: sin_googlei18n-asr_sin_000952, Target: sin, Predicted: mal +Key: sin_googlei18n-asr_sin_000890, Target: sin, Predicted: ben +Key: skr_cv_skr_000803, Target: skr, Predicted: mal +Key: skr_cv_skr_000804, Target: skr, Predicted: tam +Key: sin_googlei18n-asr_sin_000863, Target: sin, Predicted: lav +Key: sin_googlei18n-asr_sin_000866, Target: sin, Predicted: tat +Key: sin_googlei18n-asr_sin_000867, Target: sin, Predicted: gug +Key: sin_googlei18n-asr_sin_000899, Target: sin, Predicted: pan +Key: skr_cv_skr_000814, Target: skr, Predicted: ina +Key: sin_googlei18n-asr_sin_000939, Target: sin, Predicted: mlg +Key: skr_cv_skr_000824, Target: skr, Predicted: abk +Key: sin_googlei18n-asr_sin_000944, Target: sin, Predicted: asm +Key: skr_cv_skr_000826, Target: skr, Predicted: ell +Key: sin_googlei18n-asr_sin_000882, Target: sin, Predicted: mar +Key: slk_cv_slk_000884, Target: slk, Predicted: ces +Key: slk_cv_slk_000896, Target: slk, Predicted: bel +Key: slk_cv_slk_000903, Target: slk, Predicted: ces +Key: slk_cv_slk_000906, Target: slk, Predicted: bos +Key: slk_cv_slk_001005, Target: slk, Predicted: ces +Key: slk_cv_slk_000910, Target: slk, Predicted: ron +Key: slk_cv_slk_001007, Target: slk, Predicted: slv +Key: slk_cv_slk_000914, Target: slk, Predicted: tok +Key: slk_cv_slk_001011, Target: slk, Predicted: ita +Key: slk_cv_slk_000949, Target: slk, Predicted: kat +Key: slk_cv_slk_001013, Target: slk, Predicted: ces +Key: slk_cv_slk_000918, Target: slk, Predicted: hin +Key: slk_cv_slk_000982, Target: slk, Predicted: myv +Key: slk_cv_slk_000920, Target: slk, Predicted: ind +Key: slk_cv_slk_001016, Target: slk, Predicted: ces +Key: slk_cv_slk_000922, Target: slk, Predicted: nld +Key: slk_cv_slk_000924, Target: slk, Predicted: mhr +Key: slk_fleurs_slk_000004, Target: slk, Predicted: ces +Key: slk_cv_slk_000926, Target: slk, Predicted: ukr +Key: slk_cv_slk_000959, Target: slk, Predicted: kat +Key: slk_cv_slk_000929, Target: slk, Predicted: ces +Key: slk_cv_slk_000993, Target: slk, Predicted: ces +Key: slk_cv_slk_000932, Target: slk, Predicted: abk +Key: slk_cv_slk_000934, Target: slk, Predicted: kat +Key: slk_cv_slk_000999, Target: slk, Predicted: bel +Key: slk_cv_slk_000936, Target: slk, Predicted: hau +Key: slk_cv_slk_001001, Target: slk, Predicted: ces +Key: slk_cv_slk_001002, Target: slk, Predicted: ces +Key: slk_cv_slk_000939, Target: slk, Predicted: ces +Key: slv_cv_slv_000016, Target: slv, Predicted: bel +Key: slk_voxpopuli_slk_002412, Target: slk, Predicted: ces +Key: slk_voxpopuli_slk_002454, Target: slk, Predicted: pol +Key: slv_cv_slv_000025, Target: slv, Predicted: ita +Key: slv_cv_slv_000000, Target: slv, Predicted: srp +Key: slv_cv_slv_000003, Target: slv, Predicted: ukr +Key: slv_cv_slv_000004, Target: slv, Predicted: ces +Key: slv_cv_slv_000037, Target: slv, Predicted: ces +Key: slv_cv_slv_000008, Target: slv, Predicted: ces +Key: slv_cv_slv_000043, Target: slv, Predicted: ces +Key: slv_cv_slv_000047, Target: slv, Predicted: por +Key: slv_cv_slv_000050, Target: slv, Predicted: hun +Key: slv_cv_slv_000051, Target: slv, Predicted: srp +Key: slv_cv_slv_000116, Target: slv, Predicted: ell +Key: slv_cv_slv_000060, Target: slv, Predicted: slk +Key: slv_cv_slv_000065, Target: slv, Predicted: ces +Key: slv_cv_slv_000134, Target: slv, Predicted: ina +Key: slv_cv_slv_000135, Target: slv, Predicted: ita +Key: slv_fleurs_slv_000045, Target: slv, Predicted: hrv +Key: sot_googlei18n-tts_sot_000673, Target: sot, Predicted: xho +Key: sot_googlei18n-tts_sot_000674, Target: sot, Predicted: heb +Key: sot_googlei18n-tts_sot_000675, Target: sot, Predicted: cym +Key: sot_googlei18n-tts_sot_000677, Target: sot, Predicted: xho +Key: snd_fleurs_snd_000022, Target: snd, Predicted: urd +Key: sot_googlei18n-tts_sot_000680, Target: sot, Predicted: xho +Key: sot_googlei18n-tts_sot_000681, Target: sot, Predicted: xho +Key: sot_googlei18n-tts_sot_000682, Target: sot, Predicted: nso +Key: sot_googlei18n-tts_sot_000685, Target: sot, Predicted: luo +Key: sot_googlei18n-tts_sot_000686, Target: sot, Predicted: nso +Key: sot_googlei18n-tts_sot_000655, Target: sot, Predicted: nbl +Key: sot_googlei18n-tts_sot_000660, Target: sot, Predicted: kam +Key: sot_googlei18n-tts_sot_000662, Target: sot, Predicted: xho +Key: sot_googlei18n-tts_sot_000665, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001020, Target: sot, Predicted: eng +Key: sot_googlei18n-tts_sot_000703, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001021, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001053, Target: sot, Predicted: xho +Key: sot_googlei18n-tts_sot_000704, Target: sot, Predicted: wol +Key: sot_nchlt_sot_001054, Target: sot, Predicted: xho +Key: sot_googlei18n-tts_sot_000705, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001023, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001055, Target: sot, Predicted: wol +Key: sot_nchlt_sot_001024, Target: sot, Predicted: eng +Key: sot_nchlt_sot_001056, Target: sot, Predicted: oci +Key: sot_nchlt_sot_001025, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001057, Target: sot, Predicted: eng +Key: sot_googlei18n-tts_sot_000708, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001026, Target: sot, Predicted: ssw +Key: sot_nchlt_sot_001058, Target: sot, Predicted: xho +Key: sot_googlei18n-tts_sot_000741, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001027, Target: sot, Predicted: ssw +Key: sot_googlei18n-tts_sot_000710, Target: sot, Predicted: nso +Key: sot_googlei18n-tts_sot_000742, Target: sot, Predicted: xho +Key: sot_googlei18n-tts_sot_000711, Target: sot, Predicted: hat +Key: sot_googlei18n-tts_sot_000743, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001029, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001061, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001030, Target: sot, Predicted: ven +Key: sot_nchlt_sot_001062, Target: sot, Predicted: lin +Key: sot_googlei18n-tts_sot_000745, Target: sot, Predicted: eng +Key: sot_nchlt_sot_001031, Target: sot, Predicted: ven +Key: sot_nchlt_sot_001063, Target: sot, Predicted: xho +Key: sot_googlei18n-tts_sot_000746, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001032, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001064, Target: sot, Predicted: nso +Key: sot_googlei18n-tts_sot_000715, Target: sot, Predicted: wol +Key: sot_nchlt_sot_001033, Target: sot, Predicted: nso +Key: sot_googlei18n-tts_sot_000716, Target: sot, Predicted: ssw +Key: sot_googlei18n-tts_sot_000748, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001066, Target: sot, Predicted: ssw +Key: sot_nchlt_sot_001067, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001036, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001068, Target: sot, Predicted: ven +Key: sot_googlei18n-tts_sot_000719, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001037, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001069, Target: sot, Predicted: ven +Key: sot_googlei18n-tts_sot_000752, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001038, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001070, Target: sot, Predicted: umb +Key: sot_nchlt_sot_001039, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001071, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001040, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001072, Target: sot, Predicted: ssw +Key: sot_googlei18n-tts_sot_000723, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001041, Target: sot, Predicted: ssw +Key: sot_nchlt_sot_001073, Target: sot, Predicted: ssw +Key: sot_googlei18n-tts_sot_000724, Target: sot, Predicted: sna +Key: sot_googlei18n-tts_sot_000756, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001042, Target: sot, Predicted: nso +Key: sot_googlei18n-tts_sot_000757, Target: sot, Predicted: sna +Key: sot_nchlt_sot_001043, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001075, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001044, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001045, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001077, Target: sot, Predicted: nso +Key: sot_googlei18n-tts_sot_000760, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001078, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001047, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001079, Target: sot, Predicted: nep +Key: sot_googlei18n-tts_sot_000730, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001016, Target: sot, Predicted: umb +Key: sot_nchlt_sot_001048, Target: sot, Predicted: tam +Key: sot_nchlt_sot_001080, Target: sot, Predicted: lin +Key: sot_nchlt_sot_001017, Target: sot, Predicted: ssw +Key: sot_nchlt_sot_001049, Target: sot, Predicted: nor +Key: sot_nchlt_sot_001081, Target: sot, Predicted: ven +Key: sot_googlei18n-tts_sot_000732, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001018, Target: sot, Predicted: wol +Key: sot_nchlt_sot_001082, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001051, Target: sot, Predicted: nbl +Key: sot_nchlt_sot_001083, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001084, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001148, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001180, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001085, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001117, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001149, Target: sot, Predicted: ven +Key: sot_nchlt_sot_001181, Target: sot, Predicted: ven +Key: sot_nchlt_sot_001086, Target: sot, Predicted: ssw +Key: sot_nchlt_sot_001118, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001150, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001182, Target: sot, Predicted: amh +Key: sot_nchlt_sot_001087, Target: sot, Predicted: ven +Key: sot_nchlt_sot_001119, Target: sot, Predicted: mar +Key: sot_nchlt_sot_001151, Target: sot, Predicted: ven +Key: sot_nchlt_sot_001183, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001088, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001120, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001152, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001184, Target: sot, Predicted: hau +Key: sot_nchlt_sot_001089, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001122, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001091, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001123, Target: sot, Predicted: ssw +Key: sot_nchlt_sot_001155, Target: sot, Predicted: eng +Key: sot_nchlt_sot_001124, Target: sot, Predicted: ssw +Key: sot_nchlt_sot_001156, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001093, Target: sot, Predicted: orm +Key: sot_nchlt_sot_001157, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001094, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001126, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001158, Target: sot, Predicted: ssw +Key: sot_nchlt_sot_001159, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001096, Target: sot, Predicted: ven +Key: sot_nchlt_sot_001128, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001160, Target: sot, Predicted: nbl +Key: sot_nchlt_sot_001129, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001161, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001098, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001130, Target: sot, Predicted: wol +Key: sot_nchlt_sot_001099, Target: sot, Predicted: dan +Key: sot_nchlt_sot_001131, Target: sot, Predicted: ven +Key: sot_nchlt_sot_001163, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001100, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001132, Target: sot, Predicted: ven +Key: sot_nchlt_sot_001164, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001101, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001133, Target: sot, Predicted: hau +Key: sot_nchlt_sot_001102, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001134, Target: sot, Predicted: ven +Key: sot_nchlt_sot_001166, Target: sot, Predicted: hau +Key: sot_nchlt_sot_001135, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001167, Target: sot, Predicted: ssw +Key: sot_nchlt_sot_001104, Target: sot, Predicted: ven +Key: sot_nchlt_sot_001168, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001169, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001106, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001170, Target: sot, Predicted: ssw +Key: sot_nchlt_sot_001107, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001139, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001171, Target: sot, Predicted: hau +Key: sot_nchlt_sot_001108, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001172, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001109, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001141, Target: sot, Predicted: san +Key: sot_nchlt_sot_001142, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001174, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001111, Target: sot, Predicted: hat +Key: sot_nchlt_sot_001112, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001113, Target: sot, Predicted: xho +Key: sot_nchlt_sot_001145, Target: sot, Predicted: nep +Key: sot_nchlt_sot_001177, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001114, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001146, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001178, Target: sot, Predicted: wol +Key: sot_nchlt_sot_001115, Target: sot, Predicted: nso +Key: sot_nchlt_sot_001147, Target: sot, Predicted: azz +Key: sot_nchlt_sot_001179, Target: sot, Predicted: ven +Key: spa_cv_spa_000662, Target: spa, Predicted: ina +Key: spa_cv_spa_000636, Target: spa, Predicted: glg +Key: spa_cv_spa_000668, Target: spa, Predicted: ast +Key: spa_cv_spa_000669, Target: spa, Predicted: ina +Key: spa_cv_spa_000638, Target: spa, Predicted: por +Key: spa_cv_spa_000641, Target: spa, Predicted: xho +Key: spa_cv_spa_000642, Target: spa, Predicted: kat +Key: spa_cv_spa_000676, Target: spa, Predicted: glg +Key: spa_cv_spa_000680, Target: spa, Predicted: lav +Key: spa_cv_spa_000681, Target: spa, Predicted: glg +Key: spa_cv_spa_000652, Target: spa, Predicted: ina +Key: spa_cv_spa_000654, Target: spa, Predicted: bre +Key: spa_cv_spa_000687, Target: spa, Predicted: ast +Key: spa_cv_spa_000688, Target: spa, Predicted: cat +Key: spa_cv_spa_000658, Target: spa, Predicted: asm +Key: spa_cv_spa_000690, Target: spa, Predicted: bre +Key: spa_cv_spa_000659, Target: spa, Predicted: por +Key: spa_cv_spa_000692, Target: spa, Predicted: por +Key: spa_cv_spa_000694, Target: spa, Predicted: glg +Key: spa_cv_spa_000728, Target: spa, Predicted: ita +Key: spa_cv_spa_000730, Target: spa, Predicted: glg +Key: spa_cv_spa_000700, Target: spa, Predicted: ita +Key: spa_cv_spa_000703, Target: spa, Predicted: glg +Key: spa_cv_spa_000704, Target: spa, Predicted: ina +Key: spa_cv_spa_000736, Target: spa, Predicted: por +Key: spa_cv_spa_000737, Target: spa, Predicted: bre +Key: spa_mls_spa_000262, Target: spa, Predicted: ast +Key: spa_mls_spa_000264, Target: spa, Predicted: ina +Key: spa_cv_spa_000710, Target: spa, Predicted: ben +Key: spa_cv_spa_000715, Target: spa, Predicted: ces +Key: spa_cv_spa_000717, Target: spa, Predicted: ell +Key: spa_voxforge_spa_000474, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000508, Target: spa, Predicted: jav +Key: spa_mls_spa_000283, Target: spa, Predicted: ina +Key: spa_voxforge_spa_000477, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000478, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000510, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000511, Target: spa, Predicted: glg +Key: spa_mls_spa_000286, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000512, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000449, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000481, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000513, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000450, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000482, Target: spa, Predicted: glg +Key: spa_voxforge_spa_000451, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000518, Target: spa, Predicted: glg +Key: spa_voxforge_spa_000487, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000488, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000493, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000494, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000497, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000500, Target: spa, Predicted: glg +Key: spa_voxforge_spa_000501, Target: spa, Predicted: ast +Key: spa_voxforge_spa_000471, Target: spa, Predicted: ina +Key: srp_cv_srp_000004, Target: srp, Predicted: mhr +Key: srp_cv_srp_000005, Target: srp, Predicted: ukr +Key: srp_cv_srp_000037, Target: srp, Predicted: bos +Key: srp_cv_srp_000069, Target: srp, Predicted: slk +Key: srp_cv_srp_000006, Target: srp, Predicted: bel +Key: srp_cv_srp_000038, Target: srp, Predicted: bos +Key: srp_cv_srp_000070, Target: srp, Predicted: rus +Key: srp_cv_srp_000039, Target: srp, Predicted: ces +Key: srp_cv_srp_000071, Target: srp, Predicted: kat +Key: srp_cv_srp_000008, Target: srp, Predicted: bos +Key: srp_cv_srp_000072, Target: srp, Predicted: slv +Key: srp_cv_srp_000104, Target: srp, Predicted: slv +Key: srp_cv_srp_000009, Target: srp, Predicted: slv +Key: srp_cv_srp_000073, Target: srp, Predicted: slk +Key: srp_cv_srp_000010, Target: srp, Predicted: bre +Key: srp_cv_srp_000042, Target: srp, Predicted: ces +Key: srp_cv_srp_000074, Target: srp, Predicted: ukr +Key: srp_cv_srp_000012, Target: srp, Predicted: por +Key: srp_cv_srp_000044, Target: srp, Predicted: ukr +Key: srp_cv_srp_000108, Target: srp, Predicted: bos +Key: srp_cv_srp_000013, Target: srp, Predicted: ukr +Key: srp_cv_srp_000045, Target: srp, Predicted: eng +Key: srp_cv_srp_000014, Target: srp, Predicted: yue +Key: srp_cv_srp_000046, Target: srp, Predicted: kat +Key: srp_cv_srp_000047, Target: srp, Predicted: slk +Key: srp_cv_srp_000111, Target: srp, Predicted: ell +Key: srp_cv_srp_000016, Target: srp, Predicted: slk +Key: srp_cv_srp_000048, Target: srp, Predicted: slv +Key: srp_cv_srp_000017, Target: srp, Predicted: bos +Key: srp_cv_srp_000049, Target: srp, Predicted: bos +Key: srp_cv_srp_000081, Target: srp, Predicted: bel +Key: srp_cv_srp_000113, Target: srp, Predicted: ces +Key: srp_cv_srp_000018, Target: srp, Predicted: slk +Key: srp_cv_srp_000050, Target: srp, Predicted: kat +Key: srp_cv_srp_000082, Target: srp, Predicted: slk +Key: srp_cv_srp_000114, Target: srp, Predicted: bos +Key: srp_cv_srp_000019, Target: srp, Predicted: hrv +Key: srp_cv_srp_000083, Target: srp, Predicted: bos +Key: srp_cv_srp_000020, Target: srp, Predicted: hrv +Key: srp_cv_srp_000052, Target: srp, Predicted: bos +Key: srp_cv_srp_000116, Target: srp, Predicted: bul +Key: srp_cv_srp_000053, Target: srp, Predicted: ces +Key: srp_cv_srp_000117, Target: srp, Predicted: bos +Key: srp_cv_srp_000054, Target: srp, Predicted: bos +Key: srp_cv_srp_000086, Target: srp, Predicted: rus +Key: srp_cv_srp_000023, Target: srp, Predicted: kat +Key: srp_cv_srp_000055, Target: srp, Predicted: por +Key: srp_cv_srp_000087, Target: srp, Predicted: kat +Key: srp_cv_srp_000119, Target: srp, Predicted: mkd +Key: srp_cv_srp_000088, Target: srp, Predicted: mrj +Key: srp_cv_srp_000120, Target: srp, Predicted: hrv +Key: srp_cv_srp_000025, Target: srp, Predicted: bos +Key: srp_cv_srp_000089, Target: srp, Predicted: mhr +Key: srp_cv_srp_000026, Target: srp, Predicted: bos +Key: srp_cv_srp_000058, Target: srp, Predicted: ori +Key: srp_cv_srp_000090, Target: srp, Predicted: ukr +Key: srp_cv_srp_000027, Target: srp, Predicted: ces +Key: srp_cv_srp_000059, Target: srp, Predicted: rus +Key: srp_cv_srp_000123, Target: srp, Predicted: bos +Key: srp_cv_srp_000092, Target: srp, Predicted: ukr +Key: srp_cv_srp_000029, Target: srp, Predicted: slv +Key: srp_cv_srp_000126, Target: srp, Predicted: mhr +Key: srp_cv_srp_000031, Target: srp, Predicted: ces +Key: srp_cv_srp_000095, Target: srp, Predicted: slv +Key: srp_cv_srp_000032, Target: srp, Predicted: pol +Key: srp_cv_srp_000096, Target: srp, Predicted: ces +Key: srp_cv_srp_000033, Target: srp, Predicted: bos +Key: srp_cv_srp_000097, Target: srp, Predicted: rus +Key: srp_cv_srp_000129, Target: srp, Predicted: bos +Key: srp_cv_srp_000034, Target: srp, Predicted: bos +Key: srp_cv_srp_000130, Target: srp, Predicted: slv +Key: srp_cv_srp_000035, Target: srp, Predicted: ukr +Key: srp_cv_srp_000067, Target: srp, Predicted: ces +Key: srp_cv_srp_000099, Target: srp, Predicted: ell +Key: srp_cv_srp_000036, Target: srp, Predicted: bos +Key: srp_cv_srp_000100, Target: srp, Predicted: slv +Key: srp_cv_srp_000132, Target: srp, Predicted: bos +Key: srp_cv_srp_000133, Target: srp, Predicted: ces +Key: srp_cv_srp_000165, Target: srp, Predicted: bos +Key: srp_cv_srp_000134, Target: srp, Predicted: bos +Key: srp_cv_srp_000166, Target: srp, Predicted: slv +Key: srp_fleurs_srp_000051, Target: srp, Predicted: bos +Key: srp_cv_srp_000135, Target: srp, Predicted: slk +Key: srp_cv_srp_000167, Target: srp, Predicted: slv +Key: srp_cv_srp_000136, Target: srp, Predicted: bos +Key: srp_cv_srp_000168, Target: srp, Predicted: bos +Key: srp_cv_srp_000137, Target: srp, Predicted: bos +Key: srp_fleurs_srp_000054, Target: srp, Predicted: hrv +Key: srp_cv_srp_000138, Target: srp, Predicted: slk +Key: srp_cv_srp_000170, Target: srp, Predicted: bos +Key: srp_cv_srp_000139, Target: srp, Predicted: slv +Key: srp_cv_srp_000140, Target: srp, Predicted: slk +Key: srp_cv_srp_000172, Target: srp, Predicted: bos +Key: srp_fleurs_srp_000027, Target: srp, Predicted: bos +Key: ssw_nchlt_ssw_000765, Target: ssw, Predicted: zul +Key: srp_cv_srp_000175, Target: srp, Predicted: bos +Key: ssw_nchlt_ssw_000766, Target: ssw, Predicted: nbl +Key: srp_cv_srp_000176, Target: srp, Predicted: slk +Key: srp_cv_srp_000177, Target: srp, Predicted: lav +Key: srp_cv_srp_000178, Target: srp, Predicted: slk +Key: ssw_nchlt_ssw_000769, Target: ssw, Predicted: ven +Key: srp_cv_srp_000147, Target: srp, Predicted: bul +Key: srp_cv_srp_000179, Target: srp, Predicted: bos +Key: ssw_nchlt_ssw_000770, Target: ssw, Predicted: nso +Key: srp_cv_srp_000148, Target: srp, Predicted: slv +Key: ssw_nchlt_ssw_000771, Target: ssw, Predicted: zul +Key: srp_cv_srp_000151, Target: srp, Predicted: bos +Key: ssw_nchlt_ssw_000774, Target: ssw, Predicted: xho +Key: srp_cv_srp_000154, Target: srp, Predicted: bos +Key: ssw_nchlt_ssw_000777, Target: ssw, Predicted: zul +Key: srp_fleurs_srp_000040, Target: srp, Predicted: hrv +Key: srp_cv_srp_000156, Target: srp, Predicted: slv +Key: srp_fleurs_srp_000008, Target: srp, Predicted: bos +Key: srp_cv_srp_000157, Target: srp, Predicted: kmr +Key: ssw_nchlt_ssw_000780, Target: ssw, Predicted: zul +Key: ssw_nchlt_ssw_000782, Target: ssw, Predicted: zul +Key: srp_cv_srp_000160, Target: srp, Predicted: bul +Key: srp_cv_srp_000161, Target: srp, Predicted: hrv +Key: srp_cv_srp_000162, Target: srp, Predicted: kat +Key: srp_cv_srp_000163, Target: srp, Predicted: bos +Key: ssw_nchlt_ssw_000786, Target: ssw, Predicted: zul +Key: srp_cv_srp_000164, Target: srp, Predicted: slk +Key: srp_fleurs_srp_000049, Target: srp, Predicted: hrv +Key: ssw_nchlt_ssw_000820, Target: ssw, Predicted: nbl +Key: ssw_nchlt_ssw_000852, Target: ssw, Predicted: zul +Key: ssw_nchlt_ssw_000853, Target: ssw, Predicted: zul +Key: sun_googlei18n-asr_sun_000679, Target: sun, Predicted: ind +Key: ssw_nchlt_ssw_000790, Target: ssw, Predicted: zul +Key: ssw_nchlt_ssw_000854, Target: ssw, Predicted: nso +Key: sun_googlei18n-asr_sun_000682, Target: sun, Predicted: msa +Key: sun_googlei18n-asr_sun_000684, Target: sun, Predicted: jav +Key: ssw_nchlt_ssw_000827, Target: ssw, Predicted: nso +Key: ssw_nchlt_ssw_000828, Target: ssw, Predicted: xho +Key: sun_googlei18n-asr_sun_000686, Target: sun, Predicted: ind +Key: ssw_nchlt_ssw_000798, Target: ssw, Predicted: zul +Key: ssw_nchlt_ssw_000862, Target: ssw, Predicted: zul +Key: ssw_nchlt_ssw_000863, Target: ssw, Predicted: por +Key: ssw_nchlt_ssw_000800, Target: ssw, Predicted: zul +Key: ssw_nchlt_ssw_000832, Target: ssw, Predicted: zul +Key: sun_googlei18n-asr_sun_000690, Target: sun, Predicted: jav +Key: ssw_nchlt_ssw_000865, Target: ssw, Predicted: zul +Key: ssw_nchlt_ssw_000866, Target: ssw, Predicted: zul +Key: ssw_nchlt_ssw_000835, Target: ssw, Predicted: nso +Key: ssw_nchlt_ssw_000805, Target: ssw, Predicted: xho +Key: ssw_nchlt_ssw_000806, Target: ssw, Predicted: zul +Key: sun_googlei18n-asr_sun_000664, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000665, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000666, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000698, Target: sun, Predicted: ind +Key: ssw_nchlt_ssw_000809, Target: ssw, Predicted: sot +Key: ssw_nchlt_ssw_000810, Target: ssw, Predicted: zul +Key: ssw_nchlt_ssw_000842, Target: ssw, Predicted: xho +Key: ssw_nchlt_ssw_000843, Target: ssw, Predicted: zul +Key: ssw_nchlt_ssw_000845, Target: ssw, Predicted: xho +Key: ssw_nchlt_ssw_000814, Target: ssw, Predicted: zul +Key: sun_googlei18n-asr_sun_000672, Target: sun, Predicted: ind +Key: ssw_nchlt_ssw_000850, Target: ssw, Predicted: nso +Key: ssw_nchlt_ssw_000819, Target: ssw, Predicted: nso +Key: sun_googlei18n-asr_sun_000743, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000747, Target: sun, Predicted: cnh +Key: sun_googlei18n-asr_sun_000717, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000752, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000756, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000727, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000760, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000764, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000734, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000766, Target: sun, Predicted: jav +Key: sun_googlei18n-asr_sun_000735, Target: sun, Predicted: ind +Key: sun_googlei18n-asr_sun_000736, Target: sun, Predicted: mar +Key: swa_ALFFA_swa_000280, Target: swa, Predicted: pus +Key: swa_cv_swa_000652, Target: swa, Predicted: kin +Key: swa_cv_swa_000653, Target: swa, Predicted: kin +Key: swa_cv_swa_000654, Target: swa, Predicted: hin +Key: swa_cv_swa_000655, Target: swa, Predicted: bre +Key: swa_cv_swa_000656, Target: swa, Predicted: ara +Key: swe_NST_swe_000752, Target: swe, Predicted: yue +Key: swa_cv_swa_000676, Target: swa, Predicted: por +Key: swe_NST_swe_000799, Target: swe, Predicted: slk +Key: swe_NST_swe_000806, Target: swe, Predicted: lat +Key: swe_NST_swe_000871, Target: swe, Predicted: deu +Key: swe_NST_swe_000809, Target: swe, Predicted: dan +Key: swe_NST_swe_000841, Target: swe, Predicted: mri +Key: swe_NST_swe_000842, Target: swe, Predicted: afr +Key: swe_NST_swe_000843, Target: swe, Predicted: nob +Key: swe_cv_swe_000743, Target: swe, Predicted: bre +Key: swe_NST_swe_000820, Target: swe, Predicted: nno +Key: swe_NST_swe_000789, Target: swe, Predicted: nno +Key: swe_cv_swe_000752, Target: swe, Predicted: fin +Key: swe_NST_swe_000825, Target: swe, Predicted: vie +Key: swe_NST_swe_000857, Target: swe, Predicted: nor +Key: swe_NST_swe_000859, Target: swe, Predicted: nor +Key: swe_NST_swe_000860, Target: swe, Predicted: nor +Key: swe_cv_swe_000824, Target: swe, Predicted: est +Key: swe_cv_swe_000804, Target: swe, Predicted: cmn +Key: swe_cv_swe_000806, Target: swe, Predicted: nan +Key: swe_cv_swe_000807, Target: swe, Predicted: gle +Key: swe_cv_swe_000851, Target: swe, Predicted: bre +Key: tam_cv_tam_000613, Target: tam, Predicted: abk +Key: tam_cv_tam_000614, Target: tam, Predicted: kat +Key: tam_cv_tam_000615, Target: tam, Predicted: ces +Key: tam_cv_tam_000616, Target: tam, Predicted: bul +Key: tam_cv_tam_000617, Target: tam, Predicted: ces +Key: tam_cv_tam_000684, Target: tam, Predicted: ces +Key: tam_cv_tam_000686, Target: tam, Predicted: kat +Key: tam_cv_tam_000687, Target: tam, Predicted: cnh +Key: tam_googlei18n-tts_tam_000847, Target: tam, Predicted: kan +Key: tam_googlei18n-tts_tam_000817, Target: tam, Predicted: div +Key: tat_cv_tat_000014, Target: tat, Predicted: ell +Key: tat_cv_tat_000015, Target: tat, Predicted: kir +Key: tat_cv_tat_000047, Target: tat, Predicted: mrj +Key: tam_googlei18n-tts_tam_000859, Target: tam, Predicted: eng +Key: tat_cv_tat_000051, Target: tat, Predicted: kab +Key: tat_cv_tat_000052, Target: tat, Predicted: kir +Key: tam_googlei18n-tts_tam_000864, Target: tam, Predicted: tel +Key: tat_cv_tat_000023, Target: tat, Predicted: bak +Key: tat_cv_tat_000055, Target: tat, Predicted: mrj +Key: tat_cv_tat_000056, Target: tat, Predicted: bak +Key: tat_cv_tat_000058, Target: tat, Predicted: bak +Key: tat_cv_tat_000027, Target: tat, Predicted: chv +Key: tat_cv_tat_000060, Target: tat, Predicted: rus +Key: tat_cv_tat_000061, Target: tat, Predicted: chv +Key: tam_googlei18n-tts_tam_000875, Target: tam, Predicted: ben +Key: tat_cv_tat_000000, Target: tat, Predicted: cnh +Key: tam_googlei18n-tts_tam_000876, Target: tam, Predicted: mal +Key: tat_cv_tat_000065, Target: tat, Predicted: rus +Key: tat_cv_tat_000004, Target: tat, Predicted: rus +Key: tat_cv_tat_000036, Target: tat, Predicted: bak +Key: tat_cv_tat_000068, Target: tat, Predicted: slv +Key: tat_cv_tat_000038, Target: tat, Predicted: bak +Key: tat_cv_tat_000071, Target: tat, Predicted: bak +Key: tat_cv_tat_000008, Target: tat, Predicted: chv +Key: tat_cv_tat_000072, Target: tat, Predicted: uig +Key: tat_cv_tat_000073, Target: tat, Predicted: bak +Key: tat_cv_tat_000074, Target: tat, Predicted: bak +Key: tat_cv_tat_000108, Target: tat, Predicted: kat +Key: tat_cv_tat_000110, Target: tat, Predicted: kab +Key: tat_cv_tat_000080, Target: tat, Predicted: bak +Key: tat_cv_tat_000112, Target: tat, Predicted: kab +Key: tat_cv_tat_000081, Target: tat, Predicted: bak +Key: tat_cv_tat_000113, Target: tat, Predicted: mrj +Key: tat_cv_tat_000088, Target: tat, Predicted: mhr +Key: tat_cv_tat_000122, Target: tat, Predicted: est +Key: tat_cv_tat_000123, Target: tat, Predicted: bak +Key: tat_cv_tat_000093, Target: tat, Predicted: mrj +Key: tat_cv_tat_000128, Target: tat, Predicted: kir +Key: tat_cv_tat_000105, Target: tat, Predicted: chv +Key: tat_cv_tat_000106, Target: tat, Predicted: kat +Key: tgk_fleurs_tgk_000025, Target: tgk, Predicted: fas +Key: tha_cv_tha_000761, Target: tha, Predicted: lao +Key: tha_cv_tha_000764, Target: tha, Predicted: lao +Key: tha_cv_tha_000769, Target: tha, Predicted: nan +Key: tha_cv_tha_000739, Target: tha, Predicted: urd +Key: tha_cv_tha_000741, Target: tha, Predicted: yue +Key: tha_cv_tha_000774, Target: tha, Predicted: yue +Key: tha_cv_tha_000777, Target: tha, Predicted: nan +Key: tha_cv_tha_000809, Target: tha, Predicted: nan +Key: tha_cv_tha_000811, Target: tha, Predicted: yue +Key: tha_cv_tha_000812, Target: tha, Predicted: kir +Key: tok_cv_tok_000910, Target: tok, Predicted: mhr +Key: tos_mexico-el_tos_003205, Target: tos, Predicted: pus +Key: tos_mexico-el_tos_003206, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003207, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003209, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003210, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003211, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003212, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003214, Target: tos, Predicted: nso +Key: tos_mexico-el_tos_003215, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003218, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003219, Target: tos, Predicted: mon +Key: tos_mexico-el_tos_003220, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003221, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003222, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003224, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003226, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003227, Target: tos, Predicted: nso +Key: tos_mexico-el_tos_003228, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003229, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003231, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003232, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003234, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003235, Target: tos, Predicted: xho +Key: tos_mexico-el_tos_003236, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003237, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003269, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003301, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003333, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003238, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003270, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003302, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003239, Target: tos, Predicted: mon +Key: tos_mexico-el_tos_003271, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003303, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003240, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003272, Target: tos, Predicted: bod +Key: tos_mexico-el_tos_003304, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003241, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003337, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003242, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003274, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003338, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003275, Target: tos, Predicted: mon +Key: tos_mexico-el_tos_003339, Target: tos, Predicted: mon +Key: tos_mexico-el_tos_003244, Target: tos, Predicted: tgl +Key: tos_mexico-el_tos_003276, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003308, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003340, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003245, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003277, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003309, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003341, Target: tos, Predicted: spa +Key: tos_mexico-el_tos_003246, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003310, Target: tos, Predicted: mon +Key: tos_mexico-el_tos_003342, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003247, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003279, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003311, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003248, Target: tos, Predicted: abk +Key: tos_mexico-el_tos_003280, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003312, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003344, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003281, Target: tos, Predicted: hau +Key: tos_mexico-el_tos_003313, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003345, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003282, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003314, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003346, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003251, Target: tos, Predicted: mon +Key: tos_mexico-el_tos_003252, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003284, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003316, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003348, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003253, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003285, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003349, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003254, Target: tos, Predicted: mon +Key: tos_mexico-el_tos_003286, Target: tos, Predicted: tgl +Key: tos_mexico-el_tos_003318, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003255, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003287, Target: tos, Predicted: pus +Key: tos_mexico-el_tos_003256, Target: tos, Predicted: mon +Key: tos_mexico-el_tos_003288, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003320, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003352, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003257, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003289, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003321, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003290, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003322, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003354, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003259, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003323, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003355, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003260, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003292, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003324, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003356, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003261, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003293, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003325, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003357, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003262, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003294, Target: tos, Predicted: luo +Key: tos_mexico-el_tos_003326, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003358, Target: tos, Predicted: mon +Key: tos_mexico-el_tos_003327, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003359, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003360, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003265, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003329, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003266, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003298, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003330, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003362, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003267, Target: tos, Predicted: xty +Key: tos_mexico-el_tos_003299, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003331, Target: tos, Predicted: bod +Key: tos_mexico-el_tos_003268, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003300, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003332, Target: tos, Predicted: tgl +Key: tos_mexico-el_tos_003364, Target: tos, Predicted: azz +Key: tos_mexico-el_tos_003365, Target: tos, Predicted: xty +Key: tsn_googlei18n-tts_tsn_000701, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000733, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000765, Target: tsn, Predicted: sot +Key: tos_mexico-el_tos_003366, Target: tos, Predicted: azz +Key: tsn_googlei18n-tts_tsn_000702, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000734, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000766, Target: tsn, Predicted: sot +Key: tos_mexico-el_tos_003367, Target: tos, Predicted: azz +Key: tsn_googlei18n-tts_tsn_000703, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000735, Target: tsn, Predicted: eng +Key: tsn_googlei18n-tts_tsn_000767, Target: tsn, Predicted: luo +Key: tos_mexico-el_tos_003368, Target: tos, Predicted: azz +Key: tsn_googlei18n-tts_tsn_000704, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000736, Target: tsn, Predicted: asm +Key: tsn_googlei18n-tts_tsn_000768, Target: tsn, Predicted: sot +Key: tos_mexico-el_tos_003369, Target: tos, Predicted: mon +Key: tsn_googlei18n-tts_tsn_000705, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000737, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000769, Target: tsn, Predicted: nso +Key: tos_mexico-el_tos_003370, Target: tos, Predicted: xty +Key: tsn_googlei18n-tts_tsn_000706, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000738, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000770, Target: tsn, Predicted: sot +Key: tos_mexico-el_tos_003371, Target: tos, Predicted: azz +Key: tsn_googlei18n-tts_tsn_000707, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000739, Target: tsn, Predicted: swa +Key: tsn_googlei18n-tts_tsn_000771, Target: tsn, Predicted: sot +Key: tos_mexico-el_tos_003372, Target: tos, Predicted: azz +Key: tsn_googlei18n-tts_tsn_000708, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000740, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000772, Target: tsn, Predicted: dan +Key: tos_mexico-el_tos_003373, Target: tos, Predicted: slv +Key: tsn_googlei18n-tts_tsn_000709, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000741, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000773, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000710, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000742, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000774, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000679, Target: tsn, Predicted: xho +Key: tsn_googlei18n-tts_tsn_000711, Target: tsn, Predicted: sna +Key: tsn_googlei18n-tts_tsn_000743, Target: tsn, Predicted: eng +Key: tsn_googlei18n-tts_tsn_000775, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000680, Target: tsn, Predicted: deu +Key: tsn_googlei18n-tts_tsn_000712, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000744, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000776, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000681, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000713, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000745, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000777, Target: tsn, Predicted: sna +Key: tsn_googlei18n-tts_tsn_000682, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000714, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000746, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000778, Target: tsn, Predicted: eng +Key: tsn_googlei18n-tts_tsn_000683, Target: tsn, Predicted: ces +Key: tsn_googlei18n-tts_tsn_000715, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000747, Target: tsn, Predicted: abk +Key: tsn_googlei18n-tts_tsn_000779, Target: tsn, Predicted: sna +Key: tsn_googlei18n-tts_tsn_000684, Target: tsn, Predicted: swa +Key: tsn_googlei18n-tts_tsn_000716, Target: tsn, Predicted: yor +Key: tsn_googlei18n-tts_tsn_000748, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000780, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000685, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000717, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000749, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000781, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000686, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000718, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000750, Target: tsn, Predicted: eng +Key: tsn_googlei18n-tts_tsn_000782, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000687, Target: tsn, Predicted: ssw +Key: tsn_googlei18n-tts_tsn_000719, Target: tsn, Predicted: xho +Key: tsn_googlei18n-tts_tsn_000751, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000783, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000688, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000720, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000752, Target: tsn, Predicted: hat +Key: tsn_googlei18n-tts_tsn_000784, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000689, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000721, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000753, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000785, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000690, Target: tsn, Predicted: xho +Key: tsn_googlei18n-tts_tsn_000722, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000754, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000786, Target: tsn, Predicted: ssw +Key: tsn_googlei18n-tts_tsn_000691, Target: tsn, Predicted: sna +Key: tsn_googlei18n-tts_tsn_000723, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000755, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000787, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000692, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000724, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000756, Target: tsn, Predicted: eng +Key: tsn_googlei18n-tts_tsn_000788, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000693, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000725, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000757, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000789, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000694, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000726, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000758, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000790, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000695, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000727, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000759, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000791, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000696, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000728, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000760, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000792, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000697, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000729, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000761, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001040, Target: tsn, Predicted: xho +Key: tsn_googlei18n-tts_tsn_000698, Target: tsn, Predicted: swa +Key: tsn_googlei18n-tts_tsn_000730, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000762, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001041, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000699, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000731, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000763, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001042, Target: tsn, Predicted: cnh +Key: tsn_googlei18n-tts_tsn_000700, Target: tsn, Predicted: nso +Key: tsn_googlei18n-tts_tsn_000732, Target: tsn, Predicted: sot +Key: tsn_googlei18n-tts_tsn_000764, Target: tsn, Predicted: sna +Key: tsn_nchlt_tsn_001043, Target: tsn, Predicted: ssw +Key: tsn_nchlt_tsn_001044, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001076, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001108, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001140, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001045, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001077, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001109, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001141, Target: tsn, Predicted: xho +Key: tsn_nchlt_tsn_001046, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001078, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001110, Target: tsn, Predicted: xho +Key: tsn_nchlt_tsn_001142, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001047, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001079, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001111, Target: tsn, Predicted: ssw +Key: tsn_nchlt_tsn_001143, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001048, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001080, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001112, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001144, Target: tsn, Predicted: spa +Key: tsn_nchlt_tsn_001049, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001081, Target: tsn, Predicted: ell +Key: tsn_nchlt_tsn_001113, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001145, Target: tsn, Predicted: xho +Key: tsn_nchlt_tsn_001050, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001082, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001114, Target: tsn, Predicted: wol +Key: tsn_nchlt_tsn_001146, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001051, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001083, Target: tsn, Predicted: xho +Key: tsn_nchlt_tsn_001115, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001147, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001052, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001084, Target: tsn, Predicted: nya +Key: tsn_nchlt_tsn_001116, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001148, Target: tsn, Predicted: ven +Key: tsn_nchlt_tsn_001053, Target: tsn, Predicted: eng +Key: tsn_nchlt_tsn_001085, Target: tsn, Predicted: nbl +Key: tsn_nchlt_tsn_001117, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001149, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001054, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001086, Target: tsn, Predicted: xho +Key: tsn_nchlt_tsn_001118, Target: tsn, Predicted: xho +Key: tsn_nchlt_tsn_001150, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001055, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001087, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001119, Target: tsn, Predicted: ssw +Key: tsn_nchlt_tsn_001151, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001056, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001088, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001120, Target: tsn, Predicted: nep +Key: tsn_nchlt_tsn_001152, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001057, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001089, Target: tsn, Predicted: xho +Key: tsn_nchlt_tsn_001121, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001153, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001058, Target: tsn, Predicted: sin +Key: tsn_nchlt_tsn_001090, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001122, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001154, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001059, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001091, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001123, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001155, Target: tsn, Predicted: xho +Key: tsn_nchlt_tsn_001060, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001092, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001124, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001156, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001061, Target: tsn, Predicted: ssw +Key: tsn_nchlt_tsn_001093, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001125, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001157, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001062, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001094, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001126, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001158, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001063, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001095, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001127, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001159, Target: tsn, Predicted: ssw +Key: tsn_nchlt_tsn_001064, Target: tsn, Predicted: ven +Key: tsn_nchlt_tsn_001096, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001128, Target: tsn, Predicted: ben +Key: tsn_nchlt_tsn_001160, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001065, Target: tsn, Predicted: ssw +Key: tsn_nchlt_tsn_001097, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001129, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001161, Target: tsn, Predicted: sna +Key: tsn_nchlt_tsn_001066, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001098, Target: tsn, Predicted: isl +Key: tsn_nchlt_tsn_001130, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001162, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001067, Target: tsn, Predicted: xho +Key: tsn_nchlt_tsn_001099, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001131, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001163, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001068, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001100, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001132, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001164, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001069, Target: tsn, Predicted: ssw +Key: tsn_nchlt_tsn_001101, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001133, Target: tsn, Predicted: wol +Key: tsn_nchlt_tsn_001165, Target: tsn, Predicted: ssw +Key: tsn_nchlt_tsn_001070, Target: tsn, Predicted: nbl +Key: tsn_nchlt_tsn_001102, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001134, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001166, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001071, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001103, Target: tsn, Predicted: xho +Key: tsn_nchlt_tsn_001135, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001167, Target: tsn, Predicted: ven +Key: tsn_nchlt_tsn_001072, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001104, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001136, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001168, Target: tsn, Predicted: ssw +Key: tsn_nchlt_tsn_001073, Target: tsn, Predicted: xho +Key: tsn_nchlt_tsn_001105, Target: tsn, Predicted: sna +Key: tsn_nchlt_tsn_001137, Target: tsn, Predicted: nbl +Key: tsn_nchlt_tsn_001169, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001074, Target: tsn, Predicted: ssw +Key: tsn_nchlt_tsn_001106, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001138, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001170, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001075, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001107, Target: tsn, Predicted: ssw +Key: tsn_nchlt_tsn_001139, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001171, Target: tsn, Predicted: som +Key: tsn_nchlt_tsn_001172, Target: tsn, Predicted: hau +Key: tsn_nchlt_tsn_001204, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000858, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000890, Target: tso, Predicted: kam +Key: tsn_nchlt_tsn_001173, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001205, Target: tsn, Predicted: xho +Key: tso_nchlt_tso_000859, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000891, Target: tso, Predicted: nbl +Key: tsn_nchlt_tsn_001174, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001206, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000860, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000892, Target: tso, Predicted: ssw +Key: tsn_nchlt_tsn_001175, Target: tsn, Predicted: sot +Key: tsn_nchlt_tsn_001207, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000861, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000893, Target: tso, Predicted: ssw +Key: tsn_nchlt_tsn_001176, Target: tsn, Predicted: nso +Key: tsn_nchlt_tsn_001208, Target: tsn, Predicted: sot +Key: tso_nchlt_tso_000862, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000894, Target: tso, Predicted: xho +Key: tsn_nchlt_tsn_001177, Target: tsn, Predicted: sot +Key: tso_nchlt_tso_000831, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000863, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000895, Target: tso, Predicted: xho +Key: tsn_nchlt_tsn_001178, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000832, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000864, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000896, Target: tso, Predicted: ssw +Key: tsn_nchlt_tsn_001179, Target: tsn, Predicted: ven +Key: tso_nchlt_tso_000833, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000865, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000897, Target: tso, Predicted: nso +Key: tsn_nchlt_tsn_001180, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000834, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000866, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000898, Target: tso, Predicted: nbl +Key: tsn_nchlt_tsn_001181, Target: tsn, Predicted: sot +Key: tso_nchlt_tso_000835, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000867, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000899, Target: tso, Predicted: xho +Key: tsn_nchlt_tsn_001182, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000836, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000868, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000900, Target: tso, Predicted: xho +Key: tsn_nchlt_tsn_001183, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000837, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000869, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000901, Target: tso, Predicted: nso +Key: tsn_nchlt_tsn_001184, Target: tsn, Predicted: xho +Key: tso_nchlt_tso_000838, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000870, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000902, Target: tso, Predicted: nso +Key: tsn_nchlt_tsn_001185, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000839, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000871, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000903, Target: tso, Predicted: nso +Key: tsn_nchlt_tsn_001186, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000840, Target: tso, Predicted: sna +Key: tso_nchlt_tso_000872, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000904, Target: tso, Predicted: nso +Key: tsn_nchlt_tsn_001187, Target: tsn, Predicted: sot +Key: tso_nchlt_tso_000841, Target: tso, Predicted: fin +Key: tso_nchlt_tso_000873, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000905, Target: tso, Predicted: ssw +Key: tsn_nchlt_tsn_001188, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000842, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000874, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000906, Target: tso, Predicted: ssw +Key: tsn_nchlt_tsn_001189, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000843, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000875, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000907, Target: tso, Predicted: sot +Key: tsn_nchlt_tsn_001190, Target: tsn, Predicted: sot +Key: tso_nchlt_tso_000844, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000876, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000908, Target: tso, Predicted: nso +Key: tsn_nchlt_tsn_001191, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000845, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000877, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000909, Target: tso, Predicted: ssw +Key: tsn_nchlt_tsn_001192, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000846, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000878, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000910, Target: tso, Predicted: ven +Key: tsn_nchlt_tsn_001193, Target: tsn, Predicted: nbl +Key: tso_nchlt_tso_000847, Target: tso, Predicted: umb +Key: tso_nchlt_tso_000879, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000911, Target: tso, Predicted: ven +Key: tsn_nchlt_tsn_001194, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000848, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000880, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000912, Target: tso, Predicted: xho +Key: tsn_nchlt_tsn_001195, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000849, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000881, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000913, Target: tso, Predicted: ssw +Key: tsn_nchlt_tsn_001196, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000850, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000882, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000914, Target: tso, Predicted: ssw +Key: tsn_nchlt_tsn_001197, Target: tsn, Predicted: ven +Key: tso_nchlt_tso_000851, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000883, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000915, Target: tso, Predicted: nso +Key: tsn_nchlt_tsn_001198, Target: tsn, Predicted: nbl +Key: tso_nchlt_tso_000852, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000884, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000916, Target: tso, Predicted: xho +Key: tsn_nchlt_tsn_001199, Target: tsn, Predicted: nbl +Key: tso_nchlt_tso_000853, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000885, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000917, Target: tso, Predicted: ven +Key: tsn_nchlt_tsn_001200, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000854, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000886, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000918, Target: tso, Predicted: ven +Key: tsn_nchlt_tsn_001201, Target: tsn, Predicted: sot +Key: tso_nchlt_tso_000855, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000887, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000919, Target: tso, Predicted: xho +Key: tsn_nchlt_tsn_001202, Target: tsn, Predicted: nso +Key: tso_nchlt_tso_000856, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000888, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000920, Target: tso, Predicted: ssw +Key: tsn_nchlt_tsn_001203, Target: tsn, Predicted: xho +Key: tso_nchlt_tso_000857, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000889, Target: tso, Predicted: sna +Key: tso_nchlt_tso_000921, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000922, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000954, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000986, Target: tso, Predicted: sot +Key: tso_nchlt_tso_000923, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000955, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000987, Target: tso, Predicted: sna +Key: tur_cv_tur_000022, Target: tur, Predicted: kir +Key: tso_nchlt_tso_000924, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000956, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000988, Target: tso, Predicted: ssw +Key: tur_cv_tur_000023, Target: tur, Predicted: uig +Key: tso_nchlt_tso_000925, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000957, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000989, Target: tso, Predicted: ven +Key: tur_cv_tur_000024, Target: tur, Predicted: gle +Key: tso_nchlt_tso_000926, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000958, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000990, Target: tso, Predicted: sot +Key: tso_nchlt_tso_000927, Target: tso, Predicted: sot +Key: tso_nchlt_tso_000959, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000991, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000928, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000960, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000992, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000929, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000961, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000993, Target: tso, Predicted: nso +Key: tur_cv_tur_000028, Target: tur, Predicted: fas +Key: tso_nchlt_tso_000930, Target: tso, Predicted: wol +Key: tso_nchlt_tso_000962, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000994, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000931, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000963, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000995, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000932, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000964, Target: tso, Predicted: fin +Key: tso_nchlt_tso_000996, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000933, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000965, Target: tso, Predicted: nbl +Key: tur_cv_tur_000000, Target: tur, Predicted: ita +Key: tso_nchlt_tso_000934, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000966, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000935, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000967, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000936, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000968, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000937, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000969, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000938, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000970, Target: tso, Predicted: xty +Key: tso_nchlt_tso_000939, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000971, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000940, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000972, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000941, Target: tso, Predicted: sot +Key: tso_nchlt_tso_000973, Target: tso, Predicted: xty +Key: tur_cv_tur_000008, Target: tur, Predicted: tat +Key: tur_cv_tur_000040, Target: tur, Predicted: uig +Key: tso_nchlt_tso_000942, Target: tso, Predicted: ssw +Key: tso_nchlt_tso_000974, Target: tso, Predicted: ven +Key: tur_cv_tur_000041, Target: tur, Predicted: aze +Key: tso_nchlt_tso_000943, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000975, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000944, Target: tso, Predicted: xty +Key: tso_nchlt_tso_000976, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000945, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000977, Target: tso, Predicted: ven +Key: tur_cv_tur_000012, Target: tur, Predicted: aze +Key: tso_nchlt_tso_000946, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000978, Target: tso, Predicted: ven +Key: tur_cv_tur_000013, Target: tur, Predicted: bre +Key: tso_nchlt_tso_000947, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000979, Target: tso, Predicted: nso +Key: tso_nchlt_tso_000948, Target: tso, Predicted: umb +Key: tso_nchlt_tso_000980, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000949, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000981, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000950, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000982, Target: tso, Predicted: ssw +Key: tur_cv_tur_000017, Target: tur, Predicted: bak +Key: tso_nchlt_tso_000951, Target: tso, Predicted: nbl +Key: tso_nchlt_tso_000983, Target: tso, Predicted: ssw +Key: tur_cv_tur_000018, Target: tur, Predicted: kir +Key: tso_nchlt_tso_000952, Target: tso, Predicted: ven +Key: tso_nchlt_tso_000984, Target: tso, Predicted: xho +Key: tur_cv_tur_000019, Target: tur, Predicted: ara +Key: tso_nchlt_tso_000953, Target: tso, Predicted: xho +Key: tso_nchlt_tso_000985, Target: tso, Predicted: nso +Key: tur_cv_tur_000052, Target: tur, Predicted: eng +Key: tur_cv_tur_000053, Target: tur, Predicted: ell +Key: tur_cv_tur_000118, Target: tur, Predicted: kir +Key: tur_cv_tur_000119, Target: tur, Predicted: aze +Key: tur_cv_tur_000122, Target: tur, Predicted: gle +Key: tur_cv_tur_000092, Target: tur, Predicted: uzb +Key: tur_cv_tur_000124, Target: tur, Predicted: heb +Key: tur_cv_tur_000093, Target: tur, Predicted: kir +Key: tur_cv_tur_000062, Target: tur, Predicted: bre +Key: tur_cv_tur_000063, Target: tur, Predicted: ces +Key: tur_cv_tur_000066, Target: tur, Predicted: nno +Key: tur_cv_tur_000067, Target: tur, Predicted: deu +Key: tur_cv_tur_000102, Target: tur, Predicted: chv +Key: tur_cv_tur_000071, Target: tur, Predicted: aze +Key: tur_cv_tur_000104, Target: tur, Predicted: kir +Key: tur_cv_tur_000106, Target: tur, Predicted: kmr +Key: tur_cv_tur_000076, Target: tur, Predicted: aze +Key: tur_cv_tur_000077, Target: tur, Predicted: por +Key: tur_cv_tur_000081, Target: tur, Predicted: aze +Key: tur_cv_tur_000113, Target: tur, Predicted: aze +Key: tur_cv_tur_000114, Target: tur, Predicted: kir +Key: tur_cv_tur_000083, Target: tur, Predicted: kir +Key: tur_cv_tur_000115, Target: tur, Predicted: kir +Key: ukr_M-AILABS_ukr_000105, Target: ukr, Predicted: rus +Key: ukr_M-AILABS_ukr_000107, Target: ukr, Predicted: bel +Key: ukr_cv_ukr_000748, Target: ukr, Predicted: bel +Key: ukr_cv_ukr_000723, Target: ukr, Predicted: bel +Key: ukr_cv_ukr_000756, Target: ukr, Predicted: ces +Key: ukr_M-AILABS_ukr_000133, Target: ukr, Predicted: rus +Key: ukr_cv_ukr_000757, Target: ukr, Predicted: pol +Key: ukr_cv_ukr_000760, Target: ukr, Predicted: bul +Key: ukr_cv_ukr_000697, Target: ukr, Predicted: bul +Key: ukr_cv_ukr_000731, Target: ukr, Predicted: bel +Key: ukr_cv_ukr_000733, Target: ukr, Predicted: mhr +Key: ukr_cv_ukr_000678, Target: ukr, Predicted: chv +Key: urd_cv_urd_000834, Target: urd, Predicted: hin +Key: umb_fleurs_umb_000016, Target: umb, Predicted: swa +Key: urd_cv_urd_000837, Target: urd, Predicted: hin +Key: urd_cv_urd_000839, Target: urd, Predicted: div +Key: umb_fleurs_umb_000026, Target: umb, Predicted: por +Key: urd_cv_urd_000841, Target: urd, Predicted: hin +Key: urd_cv_urd_000810, Target: urd, Predicted: ori +Key: urd_cv_urd_000843, Target: urd, Predicted: skr +Key: urd_cv_urd_000813, Target: urd, Predicted: skr +Key: urd_cv_urd_000845, Target: urd, Predicted: hin +Key: urd_cv_urd_000818, Target: urd, Predicted: hin +Key: urd_cv_urd_000851, Target: urd, Predicted: hin +Key: urd_cv_urd_000852, Target: urd, Predicted: skr +Key: urd_cv_urd_000823, Target: urd, Predicted: hin +Key: urd_cv_urd_000825, Target: urd, Predicted: hin +Key: urd_cv_urd_000826, Target: urd, Predicted: hin +Key: urd_cv_urd_000829, Target: urd, Predicted: hin +Key: urd_cv_urd_000862, Target: urd, Predicted: skr +Key: urd_cv_urd_000863, Target: urd, Predicted: kat +Key: urd_cv_urd_000832, Target: urd, Predicted: hin +Key: urd_cv_urd_000864, Target: urd, Predicted: kir +Key: urd_cv_urd_000833, Target: urd, Predicted: hin +Key: urd_cv_urd_000865, Target: urd, Predicted: hin +Key: urd_cv_urd_000867, Target: urd, Predicted: skr +Key: urd_cv_urd_000899, Target: urd, Predicted: ori +Key: urd_fleurs_urd_000010, Target: urd, Predicted: hin +Key: urd_cv_urd_000868, Target: urd, Predicted: hin +Key: urd_cv_urd_000900, Target: urd, Predicted: hin +Key: urd_cv_urd_000901, Target: urd, Predicted: hin +Key: urd_cv_urd_000870, Target: urd, Predicted: kat +Key: urd_cv_urd_000871, Target: urd, Predicted: skr +Key: urd_cv_urd_000903, Target: urd, Predicted: hin +Key: urd_fleurs_urd_000017, Target: urd, Predicted: hin +Key: urd_cv_urd_000878, Target: urd, Predicted: skr +Key: urd_cv_urd_000910, Target: urd, Predicted: hin +Key: urd_fleurs_urd_000022, Target: urd, Predicted: hin +Key: urd_cv_urd_000883, Target: urd, Predicted: skr +Key: urd_fleurs_urd_000028, Target: urd, Predicted: hin +Key: urd_cv_urd_000885, Target: urd, Predicted: hin +Key: urd_cv_urd_000890, Target: urd, Predicted: hin +Key: urd_cv_urd_000923, Target: urd, Predicted: skr +Key: urd_cv_urd_000925, Target: urd, Predicted: skr +Key: urd_fleurs_urd_000037, Target: urd, Predicted: pan +Key: urd_fleurs_urd_000046, Target: urd, Predicted: hin +Key: uzb_cv_uzb_000678, Target: uzb, Predicted: uig +Key: uzb_cv_uzb_000679, Target: uzb, Predicted: ckb +Key: uzb_cv_uzb_000720, Target: uzb, Predicted: kmr +Key: uzb_cv_uzb_000758, Target: uzb, Predicted: uig +Key: uzb_cv_uzb_000664, Target: uzb, Predicted: kir +Key: uzb_cv_uzb_000699, Target: uzb, Predicted: aze +Key: uzb_fleurs_uzb_000040, Target: uzb, Predicted: aze +Key: uzb_fleurs_uzb_000009, Target: uzb, Predicted: kan +Key: ven_nchlt_ven_000917, Target: ven, Predicted: sna +Key: ven_nchlt_ven_000949, Target: ven, Predicted: sot +Key: ven_nchlt_ven_000929, Target: ven, Predicted: nso +Key: ven_nchlt_ven_000933, Target: ven, Predicted: yor +Key: ven_nchlt_ven_000965, Target: ven, Predicted: sna +Key: ven_nchlt_ven_000934, Target: ven, Predicted: yor +Key: ven_nchlt_ven_000906, Target: ven, Predicted: sna +Key: ven_nchlt_ven_000939, Target: ven, Predicted: nso +Key: ven_nchlt_ven_000908, Target: ven, Predicted: sna +Key: ven_nchlt_ven_000941, Target: ven, Predicted: nso +Key: vie_cv_vie_000981, Target: vie, Predicted: tha +Key: ven_nchlt_ven_001008, Target: ven, Predicted: nso +Key: vie_cv_vie_000983, Target: vie, Predicted: nan +Key: ven_nchlt_ven_001012, Target: ven, Predicted: lug +Key: ven_nchlt_ven_000986, Target: ven, Predicted: sot +Key: ven_nchlt_ven_001018, Target: ven, Predicted: nso +Key: vie_cv_vie_000960, Target: vie, Predicted: cnh +Key: ven_nchlt_ven_000987, Target: ven, Predicted: nso +Key: ven_nchlt_ven_001019, Target: ven, Predicted: nso +Key: vie_cv_vie_000961, Target: vie, Predicted: tha +Key: ven_nchlt_ven_000988, Target: ven, Predicted: nya +Key: vie_cv_vie_001003, Target: vie, Predicted: tha +Key: ven_nchlt_ven_000999, Target: ven, Predicted: nso +Key: vie_cv_vie_001014, Target: vie, Predicted: yue +Key: vie_cv_vie_001078, Target: vie, Predicted: tha +Key: vie_cv_vie_001015, Target: vie, Predicted: yue +Key: vie_cv_vie_001086, Target: vie, Predicted: tha +Key: vie_cv_vie_001039, Target: vie, Predicted: yue +Key: vie_cv_vie_001075, Target: vie, Predicted: tha +Key: xho_fleurs_xho_000042, Target: xho, Predicted: zul +Key: xho_fleurs_xho_000012, Target: xho, Predicted: sna +Key: xho_googlei18n-tts_xho_000801, Target: xho, Predicted: eng +Key: xho_googlei18n-tts_xho_000770, Target: xho, Predicted: eng +Key: xho_googlei18n-tts_xho_000803, Target: xho, Predicted: swa +Key: xho_googlei18n-tts_xho_000772, Target: xho, Predicted: sot +Key: xho_googlei18n-tts_xho_000805, Target: xho, Predicted: sot +Key: xho_googlei18n-tts_xho_000775, Target: xho, Predicted: zul +Key: xho_googlei18n-tts_xho_000808, Target: xho, Predicted: eng +Key: xho_nchlt_xho_000843, Target: xho, Predicted: ven +Key: xho_googlei18n-tts_xho_000842, Target: xho, Predicted: eng +Key: xho_nchlt_xho_000844, Target: xho, Predicted: nbl +Key: xho_googlei18n-tts_xho_000875, Target: xho, Predicted: sot +Key: xho_nchlt_xho_000845, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000847, Target: xho, Predicted: zul +Key: xho_nchlt_xho_000849, Target: xho, Predicted: ven +Key: xho_nchlt_xho_000852, Target: xho, Predicted: ssw +Key: xho_googlei18n-tts_xho_000819, Target: xho, Predicted: sot +Key: xho_nchlt_xho_000854, Target: xho, Predicted: nbl +Key: xho_googlei18n-tts_xho_000885, Target: xho, Predicted: sot +Key: xho_googlei18n-tts_xho_000855, Target: xho, Predicted: eng +Key: xho_nchlt_xho_000859, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000828, Target: xho, Predicted: sna +Key: xho_nchlt_xho_000860, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000862, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000831, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000863, Target: xho, Predicted: ssw +Key: xho_googlei18n-tts_xho_000830, Target: xho, Predicted: eng +Key: xho_googlei18n-tts_xho_000862, Target: xho, Predicted: eng +Key: xho_nchlt_xho_000832, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000833, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000835, Target: xho, Predicted: ven +Key: xho_nchlt_xho_000867, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000837, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000838, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000839, Target: xho, Predicted: zul +Key: xho_googlei18n-tts_xho_000870, Target: xho, Predicted: eng +Key: xho_nchlt_xho_000840, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000872, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000841, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000873, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000842, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000874, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000939, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000876, Target: xho, Predicted: ven +Key: xho_nchlt_xho_000877, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000942, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000912, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000945, Target: xho, Predicted: nso +Key: xho_nchlt_xho_000882, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000914, Target: xho, Predicted: nya +Key: xho_nchlt_xho_000884, Target: xho, Predicted: ven +Key: xho_nchlt_xho_000948, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000885, Target: xho, Predicted: sna +Key: xty_mexico-el_xty_000598, Target: xty, Predicted: hrv +Key: xho_nchlt_xho_000950, Target: xho, Predicted: sot +Key: xho_nchlt_xho_000951, Target: xho, Predicted: nso +Key: xho_nchlt_xho_000889, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000953, Target: xho, Predicted: ven +Key: xho_nchlt_xho_000954, Target: xho, Predicted: grn +Key: xho_nchlt_xho_000891, Target: xho, Predicted: ven +Key: xho_nchlt_xho_000923, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000892, Target: xho, Predicted: zul +Key: xho_nchlt_xho_000925, Target: xho, Predicted: nso +Key: xho_nchlt_xho_000957, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000927, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000959, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000896, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000960, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000897, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000898, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000930, Target: xho, Predicted: nbl +Key: xho_nchlt_xho_000931, Target: xho, Predicted: ven +Key: xho_nchlt_xho_000901, Target: xho, Predicted: zul +Key: xho_nchlt_xho_000933, Target: xho, Predicted: ven +Key: xho_nchlt_xho_000902, Target: xho, Predicted: ssw +Key: xho_nchlt_xho_000903, Target: xho, Predicted: ven +Key: xho_nchlt_xho_000905, Target: xho, Predicted: kam +Key: xho_nchlt_xho_000937, Target: xho, Predicted: ven +Key: xho_nchlt_xho_000906, Target: xho, Predicted: nbl +Key: yue_cv_yue_000062, Target: yue, Predicted: nan +Key: yue_cv_yue_000075, Target: yue, Predicted: nan +Key: yue_cv_yue_000079, Target: yue, Predicted: nan +Key: yue_cv_yue_000053, Target: yue, Predicted: nan +Key: yue_cv_yue_000087, Target: yue, Predicted: nan +Key: zul_nchlt_zul_000819, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000851, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000820, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000852, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000821, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000853, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000822, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000854, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000823, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000855, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000856, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000825, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000857, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000826, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000858, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000827, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000828, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000860, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000829, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000861, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000830, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000831, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000863, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000832, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000864, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000833, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000865, Target: zul, Predicted: ssw +Key: zul_fleurs_zul_000027, Target: zul, Predicted: hau +Key: zul_nchlt_zul_000834, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000866, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000835, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000867, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000836, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000868, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000837, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000838, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000870, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000871, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000840, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000872, Target: zul, Predicted: xho +Key: zul_fleurs_zul_000034, Target: zul, Predicted: umb +Key: zul_nchlt_zul_000841, Target: zul, Predicted: nso +Key: zul_nchlt_zul_000873, Target: zul, Predicted: nso +Key: zul_fleurs_zul_000036, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000842, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000874, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000843, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000875, Target: zul, Predicted: lin +Key: zul_nchlt_zul_000812, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000844, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000876, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000813, Target: zul, Predicted: nso +Key: zul_nchlt_zul_000845, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000877, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000814, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000846, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000878, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000815, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000847, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000879, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000816, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000848, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000880, Target: zul, Predicted: nso +Key: zul_nchlt_zul_000817, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000849, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000881, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000850, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000883, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000915, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000884, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000916, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000885, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000917, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000918, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000887, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000919, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000888, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000920, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000889, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000921, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000890, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000922, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000891, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000923, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000892, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000924, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000893, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000925, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000894, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000926, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000895, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000927, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000896, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000897, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000929, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000898, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000930, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000899, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000931, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000900, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000932, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000901, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000933, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000902, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000934, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000903, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000935, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000904, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000936, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000905, Target: zul, Predicted: ven +Key: zul_nchlt_zul_000937, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000906, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000938, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000907, Target: zul, Predicted: ssw +Key: zul_nchlt_zul_000939, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000908, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000909, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000910, Target: zul, Predicted: nbl +Key: zul_nchlt_zul_000911, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000912, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000913, Target: zul, Predicted: xho +Key: zul_nchlt_zul_000914, Target: zul, Predicted: nbl diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang/lid_inference_test.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang/lid_inference_test.log new file mode 100644 index 0000000000000000000000000000000000000000..52bcdd1898ebf1ae246cadb971a54e47f66cbc9f --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang/lid_inference_test.log @@ -0,0 +1,280 @@ +# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +# Started at Mon Jun 2 00:34:26 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_voxlingua107_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +[gpue04] 2025-06-02 00:34:58,673 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + torch.load(model_file, map_location=device), +[gpue04] 2025-06-02 00:35:15,310 (lid_inference_dist:86) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary. + warnings.warn(_create_warning_msg( +/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist. + warnings.warn( +[gpue04] 2025-06-02 00:35:15,911 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang/lids0 +[gpue04] 2025-06-02 00:36:12,151 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0 +[gpue04] 2025-06-02 00:36:44,813 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True +[gpue04] 2025-06-02 00:36:44,813 (lid_inference_dist:215) INFO: args.save_tsne_plot: False +# Accounting: time=139 threads=1 +# Ended (code 0) at Mon Jun 2 00:36:45 CDT 2025, elapsed time 139 seconds diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang/results b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang/results new file mode 100644 index 0000000000000000000000000000000000000000..ac21a62d2cc5d08ac06c54e9401eb4598353bc93 --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/dev_voxlingua107_lang_cross_train_all_no_filter_lang/results @@ -0,0 +1,126 @@ +Accuracy: 94.41% +Macro Accuracy: 93.10% +Accuracy per Language: +slv: 100.00% +aze: 91.18% +cmn: 95.65% +srp: 71.43% +hun: 100.00% +fas: 89.00% +hye: 96.00% +urd: 80.77% +spa: 92.73% +est: 94.12% +tur: 98.61% +rus: 100.00% +ita: 100.00% +ara: 95.00% +nor: 54.17% +lav: 98.95% +ukr: 100.00% +swe: 96.00% +deu: 93.90% +ell: 80.00% +isl: 100.00% +nno: 100.00% +pol: 100.00% +hrv: 75.00% +dan: 93.94% +fra: 97.00% +nld: 95.00% +mkd: 100.00% +jpn: 97.62% +por: 100.00% +fin: 97.85% +eng: 96.25% +lit: 92.31% +Key: ara_cwvnYGInNNg__U__S229---0661.830-0670.560.wav, Target: ara, Predicted: heb +Key: aze_3UUShvAQxQY__U__S199---1315.800-1322.250.wav, Target: aze, Predicted: tur +Key: aze_3qOGhbHQuAc__U__S157---1061.380-1066.120.wav, Target: aze, Predicted: tur +Key: ara_AfS6C1PXAdQ__U__S20---0104.730-0111.410.wav, Target: ara, Predicted: hau +Key: ara_TPWwuy20K_c__U__S70---0466.380-0472.600.wav, Target: ara, Predicted: hau +Key: ara_XplxxijLuFI__U__S0---0372.560-0375.230.wav, Target: ara, Predicted: heb +Key: aze_bYKK1m78ecE__U__S91---0592.500-0596.130.wav, Target: aze, Predicted: fao +Key: ara_tl39W93P0r4__U__S32---0282.970-0286.530.wav, Target: ara, Predicted: dan +Key: dan_Nyl6CuW6Qfk__U__S26---0557.690-0560.120.wav, Target: dan, Predicted: nor +Key: dan_ONZC1wL5hBw__U__S100---1407.470-1417.260.wav, Target: dan, Predicted: nno +Key: cmn_ZUzq_TIfYL4__U__S39---0442.690-0454.380.wav, Target: cmn, Predicted: yue +Key: dan_SbE2FKexCW4__U__S62---0546.280-0551.260.wav, Target: dan, Predicted: isl +Key: dan_E3vuA0Mqk1Q__U__S13---0072.140-0083.530.wav, Target: dan, Predicted: nno +Key: dan_ZZD1qu4ScPg__U__S14---0166.700-0176.010.wav, Target: dan, Predicted: hat +Key: dan_yEEcGssW0Qg__U__S112---1016.050-1020.110.wav, Target: dan, Predicted: deu +Key: eng_4y7p9R2No-4__U__S12---0266.390-0268.460.wav, Target: eng, Predicted: gle +Key: deu_4zCzyVjLkcc__U__S0---0123.750-0127.540.wav, Target: deu, Predicted: ltz +Key: deu_8L3k8XNTtNA__U__S100---2689.380-2692.180.wav, Target: deu, Predicted: fin +Key: deu_9O2haSYzftE__U__S0---0000.000-0004.200.wav, Target: deu, Predicted: yid +Key: deu_cMZO2zXTBv8__U__S100---0341.910-0344.350.wav, Target: deu, Predicted: yid +Key: ell_bw_mDLVdgtY__U__S18---0119.750-0127.200.wav, Target: ell, Predicted: isl +Key: deu_eyZqRcgGkiY__U__S126---1155.890-1162.390.wav, Target: deu, Predicted: nor +Key: eng_K977aQQpAVk__U__S106---0393.230-0397.100.wav, Target: eng, Predicted: cym +Key: est_EtWRBtavckY__U__S116---1906.220-1908.810.wav, Target: est, Predicted: fin +Key: eng_eQXHc-tJMXM__U__S11---1066.230-1077.360.wav, Target: eng, Predicted: cym +Key: est_gTl2GSJBxNw__U__S0---0000.000-0008.420.wav, Target: est, Predicted: tur +Key: est_5gWpxiFOouQ__U__S2---1635.950-1646.620.wav, Target: est, Predicted: tel +Key: est_7vZIuc9qumg__U__S21---0145.690-0153.320.wav, Target: est, Predicted: dan +Key: est_E05LlgvSMg0__U__S156---1171.030-1172.780.wav, Target: est, Predicted: fin +Key: fas_SMcjja_krx4__U__S2---0012.190-0021.730.wav, Target: fas, Predicted: tgk +Key: fas_4sboRMmC2TM__U__S212---1293.790-1307.370.wav, Target: fas, Predicted: tgk +Key: fas_XUGZwtXgvRA__U__S154---0993.540-0997.340.wav, Target: fas, Predicted: san +Key: fas_9k1oVW4Ynyw__U__S15---0097.430-0101.630.wav, Target: fas, Predicted: sqi +Key: fas_nPts67VQKRQ__U__S250---1629.010-1632.750.wav, Target: fas, Predicted: hat +Key: fas_EjSRRddYuc4__U__S58---0355.980-0359.590.wav, Target: fas, Predicted: lat +Key: fas_pt166R7v8kU__U__S13---0267.910-0272.370.wav, Target: fas, Predicted: tgk +Key: fin_C4H2GlJRkNU__U__S100---1604.910-1610.210.wav, Target: fin, Predicted: est +Key: fas_x_Di4cq4ixM__U__S100---1353.580-1358.390.wav, Target: fas, Predicted: pus +Key: fas_gLoBPMrad3E__U__S14---0097.650-0102.010.wav, Target: fas, Predicted: yid +Key: fas_zZCjOs-WwKo__U__S195---1357.430-1377.010.wav, Target: fas, Predicted: aze +Key: fas_QYwCDYVxjpo__U__S68---0428.220-0432.740.wav, Target: fas, Predicted: pus +Key: fin_S_VWbBtBey4__U__S0---0308.380-0310.650.wav, Target: fin, Predicted: glv +Key: fra_SLfpp704KI8__U__S57---0368.470-0372.910.wav, Target: fra, Predicted: rus +Key: fra_Lo_JX-8KHEw__U__S151---0284.430-0299.020.wav, Target: fra, Predicted: lin +Key: hrv_Jntmbw5_vOI__U__S291---0379.300-0383.970.wav, Target: hrv, Predicted: srp +Key: fra_jjEvNgbuptE__U__S103---0990.080-0997.340.wav, Target: fra, Predicted: hat +Key: hye_PcLE4N63O9M__U__S352---2333.340-2337.540.wav, Target: hye, Predicted: yid +Key: hye_Qmo3P38Ytek__U__S32---0245.460-0249.320.wav, Target: hye, Predicted: jav +Key: hye_qkMM0rYsa0c__U__S276---1611.690-1615.350.wav, Target: hye, Predicted: sqi +Key: hye_um6xT5Gjgus__U__S194---1224.460-1234.130.wav, Target: hye, Predicted: lat +Key: jpn_rQPhM6wNQwc__U__S47---0317.270-0323.120.wav, Target: jpn, Predicted: est +Key: lit_3svAywrL0_I__U__S149---0461.370-0464.980.wav, Target: lit, Predicted: por +Key: nld_0LhAXOxz-JU__U__S32---0243.280-0247.880.wav, Target: nld, Predicted: afr +Key: nld_0LhAXOxz-JU__U__S396---2475.670-2488.950.wav, Target: nld, Predicted: afr +Key: nld_2C5HehL-Fx0__U__S101---1125.890-1131.720.wav, Target: nld, Predicted: ltz +Key: lav_DWPBBIdz0Mo__U__S52---0339.380-0356.600.wav, Target: lav, Predicted: ukr +Key: nld_QflBX7-rF9c__U__S106---0919.840-0926.610.wav, Target: nld, Predicted: afr +Key: nor_HW_49WuFloM__U__S106---0621.590-0626.130.wav, Target: nor, Predicted: nno +Key: nor_I1vUI8va8Yc__U__S49---0294.940-0302.560.wav, Target: nor, Predicted: nno +Key: nld_7AZTxaq_37U__U__S29---0226.530-0237.250.wav, Target: nld, Predicted: afr +Key: nor_UxHL_uql05E__U__S118---0587.340-0598.790.wav, Target: nor, Predicted: nno +Key: nor_XC4Ffj9XDls__U__S105---0636.770-0655.460.wav, Target: nor, Predicted: nno +Key: nor_tV3Le8SUz_0__U__S276---1831.870-1841.550.wav, Target: nor, Predicted: nno +Key: nor_xVNA15ifyIw__U__S494---0311.220-0317.160.wav, Target: nor, Predicted: nno +Key: nor_ySVkmT8SgNM__U__S345---2245.790-2255.930.wav, Target: nor, Predicted: nno +Key: nor_0eQvHBz2Zb0__U__S0---0000.000-0018.430.wav, Target: nor, Predicted: nno +Key: nor_1KFP5wVtthQ__U__S130---0511.650-0521.160.wav, Target: nor, Predicted: nno +Key: nor_41P9Uue3YbQ__U__S38---0255.810-0264.280.wav, Target: nor, Predicted: nno +Key: nor_97e9pEtHAxg__U__S32---0201.830-0210.250.wav, Target: nor, Predicted: nno +Key: spa_BApoyHcbdls__U__S286---1705.860-1722.550.wav, Target: spa, Predicted: ast +Key: swe_CizHFWTDSnU__U__S113---0867.420-0878.560.wav, Target: swe, Predicted: nor +Key: spa_z5b-CjOOhK8__U__S251---1701.420-1708.280.wav, Target: spa, Predicted: glg +Key: srp_8dvIaAOLlGA__U__S216---1326.410-1335.490.wav, Target: srp, Predicted: hrv +Key: spa_UYBcNrx8kvQ__U__S186---2292.670-2299.590.wav, Target: spa, Predicted: kor +Key: srp_rkQhxxO5Qt4__U__S109---0820.610-0826.190.wav, Target: srp, Predicted: bos +Key: spa_Y0mzNQqBR3A__U__S151---1160.320-1166.480.wav, Target: spa, Predicted: glg +Key: swe_0x4xb4AaTy0__U__S0---0301.450-0319.780.wav, Target: swe, Predicted: nno +Key: tur_4C-efpD-DlM__U__S7---0050.890-0055.080.wav, Target: tur, Predicted: war +Key: swe_ilhngbAuxvs__U__S14---2441.740-2445.720.wav, Target: swe, Predicted: nno +Key: swe_wMAAiJhj0VA__U__S100---0564.840-0568.420.wav, Target: swe, Predicted: nno +Key: urd_o3awRytwrUY__U__S1---0290.850-0306.430.wav, Target: urd, Predicted: fas +Key: urd_pYId2x4cutY__U__S151---1539.990-1542.820.wav, Target: urd, Predicted: hin +Key: urd_J7RizO2mvm4__U__S3---0042.600-0051.180.wav, Target: urd, Predicted: san +Key: urd_ySjOb5uaA-U__U__S107---0336.690-0353.110.wav, Target: urd, Predicted: cym +Key: urd_N59t4A1mxfA__U__S101---0715.390-0720.460.wav, Target: urd, Predicted: snd +Key: urd_Tj2pngm_vuA__U__S1---0070.400-0089.660.wav, Target: urd, Predicted: hin +Key: urd_U_h8Bgywxrc__U__S0---0222.420-0227.610.wav, Target: urd, Predicted: hin +Key: urd_eTfyAm6CFB0__U__S25---0439.860-0446.680.wav, Target: urd, Predicted: hin +Key: urd_8VDrhDx37OA__U__S12---0094.320-0106.080.wav, Target: urd, Predicted: hin +Key: urd_n3l7PavcOFk__U__S0---0379.380-0397.930.wav, Target: urd, Predicted: hin diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang/lid_inference_test.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang/lid_inference_test.log new file mode 100644 index 0000000000000000000000000000000000000000..50cd6f0fdf3ce24eeb39fc1df26c6c160ea0aa61 --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang/lid_inference_test.log @@ -0,0 +1,356 @@ +# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +# Started at Mon Jun 2 00:54:21 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_fleurs_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +[gpue04] 2025-06-02 00:54:40,331 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + torch.load(model_file, map_location=device), +[gpue04] 2025-06-02 00:54:51,981 (lid_inference_dist:86) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary. + warnings.warn(_create_warning_msg( +/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist. + warnings.warn( +[gpue04] 2025-06-02 00:54:52,516 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang/lids0 +[gpue04] 2025-06-02 00:55:50,135 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0 +[gpue04] 2025-06-02 00:56:51,199 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1 +[gpue04] 2025-06-02 00:57:39,967 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2 +[gpue04] 2025-06-02 00:58:38,305 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3 +[gpue04] 2025-06-02 00:59:50,779 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4 +[gpue04] 2025-06-02 01:00:54,111 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5 +[gpue04] 2025-06-02 01:01:48,017 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6 +[gpue04] 2025-06-02 01:02:41,062 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7 +[gpue04] 2025-06-02 01:03:46,378 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8 +[gpue04] 2025-06-02 01:04:44,682 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9 +[gpue04] 2025-06-02 01:05:39,283 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10 +[gpue04] 2025-06-02 01:06:43,893 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11 +[gpue04] 2025-06-02 01:07:51,015 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12 +[gpue04] 2025-06-02 01:08:54,020 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13 +[gpue04] 2025-06-02 01:09:52,678 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14 +[gpue04] 2025-06-02 01:10:43,497 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15 +[gpue04] 2025-06-02 01:11:46,070 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 16 +[gpue04] 2025-06-02 01:13:04,592 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 17 +[gpue04] 2025-06-02 01:14:19,368 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 18 +[gpue04] 2025-06-02 01:15:13,278 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 19 +[gpue04] 2025-06-02 01:16:21,130 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 20 +[gpue04] 2025-06-02 01:17:22,122 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 21 +[gpue04] 2025-06-02 01:18:11,762 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 22 +[gpue04] 2025-06-02 01:19:30,749 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 23 +[gpue04] 2025-06-02 01:20:24,887 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 24 +[gpue04] 2025-06-02 01:21:15,131 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 25 +[gpue04] 2025-06-02 01:22:09,711 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 26 +[gpue04] 2025-06-02 01:23:05,444 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 27 +[gpue04] 2025-06-02 01:24:41,530 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 28 +[gpue04] 2025-06-02 01:25:45,696 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 29 +[gpue04] 2025-06-02 01:26:55,280 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 30 +[gpue04] 2025-06-02 01:27:56,405 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 31 +[gpue04] 2025-06-02 01:29:15,643 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 32 +[gpue04] 2025-06-02 01:30:14,980 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 33 +[gpue04] 2025-06-02 01:31:18,218 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 34 +[gpue04] 2025-06-02 01:32:29,009 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 35 +[gpue04] 2025-06-02 01:33:37,784 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 36 +[gpue04] 2025-06-02 01:34:36,080 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 37 +[gpue04] 2025-06-02 01:35:38,726 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 38 +[gpue04] 2025-06-02 01:36:47,523 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 39 +[gpue04] 2025-06-02 01:37:52,504 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 40 +[gpue04] 2025-06-02 01:38:45,251 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 41 +[gpue04] 2025-06-02 01:40:00,452 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 42 +[gpue04] 2025-06-02 01:41:11,257 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 43 +[gpue04] 2025-06-02 01:42:19,878 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 44 +[gpue04] 2025-06-02 01:43:18,828 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 45 +[gpue04] 2025-06-02 01:44:21,603 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 46 +[gpue04] 2025-06-02 01:45:16,599 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 47 +[gpue04] 2025-06-02 01:47:00,696 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 48 +[gpue04] 2025-06-02 01:48:00,085 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 49 +[gpue04] 2025-06-02 01:49:05,199 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 50 +[gpue04] 2025-06-02 01:49:59,112 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 51 +[gpue04] 2025-06-02 01:51:30,912 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 52 +[gpue04] 2025-06-02 01:52:53,232 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 53 +[gpue04] 2025-06-02 01:54:03,035 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 54 +[gpue04] 2025-06-02 01:55:01,946 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 55 +[gpue04] 2025-06-02 01:55:51,439 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 56 +[gpue04] 2025-06-02 01:56:50,653 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 57 +[gpue04] 2025-06-02 01:57:42,745 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 58 +[gpue04] 2025-06-02 01:58:36,260 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 59 +[gpue04] 2025-06-02 01:59:29,171 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 60 +[gpue04] 2025-06-02 02:00:27,515 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 61 +[gpue04] 2025-06-02 02:01:31,325 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 62 +[gpue04] 2025-06-02 02:02:34,747 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 63 +[gpue04] 2025-06-02 02:03:36,231 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 64 +[gpue04] 2025-06-02 02:04:29,492 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 65 +[gpue04] 2025-06-02 02:05:28,729 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 66 +[gpue04] 2025-06-02 02:06:31,995 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 67 +[gpue04] 2025-06-02 02:07:34,329 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 68 +[gpue04] 2025-06-02 02:08:36,453 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 69 +[gpue04] 2025-06-02 02:09:32,760 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 70 +[gpue04] 2025-06-02 02:11:06,537 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 71 +[gpue04] 2025-06-02 02:12:03,962 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 72 +[gpue04] 2025-06-02 02:13:08,461 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 73 +[gpue04] 2025-06-02 02:14:18,704 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 74 +[gpue04] 2025-06-02 02:15:32,453 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 75 +[gpue04] 2025-06-02 02:16:33,681 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 76 +[gpue04] 2025-06-02 02:17:40,005 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True +[gpue04] 2025-06-02 02:17:40,006 (lid_inference_dist:215) INFO: args.save_tsne_plot: False +# Accounting: time=5000 threads=1 +# Ended (code 0) at Mon Jun 2 02:17:41 CDT 2025, elapsed time 5000 seconds diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang/results b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang/results new file mode 100644 index 0000000000000000000000000000000000000000..9aea15480ae1e0c216f95e8bee411486bfe90e29 --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_fleurs_lang_cross_train_all_no_filter_lang/results @@ -0,0 +1,1873 @@ +Accuracy: 97.73% +Macro Accuracy: 97.63% +Accuracy per Language: +nep: 100.00% +aze: 99.67% +msa: 94.79% +ltz: 100.00% +tam: 98.31% +rus: 99.48% +nld: 99.73% +orm: 97.56% +deu: 100.00% +yue: 100.00% +kor: 100.00% +ell: 99.69% +dan: 99.89% +ara: 99.77% +ckb: 95.01% +slv: 99.88% +ibo: 98.56% +uzb: 99.19% +por: 100.00% +heb: 91.04% +kaz: 100.00% +swa: 100.00% +umb: 96.57% +ron: 99.89% +oci: 86.17% +mkd: 99.90% +guj: 99.50% +fin: 100.00% +kat: 100.00% +cat: 100.00% +fil: 99.90% +lav: 100.00% +bul: 100.00% +ast: 77.17% +slk: 98.99% +luo: 100.00% +afr: 99.62% +ceb: 97.41% +mri: 100.00% +spa: 100.00% +bel: 99.69% +hin: 87.08% +ukr: 99.73% +snd: 98.47% +isl: 100.00% +est: 100.00% +nob: 99.44% +cym: 99.41% +mar: 98.52% +pan: 98.08% +tgk: 98.33% +mlt: 100.00% +kea: 87.38% +lit: 100.00% +som: 99.51% +khm: 99.35% +cmn: 100.00% +ful: 77.73% +fas: 100.00% +nso: 98.99% +zul: 85.25% +bos: 96.65% +ces: 99.72% +eng: 100.00% +yor: 100.00% +hye: 100.00% +sna: 99.14% +ita: 100.00% +hau: 100.00% +jpn: 100.00% +pus: 99.61% +hrv: 91.36% +nya: 95.01% +ben: 100.00% +kir: 99.80% +tur: 100.00% +kam: 97.58% +tha: 99.80% +mon: 99.89% +amh: 100.00% +gle: 99.88% +vie: 100.00% +lin: 99.16% +mya: 99.20% +mal: 99.37% +kan: 99.40% +fra: 99.85% +jav: 96.02% +ori: 92.53% +hun: 99.45% +wol: 98.38% +swe: 99.47% +glg: 94.17% +pol: 100.00% +lug: 100.00% +lao: 93.33% +ind: 93.45% +tel: 99.36% +urd: 82.61% +xho: 98.66% +asm: 99.29% +srp: 81.43% +Key: afr_7982535918047944297, Target: afr, Predicted: nld +Key: ara_4248701110553593570, Target: ara, Predicted: sot +Key: asm_10610964075898474870, Target: asm, Predicted: ori +Key: asm_10594445702423021926, Target: asm, Predicted: ori +Key: asm_17734366941114381002, Target: asm, Predicted: ori +Key: asm_18372175715051087641, Target: asm, Predicted: ori +Key: asm_3152616185148212383, Target: asm, Predicted: ori +Key: asm_3217319002071130963, Target: asm, Predicted: ori +Key: asm_8545995189392304523, Target: asm, Predicted: ori +Key: ast_1018962528012367685, Target: ast, Predicted: spa +Key: ast_1019107194882278208, Target: ast, Predicted: spa +Key: ast_10202712298025475218, Target: ast, Predicted: spa +Key: ast_11526208785861614719, Target: ast, Predicted: spa +Key: ast_10234036691472934781, Target: ast, Predicted: spa +Key: ast_10852707558460862636, Target: ast, Predicted: spa +Key: ast_10287015492181877949, Target: ast, Predicted: spa +Key: ast_1030467751132371693, Target: ast, Predicted: spa +Key: ast_11640777268739912106, Target: ast, Predicted: spa +Key: ast_11004200915550690640, Target: ast, Predicted: spa +Key: ast_11686163491492610952, Target: ast, Predicted: spa +Key: ast_10419946266107677131, Target: ast, Predicted: spa +Key: ast_10010711904591060483, Target: ast, Predicted: spa +Key: ast_1002728215751471177, Target: ast, Predicted: spa +Key: ast_11770035823153823340, Target: ast, Predicted: spa +Key: ast_11141033937495123569, Target: ast, Predicted: spa +Key: ast_10113368220406117046, Target: ast, Predicted: spa +Key: ast_10123298053184808351, Target: ast, Predicted: spa +Key: ast_10588548453555278275, Target: ast, Predicted: spa +Key: ast_11865273780381513475, Target: ast, Predicted: spa +Key: ast_10184950092327794810, Target: ast, Predicted: spa +Key: ast_10679411619380069139, Target: ast, Predicted: spa +Key: ast_13240740072525227893, Target: ast, Predicted: spa +Key: ast_11938245305466612810, Target: ast, Predicted: spa +Key: ast_13811678400222347420, Target: ast, Predicted: spa +Key: ast_11950028063763346486, Target: ast, Predicted: spa +Key: ast_12527012029957838825, Target: ast, Predicted: spa +Key: ast_13269475820156350146, Target: ast, Predicted: spa +Key: ast_13874215827016371737, Target: ast, Predicted: spa +Key: ast_13953686875691913119, Target: ast, Predicted: spa +Key: ast_12089977366576225376, Target: ast, Predicted: spa +Key: ast_12646579750185781031, Target: ast, Predicted: spa +Key: ast_12733188990716589063, Target: ast, Predicted: spa +Key: ast_14043457672312978504, Target: ast, Predicted: spa +Key: ast_12739685933258384483, Target: ast, Predicted: spa +Key: ast_12264876260622584640, Target: ast, Predicted: spa +Key: ast_12282600958542769312, Target: ast, Predicted: spa +Key: ast_12925375800247088793, Target: ast, Predicted: spa +Key: ast_12932732947491320249, Target: ast, Predicted: spa +Key: ast_12377966625775160502, Target: ast, Predicted: spa +Key: ast_13607665936351396298, Target: ast, Predicted: spa +Key: ast_13016707880902980323, Target: ast, Predicted: spa +Key: ast_14195679264455408821, Target: ast, Predicted: spa +Key: ast_13063101824057149409, Target: ast, Predicted: spa +Key: ast_1428684595077736903, Target: ast, Predicted: spa +Key: ast_1246985496355482490, Target: ast, Predicted: spa +Key: ast_12484458631646684185, Target: ast, Predicted: spa +Key: ast_13233902383083779509, Target: ast, Predicted: spa +Key: ast_15637938871149925385, Target: ast, Predicted: spa +Key: ast_1616949886657507690, Target: ast, Predicted: spa +Key: ast_15008828469944934815, Target: ast, Predicted: spa +Key: ast_16188321270605169475, Target: ast, Predicted: spa +Key: ast_15680426433096932409, Target: ast, Predicted: spa +Key: ast_15685793255023012690, Target: ast, Predicted: spa +Key: ast_16279048307879284750, Target: ast, Predicted: spa +Key: ast_15798524602206825183, Target: ast, Predicted: spa +Key: ast_15807950727360964423, Target: ast, Predicted: spa +Key: ast_15813890969782657848, Target: ast, Predicted: spa +Key: ast_1637339659733354564, Target: ast, Predicted: spa +Key: ast_14670089046544945568, Target: ast, Predicted: spa +Key: ast_14673822690404785463, Target: ast, Predicted: spa +Key: ast_15357328460824151093, Target: ast, Predicted: spa +Key: ast_16449236385482858052, Target: ast, Predicted: spa +Key: ast_1477082376082928689, Target: ast, Predicted: spa +Key: ast_15391431976958494191, Target: ast, Predicted: spa +Key: ast_1539560939846228713, Target: ast, Predicted: spa +Key: ast_16030233523664889759, Target: ast, Predicted: spa +Key: ast_16529149892876666018, Target: ast, Predicted: spa +Key: ast_16530033977790565388, Target: ast, Predicted: eus +Key: ast_1550541904457477843, Target: ast, Predicted: spa +Key: ast_1486804658019814025, Target: ast, Predicted: spa +Key: ast_16625489979955327449, Target: ast, Predicted: spa +Key: ast_14908228525615533112, Target: ast, Predicted: spa +Key: ast_16156752581103778457, Target: ast, Predicted: spa +Key: ast_17137152298987659765, Target: ast, Predicted: spa +Key: ast_18128362033723500992, Target: ast, Predicted: spa +Key: ast_17158911093444883388, Target: ast, Predicted: spa +Key: ast_17627283243344911744, Target: ast, Predicted: spa +Key: ast_17206258140033552209, Target: ast, Predicted: spa +Key: ast_16689150398796785392, Target: ast, Predicted: spa +Key: ast_1818431872956058035, Target: ast, Predicted: spa +Key: ast_16722455103823570601, Target: ast, Predicted: spa +Key: ast_16729676787531914693, Target: ast, Predicted: spa +Key: ast_17259525704741365458, Target: ast, Predicted: spa +Key: ast_1772438502276679892, Target: ast, Predicted: spa +Key: ast_18238042878597519475, Target: ast, Predicted: spa +Key: ast_17735806686664724186, Target: ast, Predicted: spa +Key: ast_18242776816594767554, Target: ast, Predicted: spa +Key: ast_17275689331140189805, Target: ast, Predicted: spa +Key: ast_17772339297013499902, Target: ast, Predicted: spa +Key: ast_16930568415474127672, Target: ast, Predicted: spa +Key: ast_17294516128009650397, Target: ast, Predicted: spa +Key: ast_17798797192544155707, Target: ast, Predicted: spa +Key: ast_18313634720350775414, Target: ast, Predicted: spa +Key: ast_16962886775113810085, Target: ast, Predicted: spa +Key: ast_17359174374714859988, Target: ast, Predicted: spa +Key: ast_17838959014946263711, Target: ast, Predicted: spa +Key: ast_17888845600093549735, Target: ast, Predicted: spa +Key: ast_17000639106832872192, Target: ast, Predicted: spa +Key: ast_18436487712087047910, Target: ast, Predicted: spa +Key: ast_17002464845795073174, Target: ast, Predicted: spa +Key: ast_17948813610048627962, Target: ast, Predicted: spa +Key: ast_18010580545774200532, Target: ast, Predicted: spa +Key: ast_17500126836557716031, Target: ast, Predicted: spa +Key: ast_18066446318044545927, Target: ast, Predicted: spa +Key: ast_17538164375287207389, Target: ast, Predicted: spa +Key: ast_17549224026112129606, Target: ast, Predicted: spa +Key: ast_18087191698794044331, Target: ast, Predicted: spa +Key: ast_17136233476700623613, Target: ast, Predicted: spa +Key: ast_216125073722114286, Target: ast, Predicted: spa +Key: ast_2700544236962647514, Target: ast, Predicted: spa +Key: ast_2701911583781509419, Target: ast, Predicted: spa +Key: ast_3670169385366412979, Target: ast, Predicted: spa +Key: ast_274397472356700019, Target: ast, Predicted: spa +Key: ast_3332807774799654504, Target: ast, Predicted: spa +Key: ast_3357567160853355172, Target: ast, Predicted: spa +Key: ast_3728255539752411591, Target: ast, Predicted: spa +Key: ast_2283688304016294545, Target: ast, Predicted: spa +Key: ast_2857792129439058806, Target: ast, Predicted: spa +Key: ast_2971187738997891286, Target: ast, Predicted: spa +Key: ast_2976096987158896340, Target: ast, Predicted: spa +Key: ast_3857221991894836000, Target: ast, Predicted: spa +Key: ast_3863262372982435665, Target: ast, Predicted: spa +Key: ast_2416894401152663032, Target: ast, Predicted: spa +Key: ast_3901134422180676035, Target: ast, Predicted: spa +Key: ast_3502321205348404230, Target: ast, Predicted: spa +Key: ast_3918825665352398033, Target: ast, Predicted: spa +Key: ast_3032083893966284612, Target: ast, Predicted: spa +Key: ast_3534854014197257869, Target: ast, Predicted: spa +Key: ast_392962887499938727, Target: ast, Predicted: spa +Key: ast_3057950159965539843, Target: ast, Predicted: spa +Key: ast_3972551253015772670, Target: ast, Predicted: spa +Key: ast_3992725568508510038, Target: ast, Predicted: spa +Key: ast_3074459910866817385, Target: ast, Predicted: eus +Key: ast_3580836334624498097, Target: ast, Predicted: spa +Key: ast_2494737230967707701, Target: ast, Predicted: spa +Key: ast_3634702834984163521, Target: ast, Predicted: spa +Key: ast_2581560775183107823, Target: ast, Predicted: spa +Key: ast_5223969042787943365, Target: ast, Predicted: spa +Key: ast_5793008677135877408, Target: ast, Predicted: spa +Key: ast_4808419619312810324, Target: ast, Predicted: spa +Key: ast_5262443543137554944, Target: ast, Predicted: spa +Key: ast_5861552125306403386, Target: ast, Predicted: spa +Key: ast_4238784075004895726, Target: ast, Predicted: spa +Key: ast_4249022278227973181, Target: ast, Predicted: spa +Key: ast_5296980325519572032, Target: ast, Predicted: spa +Key: ast_425002284299112287, Target: ast, Predicted: glg +Key: ast_482775445108860550, Target: ast, Predicted: spa +Key: ast_5343455351402474807, Target: ast, Predicted: spa +Key: ast_430329744400795500, Target: ast, Predicted: spa +Key: ast_4924756937972295046, Target: ast, Predicted: spa +Key: ast_493625677855470598, Target: ast, Predicted: spa +Key: ast_4326891718581888736, Target: ast, Predicted: spa +Key: ast_4976214020639368802, Target: ast, Predicted: spa +Key: ast_4430081042485077818, Target: ast, Predicted: spa +Key: ast_4438052146961175930, Target: ast, Predicted: spa +Key: ast_5019300531152068159, Target: ast, Predicted: spa +Key: ast_4452770839038261579, Target: ast, Predicted: spa +Key: ast_4458150862033042637, Target: ast, Predicted: spa +Key: ast_4491761427541207825, Target: ast, Predicted: spa +Key: ast_5602068118823733343, Target: ast, Predicted: spa +Key: ast_6188295134313505065, Target: ast, Predicted: spa +Key: ast_4594892739423532075, Target: ast, Predicted: spa +Key: ast_5106935876017614404, Target: ast, Predicted: spa +Key: ast_5630866046407443270, Target: ast, Predicted: spa +Key: ast_618957922031476859, Target: ast, Predicted: spa +Key: ast_4654203812410850331, Target: ast, Predicted: spa +Key: ast_5114413299760706177, Target: ast, Predicted: spa +Key: ast_6223714592209899333, Target: ast, Predicted: spa +Key: ast_4654986751786320155, Target: ast, Predicted: spa +Key: ast_5728348551960058, Target: ast, Predicted: spa +Key: ast_5125223319914481245, Target: ast, Predicted: spa +Key: ast_5744731363334460729, Target: ast, Predicted: spa +Key: ast_8189907359104756854, Target: ast, Predicted: spa +Key: ast_7435163034347123884, Target: ast, Predicted: spa +Key: ast_8245941708105789636, Target: ast, Predicted: spa +Key: ast_7468054803987655365, Target: ast, Predicted: spa +Key: ast_829085619273033672, Target: ast, Predicted: spa +Key: ast_6422143269452202702, Target: ast, Predicted: spa +Key: ast_6913097427947263325, Target: ast, Predicted: spa +Key: ast_8367724236657932408, Target: ast, Predicted: spa +Key: ast_8487287252253309295, Target: ast, Predicted: spa +Key: ast_648875788430739268, Target: ast, Predicted: spa +Key: ast_698680811928529185, Target: ast, Predicted: spa +Key: ast_852225345944180798, Target: ast, Predicted: spa +Key: ast_7016945322414244246, Target: ast, Predicted: spa +Key: ast_7057111414855871603, Target: ast, Predicted: spa +Key: ast_8596286617290190057, Target: ast, Predicted: spa +Key: ast_7824181966705875106, Target: ast, Predicted: spa +Key: ast_8605886612406719945, Target: ast, Predicted: spa +Key: ast_710707614293310103, Target: ast, Predicted: spa +Key: ast_7163885068575959377, Target: ast, Predicted: spa +Key: ast_8655731334769351548, Target: ast, Predicted: spa +Key: ast_7216354341094331888, Target: ast, Predicted: spa +Key: ast_6693764780157024293, Target: ast, Predicted: spa +Key: ast_7959317035960124377, Target: ast, Predicted: spa +Key: ast_670326381621986190, Target: ast, Predicted: spa +Key: ast_8045248845368197613, Target: ast, Predicted: spa +Key: ast_7327697862180243602, Target: ast, Predicted: spa +Key: ast_8875043799537191903, Target: ast, Predicted: spa +Key: ast_7390093576848692469, Target: ast, Predicted: spa +Key: ast_8972400375322197005, Target: ast, Predicted: spa +Key: ast_9451719700710917978, Target: ast, Predicted: spa +Key: ast_9494484340332243714, Target: ast, Predicted: spa +Key: ast_9498159537645478442, Target: ast, Predicted: spa +Key: ast_9598462533436620253, Target: ast, Predicted: spa +Key: ast_9126289205089059912, Target: ast, Predicted: spa +Key: ast_9645871248489359366, Target: ast, Predicted: spa +Key: ast_9646299472670999121, Target: ast, Predicted: spa +Key: ast_9269030913469067700, Target: ast, Predicted: spa +Key: ast_9284662952512508333, Target: ast, Predicted: spa +Key: ast_9324876896562192529, Target: ast, Predicted: spa +Key: ast_9817731539084593690, Target: ast, Predicted: spa +Key: ast_935290068168582899, Target: ast, Predicted: spa +Key: ast_9411057423719919622, Target: ast, Predicted: spa +Key: ast_9913159799489334115, Target: ast, Predicted: spa +Key: aze_12821706862722561615, Target: aze, Predicted: tat +Key: aze_16916012245145320888, Target: aze, Predicted: tat +Key: aze_5689852942542504900, Target: aze, Predicted: tur +Key: bel_10487171223553496488, Target: bel, Predicted: rus +Key: bel_11282679222756815581, Target: bel, Predicted: rus +Key: bel_17876450805825628905, Target: bel, Predicted: rus +Key: bos_10330562608842463163, Target: bos, Predicted: hrv +Key: bos_10421899820102230922, Target: bos, Predicted: srp +Key: bos_10500099961107683689, Target: bos, Predicted: hrv +Key: bos_10030716725928261620, Target: bos, Predicted: srp +Key: bos_10038822400549245346, Target: bos, Predicted: hrv +Key: bos_12912945488247038585, Target: bos, Predicted: slv +Key: bos_12463762693957779670, Target: bos, Predicted: hrv +Key: bos_11956746104643988774, Target: bos, Predicted: ita +Key: bos_11528950539857665976, Target: bos, Predicted: hrv +Key: bos_12516642678295088376, Target: bos, Predicted: hrv +Key: bos_13173812582982909363, Target: bos, Predicted: hrv +Key: bos_13998744396655792964, Target: bos, Predicted: srp +Key: bos_14070025951887481544, Target: bos, Predicted: srp +Key: bos_14258320709805737333, Target: bos, Predicted: hrv +Key: bos_15512330491873513078, Target: bos, Predicted: hrv +Key: bos_14380996718476558513, Target: bos, Predicted: srp +Key: bos_1564593743264235112, Target: bos, Predicted: hrv +Key: bos_3262509684161753227, Target: bos, Predicted: hrv +Key: bos_1973601556909399752, Target: bos, Predicted: hrv +Key: bos_4870458705545499905, Target: bos, Predicted: srp +Key: bos_4167116755166940544, Target: bos, Predicted: srp +Key: bos_4174123618452137059, Target: bos, Predicted: srp +Key: bos_6761174397718209280, Target: bos, Predicted: slv +Key: bos_61798271896204697, Target: bos, Predicted: hrv +Key: bos_6281200359639755421, Target: bos, Predicted: ita +Key: bos_7464050026600935328, Target: bos, Predicted: hrv +Key: bos_635890154940116046, Target: bos, Predicted: srp +Key: bos_8125677272143640987, Target: bos, Predicted: slv +Key: bos_8125756581807901719, Target: bos, Predicted: slv +Key: bos_9665746133192718405, Target: bos, Predicted: hrv +Key: bos_9674125442388487349, Target: bos, Predicted: hrv +Key: ceb_11331439916630590551, Target: ceb, Predicted: fil +Key: ceb_10467912577691316414, Target: ceb, Predicted: tgl +Key: ceb_1118506507770020922, Target: ceb, Predicted: fil +Key: ceb_14180038138770181258, Target: ceb, Predicted: fil +Key: ceb_14399398330249951555, Target: ceb, Predicted: tgl +Key: ceb_1350418350116818273, Target: ceb, Predicted: war +Key: ceb_14414858922834741515, Target: ceb, Predicted: fil +Key: ceb_16168383003822803151, Target: ceb, Predicted: fil +Key: ceb_1633149976535972032, Target: ceb, Predicted: tgl +Key: ceb_3330003259757267088, Target: ceb, Predicted: tgl +Key: ceb_2572968374542489723, Target: ceb, Predicted: cnh +Key: ceb_2745288379882417330, Target: ceb, Predicted: tgl +Key: ceb_3096372337481636502, Target: ceb, Predicted: fil +Key: ceb_9908484136679533411, Target: ceb, Predicted: fil +Key: ces_16451498877842569126, Target: ces, Predicted: slk +Key: ces_15946562629838753884, Target: ces, Predicted: pol +Key: ckb_10144595084259792181, Target: ckb, Predicted: pus +Key: ckb_11330819624082334253, Target: ckb, Predicted: som +Key: ckb_12274128520956563282, Target: ckb, Predicted: pus +Key: ckb_13684874662065321786, Target: ckb, Predicted: pus +Key: ckb_16680370263319652147, Target: ckb, Predicted: pus +Key: ckb_15491290298532450450, Target: ckb, Predicted: pus +Key: ckb_149520202348044675, Target: ckb, Predicted: pus +Key: ckb_16433586607656826790, Target: ckb, Predicted: pus +Key: ckb_16548474187615755174, Target: ckb, Predicted: pus +Key: ckb_15977972277501545730, Target: ckb, Predicted: pus +Key: ckb_15324837629503970431, Target: ckb, Predicted: pus +Key: ckb_17403155250828649377, Target: ckb, Predicted: pus +Key: ckb_2244545244432888224, Target: ckb, Predicted: pus +Key: ckb_1868547912377686715, Target: ckb, Predicted: pus +Key: ckb_189707448772478016, Target: ckb, Predicted: pus +Key: ckb_18120696585678645554, Target: ckb, Predicted: pus +Key: ckb_17755580648127199522, Target: ckb, Predicted: pus +Key: ckb_2207649463482329838, Target: ckb, Predicted: pus +Key: ckb_2723420875579396255, Target: ckb, Predicted: pus +Key: ckb_2220574830925696791, Target: ckb, Predicted: hau +Key: ckb_4040697415003438637, Target: ckb, Predicted: amh +Key: ckb_3583147712408853065, Target: ckb, Predicted: pus +Key: ckb_4761578612456472665, Target: ckb, Predicted: pus +Key: ckb_3075594295310710561, Target: ckb, Predicted: pus +Key: ckb_3796574327341463831, Target: ckb, Predicted: amh +Key: ckb_3118668790020962928, Target: ckb, Predicted: ara +Key: ckb_3138269233922819043, Target: ckb, Predicted: pus +Key: ckb_3869710287400359711, Target: ckb, Predicted: pus +Key: ckb_4985838077310755152, Target: ckb, Predicted: pus +Key: ckb_5005091607444483674, Target: ckb, Predicted: pus +Key: ckb_6042074390680983180, Target: ckb, Predicted: pus +Key: ckb_6509269105237502230, Target: ckb, Predicted: pus +Key: ckb_6146858345898432165, Target: ckb, Predicted: pus +Key: ckb_7702853972773404050, Target: ckb, Predicted: pus +Key: ckb_8150732029269398497, Target: ckb, Predicted: pus +Key: ckb_8284039632200411542, Target: ckb, Predicted: pus +Key: ckb_723042271417019558, Target: ckb, Predicted: pus +Key: ckb_783852304899402431, Target: ckb, Predicted: pus +Key: ckb_8463239923283908221, Target: ckb, Predicted: pus +Key: ckb_7523768958606496299, Target: ckb, Predicted: pus +Key: ckb_9154296036449342919, Target: ckb, Predicted: hau +Key: ckb_9165820465851925220, Target: ckb, Predicted: pus +Key: ckb_763568131698357931, Target: ckb, Predicted: pus +Key: ckb_8040971028223594371, Target: ckb, Predicted: pus +Key: ckb_9226131421992112692, Target: ckb, Predicted: pus +Key: ckb_9759207547878295684, Target: ckb, Predicted: pus +Key: cym_10370914806389275618, Target: cym, Predicted: nno +Key: cym_11887573463509257433, Target: cym, Predicted: isl +Key: cym_4567069782789800531, Target: cym, Predicted: sot +Key: cym_6692077130913057451, Target: cym, Predicted: gle +Key: cym_9328067215944641218, Target: cym, Predicted: yor +Key: cym_9343820633077720391, Target: cym, Predicted: eng +Key: dan_4596677789826187230, Target: dan, Predicted: sna +Key: ell_16257764472226144745, Target: ell, Predicted: mkd +Key: ell_1898715818168268352, Target: ell, Predicted: mkd +Key: fil_14348334124170623940, Target: fil, Predicted: tgl +Key: fra_3693672928808080722, Target: fra, Predicted: bre +Key: ful_1069883613624056030, Target: ful, Predicted: wol +Key: ful_10008205857778443490, Target: ful, Predicted: luo +Key: ful_10010052403644506836, Target: ful, Predicted: swa +Key: ful_10728662388228092247, Target: ful, Predicted: swa +Key: ful_10057848043538805134, Target: ful, Predicted: swa +Key: ful_10883347741238177904, Target: ful, Predicted: wol +Key: ful_10116751900427129794, Target: ful, Predicted: wol +Key: ful_1017335605310391787, Target: ful, Predicted: luo +Key: ful_10233160498752884232, Target: ful, Predicted: kea +Key: ful_10299040205592410360, Target: ful, Predicted: luo +Key: ful_10307068898405525326, Target: ful, Predicted: luo +Key: ful_10338902449846824831, Target: ful, Predicted: hau +Key: ful_11184397701490529912, Target: ful, Predicted: sna +Key: ful_1038707514777266882, Target: ful, Predicted: wol +Key: ful_1120595563920955520, Target: ful, Predicted: swa +Key: ful_10511277866116826457, Target: ful, Predicted: wol +Key: ful_10565818114448566822, Target: ful, Predicted: wol +Key: ful_10697649567120142582, Target: ful, Predicted: swa +Key: ful_12180534394279310984, Target: ful, Predicted: wol +Key: ful_12190947683209205648, Target: ful, Predicted: swa +Key: ful_13080844647399285758, Target: ful, Predicted: swa +Key: ful_13878323621374837313, Target: ful, Predicted: wol +Key: ful_12283060778324178816, Target: ful, Predicted: swa +Key: ful_12324096773046019303, Target: ful, Predicted: swa +Key: ful_13943031798362794810, Target: ful, Predicted: luo +Key: ful_12366058143139241263, Target: ful, Predicted: sot +Key: ful_12509140590418859739, Target: ful, Predicted: wol +Key: ful_13194104090270829092, Target: ful, Predicted: luo +Key: ful_1251888211126803990, Target: ful, Predicted: swa +Key: ful_11841531282393546214, Target: ful, Predicted: hau +Key: ful_13368377888759058669, Target: ful, Predicted: luo +Key: ful_12688737760137856053, Target: ful, Predicted: hau +Key: ful_13370400818842813315, Target: ful, Predicted: luo +Key: ful_12735622401147703580, Target: ful, Predicted: sna +Key: ful_13403568110110853482, Target: ful, Predicted: wol +Key: ful_11983683685970854279, Target: ful, Predicted: kea +Key: ful_12863573928073103798, Target: ful, Predicted: kea +Key: ful_13714084919861467012, Target: ful, Predicted: wol +Key: ful_14398673773358293078, Target: ful, Predicted: swa +Key: ful_13733926443819716871, Target: ful, Predicted: swa +Key: ful_14483672886607051172, Target: ful, Predicted: swa +Key: ful_13776916005401500248, Target: ful, Predicted: swa +Key: ful_12124643812808416604, Target: ful, Predicted: swa +Key: ful_13777130470312630219, Target: ful, Predicted: swa +Key: ful_14749156392021098351, Target: ful, Predicted: luo +Key: ful_15568304924207012975, Target: ful, Predicted: luo +Key: ful_16314703809802231572, Target: ful, Predicted: hau +Key: ful_15023924591683379988, Target: ful, Predicted: luo +Key: ful_16360661594932316319, Target: ful, Predicted: wol +Key: ful_15084322733444906245, Target: ful, Predicted: wol +Key: ful_17366052533615700006, Target: ful, Predicted: hau +Key: ful_17403650045777356372, Target: ful, Predicted: swa +Key: ful_15743914582243417616, Target: ful, Predicted: kea +Key: ful_17432679322947478312, Target: ful, Predicted: swa +Key: ful_1578405219715055315, Target: ful, Predicted: swa +Key: ful_16532391636870449616, Target: ful, Predicted: swa +Key: ful_16541429466645538979, Target: ful, Predicted: swa +Key: ful_15861468204473125369, Target: ful, Predicted: wol +Key: ful_15866627836525068921, Target: ful, Predicted: luo +Key: ful_16763945490709264643, Target: ful, Predicted: luo +Key: ful_17637716665194080519, Target: ful, Predicted: wol +Key: ful_15339068031181916836, Target: ful, Predicted: swa +Key: ful_17767454666793958049, Target: ful, Predicted: swa +Key: ful_15350193471710383286, Target: ful, Predicted: sna +Key: ful_16149648136976286836, Target: ful, Predicted: wol +Key: ful_17820524607692546712, Target: ful, Predicted: swa +Key: ful_17839476812677639957, Target: ful, Predicted: pus +Key: ful_15504709044787082479, Target: ful, Predicted: swa +Key: ful_178665050172769243, Target: ful, Predicted: wol +Key: ful_16272922657780214032, Target: ful, Predicted: hat +Key: ful_17967960252569092871, Target: ful, Predicted: wol +Key: ful_18077836773802791532, Target: ful, Predicted: swa +Key: ful_3095676517345586599, Target: ful, Predicted: hau +Key: ful_385976524259691765, Target: ful, Predicted: hau +Key: ful_3895192166957852345, Target: ful, Predicted: hau +Key: ful_3140261054418875173, Target: ful, Predicted: swa +Key: ful_3926789435257938584, Target: ful, Predicted: wol +Key: ful_3151808443709826548, Target: ful, Predicted: swa +Key: ful_18125010363993093079, Target: ful, Predicted: swa +Key: ful_2575946208063213224, Target: ful, Predicted: swa +Key: ful_325535860421403129, Target: ful, Predicted: swa +Key: ful_2595470844850392797, Target: ful, Predicted: swa +Key: ful_3278614539600092212, Target: ful, Predicted: swa +Key: ful_18233574780994276312, Target: ful, Predicted: swa +Key: ful_4134679492316511517, Target: ful, Predicted: hau +Key: ful_3427092603675509988, Target: ful, Predicted: swa +Key: ful_2696909974417007754, Target: ful, Predicted: swa +Key: ful_3430140446049143032, Target: ful, Predicted: wol +Key: ful_4250590839031321652, Target: ful, Predicted: hau +Key: ful_1882491294451164591, Target: ful, Predicted: wol +Key: ful_3549322577141991679, Target: ful, Predicted: wol +Key: ful_4300096864921099703, Target: ful, Predicted: kea +Key: ful_3564673223637162845, Target: ful, Predicted: swa +Key: ful_4429557818842726377, Target: ful, Predicted: swa +Key: ful_3604017402069717028, Target: ful, Predicted: hau +Key: ful_2137896798138887413, Target: ful, Predicted: swa +Key: ful_4444316329148332334, Target: ful, Predicted: wol +Key: ful_2177869389392349325, Target: ful, Predicted: swa +Key: ful_2180824654205691557, Target: ful, Predicted: swa +Key: ful_4553659885779589349, Target: ful, Predicted: swa +Key: ful_3039656246932231461, Target: ful, Predicted: wol +Key: ful_2342710256702461887, Target: ful, Predicted: orm +Key: ful_3800392056761118797, Target: ful, Predicted: swa +Key: ful_2377938479309485830, Target: ful, Predicted: wol +Key: ful_3090839150907352574, Target: ful, Predicted: wol +Key: ful_5318039739731103195, Target: ful, Predicted: swa +Key: ful_6185725096249650418, Target: ful, Predicted: wol +Key: ful_5359209129657040058, Target: ful, Predicted: hau +Key: ful_6303200995784059640, Target: ful, Predicted: swa +Key: ful_4877642018597978563, Target: ful, Predicted: swa +Key: ful_6321772164537298494, Target: ful, Predicted: hau +Key: ful_5590343225256726053, Target: ful, Predicted: swa +Key: ful_5625532354229645166, Target: ful, Predicted: swa +Key: ful_4906971416102425492, Target: ful, Predicted: luo +Key: ful_4920680982871030366, Target: ful, Predicted: wol +Key: ful_7347049053766336662, Target: ful, Predicted: swa +Key: ful_6542065097866619495, Target: ful, Predicted: sna +Key: ful_735545428666644944, Target: ful, Predicted: wol +Key: ful_5763706207736726902, Target: ful, Predicted: luo +Key: ful_6713727433704915538, Target: ful, Predicted: luo +Key: ful_507524666530537228, Target: ful, Predicted: swa +Key: ful_6059012895010773870, Target: ful, Predicted: luo +Key: ful_6072148063324122544, Target: ful, Predicted: swa +Key: ful_5169678110237700990, Target: ful, Predicted: wol +Key: ful_7656285477286215112, Target: ful, Predicted: hau +Key: ful_6835825518094330604, Target: ful, Predicted: wol +Key: ful_6134224315284225430, Target: ful, Predicted: swa +Key: ful_6839931547391475896, Target: ful, Predicted: wol +Key: ful_9484986236670450766, Target: ful, Predicted: luo +Key: ful_9496101690113648110, Target: ful, Predicted: swa +Key: ful_7788603678489354973, Target: ful, Predicted: hau +Key: ful_95647032621548857, Target: ful, Predicted: snd +Key: ful_7834238156191979698, Target: ful, Predicted: luo +Key: ful_8806334423347359341, Target: ful, Predicted: swa +Key: ful_7848632951511951460, Target: ful, Predicted: luo +Key: ful_9719823493666657038, Target: ful, Predicted: swa +Key: ful_7935704314344867618, Target: ful, Predicted: hau +Key: ful_9778004675383002538, Target: ful, Predicted: wol +Key: ful_7953454598846982901, Target: ful, Predicted: swa +Key: ful_8971477126221996122, Target: ful, Predicted: hau +Key: ful_8119789574206901117, Target: ful, Predicted: swa +Key: ful_897962722940883599, Target: ful, Predicted: hau +Key: ful_9114024724214923362, Target: ful, Predicted: swa +Key: ful_830624428286569374, Target: ful, Predicted: swa +Key: ful_9925985083523729823, Target: ful, Predicted: wol +Key: ful_9355136448294698896, Target: ful, Predicted: sna +Key: ful_8568188653128781023, Target: ful, Predicted: hau +Key: gle_17948419739856900328, Target: gle, Predicted: glv +Key: glg_11371645286619558148, Target: glg, Predicted: ast +Key: glg_11537132245234629420, Target: glg, Predicted: ast +Key: glg_11573851234273661440, Target: glg, Predicted: ast +Key: glg_11022131445177976511, Target: glg, Predicted: ast +Key: glg_11820080635742761312, Target: glg, Predicted: ast +Key: glg_1118775277613247576, Target: glg, Predicted: ast +Key: glg_13784972180304651143, Target: glg, Predicted: ast +Key: glg_12533991928134231934, Target: glg, Predicted: ast +Key: glg_12565959852693044920, Target: glg, Predicted: ast +Key: glg_13948615213732699845, Target: glg, Predicted: ast +Key: glg_12336936574644650537, Target: glg, Predicted: spa +Key: glg_12356564766916224355, Target: glg, Predicted: ast +Key: glg_12363172192231689701, Target: glg, Predicted: ast +Key: glg_1281168654846638070, Target: glg, Predicted: ast +Key: glg_12911436205700792134, Target: glg, Predicted: ast +Key: glg_15559795314344440532, Target: glg, Predicted: ast +Key: glg_16186356900587335612, Target: glg, Predicted: ast +Key: glg_1619142105232038932, Target: glg, Predicted: ast +Key: glg_1582948845379825267, Target: glg, Predicted: ast +Key: glg_14797646809705245503, Target: glg, Predicted: ast +Key: glg_15465109309231668931, Target: glg, Predicted: ast +Key: glg_15496530840105191861, Target: glg, Predicted: ast +Key: glg_17878920102287592967, Target: glg, Predicted: ast +Key: glg_16732792652314113310, Target: glg, Predicted: ast +Key: glg_17323054448470979654, Target: glg, Predicted: ast +Key: glg_17995460094373016998, Target: glg, Predicted: ast +Key: glg_167899221096289564, Target: glg, Predicted: ast +Key: glg_2162030846518758713, Target: glg, Predicted: ast +Key: glg_2361977493745465538, Target: glg, Predicted: ast +Key: glg_2937607740591193838, Target: glg, Predicted: ast +Key: glg_3827496020928115654, Target: glg, Predicted: ast +Key: glg_3862435309506495540, Target: glg, Predicted: ast +Key: glg_2695986516182233446, Target: glg, Predicted: ast +Key: glg_3369082772537006954, Target: glg, Predicted: ast +Key: glg_3919431014155125712, Target: glg, Predicted: ast +Key: glg_3967020171372380024, Target: glg, Predicted: ast +Key: glg_4674219651689822753, Target: glg, Predicted: ast +Key: glg_3390421776195452043, Target: glg, Predicted: ast +Key: glg_6006275276723438090, Target: glg, Predicted: ast +Key: glg_6203749954443123717, Target: glg, Predicted: ast +Key: glg_5102710010742830482, Target: glg, Predicted: ast +Key: glg_6690347356763639754, Target: glg, Predicted: ast +Key: glg_5743250550688229262, Target: glg, Predicted: ast +Key: glg_8565245348813367140, Target: glg, Predicted: ast +Key: glg_7489142028495480727, Target: glg, Predicted: ast +Key: glg_7941392053675828494, Target: glg, Predicted: ast +Key: glg_8704977666846314135, Target: glg, Predicted: ast +Key: glg_7021757541772517151, Target: glg, Predicted: ast +Key: glg_7221128782829616485, Target: glg, Predicted: ast +Key: guj_10807135137550874009, Target: guj, Predicted: nep +Key: glg_9609398486904674283, Target: glg, Predicted: ast +Key: glg_91969771707586328, Target: glg, Predicted: ast +Key: glg_9768375587341914923, Target: glg, Predicted: ast +Key: glg_9356785447197647413, Target: glg, Predicted: ast +Key: glg_9901833843185839440, Target: glg, Predicted: ast +Key: guj_11334169955274565894, Target: guj, Predicted: mar +Key: guj_12380742375034994290, Target: guj, Predicted: hin +Key: guj_17159496041599387860, Target: guj, Predicted: ben +Key: guj_18176640343780558636, Target: guj, Predicted: ben +Key: heb_11526705735519681230, Target: heb, Predicted: ara +Key: heb_10903617939455240478, Target: heb, Predicted: ara +Key: heb_10405588310312483740, Target: heb, Predicted: ara +Key: heb_11778128680191506306, Target: heb, Predicted: ara +Key: heb_11820054623908755150, Target: heb, Predicted: ckb +Key: heb_11866478067948157498, Target: heb, Predicted: ara +Key: heb_1062031265143490849, Target: heb, Predicted: ara +Key: heb_10671296103375413199, Target: heb, Predicted: ara +Key: heb_12010375412324118246, Target: heb, Predicted: ara +Key: heb_12015737174680349393, Target: heb, Predicted: ara +Key: heb_10744260442384711132, Target: heb, Predicted: ara +Key: heb_14084216550246875859, Target: heb, Predicted: ara +Key: heb_12245272347551287353, Target: heb, Predicted: ckb +Key: heb_13586556662917749214, Target: heb, Predicted: ara +Key: heb_14314181287001146640, Target: heb, Predicted: ara +Key: heb_13170566870467032642, Target: heb, Predicted: ara +Key: heb_13672638088780268563, Target: heb, Predicted: ara +Key: heb_13177662097666235252, Target: heb, Predicted: ara +Key: heb_12660419218700563156, Target: heb, Predicted: ara +Key: heb_127949745835207457, Target: heb, Predicted: ara +Key: heb_14545994604144341648, Target: heb, Predicted: ara +Key: heb_13981864000952012586, Target: heb, Predicted: ara +Key: heb_15409247670048183127, Target: heb, Predicted: ara +Key: heb_16064089784871041574, Target: heb, Predicted: ara +Key: heb_16111125278981483867, Target: heb, Predicted: ara +Key: heb_16142358346089399782, Target: heb, Predicted: ara +Key: heb_15443384694759316031, Target: heb, Predicted: ara +Key: heb_16273471537402969711, Target: heb, Predicted: ara +Key: heb_15465539665409710566, Target: heb, Predicted: ara +Key: heb_15466348292603730337, Target: heb, Predicted: ara +Key: heb_14885722533385172285, Target: heb, Predicted: ara +Key: heb_14981203743281918118, Target: heb, Predicted: ara +Key: heb_15549337232523171759, Target: heb, Predicted: kir +Key: heb_16456974619127617639, Target: heb, Predicted: ara +Key: heb_15606475319623053141, Target: heb, Predicted: ara +Key: heb_16488683140757056346, Target: heb, Predicted: ara +Key: heb_17154226079752134082, Target: heb, Predicted: ara +Key: heb_16522631569528702810, Target: heb, Predicted: ara +Key: heb_15331612683969783035, Target: heb, Predicted: ara +Key: heb_17212403881791929618, Target: heb, Predicted: ara +Key: heb_1731152035784795960, Target: heb, Predicted: ara +Key: heb_2158126778855122738, Target: heb, Predicted: ara +Key: heb_17422642249099597140, Target: heb, Predicted: ara +Key: heb_18152938696901276132, Target: heb, Predicted: ara +Key: heb_2302906079471812116, Target: heb, Predicted: ara +Key: heb_18243260560414274552, Target: heb, Predicted: ara +Key: heb_17610207843197082137, Target: heb, Predicted: ara +Key: heb_17621479855053023638, Target: heb, Predicted: ara +Key: heb_3598140641113031108, Target: heb, Predicted: bre +Key: heb_292762740585221028, Target: heb, Predicted: ara +Key: heb_3110296995424734888, Target: heb, Predicted: ara +Key: heb_3745638736212203570, Target: heb, Predicted: ara +Key: heb_4143123695716347309, Target: heb, Predicted: ckb +Key: heb_5904907176764029270, Target: heb, Predicted: ara +Key: heb_5926191186410908722, Target: heb, Predicted: ara +Key: heb_4198258171289432534, Target: heb, Predicted: ara +Key: heb_5056734897370955044, Target: heb, Predicted: kmr +Key: heb_5133065852656902634, Target: heb, Predicted: ara +Key: heb_6383494371921548265, Target: heb, Predicted: ara +Key: heb_838864343441292617, Target: heb, Predicted: ara +Key: heb_7069344602374522850, Target: heb, Predicted: ara +Key: heb_6443462765208343074, Target: heb, Predicted: ara +Key: heb_7687004484777328385, Target: heb, Predicted: ara +Key: heb_8476398016443969997, Target: heb, Predicted: ara +Key: heb_6535547824916890906, Target: heb, Predicted: ara +Key: heb_7405484618743045434, Target: heb, Predicted: ara +Key: heb_8180293613733908401, Target: heb, Predicted: ara +Key: heb_7442509082881087391, Target: heb, Predicted: ara +Key: heb_8282382885607944575, Target: heb, Predicted: ara +Key: heb_9820397420785835550, Target: heb, Predicted: ara +Key: hin_11887255676786903166, Target: hin, Predicted: urd +Key: hin_10924000697500767031, Target: hin, Predicted: urd +Key: hin_11044357197343438539, Target: hin, Predicted: urd +Key: hin_12150230149527456841, Target: hin, Predicted: urd +Key: hin_12211625880636332335, Target: hin, Predicted: urd +Key: hin_11490162715923744769, Target: hin, Predicted: guj +Key: heb_9671640108187051049, Target: heb, Predicted: ara +Key: hin_10587318592827006604, Target: hin, Predicted: urd +Key: hin_12756212696346090124, Target: hin, Predicted: urd +Key: hin_10761312360331088376, Target: hin, Predicted: guj +Key: hin_12919174378536885221, Target: hin, Predicted: snd +Key: hin_14762987283976764594, Target: hin, Predicted: urd +Key: hin_16307283543540841715, Target: hin, Predicted: snd +Key: hin_1728637154424755425, Target: hin, Predicted: snd +Key: hin_16487389062733675441, Target: hin, Predicted: urd +Key: hin_1765177987567043565, Target: hin, Predicted: urd +Key: hin_16577983044171271983, Target: hin, Predicted: guj +Key: hin_13717677002215321761, Target: hin, Predicted: guj +Key: hin_16619231656724994690, Target: hin, Predicted: urd +Key: hin_15313654381899549036, Target: hin, Predicted: urd +Key: hin_17757682103941428900, Target: hin, Predicted: snd +Key: hin_15393261377731668660, Target: hin, Predicted: urd +Key: hin_15449373665420679878, Target: hin, Predicted: urd +Key: hin_17873467393828864636, Target: hin, Predicted: urd +Key: hin_1684304007849598848, Target: hin, Predicted: urd +Key: hin_2828193963807168217, Target: hin, Predicted: urd +Key: hin_5430720125226793398, Target: hin, Predicted: urd +Key: hin_3295919989030123828, Target: hin, Predicted: urd +Key: hin_5636625969852704727, Target: hin, Predicted: urd +Key: hin_1921302996542903288, Target: hin, Predicted: guj +Key: hin_5714389874445148824, Target: hin, Predicted: urd +Key: hin_5804236690028639622, Target: hin, Predicted: urd +Key: hin_3897753980006371820, Target: hin, Predicted: guj +Key: hin_4908398599945668748, Target: hin, Predicted: urd +Key: hin_5956554166903472780, Target: hin, Predicted: urd +Key: hin_2530253162481007097, Target: hin, Predicted: urd +Key: hin_3972087215456952159, Target: hin, Predicted: guj +Key: hin_5040068070207954482, Target: hin, Predicted: urd +Key: hin_4187574626528714176, Target: hin, Predicted: urd +Key: hin_6282079269312717787, Target: hin, Predicted: guj +Key: hin_5342217036255595560, Target: hin, Predicted: urd +Key: hin_6412098882560291653, Target: hin, Predicted: urd +Key: hin_7944552104612814736, Target: hin, Predicted: guj +Key: hin_9632791299638996282, Target: hin, Predicted: urd +Key: hin_9702619691119109001, Target: hin, Predicted: urd +Key: hin_6559956825323097558, Target: hin, Predicted: urd +Key: hin_6716012942964313180, Target: hin, Predicted: urd +Key: hin_6909376388382817686, Target: hin, Predicted: urd +Key: hin_7037862691881688112, Target: hin, Predicted: guj +Key: hin_7127722642159000820, Target: hin, Predicted: urd +Key: hrv_10506066134617086948, Target: hrv, Predicted: mkd +Key: hin_7251883826251297781, Target: hin, Predicted: pan +Key: hrv_1053454861311219472, Target: hrv, Predicted: srp +Key: hrv_10186456872158998415, Target: hrv, Predicted: slv +Key: hrv_105559179615123353, Target: hrv, Predicted: mkd +Key: hrv_10227456216907656893, Target: hrv, Predicted: mkd +Key: hrv_10705928635364349071, Target: hrv, Predicted: slv +Key: hin_7639321476329058229, Target: hin, Predicted: guj +Key: hin_7656274658476103913, Target: hin, Predicted: urd +Key: hin_9144832777583634056, Target: hin, Predicted: guj +Key: hin_9447740746764258022, Target: hin, Predicted: urd +Key: hrv_10297296377505222847, Target: hrv, Predicted: mkd +Key: hrv_10813675133068584356, Target: hrv, Predicted: slv +Key: hrv_1085520459777687919, Target: hrv, Predicted: mkd +Key: hrv_10930164652580537276, Target: hrv, Predicted: srp +Key: hrv_12547641930021004210, Target: hrv, Predicted: srp +Key: hrv_10952824580783203762, Target: hrv, Predicted: srp +Key: hrv_11499029374492154464, Target: hrv, Predicted: srp +Key: hrv_11074213299697201985, Target: hrv, Predicted: srp +Key: hrv_11135856911746771900, Target: hrv, Predicted: srp +Key: hrv_12118523850635033208, Target: hrv, Predicted: srp +Key: hrv_12204476988493158873, Target: hrv, Predicted: srp +Key: hrv_11197514843846504244, Target: hrv, Predicted: ces +Key: hrv_11762007684568667, Target: hrv, Predicted: mkd +Key: hrv_1221770310061202471, Target: hrv, Predicted: srp +Key: hrv_1302172271479039124, Target: hrv, Predicted: mkd +Key: hrv_14193061522016022360, Target: hrv, Predicted: mkd +Key: hrv_13721994391222764409, Target: hrv, Predicted: mkd +Key: hrv_1513504888665818327, Target: hrv, Predicted: slv +Key: hrv_13287274380842567138, Target: hrv, Predicted: srp +Key: hrv_13410735861663001031, Target: hrv, Predicted: slv +Key: hrv_14809802142515712641, Target: hrv, Predicted: srp +Key: hrv_15414124030766646367, Target: hrv, Predicted: srp +Key: hrv_15494758991862468581, Target: hrv, Predicted: slv +Key: hrv_16552994156241846995, Target: hrv, Predicted: srp +Key: hrv_16561115811453128126, Target: hrv, Predicted: srp +Key: hrv_16075777641284478178, Target: hrv, Predicted: srp +Key: hrv_17280933943755869388, Target: hrv, Predicted: srp +Key: hrv_15662724158042487590, Target: hrv, Predicted: srp +Key: hrv_16915213701258955738, Target: hrv, Predicted: srp +Key: hrv_16505611540690252094, Target: hrv, Predicted: srp +Key: hrv_2638340473944265163, Target: hrv, Predicted: slv +Key: hrv_2718581000612741001, Target: hrv, Predicted: mkd +Key: hrv_207661747843212507, Target: hrv, Predicted: srp +Key: hrv_18302444113425543535, Target: hrv, Predicted: mkd +Key: hrv_2345511353522308946, Target: hrv, Predicted: srp +Key: hrv_1847755676386151118, Target: hrv, Predicted: srp +Key: hrv_3883395355781726321, Target: hrv, Predicted: srp +Key: hrv_3956984824326419757, Target: hrv, Predicted: mkd +Key: hrv_3987642867382487152, Target: hrv, Predicted: srp +Key: hrv_2530012817058816955, Target: hrv, Predicted: mkd +Key: hrv_398972275951143559, Target: hrv, Predicted: mkd +Key: hrv_5279528420918230622, Target: hrv, Predicted: srp +Key: hrv_4829039672974037095, Target: hrv, Predicted: mkd +Key: hrv_5881888317366079455, Target: hrv, Predicted: slv +Key: hrv_4238512411061067241, Target: hrv, Predicted: srp +Key: hrv_5394555172233870083, Target: hrv, Predicted: srp +Key: hrv_6067547957870017684, Target: hrv, Predicted: srp +Key: hrv_4298347025520883992, Target: hrv, Predicted: srp +Key: hrv_6147511094056989038, Target: hrv, Predicted: srp +Key: hrv_4991619200707236289, Target: hrv, Predicted: slv +Key: hrv_6162337273299386316, Target: hrv, Predicted: mkd +Key: hrv_4704152041876625076, Target: hrv, Predicted: srp +Key: hrv_636493618555201295, Target: hrv, Predicted: srp +Key: hrv_6906791446850556130, Target: hrv, Predicted: srp +Key: hrv_7510940185402698631, Target: hrv, Predicted: slv +Key: hrv_7007106257572228975, Target: hrv, Predicted: ita +Key: hrv_6524623937139600893, Target: hrv, Predicted: srp +Key: hrv_7728186066240518698, Target: hrv, Predicted: mkd +Key: hrv_6579606374855718551, Target: hrv, Predicted: mkd +Key: hrv_6591949111465945004, Target: hrv, Predicted: mkd +Key: hrv_7241703087664263024, Target: hrv, Predicted: srp +Key: hrv_726447836442015076, Target: hrv, Predicted: slv +Key: hrv_6646901263278732055, Target: hrv, Predicted: srp +Key: hrv_8375246933842487725, Target: hrv, Predicted: srp +Key: hrv_9063988129381039413, Target: hrv, Predicted: srp +Key: hrv_9524709560473132258, Target: hrv, Predicted: srp +Key: hrv_8683732496675444103, Target: hrv, Predicted: mkd +Key: hrv_8753407309888871706, Target: hrv, Predicted: srp +Key: hrv_926585904888402809, Target: hrv, Predicted: slv +Key: hrv_9385399172776158912, Target: hrv, Predicted: slv +Key: hrv_9887450124022936319, Target: hrv, Predicted: srp +Key: hrv_8921430554541213917, Target: hrv, Predicted: srp +Key: hrv_9893232680614235550, Target: hrv, Predicted: srp +Key: hun_16649230919561228487, Target: hun, Predicted: fin +Key: hun_16752596014010476729, Target: hun, Predicted: fin +Key: hun_2986247380561805311, Target: hun, Predicted: fin +Key: hun_6070502549238100443, Target: hun, Predicted: fin +Key: hun_9067223779194167469, Target: hun, Predicted: est +Key: ibo_10807291442248727535, Target: ibo, Predicted: yor +Key: ibo_1226164590067606088, Target: ibo, Predicted: kam +Key: ibo_13391270813556393908, Target: ibo, Predicted: hau +Key: ibo_152276510245648628, Target: ibo, Predicted: hau +Key: ibo_14894593421607861948, Target: ibo, Predicted: hau +Key: ibo_16320880743682425605, Target: ibo, Predicted: nso +Key: ibo_1646976537126923633, Target: ibo, Predicted: umb +Key: ibo_2923212036803176645, Target: ibo, Predicted: hau +Key: ibo_2583405670614695069, Target: ibo, Predicted: nya +Key: ibo_4636305919294127456, Target: ibo, Predicted: hau +Key: ibo_469290930375981486, Target: ibo, Predicted: kin +Key: ibo_676764867472988170, Target: ibo, Predicted: yor +Key: ibo_7864362685521856556, Target: ibo, Predicted: sot +Key: ibo_8465635780532354531, Target: ibo, Predicted: hau +Key: ind_10426790486327755563, Target: ind, Predicted: jav +Key: ind_10582873098068662583, Target: ind, Predicted: jav +Key: ind_10695080633375632643, Target: ind, Predicted: jav +Key: ind_10718739153093062264, Target: ind, Predicted: jav +Key: ind_10030207972417551421, Target: ind, Predicted: jav +Key: ind_10041775983994836780, Target: ind, Predicted: jav +Key: ind_1004371778779730270, Target: ind, Predicted: jav +Key: ind_1126397733394949199, Target: ind, Predicted: jav +Key: ind_13457638346204774417, Target: ind, Predicted: jav +Key: ind_12910465532998892062, Target: ind, Predicted: jav +Key: ind_13485224400142889523, Target: ind, Predicted: jav +Key: ind_12410349848143255630, Target: ind, Predicted: jav +Key: ind_13694551591874502347, Target: ind, Predicted: jav +Key: ind_1310421265709013921, Target: ind, Predicted: jav +Key: ind_11625620854961261500, Target: ind, Predicted: jav +Key: ind_13129278352658383129, Target: ind, Predicted: jav +Key: ind_13187806086334138326, Target: ind, Predicted: jav +Key: ind_1603263828027297939, Target: ind, Predicted: jav +Key: ind_16269861796183595251, Target: ind, Predicted: jav +Key: ind_14833225615241196432, Target: ind, Predicted: jav +Key: ind_16329238611784672888, Target: ind, Predicted: jav +Key: ind_14372361299750817071, Target: ind, Predicted: jav +Key: ind_16722817494614540177, Target: ind, Predicted: jav +Key: ind_15887343284364560898, Target: ind, Predicted: jav +Key: ind_16003767185581738620, Target: ind, Predicted: jav +Key: ind_2364721781808478190, Target: ind, Predicted: jav +Key: ind_2387219639567528552, Target: ind, Predicted: kir +Key: ind_2708341615308237219, Target: ind, Predicted: jav +Key: ind_2122648877306013034, Target: ind, Predicted: jav +Key: ind_2233513352699206022, Target: ind, Predicted: jav +Key: ind_2307160667860695313, Target: ind, Predicted: jav +Key: ind_3060863748339320897, Target: ind, Predicted: jav +Key: ind_5228677418419567887, Target: ind, Predicted: jav +Key: ind_597500685919305943, Target: ind, Predicted: jav +Key: ind_4523556266535272953, Target: ind, Predicted: jav +Key: ind_4744952555395890998, Target: ind, Predicted: jav +Key: ind_5450655422482970725, Target: ind, Predicted: jav +Key: ind_5559410424421727906, Target: ind, Predicted: mar +Key: ind_6475550457404205193, Target: ind, Predicted: wol +Key: ind_9001049691913539084, Target: ind, Predicted: jav +Key: ind_8646335919182161698, Target: ind, Predicted: jav +Key: ind_9184891619867142204, Target: ind, Predicted: jav +Key: ind_67374860314916889, Target: ind, Predicted: jav +Key: ind_9660949463884628083, Target: ind, Predicted: jav +Key: ind_8503500464578299183, Target: ind, Predicted: jav +Key: jav_11986270656459162920, Target: jav, Predicted: ind +Key: jav_10320018353678592476, Target: jav, Predicted: ind +Key: jav_10795278807746062069, Target: jav, Predicted: ind +Key: jav_11752780790264575164, Target: jav, Predicted: ind +Key: jav_10907326559357611070, Target: jav, Predicted: ind +Key: jav_12967405631085473345, Target: jav, Predicted: ind +Key: jav_1344399084121838196, Target: jav, Predicted: ind +Key: jav_14809401794261814438, Target: jav, Predicted: ind +Key: jav_14841409234255496635, Target: jav, Predicted: mon +Key: jav_16604698454280783179, Target: jav, Predicted: ind +Key: jav_15088652548239920749, Target: jav, Predicted: ind +Key: jav_17434451046305288424, Target: jav, Predicted: ind +Key: jav_1594078195823164817, Target: jav, Predicted: ind +Key: jav_15997181650540811635, Target: jav, Predicted: ind +Key: jav_16983650252846351824, Target: jav, Predicted: ind +Key: jav_2318618640643459587, Target: jav, Predicted: ind +Key: jav_3189628974315092264, Target: jav, Predicted: ind +Key: jav_3221173694681105475, Target: jav, Predicted: ind +Key: jav_2554173677537257074, Target: jav, Predicted: ind +Key: jav_279535380270515435, Target: jav, Predicted: ind +Key: jav_2063835837164185316, Target: jav, Predicted: ind +Key: jav_5537966045565689997, Target: jav, Predicted: ind +Key: jav_4970619820331120620, Target: jav, Predicted: ind +Key: jav_6999994640216581640, Target: jav, Predicted: ind +Key: jav_5269750912602183500, Target: jav, Predicted: sun +Key: jav_7192065439110485038, Target: jav, Predicted: ind +Key: jav_9722828475989763047, Target: jav, Predicted: sun +Key: jav_8603110741765716499, Target: jav, Predicted: sun +Key: jav_8758594605647341919, Target: jav, Predicted: ind +Key: kam_1242016061651212744, Target: kam, Predicted: luo +Key: kam_1455301758555742062, Target: kam, Predicted: luo +Key: kam_13420369306311865100, Target: kam, Predicted: luo +Key: kam_13045083989122667223, Target: kam, Predicted: lug +Key: kam_15150110269721733005, Target: kam, Predicted: luo +Key: kam_157961075504251229, Target: kam, Predicted: zul +Key: kam_16047609881037455846, Target: kam, Predicted: swa +Key: kam_17447005591214699787, Target: kam, Predicted: swa +Key: kam_16124115010885535329, Target: kam, Predicted: swa +Key: kam_16127034024584070292, Target: kam, Predicted: swa +Key: kam_17690316600236278692, Target: kam, Predicted: luo +Key: kam_17860052992146491804, Target: kam, Predicted: swa +Key: kam_6085096344206308725, Target: kam, Predicted: luo +Key: kam_4208809560417850889, Target: kam, Predicted: luo +Key: kam_6438701065830347208, Target: kam, Predicted: luo +Key: kam_7211187970304142138, Target: kam, Predicted: luo +Key: kam_6808257960033082420, Target: kam, Predicted: luo +Key: kam_6835574604514011845, Target: kam, Predicted: luo +Key: kam_8018655713323272946, Target: kam, Predicted: swa +Key: kam_967471867191259695, Target: kam, Predicted: luo +Key: kan_14454497112521745355, Target: kan, Predicted: mal +Key: kan_2278795397297337931, Target: kan, Predicted: mal +Key: kan_17601542470188815847, Target: kan, Predicted: tam +Key: kan_18023272269047514633, Target: kan, Predicted: cym +Key: kan_5438903761990890690, Target: kan, Predicted: tam +Key: kea_11370577310892220452, Target: kea, Predicted: por +Key: kea_10023537024284874152, Target: kea, Predicted: ita +Key: kea_11450645254281300368, Target: kea, Predicted: por +Key: kea_10948659597447297834, Target: kea, Predicted: por +Key: kea_10949887230451203619, Target: kea, Predicted: por +Key: kea_11656366189750169253, Target: kea, Predicted: por +Key: kea_10327757258671853733, Target: kea, Predicted: ita +Key: kea_11058299817121451575, Target: kea, Predicted: por +Key: kea_11076533189719088516, Target: kea, Predicted: por +Key: kea_11212128483528833563, Target: kea, Predicted: oci +Key: kea_10591583497034258969, Target: kea, Predicted: por +Key: kea_13870472889580399300, Target: kea, Predicted: por +Key: kea_12028275812868902502, Target: kea, Predicted: por +Key: kea_13332474466403935880, Target: kea, Predicted: por +Key: kea_12041217662880702437, Target: kea, Predicted: por +Key: kea_12734184228471010648, Target: kea, Predicted: por +Key: kea_13382043799364373961, Target: kea, Predicted: por +Key: kea_12761914738496578506, Target: kea, Predicted: por +Key: kea_12828496938674945042, Target: kea, Predicted: por +Key: kea_12137157587968409286, Target: kea, Predicted: por +Key: kea_13503385683081391196, Target: kea, Predicted: por +Key: kea_12166135656179699570, Target: kea, Predicted: por +Key: kea_12200629229646978411, Target: kea, Predicted: por +Key: kea_13590271451577824706, Target: kea, Predicted: por +Key: kea_12283507574833588642, Target: kea, Predicted: ita +Key: kea_13611627060658051099, Target: kea, Predicted: por +Key: kea_12308175212043693828, Target: kea, Predicted: por +Key: kea_12317797419132562747, Target: kea, Predicted: por +Key: kea_12961880350010580919, Target: kea, Predicted: por +Key: kea_1235040407955579097, Target: kea, Predicted: por +Key: kea_13720501274990511945, Target: kea, Predicted: por +Key: kea_12424532967697730759, Target: kea, Predicted: por +Key: kea_1306162105932026228, Target: kea, Predicted: por +Key: kea_1374060463196740692, Target: kea, Predicted: por +Key: kea_14334944048066341274, Target: kea, Predicted: por +Key: kea_14402631003899703747, Target: kea, Predicted: oci +Key: kea_14413797605390970760, Target: kea, Predicted: por +Key: kea_15756559720995532615, Target: kea, Predicted: por +Key: kea_15222241268165118471, Target: kea, Predicted: por +Key: kea_15785343097434275974, Target: kea, Predicted: por +Key: kea_16474656917717706401, Target: kea, Predicted: por +Key: kea_16508275745535197182, Target: kea, Predicted: oci +Key: kea_14670781841400742132, Target: kea, Predicted: por +Key: kea_15267994968012737386, Target: kea, Predicted: por +Key: kea_15820143418421606260, Target: kea, Predicted: por +Key: kea_14690732557010562348, Target: kea, Predicted: por +Key: kea_15825981446765350672, Target: kea, Predicted: por +Key: kea_14701443643283431537, Target: kea, Predicted: por +Key: kea_15944610858348925975, Target: kea, Predicted: por +Key: kea_1678130674403159411, Target: kea, Predicted: por +Key: kea_15005840006257929229, Target: kea, Predicted: por +Key: kea_16218232455853863666, Target: kea, Predicted: por +Key: kea_16846378519337849962, Target: kea, Predicted: ita +Key: kea_16857697894295504817, Target: kea, Predicted: por +Key: kea_16253715537405601094, Target: kea, Predicted: por +Key: kea_1562170173413871353, Target: kea, Predicted: por +Key: kea_2177710035237734736, Target: kea, Predicted: por +Key: kea_17086716106493740163, Target: kea, Predicted: por +Key: kea_17095449410529783018, Target: kea, Predicted: por +Key: kea_17703710668441947524, Target: kea, Predicted: por +Key: kea_17112992874317603866, Target: kea, Predicted: por +Key: kea_248590141353320420, Target: kea, Predicted: por +Key: kea_17195926103756078293, Target: kea, Predicted: por +Key: kea_1862309501795884828, Target: kea, Predicted: por +Key: kea_17356206412965194573, Target: kea, Predicted: por +Key: kea_2773646806354448714, Target: kea, Predicted: por +Key: kea_17463566545251807382, Target: kea, Predicted: por +Key: kea_17464417069349009762, Target: kea, Predicted: por +Key: kea_17534201065369631609, Target: kea, Predicted: por +Key: kea_2888930183961937837, Target: kea, Predicted: por +Key: kea_2890483128225249206, Target: kea, Predicted: por +Key: kea_3514485212191406862, Target: kea, Predicted: por +Key: kea_4723416937214403911, Target: kea, Predicted: por +Key: kea_4760421816869934830, Target: kea, Predicted: por +Key: kea_4113397890146276183, Target: kea, Predicted: ita +Key: kea_4262774842290370021, Target: kea, Predicted: ita +Key: kea_4953877087358245717, Target: kea, Predicted: por +Key: kea_3156138704577040509, Target: kea, Predicted: por +Key: kea_4349008025354675953, Target: kea, Predicted: por +Key: kea_5017546362452453034, Target: kea, Predicted: por +Key: kea_4478899120966508654, Target: kea, Predicted: ita +Key: kea_3922600767444568824, Target: kea, Predicted: por +Key: kea_34012244872537127, Target: kea, Predicted: por +Key: kea_459504717075374584, Target: kea, Predicted: por +Key: kea_6733659578652753743, Target: kea, Predicted: por +Key: kea_5210147248718906327, Target: kea, Predicted: por +Key: kea_7295412942775124573, Target: kea, Predicted: ita +Key: kea_6884904234999946049, Target: kea, Predicted: por +Key: kea_5350981043758768546, Target: kea, Predicted: por +Key: kea_549960835444106667, Target: kea, Predicted: por +Key: kea_6314614134140891450, Target: kea, Predicted: por +Key: kea_752657841655742551, Target: kea, Predicted: por +Key: kea_6320088614458047720, Target: kea, Predicted: por +Key: kea_6957005070552528440, Target: kea, Predicted: por +Key: kea_5565228125392598074, Target: kea, Predicted: por +Key: kea_6995073637647979908, Target: kea, Predicted: por +Key: kea_7585596070117030295, Target: kea, Predicted: por +Key: kea_5774766842956021661, Target: kea, Predicted: por +Key: kea_8279992454119085060, Target: kea, Predicted: por +Key: kea_9441488993518695975, Target: kea, Predicted: por +Key: kea_8983270368149169064, Target: kea, Predicted: por +Key: kea_8382354646702725055, Target: kea, Predicted: por +Key: kea_9013409348402098602, Target: kea, Predicted: por +Key: kea_8402127232763004057, Target: kea, Predicted: por +Key: kea_9167632777072512568, Target: kea, Predicted: por +Key: kea_9626163960189261342, Target: kea, Predicted: por +Key: kea_9626810905026674760, Target: kea, Predicted: por +Key: kea_8161842936882513400, Target: kea, Predicted: zul +Key: kea_8704005351156506340, Target: kea, Predicted: por +Key: khm_10334747049756605425, Target: khm, Predicted: lao +Key: khm_10497588501073496528, Target: khm, Predicted: cnh +Key: khm_13229727687924019568, Target: khm, Predicted: cnh +Key: khm_16020836270012079206, Target: khm, Predicted: nep +Key: khm_8697193414050997321, Target: khm, Predicted: tha +Key: kir_12484293660119005996, Target: kir, Predicted: kaz +Key: kir_13569899049300641056, Target: kir, Predicted: kaz +Key: lao_10887463338602702992, Target: lao, Predicted: tha +Key: lao_11265992186853602922, Target: lao, Predicted: tha +Key: lao_1006200373522070745, Target: lao, Predicted: tha +Key: lao_11887598347379820904, Target: lao, Predicted: tha +Key: lao_13967848504591496835, Target: lao, Predicted: tha +Key: lao_14170982321559771103, Target: lao, Predicted: tha +Key: lao_16788493858014489978, Target: lao, Predicted: tha +Key: lao_16869693862749258217, Target: lao, Predicted: tha +Key: lao_17005144145609286708, Target: lao, Predicted: tha +Key: lao_14779177129565039564, Target: lao, Predicted: tha +Key: lao_17690709746158885116, Target: lao, Predicted: tha +Key: lao_5242334398822956714, Target: lao, Predicted: tha +Key: lao_2573597075027053599, Target: lao, Predicted: tha +Key: lao_5252626148202950305, Target: lao, Predicted: tha +Key: lao_5298567786585177784, Target: lao, Predicted: tha +Key: lao_2840990474993441772, Target: lao, Predicted: mya +Key: lao_4093873712172017946, Target: lao, Predicted: tha +Key: lao_5596283810118957871, Target: lao, Predicted: tha +Key: lao_4446692467268612894, Target: lao, Predicted: tha +Key: lao_6441665597466756945, Target: lao, Predicted: tha +Key: lao_7579288768944427147, Target: lao, Predicted: tha +Key: lao_7785608186563398290, Target: lao, Predicted: tha +Key: lao_697391896455587877, Target: lao, Predicted: tha +Key: lao_921350125794418465, Target: lao, Predicted: tha +Key: lao_7920467936081757903, Target: lao, Predicted: hun +Key: lao_9379426444336182237, Target: lao, Predicted: tha +Key: lao_7268137333437823000, Target: lao, Predicted: tha +Key: lin_14143588881603104411, Target: lin, Predicted: swa +Key: lin_6181759988724330242, Target: lin, Predicted: swa +Key: lin_6205227460245188525, Target: lin, Predicted: kam +Key: lin_5023781333105504302, Target: lin, Predicted: swa +Key: mal_11790916615074427050, Target: mal, Predicted: tam +Key: mal_13413981940425302406, Target: mal, Predicted: tam +Key: mal_2963602967050959977, Target: mal, Predicted: tel +Key: mal_4060784580455360232, Target: mal, Predicted: tam +Key: mal_7913095307002413892, Target: mal, Predicted: tam +Key: mal_7572549177910319908, Target: mal, Predicted: tam +Key: mar_12085171827775117772, Target: mar, Predicted: hin +Key: mar_12161518038097873713, Target: mar, Predicted: kan +Key: mar_17572297509713053639, Target: mar, Predicted: guj +Key: mar_16594609588987127295, Target: mar, Predicted: nep +Key: mar_1771706715401987957, Target: mar, Predicted: nep +Key: mar_1740449668720159196, Target: mar, Predicted: hin +Key: mar_3102880835466115285, Target: mar, Predicted: tel +Key: mar_3377293000824517505, Target: mar, Predicted: guj +Key: mar_290155358910906387, Target: mar, Predicted: mal +Key: mar_4450073235668634282, Target: mar, Predicted: pus +Key: mar_4634582622777921430, Target: mar, Predicted: hin +Key: mar_6983692225192195129, Target: mar, Predicted: amh +Key: mar_6065297199523085811, Target: mar, Predicted: hin +Key: mar_736297945803458785, Target: mar, Predicted: kan +Key: mar_8066350727316800792, Target: mar, Predicted: ori +Key: mkd_10508234460771157881, Target: mkd, Predicted: bul +Key: mon_6419873078571810053, Target: mon, Predicted: kir +Key: msa_11592129742562954883, Target: msa, Predicted: ind +Key: msa_10111750290465346764, Target: msa, Predicted: ind +Key: msa_10278709569280964844, Target: msa, Predicted: ind +Key: msa_1284458001199020844, Target: msa, Predicted: ind +Key: msa_1300674297372472502, Target: msa, Predicted: ind +Key: msa_13920403904525097223, Target: msa, Predicted: ind +Key: msa_13315105778793804607, Target: msa, Predicted: ind +Key: msa_13932171022482911745, Target: msa, Predicted: ind +Key: msa_1332453566006615802, Target: msa, Predicted: amh +Key: msa_15278275755320217147, Target: msa, Predicted: ind +Key: msa_15514940558237434990, Target: msa, Predicted: ind +Key: msa_16214460547845144952, Target: msa, Predicted: ind +Key: msa_14931080105285117562, Target: msa, Predicted: ind +Key: msa_17036851527779674162, Target: msa, Predicted: ind +Key: msa_14957155403923172893, Target: msa, Predicted: ind +Key: msa_16506881510047618268, Target: msa, Predicted: ind +Key: msa_17246169153905486782, Target: msa, Predicted: ind +Key: msa_15937417704324312446, Target: msa, Predicted: amh +Key: msa_16550161769222642393, Target: msa, Predicted: ind +Key: msa_16559597755638017631, Target: msa, Predicted: ind +Key: msa_3211524364394352200, Target: msa, Predicted: ind +Key: msa_3258736234714765747, Target: msa, Predicted: ind +Key: msa_17664396103973986874, Target: msa, Predicted: ind +Key: msa_3532640944784872782, Target: msa, Predicted: ind +Key: msa_3143854168108725602, Target: msa, Predicted: ind +Key: msa_5298223025114732196, Target: msa, Predicted: ind +Key: msa_5355132416195871988, Target: msa, Predicted: ind +Key: msa_4837534007396012963, Target: msa, Predicted: ind +Key: msa_6069902627787526985, Target: msa, Predicted: ind +Key: msa_4373985853873459692, Target: msa, Predicted: ind +Key: msa_8902204966494394918, Target: msa, Predicted: ind +Key: msa_83777383302230374, Target: msa, Predicted: ind +Key: msa_9171351600332066190, Target: msa, Predicted: ind +Key: msa_9295744745242697942, Target: msa, Predicted: ind +Key: msa_8171103788200625375, Target: msa, Predicted: oci +Key: msa_9441053850416013074, Target: msa, Predicted: ind +Key: msa_8275520026132069747, Target: msa, Predicted: ind +Key: msa_7385900595095863695, Target: msa, Predicted: ind +Key: msa_8277984690298636440, Target: msa, Predicted: amh +Key: mya_11719704389387835006, Target: mya, Predicted: swa +Key: mya_14572981959592002515, Target: mya, Predicted: bod +Key: mya_16880299434113297659, Target: mya, Predicted: bod +Key: mya_2719502130481624356, Target: mya, Predicted: bod +Key: mya_4667287739125951555, Target: mya, Predicted: bod +Key: mya_3383025368283713927, Target: mya, Predicted: bod +Key: mya_6560474911803614232, Target: mya, Predicted: bod +Key: nld_6442442109177507683, Target: nld, Predicted: afr +Key: nob_10301562314551190915, Target: nob, Predicted: swe +Key: nob_7656861941519176912, Target: nob, Predicted: swe +Key: nso_13628883810466986138, Target: nso, Predicted: zul +Key: nso_13267927202257223921, Target: nso, Predicted: xho +Key: nso_13958025683754945824, Target: nso, Predicted: xho +Key: nso_1670472911050657054, Target: nso, Predicted: kea +Key: nso_15520162543424505392, Target: nso, Predicted: ibo +Key: nso_1715292488357684795, Target: nso, Predicted: zul +Key: nso_5008883943313903156, Target: nso, Predicted: ibo +Key: nso_6965374829027893895, Target: nso, Predicted: zul +Key: nya_10106457194634439937, Target: nya, Predicted: lug +Key: nya_10233264068552841059, Target: nya, Predicted: wol +Key: nya_13172439978050471946, Target: nya, Predicted: spa +Key: nya_1338273132036702530, Target: nya, Predicted: sna +Key: nya_13482991765656320265, Target: nya, Predicted: lug +Key: nya_12921744305847121809, Target: nya, Predicted: swa +Key: nya_11763881159277367163, Target: nya, Predicted: nno +Key: nya_13130129174702749278, Target: nya, Predicted: swa +Key: nya_14581891094580731236, Target: nya, Predicted: sna +Key: nya_15525001354337589086, Target: nya, Predicted: nso +Key: nya_16412372432806922703, Target: nya, Predicted: swa +Key: nya_14179043511143684685, Target: nya, Predicted: ina +Key: nya_14417823488956954854, Target: nya, Predicted: hau +Key: nya_15929917562882150167, Target: nya, Predicted: gug +Key: nya_17845104593762187732, Target: nya, Predicted: eng +Key: nya_16835885437574471192, Target: nya, Predicted: lug +Key: nya_18213375420913988168, Target: nya, Predicted: spa +Key: nya_2482532613572582592, Target: nya, Predicted: lug +Key: nya_2513354752908216133, Target: nya, Predicted: nso +Key: nya_17691814905956084883, Target: nya, Predicted: lug +Key: nya_2543080883317062951, Target: nya, Predicted: lug +Key: nya_17232226097236149601, Target: nya, Predicted: lug +Key: nya_2586598392379044583, Target: nya, Predicted: spa +Key: nya_17811683938083865643, Target: nya, Predicted: lug +Key: nya_2694343727966740432, Target: nya, Predicted: ven +Key: nya_3354543193653122817, Target: nya, Predicted: hau +Key: nya_500512238056933841, Target: nya, Predicted: sna +Key: nya_5066171020738348261, Target: nya, Predicted: sna +Key: nya_5113595555274310987, Target: nya, Predicted: lug +Key: nya_3245996723536294111, Target: nya, Predicted: zul +Key: nya_5470053227889718418, Target: nya, Predicted: ven +Key: nya_7556641835060143974, Target: nya, Predicted: hau +Key: nya_6990195776330626358, Target: nya, Predicted: lug +Key: nya_7164921415117042231, Target: nya, Predicted: swa +Key: nya_8539464076546827039, Target: nya, Predicted: swa +Key: oci_10035114770268481131, Target: oci, Predicted: lin +Key: oci_10045200335963800667, Target: oci, Predicted: fra +Key: oci_10089792662956253395, Target: oci, Predicted: fra +Key: oci_10765240280265990407, Target: oci, Predicted: fra +Key: oci_10108654052356898839, Target: oci, Predicted: fra +Key: oci_10767422666262411491, Target: oci, Predicted: fra +Key: nya_9527704361406820831, Target: nya, Predicted: nso +Key: nya_9697478311105197096, Target: nya, Predicted: wol +Key: oci_10887532611586107183, Target: oci, Predicted: lin +Key: nya_9114272542528490138, Target: nya, Predicted: spa +Key: oci_12187678988410590975, Target: oci, Predicted: lin +Key: oci_12763060440991516042, Target: oci, Predicted: fra +Key: oci_1118186261985542700, Target: oci, Predicted: fra +Key: oci_1234728608810565461, Target: oci, Predicted: fra +Key: oci_12845684426379540476, Target: oci, Predicted: fra +Key: oci_1235230572818543124, Target: oci, Predicted: lin +Key: oci_11894037773693983344, Target: oci, Predicted: ina +Key: oci_12390438008058854319, Target: oci, Predicted: lin +Key: oci_12943307048282939379, Target: oci, Predicted: lin +Key: oci_11254194885989696450, Target: oci, Predicted: swa +Key: oci_11954354509598207686, Target: oci, Predicted: fra +Key: oci_11959677351390761127, Target: oci, Predicted: lin +Key: oci_11356167124222045127, Target: oci, Predicted: fra +Key: oci_12002753061572474258, Target: oci, Predicted: kea +Key: oci_11359807897999014535, Target: oci, Predicted: bre +Key: oci_12005022516190513283, Target: oci, Predicted: nso +Key: oci_12050354636534524667, Target: oci, Predicted: lin +Key: oci_13050822811222026811, Target: oci, Predicted: lin +Key: oci_14227773202293277457, Target: oci, Predicted: lin +Key: oci_14230425456430771911, Target: oci, Predicted: kea +Key: oci_14742673285720689094, Target: oci, Predicted: fra +Key: oci_14299067940127301212, Target: oci, Predicted: lin +Key: oci_14346767696111031738, Target: oci, Predicted: lin +Key: oci_14360239166218237330, Target: oci, Predicted: nso +Key: oci_14877060008445098941, Target: oci, Predicted: lin +Key: oci_13900413657066168531, Target: oci, Predicted: ita +Key: oci_14421549074026478736, Target: oci, Predicted: lin +Key: oci_14464968082818422887, Target: oci, Predicted: lin +Key: oci_14470678493261203952, Target: oci, Predicted: lin +Key: oci_14039577261107418246, Target: oci, Predicted: fra +Key: oci_14971865000970089192, Target: oci, Predicted: fra +Key: oci_13561310996844446718, Target: oci, Predicted: lin +Key: oci_14603552222275485534, Target: oci, Predicted: lin +Key: oci_15125979688517959499, Target: oci, Predicted: fra +Key: oci_16946912130847129233, Target: oci, Predicted: lin +Key: oci_16963598218837298695, Target: oci, Predicted: fra +Key: oci_17085791293703151801, Target: oci, Predicted: nso +Key: oci_1569678533502520702, Target: oci, Predicted: fra +Key: oci_17159516148371020092, Target: oci, Predicted: kea +Key: oci_15319443740009568810, Target: oci, Predicted: fra +Key: oci_17213915059671662512, Target: oci, Predicted: lin +Key: oci_17222333721294152971, Target: oci, Predicted: lin +Key: oci_16586176488415705051, Target: oci, Predicted: fra +Key: oci_15413735991738082194, Target: oci, Predicted: lin +Key: oci_16623163538133999013, Target: oci, Predicted: fra +Key: oci_15960152213912533341, Target: oci, Predicted: lin +Key: oci_17323665886719245022, Target: oci, Predicted: fra +Key: oci_1548097866879558718, Target: oci, Predicted: fra +Key: oci_16003608612435520144, Target: oci, Predicted: lin +Key: oci_15498883259209363181, Target: oci, Predicted: xho +Key: oci_16227138733838240895, Target: oci, Predicted: lin +Key: oci_278645950045181170, Target: oci, Predicted: kea +Key: oci_18390405066433627147, Target: oci, Predicted: fra +Key: oci_2395544672380227277, Target: oci, Predicted: lin +Key: oci_2859451806964580927, Target: oci, Predicted: lin +Key: oci_2401399291303101111, Target: oci, Predicted: fra +Key: oci_2862537988723592877, Target: oci, Predicted: lin +Key: oci_2465450537037327072, Target: oci, Predicted: lin +Key: oci_2497936287465812723, Target: oci, Predicted: lin +Key: oci_1884834082230414859, Target: oci, Predicted: fra +Key: oci_1920396433611304395, Target: oci, Predicted: fra +Key: oci_192245146755605494, Target: oci, Predicted: lin +Key: oci_18114114191869304725, Target: oci, Predicted: fra +Key: oci_1813623735703409032, Target: oci, Predicted: lin +Key: oci_2719491919789062330, Target: oci, Predicted: nso +Key: oci_2093016726419149325, Target: oci, Predicted: ron +Key: oci_3321046731458227092, Target: oci, Predicted: ina +Key: oci_4414232984941153132, Target: oci, Predicted: lin +Key: oci_4966264534737803148, Target: oci, Predicted: lin +Key: oci_442797991540420883, Target: oci, Predicted: lin +Key: oci_4997486703744070716, Target: oci, Predicted: lin +Key: oci_3425711327759764248, Target: oci, Predicted: fra +Key: oci_3452431826249878814, Target: oci, Predicted: fra +Key: oci_3471507779151941465, Target: oci, Predicted: fra +Key: oci_4072102901762339099, Target: oci, Predicted: fra +Key: oci_4144947263264733268, Target: oci, Predicted: lin +Key: oci_4652294378182605473, Target: oci, Predicted: fra +Key: oci_5088115619560169847, Target: oci, Predicted: swa +Key: oci_4251337780481273192, Target: oci, Predicted: lin +Key: oci_5207232958533370013, Target: oci, Predicted: fra +Key: oci_3746806009703404422, Target: oci, Predicted: lin +Key: oci_4309148831540162305, Target: oci, Predicted: fra +Key: oci_4757403258527307011, Target: oci, Predicted: lin +Key: oci_3783487835428908440, Target: oci, Predicted: kea +Key: oci_4792189983129184199, Target: oci, Predicted: fra +Key: oci_4849021251879998946, Target: oci, Predicted: fra +Key: oci_6932648805692649097, Target: oci, Predicted: lin +Key: oci_6944036571856684796, Target: oci, Predicted: fra +Key: oci_6402610575136094491, Target: oci, Predicted: fra +Key: oci_5398636369620403778, Target: oci, Predicted: lin +Key: oci_6464046320629307997, Target: oci, Predicted: ina +Key: oci_6994636997111842960, Target: oci, Predicted: ina +Key: oci_5425267487039445793, Target: oci, Predicted: lin +Key: oci_5968291094019460044, Target: oci, Predicted: fra +Key: oci_6485819516718800455, Target: oci, Predicted: lin +Key: oci_6514725888217094897, Target: oci, Predicted: lin +Key: oci_5530148634400670636, Target: oci, Predicted: lin +Key: oci_604433587118062254, Target: oci, Predicted: lin +Key: oci_7108935010194208233, Target: oci, Predicted: lin +Key: oci_6090148937806025949, Target: oci, Predicted: lin +Key: oci_5570888727036579087, Target: oci, Predicted: fra +Key: oci_558163011731938700, Target: oci, Predicted: fra +Key: oci_662763434746011465, Target: oci, Predicted: lin +Key: oci_5641642783734618043, Target: oci, Predicted: kea +Key: oci_720348032295147859, Target: oci, Predicted: ina +Key: oci_7311154359901630741, Target: oci, Predicted: lin +Key: oci_6919243241561828485, Target: oci, Predicted: lin +Key: oci_8784051546383688980, Target: oci, Predicted: lin +Key: oci_8355130754641130407, Target: oci, Predicted: bre +Key: oci_8806419492013950659, Target: oci, Predicted: lin +Key: oci_7863766883487840579, Target: oci, Predicted: ina +Key: oci_788182832077611318, Target: oci, Predicted: lin +Key: oci_8827344410125877233, Target: oci, Predicted: fra +Key: oci_792479661611380052, Target: oci, Predicted: lin +Key: oci_8872730984281443881, Target: oci, Predicted: fra +Key: oci_8960583867939494320, Target: oci, Predicted: ina +Key: oci_7965796548492097422, Target: oci, Predicted: ina +Key: oci_8515553996209440419, Target: oci, Predicted: lin +Key: oci_8529411017397435197, Target: oci, Predicted: fra +Key: oci_8115850824216530028, Target: oci, Predicted: lin +Key: oci_8622647309406498485, Target: oci, Predicted: lin +Key: oci_8624571117604846737, Target: oci, Predicted: fra +Key: oci_8640979722074290991, Target: oci, Predicted: lin +Key: oci_9220133925237440798, Target: oci, Predicted: lin +Key: oci_821720063451332210, Target: oci, Predicted: lin +Key: oci_8217566762692038572, Target: oci, Predicted: bre +Key: oci_9830304339339721305, Target: oci, Predicted: lin +Key: oci_9831556062305912293, Target: oci, Predicted: swa +Key: oci_9356094487703999823, Target: oci, Predicted: ina +Key: oci_9397977856215572, Target: oci, Predicted: nso +Key: oci_9479976141789085195, Target: oci, Predicted: kea +Key: ori_10716454304165242105, Target: ori, Predicted: pan +Key: ori_10842000332292806830, Target: ori, Predicted: pan +Key: ori_109658993300107827, Target: ori, Predicted: guj +Key: ori_11648079604769219462, Target: ori, Predicted: guj +Key: ori_1166466104726901401, Target: ori, Predicted: pan +Key: ori_11828391465034838498, Target: ori, Predicted: ben +Key: ori_12718454599028988781, Target: ori, Predicted: ben +Key: ori_12439810431044719325, Target: ori, Predicted: pan +Key: ori_13659205729028471754, Target: ori, Predicted: pan +Key: ori_12921665378125022567, Target: ori, Predicted: pan +Key: ori_12051954371611434881, Target: ori, Predicted: nep +Key: ori_1254063656157621956, Target: ori, Predicted: tel +Key: ori_12053985902125347929, Target: ori, Predicted: pan +Key: ori_12612743474789668409, Target: ori, Predicted: asm +Key: ori_12160814635680669470, Target: ori, Predicted: ben +Key: ori_12659829034181992522, Target: ori, Predicted: pan +Key: ori_14595364881600317211, Target: ori, Predicted: pus +Key: ori_14011138884456145834, Target: ori, Predicted: mar +Key: ori_14615642173121656903, Target: ori, Predicted: guj +Key: ori_1518302752387214528, Target: ori, Predicted: nep +Key: ori_15328638747121073753, Target: ori, Predicted: nep +Key: ori_153687043684510415, Target: ori, Predicted: pan +Key: ori_14772601123586174338, Target: ori, Predicted: tel +Key: ori_15546458309483733348, Target: ori, Predicted: pan +Key: ori_1555191647132318669, Target: ori, Predicted: mar +Key: ori_15029526583664657395, Target: ori, Predicted: guj +Key: ori_16408704756304694753, Target: ori, Predicted: nep +Key: ori_16731800149238765346, Target: ori, Predicted: asm +Key: ori_17285042486006238075, Target: ori, Predicted: asm +Key: ori_17833451443110352600, Target: ori, Predicted: guj +Key: ori_17882520093636488227, Target: ori, Predicted: nep +Key: ori_17891110391803856787, Target: ori, Predicted: guj +Key: ori_2090383390678744231, Target: ori, Predicted: nep +Key: ori_2186711700360652563, Target: ori, Predicted: ben +Key: ori_17525787697773550027, Target: ori, Predicted: guj +Key: ori_2229457956374815551, Target: ori, Predicted: ben +Key: ori_2253021112463896310, Target: ori, Predicted: ben +Key: ori_2783438669106573260, Target: ori, Predicted: asm +Key: ori_2290919075628687500, Target: ori, Predicted: pan +Key: ori_2797358804468943768, Target: ori, Predicted: ben +Key: ori_2409462026481215307, Target: ori, Predicted: kan +Key: ori_2462662444772885270, Target: ori, Predicted: mar +Key: ori_2981244338440434554, Target: ori, Predicted: kan +Key: ori_4190610879288846829, Target: ori, Predicted: nep +Key: ori_2709641879783847902, Target: ori, Predicted: ben +Key: ori_6255800141494218197, Target: ori, Predicted: mar +Key: ori_4331564786242332644, Target: ori, Predicted: asm +Key: ori_6282781328026356259, Target: ori, Predicted: ben +Key: ori_6565978847528005434, Target: ori, Predicted: guj +Key: ori_5652757102520751357, Target: ori, Predicted: guj +Key: ori_6649516954820601194, Target: ori, Predicted: tel +Key: ori_6666933898209756577, Target: ori, Predicted: pan +Key: ori_6711883068622594196, Target: ori, Predicted: ben +Key: ori_4745222971940497976, Target: ori, Predicted: nep +Key: ori_5973935840470467578, Target: ori, Predicted: guj +Key: ori_6918798369703733879, Target: ori, Predicted: asm +Key: ori_8353587877107298882, Target: ori, Predicted: guj +Key: ori_7732812069959564845, Target: ori, Predicted: guj +Key: ori_8407705107066164002, Target: ori, Predicted: nep +Key: ori_7107637805520281616, Target: ori, Predicted: kan +Key: ori_7209394709568206074, Target: ori, Predicted: asm +Key: ori_8549700899459932520, Target: ori, Predicted: nep +Key: ori_9345907644759232390, Target: ori, Predicted: nep +Key: ori_9373672110235547769, Target: ori, Predicted: guj +Key: ori_9635294763359140021, Target: ori, Predicted: guj +Key: pan_10695540541201012273, Target: pan, Predicted: hin +Key: ori_9978519570425282624, Target: ori, Predicted: asm +Key: orm_6273167335000591623, Target: orm, Predicted: ful +Key: pan_12290458536723623373, Target: pan, Predicted: hin +Key: pan_13468158527037882393, Target: pan, Predicted: guj +Key: pan_14754779207603625942, Target: pan, Predicted: hin +Key: pan_15924258999158666628, Target: pan, Predicted: hin +Key: pan_16973197545793408004, Target: pan, Predicted: hin +Key: pan_17668201018365750359, Target: pan, Predicted: ben +Key: pan_4194855438423333835, Target: pan, Predicted: mal +Key: pan_3683112638415344147, Target: pan, Predicted: hin +Key: pan_9584883713400315701, Target: pan, Predicted: mal +Key: pan_9621939290570706458, Target: pan, Predicted: guj +Key: pus_10349747224958645856, Target: pus, Predicted: urd +Key: pus_16512516377951882534, Target: pus, Predicted: urd +Key: ron_12949738213184570453, Target: ron, Predicted: slv +Key: rus_10021068571959461962, Target: rus, Predicted: ukr +Key: rus_11567345507381565160, Target: rus, Predicted: ukr +Key: rus_14973914077317373840, Target: rus, Predicted: ukr +Key: rus_15168957797605772339, Target: rus, Predicted: ukr +Key: slk_13271373239593195609, Target: slk, Predicted: ces +Key: slk_13632794656638588287, Target: slk, Predicted: ces +Key: slk_13719678465912843350, Target: slk, Predicted: lav +Key: slk_14707864985700181162, Target: slk, Predicted: kir +Key: slk_15635702768228927767, Target: slk, Predicted: spa +Key: slk_17759699795618990124, Target: slk, Predicted: ibo +Key: slk_383743626413610591, Target: slk, Predicted: mon +Key: slk_9547896389507947757, Target: slk, Predicted: ces +Key: slv_12784941023408096188, Target: slv, Predicted: mkd +Key: sna_12163247980833279669, Target: sna, Predicted: nso +Key: sna_12281198795183461267, Target: sna, Predicted: nya +Key: sna_14733868880234125920, Target: sna, Predicted: zul +Key: sna_18176928007039618179, Target: sna, Predicted: zul +Key: sna_375732233404443184, Target: sna, Predicted: nya +Key: sna_4649833110658933188, Target: sna, Predicted: zul +Key: sna_9569619627876551134, Target: sna, Predicted: zul +Key: sna_8850264822718355017, Target: sna, Predicted: zul +Key: snd_10931035890407772670, Target: snd, Predicted: hin +Key: snd_13956110553955506374, Target: snd, Predicted: pan +Key: snd_12812786449551127599, Target: snd, Predicted: pus +Key: snd_12867915390493355608, Target: snd, Predicted: hin +Key: snd_14715306487520336338, Target: snd, Predicted: tel +Key: snd_15101535145466596506, Target: snd, Predicted: urd +Key: snd_16118146785944658451, Target: snd, Predicted: guj +Key: snd_17416420152628563852, Target: snd, Predicted: pan +Key: snd_1802768459296125605, Target: snd, Predicted: urd +Key: snd_17660182764094277200, Target: snd, Predicted: guj +Key: snd_2318860866138896926, Target: snd, Predicted: pan +Key: snd_4335680412400837436, Target: snd, Predicted: guj +Key: snd_7446254059662185404, Target: snd, Predicted: urd +Key: snd_9833325573339770401, Target: snd, Predicted: urd +Key: snd_9398135184837581319, Target: snd, Predicted: pan +Key: som_14141698925594276772, Target: som, Predicted: orm +Key: som_16912522961496692039, Target: som, Predicted: pus +Key: som_5648620539638716837, Target: som, Predicted: hau +Key: som_6731941157034302338, Target: som, Predicted: wol +Key: som_8869678771586021598, Target: som, Predicted: war +Key: srp_10133555762101302363, Target: srp, Predicted: ita +Key: srp_10612022720733778497, Target: srp, Predicted: ukr +Key: srp_10692529857731497529, Target: srp, Predicted: bos +Key: srp_12813362938308778530, Target: srp, Predicted: hrv +Key: srp_11340712043539403058, Target: srp, Predicted: hrv +Key: srp_10887538631131934548, Target: srp, Predicted: hrv +Key: srp_12823258331730624548, Target: srp, Predicted: hrv +Key: srp_10887767311962801524, Target: srp, Predicted: hrv +Key: srp_11421018368737558425, Target: srp, Predicted: hrv +Key: srp_1218977116116537053, Target: srp, Predicted: hrv +Key: srp_12836487645881379809, Target: srp, Predicted: hrv +Key: srp_12898020913220115354, Target: srp, Predicted: slv +Key: srp_12351238112433640566, Target: srp, Predicted: hrv +Key: srp_11468751733696581296, Target: srp, Predicted: hrv +Key: srp_11473412062427289648, Target: srp, Predicted: bos +Key: srp_11548285736388746487, Target: srp, Predicted: hrv +Key: srp_12446713251027730788, Target: srp, Predicted: bos +Key: srp_11057548844562062112, Target: srp, Predicted: hrv +Key: srp_11613754600803291364, Target: srp, Predicted: hrv +Key: srp_13148136760149614702, Target: srp, Predicted: hrv +Key: srp_13150940717764852133, Target: srp, Predicted: hrv +Key: srp_11660627579021677534, Target: srp, Predicted: slv +Key: srp_12597168727263941920, Target: srp, Predicted: bos +Key: srp_12612523099659534933, Target: srp, Predicted: hrv +Key: srp_11194968110504455746, Target: srp, Predicted: hrv +Key: srp_13281712431067425150, Target: srp, Predicted: hrv +Key: srp_12643216583758875465, Target: srp, Predicted: hrv +Key: srp_12689371601097882687, Target: srp, Predicted: hrv +Key: srp_12781466767444155566, Target: srp, Predicted: bos +Key: srp_11280345007752552040, Target: srp, Predicted: pol +Key: srp_14087303640117784984, Target: srp, Predicted: hrv +Key: srp_1561197198911191967, Target: srp, Predicted: bos +Key: srp_14869679311281226051, Target: srp, Predicted: bos +Key: srp_14142512391140910693, Target: srp, Predicted: hrv +Key: srp_13571981841213205265, Target: srp, Predicted: hrv +Key: srp_15784875444534085094, Target: srp, Predicted: hrv +Key: srp_14446354744328814087, Target: srp, Predicted: hrv +Key: srp_1524602416579736867, Target: srp, Predicted: bos +Key: srp_15965985615688495037, Target: srp, Predicted: bos +Key: srp_13733312980985457459, Target: srp, Predicted: bos +Key: srp_14523985400297594582, Target: srp, Predicted: hrv +Key: srp_13748868233695251269, Target: srp, Predicted: lit +Key: srp_14566638703835303545, Target: srp, Predicted: hrv +Key: srp_15323123435233914979, Target: srp, Predicted: bos +Key: srp_15327410018752054935, Target: srp, Predicted: hrv +Key: srp_16010901362751064407, Target: srp, Predicted: hrv +Key: srp_16032391823712981203, Target: srp, Predicted: bos +Key: srp_13795975814032988280, Target: srp, Predicted: slv +Key: srp_14798684010719157229, Target: srp, Predicted: bos +Key: srp_17760129771668465558, Target: srp, Predicted: bos +Key: srp_17055330937656763140, Target: srp, Predicted: hrv +Key: srp_2050155080273896517, Target: srp, Predicted: hrv +Key: srp_16468924855521330525, Target: srp, Predicted: hrv +Key: srp_17104982163933582656, Target: srp, Predicted: slv +Key: srp_17130881798881632639, Target: srp, Predicted: hrv +Key: srp_1790418761023712291, Target: srp, Predicted: hrv +Key: srp_17913261368920506986, Target: srp, Predicted: hrv +Key: srp_2157416035830743830, Target: srp, Predicted: hrv +Key: srp_16597125885542110325, Target: srp, Predicted: hrv +Key: srp_17332768808358277313, Target: srp, Predicted: bos +Key: srp_16714408230381531453, Target: srp, Predicted: hrv +Key: srp_230885809902828017, Target: srp, Predicted: bos +Key: srp_16777469404720242321, Target: srp, Predicted: hrv +Key: srp_1822675761759546186, Target: srp, Predicted: hrv +Key: srp_18262284411831239292, Target: srp, Predicted: hrv +Key: srp_16836675518437273627, Target: srp, Predicted: hrv +Key: srp_17581686081703595796, Target: srp, Predicted: hrv +Key: srp_18313076778692973044, Target: srp, Predicted: hrv +Key: srp_2418469796903974385, Target: srp, Predicted: por +Key: srp_2457237787317605110, Target: srp, Predicted: hrv +Key: srp_16939451472070369048, Target: srp, Predicted: hrv +Key: srp_17641484960865465742, Target: srp, Predicted: hrv +Key: srp_2476769423608175841, Target: srp, Predicted: hrv +Key: srp_16977110776956870235, Target: srp, Predicted: bos +Key: srp_17711408051965203127, Target: srp, Predicted: hrv +Key: srp_16978919199115158402, Target: srp, Predicted: bos +Key: srp_4233312582066737752, Target: srp, Predicted: hrv +Key: srp_3535801206488915714, Target: srp, Predicted: hrv +Key: srp_5284787053596506328, Target: srp, Predicted: bos +Key: srp_4271865598148278757, Target: srp, Predicted: bos +Key: srp_4287907716650227291, Target: srp, Predicted: hrv +Key: srp_3725694752773049664, Target: srp, Predicted: bos +Key: srp_3752033416981220528, Target: srp, Predicted: bos +Key: srp_4662479571104272772, Target: srp, Predicted: hrv +Key: srp_5477995090434713681, Target: srp, Predicted: hrv +Key: srp_3073441808531574745, Target: srp, Predicted: bos +Key: srp_5587887704225940987, Target: srp, Predicted: slv +Key: srp_401415834133928697, Target: srp, Predicted: bos +Key: srp_4811391356855624115, Target: srp, Predicted: hrv +Key: srp_4036589803044236257, Target: srp, Predicted: hrv +Key: srp_4857740665202028077, Target: srp, Predicted: hrv +Key: srp_3298070003305091847, Target: srp, Predicted: hrv +Key: srp_5722965903727582680, Target: srp, Predicted: bos +Key: srp_3300633125476658427, Target: srp, Predicted: ita +Key: srp_3345230356332427449, Target: srp, Predicted: hrv +Key: srp_335080029527666830, Target: srp, Predicted: bos +Key: srp_3418866641301312995, Target: srp, Predicted: bos +Key: srp_5080412504890316633, Target: srp, Predicted: hrv +Key: srp_4206651015049398499, Target: srp, Predicted: hrv +Key: srp_5086438909906887600, Target: srp, Predicted: hrv +Key: srp_6958511098191772345, Target: srp, Predicted: bos +Key: srp_8691278146942049620, Target: srp, Predicted: bos +Key: srp_6970563360433186511, Target: srp, Predicted: bos +Key: srp_7069614473823453984, Target: srp, Predicted: bos +Key: srp_8806908509714112244, Target: srp, Predicted: hrv +Key: srp_6339484391901060882, Target: srp, Predicted: bos +Key: srp_634644504378566709, Target: srp, Predicted: hrv +Key: srp_8869891018025818143, Target: srp, Predicted: bos +Key: srp_8895409715503085827, Target: srp, Predicted: hrv +Key: srp_7339755553258307485, Target: srp, Predicted: slv +Key: srp_8903821551900780948, Target: srp, Predicted: hrv +Key: srp_7370153204166404108, Target: srp, Predicted: bos +Key: srp_891846840334889412, Target: srp, Predicted: bos +Key: srp_8245727986274315516, Target: srp, Predicted: bos +Key: srp_7522916916741686314, Target: srp, Predicted: hrv +Key: srp_7527906355169222257, Target: srp, Predicted: hrv +Key: srp_9182989541276590345, Target: srp, Predicted: hrv +Key: srp_7542757110562297554, Target: srp, Predicted: bos +Key: srp_8338313137366015486, Target: srp, Predicted: bos +Key: srp_7628559925561231457, Target: srp, Predicted: hrv +Key: srp_8544461313029052405, Target: srp, Predicted: hrv +Key: srp_6859472637837584512, Target: srp, Predicted: hrv +Key: srp_8587534906307764120, Target: srp, Predicted: bos +Key: srp_9441124599701705331, Target: srp, Predicted: hrv +Key: srp_9600003180782445157, Target: srp, Predicted: hrv +Key: srp_9672695547465118938, Target: srp, Predicted: hrv +Key: srp_9753998011115250801, Target: srp, Predicted: hrv +Key: srp_9857456545548909543, Target: srp, Predicted: bos +Key: srp_985891529643953825, Target: srp, Predicted: bos +Key: srp_9924022485839799273, Target: srp, Predicted: hrv +Key: swe_10070024341607636465, Target: swe, Predicted: nno +Key: swe_11122883148996247132, Target: swe, Predicted: ron +Key: swe_7361101927076262644, Target: swe, Predicted: nno +Key: swe_989470138399558072, Target: swe, Predicted: nob +Key: tam_10072217146537983584, Target: tam, Predicted: tel +Key: tam_12630471432197299137, Target: tam, Predicted: tel +Key: tam_11168727768279127482, Target: tam, Predicted: tel +Key: tam_14516850308917653491, Target: tam, Predicted: mal +Key: tam_13871963685686168470, Target: tam, Predicted: kan +Key: tam_15174502076958430315, Target: tam, Predicted: mal +Key: tam_193650557789856037, Target: tam, Predicted: mal +Key: tam_5168814581281946692, Target: tam, Predicted: mal +Key: tam_809206326947205371, Target: tam, Predicted: mal +Key: tam_8984584548068584801, Target: tam, Predicted: tel +Key: tel_16491693500662842324, Target: tel, Predicted: tam +Key: tel_5530909766127012291, Target: tel, Predicted: tam +Key: tel_4978090353589600213, Target: tel, Predicted: tam +Key: tgk_12038985123290448268, Target: tgk, Predicted: ckb +Key: tgk_10533746227658192753, Target: tgk, Predicted: ckb +Key: tgk_13766450087259196814, Target: tgk, Predicted: ckb +Key: tgk_13890057497568118847, Target: tgk, Predicted: ckb +Key: tgk_18313751795463159501, Target: tgk, Predicted: ckb +Key: tgk_16606279975798113857, Target: tgk, Predicted: ckb +Key: tgk_2811744375061220005, Target: tgk, Predicted: fas +Key: tgk_4677509700075763495, Target: tgk, Predicted: fas +Key: tgk_5711492026403226793, Target: tgk, Predicted: fas +Key: tgk_4026197175952387178, Target: tgk, Predicted: hye +Key: tha_14009510506874278149, Target: tha, Predicted: lao +Key: tha_16547208074444766367, Target: tha, Predicted: lao +Key: ukr_4600941743658948385, Target: ukr, Predicted: rus +Key: ukr_4728895735813456314, Target: ukr, Predicted: rus +Key: umb_15327131374077531808, Target: umb, Predicted: lin +Key: umb_16643059632751075901, Target: umb, Predicted: xho +Key: umb_16887749978225736296, Target: umb, Predicted: swa +Key: umb_14963243486244077852, Target: umb, Predicted: nso +Key: umb_15038502058577647135, Target: umb, Predicted: swa +Key: umb_17501231758192557480, Target: umb, Predicted: luo +Key: umb_4299344406693017609, Target: umb, Predicted: xho +Key: umb_1972824399443220996, Target: umb, Predicted: lin +Key: umb_2017429004702861148, Target: umb, Predicted: lin +Key: umb_6389421232262193714, Target: umb, Predicted: grn +Key: umb_46806884485517499, Target: umb, Predicted: swa +Key: umb_6575652888668719802, Target: umb, Predicted: xho +Key: urd_11857138524629511643, Target: urd, Predicted: hin +Key: urd_11935094216278255680, Target: urd, Predicted: hin +Key: urd_10611851645626104097, Target: urd, Predicted: hin +Key: urd_10624132568857900978, Target: urd, Predicted: hin +Key: urd_12000653928344037649, Target: urd, Predicted: hin +Key: umb_9176405967749309567, Target: umb, Predicted: lin +Key: urd_1216329082813867564, Target: urd, Predicted: hin +Key: urd_12266818228956444261, Target: urd, Predicted: hin +Key: urd_12874697479281866232, Target: urd, Predicted: hin +Key: urd_13196367225357384662, Target: urd, Predicted: hin +Key: urd_17284988443414556501, Target: urd, Predicted: hin +Key: urd_1332602859647690673, Target: urd, Predicted: hin +Key: urd_2446224150806640796, Target: urd, Predicted: hin +Key: urd_15513775406043850283, Target: urd, Predicted: hin +Key: urd_2557131221344299698, Target: urd, Predicted: hin +Key: urd_1362203696835771765, Target: urd, Predicted: hin +Key: urd_17732883011657798714, Target: urd, Predicted: hin +Key: urd_1576050626509555235, Target: urd, Predicted: hin +Key: urd_15920857700902050686, Target: urd, Predicted: hin +Key: urd_17844323058432756473, Target: urd, Predicted: hin +Key: urd_13847001784999338851, Target: urd, Predicted: hin +Key: urd_16266308329229766642, Target: urd, Predicted: hin +Key: urd_16347407517219196829, Target: urd, Predicted: hin +Key: urd_14079228178881184819, Target: urd, Predicted: hin +Key: urd_16377121192027225223, Target: urd, Predicted: hin +Key: urd_16402832591682278554, Target: urd, Predicted: hin +Key: urd_18328037278962906455, Target: urd, Predicted: hin +Key: urd_14346938727323136720, Target: urd, Predicted: hin +Key: urd_3273024814822561875, Target: urd, Predicted: hin +Key: urd_14391769455756671345, Target: urd, Predicted: hin +Key: urd_16529599399265855569, Target: urd, Predicted: hin +Key: urd_14700870390991624182, Target: urd, Predicted: hin +Key: urd_206990840524233223, Target: urd, Predicted: hin +Key: urd_16943934763576540681, Target: urd, Predicted: pan +Key: urd_2212136085421978601, Target: urd, Predicted: hin +Key: urd_3809805322037428580, Target: urd, Predicted: hin +Key: urd_5291579398816048860, Target: urd, Predicted: hin +Key: urd_6980771525190617518, Target: urd, Predicted: hin +Key: urd_9020584967112911516, Target: urd, Predicted: hin +Key: urd_3949745999494304343, Target: urd, Predicted: hin +Key: urd_7078789154147826126, Target: urd, Predicted: hin +Key: urd_7097519219832988560, Target: urd, Predicted: hin +Key: urd_7228450244189603803, Target: urd, Predicted: hin +Key: urd_5871438281700834673, Target: urd, Predicted: hin +Key: urd_4285722114420280535, Target: urd, Predicted: hin +Key: urd_7791195432941232482, Target: urd, Predicted: hin +Key: urd_9812158965425579976, Target: urd, Predicted: hin +Key: urd_9994793090633271833, Target: urd, Predicted: hin +Key: urd_4571714796538875882, Target: urd, Predicted: hin +Key: urd_6386367732363102384, Target: urd, Predicted: hin +Key: urd_6503295717676582815, Target: urd, Predicted: hin +Key: urd_6510587710189564968, Target: urd, Predicted: hin +Key: urd_6946534457899862790, Target: urd, Predicted: hin +Key: uzb_13313597062404594815, Target: uzb, Predicted: kir +Key: uzb_16567258102160882103, Target: uzb, Predicted: kir +Key: uzb_17482658228095102970, Target: uzb, Predicted: chv +Key: uzb_302679039007787707, Target: uzb, Predicted: kir +Key: uzb_6517103134674269945, Target: uzb, Predicted: kir +Key: uzb_9885665600324059610, Target: uzb, Predicted: aze +Key: uzb_9504394072718385531, Target: uzb, Predicted: pus +Key: wol_1027260428980230992, Target: wol, Predicted: ful +Key: wol_1381733371359486757, Target: wol, Predicted: ful +Key: wol_1970196440175973515, Target: wol, Predicted: kea +Key: wol_1740687145003300159, Target: wol, Predicted: ful +Key: wol_16077803617402724834, Target: wol, Predicted: ful +Key: wol_16749038240930993889, Target: wol, Predicted: ful +Key: xho_10559235104824188329, Target: xho, Predicted: zul +Key: xho_12153143281768834747, Target: xho, Predicted: zul +Key: xho_15266051011306359135, Target: xho, Predicted: zul +Key: xho_14007129089646524578, Target: xho, Predicted: zul +Key: xho_18275342378884488139, Target: xho, Predicted: zul +Key: xho_3034291871833168684, Target: xho, Predicted: nya +Key: xho_4920442185906289723, Target: xho, Predicted: nya +Key: xho_616782905345928351, Target: xho, Predicted: zul +Key: xho_6782259076560989445, Target: xho, Predicted: zul +Key: xho_6391198370117725844, Target: xho, Predicted: zul +Key: xho_5960486203143084684, Target: xho, Predicted: zul +Key: xho_86315000786011260, Target: xho, Predicted: zul +Key: xho_8768198074988645428, Target: xho, Predicted: zul +Key: xho_9549900889219191882, Target: xho, Predicted: zul +Key: zul_10809734731250783487, Target: zul, Predicted: nso +Key: zul_11409890281613077463, Target: zul, Predicted: xho +Key: zul_10179620851045022176, Target: zul, Predicted: xho +Key: zul_11509170332495831206, Target: zul, Predicted: xho +Key: zul_10236834525194481245, Target: zul, Predicted: xho +Key: zul_10907322559052670368, Target: zul, Predicted: xho +Key: zul_11592657540288168149, Target: zul, Predicted: xho +Key: zul_10983276434682369544, Target: zul, Predicted: xho +Key: zul_10376816655783433209, Target: zul, Predicted: xho +Key: zul_10994116875143238143, Target: zul, Predicted: xho +Key: zul_1171797286482634752, Target: zul, Predicted: xho +Key: zul_1046638178426816339, Target: zul, Predicted: xho +Key: zul_10495157933204271581, Target: zul, Predicted: xho +Key: zul_11067700913855637129, Target: zul, Predicted: xho +Key: zul_11078112223849221317, Target: zul, Predicted: xho +Key: zul_1054214380299464529, Target: zul, Predicted: xho +Key: zul_10560577862918063751, Target: zul, Predicted: xho +Key: zul_10612834763345893993, Target: zul, Predicted: xho +Key: zul_11914632126540692002, Target: zul, Predicted: xho +Key: zul_11105652702009375708, Target: zul, Predicted: xho +Key: zul_11924556302667429240, Target: zul, Predicted: xho +Key: zul_11142811248248894758, Target: zul, Predicted: sna +Key: zul_10782531262571389975, Target: zul, Predicted: xho +Key: zul_13118365232040789892, Target: zul, Predicted: xho +Key: zul_13718316347215643899, Target: zul, Predicted: xho +Key: zul_12755467465915560922, Target: zul, Predicted: xho +Key: zul_13191353774423230015, Target: zul, Predicted: xho +Key: zul_12785699444233336839, Target: zul, Predicted: xho +Key: zul_1389046613828636518, Target: zul, Predicted: xho +Key: zul_12377215388363939668, Target: zul, Predicted: xho +Key: zul_1391250393057501851, Target: zul, Predicted: xho +Key: zul_12426319052289023348, Target: zul, Predicted: xho +Key: zul_13479411556064775372, Target: zul, Predicted: xho +Key: zul_14128268918226480768, Target: zul, Predicted: xho +Key: zul_13065895441509923056, Target: zul, Predicted: xho +Key: zul_1267515324770503764, Target: zul, Predicted: xho +Key: zul_13075246235803834815, Target: zul, Predicted: xho +Key: zul_13089615627008466387, Target: zul, Predicted: xho +Key: zul_13707062503823962165, Target: zul, Predicted: xho +Key: zul_14315829575220799165, Target: zul, Predicted: xho +Key: zul_14355295855384382179, Target: zul, Predicted: xho +Key: zul_14423993318256884114, Target: zul, Predicted: xho +Key: zul_15711553143112509454, Target: zul, Predicted: xho +Key: zul_15101216129834269094, Target: zul, Predicted: xho +Key: zul_14490061345748017555, Target: zul, Predicted: xho +Key: zul_15147888745640013856, Target: zul, Predicted: xho +Key: zul_14521902939516418842, Target: zul, Predicted: nya +Key: zul_14668736518060979799, Target: zul, Predicted: xho +Key: zul_14700682974567023750, Target: zul, Predicted: xho +Key: zul_15321449259257839932, Target: zul, Predicted: xho +Key: zul_14733806593232975564, Target: zul, Predicted: xho +Key: zul_15327897328875814604, Target: zul, Predicted: xho +Key: zul_14794942763283370618, Target: zul, Predicted: xho +Key: zul_14853985325623906611, Target: zul, Predicted: xho +Key: zul_16061353203439057515, Target: zul, Predicted: xho +Key: zul_16681332751694202196, Target: zul, Predicted: xho +Key: zul_16721743882981230318, Target: zul, Predicted: xho +Key: zul_16761241988426650983, Target: zul, Predicted: xho +Key: zul_17546214278518218592, Target: zul, Predicted: xho +Key: zul_17583949574078769681, Target: zul, Predicted: xho +Key: zul_17995276332497596342, Target: zul, Predicted: xho +Key: zul_16955493551322873871, Target: zul, Predicted: xho +Key: zul_17027247119280842878, Target: zul, Predicted: nso +Key: zul_17656160082310691093, Target: zul, Predicted: xho +Key: zul_17167034721249735771, Target: zul, Predicted: xho +Key: zul_2343414618182766422, Target: zul, Predicted: xho +Key: zul_18271961916832531734, Target: zul, Predicted: xho +Key: zul_17204427068602077252, Target: zul, Predicted: xho +Key: zul_17782653358019406676, Target: zul, Predicted: xho +Key: zul_2408763893860841231, Target: zul, Predicted: xho +Key: zul_18276706113466358513, Target: zul, Predicted: xho +Key: zul_246258304840422492, Target: zul, Predicted: xho +Key: zul_1723972430829964710, Target: zul, Predicted: xho +Key: zul_17352857465328823646, Target: zul, Predicted: xho +Key: zul_17459043251586815775, Target: zul, Predicted: xho +Key: zul_17947549290254644344, Target: zul, Predicted: xho +Key: zul_2639609781275822566, Target: zul, Predicted: xho +Key: zul_3917635891827436019, Target: zul, Predicted: xho +Key: zul_270411754646128444, Target: zul, Predicted: xho +Key: zul_4559834856262506784, Target: zul, Predicted: xho +Key: zul_2773483298866671616, Target: zul, Predicted: xho +Key: zul_4714063675197328407, Target: zul, Predicted: xho +Key: zul_3442800430556158035, Target: zul, Predicted: xho +Key: zul_4197054920430393886, Target: zul, Predicted: xho +Key: zul_4835654573245514019, Target: zul, Predicted: xho +Key: zul_2941140575563461723, Target: zul, Predicted: xho +Key: zul_3601170786569559965, Target: zul, Predicted: xho +Key: zul_4962184807556646656, Target: zul, Predicted: xho +Key: zul_4450539344786232483, Target: zul, Predicted: xho +Key: zul_3084409340625402276, Target: zul, Predicted: nso +Key: zul_3099857952362101284, Target: zul, Predicted: xho +Key: zul_3834275295619649697, Target: zul, Predicted: xho +Key: zul_5869381307850846219, Target: zul, Predicted: xho +Key: zul_6470983311533884443, Target: zul, Predicted: xho +Key: zul_7009133448207432548, Target: zul, Predicted: xho +Key: zul_650283009783069716, Target: zul, Predicted: nya +Key: zul_529878050342905902, Target: zul, Predicted: nya +Key: zul_5991249782636792071, Target: zul, Predicted: xho +Key: zul_6686769581515224661, Target: zul, Predicted: xho +Key: zul_5429551459349048083, Target: zul, Predicted: xho +Key: zul_6015575423607156616, Target: zul, Predicted: xho +Key: zul_6726370617263138514, Target: zul, Predicted: xho +Key: zul_6066399405890294607, Target: zul, Predicted: xho +Key: zul_7277097140077984872, Target: zul, Predicted: xho +Key: zul_5560032758787608109, Target: zul, Predicted: xho +Key: zul_5667226768747434598, Target: zul, Predicted: xho +Key: zul_5668929967350610644, Target: zul, Predicted: nso +Key: zul_7523767784591847777, Target: zul, Predicted: xho +Key: zul_5669176561009418559, Target: zul, Predicted: xho +Key: zul_5769176005359762548, Target: zul, Predicted: xho +Key: zul_9043419625315175103, Target: zul, Predicted: xho +Key: zul_8524966875470859958, Target: zul, Predicted: xho +Key: zul_7926942854117551875, Target: zul, Predicted: xho +Key: zul_9209346979325226094, Target: zul, Predicted: xho +Key: zul_7942277099338193778, Target: zul, Predicted: xho +Key: zul_9899330977251327686, Target: zul, Predicted: xho +Key: zul_7961033240318097419, Target: zul, Predicted: xho +Key: zul_8760444817559521951, Target: zul, Predicted: xho +Key: zul_805676243542905718, Target: zul, Predicted: xho +Key: zul_879121530691385768, Target: zul, Predicted: xho +Key: zul_8153772986868539824, Target: zul, Predicted: nso +Key: zul_9436652657617642646, Target: zul, Predicted: xho +Key: zul_8221196711015201977, Target: zul, Predicted: xho +Key: zul_9599586213899741723, Target: zul, Predicted: xho +Key: zul_8966644467088601433, Target: zul, Predicted: xho +Key: zul_9020496956091559092, Target: zul, Predicted: xho diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang/lid_inference_test.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang/lid_inference_test.log new file mode 100644 index 0000000000000000000000000000000000000000..4b2a56c6b217945c551d73081995af3c9ffd8ce5 --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang/lid_inference_test.log @@ -0,0 +1,295 @@ +# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +# Started at Mon Jun 2 00:36:46 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/test_voxpopuli_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +[gpue04] 2025-06-02 00:37:06,533 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + torch.load(model_file, map_location=device), +[gpue04] 2025-06-02 00:37:18,559 (lid_inference_dist:86) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary. + warnings.warn(_create_warning_msg( +/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist. + warnings.warn( +[gpue04] 2025-06-02 00:37:19,091 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang/lids0 +[gpue04] 2025-06-02 00:38:18,371 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0 +[gpue04] 2025-06-02 00:39:13,446 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1 +[gpue04] 2025-06-02 00:40:09,054 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2 +[gpue04] 2025-06-02 00:41:05,743 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3 +[gpue04] 2025-06-02 00:42:03,918 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4 +[gpue04] 2025-06-02 00:43:01,470 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5 +[gpue04] 2025-06-02 00:44:00,210 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6 +[gpue04] 2025-06-02 00:45:05,238 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7 +[gpue04] 2025-06-02 00:46:04,557 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8 +[gpue04] 2025-06-02 00:47:20,175 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9 +[gpue04] 2025-06-02 00:48:16,561 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10 +[gpue04] 2025-06-02 00:49:08,874 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11 +[gpue04] 2025-06-02 00:50:08,094 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12 +[gpue04] 2025-06-02 00:51:11,746 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13 +[gpue04] 2025-06-02 00:52:10,238 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14 +[gpue04] 2025-06-02 00:53:15,344 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15 +[gpue04] 2025-06-02 00:54:19,104 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True +[gpue04] 2025-06-02 00:54:19,105 (lid_inference_dist:215) INFO: args.save_tsne_plot: False +# Accounting: time=1054 threads=1 +# Ended (code 0) at Mon Jun 2 00:54:20 CDT 2025, elapsed time 1054 seconds diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang/results b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang/results new file mode 100644 index 0000000000000000000000000000000000000000..94f35dadd4eb741d618508ca831f685d4aa6966d --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/inference/valid.accuracy.best/test_voxpopuli_lang_cross_train_all_no_filter_lang/results @@ -0,0 +1,197 @@ +Accuracy: 98.95% +Macro Accuracy: 99.09% +Accuracy per Language: +hrv: 99.40% +ces: 98.75% +spa: 97.64% +hun: 98.83% +pol: 98.31% +slk: 97.35% +nld: 99.12% +eng: 99.51% +est: 100.00% +ron: 99.49% +slv: 99.35% +ita: 99.24% +lit: 100.00% +fra: 99.31% +deu: 99.29% +fin: 99.79% +Key: ces_20110609-0900-PLENARY-4-cs_20110609-11:20:13_0, Target: ces, Predicted: cym +Key: ces_20130610-0900-PLENARY-15-cs_20130610-20:51:10_5, Target: ces, Predicted: slk +Key: ces_20141126-0900-PLENARY-14-cs_20141126-18:28:01_1, Target: ces, Predicted: slk +Key: ces_20150209-0900-PLENARY-11-cs_20150209-21:09:35_2, Target: ces, Predicted: hun +Key: ces_20170403-0900-PLENARY-17-cs_20170403-20:24:45_0, Target: ces, Predicted: deu +Key: ces_20180614-0900-PLENARY-5-cs_20180614-11:09:11_0, Target: ces, Predicted: pol +Key: ces_20180614-0900-PLENARY-cs_20180614-11:09:11_0, Target: ces, Predicted: pol +Key: ces_20180612-0900-PLENARY-14-cs_20180612-17:15:35_13, Target: ces, Predicted: pol +Key: deu_20090202-0900-PLENARY-13-de_20090202-22:12:47_16, Target: deu, Predicted: ces +Key: ces_20180912-0900-PLENARY-widetrim-cs_20180912-16:32:19_1, Target: ces, Predicted: fra +Key: ces_20180912-0900-PLENARY-widetrim-cs_20180912-16:32:19_2, Target: ces, Predicted: fra +Key: ces_20180912-0900-PLENARY-widetrim-cs_20180912-16:32:19_3, Target: ces, Predicted: fra +Key: ces_20180912-0900-PLENARY-widetrim-cs_20180912-16:32:19_5, Target: ces, Predicted: fra +Key: ces_20180912-0900-PLENARY-widetrim-cs_20180912-16:32:19_6, Target: ces, Predicted: fra +Key: ces_20180612-0900-PLENARY-cs_20180612-17:15:35_14, Target: ces, Predicted: pol +Key: deu_20111116-0900-PLENARY-3-de_20111116-11:38:53_0, Target: deu, Predicted: ltz +Key: deu_20111024-0900-PLENARY-10-de_20111024-17:46:08_0, Target: deu, Predicted: eng +Key: deu_20131021-0900-PLENARY-10-de_20131021-19:11:07_0, Target: deu, Predicted: ell +Key: deu_20131022-0900-PLENARY-4-de_20131022-09:24:30_14, Target: deu, Predicted: hrv +Key: deu_20131022-0900-PLENARY-4-de_20131022-08:42:26_10, Target: deu, Predicted: nld +Key: deu_20160511-0900-PLENARY-14-de_20160511-15:48:52_1, Target: deu, Predicted: fra +Key: deu_20170314-0900-PLENARY-13-de_20170314-20:56:04_2, Target: deu, Predicted: nld +Key: deu_20170613-0900-PLENARY-20-de_20170613-22:55:01_13, Target: deu, Predicted: slv +Key: deu_20171025-0900-PLENARY-21-de_20171025-19:19:40_0, Target: deu, Predicted: ina +Key: deu_20180611-0900-PLENARY-11-de_20180611-18:10:02_0, Target: deu, Predicted: ron +Key: deu_20180912-0900-PLENARY-widetrim-de_20180912-16:34:37_1, Target: deu, Predicted: ell +Key: deu_20180912-0900-PLENARY-widetrim-de_20180912-19:37:22_2, Target: deu, Predicted: fra +Key: deu_20180912-0900-PLENARY-widetrim-de_20180912-20:47:17_1, Target: deu, Predicted: slk +Key: eng_20110310-0900-PLENARY-5-en_20110310-10:53:26_3, Target: eng, Predicted: hun +Key: eng_20120912-0900-PLENARY-9-en_20120912-16:27:37_7, Target: eng, Predicted: slv +Key: eng_20131022-0900-PLENARY-20-en_20131022-22:05:54_6, Target: eng, Predicted: nld +Key: eng_20131023-0900-PLENARY-11-en_20131023-17:16:39_6, Target: eng, Predicted: ces +Key: eng_20171114-0900-PLENARY-14-en_20171114-15:46:05_9, Target: eng, Predicted: ron +Key: eng_20180911-0900-PLENARY-witholdRO-en_20180911-18:37:21_2, Target: eng, Predicted: deu +Key: eng_20180613-0900-PLENARY-15-en_20180613-15:21:04_16, Target: eng, Predicted: deu +Key: eng_20180613-0900-PLENARY-15-en_20180613-15:21:04_6, Target: eng, Predicted: deu +Key: eng_20200914-0900-PLENARY-en_20200914-21:39:43_1, Target: eng, Predicted: slv +Key: fin_20140313-0900-PLENARY-14-fi_20140313-13:36:53_0, Target: fin, Predicted: ell +Key: fra_20111130-0900-PLENARY-11-fr_20111130-16:35:45_18, Target: fra, Predicted: nld +Key: fra_20111130-0900-PLENARY-11-fr_20111130-16:35:45_19, Target: fra, Predicted: pol +Key: fra_20131022-0900-PLENARY-14-fr_20131022-16:32:57_5, Target: fra, Predicted: ron +Key: fra_20140225-0900-PLENARY-11-fr_20140225-15:56:55_0, Target: fra, Predicted: deu +Key: fra_20140312-0900-PLENARY-15-fr_20140312-20:54:27_9, Target: fra, Predicted: ell +Key: fra_20160704-0900-PLENARY-13-fr_20160704-20:03:29_0, Target: fra, Predicted: nno +Key: fra_20170912-0900-PLENARY-21-fr_20170912-20:09:57_0, Target: fra, Predicted: deu +Key: fra_20180530-0900-PLENARY-3-fr_20180530-11:02:02_4, Target: fra, Predicted: ron +Key: fra_20180912-0900-PLENARY-widetrim-fr_20180912-19:32:09_2, Target: fra, Predicted: slk +Key: fra_20180912-0900-PLENARY-widetrim-fr_20180912-19:32:09_3, Target: fra, Predicted: fin +Key: fra_20180912-0900-PLENARY-widetrim-fr_20180912-19:32:09_5, Target: fra, Predicted: fin +Key: fra_20201019-0900-PLENARY-fr_20201019-19:35:21_8, Target: fra, Predicted: ita +Key: hrv_20140114-0900-PLENARY-6-hr_20140114-13:40:47_0, Target: hrv, Predicted: eng +Key: hrv_20151216-0900-PLENARY-16-hr_20151216-20:01:08_3, Target: hrv, Predicted: ita +Key: hrv_20170213-0900-PLENARY-18-hr_20170213-22:14:46_3, Target: hrv, Predicted: nld +Key: hun_20090203-0900-PLENARY-13-hu_20090203-21:55:15_8, Target: hun, Predicted: slv +Key: hun_20090204-0900-PLENARY-3-hu_20090204-10:53:37_0, Target: hun, Predicted: fra +Key: hrv_20181022-0900-PLENARY-hr_20181022-22:55:28_7, Target: hrv, Predicted: bos +Key: hun_20110117-0900-PLENARY-14-hu_20110117-21:41:35_0, Target: hun, Predicted: ita +Key: hun_20120313-0900-PLENARY-10-hu_20120313-17:33:54_0, Target: hun, Predicted: isl +Key: hun_20120313-0900-PLENARY-6-hu_20120313-12:45:53_0, Target: hun, Predicted: ita +Key: hun_20160414-0900-PLENARY-10-hu_20160414-13:26:41_0, Target: hun, Predicted: fin +Key: hun_20171212-0900-PLENARY-15-hu_20171212-16:18:30_0, Target: hun, Predicted: ron +Key: hun_20170705-0900-PLENARY-8-hu_20170705-12:19:50_0, Target: hun, Predicted: fra +Key: ita_20130521-0900-PLENARY-10-it_20130521-17:59:38_11, Target: ita, Predicted: spa +Key: hun_20180912-0900-PLENARY-widetrim-hu_20180912-22:41:34_2, Target: hun, Predicted: eng +Key: hun_20180912-0900-PLENARY-widetrim-hu_20180912-22:41:34_3, Target: hun, Predicted: eng +Key: hun_20180912-0900-PLENARY-widetrim-hu_20180912-22:41:34_4, Target: hun, Predicted: eng +Key: hun_20180912-0900-PLENARY-widetrim-hu_20180912-22:41:34_5, Target: hun, Predicted: eng +Key: hun_20180912-0900-PLENARY-widetrim-hu_20180912-22:41:34_6, Target: hun, Predicted: eng +Key: ita_20140702-0900-PLENARY-12-it_20140702-16:59:58_1, Target: ita, Predicted: spa +Key: ita_20140225-0900-PLENARY-6-it_20140225-13:50:00_0, Target: ita, Predicted: fra +Key: ita_20140226-0900-PLENARY-3-it_20140226-09:26:22_1, Target: ita, Predicted: fra +Key: ita_20140226-0900-PLENARY-3-it_20140226-09:26:22_7, Target: ita, Predicted: spa +Key: ita_20141021-0900-PLENARY-4-it_20141021-09:37:26_4, Target: ita, Predicted: ces +Key: ita_20151125-0900-PLENARY-7-it_20151125-12:02:07_15, Target: ita, Predicted: deu +Key: ita_20151125-0900-PLENARY-7-it_20151125-12:02:07_12, Target: ita, Predicted: ces +Key: ita_20180613-0900-PLENARY-17-it_20180613-17:05:13_2, Target: ita, Predicted: spa +Key: nld_20090311-0900-PLENARY-20-nl_20090311-21:13:31_16, Target: nld, Predicted: ces +Key: nld_20101019-0900-PLENARY-11-nl_20101019-18:14:25_20, Target: nld, Predicted: azz +Key: nld_20100120-0900-PLENARY-13-nl_20100120-21:54:24_0, Target: nld, Predicted: ell +Key: nld_20140116-0900-PLENARY-7-nl_20140116-12:49:10_0, Target: nld, Predicted: hun +Key: nld_20141021-0900-PLENARY-11-nl_20141021-17:19:09_13, Target: nld, Predicted: fra +Key: nld_20141021-0900-PLENARY-16-nl_20141021-22:35:16_6, Target: nld, Predicted: swe +Key: nld_20170613-0900-PLENARY-13-nl_20170613-16:02:56_0, Target: nld, Predicted: fra +Key: nld_20180315-0900-PLENARY-3-nl_20180315-09:28:48_0, Target: nld, Predicted: hun +Key: nld_20180912-0900-PLENARY-widetrim-nl_20180912-16:04:33_1, Target: nld, Predicted: deu +Key: nld_20180912-0900-PLENARY-widetrim-nl_20180912-16:04:33_3, Target: nld, Predicted: spa +Key: pol_20090324-0900-PLENARY-3-pl_20090324-09:54:57_10, Target: pol, Predicted: slk +Key: pol_20091124-0900-PLENARY-19-pl_20091124-23:31:52_0, Target: pol, Predicted: ukr +Key: pol_20091124-0900-PLENARY-19-pl_20091124-23:31:52_1, Target: pol, Predicted: ukr +Key: pol_20091124-0900-PLENARY-19-pl_20091124-23:31:52_2, Target: pol, Predicted: ukr +Key: pol_20091124-0900-PLENARY-19-pl_20091124-23:31:52_4, Target: pol, Predicted: ukr +Key: pol_20110512-0900-PLENARY-3-pl_20110512-11:04:51_2, Target: pol, Predicted: bel +Key: pol_20110512-0900-PLENARY-3-pl_20110512-11:04:51_3, Target: pol, Predicted: bel +Key: pol_20110705-0900-PLENARY-5-pl_20110705-12:17:25_0, Target: pol, Predicted: ita +Key: pol_20110706-0900-PLENARY-4-pl_20110706-13:12:02_0, Target: pol, Predicted: ita +Key: pol_20110915-0900-PLENARY-3-pl_20110915-09:28:56_3, Target: pol, Predicted: ukr +Key: pol_20111116-0900-PLENARY-9-pl_20111116-17:01:22_0, Target: pol, Predicted: eng +Key: pol_20111026-0900-PLENARY-15-pl_20111026-18:51:55_8, Target: pol, Predicted: ces +Key: pol_20111213-0900-PLENARY-10-pl_20111213-16:58:12_1, Target: pol, Predicted: bel +Key: pol_20111213-0900-PLENARY-5-pl_20111213-11:07:05_3, Target: pol, Predicted: ukr +Key: pol_20111213-0900-PLENARY-5-pl_20111213-11:07:05_4, Target: pol, Predicted: ukr +Key: pol_20120201-0900-PLENARY-13-pl_20120201-20:56:53_1, Target: pol, Predicted: bel +Key: pol_20120201-0900-PLENARY-13-pl_20120201-20:56:53_2, Target: pol, Predicted: bel +Key: pol_20120201-0900-PLENARY-13-pl_20120201-20:56:53_3, Target: pol, Predicted: bel +Key: pol_20120201-0900-PLENARY-13-pl_20120201-20:56:53_5, Target: pol, Predicted: bel +Key: pol_20120201-0900-PLENARY-13-pl_20120201-20:56:53_6, Target: pol, Predicted: ukr +Key: pol_20120611-0900-PLENARY-13-pl_20120611-17:23:16_0, Target: pol, Predicted: aze +Key: pol_20120614-0900-PLENARY-5-pl_20120614-11:12:48_3, Target: pol, Predicted: ukr +Key: pol_20131008-0900-PLENARY-3-pl_20131008-10:35:17_0, Target: pol, Predicted: nld +Key: pol_20131021-0900-PLENARY-12-pl_20131021-20:34:35_0, Target: pol, Predicted: eng +Key: pol_20151125-0900-PLENARY-16-pl_20151125-16:59:57_0, Target: pol, Predicted: hun +Key: pol_20170313-0900-PLENARY-11-pl_20170313-19:41:51_0, Target: pol, Predicted: deu +Key: pol_20180115-0900-PLENARY-11-pl_20180115-19:09:08_0, Target: pol, Predicted: ell +Key: pol_20180207-0900-PLENARY-9-pl_20180207-13:13:22_0, Target: pol, Predicted: deu +Key: pol_20171115-0900-PLENARY-4-pl_20171115-09:24:40_56, Target: pol, Predicted: fra +Key: pol_20180313-0900-PLENARY-18-pl_20180313-21:30:24_0, Target: pol, Predicted: spa +Key: pol_20180704-0900-PLENARY-pl_20180704-11:34:24_0, Target: pol, Predicted: sna +Key: ron_20090309-0900-PLENARY-14-ro_20090309-21:35:00_0, Target: ron, Predicted: deu +Key: ron_20090310-0900-PLENARY-19-ro_20090310-21:35:23_0, Target: ron, Predicted: eng +Key: ron_20130312-0900-PLENARY-5-ro_20130312-10:44:27_4, Target: ron, Predicted: slv +Key: ron_20140416-0900-PLENARY-4-ro_20140416-11:26:09_0, Target: ron, Predicted: hun +Key: ron_20180207-0900-PLENARY-17-ro_20180207-18:27:19_0, Target: ron, Predicted: ell +Key: ron_20180207-0900-PLENARY-17-ro_20180207-17:49:18_13, Target: ron, Predicted: ell +Key: ron_20180613-0900-PLENARY-6-ro_20180613-12:38:55_0, Target: ron, Predicted: ita +Key: slk_20090310-0900-PLENARY-9-sk_20090310-13:41:29_0, Target: slk, Predicted: eng +Key: slk_20091124-0900-PLENARY-19-sk_20091124-23:19:02_13, Target: slk, Predicted: ces +Key: slk_20091124-0900-PLENARY-19-sk_20091124-23:19:02_14, Target: slk, Predicted: ces +Key: slk_20130312-0900-PLENARY-11-sk_20130312-14:03:05_5, Target: slk, Predicted: ces +Key: slk_20091124-0900-PLENARY-19-sk_20091124-23:19:02_2, Target: slk, Predicted: ces +Key: slk_20090421-0900-PLENARY-23-sk_20090421-23:31:18_16, Target: slk, Predicted: ces +Key: slk_20091124-0900-PLENARY-19-sk_20091124-23:19:02_9, Target: slk, Predicted: ces +Key: slk_20131210-0900-PLENARY-11-sk_20131210-14:35:10_1, Target: slk, Predicted: ces +Key: slk_20131120-0900-PLENARY-12-sk_20131120-14:39:04_0, Target: slk, Predicted: ita +Key: slk_20150908-0900-PLENARY-12-sk_20150908-16:31:31_0, Target: slk, Predicted: nno +Key: slk_20150211-0900-PLENARY-10-sk_20150211-16:15:34_12, Target: slk, Predicted: ces +Key: slk_20151124-0900-PLENARY-13-sk_20151124-20:29:15_5, Target: slk, Predicted: ces +Key: slk_20180312-0900-PLENARY-20-sk_20180312-22:35:26_1, Target: slk, Predicted: ces +Key: slk_20180612-0900-PLENARY-8-sk_20180612-13:22:33_3, Target: slk, Predicted: ces +Key: slk_20180612-0900-PLENARY-sk_20180612-13:22:33_4, Target: slk, Predicted: ces +Key: slk_20201021-0900-PLENARY-sk_20201021-16:00:55_13, Target: slk, Predicted: ces +Key: slv_20171114-0900-PLENARY-14-sl_20171114-16:22:58_11, Target: slv, Predicted: hrv +Key: slv_20170704-0900-PLENARY-22-sl_20170704-23:03:46_0, Target: slv, Predicted: deu +Key: spa_20090203-0900-PLENARY-14-es_20090203-22:21:14_10, Target: spa, Predicted: ron +Key: spa_20090505-0900-PLENARY-3-es_20090505-09:56:00_7, Target: spa, Predicted: slv +Key: spa_20091215-0900-PLENARY-14-es_20091215-22:05:17_2, Target: spa, Predicted: ita +Key: spa_20100120-0900-PLENARY-5-es_20100120-12:42:38_2, Target: spa, Predicted: lit +Key: spa_20100615-0900-PLENARY-14-es_20100615-21:30:17_28, Target: spa, Predicted: ita +Key: spa_20140114-0900-PLENARY-6-es_20140114-13:41:54_0, Target: spa, Predicted: ita +Key: spa_20140114-0900-PLENARY-6-es_20140114-13:41:54_2, Target: spa, Predicted: ita +Key: spa_20141126-0900-PLENARY-13-es_20141126-16:38:25_2, Target: spa, Predicted: eng +Key: spa_20141126-0900-PLENARY-13-es_20141126-16:38:25_3, Target: spa, Predicted: eng +Key: spa_20141126-0900-PLENARY-13-es_20141126-16:38:25_4, Target: spa, Predicted: eng +Key: spa_20151007-0900-PLENARY-6-es_20151007-12:04:24_3, Target: spa, Predicted: fra +Key: spa_20151007-0900-PLENARY-6-es_20151007-12:04:24_5, Target: spa, Predicted: deu +Key: spa_20151007-0900-PLENARY-7-es_20151007-12:04:24_3, Target: spa, Predicted: deu +Key: spa_20151007-0900-PLENARY-7-es_20151007-12:04:24_5, Target: spa, Predicted: deu +Key: spa_20170216-0900-PLENARY-3-es_20170216-09:41:17_6, Target: spa, Predicted: ces +Key: spa_20170216-0900-PLENARY-5-es_20170216-10:42:11_1, Target: spa, Predicted: ces +Key: spa_20170404-0900-PLENARY-18-es_20170404-18:36:59_2, Target: spa, Predicted: deu +Key: spa_20170704-0900-PLENARY-21-es_20170704-21:50:24_2, Target: spa, Predicted: ita +Key: spa_20170704-0900-PLENARY-21-es_20170704-21:50:24_3, Target: spa, Predicted: ita +Key: spa_20170704-0900-PLENARY-21-es_20170704-21:50:24_4, Target: spa, Predicted: ita +Key: spa_20170704-0900-PLENARY-21-es_20170704-21:50:24_5, Target: spa, Predicted: ita +Key: spa_20171004-0900-PLENARY-3-es_20171004-10:28:48_0, Target: spa, Predicted: deu +Key: spa_20180529-0900-PLENARY-18-es_20180529-16:25:55_8, Target: spa, Predicted: ces +Key: spa_20180912-0900-PLENARY-widetrim-es_20180912-22:22:18_2, Target: spa, Predicted: deu +Key: spa_20180912-0900-PLENARY-widetrim-es_20180912-22:22:18_3, Target: spa, Predicted: deu +Key: spa_20180912-0900-PLENARY-widetrim-es_20180912-22:22:18_4, Target: spa, Predicted: deu +Key: spa_20180612-0900-PLENARY-14-es_20180612-17:34:51_0, Target: spa, Predicted: ita +Key: spa_20180912-0900-PLENARY-widetrim-es_20180912-22:22:18_5, Target: spa, Predicted: deu +Key: spa_20180612-0900-PLENARY-14-es_20180612-17:34:51_3, Target: spa, Predicted: ita +Key: spa_20180612-0900-PLENARY-14-es_20180612-17:34:51_4, Target: spa, Predicted: ita +Key: spa_20180612-0900-PLENARY-14-es_20180612-17:34:51_5, Target: spa, Predicted: ita +Key: spa_20180612-0900-PLENARY-es_20180612-17:34:51_0, Target: spa, Predicted: ita +Key: spa_20180612-0900-PLENARY-es_20180612-17:34:51_3, Target: spa, Predicted: ita +Key: spa_20180612-0900-PLENARY-es_20180612-17:34:51_4, Target: spa, Predicted: ita +Key: spa_20180612-0900-PLENARY-es_20180612-17:34:51_5, Target: spa, Predicted: ita +Key: spa_20180912-0900-PLENARY-widetrim-es_20180912-22:22:18_1, Target: spa, Predicted: deu diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/lid_inference_test.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/lid_inference_test.log new file mode 100644 index 0000000000000000000000000000000000000000..bd0aa4d0899bd08f066567e940b6644d0705dc5a --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/lid_inference_test.log @@ -0,0 +1,300 @@ +# python3 -m espnet2.bin.lid_inference_dist --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +# Started at Mon Jun 2 02:37:15 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_inference_dist.py --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang --dtype float32 --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/wav.scp,speech,sound --data_path_and_name_and_type dump/raw/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/utt2spk,lid_labels,text --valid_batch_size 4 --lid_train_config exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml --lid_model_file exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/valid.accuracy.best.pth --use_preprocessor true --fix_duration false --num_workers 32 --extract_embd false --save_every 1000 --resume true --save_embd_per_utt true --save_embd_avg_lang true --save_tsne_plot false --ngpu 1 --multiprocessing_distributed True +[gpue04] 2025-06-02 02:37:35,038 (abs_task:2406) INFO: config file: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py:2429: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + torch.load(model_file, map_location=device), +[gpue04] 2025-06-02 02:37:46,607 (lid_inference_dist:86) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary. + warnings.warn(_create_warning_msg( +/work/nvme/bbjs/qwang20/espnet/espnet2/train/reporter.py:321: UserWarning: The stats of the previous epoch=-1doesn't exist. + warnings.warn( +[gpue04] 2025-06-02 02:37:47,156 (lid_trainer:102) INFO: [Rank 0] Resume: 0 utterances found in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/inference/valid.accuracy.best/dev_babel_over_10s_lang_cross_train_all_no_filter_lang/lids0 +[gpue04] 2025-06-02 02:38:41,828 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 0 +[gpue04] 2025-06-02 02:39:27,483 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 1 +[gpue04] 2025-06-02 02:40:15,909 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 2 +[gpue04] 2025-06-02 02:41:08,571 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 3 +[gpue04] 2025-06-02 02:41:56,182 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 4 +[gpue04] 2025-06-02 02:42:40,736 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 5 +[gpue04] 2025-06-02 02:43:27,814 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 6 +[gpue04] 2025-06-02 02:44:10,740 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 7 +[gpue04] 2025-06-02 02:44:52,065 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 8 +[gpue04] 2025-06-02 02:45:40,635 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 9 +[gpue04] 2025-06-02 02:46:28,394 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 10 +[gpue04] 2025-06-02 02:47:09,502 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 11 +[gpue04] 2025-06-02 02:47:59,978 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 12 +[gpue04] 2025-06-02 02:48:52,866 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 13 +[gpue04] 2025-06-02 02:49:41,279 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 14 +[gpue04] 2025-06-02 02:50:32,817 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 15 +[gpue04] 2025-06-02 02:51:20,444 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 16 +[gpue04] 2025-06-02 02:52:09,714 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 17 +[gpue04] 2025-06-02 02:52:55,108 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 18 +[gpue04] 2025-06-02 02:53:50,212 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 19 +[gpue04] 2025-06-02 02:54:31,533 (lid_trainer:207) INFO: [Rank 0] Saved 1000 utts at step 20 +[gpue04] 2025-06-02 02:55:19,223 (lid_inference_dist:200) INFO: args.save_embd_per_utt: True +[gpue04] 2025-06-02 02:55:19,224 (lid_inference_dist:215) INFO: args.save_tsne_plot: False +# Accounting: time=1085 threads=1 +# Ended (code 0) at Mon Jun 2 02:55:20 CDT 2025, elapsed time 1085 seconds diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.1.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.1.log new file mode 100644 index 0000000000000000000000000000000000000000..0dd5ff7331db7cf5692e297ea7e31443e168d6da --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.1.log @@ -0,0 +1,390 @@ +# python3 -m espnet2.bin.lid_train --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True +# Started at Wed Jun 4 20:24:52 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_train.py --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +[gpue08] 2025-06-04 20:25:25,391 (abs_task:1420) INFO: pytorch.version=2.4.0+cu118, cuda.available=True, cudnn.version=90100, cudnn.benchmark=True, cudnn.deterministic=False +[gpue08] 2025-06-04 20:25:25,398 (abs_task:1421) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +[gpue08] 2025-06-04 20:25:25,398 (abs_task:1424) INFO: Optimizer: +Adam ( +Parameter Group 0 + amsgrad: False + betas: [0.9, 0.98] + capturable: False + differentiable: False + eps: 1e-08 + foreach: None + fused: None + initial_lr: 1e-05 + lr: 6.0032e-06 + maximize: False + weight_decay: 0 +) +[gpue08] 2025-06-04 20:25:25,398 (abs_task:1425) INFO: Scheduler: TristageLR(warmup_steps=1250)(hold_steps=5000)(decay_steps=6250)(init_lr_scale=0.6)(final_lr_scale=0.1)(decay_factor=0.00036841361487904725) +[gpue08] 2025-06-04 20:25:25,404 (abs_task:1434) INFO: Saving the configuration in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/config.yaml +[gpue08] 2025-06-04 20:25:25,693 (preprocessor:2245) INFO: Using lang2vec geo +# Accounting: time=218 threads=1 +# Ended (code 0) at Wed Jun 4 20:25:32 CDT 2025, elapsed time 218 seconds +[gpue08] 2025-06-04 20:25:41,611 (abs_task:1899) WARNING: Reading dump/raw/train_all_no_filter_lang/category2utt +[gpue08] 2025-06-04 20:25:41,660 (abs_task:1946) WARNING: Reading dump/raw/train_all_no_filter_lang/dataset2utt +[gpue08] 2025-06-04 20:25:41,663 (abs_task:1962) WARNING: Reading dump/raw/train_all_no_filter_lang/utt2dataset +[gpue08] 2025-06-04 20:27:58,237 (abs_task:1997) INFO: [train] dataset: +ESPnetDataset( + speech: {"path": "dump/raw/train_all_no_filter_lang/wav.scp", "type": "sound"} + lid_labels: {"path": "dump/raw/train_all_no_filter_lang/utt2spk", "type": "text"} + preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)) +[gpue08] 2025-06-04 20:27:58,256 (abs_task:1998) INFO: [train] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False) +[gpue08] 2025-06-04 20:27:58,256 (abs_task:1999) INFO: [train] collate_fn: (float_pad_value=0.0, int_pad_value=0.0) +[gpue08] 2025-06-04 20:27:58,256 (abs_task:2000) INFO: [train] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=727476, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3) +[gpue08] 2025-06-04 20:27:58,323 (abs_task:2001) INFO: [train] mini-batch sizes summary: N-batch=727476, mean=6.0, min=1, max=6 +[gpue08] 2025-06-04 20:27:58,742 (preprocessor:2245) INFO: Using lang2vec geo +[gpue08] 2025-06-04 20:28:11,299 (abs_task:1899) WARNING: Reading dump/raw/dev_ml_superb2_lang/category2utt +[gpue08] 2025-06-04 20:28:11,301 (abs_task:1946) WARNING: Reading dump/raw/dev_ml_superb2_lang/dataset2utt +[gpue08] 2025-06-04 20:28:11,302 (abs_task:1962) WARNING: Reading dump/raw/dev_ml_superb2_lang/utt2dataset +[gpue08] 2025-06-04 20:28:12,337 (abs_task:1997) INFO: [valid] dataset: +ESPnetDataset( + speech: {"path": "dump/raw/dev_ml_superb2_lang/wav.scp", "type": "sound"} + lid_labels: {"path": "dump/raw/dev_ml_superb2_lang/utt2spk", "type": "text"} + preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)) +[gpue08] 2025-06-04 20:28:12,337 (abs_task:1998) INFO: [valid] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False) +[gpue08] 2025-06-04 20:28:12,338 (abs_task:1999) INFO: [valid] collate_fn: (float_pad_value=0.0, int_pad_value=0.0) +[gpue08] 2025-06-04 20:28:12,338 (abs_task:2000) INFO: [valid] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=4722, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3) +[gpue08] 2025-06-04 20:28:12,338 (abs_task:2001) INFO: [valid] mini-batch sizes summary: N-batch=4722, mean=6.0, min=4, max=6 +wandb: Currently logged in as: qingzhew (qingzhew-carnegie-mellon-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin +wandb: Tracking run with wandb version 0.19.10 +wandb: Run data is saved locally in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_202812-6dkg2ayp +wandb: Run `wandb offline` to turn off syncing. +wandb: Syncing run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch +wandb: ⭐️ View project at https://wandb.ai/qingzhew-carnegie-mellon-university/lid +wandb: 🚀 View run at https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/6dkg2ayp +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:218: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead. + scaler = GradScaler() +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:159: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + states = torch.load( +[gpue08] 2025-06-04 20:28:22,171 (trainer:176) INFO: The training was resumed using exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/checkpoint.pth +[gpue08] 2025-06-04 20:28:22,239 (trainer:251) INFO: Frontend featurizer weights for each layer: +Parameter containing: +tensor([-0.0056, -0.0141, -0.0168, -0.0187, -0.0203, -0.0225, -0.0231, -0.0246, + -0.0253, -0.0252, -0.0254, -0.0241, -0.0226, -0.0200, -0.0162, -0.0120, + -0.0095, -0.0059, -0.0017, 0.0058, 0.0097, 0.0142, 0.0175, 0.0196, + 0.0211, 0.0224, 0.0228, 0.0230, 0.0226, 0.0224, 0.0215, 0.0210, + 0.0196, 0.0176, 0.0157, 0.0126, 0.0095, 0.0070, 0.0051, 0.0037, + 0.0020, -0.0003, -0.0030, -0.0056, -0.0076, -0.0090, -0.0096, -0.0102, + -0.0102], device='cuda:0', requires_grad=True) +[gpue08] 2025-06-04 20:28:22,239 (trainer:267) INFO: Error: 'Linear' object is not subscriptable +[gpue08] 2025-06-04 20:28:22,240 (trainer:272) INFO: cos_mp: 1.0 +[gpue08] 2025-06-04 20:28:22,240 (trainer:273) INFO: easy_margin: False +[gpue08] 2025-06-04 20:28:22,240 (trainer:281) WARNING: The training has already reached at max_epoch: 34 +[gpue08] 2025-06-04 20:28:22,253 (trainer:541) INFO: The training was finished at 33 epochs +[gpue08] 2025-06-04 20:28:22,253 (average_nbest_models:69) INFO: Averaging 2best models: criterion="valid.accuracy": exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/valid.accuracy.ave_2best.pth +/work/nvme/bbjs/qwang20/espnet/espnet2/main_funcs/average_nbest_models.py:77: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + _loaded[e] = torch.load( +[gpue08] 2025-06-04 20:28:27,695 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.32.attention.2.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,695 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.36.attention.2.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,696 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.40.attention.2.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,697 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.44.attention.2.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,697 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.32.bn.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,698 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.36.bn.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,698 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.40.bn.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,699 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.44.bn.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,701 (average_nbest_models:96) INFO: Accumulating encoder.bn.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,701 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn1.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,701 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.0.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.1.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.2.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.3.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.4.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.5.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.6.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn3.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,702 (average_nbest_models:96) INFO: Accumulating encoder.layer1.se.se.3.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn1.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.0.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.1.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.2.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.3.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.4.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,703 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.5.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,704 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.6.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,704 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn3.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,704 (average_nbest_models:96) INFO: Accumulating encoder.layer2.se.se.3.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,704 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn1.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,704 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.0.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.1.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.2.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.3.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.4.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.5.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.6.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn3.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,705 (average_nbest_models:96) INFO: Accumulating encoder.layer3.se.se.3.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,707 (average_nbest_models:96) INFO: Accumulating pooling.attention.2.num_batches_tracked instead of averaging +[gpue08] 2025-06-04 20:28:27,707 (average_nbest_models:96) INFO: Accumulating projector.bn.num_batches_tracked instead of averaging +wandb: +wandb: 🚀 View run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/6dkg2ayp +wandb: ⭐️ View project at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid +wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) +wandb: Find logs at: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_202812-6dkg2ayp/logs +# Accounting: time=221 threads=1 +# Ended (code 0) at Wed Jun 4 20:28:33 CDT 2025, elapsed time 221 seconds diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.2.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.2.log new file mode 100644 index 0000000000000000000000000000000000000000..07a4d8a4c27bfc13583aec480a3f36330f1f998b --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.2.log @@ -0,0 +1,441 @@ +# python3 -m espnet2.bin.lid_train --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True +# Started at Wed Jun 4 20:21:54 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_train.py --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +[gpue06] 2025-06-04 20:22:27,532 (abs_task:1420) INFO: pytorch.version=2.4.0+cu118, cuda.available=True, cudnn.version=90100, cudnn.benchmark=True, cudnn.deterministic=False +[gpue06] 2025-06-04 20:22:27,538 (abs_task:1421) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +[gpue06] 2025-06-04 20:22:27,538 (abs_task:1424) INFO: Optimizer: +Adam ( +Parameter Group 0 + amsgrad: False + betas: [0.9, 0.98] + capturable: False + differentiable: False + eps: 1e-08 + foreach: None + fused: None + initial_lr: 1e-05 + lr: 6.0032e-06 + maximize: False + weight_decay: 0 +) +[gpue06] 2025-06-04 20:22:27,538 (abs_task:1425) INFO: Scheduler: TristageLR(warmup_steps=1250)(hold_steps=5000)(decay_steps=6250)(init_lr_scale=0.6)(final_lr_scale=0.1)(decay_factor=0.00036841361487904725) +[gpue06] 2025-06-04 20:22:27,544 (abs_task:1434) INFO: Saving the configuration in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/config.yaml +[gpue06] 2025-06-04 20:22:27,823 (preprocessor:2245) INFO: Using lang2vec geo +[gpue06] 2025-06-04 20:22:43,726 (abs_task:1899) WARNING: Reading dump/raw/train_all_no_filter_lang/category2utt +[gpue06] 2025-06-04 20:22:43,727 (abs_task:1946) WARNING: Reading dump/raw/train_all_no_filter_lang/dataset2utt +[gpue06] 2025-06-04 20:22:43,729 (abs_task:1962) WARNING: Reading dump/raw/train_all_no_filter_lang/utt2dataset +[gpue06] 2025-06-04 20:24:57,630 (abs_task:1997) INFO: [train] dataset: +ESPnetDataset( + speech: {"path": "dump/raw/train_all_no_filter_lang/wav.scp", "type": "sound"} + lid_labels: {"path": "dump/raw/train_all_no_filter_lang/utt2spk", "type": "text"} + preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)) +[gpue06] 2025-06-04 20:24:57,648 (abs_task:1998) INFO: [train] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False) +[gpue06] 2025-06-04 20:24:57,648 (abs_task:1999) INFO: [train] collate_fn: (float_pad_value=0.0, int_pad_value=0.0) +[gpue06] 2025-06-04 20:24:57,649 (abs_task:2000) INFO: [train] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=727472, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3) +[gpue06] 2025-06-04 20:24:57,715 (abs_task:2001) INFO: [train] mini-batch sizes summary: N-batch=727472, mean=6.0, min=1, max=6 +[gpue06] 2025-06-04 20:24:58,116 (preprocessor:2245) INFO: Using lang2vec geo +[gpue06] 2025-06-04 20:25:10,662 (abs_task:1899) WARNING: Reading dump/raw/dev_ml_superb2_lang/category2utt +[gpue06] 2025-06-04 20:25:10,664 (abs_task:1946) WARNING: Reading dump/raw/dev_ml_superb2_lang/dataset2utt +[gpue06] 2025-06-04 20:25:10,666 (abs_task:1962) WARNING: Reading dump/raw/dev_ml_superb2_lang/utt2dataset +[gpue06] 2025-06-04 20:25:11,695 (abs_task:1997) INFO: [valid] dataset: +ESPnetDataset( + speech: {"path": "dump/raw/dev_ml_superb2_lang/wav.scp", "type": "sound"} + lid_labels: {"path": "dump/raw/dev_ml_superb2_lang/utt2spk", "type": "text"} + preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)) +[gpue06] 2025-06-04 20:25:11,696 (abs_task:1998) INFO: [valid] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False) +[gpue06] 2025-06-04 20:25:11,696 (abs_task:1999) INFO: [valid] collate_fn: (float_pad_value=0.0, int_pad_value=0.0) +[gpue06] 2025-06-04 20:25:11,696 (abs_task:2000) INFO: [valid] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=4722, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3) +[gpue06] 2025-06-04 20:25:11,696 (abs_task:2001) INFO: [valid] mini-batch sizes summary: N-batch=4722, mean=6.0, min=4, max=6 +wandb: Currently logged in as: qingzhew (qingzhew-carnegie-mellon-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin +wandb: Tracking run with wandb version 0.19.10 +wandb: Run data is saved locally in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_202512-0zfdmaq1 +wandb: Run `wandb offline` to turn off syncing. +wandb: Resuming run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch +wandb: ⭐️ View project at https://wandb.ai/qingzhew-carnegie-mellon-university/lid +wandb: 🚀 View run at https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/0zfdmaq1 +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:218: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead. + scaler = GradScaler() +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:159: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + states = torch.load( +[gpue06] 2025-06-04 20:25:22,045 (trainer:176) INFO: The training was resumed using exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/checkpoint.pth +[gpue06] 2025-06-04 20:25:22,123 (trainer:251) INFO: Frontend featurizer weights for each layer: +Parameter containing: +tensor([-0.0056, -0.0141, -0.0168, -0.0187, -0.0203, -0.0225, -0.0231, -0.0246, + -0.0253, -0.0252, -0.0254, -0.0241, -0.0226, -0.0200, -0.0162, -0.0120, + -0.0095, -0.0059, -0.0017, 0.0058, 0.0097, 0.0142, 0.0175, 0.0196, + 0.0211, 0.0224, 0.0228, 0.0230, 0.0226, 0.0224, 0.0215, 0.0210, + 0.0196, 0.0176, 0.0157, 0.0126, 0.0095, 0.0070, 0.0051, 0.0037, + 0.0020, -0.0003, -0.0030, -0.0056, -0.0076, -0.0090, -0.0096, -0.0102, + -0.0102], device='cuda:0', requires_grad=True) +[gpue06] 2025-06-04 20:25:22,124 (trainer:267) INFO: Error: 'Linear' object is not subscriptable +[gpue06] 2025-06-04 20:25:22,124 (trainer:272) INFO: cos_mp: 1.0 +[gpue06] 2025-06-04 20:25:22,124 (trainer:273) INFO: easy_margin: False +[gpue06] 2025-06-04 20:25:22,124 (trainer:281) WARNING: The training has already reached at max_epoch: 34 +[gpue06] 2025-06-04 20:25:22,135 (trainer:541) INFO: The training was finished at 33 epochs +[gpue06] 2025-06-04 20:25:22,136 (average_nbest_models:69) INFO: Averaging 2best models: criterion="valid.accuracy": exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/valid.accuracy.ave_2best.pth +/work/nvme/bbjs/qwang20/espnet/espnet2/main_funcs/average_nbest_models.py:77: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + _loaded[e] = torch.load( +[gpue06] 2025-06-04 20:25:27,720 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.32.attention.2.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,721 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.36.attention.2.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,722 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.40.attention.2.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,723 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.44.attention.2.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,723 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.32.bn.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,723 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.36.bn.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,724 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.40.bn.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,724 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.44.bn.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,727 (average_nbest_models:96) INFO: Accumulating encoder.bn.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,727 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn1.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,727 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.0.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.1.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.2.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.3.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.4.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.5.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.6.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn3.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,728 (average_nbest_models:96) INFO: Accumulating encoder.layer1.se.se.3.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn1.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.0.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.1.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.2.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.3.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,729 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.4.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,730 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.5.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,730 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.6.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,730 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn3.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,730 (average_nbest_models:96) INFO: Accumulating encoder.layer2.se.se.3.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,730 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn1.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.0.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.1.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.2.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.3.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.4.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.5.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.6.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,731 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn3.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,732 (average_nbest_models:96) INFO: Accumulating encoder.layer3.se.se.3.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,733 (average_nbest_models:96) INFO: Accumulating pooling.attention.2.num_batches_tracked instead of averaging +[gpue06] 2025-06-04 20:25:27,733 (average_nbest_models:96) INFO: Accumulating projector.bn.num_batches_tracked instead of averaging +wandb: +wandb: +wandb: Run summary: +wandb: epoch 50 +wandb: iteration 100000 +wandb: metrics/accuracy 0.95477 +wandb: metrics/backward_time 0.96399 +wandb: metrics/class_loss 1.09721 +wandb: metrics/clip 0 +wandb: metrics/forward_time 0.28474 +wandb: metrics/geo_loss_all 0.10049 +wandb: metrics/geo_loss_downstream 0.15867 +wandb: metrics/grad_norm 59.37694 +wandb: metrics/inter_geo_loss_layer32 0.01366 +wandb: metrics/inter_geo_loss_layer36 0.01344 +wandb: metrics/inter_geo_loss_layer40 0.01298 +wandb: metrics/inter_geo_loss_layer44 0.01279 +wandb: metrics/inter_geo_loss_mean 0.01322 +wandb: metrics/iter_time 0.00022 +wandb: metrics/loss 0.22447 +wandb: metrics/loss_scale 268435456 +wandb: metrics/optim0_lr0 0.0 +wandb: metrics/optim_step_time 0.03651 +wandb: train/train_accuracy_epoch 0.95477 +wandb: train/train_backward_time_epoch 0.96399 +wandb: train/train_class_loss_epoch 1.09721 +wandb: train/train_clip_epoch 0 +wandb: train/train_forward_time_epoch 0.28474 +wandb: train/train_geo_loss_all_epoch 0.10049 +wandb: train/train_geo_loss_downstream_epoch 0.15867 +wandb: train/train_gpu_max_cached_mem_GB_epoch 130.68359 +wandb: train/train_grad_norm_epoch 59.37694 +wandb: train/train_inter_geo_loss_layer32_epoch 0.01366 +wandb: train/train_inter_geo_loss_layer36_epoch 0.01344 +wandb: train/train_inter_geo_loss_layer40_epoch 0.01298 +wandb: train/train_inter_geo_loss_layer44_epoch 0.01279 +wandb: train/train_inter_geo_loss_mean_epoch 0.01322 +wandb: train/train_iter_time_epoch 0.00022 +wandb: train/train_loss_epoch 0.22447 +wandb: train/train_loss_scale_epoch 268435456 +wandb: train/train_optim0_lr0_epoch 0.0 +wandb: train/train_optim_step_time_epoch 0.03651 +wandb: train/train_time 5.07792 +wandb: train/train_train_time_epoch 5.07792 +wandb: valid/valid_accuracy_epoch 0.89594 +wandb: valid/valid_class_loss_epoch 2.57223 +wandb: valid/valid_geo_loss_all_epoch 0.13273 +wandb: valid/valid_geo_loss_downstream_epoch 0.20731 +wandb: valid/valid_gpu_max_cached_mem_GB_epoch 130.68359 +wandb: valid/valid_inter_geo_loss_layer32_epoch 0.01976 +wandb: valid/valid_inter_geo_loss_layer36_epoch 0.02211 +wandb: valid/valid_inter_geo_loss_layer40_epoch 0.02097 +wandb: valid/valid_inter_geo_loss_layer44_epoch 0.02058 +wandb: valid/valid_inter_geo_loss_mean_epoch 0.02086 +wandb: valid/valid_loss_epoch 2.08433 +wandb: +wandb: 🚀 View run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/0zfdmaq1 +wandb: ⭐️ View project at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid +wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) +wandb: Find logs at: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_202512-0zfdmaq1/logs diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.3.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.3.log new file mode 100644 index 0000000000000000000000000000000000000000..f238b58ae55e8a35a6e052bd48dd453741e8b4d4 --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.3.log @@ -0,0 +1,460 @@ +# python3 -m espnet2.bin.lid_train --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True +# Started at Mon Jun 2 08:00:04 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_train.py --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +[gpue01] 2025-06-02 08:00:37,184 (abs_task:1420) INFO: pytorch.version=2.4.0+cu118, cuda.available=True, cudnn.version=90100, cudnn.benchmark=True, cudnn.deterministic=False +[gpue01] 2025-06-02 08:00:37,190 (abs_task:1421) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +[gpue01] 2025-06-02 08:00:37,190 (abs_task:1424) INFO: Optimizer: +Adam ( +Parameter Group 0 + amsgrad: False + betas: [0.9, 0.98] + capturable: False + differentiable: False + eps: 1e-08 + foreach: None + fused: None + initial_lr: 1e-05 + lr: 6.0032e-06 + maximize: False + weight_decay: 0 +) +[gpue01] 2025-06-02 08:00:37,190 (abs_task:1425) INFO: Scheduler: TristageLR(warmup_steps=1250)(hold_steps=5000)(decay_steps=6250)(init_lr_scale=0.6)(final_lr_scale=0.1)(decay_factor=0.00036841361487904725) +[gpue01] 2025-06-02 08:00:37,195 (abs_task:1434) INFO: Saving the configuration in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml +[gpue01] 2025-06-02 08:00:37,476 (preprocessor:2245) INFO: Using lang2vec geo +[gpue01] 2025-06-02 08:00:53,379 (abs_task:1899) WARNING: Reading dump/raw/train_all_no_filter_lang/category2utt +[gpue01] 2025-06-02 08:00:53,380 (abs_task:1946) WARNING: Reading dump/raw/train_all_no_filter_lang/dataset2utt +[gpue01] 2025-06-02 08:00:53,382 (abs_task:1962) WARNING: Reading dump/raw/train_all_no_filter_lang/utt2dataset +[gpue01] 2025-06-02 08:03:08,576 (abs_task:1997) INFO: [train] dataset: +ESPnetDataset( + speech: {"path": "dump/raw/train_all_no_filter_lang/wav.scp", "type": "sound"} + lid_labels: {"path": "dump/raw/train_all_no_filter_lang/utt2spk", "type": "text"} + preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)) +[gpue01] 2025-06-02 08:03:08,577 (abs_task:1998) INFO: [train] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False) +[gpue01] 2025-06-02 08:03:08,577 (abs_task:1999) INFO: [train] collate_fn: (float_pad_value=0.0, int_pad_value=0.0) +[gpue01] 2025-06-02 08:03:08,577 (abs_task:2000) INFO: [train] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=727460, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3) +[gpue01] 2025-06-02 08:03:08,642 (abs_task:2001) INFO: [train] mini-batch sizes summary: N-batch=727460, mean=6.0, min=1, max=6 +[gpue01] 2025-06-02 08:03:09,071 (preprocessor:2245) INFO: Using lang2vec geo +[gpue01] 2025-06-02 08:03:21,631 (abs_task:1899) WARNING: Reading dump/raw/dev_ml_superb2_lang/category2utt +[gpue01] 2025-06-02 08:03:21,632 (abs_task:1946) WARNING: Reading dump/raw/dev_ml_superb2_lang/dataset2utt +[gpue01] 2025-06-02 08:03:21,633 (abs_task:1962) WARNING: Reading dump/raw/dev_ml_superb2_lang/utt2dataset +[gpue01] 2025-06-02 08:03:22,657 (abs_task:1997) INFO: [valid] dataset: +ESPnetDataset( + speech: {"path": "dump/raw/dev_ml_superb2_lang/wav.scp", "type": "sound"} + lid_labels: {"path": "dump/raw/dev_ml_superb2_lang/utt2spk", "type": "text"} + preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)) +[gpue01] 2025-06-02 08:03:22,657 (abs_task:1998) INFO: [valid] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False) +[gpue01] 2025-06-02 08:03:22,657 (abs_task:1999) INFO: [valid] collate_fn: (float_pad_value=0.0, int_pad_value=0.0) +[gpue01] 2025-06-02 08:03:22,657 (abs_task:2000) INFO: [valid] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=4722, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3) +[gpue01] 2025-06-02 08:03:22,658 (abs_task:2001) INFO: [valid] mini-batch sizes summary: N-batch=4722, mean=6.0, min=4, max=6 +wandb: Currently logged in as: qingzhew (qingzhew-carnegie-mellon-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin +wandb: Tracking run with wandb version 0.19.10 +wandb: Run data is saved locally in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/wandb/run-20250602_080323-0zfdmaq1 +wandb: Run `wandb offline` to turn off syncing. +wandb: Resuming run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3 +wandb: ⭐️ View project at https://wandb.ai/qingzhew-carnegie-mellon-university/lid +wandb: 🚀 View run at https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/0zfdmaq1 +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:218: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead. + scaler = GradScaler() +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:159: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + states = torch.load( +[gpue01] 2025-06-02 08:03:32,100 (trainer:176) INFO: The training was resumed using exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/checkpoint.pth +[gpue01] 2025-06-02 08:03:32,199 (trainer:251) INFO: Frontend featurizer weights for each layer: +Parameter containing: +tensor([-0.0056, -0.0140, -0.0167, -0.0186, -0.0202, -0.0224, -0.0230, -0.0245, + -0.0252, -0.0250, -0.0253, -0.0240, -0.0225, -0.0199, -0.0161, -0.0120, + -0.0094, -0.0058, -0.0017, 0.0059, 0.0098, 0.0142, 0.0175, 0.0197, + 0.0211, 0.0224, 0.0228, 0.0230, 0.0225, 0.0223, 0.0215, 0.0209, + 0.0195, 0.0176, 0.0156, 0.0126, 0.0094, 0.0070, 0.0050, 0.0036, + 0.0019, -0.0004, -0.0031, -0.0057, -0.0077, -0.0090, -0.0097, -0.0103, + -0.0103], device='cuda:0', requires_grad=True) +[gpue01] 2025-06-02 08:03:32,200 (trainer:267) INFO: Error: 'Linear' object is not subscriptable +[gpue01] 2025-06-02 08:03:32,200 (trainer:272) INFO: cos_mp: 1.0 +[gpue01] 2025-06-02 08:03:32,200 (trainer:273) INFO: easy_margin: False +[gpue01] 2025-06-02 08:03:32,211 (trainer:347) INFO: 29/50epoch started +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:645: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead. + with autocast( +[gpue01] 2025-06-02 08:08:32,675 (trainer:816) INFO: 29epoch:train:1-100batch: iter_time=0.003, forward_time=0.394, class_loss=1.038, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.213, accuracy=0.960, backward_time=1.214, grad_norm=40.433, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=5.725e-07, train_time=6.534 +[gpue01] 2025-06-02 08:10:53,126 (trainer:816) INFO: 29epoch:train:101-200batch: iter_time=8.669e-05, forward_time=0.352, class_loss=1.234, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.106, loss=0.252, accuracy=0.947, backward_time=1.033, grad_norm=67.550, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=5.672e-07, train_time=5.618 +[gpue01] 2025-06-02 08:13:06,132 (trainer:816) INFO: 29epoch:train:201-300batch: iter_time=8.747e-05, forward_time=0.336, class_loss=1.151, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.012, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.235, accuracy=0.948, backward_time=0.973, grad_norm=55.134, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.620e-07, train_time=5.320 +[gpue01] 2025-06-02 08:15:18,146 (trainer:816) INFO: 29epoch:train:301-400batch: iter_time=9.459e-05, forward_time=0.338, class_loss=1.302, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.266, accuracy=0.950, backward_time=0.962, grad_norm=56.978, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=5.569e-07, train_time=5.280 +[gpue01] 2025-06-02 08:17:18,134 (trainer:816) INFO: 29epoch:train:401-500batch: iter_time=9.884e-05, forward_time=0.300, class_loss=1.251, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.255, accuracy=0.945, backward_time=0.879, grad_norm=50.561, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.518e-07, train_time=4.799 +[gpue01] 2025-06-02 08:19:20,112 (trainer:816) INFO: 29epoch:train:501-600batch: iter_time=9.952e-05, forward_time=0.288, class_loss=0.865, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.178, accuracy=0.968, backward_time=0.911, grad_norm=51.561, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=5.467e-07, train_time=4.879 +[gpue01] 2025-06-02 08:21:24,824 (trainer:816) INFO: 29epoch:train:601-700batch: iter_time=8.929e-05, forward_time=0.286, class_loss=0.922, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.190, accuracy=0.962, backward_time=0.941, grad_norm=48.480, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.417e-07, train_time=4.988 +[gpue01] 2025-06-02 08:23:15,407 (trainer:816) INFO: 29epoch:train:701-800batch: iter_time=1.002e-04, forward_time=0.255, class_loss=1.501, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.305, accuracy=0.943, backward_time=0.830, grad_norm=74.153, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.367e-07, train_time=4.423 +[gpue01] 2025-06-02 08:25:12,844 (trainer:816) INFO: 29epoch:train:801-900batch: iter_time=9.754e-05, forward_time=0.252, class_loss=1.060, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.217, accuracy=0.957, backward_time=0.902, grad_norm=54.887, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=5.318e-07, train_time=4.697 +[gpue01] 2025-06-02 08:27:12,890 (trainer:816) INFO: 29epoch:train:901-1000batch: iter_time=9.747e-05, forward_time=0.244, class_loss=1.193, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.244, accuracy=0.952, backward_time=0.937, grad_norm=42.200, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.269e-07, train_time=4.801 +[gpue01] 2025-06-02 08:29:03,462 (trainer:816) INFO: 29epoch:train:1001-1100batch: iter_time=9.858e-05, forward_time=0.249, class_loss=1.149, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.235, accuracy=0.953, backward_time=0.835, grad_norm=58.891, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.221e-07, train_time=4.422 +[gpue01] 2025-06-02 08:31:01,649 (trainer:816) INFO: 29epoch:train:1101-1200batch: iter_time=1.007e-04, forward_time=0.238, class_loss=1.181, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.241, accuracy=0.949, backward_time=0.922, grad_norm=100.141, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.173e-07, train_time=4.727 +[gpue01] 2025-06-02 08:32:49,871 (trainer:816) INFO: 29epoch:train:1201-1300batch: iter_time=9.155e-05, forward_time=0.239, class_loss=1.129, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.231, accuracy=0.948, backward_time=0.822, grad_norm=46.727, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.126e-07, train_time=4.328 +[gpue01] 2025-06-02 08:34:45,411 (trainer:816) INFO: 29epoch:train:1301-1400batch: iter_time=9.736e-05, forward_time=0.264, class_loss=1.095, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.224, accuracy=0.952, backward_time=0.870, grad_norm=40.216, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=5.079e-07, train_time=4.621 +[gpue01] 2025-06-02 08:36:35,310 (trainer:816) INFO: 29epoch:train:1401-1500batch: iter_time=9.289e-05, forward_time=0.240, class_loss=1.256, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.256, accuracy=0.948, backward_time=0.837, grad_norm=65.161, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.032e-07, train_time=4.395 +[gpue01] 2025-06-02 08:38:33,818 (trainer:816) INFO: 29epoch:train:1501-1600batch: iter_time=9.491e-05, forward_time=0.265, class_loss=1.038, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.213, accuracy=0.965, backward_time=0.900, grad_norm=51.130, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.986e-07, train_time=4.740 +[gpue01] 2025-06-02 08:40:35,271 (trainer:816) INFO: 29epoch:train:1601-1700batch: iter_time=9.165e-05, forward_time=0.272, class_loss=0.837, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.173, accuracy=0.965, backward_time=0.921, grad_norm=52.871, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.940e-07, train_time=4.857 +[gpue01] 2025-06-02 08:42:35,412 (trainer:816) INFO: 29epoch:train:1701-1800batch: iter_time=8.960e-05, forward_time=0.266, class_loss=1.142, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.234, accuracy=0.953, backward_time=0.915, grad_norm=50.981, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=4.895e-07, train_time=4.805 +[gpue01] 2025-06-02 08:44:31,106 (trainer:816) INFO: 29epoch:train:1801-1900batch: iter_time=1.021e-04, forward_time=0.230, class_loss=1.361, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.277, accuracy=0.945, backward_time=0.906, grad_norm=76.043, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.038, optim0_lr0=4.850e-07, train_time=4.627 +[gpue01] 2025-06-02 08:46:36,834 (trainer:816) INFO: 29epoch:train:1901-2000batch: iter_time=8.900e-05, forward_time=0.264, class_loss=1.395, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.284, accuracy=0.945, backward_time=0.973, grad_norm=59.458, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.806e-07, train_time=5.028 +[gpue01] 2025-06-02 09:09:51,218 (trainer:401) INFO: 29epoch results: [train] iter_time=2.479e-04, forward_time=0.279, class_loss=1.155, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.236, accuracy=0.953, backward_time=0.924, grad_norm=57.178, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=5.253e-07, train_time=4.894, time=43 minutes and 4.77 seconds, total_count=58000, gpu_max_cached_mem_GB=79.320, [valid] class_loss=2.591, geo_loss_downstream=0.207, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.133, loss=2.099, accuracy=0.894, time=23 minutes and 14.24 seconds, total_count=136938, gpu_max_cached_mem_GB=79.320 +[gpue01] 2025-06-02 09:10:04,925 (trainer:467) INFO: There are no improvements in this epoch +[gpue01] 2025-06-02 09:10:04,945 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/27epoch.pth, exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/28epoch.pth +[gpue01] 2025-06-02 09:10:04,945 (trainer:335) INFO: 30/50epoch started. Estimated time to finish: 23 hours, 17 minutes and 27.41 seconds +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:645: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead. + with autocast( +[gpue01] 2025-06-02 09:14:40,846 (trainer:816) INFO: 30epoch:train:1-100batch: iter_time=0.008, forward_time=0.415, class_loss=1.226, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.250, accuracy=0.952, backward_time=0.915, grad_norm=53.097, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=4.762e-07, train_time=5.456 +[gpue01] 2025-06-02 09:16:39,711 (trainer:816) INFO: 30epoch:train:101-200batch: iter_time=9.637e-05, forward_time=0.353, class_loss=1.158, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.237, accuracy=0.953, backward_time=0.815, grad_norm=71.507, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.718e-07, train_time=4.754 +[gpue01] 2025-06-02 09:19:02,535 (trainer:816) INFO: 30epoch:train:201-300batch: iter_time=9.507e-05, forward_time=0.368, class_loss=1.156, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.236, accuracy=0.952, backward_time=1.041, grad_norm=46.785, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.675e-07, train_time=5.712 +[gpue01] 2025-06-02 09:21:05,464 (trainer:816) INFO: 30epoch:train:301-400batch: iter_time=1.003e-04, forward_time=0.318, class_loss=1.121, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.229, accuracy=0.951, backward_time=0.891, grad_norm=48.383, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.632e-07, train_time=4.916 +[gpue01] 2025-06-02 09:22:55,038 (trainer:816) INFO: 30epoch:train:401-500batch: iter_time=9.499e-05, forward_time=0.284, class_loss=1.061, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.217, accuracy=0.962, backward_time=0.792, grad_norm=77.788, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.589e-07, train_time=4.382 +[gpue01] 2025-06-02 09:24:57,833 (trainer:816) INFO: 30epoch:train:501-600batch: iter_time=1.014e-04, forward_time=0.290, class_loss=1.085, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.222, accuracy=0.963, backward_time=0.918, grad_norm=59.019, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.547e-07, train_time=4.911 +[gpue01] 2025-06-02 09:26:53,859 (trainer:816) INFO: 30epoch:train:601-700batch: iter_time=9.895e-05, forward_time=0.280, class_loss=1.015, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.208, accuracy=0.965, backward_time=0.858, grad_norm=55.588, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=4.506e-07, train_time=4.640 +[gpue01] 2025-06-02 09:29:00,707 (trainer:816) INFO: 30epoch:train:701-800batch: iter_time=1.006e-04, forward_time=0.274, class_loss=1.328, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.271, accuracy=0.943, backward_time=0.974, grad_norm=65.465, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.464e-07, train_time=5.073 +[gpue01] 2025-06-02 09:31:13,341 (trainer:816) INFO: 30epoch:train:801-900batch: iter_time=1.231e-04, forward_time=0.272, class_loss=1.336, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.272, accuracy=0.947, backward_time=1.035, grad_norm=50.503, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.423e-07, train_time=5.305 +[gpue01] 2025-06-02 09:33:12,121 (trainer:816) INFO: 30epoch:train:901-1000batch: iter_time=1.060e-04, forward_time=0.241, class_loss=1.184, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.242, accuracy=0.955, backward_time=0.926, grad_norm=48.061, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.383e-07, train_time=4.750 +[gpue01] 2025-06-02 09:35:05,485 (trainer:816) INFO: 30epoch:train:1001-1100batch: iter_time=1.136e-04, forward_time=0.232, class_loss=1.135, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.232, accuracy=0.948, backward_time=0.881, grad_norm=41.919, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=4.343e-07, train_time=4.534 +[gpue01] 2025-06-02 09:37:02,151 (trainer:816) INFO: 30epoch:train:1101-1200batch: iter_time=1.137e-04, forward_time=0.245, class_loss=1.438, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.293, accuracy=0.942, backward_time=0.902, grad_norm=75.206, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=4.303e-07, train_time=4.666 +[gpue01] 2025-06-02 09:38:59,107 (trainer:816) INFO: 30epoch:train:1201-1300batch: iter_time=1.021e-04, forward_time=0.252, class_loss=1.081, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.221, accuracy=0.962, backward_time=0.897, grad_norm=52.787, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.263e-07, train_time=4.677 +[gpue01] 2025-06-02 09:40:58,086 (trainer:816) INFO: 30epoch:train:1301-1400batch: iter_time=1.154e-04, forward_time=0.263, class_loss=1.120, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.229, accuracy=0.955, backward_time=0.906, grad_norm=47.309, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.224e-07, train_time=4.758 +[gpue01] 2025-06-02 09:42:58,353 (trainer:816) INFO: 30epoch:train:1401-1500batch: iter_time=1.042e-04, forward_time=0.265, class_loss=1.275, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.260, accuracy=0.942, backward_time=0.918, grad_norm=74.393, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=4.186e-07, train_time=4.810 +[gpue01] 2025-06-02 09:44:43,549 (trainer:816) INFO: 30epoch:train:1501-1600batch: iter_time=1.006e-04, forward_time=0.231, class_loss=1.188, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.243, accuracy=0.950, backward_time=0.799, grad_norm=68.925, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=4.147e-07, train_time=4.207 +[gpue01] 2025-06-02 09:46:43,647 (trainer:816) INFO: 30epoch:train:1601-1700batch: iter_time=9.242e-05, forward_time=0.278, class_loss=1.219, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.249, accuracy=0.947, backward_time=0.903, grad_norm=38.583, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.109e-07, train_time=4.803 +[gpue01] 2025-06-02 09:48:51,075 (trainer:816) INFO: 30epoch:train:1701-1800batch: iter_time=1.070e-04, forward_time=0.275, class_loss=1.158, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.237, accuracy=0.957, backward_time=0.979, grad_norm=72.308, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.072e-07, train_time=5.096 +[gpue01] 2025-06-02 09:50:42,998 (trainer:816) INFO: 30epoch:train:1801-1900batch: iter_time=9.723e-05, forward_time=0.249, class_loss=1.209, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.247, accuracy=0.948, backward_time=0.849, grad_norm=54.531, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.034e-07, train_time=4.476 +[gpue01] 2025-06-02 09:52:36,303 (trainer:816) INFO: 30epoch:train:1901-2000batch: iter_time=9.175e-05, forward_time=0.226, class_loss=1.016, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.208, accuracy=0.953, backward_time=0.887, grad_norm=55.463, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.997e-07, train_time=4.531 +[gpue01] 2025-06-02 10:15:46,744 (trainer:401) INFO: 30epoch results: [train] iter_time=4.751e-04, forward_time=0.281, class_loss=1.175, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.240, accuracy=0.952, backward_time=0.904, grad_norm=57.881, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=4.369e-07, train_time=4.823, time=42 minutes and 31.56 seconds, total_count=60000, gpu_max_cached_mem_GB=81.803, [valid] class_loss=2.597, geo_loss_downstream=0.220, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.141, loss=2.106, accuracy=0.895, time=23 minutes and 10.24 seconds, total_count=141660, gpu_max_cached_mem_GB=81.803 +[gpue01] 2025-06-02 10:15:59,998 (trainer:467) INFO: There are no improvements in this epoch +[gpue01] 2025-06-02 10:16:00,022 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/29epoch.pth +[gpue01] 2025-06-02 10:16:00,023 (trainer:335) INFO: 31/50epoch started. Estimated time to finish: 22 hours, 4 minutes and 38.11 seconds +[gpue01] 2025-06-02 10:20:30,892 (trainer:816) INFO: 31epoch:train:1-100batch: iter_time=0.002, forward_time=0.401, class_loss=1.160, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.237, accuracy=0.952, backward_time=0.896, grad_norm=73.411, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=3.961e-07, train_time=5.300 +[gpue01] 2025-06-02 10:22:38,629 (trainer:816) INFO: 31epoch:train:101-200batch: iter_time=9.710e-05, forward_time=0.364, class_loss=1.277, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.261, accuracy=0.948, backward_time=0.893, grad_norm=65.669, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.924e-07, train_time=5.109 +[gpue01] 2025-06-02 10:24:35,032 (trainer:816) INFO: 31epoch:train:201-300batch: iter_time=9.069e-05, forward_time=0.330, class_loss=1.280, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.261, accuracy=0.948, backward_time=0.813, grad_norm=67.795, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.888e-07, train_time=4.655 +[gpue01] 2025-06-02 10:26:50,531 (trainer:816) INFO: 31epoch:train:301-400batch: iter_time=9.965e-05, forward_time=0.337, class_loss=1.468, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.299, accuracy=0.942, backward_time=0.999, grad_norm=90.267, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=3.853e-07, train_time=5.419 +[gpue01] 2025-06-02 10:28:43,674 (trainer:816) INFO: 31epoch:train:401-500batch: iter_time=9.749e-05, forward_time=0.288, class_loss=1.018, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.209, accuracy=0.960, backward_time=0.823, grad_norm=47.089, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.817e-07, train_time=4.525 +[gpue01] 2025-06-02 10:30:42,003 (trainer:816) INFO: 31epoch:train:501-600batch: iter_time=1.059e-04, forward_time=0.283, class_loss=1.170, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.239, accuracy=0.953, backward_time=0.879, grad_norm=56.840, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.782e-07, train_time=4.732 +[gpue01] 2025-06-02 10:32:49,253 (trainer:816) INFO: 31epoch:train:601-700batch: iter_time=9.643e-05, forward_time=0.288, class_loss=1.159, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.237, accuracy=0.955, backward_time=0.965, grad_norm=71.334, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.748e-07, train_time=5.089 +[gpue01] 2025-06-02 10:34:41,795 (trainer:816) INFO: 31epoch:train:701-800batch: iter_time=9.690e-05, forward_time=0.263, class_loss=1.211, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.247, accuracy=0.952, backward_time=0.841, grad_norm=58.619, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.713e-07, train_time=4.501 +[gpue01] 2025-06-02 10:36:43,614 (trainer:816) INFO: 31epoch:train:801-900batch: iter_time=9.865e-05, forward_time=0.262, class_loss=1.068, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.219, accuracy=0.958, backward_time=0.938, grad_norm=79.938, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.679e-07, train_time=4.872 +[gpue01] 2025-06-02 10:38:47,447 (trainer:816) INFO: 31epoch:train:901-1000batch: iter_time=1.059e-04, forward_time=0.263, class_loss=1.064, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.218, accuracy=0.955, backward_time=0.955, grad_norm=56.857, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.646e-07, train_time=4.953 +[gpue01] 2025-06-02 10:40:54,941 (trainer:816) INFO: 31epoch:train:1001-1100batch: iter_time=1.020e-04, forward_time=0.273, class_loss=1.107, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.227, accuracy=0.957, backward_time=0.981, grad_norm=74.898, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.612e-07, train_time=5.099 +[gpue01] 2025-06-02 10:42:58,685 (trainer:816) INFO: 31epoch:train:1101-1200batch: iter_time=9.935e-05, forward_time=0.250, class_loss=1.028, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.211, accuracy=0.960, backward_time=0.967, grad_norm=52.780, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.579e-07, train_time=4.949 +[gpue01] 2025-06-02 10:45:01,783 (trainer:816) INFO: 31epoch:train:1201-1300batch: iter_time=9.316e-05, forward_time=0.254, class_loss=1.400, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.285, accuracy=0.943, backward_time=0.958, grad_norm=58.518, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.546e-07, train_time=4.923 +[gpue01] 2025-06-02 10:47:06,260 (trainer:816) INFO: 31epoch:train:1301-1400batch: iter_time=1.076e-04, forward_time=0.263, class_loss=0.926, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.190, accuracy=0.961, backward_time=0.962, grad_norm=50.230, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.514e-07, train_time=4.978 +[gpue01] 2025-06-02 10:48:58,412 (trainer:816) INFO: 31epoch:train:1401-1500batch: iter_time=9.517e-05, forward_time=0.235, class_loss=1.449, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.295, accuracy=0.940, backward_time=0.867, grad_norm=53.965, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.481e-07, train_time=4.485 +[gpue01] 2025-06-02 10:50:59,774 (trainer:816) INFO: 31epoch:train:1501-1600batch: iter_time=1.044e-04, forward_time=0.242, class_loss=0.961, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.197, accuracy=0.960, backward_time=0.951, grad_norm=72.314, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.450e-07, train_time=4.854 +[gpue01] 2025-06-02 10:53:04,143 (trainer:816) INFO: 31epoch:train:1601-1700batch: iter_time=9.928e-05, forward_time=0.267, class_loss=1.200, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.245, accuracy=0.953, backward_time=0.957, grad_norm=59.742, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.418e-07, train_time=4.974 +[gpue01] 2025-06-02 10:55:04,940 (trainer:816) INFO: 31epoch:train:1701-1800batch: iter_time=9.423e-05, forward_time=0.243, class_loss=0.887, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.183, accuracy=0.965, backward_time=0.944, grad_norm=30.131, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.387e-07, train_time=4.831 +[gpue01] 2025-06-02 10:57:19,283 (trainer:816) INFO: 31epoch:train:1801-1900batch: iter_time=1.037e-04, forward_time=0.271, class_loss=1.390, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.283, accuracy=0.942, backward_time=1.053, grad_norm=51.783, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.356e-07, train_time=5.373 +[gpue01] 2025-06-02 10:59:05,692 (trainer:816) INFO: 31epoch:train:1901-2000batch: iter_time=9.012e-05, forward_time=0.236, class_loss=1.287, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.263, accuracy=0.950, backward_time=0.808, grad_norm=56.740, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.325e-07, train_time=4.256 +[gpue01] 2025-06-02 11:22:28,488 (trainer:401) INFO: 31epoch results: [train] iter_time=1.963e-04, forward_time=0.281, class_loss=1.176, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.240, accuracy=0.953, backward_time=0.922, grad_norm=61.446, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.634e-07, train_time=4.894, time=43 minutes and 5.9 seconds, total_count=62000, gpu_max_cached_mem_GB=81.803, [valid] class_loss=2.593, geo_loss_downstream=0.212, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.021, geo_loss_all=0.135, loss=2.102, accuracy=0.895, time=23 minutes and 22.56 seconds, total_count=146382, gpu_max_cached_mem_GB=81.803 +[gpue01] 2025-06-02 11:22:41,952 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue01] 2025-06-02 11:22:41,971 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/30epoch.pth +[gpue01] 2025-06-02 11:22:41,972 (trainer:335) INFO: 32/50epoch started. Estimated time to finish: 21 hours, 1 minute and 21.81 seconds +[gpue01] 2025-06-02 11:27:07,026 (trainer:816) INFO: 32epoch:train:1-100batch: iter_time=0.003, forward_time=0.400, class_loss=1.536, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.312, accuracy=0.938, backward_time=0.832, grad_norm=74.839, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=3.294e-07, train_time=5.039 +[gpue01] 2025-06-02 11:29:21,087 (trainer:816) INFO: 32epoch:train:101-200batch: iter_time=9.411e-05, forward_time=0.386, class_loss=1.352, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.276, accuracy=0.945, backward_time=0.935, grad_norm=43.988, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.264e-07, train_time=5.362 +[gpue01] 2025-06-02 11:31:30,188 (trainer:816) INFO: 32epoch:train:201-300batch: iter_time=9.077e-05, forward_time=0.347, class_loss=1.172, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.240, accuracy=0.957, backward_time=0.924, grad_norm=62.186, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.234e-07, train_time=5.163 +[gpue01] 2025-06-02 11:33:47,390 (trainer:816) INFO: 32epoch:train:301-400batch: iter_time=1.018e-04, forward_time=0.344, class_loss=1.022, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.210, accuracy=0.957, backward_time=1.009, grad_norm=44.390, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.205e-07, train_time=5.487 +[gpue01] 2025-06-02 11:35:49,789 (trainer:816) INFO: 32epoch:train:401-500batch: iter_time=1.054e-04, forward_time=0.297, class_loss=1.172, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.240, accuracy=0.952, backward_time=0.908, grad_norm=75.410, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.175e-07, train_time=4.895 +[gpue01] 2025-06-02 11:37:41,933 (trainer:816) INFO: 32epoch:train:501-600batch: iter_time=9.698e-05, forward_time=0.280, class_loss=1.211, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.247, accuracy=0.953, backward_time=0.819, grad_norm=79.169, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.146e-07, train_time=4.485 +[gpue01] 2025-06-02 11:39:46,309 (trainer:816) INFO: 32epoch:train:601-700batch: iter_time=9.339e-05, forward_time=0.293, class_loss=1.218, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.249, accuracy=0.953, backward_time=0.930, grad_norm=58.856, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.117e-07, train_time=4.974 +[gpue01] 2025-06-02 11:41:54,707 (trainer:816) INFO: 32epoch:train:701-800batch: iter_time=1.009e-04, forward_time=0.292, class_loss=0.833, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.172, accuracy=0.968, backward_time=0.972, grad_norm=58.946, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.036, optim0_lr0=3.089e-07, train_time=5.135 +[gpue01] 2025-06-02 11:44:02,832 (trainer:816) INFO: 32epoch:train:801-900batch: iter_time=9.829e-05, forward_time=0.264, class_loss=1.373, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.280, accuracy=0.942, backward_time=0.998, grad_norm=62.452, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.060e-07, train_time=5.124 +[gpue01] 2025-06-02 11:45:51,409 (trainer:816) INFO: 32epoch:train:901-1000batch: iter_time=9.919e-05, forward_time=0.229, class_loss=0.924, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.012, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.190, accuracy=0.963, backward_time=0.834, grad_norm=42.125, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=3.032e-07, train_time=4.342 +[gpue01] 2025-06-02 11:47:40,644 (trainer:816) INFO: 32epoch:train:1001-1100batch: iter_time=1.009e-04, forward_time=0.222, class_loss=0.990, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.203, accuracy=0.957, backward_time=0.848, grad_norm=42.052, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.004e-07, train_time=4.369 +[gpue01] 2025-06-02 11:49:25,715 (trainer:816) INFO: 32epoch:train:1101-1200batch: iter_time=9.471e-05, forward_time=0.231, class_loss=1.161, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.237, accuracy=0.953, backward_time=0.797, grad_norm=49.309, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=2.977e-07, train_time=4.202 +[gpue01] 2025-06-02 11:51:08,460 (trainer:816) INFO: 32epoch:train:1201-1300batch: iter_time=9.436e-05, forward_time=0.238, class_loss=0.849, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.012, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.175, accuracy=0.970, backward_time=0.768, grad_norm=53.095, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=2.950e-07, train_time=4.109 +[gpue01] 2025-06-02 11:53:07,433 (trainer:816) INFO: 32epoch:train:1301-1400batch: iter_time=1.007e-04, forward_time=0.257, class_loss=1.168, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.239, accuracy=0.952, backward_time=0.912, grad_norm=45.273, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=2.923e-07, train_time=4.758 +[gpue01] 2025-06-02 11:55:01,420 (trainer:816) INFO: 32epoch:train:1401-1500batch: iter_time=9.979e-05, forward_time=0.244, class_loss=1.106, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.226, accuracy=0.957, backward_time=0.875, grad_norm=36.384, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=2.896e-07, train_time=4.559 +[gpue01] 2025-06-02 11:56:54,340 (trainer:816) INFO: 32epoch:train:1501-1600batch: iter_time=9.456e-05, forward_time=0.253, class_loss=0.688, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.012, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.012, geo_loss_all=0.102, loss=0.143, accuracy=0.973, backward_time=0.855, grad_norm=53.800, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=2.869e-07, train_time=4.516 +[gpue01] 2025-06-02 11:58:55,292 (trainer:816) INFO: 32epoch:train:1601-1700batch: iter_time=1.007e-04, forward_time=0.249, class_loss=1.404, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.286, accuracy=0.938, backward_time=0.940, grad_norm=57.055, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=2.843e-07, train_time=4.837 +[gpue01] 2025-06-02 12:00:48,312 (trainer:816) INFO: 32epoch:train:1701-1800batch: iter_time=1.005e-04, forward_time=0.230, class_loss=1.274, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.260, accuracy=0.943, backward_time=0.879, grad_norm=44.005, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=2.817e-07, train_time=4.520 +[gpue01] 2025-06-02 12:02:52,056 (trainer:816) INFO: 32epoch:train:1801-1900batch: iter_time=1.037e-04, forward_time=0.257, class_loss=0.952, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.195, accuracy=0.962, backward_time=0.960, grad_norm=41.989, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=2.791e-07, train_time=4.949 +[gpue01] 2025-06-02 12:04:49,603 (trainer:816) INFO: 32epoch:train:1901-2000batch: iter_time=1.031e-04, forward_time=0.260, class_loss=0.932, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.192, accuracy=0.965, backward_time=0.894, grad_norm=54.873, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.034, optim0_lr0=2.765e-07, train_time=4.701 +[gpue01] 2025-06-02 12:28:09,414 (trainer:401) INFO: 32epoch results: [train] iter_time=2.404e-04, forward_time=0.279, class_loss=1.117, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.229, accuracy=0.955, backward_time=0.894, grad_norm=54.010, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.035, optim0_lr0=3.023e-07, train_time=4.776, time=42 minutes and 7.83 seconds, total_count=64000, gpu_max_cached_mem_GB=82.354, [valid] class_loss=2.619, geo_loss_downstream=0.221, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.141, loss=2.123, accuracy=0.895, time=23 minutes and 19.61 seconds, total_count=151104, gpu_max_cached_mem_GB=82.354 +[gpue01] 2025-06-02 12:28:22,860 (trainer:467) INFO: There are no improvements in this epoch +[gpue01] 2025-06-02 12:28:22,878 (trainer:335) INFO: 33/50epoch started. Estimated time to finish: 19 hours, 51 minutes and 48 seconds +[gpue01] 2025-06-02 12:32:47,324 (trainer:816) INFO: 33epoch:train:1-100batch: iter_time=0.002, forward_time=0.393, class_loss=1.030, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.211, accuracy=0.962, backward_time=0.823, grad_norm=47.793, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.740e-07, train_time=4.976 +[gpue01] 2025-06-02 12:35:01,310 (trainer:816) INFO: 33epoch:train:101-200batch: iter_time=9.706e-05, forward_time=0.376, class_loss=1.497, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.305, accuracy=0.933, backward_time=0.945, grad_norm=58.238, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.715e-07, train_time=5.359 +[gpue01] 2025-06-02 12:37:11,584 (trainer:816) INFO: 33epoch:train:201-300batch: iter_time=1.017e-04, forward_time=0.353, class_loss=0.605, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.126, accuracy=0.980, backward_time=0.929, grad_norm=49.261, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.690e-07, train_time=5.210 +[gpue01] 2025-06-02 12:39:22,119 (trainer:816) INFO: 33epoch:train:301-400batch: iter_time=1.060e-04, forward_time=0.338, class_loss=1.064, geo_loss_downstream=0.160, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.218, accuracy=0.960, backward_time=0.948, grad_norm=72.623, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.036, optim0_lr0=2.665e-07, train_time=5.221 +[gpue01] 2025-06-02 12:41:30,505 (trainer:816) INFO: 33epoch:train:401-500batch: iter_time=1.061e-04, forward_time=0.305, class_loss=1.219, geo_loss_downstream=0.160, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.249, accuracy=0.950, backward_time=0.960, grad_norm=49.901, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.036, optim0_lr0=2.641e-07, train_time=5.135 +[gpue01] 2025-06-02 12:43:31,704 (trainer:816) INFO: 33epoch:train:501-600batch: iter_time=1.046e-04, forward_time=0.294, class_loss=1.601, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.325, accuracy=0.937, backward_time=0.898, grad_norm=77.095, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.617e-07, train_time=4.847 +[gpue01] 2025-06-02 12:45:34,686 (trainer:816) INFO: 33epoch:train:601-700batch: iter_time=9.555e-05, forward_time=0.290, class_loss=1.288, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.263, accuracy=0.950, backward_time=0.920, grad_norm=59.251, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.593e-07, train_time=4.918 +[gpue01] 2025-06-02 12:47:38,091 (trainer:816) INFO: 33epoch:train:701-800batch: iter_time=9.700e-05, forward_time=0.266, class_loss=1.299, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.265, accuracy=0.953, backward_time=0.948, grad_norm=62.744, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.569e-07, train_time=4.935 +[gpue01] 2025-06-02 12:49:30,107 (trainer:816) INFO: 33epoch:train:801-900batch: iter_time=1.092e-04, forward_time=0.250, class_loss=1.356, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.276, accuracy=0.945, backward_time=0.850, grad_norm=72.159, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.545e-07, train_time=4.480 +[gpue01] 2025-06-02 12:51:37,122 (trainer:816) INFO: 33epoch:train:901-1000batch: iter_time=1.187e-04, forward_time=0.278, class_loss=1.025, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.210, accuracy=0.962, backward_time=0.972, grad_norm=63.217, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.522e-07, train_time=5.080 +[gpue01] 2025-06-02 12:53:44,686 (trainer:816) INFO: 33epoch:train:1001-1100batch: iter_time=1.035e-04, forward_time=0.249, class_loss=1.190, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.243, accuracy=0.955, backward_time=1.007, grad_norm=72.056, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.034, optim0_lr0=2.499e-07, train_time=5.102 +[gpue01] 2025-06-02 12:55:38,967 (trainer:816) INFO: 33epoch:train:1101-1200batch: iter_time=1.035e-04, forward_time=0.249, class_loss=0.918, geo_loss_downstream=0.161, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.189, accuracy=0.968, backward_time=0.873, grad_norm=51.495, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.034, optim0_lr0=2.476e-07, train_time=4.570 +[gpue01] 2025-06-02 12:57:49,673 (trainer:816) INFO: 33epoch:train:1201-1300batch: iter_time=1.087e-04, forward_time=0.276, class_loss=0.972, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.200, accuracy=0.958, backward_time=1.010, grad_norm=58.794, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.453e-07, train_time=5.227 +[gpue01] 2025-06-02 12:59:38,197 (trainer:816) INFO: 33epoch:train:1301-1400batch: iter_time=1.066e-04, forward_time=0.231, class_loss=1.716, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.348, accuracy=0.918, backward_time=0.833, grad_norm=63.048, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.034, optim0_lr0=2.431e-07, train_time=4.340 +[gpue01] 2025-06-02 13:01:29,801 (trainer:816) INFO: 33epoch:train:1401-1500batch: iter_time=1.127e-04, forward_time=0.251, class_loss=1.037, geo_loss_downstream=0.160, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.101, loss=0.213, accuracy=0.957, backward_time=0.845, grad_norm=49.476, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.409e-07, train_time=4.463 +[gpue01] 2025-06-02 13:03:25,989 (trainer:816) INFO: 33epoch:train:1501-1600batch: iter_time=1.010e-04, forward_time=0.253, class_loss=1.036, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.212, accuracy=0.957, backward_time=0.887, grad_norm=52.597, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.034, optim0_lr0=2.387e-07, train_time=4.647 +[gpue01] 2025-06-02 13:05:27,481 (trainer:816) INFO: 33epoch:train:1601-1700batch: iter_time=9.690e-05, forward_time=0.251, class_loss=1.181, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.241, accuracy=0.950, backward_time=0.944, grad_norm=62.438, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.365e-07, train_time=4.859 +[gpue01] 2025-06-02 13:07:19,919 (trainer:816) INFO: 33epoch:train:1701-1800batch: iter_time=9.964e-05, forward_time=0.227, class_loss=0.954, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.196, accuracy=0.963, backward_time=0.877, grad_norm=71.114, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.343e-07, train_time=4.497 +[gpue01] 2025-06-02 13:09:14,103 (trainer:816) INFO: 33epoch:train:1801-1900batch: iter_time=9.797e-05, forward_time=0.250, class_loss=1.066, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.218, accuracy=0.957, backward_time=0.871, grad_norm=49.760, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.321e-07, train_time=4.567 +[gpue01] 2025-06-02 13:11:13,214 (trainer:816) INFO: 33epoch:train:1901-2000batch: iter_time=9.810e-05, forward_time=0.248, class_loss=0.987, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.102, loss=0.203, accuracy=0.958, backward_time=0.923, grad_norm=45.787, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.034, optim0_lr0=2.300e-07, train_time=4.764 +[gpue01] 2025-06-02 13:34:34,616 (trainer:401) INFO: 33epoch results: [train] iter_time=2.171e-04, forward_time=0.281, class_loss=1.152, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.102, loss=0.236, accuracy=0.954, backward_time=0.913, grad_norm=59.442, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.035, optim0_lr0=2.514e-07, train_time=4.860, time=42 minutes and 50.54 seconds, total_count=66000, gpu_max_cached_mem_GB=82.354, [valid] class_loss=2.631, geo_loss_downstream=0.208, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.133, loss=2.131, accuracy=0.893, time=23 minutes and 21.19 seconds, total_count=155826, gpu_max_cached_mem_GB=82.354 +[gpue01] 2025-06-02 13:34:48,118 (trainer:467) INFO: There are no improvements in this epoch +[gpue01] 2025-06-02 13:34:48,138 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/32epoch.pth +[gpue01] 2025-06-02 13:34:48,138 (trainer:335) INFO: 34/50epoch started. Estimated time to finish: 18 hours, 46 minutes and 18.15 seconds +[gpue01] 2025-06-02 13:39:13,564 (trainer:816) INFO: 34epoch:train:1-100batch: iter_time=0.003, forward_time=0.403, class_loss=1.001, geo_loss_downstream=0.162, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.103, loss=0.205, accuracy=0.962, backward_time=0.814, grad_norm=34.245, clip=0.000e+00, loss_scale=1.678e+07, optim_step_time=0.036, optim0_lr0=2.279e-07, train_time=4.988 diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.4.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.4.log new file mode 100644 index 0000000000000000000000000000000000000000..eb5ba50aa10fa8ef238d38919704f2fdcf8deaa8 --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.4.log @@ -0,0 +1,1116 @@ +# python3 -m espnet2.bin.lid_train --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True +# Started at Sat May 31 17:31:16 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_train.py --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +[gpue05] 2025-05-31 17:32:03,807 (abs_task:1420) INFO: pytorch.version=2.4.0+cu118, cuda.available=True, cudnn.version=90100, cudnn.benchmark=True, cudnn.deterministic=False +[gpue05] 2025-05-31 17:32:03,814 (abs_task:1421) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +[gpue05] 2025-05-31 17:32:03,814 (abs_task:1424) INFO: Optimizer: +Adam ( +Parameter Group 0 + amsgrad: False + betas: [0.9, 0.98] + capturable: False + differentiable: False + eps: 1e-08 + foreach: None + fused: None + initial_lr: 1e-05 + lr: 6.0032e-06 + maximize: False + weight_decay: 0 +) +[gpue05] 2025-05-31 17:32:03,814 (abs_task:1425) INFO: Scheduler: TristageLR(warmup_steps=1250)(hold_steps=5000)(decay_steps=6250)(init_lr_scale=0.6)(final_lr_scale=0.1)(decay_factor=0.00036841361487904725) +[gpue05] 2025-05-31 17:32:03,819 (abs_task:1434) INFO: Saving the configuration in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/config.yaml +[gpue05] 2025-05-31 17:32:04,123 (preprocessor:2245) INFO: Using lang2vec geo +[gpue05] 2025-05-31 17:32:21,208 (abs_task:1899) WARNING: Reading dump/raw/train_all_no_filter_lang/category2utt +[gpue05] 2025-05-31 17:32:21,222 (abs_task:1946) WARNING: Reading dump/raw/train_all_no_filter_lang/dataset2utt +[gpue05] 2025-05-31 17:32:21,226 (abs_task:1962) WARNING: Reading dump/raw/train_all_no_filter_lang/utt2dataset +[gpue05] 2025-05-31 17:34:39,414 (abs_task:1997) INFO: [train] dataset: +ESPnetDataset( + speech: {"path": "dump/raw/train_all_no_filter_lang/wav.scp", "type": "sound"} + lid_labels: {"path": "dump/raw/train_all_no_filter_lang/utt2spk", "type": "text"} + preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)) +[gpue05] 2025-05-31 17:34:39,414 (abs_task:1998) INFO: [train] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False) +[gpue05] 2025-05-31 17:34:39,414 (abs_task:1999) INFO: [train] collate_fn: (float_pad_value=0.0, int_pad_value=0.0) +[gpue05] 2025-05-31 17:34:39,414 (abs_task:2000) INFO: [train] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=727457, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3) +[gpue05] 2025-05-31 17:34:39,482 (abs_task:2001) INFO: [train] mini-batch sizes summary: N-batch=727457, mean=6.0, min=1, max=6 +[gpue05] 2025-05-31 17:34:39,905 (preprocessor:2245) INFO: Using lang2vec geo +[gpue05] 2025-05-31 17:34:52,738 (abs_task:1899) WARNING: Reading dump/raw/dev_ml_superb2_lang/category2utt +[gpue05] 2025-05-31 17:34:52,740 (abs_task:1946) WARNING: Reading dump/raw/dev_ml_superb2_lang/dataset2utt +[gpue05] 2025-05-31 17:34:52,741 (abs_task:1962) WARNING: Reading dump/raw/dev_ml_superb2_lang/utt2dataset +[gpue05] 2025-05-31 17:34:53,801 (abs_task:1997) INFO: [valid] dataset: +ESPnetDataset( + speech: {"path": "dump/raw/dev_ml_superb2_lang/wav.scp", "type": "sound"} + lid_labels: {"path": "dump/raw/dev_ml_superb2_lang/utt2spk", "type": "text"} + preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)) +[gpue05] 2025-05-31 17:34:53,801 (abs_task:1998) INFO: [valid] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False) +[gpue05] 2025-05-31 17:34:53,801 (abs_task:1999) INFO: [valid] collate_fn: (float_pad_value=0.0, int_pad_value=0.0) +[gpue05] 2025-05-31 17:34:53,801 (abs_task:2000) INFO: [valid] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=4722, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3) +[gpue05] 2025-05-31 17:34:53,802 (abs_task:2001) INFO: [valid] mini-batch sizes summary: N-batch=4722, mean=6.0, min=4, max=6 +wandb: Currently logged in as: qingzhew (qingzhew-carnegie-mellon-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin +wandb: Tracking run with wandb version 0.19.10 +wandb: Run data is saved locally in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/wandb/run-20250531_173454-0zfdmaq1 +wandb: Run `wandb offline` to turn off syncing. +wandb: Syncing run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3 +wandb: ⭐️ View project at https://wandb.ai/qingzhew-carnegie-mellon-university/lid +wandb: 🚀 View run at https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/0zfdmaq1 +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:218: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead. + scaler = GradScaler() +[gpue05] 2025-05-31 17:34:55,069 (trainer:347) INFO: 1/50epoch started +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:645: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead. + with autocast( +[gpue05] 2025-05-31 17:37:39,570 (trainer:816) INFO: 1epoch:train:1-100batch: iter_time=0.001, forward_time=0.393, class_loss=16.908, geo_loss_downstream=0.400, inter_geo_loss_layer32=0.407, inter_geo_loss_layer36=0.404, inter_geo_loss_layer40=0.407, inter_geo_loss_layer44=0.402, inter_geo_loss_mean=0.405, geo_loss_all=0.402, loss=3.402, accuracy=0.008, backward_time=1.233, grad_norm=413.667, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.012, optim0_lr0=6.045e-06, train_time=6.577 +[gpue05] 2025-05-31 17:40:03,730 (trainer:816) INFO: 1epoch:train:101-200batch: iter_time=1.196e-04, forward_time=0.355, class_loss=16.735, geo_loss_downstream=0.400, inter_geo_loss_layer32=0.395, inter_geo_loss_layer36=0.393, inter_geo_loss_layer40=0.398, inter_geo_loss_layer44=0.391, inter_geo_loss_mean=0.394, geo_loss_all=0.398, loss=3.367, accuracy=0.012, backward_time=1.073, grad_norm=454.294, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=6.125e-06, train_time=5.766 +[gpue05] 2025-05-31 17:42:01,071 (trainer:816) INFO: 1epoch:train:201-300batch: iter_time=1.173e-04, forward_time=0.303, class_loss=16.708, geo_loss_downstream=0.400, inter_geo_loss_layer32=0.385, inter_geo_loss_layer36=0.386, inter_geo_loss_layer40=0.392, inter_geo_loss_layer44=0.386, inter_geo_loss_mean=0.387, geo_loss_all=0.395, loss=3.361, accuracy=0.012, backward_time=0.856, grad_norm=397.404, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=6.205e-06, train_time=4.693 +[gpue05] 2025-05-31 17:43:51,571 (trainer:816) INFO: 1epoch:train:301-400batch: iter_time=1.205e-04, forward_time=0.296, class_loss=16.596, geo_loss_downstream=0.402, inter_geo_loss_layer32=0.379, inter_geo_loss_layer36=0.381, inter_geo_loss_layer40=0.386, inter_geo_loss_layer44=0.382, inter_geo_loss_mean=0.382, geo_loss_all=0.394, loss=3.339, accuracy=0.020, backward_time=0.792, grad_norm=376.648, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=6.285e-06, train_time=4.420 +[gpue05] 2025-05-31 17:45:58,772 (trainer:816) INFO: 1epoch:train:401-500batch: iter_time=1.176e-04, forward_time=0.310, class_loss=16.513, geo_loss_downstream=0.401, inter_geo_loss_layer32=0.373, inter_geo_loss_layer36=0.378, inter_geo_loss_layer40=0.380, inter_geo_loss_layer44=0.376, inter_geo_loss_mean=0.377, geo_loss_all=0.391, loss=3.322, accuracy=0.027, backward_time=0.946, grad_norm=382.586, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=6.365e-06, train_time=5.088 +[gpue05] 2025-05-31 17:48:12,668 (trainer:816) INFO: 1epoch:train:501-600batch: iter_time=1.239e-04, forward_time=0.318, class_loss=16.386, geo_loss_downstream=0.400, inter_geo_loss_layer32=0.369, inter_geo_loss_layer36=0.371, inter_geo_loss_layer40=0.376, inter_geo_loss_layer44=0.370, inter_geo_loss_mean=0.372, geo_loss_all=0.389, loss=3.297, accuracy=0.025, backward_time=1.006, grad_norm=343.142, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=6.445e-06, train_time=5.355 +[gpue05] 2025-05-31 17:50:16,963 (trainer:816) INFO: 1epoch:train:601-700batch: iter_time=1.134e-04, forward_time=0.284, class_loss=16.296, geo_loss_downstream=0.401, inter_geo_loss_layer32=0.367, inter_geo_loss_layer36=0.370, inter_geo_loss_layer40=0.372, inter_geo_loss_layer44=0.367, inter_geo_loss_mean=0.369, geo_loss_all=0.388, loss=3.279, accuracy=0.028, backward_time=0.944, grad_norm=301.737, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=6.525e-06, train_time=4.971 +[gpue05] 2025-05-31 17:52:18,247 (trainer:816) INFO: 1epoch:train:701-800batch: iter_time=1.160e-04, forward_time=0.254, class_loss=16.312, geo_loss_downstream=0.401, inter_geo_loss_layer32=0.363, inter_geo_loss_layer36=0.369, inter_geo_loss_layer40=0.368, inter_geo_loss_layer44=0.362, inter_geo_loss_mean=0.366, geo_loss_all=0.387, loss=3.282, accuracy=0.030, backward_time=0.943, grad_norm=438.051, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=6.605e-06, train_time=4.851 +[gpue05] 2025-05-31 17:54:10,701 (trainer:816) INFO: 1epoch:train:801-900batch: iter_time=1.191e-04, forward_time=0.244, class_loss=16.186, geo_loss_downstream=0.400, inter_geo_loss_layer32=0.363, inter_geo_loss_layer36=0.365, inter_geo_loss_layer40=0.365, inter_geo_loss_layer44=0.356, inter_geo_loss_mean=0.362, geo_loss_all=0.385, loss=3.256, accuracy=0.042, backward_time=0.865, grad_norm=304.488, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=6.685e-06, train_time=4.498 +[gpue05] 2025-05-31 17:56:03,384 (trainer:816) INFO: 1epoch:train:901-1000batch: iter_time=1.178e-04, forward_time=0.249, class_loss=16.074, geo_loss_downstream=0.401, inter_geo_loss_layer32=0.361, inter_geo_loss_layer36=0.364, inter_geo_loss_layer40=0.363, inter_geo_loss_layer44=0.354, inter_geo_loss_mean=0.360, geo_loss_all=0.385, loss=3.234, accuracy=0.070, backward_time=0.861, grad_norm=336.400, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=6.765e-06, train_time=4.507 +[gpue05] 2025-05-31 17:58:20,078 (trainer:816) INFO: 1epoch:train:1001-1100batch: iter_time=1.249e-04, forward_time=0.281, class_loss=15.986, geo_loss_downstream=0.401, inter_geo_loss_layer32=0.357, inter_geo_loss_layer36=0.361, inter_geo_loss_layer40=0.361, inter_geo_loss_layer44=0.352, inter_geo_loss_mean=0.358, geo_loss_all=0.384, loss=3.216, accuracy=0.080, backward_time=1.071, grad_norm=305.542, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=6.845e-06, train_time=5.467 +[gpue05] 2025-05-31 18:00:12,183 (trainer:816) INFO: 1epoch:train:1101-1200batch: iter_time=1.238e-04, forward_time=0.218, class_loss=15.872, geo_loss_downstream=0.399, inter_geo_loss_layer32=0.355, inter_geo_loss_layer36=0.357, inter_geo_loss_layer40=0.356, inter_geo_loss_layer44=0.346, inter_geo_loss_mean=0.354, geo_loss_all=0.381, loss=3.193, accuracy=0.085, backward_time=0.887, grad_norm=250.696, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=6.925e-06, train_time=4.484 +[gpue05] 2025-05-31 18:02:12,873 (trainer:816) INFO: 1epoch:train:1201-1300batch: iter_time=1.168e-04, forward_time=0.256, class_loss=15.644, geo_loss_downstream=0.401, inter_geo_loss_layer32=0.355, inter_geo_loss_layer36=0.356, inter_geo_loss_layer40=0.354, inter_geo_loss_layer44=0.346, inter_geo_loss_mean=0.353, geo_loss_all=0.382, loss=3.148, accuracy=0.107, backward_time=0.936, grad_norm=312.631, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=7.005e-06, train_time=4.827 +[gpue05] 2025-05-31 18:04:08,967 (trainer:816) INFO: 1epoch:train:1301-1400batch: iter_time=1.222e-04, forward_time=0.254, class_loss=15.540, geo_loss_downstream=0.402, inter_geo_loss_layer32=0.353, inter_geo_loss_layer36=0.355, inter_geo_loss_layer40=0.353, inter_geo_loss_layer44=0.347, inter_geo_loss_mean=0.352, geo_loss_all=0.382, loss=3.127, accuracy=0.137, backward_time=0.891, grad_norm=308.752, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=7.085e-06, train_time=4.643 +[gpue05] 2025-05-31 18:06:06,644 (trainer:816) INFO: 1epoch:train:1401-1500batch: iter_time=1.282e-04, forward_time=0.256, class_loss=15.322, geo_loss_downstream=0.400, inter_geo_loss_layer32=0.351, inter_geo_loss_layer36=0.352, inter_geo_loss_layer40=0.349, inter_geo_loss_layer44=0.343, inter_geo_loss_mean=0.349, geo_loss_all=0.380, loss=3.083, accuracy=0.140, backward_time=0.905, grad_norm=288.654, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=7.165e-06, train_time=4.707 +[gpue05] 2025-05-31 18:07:59,624 (trainer:816) INFO: 1epoch:train:1501-1600batch: iter_time=1.192e-04, forward_time=0.248, class_loss=15.133, geo_loss_downstream=0.403, inter_geo_loss_layer32=0.351, inter_geo_loss_layer36=0.351, inter_geo_loss_layer40=0.349, inter_geo_loss_layer44=0.342, inter_geo_loss_mean=0.348, geo_loss_all=0.381, loss=3.046, accuracy=0.175, backward_time=0.866, grad_norm=330.888, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=7.245e-06, train_time=4.519 +[gpue05] 2025-05-31 18:09:49,020 (trainer:816) INFO: 1epoch:train:1601-1700batch: iter_time=1.155e-04, forward_time=0.230, class_loss=14.917, geo_loss_downstream=0.402, inter_geo_loss_layer32=0.349, inter_geo_loss_layer36=0.349, inter_geo_loss_layer40=0.346, inter_geo_loss_layer44=0.341, inter_geo_loss_mean=0.346, geo_loss_all=0.380, loss=3.002, accuracy=0.214, backward_time=0.847, grad_norm=358.414, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=7.325e-06, train_time=4.375 +[gpue05] 2025-05-31 18:11:51,968 (trainer:816) INFO: 1epoch:train:1701-1800batch: iter_time=1.222e-04, forward_time=0.269, class_loss=14.572, geo_loss_downstream=0.400, inter_geo_loss_layer32=0.346, inter_geo_loss_layer36=0.346, inter_geo_loss_layer40=0.343, inter_geo_loss_layer44=0.341, inter_geo_loss_mean=0.344, geo_loss_all=0.378, loss=2.933, accuracy=0.250, backward_time=0.944, grad_norm=242.357, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=7.405e-06, train_time=4.917 +[gpue05] 2025-05-31 18:13:54,591 (trainer:816) INFO: 1epoch:train:1801-1900batch: iter_time=1.325e-04, forward_time=0.256, class_loss=14.230, geo_loss_downstream=0.400, inter_geo_loss_layer32=0.345, inter_geo_loss_layer36=0.346, inter_geo_loss_layer40=0.341, inter_geo_loss_layer44=0.338, inter_geo_loss_mean=0.343, geo_loss_all=0.377, loss=2.865, accuracy=0.288, backward_time=0.955, grad_norm=226.058, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=7.485e-06, train_time=4.904 +[gpue05] 2025-05-31 18:15:46,657 (trainer:816) INFO: 1epoch:train:1901-2000batch: iter_time=1.131e-04, forward_time=0.244, class_loss=13.455, geo_loss_downstream=0.401, inter_geo_loss_layer32=0.344, inter_geo_loss_layer36=0.343, inter_geo_loss_layer40=0.339, inter_geo_loss_layer44=0.336, inter_geo_loss_mean=0.341, geo_loss_all=0.377, loss=2.710, accuracy=0.400, backward_time=0.861, grad_norm=213.992, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=7.565e-06, train_time=4.482 +[gpue05] 2025-05-31 18:39:02,727 (trainer:401) INFO: 1epoch results: [train] iter_time=1.732e-04, forward_time=0.276, class_loss=15.769, geo_loss_downstream=0.401, inter_geo_loss_layer32=0.363, inter_geo_loss_layer36=0.365, inter_geo_loss_layer40=0.365, inter_geo_loss_layer44=0.359, inter_geo_loss_mean=0.363, geo_loss_all=0.386, loss=3.173, accuracy=0.108, backward_time=0.934, grad_norm=329.320, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=6.805e-06, train_time=4.903, time=40 minutes and 51.76 seconds, total_count=2000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=15.123, geo_loss_downstream=0.526, inter_geo_loss_layer32=0.378, inter_geo_loss_layer36=0.390, inter_geo_loss_layer40=0.400, inter_geo_loss_layer44=0.469, inter_geo_loss_mean=0.409, geo_loss_all=0.479, loss=12.195, accuracy=0.185, time=23 minutes and 15.89 seconds, total_count=4722, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-05-31 18:39:17,712 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-05-31 18:39:17,714 (trainer:335) INFO: 2/50epoch started. Estimated time to finish: 2 days, 4 hours and 34 minutes +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:645: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead. + with autocast( +[gpue05] 2025-05-31 18:43:48,565 (trainer:816) INFO: 2epoch:train:1-100batch: iter_time=0.007, forward_time=0.383, class_loss=12.894, geo_loss_downstream=0.401, inter_geo_loss_layer32=0.344, inter_geo_loss_layer36=0.343, inter_geo_loss_layer40=0.339, inter_geo_loss_layer44=0.335, inter_geo_loss_mean=0.340, geo_loss_all=0.377, loss=2.598, accuracy=0.435, backward_time=0.885, grad_norm=179.997, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=7.645e-06, train_time=5.181 +[gpue05] 2025-05-31 18:45:43,790 (trainer:816) INFO: 2epoch:train:101-200batch: iter_time=1.096e-04, forward_time=0.340, class_loss=12.353, geo_loss_downstream=0.406, inter_geo_loss_layer32=0.345, inter_geo_loss_layer36=0.344, inter_geo_loss_layer40=0.340, inter_geo_loss_layer44=0.336, inter_geo_loss_mean=0.341, geo_loss_all=0.380, loss=2.490, accuracy=0.465, backward_time=0.796, grad_norm=177.719, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=7.725e-06, train_time=4.608 +[gpue05] 2025-05-31 18:47:51,110 (trainer:816) INFO: 2epoch:train:201-300batch: iter_time=1.166e-04, forward_time=0.335, class_loss=11.412, geo_loss_downstream=0.408, inter_geo_loss_layer32=0.344, inter_geo_loss_layer36=0.344, inter_geo_loss_layer40=0.340, inter_geo_loss_layer44=0.336, inter_geo_loss_mean=0.341, geo_loss_all=0.382, loss=2.301, accuracy=0.503, backward_time=0.922, grad_norm=180.288, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=7.805e-06, train_time=5.092 +[gpue05] 2025-05-31 18:49:53,254 (trainer:816) INFO: 2epoch:train:301-400batch: iter_time=1.160e-04, forward_time=0.311, class_loss=11.202, geo_loss_downstream=0.410, inter_geo_loss_layer32=0.344, inter_geo_loss_layer36=0.344, inter_geo_loss_layer40=0.339, inter_geo_loss_layer44=0.338, inter_geo_loss_mean=0.341, geo_loss_all=0.382, loss=2.259, accuracy=0.562, backward_time=0.895, grad_norm=174.887, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=7.885e-06, train_time=4.885 +[gpue05] 2025-05-31 18:51:50,139 (trainer:816) INFO: 2epoch:train:401-500batch: iter_time=1.119e-04, forward_time=0.296, class_loss=10.138, geo_loss_downstream=0.412, inter_geo_loss_layer32=0.340, inter_geo_loss_layer36=0.341, inter_geo_loss_layer40=0.336, inter_geo_loss_layer44=0.334, inter_geo_loss_mean=0.338, geo_loss_all=0.383, loss=2.047, accuracy=0.599, backward_time=0.857, grad_norm=151.208, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=7.965e-06, train_time=4.675 +[gpue05] 2025-05-31 18:53:56,337 (trainer:816) INFO: 2epoch:train:501-600batch: iter_time=1.214e-04, forward_time=0.291, class_loss=9.616, geo_loss_downstream=0.415, inter_geo_loss_layer32=0.339, inter_geo_loss_layer36=0.339, inter_geo_loss_layer40=0.334, inter_geo_loss_layer44=0.332, inter_geo_loss_mean=0.336, geo_loss_all=0.384, loss=1.942, accuracy=0.643, backward_time=0.955, grad_norm=146.249, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.045e-06, train_time=5.047 +[gpue05] 2025-05-31 18:55:52,245 (trainer:816) INFO: 2epoch:train:601-700batch: iter_time=1.079e-04, forward_time=0.269, class_loss=9.156, geo_loss_downstream=0.417, inter_geo_loss_layer32=0.337, inter_geo_loss_layer36=0.338, inter_geo_loss_layer40=0.334, inter_geo_loss_layer44=0.331, inter_geo_loss_mean=0.335, geo_loss_all=0.384, loss=1.850, accuracy=0.642, backward_time=0.872, grad_norm=141.884, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.125e-06, train_time=4.636 +[gpue05] 2025-05-31 18:57:47,923 (trainer:816) INFO: 2epoch:train:701-800batch: iter_time=1.117e-04, forward_time=0.263, class_loss=8.223, geo_loss_downstream=0.420, inter_geo_loss_layer32=0.335, inter_geo_loss_layer36=0.336, inter_geo_loss_layer40=0.332, inter_geo_loss_layer44=0.331, inter_geo_loss_mean=0.333, geo_loss_all=0.386, loss=1.664, accuracy=0.713, backward_time=0.878, grad_norm=140.793, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.205e-06, train_time=4.626 +[gpue05] 2025-05-31 18:59:39,720 (trainer:816) INFO: 2epoch:train:801-900batch: iter_time=1.069e-04, forward_time=0.238, class_loss=7.909, geo_loss_downstream=0.421, inter_geo_loss_layer32=0.334, inter_geo_loss_layer36=0.336, inter_geo_loss_layer40=0.331, inter_geo_loss_layer44=0.330, inter_geo_loss_mean=0.333, geo_loss_all=0.386, loss=1.601, accuracy=0.727, backward_time=0.863, grad_norm=147.118, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.285e-06, train_time=4.471 +[gpue05] 2025-05-31 19:01:29,860 (trainer:816) INFO: 2epoch:train:901-1000batch: iter_time=1.236e-04, forward_time=0.228, class_loss=6.751, geo_loss_downstream=0.424, inter_geo_loss_layer32=0.331, inter_geo_loss_layer36=0.333, inter_geo_loss_layer40=0.329, inter_geo_loss_layer44=0.328, inter_geo_loss_mean=0.330, geo_loss_all=0.387, loss=1.369, accuracy=0.778, backward_time=0.858, grad_norm=127.276, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.365e-06, train_time=4.405 +[gpue05] 2025-05-31 19:03:31,899 (trainer:816) INFO: 2epoch:train:1001-1100batch: iter_time=1.122e-04, forward_time=0.256, class_loss=6.284, geo_loss_downstream=0.424, inter_geo_loss_layer32=0.330, inter_geo_loss_layer36=0.332, inter_geo_loss_layer40=0.327, inter_geo_loss_layer44=0.325, inter_geo_loss_mean=0.328, geo_loss_all=0.386, loss=1.276, accuracy=0.789, backward_time=0.950, grad_norm=124.875, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.445e-06, train_time=4.881 +[gpue05] 2025-05-31 19:05:35,327 (trainer:816) INFO: 2epoch:train:1101-1200batch: iter_time=1.169e-04, forward_time=0.250, class_loss=6.309, geo_loss_downstream=0.423, inter_geo_loss_layer32=0.327, inter_geo_loss_layer36=0.328, inter_geo_loss_layer40=0.323, inter_geo_loss_layer44=0.323, inter_geo_loss_mean=0.325, geo_loss_all=0.384, loss=1.281, accuracy=0.792, backward_time=0.970, grad_norm=140.286, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.525e-06, train_time=4.936 +[gpue05] 2025-05-31 19:07:25,297 (trainer:816) INFO: 2epoch:train:1201-1300batch: iter_time=1.168e-04, forward_time=0.230, class_loss=6.474, geo_loss_downstream=0.423, inter_geo_loss_layer32=0.327, inter_geo_loss_layer36=0.329, inter_geo_loss_layer40=0.325, inter_geo_loss_layer44=0.324, inter_geo_loss_mean=0.326, geo_loss_all=0.384, loss=1.314, accuracy=0.767, backward_time=0.853, grad_norm=134.774, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=8.605e-06, train_time=4.398 +[gpue05] 2025-05-31 19:09:08,355 (trainer:816) INFO: 2epoch:train:1301-1400batch: iter_time=1.147e-04, forward_time=0.229, class_loss=5.741, geo_loss_downstream=0.425, inter_geo_loss_layer32=0.324, inter_geo_loss_layer36=0.326, inter_geo_loss_layer40=0.321, inter_geo_loss_layer44=0.321, inter_geo_loss_mean=0.323, geo_loss_all=0.384, loss=1.167, accuracy=0.811, backward_time=0.785, grad_norm=133.627, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.685e-06, train_time=4.122 +[gpue05] 2025-05-31 19:10:54,645 (trainer:816) INFO: 2epoch:train:1401-1500batch: iter_time=1.158e-04, forward_time=0.230, class_loss=5.591, geo_loss_downstream=0.423, inter_geo_loss_layer32=0.320, inter_geo_loss_layer36=0.323, inter_geo_loss_layer40=0.317, inter_geo_loss_layer44=0.318, inter_geo_loss_mean=0.320, geo_loss_all=0.382, loss=1.137, accuracy=0.785, backward_time=0.817, grad_norm=175.555, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.765e-06, train_time=4.251 +[gpue05] 2025-05-31 19:12:51,917 (trainer:816) INFO: 2epoch:train:1501-1600batch: iter_time=1.104e-04, forward_time=0.252, class_loss=5.479, geo_loss_downstream=0.423, inter_geo_loss_layer32=0.319, inter_geo_loss_layer36=0.321, inter_geo_loss_layer40=0.316, inter_geo_loss_layer44=0.317, inter_geo_loss_mean=0.318, geo_loss_all=0.381, loss=1.115, accuracy=0.812, backward_time=0.905, grad_norm=151.047, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.845e-06, train_time=4.690 +[gpue05] 2025-05-31 19:14:51,577 (trainer:816) INFO: 2epoch:train:1601-1700batch: iter_time=1.104e-04, forward_time=0.244, class_loss=5.254, geo_loss_downstream=0.425, inter_geo_loss_layer32=0.317, inter_geo_loss_layer36=0.320, inter_geo_loss_layer40=0.315, inter_geo_loss_layer44=0.316, inter_geo_loss_mean=0.317, geo_loss_all=0.381, loss=1.070, accuracy=0.808, backward_time=0.938, grad_norm=147.810, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.925e-06, train_time=4.786 +[gpue05] 2025-05-31 19:16:38,398 (trainer:816) INFO: 2epoch:train:1701-1800batch: iter_time=1.134e-04, forward_time=0.225, class_loss=5.028, geo_loss_downstream=0.424, inter_geo_loss_layer32=0.314, inter_geo_loss_layer36=0.317, inter_geo_loss_layer40=0.312, inter_geo_loss_layer44=0.313, inter_geo_loss_mean=0.314, geo_loss_all=0.380, loss=1.025, accuracy=0.808, backward_time=0.826, grad_norm=162.478, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=9.005e-06, train_time=4.272 +[gpue05] 2025-05-31 19:18:28,040 (trainer:816) INFO: 2epoch:train:1801-1900batch: iter_time=1.188e-04, forward_time=0.237, class_loss=4.395, geo_loss_downstream=0.427, inter_geo_loss_layer32=0.311, inter_geo_loss_layer36=0.314, inter_geo_loss_layer40=0.310, inter_geo_loss_layer44=0.312, inter_geo_loss_mean=0.312, geo_loss_all=0.381, loss=0.898, accuracy=0.835, backward_time=0.843, grad_norm=134.039, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=9.085e-06, train_time=4.385 +[gpue05] 2025-05-31 19:20:16,465 (trainer:816) INFO: 2epoch:train:1901-2000batch: iter_time=1.052e-04, forward_time=0.231, class_loss=4.382, geo_loss_downstream=0.422, inter_geo_loss_layer32=0.308, inter_geo_loss_layer36=0.310, inter_geo_loss_layer40=0.307, inter_geo_loss_layer44=0.308, inter_geo_loss_mean=0.308, geo_loss_all=0.377, loss=0.895, accuracy=0.825, backward_time=0.837, grad_norm=105.216, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=9.165e-06, train_time=4.336 +[gpue05] 2025-05-31 19:43:31,965 (trainer:401) INFO: 2epoch results: [train] iter_time=4.753e-04, forward_time=0.267, class_loss=7.730, geo_loss_downstream=0.419, inter_geo_loss_layer32=0.329, inter_geo_loss_layer36=0.331, inter_geo_loss_layer40=0.326, inter_geo_loss_layer44=0.325, inter_geo_loss_mean=0.328, geo_loss_all=0.382, loss=1.565, accuracy=0.705, backward_time=0.875, grad_norm=148.856, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=8.405e-06, train_time=4.634, time=40 minutes and 58.99 seconds, total_count=4000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=6.930, geo_loss_downstream=0.494, inter_geo_loss_layer32=0.329, inter_geo_loss_layer36=0.350, inter_geo_loss_layer40=0.352, inter_geo_loss_layer44=0.395, inter_geo_loss_mean=0.356, geo_loss_all=0.439, loss=5.632, accuracy=0.699, time=23 minutes and 15.23 seconds, total_count=9444, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-05-31 19:43:46,292 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-05-31 19:43:46,296 (trainer:335) INFO: 3/50epoch started. Estimated time to finish: 2 days, 3 hours and 32 minutes +[gpue05] 2025-05-31 19:48:26,596 (trainer:816) INFO: 3epoch:train:1-100batch: iter_time=0.002, forward_time=0.410, class_loss=4.342, geo_loss_downstream=0.422, inter_geo_loss_layer32=0.306, inter_geo_loss_layer36=0.309, inter_geo_loss_layer40=0.306, inter_geo_loss_layer44=0.306, inter_geo_loss_mean=0.307, geo_loss_all=0.376, loss=0.887, accuracy=0.830, backward_time=0.957, grad_norm=112.795, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=9.245e-06, train_time=5.553 +[gpue05] 2025-05-31 19:50:19,850 (trainer:816) INFO: 3epoch:train:101-200batch: iter_time=1.108e-04, forward_time=0.351, class_loss=4.384, geo_loss_downstream=0.420, inter_geo_loss_layer32=0.304, inter_geo_loss_layer36=0.308, inter_geo_loss_layer40=0.304, inter_geo_loss_layer44=0.307, inter_geo_loss_mean=0.306, geo_loss_all=0.374, loss=0.895, accuracy=0.838, backward_time=0.765, grad_norm=167.850, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=9.325e-06, train_time=4.529 +[gpue05] 2025-05-31 19:52:11,712 (trainer:816) INFO: 3epoch:train:201-300batch: iter_time=1.094e-04, forward_time=0.310, class_loss=4.534, geo_loss_downstream=0.421, inter_geo_loss_layer32=0.301, inter_geo_loss_layer36=0.304, inter_geo_loss_layer40=0.300, inter_geo_loss_layer44=0.304, inter_geo_loss_mean=0.302, geo_loss_all=0.373, loss=0.926, accuracy=0.822, backward_time=0.792, grad_norm=130.832, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=9.405e-06, train_time=4.474 +[gpue05] 2025-05-31 19:54:41,990 (trainer:816) INFO: 3epoch:train:301-400batch: iter_time=1.167e-04, forward_time=0.361, class_loss=4.646, geo_loss_downstream=0.420, inter_geo_loss_layer32=0.300, inter_geo_loss_layer36=0.304, inter_geo_loss_layer40=0.299, inter_geo_loss_layer44=0.302, inter_geo_loss_mean=0.301, geo_loss_all=0.373, loss=0.948, accuracy=0.825, backward_time=1.128, grad_norm=163.786, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=9.485e-06, train_time=6.010 +[gpue05] 2025-05-31 19:56:44,509 (trainer:816) INFO: 3epoch:train:401-500batch: iter_time=1.166e-04, forward_time=0.304, class_loss=3.856, geo_loss_downstream=0.420, inter_geo_loss_layer32=0.295, inter_geo_loss_layer36=0.298, inter_geo_loss_layer40=0.294, inter_geo_loss_layer44=0.298, inter_geo_loss_mean=0.296, geo_loss_all=0.371, loss=0.790, accuracy=0.851, backward_time=0.905, grad_norm=106.010, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=9.565e-06, train_time=4.900 +[gpue05] 2025-05-31 19:58:48,354 (trainer:816) INFO: 3epoch:train:501-600batch: iter_time=1.117e-04, forward_time=0.280, class_loss=4.216, geo_loss_downstream=0.421, inter_geo_loss_layer32=0.292, inter_geo_loss_layer36=0.296, inter_geo_loss_layer40=0.292, inter_geo_loss_layer44=0.294, inter_geo_loss_mean=0.294, geo_loss_all=0.370, loss=0.862, accuracy=0.835, backward_time=0.943, grad_norm=133.867, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=9.645e-06, train_time=4.953 +[gpue05] 2025-05-31 20:00:49,922 (trainer:816) INFO: 3epoch:train:601-700batch: iter_time=1.092e-04, forward_time=0.278, class_loss=3.496, geo_loss_downstream=0.422, inter_geo_loss_layer32=0.289, inter_geo_loss_layer36=0.294, inter_geo_loss_layer40=0.289, inter_geo_loss_layer44=0.292, inter_geo_loss_mean=0.291, geo_loss_all=0.370, loss=0.718, accuracy=0.865, backward_time=0.923, grad_norm=115.750, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=9.725e-06, train_time=4.862 +[gpue05] 2025-05-31 20:03:04,503 (trainer:816) INFO: 3epoch:train:701-800batch: iter_time=1.166e-04, forward_time=0.272, class_loss=4.032, geo_loss_downstream=0.421, inter_geo_loss_layer32=0.286, inter_geo_loss_layer36=0.290, inter_geo_loss_layer40=0.286, inter_geo_loss_layer44=0.289, inter_geo_loss_mean=0.288, geo_loss_all=0.368, loss=0.825, accuracy=0.840, backward_time=1.060, grad_norm=104.107, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=9.805e-06, train_time=5.382 +[gpue05] 2025-05-31 20:04:53,780 (trainer:816) INFO: 3epoch:train:801-900batch: iter_time=1.132e-04, forward_time=0.238, class_loss=3.595, geo_loss_downstream=0.419, inter_geo_loss_layer32=0.284, inter_geo_loss_layer36=0.289, inter_geo_loss_layer40=0.283, inter_geo_loss_layer44=0.287, inter_geo_loss_mean=0.286, geo_loss_all=0.366, loss=0.737, accuracy=0.866, backward_time=0.840, grad_norm=114.351, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=9.885e-06, train_time=4.370 +[gpue05] 2025-05-31 20:06:50,339 (trainer:816) INFO: 3epoch:train:901-1000batch: iter_time=1.177e-04, forward_time=0.254, class_loss=3.785, geo_loss_downstream=0.420, inter_geo_loss_layer32=0.281, inter_geo_loss_layer36=0.285, inter_geo_loss_layer40=0.281, inter_geo_loss_layer44=0.283, inter_geo_loss_mean=0.283, geo_loss_all=0.365, loss=0.775, accuracy=0.855, backward_time=0.896, grad_norm=109.141, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=9.965e-06, train_time=4.662 +[gpue05] 2025-05-31 20:08:48,724 (trainer:816) INFO: 3epoch:train:1001-1100batch: iter_time=1.243e-04, forward_time=0.246, class_loss=3.333, geo_loss_downstream=0.419, inter_geo_loss_layer32=0.277, inter_geo_loss_layer36=0.283, inter_geo_loss_layer40=0.277, inter_geo_loss_layer44=0.280, inter_geo_loss_mean=0.279, geo_loss_all=0.363, loss=0.685, accuracy=0.878, backward_time=0.922, grad_norm=102.072, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.735 +[gpue05] 2025-05-31 20:10:57,550 (trainer:816) INFO: 3epoch:train:1101-1200batch: iter_time=1.118e-04, forward_time=0.268, class_loss=4.191, geo_loss_downstream=0.413, inter_geo_loss_layer32=0.274, inter_geo_loss_layer36=0.279, inter_geo_loss_layer40=0.274, inter_geo_loss_layer44=0.277, inter_geo_loss_mean=0.276, geo_loss_all=0.358, loss=0.856, accuracy=0.831, backward_time=1.005, grad_norm=124.531, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=5.152 +[gpue05] 2025-05-31 20:13:02,199 (trainer:816) INFO: 3epoch:train:1201-1300batch: iter_time=1.068e-04, forward_time=0.260, class_loss=4.029, geo_loss_downstream=0.415, inter_geo_loss_layer32=0.271, inter_geo_loss_layer36=0.277, inter_geo_loss_layer40=0.271, inter_geo_loss_layer44=0.276, inter_geo_loss_mean=0.274, geo_loss_all=0.359, loss=0.824, accuracy=0.848, backward_time=0.971, grad_norm=106.590, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.985 +[gpue05] 2025-05-31 20:14:57,938 (trainer:816) INFO: 3epoch:train:1301-1400batch: iter_time=1.108e-04, forward_time=0.238, class_loss=3.629, geo_loss_downstream=0.415, inter_geo_loss_layer32=0.267, inter_geo_loss_layer36=0.273, inter_geo_loss_layer40=0.269, inter_geo_loss_layer44=0.273, inter_geo_loss_mean=0.270, geo_loss_all=0.357, loss=0.744, accuracy=0.857, backward_time=0.902, grad_norm=138.484, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.629 +[gpue05] 2025-05-31 20:16:53,319 (trainer:816) INFO: 3epoch:train:1401-1500batch: iter_time=1.226e-04, forward_time=0.243, class_loss=3.697, geo_loss_downstream=0.414, inter_geo_loss_layer32=0.265, inter_geo_loss_layer36=0.271, inter_geo_loss_layer40=0.265, inter_geo_loss_layer44=0.269, inter_geo_loss_mean=0.268, geo_loss_all=0.355, loss=0.757, accuracy=0.863, backward_time=0.895, grad_norm=135.106, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.615 +[gpue05] 2025-05-31 20:18:44,120 (trainer:816) INFO: 3epoch:train:1501-1600batch: iter_time=1.227e-04, forward_time=0.219, class_loss=3.552, geo_loss_downstream=0.415, inter_geo_loss_layer32=0.260, inter_geo_loss_layer36=0.267, inter_geo_loss_layer40=0.261, inter_geo_loss_layer44=0.265, inter_geo_loss_mean=0.263, geo_loss_all=0.354, loss=0.728, accuracy=0.867, backward_time=0.873, grad_norm=123.650, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.431 +[gpue05] 2025-05-31 20:20:41,798 (trainer:816) INFO: 3epoch:train:1601-1700batch: iter_time=1.068e-04, forward_time=0.252, class_loss=3.178, geo_loss_downstream=0.415, inter_geo_loss_layer32=0.256, inter_geo_loss_layer36=0.262, inter_geo_loss_layer40=0.256, inter_geo_loss_layer44=0.261, inter_geo_loss_mean=0.259, geo_loss_all=0.352, loss=0.653, accuracy=0.885, backward_time=0.909, grad_norm=138.663, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.706 +[gpue05] 2025-05-31 20:22:31,000 (trainer:816) INFO: 3epoch:train:1701-1800batch: iter_time=1.138e-04, forward_time=0.224, class_loss=3.439, geo_loss_downstream=0.412, inter_geo_loss_layer32=0.251, inter_geo_loss_layer36=0.258, inter_geo_loss_layer40=0.252, inter_geo_loss_layer44=0.257, inter_geo_loss_mean=0.255, geo_loss_all=0.349, loss=0.705, accuracy=0.871, backward_time=0.852, grad_norm=92.478, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.367 +[gpue05] 2025-05-31 20:24:25,176 (trainer:816) INFO: 3epoch:train:1801-1900batch: iter_time=1.119e-04, forward_time=0.248, class_loss=3.475, geo_loss_downstream=0.412, inter_geo_loss_layer32=0.248, inter_geo_loss_layer36=0.254, inter_geo_loss_layer40=0.249, inter_geo_loss_layer44=0.254, inter_geo_loss_mean=0.251, geo_loss_all=0.348, loss=0.712, accuracy=0.880, backward_time=0.879, grad_norm=93.175, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.566 +[gpue05] 2025-05-31 20:26:28,176 (trainer:816) INFO: 3epoch:train:1901-2000batch: iter_time=1.088e-04, forward_time=0.247, class_loss=3.467, geo_loss_downstream=0.410, inter_geo_loss_layer32=0.243, inter_geo_loss_layer36=0.250, inter_geo_loss_layer40=0.243, inter_geo_loss_layer44=0.248, inter_geo_loss_mean=0.246, geo_loss_all=0.345, loss=0.711, accuracy=0.868, backward_time=0.969, grad_norm=116.055, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.919 +[gpue05] 2025-05-31 20:49:40,763 (trainer:401) INFO: 3epoch results: [train] iter_time=2.047e-04, forward_time=0.275, class_loss=3.844, geo_loss_downstream=0.417, inter_geo_loss_layer32=0.278, inter_geo_loss_layer36=0.283, inter_geo_loss_layer40=0.278, inter_geo_loss_layer44=0.281, inter_geo_loss_mean=0.280, geo_loss_all=0.362, loss=0.787, accuracy=0.854, backward_time=0.919, grad_norm=121.465, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=9.802e-06, train_time=4.840, time=42 minutes and 42.13 seconds, total_count=6000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=5.526, geo_loss_downstream=0.469, inter_geo_loss_layer32=0.258, inter_geo_loss_layer36=0.272, inter_geo_loss_layer40=0.281, inter_geo_loss_layer44=0.300, inter_geo_loss_mean=0.278, geo_loss_all=0.393, loss=4.499, accuracy=0.788, time=23 minutes and 12.31 seconds, total_count=14166, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-05-31 20:49:54,485 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-05-31 20:49:54,490 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/1epoch.pth +[gpue05] 2025-05-31 20:49:54,490 (trainer:335) INFO: 4/50epoch started. Estimated time to finish: 2 days, 2 hours and 54 minutes +[gpue05] 2025-05-31 20:54:18,235 (trainer:816) INFO: 4epoch:train:1-100batch: iter_time=0.003, forward_time=0.390, class_loss=3.363, geo_loss_downstream=0.407, inter_geo_loss_layer32=0.240, inter_geo_loss_layer36=0.247, inter_geo_loss_layer40=0.239, inter_geo_loss_layer44=0.245, inter_geo_loss_mean=0.243, geo_loss_all=0.342, loss=0.690, accuracy=0.880, backward_time=0.801, grad_norm=97.351, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.861 +[gpue05] 2025-05-31 20:56:28,698 (trainer:816) INFO: 4epoch:train:101-200batch: iter_time=1.129e-04, forward_time=0.364, class_loss=3.095, geo_loss_downstream=0.407, inter_geo_loss_layer32=0.233, inter_geo_loss_layer36=0.242, inter_geo_loss_layer40=0.234, inter_geo_loss_layer44=0.240, inter_geo_loss_mean=0.237, geo_loss_all=0.339, loss=0.636, accuracy=0.890, backward_time=0.924, grad_norm=122.957, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.012, optim0_lr0=1.000e-05, train_time=5.218 +[gpue05] 2025-05-31 20:58:24,756 (trainer:816) INFO: 4epoch:train:201-300batch: iter_time=1.094e-04, forward_time=0.307, class_loss=3.127, geo_loss_downstream=0.408, inter_geo_loss_layer32=0.229, inter_geo_loss_layer36=0.237, inter_geo_loss_layer40=0.228, inter_geo_loss_layer44=0.234, inter_geo_loss_mean=0.232, geo_loss_all=0.338, loss=0.642, accuracy=0.873, backward_time=0.838, grad_norm=99.733, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.642 +[gpue05] 2025-05-31 21:00:25,127 (trainer:816) INFO: 4epoch:train:301-400batch: iter_time=1.171e-04, forward_time=0.310, class_loss=3.638, geo_loss_downstream=0.403, inter_geo_loss_layer32=0.224, inter_geo_loss_layer36=0.232, inter_geo_loss_layer40=0.223, inter_geo_loss_layer44=0.230, inter_geo_loss_mean=0.227, geo_loss_all=0.333, loss=0.744, accuracy=0.853, backward_time=0.878, grad_norm=122.009, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.814 +[gpue05] 2025-05-31 21:02:18,365 (trainer:816) INFO: 4epoch:train:401-500batch: iter_time=1.225e-04, forward_time=0.284, class_loss=3.527, geo_loss_downstream=0.404, inter_geo_loss_layer32=0.219, inter_geo_loss_layer36=0.227, inter_geo_loss_layer40=0.220, inter_geo_loss_layer44=0.224, inter_geo_loss_mean=0.222, geo_loss_all=0.331, loss=0.722, accuracy=0.873, backward_time=0.832, grad_norm=104.898, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.529 +[gpue05] 2025-05-31 21:04:15,258 (trainer:816) INFO: 4epoch:train:501-600batch: iter_time=1.108e-04, forward_time=0.274, class_loss=3.104, geo_loss_downstream=0.403, inter_geo_loss_layer32=0.213, inter_geo_loss_layer36=0.222, inter_geo_loss_layer40=0.213, inter_geo_loss_layer44=0.220, inter_geo_loss_mean=0.217, geo_loss_all=0.329, loss=0.637, accuracy=0.897, backward_time=0.879, grad_norm=120.438, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.675 +[gpue05] 2025-05-31 21:06:19,732 (trainer:816) INFO: 4epoch:train:601-700batch: iter_time=1.122e-04, forward_time=0.277, class_loss=3.076, geo_loss_downstream=0.403, inter_geo_loss_layer32=0.209, inter_geo_loss_layer36=0.218, inter_geo_loss_layer40=0.209, inter_geo_loss_layer44=0.216, inter_geo_loss_mean=0.213, geo_loss_all=0.327, loss=0.632, accuracy=0.886, backward_time=0.953, grad_norm=117.739, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.978 +[gpue05] 2025-05-31 21:08:17,794 (trainer:816) INFO: 4epoch:train:701-800batch: iter_time=1.088e-04, forward_time=0.262, class_loss=3.151, geo_loss_downstream=0.402, inter_geo_loss_layer32=0.202, inter_geo_loss_layer36=0.213, inter_geo_loss_layer40=0.204, inter_geo_loss_layer44=0.210, inter_geo_loss_mean=0.207, geo_loss_all=0.324, loss=0.646, accuracy=0.868, backward_time=0.902, grad_norm=108.111, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.722 +[gpue05] 2025-05-31 21:10:19,204 (trainer:816) INFO: 4epoch:train:801-900batch: iter_time=1.154e-04, forward_time=0.244, class_loss=2.905, geo_loss_downstream=0.402, inter_geo_loss_layer32=0.196, inter_geo_loss_layer36=0.207, inter_geo_loss_layer40=0.197, inter_geo_loss_layer44=0.205, inter_geo_loss_mean=0.201, geo_loss_all=0.322, loss=0.597, accuracy=0.907, backward_time=0.955, grad_norm=90.841, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.856 +[gpue05] 2025-05-31 21:12:12,018 (trainer:816) INFO: 4epoch:train:901-1000batch: iter_time=1.131e-04, forward_time=0.241, class_loss=3.392, geo_loss_downstream=0.400, inter_geo_loss_layer32=0.193, inter_geo_loss_layer36=0.203, inter_geo_loss_layer40=0.194, inter_geo_loss_layer44=0.201, inter_geo_loss_mean=0.198, geo_loss_all=0.319, loss=0.694, accuracy=0.866, backward_time=0.872, grad_norm=91.737, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.512 +[gpue05] 2025-05-31 21:14:05,212 (trainer:816) INFO: 4epoch:train:1001-1100batch: iter_time=1.158e-04, forward_time=0.248, class_loss=3.285, geo_loss_downstream=0.398, inter_geo_loss_layer32=0.186, inter_geo_loss_layer36=0.198, inter_geo_loss_layer40=0.186, inter_geo_loss_layer44=0.195, inter_geo_loss_mean=0.191, geo_loss_all=0.315, loss=0.673, accuracy=0.870, backward_time=0.869, grad_norm=123.912, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.527 +[gpue05] 2025-05-31 21:15:56,068 (trainer:816) INFO: 4epoch:train:1101-1200batch: iter_time=1.307e-04, forward_time=0.248, class_loss=2.948, geo_loss_downstream=0.399, inter_geo_loss_layer32=0.180, inter_geo_loss_layer36=0.192, inter_geo_loss_layer40=0.181, inter_geo_loss_layer44=0.188, inter_geo_loss_mean=0.185, geo_loss_all=0.314, loss=0.605, accuracy=0.890, backward_time=0.844, grad_norm=123.629, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.434 +[gpue05] 2025-05-31 21:17:57,990 (trainer:816) INFO: 4epoch:train:1201-1300batch: iter_time=1.125e-04, forward_time=0.251, class_loss=2.962, geo_loss_downstream=0.397, inter_geo_loss_layer32=0.173, inter_geo_loss_layer36=0.183, inter_geo_loss_layer40=0.173, inter_geo_loss_layer44=0.182, inter_geo_loss_mean=0.178, geo_loss_all=0.309, loss=0.608, accuracy=0.887, backward_time=0.953, grad_norm=130.750, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.876 +[gpue05] 2025-05-31 21:19:57,895 (trainer:816) INFO: 4epoch:train:1301-1400batch: iter_time=1.146e-04, forward_time=0.261, class_loss=2.664, geo_loss_downstream=0.396, inter_geo_loss_layer32=0.166, inter_geo_loss_layer36=0.178, inter_geo_loss_layer40=0.169, inter_geo_loss_layer44=0.176, inter_geo_loss_mean=0.172, geo_loss_all=0.306, loss=0.548, accuracy=0.902, backward_time=0.923, grad_norm=122.812, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.795 +[gpue05] 2025-05-31 21:22:01,401 (trainer:816) INFO: 4epoch:train:1401-1500batch: iter_time=1.149e-04, forward_time=0.250, class_loss=3.186, geo_loss_downstream=0.396, inter_geo_loss_layer32=0.162, inter_geo_loss_layer36=0.173, inter_geo_loss_layer40=0.162, inter_geo_loss_layer44=0.170, inter_geo_loss_mean=0.167, geo_loss_all=0.305, loss=0.652, accuracy=0.873, backward_time=0.969, grad_norm=102.102, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.940 +[gpue05] 2025-05-31 21:23:59,040 (trainer:816) INFO: 4epoch:train:1501-1600batch: iter_time=1.177e-04, forward_time=0.252, class_loss=2.872, geo_loss_downstream=0.391, inter_geo_loss_layer32=0.158, inter_geo_loss_layer36=0.167, inter_geo_loss_layer40=0.157, inter_geo_loss_layer44=0.166, inter_geo_loss_mean=0.162, geo_loss_all=0.300, loss=0.589, accuracy=0.888, backward_time=0.909, grad_norm=89.655, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.705 +[gpue05] 2025-05-31 21:25:53,594 (trainer:816) INFO: 4epoch:train:1601-1700batch: iter_time=1.057e-04, forward_time=0.227, class_loss=3.014, geo_loss_downstream=0.393, inter_geo_loss_layer32=0.152, inter_geo_loss_layer36=0.163, inter_geo_loss_layer40=0.152, inter_geo_loss_layer44=0.160, inter_geo_loss_mean=0.157, geo_loss_all=0.298, loss=0.618, accuracy=0.880, backward_time=0.903, grad_norm=115.926, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.581 +[gpue05] 2025-05-31 21:27:45,937 (trainer:816) INFO: 4epoch:train:1701-1800batch: iter_time=1.118e-04, forward_time=0.237, class_loss=3.035, geo_loss_downstream=0.390, inter_geo_loss_layer32=0.148, inter_geo_loss_layer36=0.159, inter_geo_loss_layer40=0.148, inter_geo_loss_layer44=0.155, inter_geo_loss_mean=0.153, geo_loss_all=0.295, loss=0.622, accuracy=0.882, backward_time=0.870, grad_norm=125.319, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.493 +[gpue05] 2025-05-31 21:29:55,261 (trainer:816) INFO: 4epoch:train:1801-1900batch: iter_time=1.189e-04, forward_time=0.263, class_loss=2.590, geo_loss_downstream=0.391, inter_geo_loss_layer32=0.142, inter_geo_loss_layer36=0.154, inter_geo_loss_layer40=0.143, inter_geo_loss_layer44=0.150, inter_geo_loss_mean=0.147, geo_loss_all=0.294, loss=0.533, accuracy=0.896, backward_time=1.015, grad_norm=103.867, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=5.172 +[gpue05] 2025-05-31 21:31:51,658 (trainer:816) INFO: 4epoch:train:1901-2000batch: iter_time=1.057e-04, forward_time=0.247, class_loss=3.169, geo_loss_downstream=0.388, inter_geo_loss_layer32=0.138, inter_geo_loss_layer36=0.148, inter_geo_loss_layer40=0.139, inter_geo_loss_layer44=0.147, inter_geo_loss_mean=0.143, geo_loss_all=0.290, loss=0.648, accuracy=0.875, backward_time=0.902, grad_norm=112.363, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.655 +[gpue05] 2025-05-31 21:55:16,489 (trainer:401) INFO: 4epoch results: [train] iter_time=2.376e-04, forward_time=0.272, class_loss=3.105, geo_loss_downstream=0.399, inter_geo_loss_layer32=0.188, inter_geo_loss_layer36=0.198, inter_geo_loss_layer40=0.189, inter_geo_loss_layer44=0.196, inter_geo_loss_mean=0.193, geo_loss_all=0.316, loss=0.637, accuracy=0.882, backward_time=0.900, grad_norm=111.307, clip=0.000e+00, loss_scale=6.554e+04, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.749, time=41 minutes and 57.5 seconds, total_count=8000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=4.762, geo_loss_downstream=0.438, inter_geo_loss_layer32=0.152, inter_geo_loss_layer36=0.161, inter_geo_loss_layer40=0.236, inter_geo_loss_layer44=0.165, inter_geo_loss_mean=0.178, geo_loss_all=0.334, loss=3.876, accuracy=0.807, time=23 minutes and 24.48 seconds, total_count=18888, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-05-31 21:55:30,252 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-05-31 21:55:30,259 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/2epoch.pth +[gpue05] 2025-05-31 21:55:30,259 (trainer:335) INFO: 5/50epoch started. Estimated time to finish: 2 days, 1 hour and 56 minutes +[gpue05] 2025-05-31 21:59:55,363 (trainer:816) INFO: 5epoch:train:1-100batch: iter_time=0.003, forward_time=0.399, class_loss=2.947, geo_loss_downstream=0.391, inter_geo_loss_layer32=0.131, inter_geo_loss_layer36=0.145, inter_geo_loss_layer40=0.134, inter_geo_loss_layer44=0.141, inter_geo_loss_mean=0.138, geo_loss_all=0.290, loss=0.604, accuracy=0.876, backward_time=0.821, grad_norm=116.658, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.982 +[gpue05] 2025-05-31 22:02:02,819 (trainer:816) INFO: 5epoch:train:101-200batch: iter_time=1.136e-04, forward_time=0.355, class_loss=2.660, geo_loss_downstream=0.388, inter_geo_loss_layer32=0.126, inter_geo_loss_layer36=0.137, inter_geo_loss_layer40=0.129, inter_geo_loss_layer44=0.135, inter_geo_loss_mean=0.132, geo_loss_all=0.285, loss=0.546, accuracy=0.908, backward_time=0.904, grad_norm=115.169, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.098 +[gpue05] 2025-05-31 22:04:12,573 (trainer:816) INFO: 5epoch:train:201-300batch: iter_time=1.139e-04, forward_time=0.340, class_loss=2.940, geo_loss_downstream=0.386, inter_geo_loss_layer32=0.123, inter_geo_loss_layer36=0.134, inter_geo_loss_layer40=0.122, inter_geo_loss_layer44=0.129, inter_geo_loss_mean=0.127, geo_loss_all=0.282, loss=0.602, accuracy=0.895, backward_time=0.942, grad_norm=128.349, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.189 +[gpue05] 2025-05-31 22:06:25,450 (trainer:816) INFO: 5epoch:train:301-400batch: iter_time=1.223e-04, forward_time=0.327, class_loss=2.784, geo_loss_downstream=0.387, inter_geo_loss_layer32=0.119, inter_geo_loss_layer36=0.128, inter_geo_loss_layer40=0.119, inter_geo_loss_layer44=0.124, inter_geo_loss_mean=0.123, geo_loss_all=0.281, loss=0.571, accuracy=0.897, backward_time=0.989, grad_norm=107.064, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.314 +[gpue05] 2025-05-31 22:08:13,070 (trainer:816) INFO: 5epoch:train:401-500batch: iter_time=1.175e-04, forward_time=0.271, class_loss=2.332, geo_loss_downstream=0.384, inter_geo_loss_layer32=0.113, inter_geo_loss_layer36=0.122, inter_geo_loss_layer40=0.114, inter_geo_loss_layer44=0.122, inter_geo_loss_mean=0.118, geo_loss_all=0.277, loss=0.480, accuracy=0.902, backward_time=0.789, grad_norm=118.627, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.304 +[gpue05] 2025-05-31 22:10:15,300 (trainer:816) INFO: 5epoch:train:501-600batch: iter_time=1.164e-04, forward_time=0.293, class_loss=3.032, geo_loss_downstream=0.381, inter_geo_loss_layer32=0.108, inter_geo_loss_layer36=0.119, inter_geo_loss_layer40=0.111, inter_geo_loss_layer44=0.117, inter_geo_loss_mean=0.114, geo_loss_all=0.274, loss=0.620, accuracy=0.872, backward_time=0.913, grad_norm=115.729, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.888 +[gpue05] 2025-05-31 22:12:24,523 (trainer:816) INFO: 5epoch:train:601-700batch: iter_time=1.109e-04, forward_time=0.295, class_loss=2.725, geo_loss_downstream=0.381, inter_geo_loss_layer32=0.106, inter_geo_loss_layer36=0.116, inter_geo_loss_layer40=0.108, inter_geo_loss_layer44=0.113, inter_geo_loss_mean=0.111, geo_loss_all=0.273, loss=0.559, accuracy=0.900, backward_time=0.981, grad_norm=92.440, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.168 +[gpue05] 2025-05-31 22:14:23,528 (trainer:816) INFO: 5epoch:train:701-800batch: iter_time=1.131e-04, forward_time=0.248, class_loss=2.925, geo_loss_downstream=0.382, inter_geo_loss_layer32=0.103, inter_geo_loss_layer36=0.113, inter_geo_loss_layer40=0.103, inter_geo_loss_layer44=0.107, inter_geo_loss_mean=0.106, geo_loss_all=0.272, loss=0.599, accuracy=0.873, backward_time=0.926, grad_norm=95.313, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.759 +[gpue05] 2025-05-31 22:16:22,902 (trainer:816) INFO: 5epoch:train:801-900batch: iter_time=1.161e-04, forward_time=0.251, class_loss=2.752, geo_loss_downstream=0.380, inter_geo_loss_layer32=0.098, inter_geo_loss_layer36=0.105, inter_geo_loss_layer40=0.099, inter_geo_loss_layer44=0.104, inter_geo_loss_mean=0.102, geo_loss_all=0.269, loss=0.564, accuracy=0.892, backward_time=0.928, grad_norm=99.259, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.774 +[gpue05] 2025-05-31 22:18:10,630 (trainer:816) INFO: 5epoch:train:901-1000batch: iter_time=1.222e-04, forward_time=0.226, class_loss=2.551, geo_loss_downstream=0.379, inter_geo_loss_layer32=0.094, inter_geo_loss_layer36=0.105, inter_geo_loss_layer40=0.096, inter_geo_loss_layer44=0.104, inter_geo_loss_mean=0.100, geo_loss_all=0.267, loss=0.524, accuracy=0.898, backward_time=0.835, grad_norm=102.889, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.308 +[gpue05] 2025-05-31 22:20:09,185 (trainer:816) INFO: 5epoch:train:1001-1100batch: iter_time=1.142e-04, forward_time=0.238, class_loss=2.579, geo_loss_downstream=0.377, inter_geo_loss_layer32=0.090, inter_geo_loss_layer36=0.100, inter_geo_loss_layer40=0.090, inter_geo_loss_layer44=0.098, inter_geo_loss_mean=0.094, geo_loss_all=0.264, loss=0.529, accuracy=0.902, backward_time=0.932, grad_norm=99.120, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.741 +[gpue05] 2025-05-31 22:22:02,751 (trainer:816) INFO: 5epoch:train:1101-1200batch: iter_time=1.114e-04, forward_time=0.249, class_loss=2.986, geo_loss_downstream=0.376, inter_geo_loss_layer32=0.088, inter_geo_loss_layer36=0.097, inter_geo_loss_layer40=0.091, inter_geo_loss_layer44=0.098, inter_geo_loss_mean=0.094, geo_loss_all=0.263, loss=0.610, accuracy=0.882, backward_time=0.871, grad_norm=91.175, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.542 +[gpue05] 2025-05-31 22:24:06,329 (trainer:816) INFO: 5epoch:train:1201-1300batch: iter_time=1.156e-04, forward_time=0.248, class_loss=3.087, geo_loss_downstream=0.374, inter_geo_loss_layer32=0.088, inter_geo_loss_layer36=0.093, inter_geo_loss_layer40=0.087, inter_geo_loss_layer44=0.094, inter_geo_loss_mean=0.090, geo_loss_all=0.260, loss=0.630, accuracy=0.875, backward_time=0.973, grad_norm=118.183, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.942 +[gpue05] 2025-05-31 22:26:00,241 (trainer:816) INFO: 5epoch:train:1301-1400batch: iter_time=1.127e-04, forward_time=0.234, class_loss=2.543, geo_loss_downstream=0.374, inter_geo_loss_layer32=0.082, inter_geo_loss_layer36=0.090, inter_geo_loss_layer40=0.084, inter_geo_loss_layer44=0.090, inter_geo_loss_mean=0.086, geo_loss_all=0.259, loss=0.522, accuracy=0.907, backward_time=0.889, grad_norm=131.653, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.556 +[gpue05] 2025-05-31 22:27:50,442 (trainer:816) INFO: 5epoch:train:1401-1500batch: iter_time=1.139e-04, forward_time=0.227, class_loss=2.696, geo_loss_downstream=0.375, inter_geo_loss_layer32=0.080, inter_geo_loss_layer36=0.086, inter_geo_loss_layer40=0.082, inter_geo_loss_layer44=0.087, inter_geo_loss_mean=0.084, geo_loss_all=0.259, loss=0.552, accuracy=0.902, backward_time=0.859, grad_norm=118.347, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.407 +[gpue05] 2025-05-31 22:29:41,711 (trainer:816) INFO: 5epoch:train:1501-1600batch: iter_time=1.193e-04, forward_time=0.231, class_loss=2.942, geo_loss_downstream=0.374, inter_geo_loss_layer32=0.081, inter_geo_loss_layer36=0.088, inter_geo_loss_layer40=0.080, inter_geo_loss_layer44=0.084, inter_geo_loss_mean=0.083, geo_loss_all=0.258, loss=0.601, accuracy=0.882, backward_time=0.866, grad_norm=108.894, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.450 +[gpue05] 2025-05-31 22:31:46,637 (trainer:816) INFO: 5epoch:train:1601-1700batch: iter_time=1.151e-04, forward_time=0.255, class_loss=2.725, geo_loss_downstream=0.372, inter_geo_loss_layer32=0.077, inter_geo_loss_layer36=0.083, inter_geo_loss_layer40=0.078, inter_geo_loss_layer44=0.082, inter_geo_loss_mean=0.080, geo_loss_all=0.255, loss=0.558, accuracy=0.890, backward_time=0.981, grad_norm=113.583, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.996 +[gpue05] 2025-05-31 22:33:42,765 (trainer:816) INFO: 5epoch:train:1701-1800batch: iter_time=1.167e-04, forward_time=0.239, class_loss=2.970, geo_loss_downstream=0.373, inter_geo_loss_layer32=0.072, inter_geo_loss_layer36=0.080, inter_geo_loss_layer40=0.074, inter_geo_loss_layer44=0.078, inter_geo_loss_mean=0.076, geo_loss_all=0.254, loss=0.607, accuracy=0.885, backward_time=0.904, grad_norm=111.413, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.644 +[gpue05] 2025-05-31 22:35:37,516 (trainer:816) INFO: 5epoch:train:1801-1900batch: iter_time=1.145e-04, forward_time=0.228, class_loss=2.682, geo_loss_downstream=0.371, inter_geo_loss_layer32=0.073, inter_geo_loss_layer36=0.076, inter_geo_loss_layer40=0.072, inter_geo_loss_layer44=0.076, inter_geo_loss_mean=0.074, geo_loss_all=0.252, loss=0.549, accuracy=0.892, backward_time=0.904, grad_norm=106.706, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.589 +[gpue05] 2025-05-31 22:37:28,676 (trainer:816) INFO: 5epoch:train:1901-2000batch: iter_time=1.095e-04, forward_time=0.240, class_loss=2.555, geo_loss_downstream=0.370, inter_geo_loss_layer32=0.071, inter_geo_loss_layer36=0.076, inter_geo_loss_layer40=0.071, inter_geo_loss_layer44=0.074, inter_geo_loss_mean=0.073, geo_loss_all=0.251, loss=0.524, accuracy=0.903, backward_time=0.854, grad_norm=94.598, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.446 +[gpue05] 2025-05-31 23:00:52,043 (trainer:401) INFO: 5epoch results: [train] iter_time=2.551e-04, forward_time=0.270, class_loss=2.771, geo_loss_downstream=0.379, inter_geo_loss_layer32=0.096, inter_geo_loss_layer36=0.105, inter_geo_loss_layer40=0.097, inter_geo_loss_layer44=0.103, inter_geo_loss_mean=0.100, geo_loss_all=0.267, loss=0.567, accuracy=0.892, backward_time=0.903, grad_norm=109.258, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.755, time=41 minutes and 58.66 seconds, total_count=10000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=4.148, geo_loss_downstream=0.440, inter_geo_loss_layer32=0.094, inter_geo_loss_layer36=0.130, inter_geo_loss_layer40=0.106, inter_geo_loss_layer44=0.111, inter_geo_loss_mean=0.110, geo_loss_all=0.308, loss=3.380, accuracy=0.827, time=23 minutes and 23.11 seconds, total_count=23610, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-05-31 23:01:06,173 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-05-31 23:01:06,179 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/3epoch.pth +[gpue05] 2025-05-31 23:01:06,179 (trainer:335) INFO: 6/50epoch started. Estimated time to finish: 2 days, 55 minutes and 39.99 seconds +[gpue05] 2025-05-31 23:05:36,172 (trainer:816) INFO: 6epoch:train:1-100batch: iter_time=0.002, forward_time=0.401, class_loss=2.084, geo_loss_downstream=0.368, inter_geo_loss_layer32=0.068, inter_geo_loss_layer36=0.075, inter_geo_loss_layer40=0.070, inter_geo_loss_layer44=0.074, inter_geo_loss_mean=0.072, geo_loss_all=0.249, loss=0.429, accuracy=0.921, backward_time=0.855, grad_norm=85.994, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.114 +[gpue05] 2025-05-31 23:07:50,142 (trainer:816) INFO: 6epoch:train:101-200batch: iter_time=1.173e-04, forward_time=0.362, class_loss=2.603, geo_loss_downstream=0.367, inter_geo_loss_layer32=0.068, inter_geo_loss_layer36=0.072, inter_geo_loss_layer40=0.072, inter_geo_loss_layer44=0.072, inter_geo_loss_mean=0.071, geo_loss_all=0.249, loss=0.533, accuracy=0.905, backward_time=0.963, grad_norm=107.109, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.358 +[gpue05] 2025-05-31 23:10:00,281 (trainer:816) INFO: 6epoch:train:201-300batch: iter_time=1.147e-04, forward_time=0.339, class_loss=2.750, geo_loss_downstream=0.368, inter_geo_loss_layer32=0.064, inter_geo_loss_layer36=0.072, inter_geo_loss_layer40=0.069, inter_geo_loss_layer44=0.066, inter_geo_loss_mean=0.068, geo_loss_all=0.248, loss=0.562, accuracy=0.897, backward_time=0.946, grad_norm=111.804, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.205 +[gpue05] 2025-05-31 23:11:50,028 (trainer:816) INFO: 6epoch:train:301-400batch: iter_time=1.118e-04, forward_time=0.290, class_loss=2.596, geo_loss_downstream=0.366, inter_geo_loss_layer32=0.067, inter_geo_loss_layer36=0.070, inter_geo_loss_layer40=0.067, inter_geo_loss_layer44=0.069, inter_geo_loss_mean=0.068, geo_loss_all=0.247, loss=0.531, accuracy=0.893, backward_time=0.791, grad_norm=107.381, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.389 +[gpue05] 2025-05-31 23:13:50,676 (trainer:816) INFO: 6epoch:train:401-500batch: iter_time=1.173e-04, forward_time=0.301, class_loss=2.085, geo_loss_downstream=0.364, inter_geo_loss_layer32=0.061, inter_geo_loss_layer36=0.067, inter_geo_loss_layer40=0.065, inter_geo_loss_layer44=0.065, inter_geo_loss_mean=0.065, geo_loss_all=0.244, loss=0.429, accuracy=0.927, backward_time=0.890, grad_norm=91.691, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.825 +[gpue05] 2025-05-31 23:15:55,101 (trainer:816) INFO: 6epoch:train:501-600batch: iter_time=1.258e-04, forward_time=0.287, class_loss=2.429, geo_loss_downstream=0.361, inter_geo_loss_layer32=0.065, inter_geo_loss_layer36=0.067, inter_geo_loss_layer40=0.066, inter_geo_loss_layer44=0.065, inter_geo_loss_mean=0.066, geo_loss_all=0.243, loss=0.498, accuracy=0.910, backward_time=0.943, grad_norm=106.758, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.976 +[gpue05] 2025-05-31 23:17:56,106 (trainer:816) INFO: 6epoch:train:601-700batch: iter_time=1.120e-04, forward_time=0.277, class_loss=2.787, geo_loss_downstream=0.362, inter_geo_loss_layer32=0.063, inter_geo_loss_layer36=0.070, inter_geo_loss_layer40=0.064, inter_geo_loss_layer44=0.064, inter_geo_loss_mean=0.065, geo_loss_all=0.243, loss=0.570, accuracy=0.890, backward_time=0.916, grad_norm=124.137, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.839 +[gpue05] 2025-05-31 23:19:54,907 (trainer:816) INFO: 6epoch:train:701-800batch: iter_time=1.191e-04, forward_time=0.259, class_loss=2.442, geo_loss_downstream=0.362, inter_geo_loss_layer32=0.062, inter_geo_loss_layer36=0.066, inter_geo_loss_layer40=0.062, inter_geo_loss_layer44=0.060, inter_geo_loss_mean=0.062, geo_loss_all=0.242, loss=0.501, accuracy=0.908, backward_time=0.915, grad_norm=114.097, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.751 +[gpue05] 2025-05-31 23:21:55,685 (trainer:816) INFO: 6epoch:train:801-900batch: iter_time=1.093e-04, forward_time=0.244, class_loss=2.415, geo_loss_downstream=0.362, inter_geo_loss_layer32=0.062, inter_geo_loss_layer36=0.063, inter_geo_loss_layer40=0.060, inter_geo_loss_layer44=0.063, inter_geo_loss_mean=0.062, geo_loss_all=0.242, loss=0.495, accuracy=0.903, backward_time=0.948, grad_norm=105.562, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.830 +[gpue05] 2025-05-31 23:24:03,883 (trainer:816) INFO: 6epoch:train:901-1000batch: iter_time=1.161e-04, forward_time=0.279, class_loss=2.453, geo_loss_downstream=0.363, inter_geo_loss_layer32=0.062, inter_geo_loss_layer36=0.063, inter_geo_loss_layer40=0.063, inter_geo_loss_layer44=0.062, inter_geo_loss_mean=0.062, geo_loss_all=0.243, loss=0.503, accuracy=0.903, backward_time=0.987, grad_norm=96.834, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.127 +[gpue05] 2025-05-31 23:25:50,133 (trainer:816) INFO: 6epoch:train:1001-1100batch: iter_time=1.107e-04, forward_time=0.234, class_loss=3.065, geo_loss_downstream=0.362, inter_geo_loss_layer32=0.057, inter_geo_loss_layer36=0.062, inter_geo_loss_layer40=0.060, inter_geo_loss_layer44=0.062, inter_geo_loss_mean=0.060, geo_loss_all=0.241, loss=0.625, accuracy=0.875, backward_time=0.812, grad_norm=116.967, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.249 +[gpue05] 2025-05-31 23:27:39,707 (trainer:816) INFO: 6epoch:train:1101-1200batch: iter_time=1.101e-04, forward_time=0.237, class_loss=2.786, geo_loss_downstream=0.359, inter_geo_loss_layer32=0.062, inter_geo_loss_layer36=0.066, inter_geo_loss_layer40=0.063, inter_geo_loss_layer44=0.062, inter_geo_loss_mean=0.063, geo_loss_all=0.241, loss=0.569, accuracy=0.891, backward_time=0.841, grad_norm=86.273, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.382 +[gpue05] 2025-05-31 23:29:40,725 (trainer:816) INFO: 6epoch:train:1201-1300batch: iter_time=1.071e-04, forward_time=0.251, class_loss=2.471, geo_loss_downstream=0.358, inter_geo_loss_layer32=0.056, inter_geo_loss_layer36=0.062, inter_geo_loss_layer40=0.058, inter_geo_loss_layer44=0.062, inter_geo_loss_mean=0.059, geo_loss_all=0.239, loss=0.506, accuracy=0.905, backward_time=0.942, grad_norm=78.988, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.840 +[gpue05] 2025-05-31 23:31:37,706 (trainer:816) INFO: 6epoch:train:1301-1400batch: iter_time=1.101e-04, forward_time=0.243, class_loss=2.285, geo_loss_downstream=0.359, inter_geo_loss_layer32=0.057, inter_geo_loss_layer36=0.062, inter_geo_loss_layer40=0.060, inter_geo_loss_layer44=0.061, inter_geo_loss_mean=0.060, geo_loss_all=0.239, loss=0.469, accuracy=0.913, backward_time=0.912, grad_norm=111.610, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.679 +[gpue05] 2025-05-31 23:33:43,608 (trainer:816) INFO: 6epoch:train:1401-1500batch: iter_time=1.147e-04, forward_time=0.254, class_loss=2.365, geo_loss_downstream=0.357, inter_geo_loss_layer32=0.055, inter_geo_loss_layer36=0.062, inter_geo_loss_layer40=0.058, inter_geo_loss_layer44=0.057, inter_geo_loss_mean=0.058, geo_loss_all=0.238, loss=0.485, accuracy=0.898, backward_time=0.990, grad_norm=95.885, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=5.035 +[gpue05] 2025-05-31 23:35:41,704 (trainer:816) INFO: 6epoch:train:1501-1600batch: iter_time=1.261e-04, forward_time=0.245, class_loss=2.548, geo_loss_downstream=0.356, inter_geo_loss_layer32=0.057, inter_geo_loss_layer36=0.061, inter_geo_loss_layer40=0.056, inter_geo_loss_layer44=0.057, inter_geo_loss_mean=0.058, geo_loss_all=0.237, loss=0.521, accuracy=0.898, backward_time=0.921, grad_norm=95.963, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.723 +[gpue05] 2025-05-31 23:37:32,620 (trainer:816) INFO: 6epoch:train:1601-1700batch: iter_time=1.077e-04, forward_time=0.248, class_loss=1.905, geo_loss_downstream=0.355, inter_geo_loss_layer32=0.053, inter_geo_loss_layer36=0.059, inter_geo_loss_layer40=0.054, inter_geo_loss_layer44=0.056, inter_geo_loss_mean=0.055, geo_loss_all=0.235, loss=0.393, accuracy=0.927, backward_time=0.845, grad_norm=88.868, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.436 +[gpue05] 2025-05-31 23:39:22,944 (trainer:816) INFO: 6epoch:train:1701-1800batch: iter_time=1.055e-04, forward_time=0.241, class_loss=2.263, geo_loss_downstream=0.356, inter_geo_loss_layer32=0.055, inter_geo_loss_layer36=0.055, inter_geo_loss_layer40=0.056, inter_geo_loss_layer44=0.054, inter_geo_loss_mean=0.055, geo_loss_all=0.236, loss=0.464, accuracy=0.913, backward_time=0.846, grad_norm=87.747, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.412 +[gpue05] 2025-05-31 23:41:22,302 (trainer:816) INFO: 6epoch:train:1801-1900batch: iter_time=1.093e-04, forward_time=0.254, class_loss=2.340, geo_loss_downstream=0.352, inter_geo_loss_layer32=0.054, inter_geo_loss_layer36=0.054, inter_geo_loss_layer40=0.055, inter_geo_loss_layer44=0.055, inter_geo_loss_mean=0.055, geo_loss_all=0.233, loss=0.480, accuracy=0.903, backward_time=0.919, grad_norm=78.874, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.774 +[gpue05] 2025-05-31 23:43:09,394 (trainer:816) INFO: 6epoch:train:1901-2000batch: iter_time=1.023e-04, forward_time=0.226, class_loss=2.427, geo_loss_downstream=0.352, inter_geo_loss_layer32=0.053, inter_geo_loss_layer36=0.055, inter_geo_loss_layer40=0.053, inter_geo_loss_layer44=0.057, inter_geo_loss_mean=0.055, geo_loss_all=0.233, loss=0.497, accuracy=0.902, backward_time=0.828, grad_norm=97.008, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.283 +[gpue05] 2025-06-01 00:06:35,565 (trainer:401) INFO: 6epoch results: [train] iter_time=1.858e-04, forward_time=0.274, class_loss=2.455, geo_loss_downstream=0.360, inter_geo_loss_layer32=0.060, inter_geo_loss_layer36=0.064, inter_geo_loss_layer40=0.061, inter_geo_loss_layer44=0.062, inter_geo_loss_mean=0.062, geo_loss_all=0.241, loss=0.503, accuracy=0.904, backward_time=0.900, grad_norm=99.478, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.761, time=42 minutes and 3.49 seconds, total_count=12000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=4.033, geo_loss_downstream=0.410, inter_geo_loss_layer32=0.078, inter_geo_loss_layer36=0.090, inter_geo_loss_layer40=0.091, inter_geo_loss_layer44=0.098, inter_geo_loss_mean=0.089, geo_loss_all=0.282, loss=3.283, accuracy=0.832, time=23 minutes and 25.88 seconds, total_count=28332, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 00:06:50,140 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 00:06:50,147 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/4epoch.pth +[gpue05] 2025-06-01 00:06:50,147 (trainer:335) INFO: 7/50epoch started. Estimated time to finish: 1 day, 23 hours and 54 minutes +[gpue05] 2025-06-01 00:11:18,110 (trainer:816) INFO: 7epoch:train:1-100batch: iter_time=0.002, forward_time=0.396, class_loss=2.518, geo_loss_downstream=0.350, inter_geo_loss_layer32=0.057, inter_geo_loss_layer36=0.054, inter_geo_loss_layer40=0.056, inter_geo_loss_layer44=0.057, inter_geo_loss_mean=0.056, geo_loss_all=0.232, loss=0.515, accuracy=0.902, backward_time=0.847, grad_norm=106.871, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.065 +[gpue05] 2025-06-01 00:13:21,139 (trainer:816) INFO: 7epoch:train:101-200batch: iter_time=1.055e-04, forward_time=0.361, class_loss=2.612, geo_loss_downstream=0.351, inter_geo_loss_layer32=0.054, inter_geo_loss_layer36=0.055, inter_geo_loss_layer40=0.055, inter_geo_loss_layer44=0.056, inter_geo_loss_mean=0.055, geo_loss_all=0.232, loss=0.534, accuracy=0.886, backward_time=0.852, grad_norm=104.420, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.920 +[gpue05] 2025-06-01 00:15:30,031 (trainer:816) INFO: 7epoch:train:201-300batch: iter_time=9.992e-05, forward_time=0.348, class_loss=2.442, geo_loss_downstream=0.349, inter_geo_loss_layer32=0.054, inter_geo_loss_layer36=0.058, inter_geo_loss_layer40=0.054, inter_geo_loss_layer44=0.055, inter_geo_loss_mean=0.055, geo_loss_all=0.232, loss=0.500, accuracy=0.897, backward_time=0.925, grad_norm=70.039, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.155 +[gpue05] 2025-06-01 00:17:24,921 (trainer:816) INFO: 7epoch:train:301-400batch: iter_time=1.054e-04, forward_time=0.303, class_loss=2.676, geo_loss_downstream=0.349, inter_geo_loss_layer32=0.051, inter_geo_loss_layer36=0.055, inter_geo_loss_layer40=0.053, inter_geo_loss_layer44=0.055, inter_geo_loss_mean=0.054, geo_loss_all=0.231, loss=0.547, accuracy=0.890, backward_time=0.831, grad_norm=93.159, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.595 +[gpue05] 2025-06-01 00:19:14,606 (trainer:816) INFO: 7epoch:train:401-500batch: iter_time=9.522e-05, forward_time=0.276, class_loss=2.315, geo_loss_downstream=0.346, inter_geo_loss_layer32=0.050, inter_geo_loss_layer36=0.053, inter_geo_loss_layer40=0.051, inter_geo_loss_layer44=0.051, inter_geo_loss_mean=0.051, geo_loss_all=0.228, loss=0.474, accuracy=0.915, backward_time=0.805, grad_norm=77.336, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.387 +[gpue05] 2025-06-01 00:21:24,485 (trainer:816) INFO: 7epoch:train:501-600batch: iter_time=9.692e-05, forward_time=0.297, class_loss=2.686, geo_loss_downstream=0.347, inter_geo_loss_layer32=0.052, inter_geo_loss_layer36=0.053, inter_geo_loss_layer40=0.050, inter_geo_loss_layer44=0.051, inter_geo_loss_mean=0.051, geo_loss_all=0.229, loss=0.549, accuracy=0.895, backward_time=0.985, grad_norm=139.592, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=5.194 +[gpue05] 2025-06-01 00:23:28,597 (trainer:816) INFO: 7epoch:train:601-700batch: iter_time=9.304e-05, forward_time=0.277, class_loss=2.854, geo_loss_downstream=0.344, inter_geo_loss_layer32=0.050, inter_geo_loss_layer36=0.051, inter_geo_loss_layer40=0.051, inter_geo_loss_layer44=0.049, inter_geo_loss_mean=0.050, geo_loss_all=0.226, loss=0.582, accuracy=0.892, backward_time=0.950, grad_norm=103.519, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.964 +[gpue05] 2025-06-01 00:25:32,808 (trainer:816) INFO: 7epoch:train:701-800batch: iter_time=9.420e-05, forward_time=0.275, class_loss=2.302, geo_loss_downstream=0.345, inter_geo_loss_layer32=0.051, inter_geo_loss_layer36=0.052, inter_geo_loss_layer40=0.050, inter_geo_loss_layer44=0.054, inter_geo_loss_mean=0.052, geo_loss_all=0.228, loss=0.472, accuracy=0.908, backward_time=0.950, grad_norm=86.813, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.968 +[gpue05] 2025-06-01 00:27:37,301 (trainer:816) INFO: 7epoch:train:801-900batch: iter_time=9.941e-05, forward_time=0.259, class_loss=2.626, geo_loss_downstream=0.343, inter_geo_loss_layer32=0.050, inter_geo_loss_layer36=0.053, inter_geo_loss_layer40=0.053, inter_geo_loss_layer44=0.050, inter_geo_loss_mean=0.051, geo_loss_all=0.226, loss=0.537, accuracy=0.890, backward_time=0.971, grad_norm=78.196, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.979 +[gpue05] 2025-06-01 00:29:32,590 (trainer:816) INFO: 7epoch:train:901-1000batch: iter_time=9.941e-05, forward_time=0.241, class_loss=2.593, geo_loss_downstream=0.343, inter_geo_loss_layer32=0.050, inter_geo_loss_layer36=0.050, inter_geo_loss_layer40=0.048, inter_geo_loss_layer44=0.050, inter_geo_loss_mean=0.050, geo_loss_all=0.226, loss=0.530, accuracy=0.895, backward_time=0.896, grad_norm=113.077, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.611 +[gpue05] 2025-06-01 00:31:23,347 (trainer:816) INFO: 7epoch:train:1001-1100batch: iter_time=9.163e-05, forward_time=0.240, class_loss=2.558, geo_loss_downstream=0.341, inter_geo_loss_layer32=0.047, inter_geo_loss_layer36=0.050, inter_geo_loss_layer40=0.049, inter_geo_loss_layer44=0.053, inter_geo_loss_mean=0.049, geo_loss_all=0.225, loss=0.523, accuracy=0.895, backward_time=0.851, grad_norm=99.351, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.430 +[gpue05] 2025-06-01 00:33:12,887 (trainer:816) INFO: 7epoch:train:1101-1200batch: iter_time=9.269e-05, forward_time=0.237, class_loss=2.345, geo_loss_downstream=0.342, inter_geo_loss_layer32=0.045, inter_geo_loss_layer36=0.048, inter_geo_loss_layer40=0.049, inter_geo_loss_layer44=0.051, inter_geo_loss_mean=0.048, geo_loss_all=0.224, loss=0.480, accuracy=0.910, backward_time=0.843, grad_norm=91.271, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.381 +[gpue05] 2025-06-01 00:35:24,153 (trainer:816) INFO: 7epoch:train:1201-1300batch: iter_time=9.357e-05, forward_time=0.261, class_loss=2.356, geo_loss_downstream=0.341, inter_geo_loss_layer32=0.048, inter_geo_loss_layer36=0.050, inter_geo_loss_layer40=0.049, inter_geo_loss_layer44=0.053, inter_geo_loss_mean=0.050, geo_loss_all=0.224, loss=0.482, accuracy=0.915, backward_time=1.038, grad_norm=107.503, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.250 +[gpue05] 2025-06-01 00:37:39,277 (trainer:816) INFO: 7epoch:train:1301-1400batch: iter_time=9.304e-05, forward_time=0.264, class_loss=2.420, geo_loss_downstream=0.339, inter_geo_loss_layer32=0.046, inter_geo_loss_layer36=0.050, inter_geo_loss_layer40=0.048, inter_geo_loss_layer44=0.050, inter_geo_loss_mean=0.048, geo_loss_all=0.223, loss=0.495, accuracy=0.905, backward_time=1.073, grad_norm=94.424, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=5.404 +[gpue05] 2025-06-01 00:39:32,235 (trainer:816) INFO: 7epoch:train:1401-1500batch: iter_time=9.660e-05, forward_time=0.223, class_loss=2.537, geo_loss_downstream=0.339, inter_geo_loss_layer32=0.046, inter_geo_loss_layer36=0.049, inter_geo_loss_layer40=0.048, inter_geo_loss_layer44=0.051, inter_geo_loss_mean=0.048, geo_loss_all=0.223, loss=0.519, accuracy=0.902, backward_time=0.892, grad_norm=95.812, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.518 +[gpue05] 2025-06-01 00:41:37,575 (trainer:816) INFO: 7epoch:train:1501-1600batch: iter_time=1.100e-04, forward_time=0.259, class_loss=2.758, geo_loss_downstream=0.337, inter_geo_loss_layer32=0.046, inter_geo_loss_layer36=0.049, inter_geo_loss_layer40=0.045, inter_geo_loss_layer44=0.049, inter_geo_loss_mean=0.047, geo_loss_all=0.221, loss=0.563, accuracy=0.895, backward_time=0.981, grad_norm=88.046, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.013 +[gpue05] 2025-06-01 00:43:39,733 (trainer:816) INFO: 7epoch:train:1601-1700batch: iter_time=1.006e-04, forward_time=0.251, class_loss=2.507, geo_loss_downstream=0.337, inter_geo_loss_layer32=0.047, inter_geo_loss_layer36=0.049, inter_geo_loss_layer40=0.047, inter_geo_loss_layer44=0.046, inter_geo_loss_mean=0.047, geo_loss_all=0.221, loss=0.512, accuracy=0.898, backward_time=0.955, grad_norm=95.253, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.886 +[gpue05] 2025-06-01 00:45:49,217 (trainer:816) INFO: 7epoch:train:1701-1800batch: iter_time=1.017e-04, forward_time=0.264, class_loss=2.155, geo_loss_downstream=0.333, inter_geo_loss_layer32=0.043, inter_geo_loss_layer36=0.047, inter_geo_loss_layer40=0.045, inter_geo_loss_layer44=0.044, inter_geo_loss_mean=0.045, geo_loss_all=0.218, loss=0.442, accuracy=0.920, backward_time=1.015, grad_norm=85.422, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.179 +[gpue05] 2025-06-01 00:47:51,901 (trainer:816) INFO: 7epoch:train:1801-1900batch: iter_time=1.087e-04, forward_time=0.256, class_loss=2.468, geo_loss_downstream=0.334, inter_geo_loss_layer32=0.044, inter_geo_loss_layer36=0.045, inter_geo_loss_layer40=0.044, inter_geo_loss_layer44=0.045, inter_geo_loss_mean=0.044, geo_loss_all=0.218, loss=0.505, accuracy=0.897, backward_time=0.956, grad_norm=103.569, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.907 +[gpue05] 2025-06-01 00:49:55,416 (trainer:816) INFO: 7epoch:train:1901-2000batch: iter_time=9.146e-05, forward_time=0.244, class_loss=2.669, geo_loss_downstream=0.334, inter_geo_loss_layer32=0.044, inter_geo_loss_layer36=0.047, inter_geo_loss_layer40=0.047, inter_geo_loss_layer44=0.043, inter_geo_loss_mean=0.045, geo_loss_all=0.218, loss=0.545, accuracy=0.896, backward_time=0.976, grad_norm=98.926, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.940 +[gpue05] 2025-06-01 01:13:18,335 (trainer:401) INFO: 7epoch results: [train] iter_time=2.028e-04, forward_time=0.277, class_loss=2.520, geo_loss_downstream=0.342, inter_geo_loss_layer32=0.049, inter_geo_loss_layer36=0.051, inter_geo_loss_layer40=0.050, inter_geo_loss_layer44=0.051, inter_geo_loss_mean=0.050, geo_loss_all=0.225, loss=0.515, accuracy=0.900, backward_time=0.930, grad_norm=96.630, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.887, time=43 minutes and 5.55 seconds, total_count=14000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=4.123, geo_loss_downstream=0.385, inter_geo_loss_layer32=0.063, inter_geo_loss_layer36=0.072, inter_geo_loss_layer40=0.071, inter_geo_loss_layer44=0.076, inter_geo_loss_mean=0.071, geo_loss_all=0.259, loss=3.351, accuracy=0.827, time=23 minutes and 22.63 seconds, total_count=33054, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 01:13:33,596 (trainer:467) INFO: There are no improvements in this epoch +[gpue05] 2025-06-01 01:13:33,602 (trainer:335) INFO: 8/50epoch started. Estimated time to finish: 1 day, 22 hours and 57 minutes +[gpue05] 2025-06-01 01:18:06,445 (trainer:816) INFO: 8epoch:train:1-100batch: iter_time=0.005, forward_time=0.390, class_loss=2.198, geo_loss_downstream=0.333, inter_geo_loss_layer32=0.043, inter_geo_loss_layer36=0.045, inter_geo_loss_layer40=0.047, inter_geo_loss_layer44=0.046, inter_geo_loss_mean=0.045, geo_loss_all=0.218, loss=0.450, accuracy=0.912, backward_time=0.896, grad_norm=84.362, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.239 +[gpue05] 2025-06-01 01:20:02,962 (trainer:816) INFO: 8epoch:train:101-200batch: iter_time=1.004e-04, forward_time=0.342, class_loss=2.251, geo_loss_downstream=0.330, inter_geo_loss_layer32=0.042, inter_geo_loss_layer36=0.047, inter_geo_loss_layer40=0.044, inter_geo_loss_layer44=0.043, inter_geo_loss_mean=0.044, geo_loss_all=0.216, loss=0.461, accuracy=0.897, backward_time=0.809, grad_norm=91.262, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.660 +[gpue05] 2025-06-01 01:22:02,523 (trainer:816) INFO: 8epoch:train:201-300batch: iter_time=9.924e-05, forward_time=0.327, class_loss=2.185, geo_loss_downstream=0.330, inter_geo_loss_layer32=0.044, inter_geo_loss_layer36=0.046, inter_geo_loss_layer40=0.044, inter_geo_loss_layer44=0.043, inter_geo_loss_mean=0.044, geo_loss_all=0.216, loss=0.448, accuracy=0.918, backward_time=0.854, grad_norm=116.012, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.782 +[gpue05] 2025-06-01 01:23:52,816 (trainer:816) INFO: 8epoch:train:301-400batch: iter_time=1.009e-04, forward_time=0.292, class_loss=1.950, geo_loss_downstream=0.330, inter_geo_loss_layer32=0.042, inter_geo_loss_layer36=0.046, inter_geo_loss_layer40=0.043, inter_geo_loss_layer44=0.043, inter_geo_loss_mean=0.043, geo_loss_all=0.215, loss=0.401, accuracy=0.920, backward_time=0.794, grad_norm=70.419, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.411 +[gpue05] 2025-06-01 01:25:55,671 (trainer:816) INFO: 8epoch:train:401-500batch: iter_time=1.051e-04, forward_time=0.299, class_loss=2.501, geo_loss_downstream=0.331, inter_geo_loss_layer32=0.043, inter_geo_loss_layer36=0.046, inter_geo_loss_layer40=0.043, inter_geo_loss_layer44=0.043, inter_geo_loss_mean=0.044, geo_loss_all=0.216, loss=0.511, accuracy=0.897, backward_time=0.915, grad_norm=107.394, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.913 +[gpue05] 2025-06-01 01:28:06,765 (trainer:816) INFO: 8epoch:train:501-600batch: iter_time=1.083e-04, forward_time=0.301, class_loss=2.261, geo_loss_downstream=0.329, inter_geo_loss_layer32=0.043, inter_geo_loss_layer36=0.045, inter_geo_loss_layer40=0.043, inter_geo_loss_layer44=0.045, inter_geo_loss_mean=0.044, geo_loss_all=0.215, loss=0.463, accuracy=0.912, backward_time=0.995, grad_norm=82.445, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.243 +[gpue05] 2025-06-01 01:30:08,993 (trainer:816) INFO: 8epoch:train:601-700batch: iter_time=9.984e-05, forward_time=0.283, class_loss=2.622, geo_loss_downstream=0.328, inter_geo_loss_layer32=0.044, inter_geo_loss_layer36=0.046, inter_geo_loss_layer40=0.043, inter_geo_loss_layer44=0.045, inter_geo_loss_mean=0.045, geo_loss_all=0.215, loss=0.535, accuracy=0.896, backward_time=0.923, grad_norm=99.987, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.888 +[gpue05] 2025-06-01 01:32:15,975 (trainer:816) INFO: 8epoch:train:701-800batch: iter_time=1.087e-04, forward_time=0.275, class_loss=2.590, geo_loss_downstream=0.322, inter_geo_loss_layer32=0.041, inter_geo_loss_layer36=0.044, inter_geo_loss_layer40=0.045, inter_geo_loss_layer44=0.046, inter_geo_loss_mean=0.044, geo_loss_all=0.211, loss=0.529, accuracy=0.893, backward_time=0.979, grad_norm=73.295, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.079 +[gpue05] 2025-06-01 01:34:35,777 (trainer:816) INFO: 8epoch:train:801-900batch: iter_time=1.393e-04, forward_time=0.276, class_loss=2.359, geo_loss_downstream=0.328, inter_geo_loss_layer32=0.044, inter_geo_loss_layer36=0.047, inter_geo_loss_layer40=0.045, inter_geo_loss_layer44=0.045, inter_geo_loss_mean=0.045, geo_loss_all=0.215, loss=0.483, accuracy=0.905, backward_time=1.108, grad_norm=94.526, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.591 +[gpue05] 2025-06-01 01:36:33,703 (trainer:816) INFO: 8epoch:train:901-1000batch: iter_time=1.107e-04, forward_time=0.245, class_loss=2.473, geo_loss_downstream=0.324, inter_geo_loss_layer32=0.041, inter_geo_loss_layer36=0.043, inter_geo_loss_layer40=0.040, inter_geo_loss_layer44=0.043, inter_geo_loss_mean=0.042, geo_loss_all=0.211, loss=0.505, accuracy=0.902, backward_time=0.920, grad_norm=85.942, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=4.716 +[gpue05] 2025-06-01 01:38:23,888 (trainer:816) INFO: 8epoch:train:1001-1100batch: iter_time=1.069e-04, forward_time=0.211, class_loss=2.505, geo_loss_downstream=0.324, inter_geo_loss_layer32=0.041, inter_geo_loss_layer36=0.044, inter_geo_loss_layer40=0.041, inter_geo_loss_layer44=0.042, inter_geo_loss_mean=0.042, geo_loss_all=0.211, loss=0.512, accuracy=0.902, backward_time=0.875, grad_norm=88.247, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.407 +[gpue05] 2025-06-01 01:40:22,852 (trainer:816) INFO: 8epoch:train:1101-1200batch: iter_time=1.105e-04, forward_time=0.261, class_loss=2.170, geo_loss_downstream=0.325, inter_geo_loss_layer32=0.041, inter_geo_loss_layer36=0.041, inter_geo_loss_layer40=0.040, inter_geo_loss_layer44=0.041, inter_geo_loss_mean=0.041, geo_loss_all=0.211, loss=0.445, accuracy=0.907, backward_time=0.913, grad_norm=94.298, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=4.758 +[gpue05] 2025-06-01 01:42:21,107 (trainer:816) INFO: 8epoch:train:1201-1300batch: iter_time=9.616e-05, forward_time=0.251, class_loss=2.061, geo_loss_downstream=0.324, inter_geo_loss_layer32=0.040, inter_geo_loss_layer36=0.044, inter_geo_loss_layer40=0.039, inter_geo_loss_layer44=0.041, inter_geo_loss_mean=0.041, geo_loss_all=0.211, loss=0.423, accuracy=0.913, backward_time=0.915, grad_norm=77.080, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=4.729 +[gpue05] 2025-06-01 01:44:27,220 (trainer:816) INFO: 8epoch:train:1301-1400batch: iter_time=1.000e-04, forward_time=0.281, class_loss=2.149, geo_loss_downstream=0.320, inter_geo_loss_layer32=0.040, inter_geo_loss_layer36=0.041, inter_geo_loss_layer40=0.038, inter_geo_loss_layer44=0.041, inter_geo_loss_mean=0.040, geo_loss_all=0.208, loss=0.440, accuracy=0.915, backward_time=0.964, grad_norm=109.376, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.044 +[gpue05] 2025-06-01 01:46:22,104 (trainer:816) INFO: 8epoch:train:1401-1500batch: iter_time=1.138e-04, forward_time=0.249, class_loss=2.558, geo_loss_downstream=0.321, inter_geo_loss_layer32=0.040, inter_geo_loss_layer36=0.043, inter_geo_loss_layer40=0.039, inter_geo_loss_layer44=0.041, inter_geo_loss_mean=0.041, geo_loss_all=0.209, loss=0.522, accuracy=0.893, backward_time=0.882, grad_norm=86.709, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.595 +[gpue05] 2025-06-01 01:48:15,192 (trainer:816) INFO: 8epoch:train:1501-1600batch: iter_time=1.131e-04, forward_time=0.237, class_loss=2.479, geo_loss_downstream=0.322, inter_geo_loss_layer32=0.039, inter_geo_loss_layer36=0.041, inter_geo_loss_layer40=0.039, inter_geo_loss_layer44=0.042, inter_geo_loss_mean=0.040, geo_loss_all=0.209, loss=0.506, accuracy=0.897, backward_time=0.877, grad_norm=80.134, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=4.523 +[gpue05] 2025-06-01 01:50:21,683 (trainer:816) INFO: 8epoch:train:1601-1700batch: iter_time=1.147e-04, forward_time=0.262, class_loss=2.154, geo_loss_downstream=0.319, inter_geo_loss_layer32=0.038, inter_geo_loss_layer36=0.043, inter_geo_loss_layer40=0.039, inter_geo_loss_layer44=0.040, inter_geo_loss_mean=0.040, geo_loss_all=0.208, loss=0.441, accuracy=0.912, backward_time=0.988, grad_norm=91.820, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.059 +[gpue05] 2025-06-01 01:52:22,340 (trainer:816) INFO: 8epoch:train:1701-1800batch: iter_time=1.216e-04, forward_time=0.271, class_loss=2.445, geo_loss_downstream=0.321, inter_geo_loss_layer32=0.040, inter_geo_loss_layer36=0.042, inter_geo_loss_layer40=0.040, inter_geo_loss_layer44=0.042, inter_geo_loss_mean=0.041, geo_loss_all=0.209, loss=0.499, accuracy=0.908, backward_time=0.919, grad_norm=92.792, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.826 +[gpue05] 2025-06-01 01:54:12,119 (trainer:816) INFO: 8epoch:train:1801-1900batch: iter_time=9.913e-05, forward_time=0.228, class_loss=2.254, geo_loss_downstream=0.319, inter_geo_loss_layer32=0.038, inter_geo_loss_layer36=0.041, inter_geo_loss_layer40=0.040, inter_geo_loss_layer44=0.039, inter_geo_loss_mean=0.040, geo_loss_all=0.208, loss=0.461, accuracy=0.910, backward_time=0.855, grad_norm=83.771, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=4.390 +[gpue05] 2025-06-01 01:56:24,471 (trainer:816) INFO: 8epoch:train:1901-2000batch: iter_time=1.076e-04, forward_time=0.267, class_loss=2.219, geo_loss_downstream=0.317, inter_geo_loss_layer32=0.039, inter_geo_loss_layer36=0.042, inter_geo_loss_layer40=0.040, inter_geo_loss_layer44=0.038, inter_geo_loss_mean=0.040, geo_loss_all=0.206, loss=0.454, accuracy=0.910, backward_time=1.040, grad_norm=83.055, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.293 +[gpue05] 2025-06-01 02:19:59,298 (trainer:401) INFO: 8epoch results: [train] iter_time=3.317e-04, forward_time=0.277, class_loss=2.319, geo_loss_downstream=0.325, inter_geo_loss_layer32=0.041, inter_geo_loss_layer36=0.044, inter_geo_loss_layer40=0.042, inter_geo_loss_layer44=0.042, inter_geo_loss_mean=0.042, geo_loss_all=0.212, loss=0.474, accuracy=0.906, backward_time=0.921, grad_norm=89.646, clip=0.000e+00, loss_scale=1.311e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.857, time=42 minutes and 51.17 seconds, total_count=16000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=3.747, geo_loss_downstream=0.372, inter_geo_loss_layer32=0.056, inter_geo_loss_layer36=0.063, inter_geo_loss_layer40=0.061, inter_geo_loss_layer44=0.069, inter_geo_loss_mean=0.062, geo_loss_all=0.248, loss=3.047, accuracy=0.840, time=23 minutes and 34.51 seconds, total_count=37776, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 02:20:13,876 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 02:20:13,885 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/5epoch.pth, exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/7epoch.pth +[gpue05] 2025-06-01 02:20:13,885 (trainer:335) INFO: 9/50epoch started. Estimated time to finish: 1 day, 21 hours and 57 minutes +[gpue05] 2025-06-01 02:25:05,132 (trainer:816) INFO: 9epoch:train:1-100batch: iter_time=0.002, forward_time=0.417, class_loss=2.079, geo_loss_downstream=0.316, inter_geo_loss_layer32=0.038, inter_geo_loss_layer36=0.039, inter_geo_loss_layer40=0.038, inter_geo_loss_layer44=0.039, inter_geo_loss_mean=0.039, geo_loss_all=0.205, loss=0.426, accuracy=0.915, backward_time=1.016, grad_norm=88.816, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.828 +[gpue05] 2025-06-01 02:27:29,647 (trainer:816) INFO: 9epoch:train:101-200batch: iter_time=1.505e-04, forward_time=0.396, class_loss=2.086, geo_loss_downstream=0.316, inter_geo_loss_layer32=0.039, inter_geo_loss_layer36=0.041, inter_geo_loss_layer40=0.038, inter_geo_loss_layer44=0.039, inter_geo_loss_mean=0.039, geo_loss_all=0.205, loss=0.427, accuracy=0.918, backward_time=1.034, grad_norm=86.932, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.780 +[gpue05] 2025-06-01 02:29:39,385 (trainer:816) INFO: 9epoch:train:201-300batch: iter_time=1.333e-04, forward_time=0.341, class_loss=2.442, geo_loss_downstream=0.315, inter_geo_loss_layer32=0.038, inter_geo_loss_layer36=0.040, inter_geo_loss_layer40=0.037, inter_geo_loss_layer44=0.039, inter_geo_loss_mean=0.038, geo_loss_all=0.205, loss=0.499, accuracy=0.903, backward_time=0.941, grad_norm=87.873, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.189 +[gpue05] 2025-06-01 02:31:41,911 (trainer:816) INFO: 9epoch:train:301-400batch: iter_time=1.410e-04, forward_time=0.309, class_loss=2.139, geo_loss_downstream=0.314, inter_geo_loss_layer32=0.036, inter_geo_loss_layer36=0.037, inter_geo_loss_layer40=0.034, inter_geo_loss_layer44=0.037, inter_geo_loss_mean=0.036, geo_loss_all=0.203, loss=0.438, accuracy=0.912, backward_time=0.901, grad_norm=82.642, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.900 +[gpue05] 2025-06-01 02:33:55,458 (trainer:816) INFO: 9epoch:train:401-500batch: iter_time=1.485e-04, forward_time=0.323, class_loss=2.246, geo_loss_downstream=0.315, inter_geo_loss_layer32=0.036, inter_geo_loss_layer36=0.039, inter_geo_loss_layer40=0.036, inter_geo_loss_layer44=0.038, inter_geo_loss_mean=0.037, geo_loss_all=0.204, loss=0.459, accuracy=0.913, backward_time=0.998, grad_norm=91.331, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.341 +[gpue05] 2025-06-01 02:36:06,174 (trainer:816) INFO: 9epoch:train:501-600batch: iter_time=1.441e-04, forward_time=0.304, class_loss=2.648, geo_loss_downstream=0.313, inter_geo_loss_layer32=0.037, inter_geo_loss_layer36=0.038, inter_geo_loss_layer40=0.038, inter_geo_loss_layer44=0.037, inter_geo_loss_mean=0.038, geo_loss_all=0.203, loss=0.540, accuracy=0.893, backward_time=0.988, grad_norm=104.114, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.228 +[gpue05] 2025-06-01 02:38:00,183 (trainer:816) INFO: 9epoch:train:601-700batch: iter_time=1.369e-04, forward_time=0.263, class_loss=2.391, geo_loss_downstream=0.312, inter_geo_loss_layer32=0.037, inter_geo_loss_layer36=0.040, inter_geo_loss_layer40=0.037, inter_geo_loss_layer44=0.039, inter_geo_loss_mean=0.038, geo_loss_all=0.202, loss=0.488, accuracy=0.902, backward_time=0.861, grad_norm=80.403, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=4.560 +[gpue05] 2025-06-01 02:39:58,959 (trainer:816) INFO: 9epoch:train:701-800batch: iter_time=1.431e-04, forward_time=0.266, class_loss=2.197, geo_loss_downstream=0.311, inter_geo_loss_layer32=0.037, inter_geo_loss_layer36=0.040, inter_geo_loss_layer40=0.038, inter_geo_loss_layer44=0.038, inter_geo_loss_mean=0.038, geo_loss_all=0.202, loss=0.449, accuracy=0.913, backward_time=0.905, grad_norm=90.698, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.750 +[gpue05] 2025-06-01 02:42:12,114 (trainer:816) INFO: 9epoch:train:801-900batch: iter_time=1.510e-04, forward_time=0.276, class_loss=1.845, geo_loss_downstream=0.312, inter_geo_loss_layer32=0.036, inter_geo_loss_layer36=0.037, inter_geo_loss_layer40=0.037, inter_geo_loss_layer44=0.038, inter_geo_loss_mean=0.037, geo_loss_all=0.202, loss=0.379, accuracy=0.927, backward_time=1.042, grad_norm=97.292, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.325 +[gpue05] 2025-06-01 02:44:05,441 (trainer:816) INFO: 9epoch:train:901-1000batch: iter_time=1.495e-04, forward_time=0.234, class_loss=2.290, geo_loss_downstream=0.309, inter_geo_loss_layer32=0.036, inter_geo_loss_layer36=0.037, inter_geo_loss_layer40=0.035, inter_geo_loss_layer44=0.036, inter_geo_loss_mean=0.036, geo_loss_all=0.200, loss=0.468, accuracy=0.910, backward_time=0.884, grad_norm=91.323, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.532 +[gpue05] 2025-06-01 02:45:59,890 (trainer:816) INFO: 9epoch:train:1001-1100batch: iter_time=1.420e-04, forward_time=0.245, class_loss=2.198, geo_loss_downstream=0.312, inter_geo_loss_layer32=0.037, inter_geo_loss_layer36=0.038, inter_geo_loss_layer40=0.038, inter_geo_loss_layer44=0.037, inter_geo_loss_mean=0.037, geo_loss_all=0.202, loss=0.450, accuracy=0.905, backward_time=0.883, grad_norm=121.731, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.577 +[gpue05] 2025-06-01 02:47:58,800 (trainer:816) INFO: 9epoch:train:1101-1200batch: iter_time=1.423e-04, forward_time=0.260, class_loss=2.394, geo_loss_downstream=0.308, inter_geo_loss_layer32=0.036, inter_geo_loss_layer36=0.039, inter_geo_loss_layer40=0.037, inter_geo_loss_layer44=0.036, inter_geo_loss_mean=0.037, geo_loss_all=0.200, loss=0.489, accuracy=0.903, backward_time=0.912, grad_norm=98.103, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.756 +[gpue05] 2025-06-01 02:49:46,790 (trainer:816) INFO: 9epoch:train:1201-1300batch: iter_time=1.292e-04, forward_time=0.230, class_loss=1.756, geo_loss_downstream=0.308, inter_geo_loss_layer32=0.035, inter_geo_loss_layer36=0.035, inter_geo_loss_layer40=0.034, inter_geo_loss_layer44=0.036, inter_geo_loss_mean=0.035, geo_loss_all=0.199, loss=0.361, accuracy=0.933, backward_time=0.833, grad_norm=70.576, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.319 +[gpue05] 2025-06-01 02:51:39,838 (trainer:816) INFO: 9epoch:train:1301-1400batch: iter_time=1.459e-04, forward_time=0.244, class_loss=2.085, geo_loss_downstream=0.306, inter_geo_loss_layer32=0.034, inter_geo_loss_layer36=0.035, inter_geo_loss_layer40=0.034, inter_geo_loss_layer44=0.035, inter_geo_loss_mean=0.034, geo_loss_all=0.198, loss=0.427, accuracy=0.923, backward_time=0.870, grad_norm=64.528, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.521 +[gpue05] 2025-06-01 02:53:15,963 (trainer:816) INFO: 9epoch:train:1401-1500batch: iter_time=1.410e-04, forward_time=0.210, class_loss=2.584, geo_loss_downstream=0.304, inter_geo_loss_layer32=0.036, inter_geo_loss_layer36=0.037, inter_geo_loss_layer40=0.037, inter_geo_loss_layer44=0.035, inter_geo_loss_mean=0.036, geo_loss_all=0.197, loss=0.527, accuracy=0.903, backward_time=0.733, grad_norm=94.171, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=3.844 +[gpue05] 2025-06-01 02:55:03,699 (trainer:816) INFO: 9epoch:train:1501-1600batch: iter_time=1.400e-04, forward_time=0.242, class_loss=1.792, geo_loss_downstream=0.305, inter_geo_loss_layer32=0.034, inter_geo_loss_layer36=0.035, inter_geo_loss_layer40=0.035, inter_geo_loss_layer44=0.037, inter_geo_loss_mean=0.035, geo_loss_all=0.197, loss=0.368, accuracy=0.927, backward_time=0.818, grad_norm=79.291, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.309 +[gpue05] 2025-06-01 02:57:08,995 (trainer:816) INFO: 9epoch:train:1601-1700batch: iter_time=1.450e-04, forward_time=0.270, class_loss=2.331, geo_loss_downstream=0.304, inter_geo_loss_layer32=0.036, inter_geo_loss_layer36=0.037, inter_geo_loss_layer40=0.035, inter_geo_loss_layer44=0.035, inter_geo_loss_mean=0.036, geo_loss_all=0.197, loss=0.476, accuracy=0.900, backward_time=0.967, grad_norm=97.221, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.011 +[gpue05] 2025-06-01 02:59:03,381 (trainer:816) INFO: 9epoch:train:1701-1800batch: iter_time=1.435e-04, forward_time=0.250, class_loss=2.192, geo_loss_downstream=0.302, inter_geo_loss_layer32=0.033, inter_geo_loss_layer36=0.036, inter_geo_loss_layer40=0.034, inter_geo_loss_layer44=0.035, inter_geo_loss_mean=0.034, geo_loss_all=0.195, loss=0.448, accuracy=0.915, backward_time=0.876, grad_norm=85.535, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.575 +[gpue05] 2025-06-01 03:01:03,771 (trainer:816) INFO: 9epoch:train:1801-1900batch: iter_time=1.477e-04, forward_time=0.266, class_loss=2.596, geo_loss_downstream=0.305, inter_geo_loss_layer32=0.036, inter_geo_loss_layer36=0.036, inter_geo_loss_layer40=0.035, inter_geo_loss_layer44=0.035, inter_geo_loss_mean=0.036, geo_loss_all=0.197, loss=0.529, accuracy=0.901, backward_time=0.922, grad_norm=89.967, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.815 +[gpue05] 2025-06-01 03:02:50,857 (trainer:816) INFO: 9epoch:train:1901-2000batch: iter_time=1.387e-04, forward_time=0.229, class_loss=2.116, geo_loss_downstream=0.301, inter_geo_loss_layer32=0.033, inter_geo_loss_layer36=0.035, inter_geo_loss_layer40=0.034, inter_geo_loss_layer44=0.034, inter_geo_loss_mean=0.034, geo_loss_all=0.194, loss=0.433, accuracy=0.918, backward_time=0.825, grad_norm=74.717, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.283 +[gpue05] 2025-06-01 03:26:19,143 (trainer:401) INFO: 9epoch results: [train] iter_time=2.415e-04, forward_time=0.279, class_loss=2.220, geo_loss_downstream=0.309, inter_geo_loss_layer32=0.036, inter_geo_loss_layer36=0.037, inter_geo_loss_layer40=0.036, inter_geo_loss_layer44=0.037, inter_geo_loss_mean=0.037, geo_loss_all=0.200, loss=0.454, accuracy=0.912, backward_time=0.910, grad_norm=88.863, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.822, time=42 minutes and 37.24 seconds, total_count=18000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=3.884, geo_loss_downstream=0.353, inter_geo_loss_layer32=0.052, inter_geo_loss_layer36=0.061, inter_geo_loss_layer40=0.060, inter_geo_loss_layer44=0.068, inter_geo_loss_mean=0.060, geo_loss_all=0.236, loss=3.154, accuracy=0.834, time=23 minutes and 28.01 seconds, total_count=42498, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 03:26:33,251 (trainer:467) INFO: There are no improvements in this epoch +[gpue05] 2025-06-01 03:26:33,257 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/6epoch.pth +[gpue05] 2025-06-01 03:26:33,258 (trainer:335) INFO: 10/50epoch started. Estimated time to finish: 1 day, 20 hours and 55 minutes +[gpue05] 2025-06-01 03:31:08,597 (trainer:816) INFO: 10epoch:train:1-100batch: iter_time=0.005, forward_time=0.396, class_loss=2.792, geo_loss_downstream=0.300, inter_geo_loss_layer32=0.034, inter_geo_loss_layer36=0.036, inter_geo_loss_layer40=0.034, inter_geo_loss_layer44=0.036, inter_geo_loss_mean=0.035, geo_loss_all=0.194, loss=0.568, accuracy=0.875, backward_time=0.916, grad_norm=94.990, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.358 +[gpue05] 2025-06-01 03:33:08,370 (trainer:816) INFO: 10epoch:train:101-200batch: iter_time=1.276e-04, forward_time=0.352, class_loss=2.076, geo_loss_downstream=0.299, inter_geo_loss_layer32=0.032, inter_geo_loss_layer36=0.034, inter_geo_loss_layer40=0.033, inter_geo_loss_layer44=0.033, inter_geo_loss_mean=0.033, geo_loss_all=0.193, loss=0.425, accuracy=0.907, backward_time=0.829, grad_norm=89.949, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.790 +[gpue05] 2025-06-01 03:35:26,059 (trainer:816) INFO: 10epoch:train:201-300batch: iter_time=1.269e-04, forward_time=0.366, class_loss=2.086, geo_loss_downstream=0.299, inter_geo_loss_layer32=0.033, inter_geo_loss_layer36=0.034, inter_geo_loss_layer40=0.033, inter_geo_loss_layer44=0.033, inter_geo_loss_mean=0.033, geo_loss_all=0.193, loss=0.427, accuracy=0.918, backward_time=0.996, grad_norm=84.773, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.507 +[gpue05] 2025-06-01 03:37:31,179 (trainer:816) INFO: 10epoch:train:301-400batch: iter_time=1.338e-04, forward_time=0.323, class_loss=1.824, geo_loss_downstream=0.300, inter_geo_loss_layer32=0.033, inter_geo_loss_layer36=0.034, inter_geo_loss_layer40=0.034, inter_geo_loss_layer44=0.034, inter_geo_loss_mean=0.034, geo_loss_all=0.193, loss=0.374, accuracy=0.933, backward_time=0.913, grad_norm=97.199, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.004 +[gpue05] 2025-06-01 03:39:25,950 (trainer:816) INFO: 10epoch:train:401-500batch: iter_time=1.335e-04, forward_time=0.290, class_loss=2.207, geo_loss_downstream=0.299, inter_geo_loss_layer32=0.033, inter_geo_loss_layer36=0.034, inter_geo_loss_layer40=0.034, inter_geo_loss_layer44=0.034, inter_geo_loss_mean=0.034, geo_loss_all=0.193, loss=0.451, accuracy=0.910, backward_time=0.840, grad_norm=107.427, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.013, optim0_lr0=1.000e-05, train_time=4.590 +[gpue05] 2025-06-01 03:41:22,001 (trainer:816) INFO: 10epoch:train:501-600batch: iter_time=1.295e-04, forward_time=0.264, class_loss=2.427, geo_loss_downstream=0.296, inter_geo_loss_layer32=0.032, inter_geo_loss_layer36=0.034, inter_geo_loss_layer40=0.033, inter_geo_loss_layer44=0.033, inter_geo_loss_mean=0.033, geo_loss_all=0.191, loss=0.495, accuracy=0.902, backward_time=0.881, grad_norm=85.193, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.641 +[gpue05] 2025-06-01 03:43:32,695 (trainer:816) INFO: 10epoch:train:601-700batch: iter_time=1.328e-04, forward_time=0.296, class_loss=2.268, geo_loss_downstream=0.296, inter_geo_loss_layer32=0.033, inter_geo_loss_layer36=0.035, inter_geo_loss_layer40=0.034, inter_geo_loss_layer44=0.033, inter_geo_loss_mean=0.034, geo_loss_all=0.191, loss=0.463, accuracy=0.902, backward_time=0.995, grad_norm=101.342, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.227 +[gpue05] 2025-06-01 03:45:36,259 (trainer:816) INFO: 10epoch:train:701-800batch: iter_time=1.414e-04, forward_time=0.273, class_loss=2.169, geo_loss_downstream=0.297, inter_geo_loss_layer32=0.033, inter_geo_loss_layer36=0.035, inter_geo_loss_layer40=0.035, inter_geo_loss_layer44=0.034, inter_geo_loss_mean=0.034, geo_loss_all=0.192, loss=0.443, accuracy=0.905, backward_time=0.946, grad_norm=74.852, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.942 +[gpue05] 2025-06-01 03:47:40,343 (trainer:816) INFO: 10epoch:train:801-900batch: iter_time=1.385e-04, forward_time=0.267, class_loss=2.210, geo_loss_downstream=0.295, inter_geo_loss_layer32=0.031, inter_geo_loss_layer36=0.033, inter_geo_loss_layer40=0.033, inter_geo_loss_layer44=0.032, inter_geo_loss_mean=0.032, geo_loss_all=0.190, loss=0.452, accuracy=0.908, backward_time=0.958, grad_norm=80.657, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.963 +[gpue05] 2025-06-01 03:49:31,022 (trainer:816) INFO: 10epoch:train:901-1000batch: iter_time=1.373e-04, forward_time=0.245, class_loss=2.201, geo_loss_downstream=0.295, inter_geo_loss_layer32=0.033, inter_geo_loss_layer36=0.035, inter_geo_loss_layer40=0.033, inter_geo_loss_layer44=0.034, inter_geo_loss_mean=0.034, geo_loss_all=0.191, loss=0.450, accuracy=0.912, backward_time=0.845, grad_norm=70.034, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.426 +[gpue05] 2025-06-01 03:51:13,079 (trainer:816) INFO: 10epoch:train:1001-1100batch: iter_time=1.697e-04, forward_time=0.226, class_loss=1.760, geo_loss_downstream=0.292, inter_geo_loss_layer32=0.032, inter_geo_loss_layer36=0.033, inter_geo_loss_layer40=0.033, inter_geo_loss_layer44=0.033, inter_geo_loss_mean=0.033, geo_loss_all=0.188, loss=0.361, accuracy=0.925, backward_time=0.777, grad_norm=80.366, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.081 +[gpue05] 2025-06-01 03:53:23,419 (trainer:816) INFO: 10epoch:train:1101-1200batch: iter_time=1.485e-04, forward_time=0.268, class_loss=2.045, geo_loss_downstream=0.295, inter_geo_loss_layer32=0.031, inter_geo_loss_layer36=0.033, inter_geo_loss_layer40=0.034, inter_geo_loss_layer44=0.034, inter_geo_loss_mean=0.033, geo_loss_all=0.190, loss=0.419, accuracy=0.920, backward_time=1.020, grad_norm=96.481, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.213 +[gpue05] 2025-06-01 03:55:30,422 (trainer:816) INFO: 10epoch:train:1201-1300batch: iter_time=1.513e-04, forward_time=0.242, class_loss=1.911, geo_loss_downstream=0.293, inter_geo_loss_layer32=0.030, inter_geo_loss_layer36=0.031, inter_geo_loss_layer40=0.030, inter_geo_loss_layer44=0.031, inter_geo_loss_mean=0.030, geo_loss_all=0.188, loss=0.392, accuracy=0.923, backward_time=1.013, grad_norm=90.164, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.079 +[gpue05] 2025-06-01 03:57:27,892 (trainer:816) INFO: 10epoch:train:1301-1400batch: iter_time=1.498e-04, forward_time=0.245, class_loss=1.899, geo_loss_downstream=0.293, inter_geo_loss_layer32=0.031, inter_geo_loss_layer36=0.031, inter_geo_loss_layer40=0.031, inter_geo_loss_layer44=0.031, inter_geo_loss_mean=0.031, geo_loss_all=0.188, loss=0.389, accuracy=0.927, backward_time=0.913, grad_norm=80.127, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.698 +[gpue05] 2025-06-01 03:59:29,620 (trainer:816) INFO: 10epoch:train:1401-1500batch: iter_time=1.520e-04, forward_time=0.261, class_loss=2.349, geo_loss_downstream=0.290, inter_geo_loss_layer32=0.032, inter_geo_loss_layer36=0.033, inter_geo_loss_layer40=0.032, inter_geo_loss_layer44=0.032, inter_geo_loss_mean=0.032, geo_loss_all=0.187, loss=0.479, accuracy=0.905, backward_time=0.940, grad_norm=87.815, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.868 +[gpue05] 2025-06-01 04:01:25,575 (trainer:816) INFO: 10epoch:train:1501-1600batch: iter_time=1.532e-04, forward_time=0.258, class_loss=2.638, geo_loss_downstream=0.290, inter_geo_loss_layer32=0.032, inter_geo_loss_layer36=0.033, inter_geo_loss_layer40=0.033, inter_geo_loss_layer44=0.034, inter_geo_loss_mean=0.033, geo_loss_all=0.187, loss=0.537, accuracy=0.896, backward_time=0.886, grad_norm=126.124, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.637 +[gpue05] 2025-06-01 04:03:18,378 (trainer:816) INFO: 10epoch:train:1601-1700batch: iter_time=1.337e-04, forward_time=0.253, class_loss=2.017, geo_loss_downstream=0.290, inter_geo_loss_layer32=0.030, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.031, inter_geo_loss_layer44=0.031, inter_geo_loss_mean=0.031, geo_loss_all=0.186, loss=0.413, accuracy=0.920, backward_time=0.858, grad_norm=85.484, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.511 +[gpue05] 2025-06-01 04:05:20,619 (trainer:816) INFO: 10epoch:train:1701-1800batch: iter_time=1.465e-04, forward_time=0.241, class_loss=1.953, geo_loss_downstream=0.290, inter_geo_loss_layer32=0.030, inter_geo_loss_layer36=0.032, inter_geo_loss_layer40=0.032, inter_geo_loss_layer44=0.032, inter_geo_loss_mean=0.031, geo_loss_all=0.187, loss=0.400, accuracy=0.927, backward_time=0.965, grad_norm=73.178, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.889 +[gpue05] 2025-06-01 04:07:12,084 (trainer:816) INFO: 10epoch:train:1801-1900batch: iter_time=1.459e-04, forward_time=0.239, class_loss=1.890, geo_loss_downstream=0.285, inter_geo_loss_layer32=0.029, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.029, inter_geo_loss_layer44=0.030, inter_geo_loss_mean=0.029, geo_loss_all=0.183, loss=0.387, accuracy=0.923, backward_time=0.859, grad_norm=73.514, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.458 +[gpue05] 2025-06-01 04:09:13,984 (trainer:816) INFO: 10epoch:train:1901-2000batch: iter_time=1.365e-04, forward_time=0.257, class_loss=2.011, geo_loss_downstream=0.287, inter_geo_loss_layer32=0.031, inter_geo_loss_layer36=0.032, inter_geo_loss_layer40=0.032, inter_geo_loss_layer44=0.031, inter_geo_loss_mean=0.032, geo_loss_all=0.185, loss=0.411, accuracy=0.923, backward_time=0.946, grad_norm=70.561, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.875 +[gpue05] 2025-06-01 04:32:49,356 (trainer:401) INFO: 10epoch results: [train] iter_time=4.003e-04, forward_time=0.278, class_loss=2.136, geo_loss_downstream=0.294, inter_geo_loss_layer32=0.032, inter_geo_loss_layer36=0.033, inter_geo_loss_layer40=0.033, inter_geo_loss_layer44=0.033, inter_geo_loss_mean=0.033, geo_loss_all=0.189, loss=0.437, accuracy=0.913, backward_time=0.915, grad_norm=87.512, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.838, time=42 minutes and 41 seconds, total_count=20000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=3.565, geo_loss_downstream=0.342, inter_geo_loss_layer32=0.045, inter_geo_loss_layer36=0.054, inter_geo_loss_layer40=0.056, inter_geo_loss_layer44=0.057, inter_geo_loss_mean=0.053, geo_loss_all=0.226, loss=2.897, accuracy=0.849, time=23 minutes and 35.09 seconds, total_count=47220, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 04:33:05,308 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 04:33:05,323 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/9epoch.pth +[gpue05] 2025-06-01 04:33:05,323 (trainer:335) INFO: 11/50epoch started. Estimated time to finish: 1 day, 19 hours and 52 minutes +[gpue05] 2025-06-01 04:37:59,302 (trainer:816) INFO: 11epoch:train:1-100batch: iter_time=0.004, forward_time=0.403, class_loss=2.164, geo_loss_downstream=0.286, inter_geo_loss_layer32=0.030, inter_geo_loss_layer36=0.031, inter_geo_loss_layer40=0.031, inter_geo_loss_layer44=0.031, inter_geo_loss_mean=0.031, geo_loss_all=0.184, loss=0.442, accuracy=0.928, backward_time=1.048, grad_norm=84.219, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.892 +[gpue05] 2025-06-01 04:40:07,032 (trainer:816) INFO: 11epoch:train:101-200batch: iter_time=1.275e-04, forward_time=0.367, class_loss=1.906, geo_loss_downstream=0.286, inter_geo_loss_layer32=0.029, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.029, inter_geo_loss_layer44=0.029, inter_geo_loss_mean=0.029, geo_loss_all=0.184, loss=0.390, accuracy=0.922, backward_time=0.893, grad_norm=79.528, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.108 +[gpue05] 2025-06-01 04:42:11,952 (trainer:816) INFO: 11epoch:train:201-300batch: iter_time=1.213e-04, forward_time=0.339, class_loss=2.410, geo_loss_downstream=0.285, inter_geo_loss_layer32=0.030, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.030, inter_geo_loss_layer44=0.030, inter_geo_loss_mean=0.030, geo_loss_all=0.183, loss=0.491, accuracy=0.898, backward_time=0.895, grad_norm=105.869, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=4.996 +[gpue05] 2025-06-01 04:44:11,679 (trainer:816) INFO: 11epoch:train:301-400batch: iter_time=1.157e-04, forward_time=0.315, class_loss=2.181, geo_loss_downstream=0.284, inter_geo_loss_layer32=0.029, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.028, inter_geo_loss_layer44=0.028, inter_geo_loss_mean=0.029, geo_loss_all=0.182, loss=0.445, accuracy=0.910, backward_time=0.867, grad_norm=88.829, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.788 +[gpue05] 2025-06-01 04:46:14,816 (trainer:816) INFO: 11epoch:train:401-500batch: iter_time=1.066e-04, forward_time=0.319, class_loss=1.990, geo_loss_downstream=0.281, inter_geo_loss_layer32=0.028, inter_geo_loss_layer36=0.029, inter_geo_loss_layer40=0.028, inter_geo_loss_layer44=0.028, inter_geo_loss_mean=0.028, geo_loss_all=0.180, loss=0.407, accuracy=0.920, backward_time=0.896, grad_norm=86.842, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=4.925 +[gpue05] 2025-06-01 04:48:00,489 (trainer:816) INFO: 11epoch:train:501-600batch: iter_time=1.222e-04, forward_time=0.257, class_loss=2.162, geo_loss_downstream=0.280, inter_geo_loss_layer32=0.028, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.028, inter_geo_loss_layer44=0.029, inter_geo_loss_mean=0.029, geo_loss_all=0.180, loss=0.441, accuracy=0.908, backward_time=0.783, grad_norm=94.488, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.226 +[gpue05] 2025-06-01 04:50:07,275 (trainer:816) INFO: 11epoch:train:601-700batch: iter_time=1.126e-04, forward_time=0.295, class_loss=2.557, geo_loss_downstream=0.281, inter_geo_loss_layer32=0.028, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.029, inter_geo_loss_layer44=0.030, inter_geo_loss_mean=0.029, geo_loss_all=0.181, loss=0.520, accuracy=0.893, backward_time=0.956, grad_norm=91.163, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.071 +[gpue05] 2025-06-01 04:51:58,521 (trainer:816) INFO: 11epoch:train:701-800batch: iter_time=1.064e-04, forward_time=0.262, class_loss=2.844, geo_loss_downstream=0.282, inter_geo_loss_layer32=0.029, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.030, inter_geo_loss_layer44=0.031, inter_geo_loss_mean=0.030, geo_loss_all=0.181, loss=0.578, accuracy=0.883, backward_time=0.834, grad_norm=109.370, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.449 +[gpue05] 2025-06-01 04:53:59,796 (trainer:816) INFO: 11epoch:train:801-900batch: iter_time=1.190e-04, forward_time=0.258, class_loss=2.397, geo_loss_downstream=0.281, inter_geo_loss_layer32=0.028, inter_geo_loss_layer36=0.029, inter_geo_loss_layer40=0.028, inter_geo_loss_layer44=0.029, inter_geo_loss_mean=0.029, geo_loss_all=0.180, loss=0.488, accuracy=0.905, backward_time=0.940, grad_norm=99.868, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.850 +[gpue05] 2025-06-01 04:56:02,084 (trainer:816) INFO: 11epoch:train:901-1000batch: iter_time=1.100e-04, forward_time=0.254, class_loss=2.541, geo_loss_downstream=0.279, inter_geo_loss_layer32=0.029, inter_geo_loss_layer36=0.029, inter_geo_loss_layer40=0.029, inter_geo_loss_layer44=0.030, inter_geo_loss_mean=0.030, geo_loss_all=0.179, loss=0.517, accuracy=0.895, backward_time=0.954, grad_norm=73.975, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.891 +[gpue05] 2025-06-01 04:58:13,284 (trainer:816) INFO: 11epoch:train:1001-1100batch: iter_time=1.169e-04, forward_time=0.263, class_loss=2.134, geo_loss_downstream=0.279, inter_geo_loss_layer32=0.028, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.029, inter_geo_loss_layer44=0.031, inter_geo_loss_mean=0.029, geo_loss_all=0.179, loss=0.436, accuracy=0.918, backward_time=1.034, grad_norm=86.462, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.247 +[gpue05] 2025-06-01 05:00:16,565 (trainer:816) INFO: 11epoch:train:1101-1200batch: iter_time=1.186e-04, forward_time=0.258, class_loss=2.410, geo_loss_downstream=0.278, inter_geo_loss_layer32=0.029, inter_geo_loss_layer36=0.031, inter_geo_loss_layer40=0.031, inter_geo_loss_layer44=0.031, inter_geo_loss_mean=0.030, geo_loss_all=0.179, loss=0.491, accuracy=0.907, backward_time=0.960, grad_norm=83.519, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=4.930 +[gpue05] 2025-06-01 05:02:17,246 (trainer:816) INFO: 11epoch:train:1201-1300batch: iter_time=1.288e-04, forward_time=0.255, class_loss=2.247, geo_loss_downstream=0.277, inter_geo_loss_layer32=0.028, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.031, inter_geo_loss_layer44=0.032, inter_geo_loss_mean=0.030, geo_loss_all=0.178, loss=0.458, accuracy=0.915, backward_time=0.936, grad_norm=87.912, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=4.826 +[gpue05] 2025-06-01 05:04:20,110 (trainer:816) INFO: 11epoch:train:1301-1400batch: iter_time=1.160e-04, forward_time=0.259, class_loss=2.496, geo_loss_downstream=0.278, inter_geo_loss_layer32=0.028, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.030, inter_geo_loss_layer44=0.031, inter_geo_loss_mean=0.030, geo_loss_all=0.179, loss=0.508, accuracy=0.893, backward_time=0.953, grad_norm=91.671, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.914 +[gpue05] 2025-06-01 05:06:33,249 (trainer:816) INFO: 11epoch:train:1401-1500batch: iter_time=1.115e-04, forward_time=0.275, class_loss=2.002, geo_loss_downstream=0.277, inter_geo_loss_layer32=0.028, inter_geo_loss_layer36=0.029, inter_geo_loss_layer40=0.030, inter_geo_loss_layer44=0.032, inter_geo_loss_mean=0.030, geo_loss_all=0.178, loss=0.409, accuracy=0.921, backward_time=1.042, grad_norm=95.046, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.325 +[gpue05] 2025-06-01 05:08:26,037 (trainer:816) INFO: 11epoch:train:1501-1600batch: iter_time=1.196e-04, forward_time=0.228, class_loss=1.952, geo_loss_downstream=0.275, inter_geo_loss_layer32=0.027, inter_geo_loss_layer36=0.028, inter_geo_loss_layer40=0.029, inter_geo_loss_layer44=0.030, inter_geo_loss_mean=0.029, geo_loss_all=0.177, loss=0.399, accuracy=0.923, backward_time=0.882, grad_norm=75.980, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.511 +[gpue05] 2025-06-01 05:10:17,277 (trainer:816) INFO: 11epoch:train:1601-1700batch: iter_time=1.025e-04, forward_time=0.241, class_loss=2.376, geo_loss_downstream=0.275, inter_geo_loss_layer32=0.027, inter_geo_loss_layer36=0.029, inter_geo_loss_layer40=0.029, inter_geo_loss_layer44=0.030, inter_geo_loss_mean=0.029, geo_loss_all=0.177, loss=0.484, accuracy=0.893, backward_time=0.856, grad_norm=116.159, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.449 +[gpue05] 2025-06-01 05:12:12,694 (trainer:816) INFO: 11epoch:train:1701-1800batch: iter_time=1.206e-04, forward_time=0.249, class_loss=1.940, geo_loss_downstream=0.274, inter_geo_loss_layer32=0.027, inter_geo_loss_layer36=0.028, inter_geo_loss_layer40=0.028, inter_geo_loss_layer44=0.030, inter_geo_loss_mean=0.028, geo_loss_all=0.176, loss=0.397, accuracy=0.923, backward_time=0.890, grad_norm=66.480, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.616 +[gpue05] 2025-06-01 05:14:29,528 (trainer:816) INFO: 11epoch:train:1801-1900batch: iter_time=1.437e-04, forward_time=0.323, class_loss=2.045, geo_loss_downstream=0.273, inter_geo_loss_layer32=0.027, inter_geo_loss_layer36=0.028, inter_geo_loss_layer40=0.027, inter_geo_loss_layer44=0.028, inter_geo_loss_mean=0.028, geo_loss_all=0.175, loss=0.418, accuracy=0.922, backward_time=1.028, grad_norm=86.312, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.472 +[gpue05] 2025-06-01 05:16:18,448 (trainer:816) INFO: 11epoch:train:1901-2000batch: iter_time=1.164e-04, forward_time=0.241, class_loss=2.026, geo_loss_downstream=0.272, inter_geo_loss_layer32=0.027, inter_geo_loss_layer36=0.027, inter_geo_loss_layer40=0.027, inter_geo_loss_layer44=0.028, inter_geo_loss_mean=0.027, geo_loss_all=0.174, loss=0.414, accuracy=0.918, backward_time=0.826, grad_norm=69.920, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.356 +[gpue05] 2025-06-01 05:39:49,243 (trainer:401) INFO: 11epoch results: [train] iter_time=3.349e-04, forward_time=0.283, class_loss=2.239, geo_loss_downstream=0.279, inter_geo_loss_layer32=0.028, inter_geo_loss_layer36=0.029, inter_geo_loss_layer40=0.029, inter_geo_loss_layer44=0.030, inter_geo_loss_mean=0.029, geo_loss_all=0.179, loss=0.457, accuracy=0.910, backward_time=0.924, grad_norm=88.681, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=4.892, time=43 minutes and 13.42 seconds, total_count=22000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=3.712, geo_loss_downstream=0.336, inter_geo_loss_layer32=0.041, inter_geo_loss_layer36=0.047, inter_geo_loss_layer40=0.049, inter_geo_loss_layer44=0.052, inter_geo_loss_mean=0.047, geo_loss_all=0.220, loss=3.014, accuracy=0.847, time=23 minutes and 30.48 seconds, total_count=51942, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 05:40:03,539 (trainer:467) INFO: There are no improvements in this epoch +[gpue05] 2025-06-01 05:40:03,553 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/8epoch.pth +[gpue05] 2025-06-01 05:40:03,554 (trainer:335) INFO: 12/50epoch started. Estimated time to finish: 1 day, 18 hours and 50 minutes +[gpue05] 2025-06-01 05:44:38,797 (trainer:816) INFO: 12epoch:train:1-100batch: iter_time=0.003, forward_time=0.400, class_loss=1.989, geo_loss_downstream=0.270, inter_geo_loss_layer32=0.027, inter_geo_loss_layer36=0.027, inter_geo_loss_layer40=0.028, inter_geo_loss_layer44=0.029, inter_geo_loss_mean=0.028, geo_loss_all=0.173, loss=0.407, accuracy=0.928, backward_time=0.900, grad_norm=89.930, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.299 +[gpue05] 2025-06-01 05:46:50,480 (trainer:816) INFO: 12epoch:train:101-200batch: iter_time=1.254e-04, forward_time=0.368, class_loss=2.215, geo_loss_downstream=0.271, inter_geo_loss_layer32=0.026, inter_geo_loss_layer36=0.029, inter_geo_loss_layer40=0.029, inter_geo_loss_layer44=0.031, inter_geo_loss_mean=0.029, geo_loss_all=0.174, loss=0.452, accuracy=0.910, backward_time=0.934, grad_norm=83.572, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.267 +[gpue05] 2025-06-01 05:48:53,157 (trainer:816) INFO: 12epoch:train:201-300batch: iter_time=1.235e-04, forward_time=0.337, class_loss=1.761, geo_loss_downstream=0.268, inter_geo_loss_layer32=0.026, inter_geo_loss_layer36=0.027, inter_geo_loss_layer40=0.028, inter_geo_loss_layer44=0.029, inter_geo_loss_mean=0.027, geo_loss_all=0.172, loss=0.361, accuracy=0.932, backward_time=0.873, grad_norm=73.932, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.906 +[gpue05] 2025-06-01 05:50:57,412 (trainer:816) INFO: 12epoch:train:301-400batch: iter_time=1.284e-04, forward_time=0.319, class_loss=1.660, geo_loss_downstream=0.270, inter_geo_loss_layer32=0.026, inter_geo_loss_layer36=0.027, inter_geo_loss_layer40=0.028, inter_geo_loss_layer44=0.028, inter_geo_loss_mean=0.027, geo_loss_all=0.173, loss=0.341, accuracy=0.927, backward_time=0.910, grad_norm=88.056, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.969 +[gpue05] 2025-06-01 05:53:06,395 (trainer:816) INFO: 12epoch:train:401-500batch: iter_time=1.371e-04, forward_time=0.311, class_loss=1.789, geo_loss_downstream=0.268, inter_geo_loss_layer32=0.026, inter_geo_loss_layer36=0.027, inter_geo_loss_layer40=0.028, inter_geo_loss_layer44=0.029, inter_geo_loss_mean=0.027, geo_loss_all=0.172, loss=0.366, accuracy=0.928, backward_time=0.964, grad_norm=73.917, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.159 +[gpue05] 2025-06-01 05:55:12,936 (trainer:816) INFO: 12epoch:train:501-600batch: iter_time=1.237e-04, forward_time=0.288, class_loss=1.543, geo_loss_downstream=0.269, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.026, inter_geo_loss_layer40=0.026, inter_geo_loss_layer44=0.027, inter_geo_loss_mean=0.026, geo_loss_all=0.172, loss=0.317, accuracy=0.940, backward_time=0.963, grad_norm=59.525, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.061 +[gpue05] 2025-06-01 05:57:19,619 (trainer:816) INFO: 12epoch:train:601-700batch: iter_time=1.179e-04, forward_time=0.300, class_loss=1.912, geo_loss_downstream=0.266, inter_geo_loss_layer32=0.026, inter_geo_loss_layer36=0.027, inter_geo_loss_layer40=0.027, inter_geo_loss_layer44=0.028, inter_geo_loss_mean=0.027, geo_loss_all=0.171, loss=0.391, accuracy=0.923, backward_time=0.951, grad_norm=72.603, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.067 +[gpue05] 2025-06-01 05:59:25,103 (trainer:816) INFO: 12epoch:train:701-800batch: iter_time=1.275e-04, forward_time=0.261, class_loss=2.128, geo_loss_downstream=0.266, inter_geo_loss_layer32=0.025, inter_geo_loss_layer36=0.026, inter_geo_loss_layer40=0.027, inter_geo_loss_layer44=0.027, inter_geo_loss_mean=0.026, geo_loss_all=0.170, loss=0.434, accuracy=0.915, backward_time=0.979, grad_norm=98.991, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.012, optim0_lr0=1.000e-05, train_time=5.019 +[gpue05] 2025-06-01 06:01:23,452 (trainer:816) INFO: 12epoch:train:801-900batch: iter_time=1.220e-04, forward_time=0.257, class_loss=2.218, geo_loss_downstream=0.264, inter_geo_loss_layer32=0.026, inter_geo_loss_layer36=0.027, inter_geo_loss_layer40=0.026, inter_geo_loss_layer44=0.026, inter_geo_loss_mean=0.026, geo_loss_all=0.169, loss=0.452, accuracy=0.908, backward_time=0.912, grad_norm=85.268, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.733 +[gpue05] 2025-06-01 06:03:24,347 (trainer:816) INFO: 12epoch:train:901-1000batch: iter_time=1.185e-04, forward_time=0.261, class_loss=2.510, geo_loss_downstream=0.267, inter_geo_loss_layer32=0.025, inter_geo_loss_layer36=0.027, inter_geo_loss_layer40=0.026, inter_geo_loss_layer44=0.027, inter_geo_loss_mean=0.026, geo_loss_all=0.171, loss=0.511, accuracy=0.897, backward_time=0.933, grad_norm=111.232, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.835 +[gpue05] 2025-06-01 06:05:18,809 (trainer:816) INFO: 12epoch:train:1001-1100batch: iter_time=1.211e-04, forward_time=0.243, class_loss=2.007, geo_loss_downstream=0.265, inter_geo_loss_layer32=0.026, inter_geo_loss_layer36=0.027, inter_geo_loss_layer40=0.027, inter_geo_loss_layer44=0.027, inter_geo_loss_mean=0.027, geo_loss_all=0.170, loss=0.410, accuracy=0.925, backward_time=0.886, grad_norm=73.344, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.578 +[gpue05] 2025-06-01 06:07:14,986 (trainer:816) INFO: 12epoch:train:1101-1200batch: iter_time=1.154e-04, forward_time=0.255, class_loss=1.731, geo_loss_downstream=0.264, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.025, inter_geo_loss_layer40=0.025, inter_geo_loss_layer44=0.026, inter_geo_loss_mean=0.025, geo_loss_all=0.168, loss=0.355, accuracy=0.938, backward_time=0.891, grad_norm=80.978, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.646 +[gpue05] 2025-06-01 06:09:07,236 (trainer:816) INFO: 12epoch:train:1201-1300batch: iter_time=1.151e-04, forward_time=0.244, class_loss=1.785, geo_loss_downstream=0.263, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.026, inter_geo_loss_layer40=0.025, inter_geo_loss_layer44=0.026, inter_geo_loss_mean=0.025, geo_loss_all=0.168, loss=0.365, accuracy=0.938, backward_time=0.863, grad_norm=98.691, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.489 +[gpue05] 2025-06-01 06:11:07,598 (trainer:816) INFO: 12epoch:train:1301-1400batch: iter_time=1.149e-04, forward_time=0.258, class_loss=2.346, geo_loss_downstream=0.261, inter_geo_loss_layer32=0.025, inter_geo_loss_layer36=0.026, inter_geo_loss_layer40=0.026, inter_geo_loss_layer44=0.027, inter_geo_loss_mean=0.026, geo_loss_all=0.167, loss=0.478, accuracy=0.903, backward_time=0.929, grad_norm=90.990, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.814 +[gpue05] 2025-06-01 06:13:00,459 (trainer:816) INFO: 12epoch:train:1401-1500batch: iter_time=1.188e-04, forward_time=0.242, class_loss=2.135, geo_loss_downstream=0.260, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.025, inter_geo_loss_layer40=0.026, inter_geo_loss_layer44=0.026, inter_geo_loss_mean=0.025, geo_loss_all=0.166, loss=0.435, accuracy=0.920, backward_time=0.870, grad_norm=93.442, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.514 +[gpue05] 2025-06-01 06:14:57,096 (trainer:816) INFO: 12epoch:train:1501-1600batch: iter_time=1.173e-04, forward_time=0.244, class_loss=2.108, geo_loss_downstream=0.262, inter_geo_loss_layer32=0.025, inter_geo_loss_layer36=0.025, inter_geo_loss_layer40=0.026, inter_geo_loss_layer44=0.026, inter_geo_loss_mean=0.025, geo_loss_all=0.167, loss=0.430, accuracy=0.915, backward_time=0.906, grad_norm=76.881, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.665 +[gpue05] 2025-06-01 06:16:58,796 (trainer:816) INFO: 12epoch:train:1601-1700batch: iter_time=1.171e-04, forward_time=0.259, class_loss=2.278, geo_loss_downstream=0.261, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.026, inter_geo_loss_layer40=0.025, inter_geo_loss_layer44=0.027, inter_geo_loss_mean=0.026, geo_loss_all=0.167, loss=0.464, accuracy=0.901, backward_time=0.942, grad_norm=113.326, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.867 +[gpue05] 2025-06-01 06:19:06,740 (trainer:816) INFO: 12epoch:train:1701-1800batch: iter_time=1.204e-04, forward_time=0.274, class_loss=1.710, geo_loss_downstream=0.259, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.025, inter_geo_loss_layer40=0.024, inter_geo_loss_layer44=0.026, inter_geo_loss_mean=0.025, geo_loss_all=0.166, loss=0.350, accuracy=0.930, backward_time=0.990, grad_norm=78.573, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.117 +[gpue05] 2025-06-01 06:21:04,388 (trainer:816) INFO: 12epoch:train:1801-1900batch: iter_time=1.168e-04, forward_time=0.242, class_loss=1.949, geo_loss_downstream=0.259, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.024, inter_geo_loss_layer40=0.025, inter_geo_loss_layer44=0.025, inter_geo_loss_mean=0.025, geo_loss_all=0.165, loss=0.398, accuracy=0.920, backward_time=0.919, grad_norm=69.398, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.705 +[gpue05] 2025-06-01 06:23:04,952 (trainer:816) INFO: 12epoch:train:1901-2000batch: iter_time=1.109e-04, forward_time=0.250, class_loss=1.874, geo_loss_downstream=0.259, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.025, inter_geo_loss_layer40=0.025, inter_geo_loss_layer44=0.025, inter_geo_loss_mean=0.025, geo_loss_all=0.165, loss=0.383, accuracy=0.928, backward_time=0.940, grad_norm=90.828, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.822 +[gpue05] 2025-06-01 06:46:29,719 (trainer:401) INFO: 12epoch results: [train] iter_time=2.629e-04, forward_time=0.281, class_loss=1.982, geo_loss_downstream=0.265, inter_geo_loss_layer32=0.025, inter_geo_loss_layer36=0.026, inter_geo_loss_layer40=0.026, inter_geo_loss_layer44=0.027, inter_geo_loss_mean=0.026, geo_loss_all=0.169, loss=0.405, accuracy=0.921, backward_time=0.923, grad_norm=85.174, clip=0.000e+00, loss_scale=2.621e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.877, time=43 minutes and 1.76 seconds, total_count=24000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=3.648, geo_loss_downstream=0.301, inter_geo_loss_layer32=0.034, inter_geo_loss_layer36=0.039, inter_geo_loss_layer40=0.042, inter_geo_loss_layer44=0.047, inter_geo_loss_mean=0.041, geo_loss_all=0.197, loss=2.957, accuracy=0.850, time=23 minutes and 24.38 seconds, total_count=56664, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 06:46:44,082 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 06:46:44,112 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/11epoch.pth +[gpue05] 2025-06-01 06:46:44,113 (trainer:335) INFO: 13/50epoch started. Estimated time to finish: 1 day, 17 hours and 47 minutes +[gpue05] 2025-06-01 06:51:18,214 (trainer:816) INFO: 13epoch:train:1-100batch: iter_time=0.004, forward_time=0.405, class_loss=1.802, geo_loss_downstream=0.260, inter_geo_loss_layer32=0.025, inter_geo_loss_layer36=0.025, inter_geo_loss_layer40=0.026, inter_geo_loss_layer44=0.026, inter_geo_loss_mean=0.025, geo_loss_all=0.166, loss=0.369, accuracy=0.933, backward_time=0.902, grad_norm=70.728, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.011, optim0_lr0=1.000e-05, train_time=5.339 +[gpue05] 2025-06-01 06:53:25,725 (trainer:816) INFO: 13epoch:train:101-200batch: iter_time=1.209e-04, forward_time=0.359, class_loss=2.211, geo_loss_downstream=0.257, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.025, inter_geo_loss_layer40=0.024, inter_geo_loss_layer44=0.025, inter_geo_loss_mean=0.025, geo_loss_all=0.164, loss=0.450, accuracy=0.911, backward_time=0.900, grad_norm=66.522, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.100 +[gpue05] 2025-06-01 06:55:24,123 (trainer:816) INFO: 13epoch:train:201-300batch: iter_time=1.227e-04, forward_time=0.329, class_loss=1.933, geo_loss_downstream=0.260, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.026, inter_geo_loss_layer40=0.026, inter_geo_loss_layer44=0.026, inter_geo_loss_mean=0.026, geo_loss_all=0.166, loss=0.395, accuracy=0.930, backward_time=0.838, grad_norm=92.439, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=1.000e-05, train_time=4.735 +[gpue05] 2025-06-01 06:57:27,642 (trainer:816) INFO: 13epoch:train:301-400batch: iter_time=1.295e-04, forward_time=0.322, class_loss=1.731, geo_loss_downstream=0.255, inter_geo_loss_layer32=0.023, inter_geo_loss_layer36=0.024, inter_geo_loss_layer40=0.024, inter_geo_loss_layer44=0.025, inter_geo_loss_mean=0.024, geo_loss_all=0.163, loss=0.354, accuracy=0.927, backward_time=0.896, grad_norm=111.392, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.940 +[gpue05] 2025-06-01 06:59:34,465 (trainer:816) INFO: 13epoch:train:401-500batch: iter_time=1.323e-04, forward_time=0.308, class_loss=1.576, geo_loss_downstream=0.255, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.025, inter_geo_loss_layer40=0.025, inter_geo_loss_layer44=0.025, inter_geo_loss_mean=0.025, geo_loss_all=0.163, loss=0.323, accuracy=0.940, backward_time=0.944, grad_norm=66.931, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.072 +[gpue05] 2025-06-01 07:01:38,435 (trainer:816) INFO: 13epoch:train:501-600batch: iter_time=1.149e-04, forward_time=0.293, class_loss=2.569, geo_loss_downstream=0.254, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.024, inter_geo_loss_layer40=0.025, inter_geo_loss_layer44=0.025, inter_geo_loss_mean=0.024, geo_loss_all=0.162, loss=0.522, accuracy=0.896, backward_time=0.931, grad_norm=70.137, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.958 +[gpue05] 2025-06-01 07:03:36,980 (trainer:816) INFO: 13epoch:train:601-700batch: iter_time=1.161e-04, forward_time=0.271, class_loss=2.410, geo_loss_downstream=0.253, inter_geo_loss_layer32=0.023, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.024, inter_geo_loss_mean=0.023, geo_loss_all=0.161, loss=0.490, accuracy=0.896, backward_time=0.899, grad_norm=58.666, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.741 +[gpue05] 2025-06-01 07:05:42,394 (trainer:816) INFO: 13epoch:train:701-800batch: iter_time=1.172e-04, forward_time=0.271, class_loss=1.760, geo_loss_downstream=0.253, inter_geo_loss_layer32=0.023, inter_geo_loss_layer36=0.024, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.024, inter_geo_loss_mean=0.024, geo_loss_all=0.161, loss=0.360, accuracy=0.935, backward_time=0.967, grad_norm=79.680, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=5.016 +[gpue05] 2025-06-01 07:07:44,140 (trainer:816) INFO: 13epoch:train:801-900batch: iter_time=1.215e-04, forward_time=0.262, class_loss=1.952, geo_loss_downstream=0.252, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.024, inter_geo_loss_layer40=0.024, inter_geo_loss_layer44=0.024, inter_geo_loss_mean=0.023, geo_loss_all=0.160, loss=0.398, accuracy=0.927, backward_time=0.941, grad_norm=81.095, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.869 +[gpue05] 2025-06-01 07:09:38,037 (trainer:816) INFO: 13epoch:train:901-1000batch: iter_time=1.206e-04, forward_time=0.226, class_loss=2.078, geo_loss_downstream=0.250, inter_geo_loss_layer32=0.023, inter_geo_loss_layer36=0.024, inter_geo_loss_layer40=0.024, inter_geo_loss_layer44=0.024, inter_geo_loss_mean=0.024, geo_loss_all=0.159, loss=0.424, accuracy=0.912, backward_time=0.899, grad_norm=72.235, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=1.000e-05, train_time=4.555 +[gpue05] 2025-06-01 07:11:32,736 (trainer:816) INFO: 13epoch:train:1001-1100batch: iter_time=1.192e-04, forward_time=0.243, class_loss=1.732, geo_loss_downstream=0.250, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.023, inter_geo_loss_mean=0.023, geo_loss_all=0.159, loss=0.354, accuracy=0.935, backward_time=0.889, grad_norm=75.686, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=9.949e-06, train_time=4.587 +[gpue05] 2025-06-01 07:13:37,665 (trainer:816) INFO: 13epoch:train:1101-1200batch: iter_time=1.262e-04, forward_time=0.237, class_loss=1.958, geo_loss_downstream=0.251, inter_geo_loss_layer32=0.023, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.023, inter_geo_loss_mean=0.023, geo_loss_all=0.160, loss=0.400, accuracy=0.920, backward_time=0.998, grad_norm=71.043, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=9.857e-06, train_time=4.996 +[gpue05] 2025-06-01 07:15:30,308 (trainer:816) INFO: 13epoch:train:1201-1300batch: iter_time=1.198e-04, forward_time=0.228, class_loss=1.723, geo_loss_downstream=0.249, inter_geo_loss_layer32=0.023, inter_geo_loss_layer36=0.024, inter_geo_loss_layer40=0.024, inter_geo_loss_layer44=0.023, inter_geo_loss_mean=0.023, geo_loss_all=0.159, loss=0.353, accuracy=0.930, backward_time=0.883, grad_norm=67.336, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=9.767e-06, train_time=4.505 +[gpue05] 2025-06-01 07:17:21,359 (trainer:816) INFO: 13epoch:train:1301-1400batch: iter_time=1.252e-04, forward_time=0.234, class_loss=2.330, geo_loss_downstream=0.248, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.023, inter_geo_loss_mean=0.023, geo_loss_all=0.158, loss=0.474, accuracy=0.907, backward_time=0.861, grad_norm=82.088, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=9.677e-06, train_time=4.441 +[gpue05] 2025-06-01 07:19:10,841 (trainer:816) INFO: 13epoch:train:1401-1500batch: iter_time=1.250e-04, forward_time=0.247, class_loss=1.839, geo_loss_downstream=0.247, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.023, inter_geo_loss_mean=0.023, geo_loss_all=0.157, loss=0.376, accuracy=0.922, backward_time=0.831, grad_norm=72.532, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=9.589e-06, train_time=4.379 +[gpue05] 2025-06-01 07:21:04,673 (trainer:816) INFO: 13epoch:train:1501-1600batch: iter_time=1.153e-04, forward_time=0.249, class_loss=2.180, geo_loss_downstream=0.246, inter_geo_loss_layer32=0.023, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.023, inter_geo_loss_mean=0.023, geo_loss_all=0.157, loss=0.444, accuracy=0.912, backward_time=0.872, grad_norm=71.077, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=9.501e-06, train_time=4.553 +[gpue05] 2025-06-01 07:23:00,431 (trainer:816) INFO: 13epoch:train:1601-1700batch: iter_time=1.082e-04, forward_time=0.241, class_loss=1.774, geo_loss_downstream=0.248, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.023, inter_geo_loss_mean=0.022, geo_loss_all=0.158, loss=0.363, accuracy=0.928, backward_time=0.902, grad_norm=73.814, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=9.414e-06, train_time=4.630 +[gpue05] 2025-06-01 07:24:46,675 (trainer:816) INFO: 13epoch:train:1701-1800batch: iter_time=1.190e-04, forward_time=0.223, class_loss=2.054, geo_loss_downstream=0.246, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.023, inter_geo_loss_mean=0.023, geo_loss_all=0.156, loss=0.419, accuracy=0.922, backward_time=0.823, grad_norm=64.260, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=9.327e-06, train_time=4.249 +[gpue05] 2025-06-01 07:26:27,448 (trainer:816) INFO: 13epoch:train:1801-1900batch: iter_time=1.142e-04, forward_time=0.236, class_loss=1.445, geo_loss_downstream=0.244, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.155, loss=0.297, accuracy=0.947, backward_time=0.756, grad_norm=80.806, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=9.242e-06, train_time=4.030 +[gpue05] 2025-06-01 07:28:34,306 (trainer:816) INFO: 13epoch:train:1901-2000batch: iter_time=1.107e-04, forward_time=0.253, class_loss=1.516, geo_loss_downstream=0.244, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.155, loss=0.311, accuracy=0.937, backward_time=1.001, grad_norm=81.176, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=9.157e-06, train_time=5.074 +[gpue05] 2025-06-01 07:52:06,139 (trainer:401) INFO: 13epoch results: [train] iter_time=3.366e-04, forward_time=0.272, class_loss=1.929, geo_loss_downstream=0.251, inter_geo_loss_layer32=0.023, inter_geo_loss_layer36=0.024, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.024, inter_geo_loss_mean=0.023, geo_loss_all=0.160, loss=0.394, accuracy=0.923, backward_time=0.896, grad_norm=75.482, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=9.774e-06, train_time=4.738, time=41 minutes and 50.51 seconds, total_count=26000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=3.407, geo_loss_downstream=0.275, inter_geo_loss_layer32=0.030, inter_geo_loss_layer36=0.035, inter_geo_loss_layer40=0.034, inter_geo_loss_layer44=0.037, inter_geo_loss_mean=0.034, geo_loss_all=0.178, loss=2.761, accuracy=0.860, time=23 minutes and 31.5 seconds, total_count=61386, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 07:52:20,205 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 07:52:20,341 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/10epoch.pth +[gpue05] 2025-06-01 07:52:20,342 (trainer:335) INFO: 14/50epoch started. Estimated time to finish: 1 day, 16 hours and 40 minutes +[gpue05] 2025-06-01 07:56:49,716 (trainer:816) INFO: 14epoch:train:1-100batch: iter_time=0.003, forward_time=0.393, class_loss=1.914, geo_loss_downstream=0.244, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.155, loss=0.391, accuracy=0.926, backward_time=0.858, grad_norm=94.982, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=9.073e-06, train_time=5.105 +[gpue05] 2025-06-01 07:59:10,934 (trainer:816) INFO: 14epoch:train:101-200batch: iter_time=1.076e-04, forward_time=0.369, class_loss=1.576, geo_loss_downstream=0.242, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.021, geo_loss_all=0.154, loss=0.323, accuracy=0.942, backward_time=1.028, grad_norm=67.841, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=8.990e-06, train_time=5.648 +[gpue05] 2025-06-01 08:01:21,726 (trainer:816) INFO: 14epoch:train:201-300batch: iter_time=1.084e-04, forward_time=0.350, class_loss=2.039, geo_loss_downstream=0.243, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.023, geo_loss_all=0.155, loss=0.416, accuracy=0.910, backward_time=0.942, grad_norm=78.730, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=8.908e-06, train_time=5.231 +[gpue05] 2025-06-01 08:03:22,375 (trainer:816) INFO: 14epoch:train:301-400batch: iter_time=1.244e-04, forward_time=0.307, class_loss=1.713, geo_loss_downstream=0.243, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.023, inter_geo_loss_mean=0.022, geo_loss_all=0.155, loss=0.350, accuracy=0.935, backward_time=0.883, grad_norm=81.702, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=8.826e-06, train_time=4.825 +[gpue05] 2025-06-01 08:05:27,892 (trainer:816) INFO: 14epoch:train:401-500batch: iter_time=1.246e-04, forward_time=0.307, class_loss=1.825, geo_loss_downstream=0.241, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.153, loss=0.373, accuracy=0.925, backward_time=0.933, grad_norm=78.514, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=8.745e-06, train_time=5.020 +[gpue05] 2025-06-01 08:07:36,367 (trainer:816) INFO: 14epoch:train:501-600batch: iter_time=1.222e-04, forward_time=0.279, class_loss=1.523, geo_loss_downstream=0.241, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.021, geo_loss_all=0.153, loss=0.312, accuracy=0.937, backward_time=0.992, grad_norm=65.314, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=8.665e-06, train_time=5.138 +[gpue05] 2025-06-01 08:09:35,407 (trainer:816) INFO: 14epoch:train:601-700batch: iter_time=1.132e-04, forward_time=0.271, class_loss=1.469, geo_loss_downstream=0.238, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.021, geo_loss_all=0.151, loss=0.301, accuracy=0.938, backward_time=0.904, grad_norm=57.642, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=8.585e-06, train_time=4.761 +[gpue05] 2025-06-01 08:11:21,434 (trainer:816) INFO: 14epoch:train:701-800batch: iter_time=1.089e-04, forward_time=0.236, class_loss=1.839, geo_loss_downstream=0.239, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.022, geo_loss_all=0.152, loss=0.375, accuracy=0.930, backward_time=0.807, grad_norm=84.407, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=8.507e-06, train_time=4.240 +[gpue05] 2025-06-01 08:13:18,668 (trainer:816) INFO: 14epoch:train:801-900batch: iter_time=1.247e-04, forward_time=0.262, class_loss=1.954, geo_loss_downstream=0.238, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.021, geo_loss_all=0.152, loss=0.398, accuracy=0.918, backward_time=0.894, grad_norm=79.926, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=8.429e-06, train_time=4.689 +[gpue05] 2025-06-01 08:15:10,693 (trainer:816) INFO: 14epoch:train:901-1000batch: iter_time=1.323e-04, forward_time=0.232, class_loss=1.998, geo_loss_downstream=0.239, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.152, loss=0.407, accuracy=0.923, backward_time=0.872, grad_norm=88.323, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=8.351e-06, train_time=4.480 +[gpue05] 2025-06-01 08:16:49,804 (trainer:816) INFO: 14epoch:train:1001-1100batch: iter_time=1.147e-04, forward_time=0.207, class_loss=1.872, geo_loss_downstream=0.239, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.021, geo_loss_all=0.152, loss=0.382, accuracy=0.928, backward_time=0.767, grad_norm=86.549, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=8.275e-06, train_time=3.964 +[gpue05] 2025-06-01 08:18:38,056 (trainer:816) INFO: 14epoch:train:1101-1200batch: iter_time=1.186e-04, forward_time=0.232, class_loss=1.625, geo_loss_downstream=0.236, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.151, loss=0.333, accuracy=0.935, backward_time=0.834, grad_norm=99.619, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=8.199e-06, train_time=4.329 +[gpue05] 2025-06-01 08:20:32,813 (trainer:816) INFO: 14epoch:train:1201-1300batch: iter_time=1.156e-04, forward_time=0.242, class_loss=2.118, geo_loss_downstream=0.236, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.021, geo_loss_all=0.150, loss=0.431, accuracy=0.910, backward_time=0.890, grad_norm=69.473, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=8.124e-06, train_time=4.589 +[gpue05] 2025-06-01 08:22:33,863 (trainer:816) INFO: 14epoch:train:1301-1400batch: iter_time=1.209e-04, forward_time=0.244, class_loss=1.848, geo_loss_downstream=0.236, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.021, geo_loss_all=0.150, loss=0.377, accuracy=0.928, backward_time=0.951, grad_norm=82.260, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=8.049e-06, train_time=4.841 +[gpue05] 2025-06-01 08:24:31,122 (trainer:816) INFO: 14epoch:train:1401-1500batch: iter_time=1.204e-04, forward_time=0.243, class_loss=1.892, geo_loss_downstream=0.235, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.149, loss=0.386, accuracy=0.925, backward_time=0.915, grad_norm=73.568, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.011, optim0_lr0=7.976e-06, train_time=4.690 +[gpue05] 2025-06-01 08:26:34,745 (trainer:816) INFO: 14epoch:train:1501-1600batch: iter_time=1.158e-04, forward_time=0.246, class_loss=1.581, geo_loss_downstream=0.235, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.021, geo_loss_all=0.149, loss=0.324, accuracy=0.935, backward_time=0.975, grad_norm=86.090, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.902e-06, train_time=4.944 +[gpue05] 2025-06-01 08:28:32,563 (trainer:816) INFO: 14epoch:train:1601-1700batch: iter_time=1.124e-04, forward_time=0.251, class_loss=2.053, geo_loss_downstream=0.234, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.021, geo_loss_all=0.149, loss=0.418, accuracy=0.910, backward_time=0.912, grad_norm=79.038, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.830e-06, train_time=4.712 +[gpue05] 2025-06-01 08:30:36,694 (trainer:816) INFO: 14epoch:train:1701-1800batch: iter_time=1.293e-04, forward_time=0.249, class_loss=1.837, geo_loss_downstream=0.234, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.021, geo_loss_all=0.149, loss=0.375, accuracy=0.922, backward_time=0.977, grad_norm=92.179, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.758e-06, train_time=4.964 +[gpue05] 2025-06-01 08:32:37,100 (trainer:816) INFO: 14epoch:train:1801-1900batch: iter_time=1.163e-04, forward_time=0.261, class_loss=1.777, geo_loss_downstream=0.235, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.021, geo_loss_all=0.149, loss=0.363, accuracy=0.928, backward_time=0.926, grad_norm=72.522, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.687e-06, train_time=4.815 +[gpue05] 2025-06-01 08:34:23,480 (trainer:816) INFO: 14epoch:train:1901-2000batch: iter_time=1.108e-04, forward_time=0.227, class_loss=1.670, geo_loss_downstream=0.234, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.148, loss=0.341, accuracy=0.933, backward_time=0.819, grad_norm=68.198, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.617e-06, train_time=4.254 +[gpue05] 2025-06-01 08:57:57,451 (trainer:401) INFO: 14epoch results: [train] iter_time=2.513e-04, forward_time=0.270, class_loss=1.806, geo_loss_downstream=0.238, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.021, geo_loss_all=0.151, loss=0.369, accuracy=0.927, backward_time=0.904, grad_norm=79.344, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=8.325e-06, train_time=4.762, time=42 minutes and 3.3 seconds, total_count=28000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=3.421, geo_loss_downstream=0.275, inter_geo_loss_layer32=0.029, inter_geo_loss_layer36=0.033, inter_geo_loss_layer40=0.033, inter_geo_loss_layer44=0.035, inter_geo_loss_mean=0.033, geo_loss_all=0.178, loss=2.773, accuracy=0.858, time=23 minutes and 33.69 seconds, total_count=66108, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 08:58:11,988 (trainer:467) INFO: There are no improvements in this epoch +[gpue05] 2025-06-01 08:58:12,078 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/12epoch.pth +[gpue05] 2025-06-01 08:58:12,078 (trainer:335) INFO: 15/50epoch started. Estimated time to finish: 1 day, 15 hours and 34 minutes +[gpue05] 2025-06-01 09:02:38,733 (trainer:816) INFO: 15epoch:train:1-100batch: iter_time=0.001, forward_time=0.405, class_loss=1.871, geo_loss_downstream=0.231, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.019, inter_geo_loss_mean=0.019, geo_loss_all=0.147, loss=0.381, accuracy=0.930, backward_time=0.842, grad_norm=78.901, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.012, optim0_lr0=7.547e-06, train_time=5.082 +[gpue05] 2025-06-01 09:04:52,492 (trainer:816) INFO: 15epoch:train:101-200batch: iter_time=1.098e-04, forward_time=0.369, class_loss=1.602, geo_loss_downstream=0.231, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.147, loss=0.328, accuracy=0.932, backward_time=0.953, grad_norm=57.554, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.478e-06, train_time=5.350 +[gpue05] 2025-06-01 09:06:59,781 (trainer:816) INFO: 15epoch:train:201-300batch: iter_time=1.107e-04, forward_time=0.349, class_loss=2.120, geo_loss_downstream=0.232, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.147, loss=0.431, accuracy=0.913, backward_time=0.908, grad_norm=74.733, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.409e-06, train_time=5.091 +[gpue05] 2025-06-01 09:08:52,276 (trainer:816) INFO: 15epoch:train:301-400batch: iter_time=1.249e-04, forward_time=0.302, class_loss=1.470, geo_loss_downstream=0.229, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.019, geo_loss_all=0.145, loss=0.301, accuracy=0.942, backward_time=0.806, grad_norm=78.721, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.341e-06, train_time=4.499 +[gpue05] 2025-06-01 09:11:01,507 (trainer:816) INFO: 15epoch:train:401-500batch: iter_time=1.207e-04, forward_time=0.312, class_loss=1.538, geo_loss_downstream=0.230, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.146, loss=0.315, accuracy=0.942, backward_time=0.965, grad_norm=61.696, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.274e-06, train_time=5.169 +[gpue05] 2025-06-01 09:13:05,862 (trainer:816) INFO: 15epoch:train:501-600batch: iter_time=1.251e-04, forward_time=0.286, class_loss=1.610, geo_loss_downstream=0.229, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.145, loss=0.329, accuracy=0.935, backward_time=0.941, grad_norm=62.756, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.207e-06, train_time=4.973 +[gpue05] 2025-06-01 09:15:03,941 (trainer:816) INFO: 15epoch:train:601-700batch: iter_time=1.146e-04, forward_time=0.273, class_loss=1.675, geo_loss_downstream=0.228, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.020, geo_loss_all=0.145, loss=0.342, accuracy=0.940, backward_time=0.892, grad_norm=79.131, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.141e-06, train_time=4.722 +[gpue05] 2025-06-01 09:17:02,755 (trainer:816) INFO: 15epoch:train:701-800batch: iter_time=1.175e-04, forward_time=0.259, class_loss=1.268, geo_loss_downstream=0.227, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.019, inter_geo_loss_mean=0.019, geo_loss_all=0.144, loss=0.261, accuracy=0.952, backward_time=0.914, grad_norm=71.073, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.076e-06, train_time=4.752 +[gpue05] 2025-06-01 09:18:57,108 (trainer:816) INFO: 15epoch:train:801-900batch: iter_time=1.178e-04, forward_time=0.255, class_loss=1.515, geo_loss_downstream=0.229, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.021, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.145, loss=0.310, accuracy=0.937, backward_time=0.872, grad_norm=76.551, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=7.011e-06, train_time=4.573 +[gpue05] 2025-06-01 09:20:49,820 (trainer:816) INFO: 15epoch:train:901-1000batch: iter_time=1.185e-04, forward_time=0.245, class_loss=1.373, geo_loss_downstream=0.227, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.019, inter_geo_loss_mean=0.019, geo_loss_all=0.144, loss=0.282, accuracy=0.942, backward_time=0.865, grad_norm=58.530, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.946e-06, train_time=4.508 +[gpue05] 2025-06-01 09:22:44,671 (trainer:816) INFO: 15epoch:train:1001-1100batch: iter_time=1.248e-04, forward_time=0.245, class_loss=1.522, geo_loss_downstream=0.225, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.143, loss=0.312, accuracy=0.938, backward_time=0.886, grad_norm=76.419, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.883e-06, train_time=4.593 +[gpue05] 2025-06-01 09:24:33,135 (trainer:816) INFO: 15epoch:train:1101-1200batch: iter_time=1.235e-04, forward_time=0.240, class_loss=1.364, geo_loss_downstream=0.225, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.143, loss=0.280, accuracy=0.940, backward_time=0.827, grad_norm=55.285, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.820e-06, train_time=4.338 +[gpue05] 2025-06-01 09:26:23,544 (trainer:816) INFO: 15epoch:train:1201-1300batch: iter_time=1.166e-04, forward_time=0.233, class_loss=1.917, geo_loss_downstream=0.224, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.019, inter_geo_loss_mean=0.019, geo_loss_all=0.142, loss=0.390, accuracy=0.923, backward_time=0.854, grad_norm=77.165, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.757e-06, train_time=4.416 +[gpue05] 2025-06-01 09:28:11,372 (trainer:816) INFO: 15epoch:train:1301-1400batch: iter_time=1.254e-04, forward_time=0.230, class_loss=1.550, geo_loss_downstream=0.224, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.142, loss=0.317, accuracy=0.938, backward_time=0.830, grad_norm=80.716, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.695e-06, train_time=4.312 +[gpue05] 2025-06-01 09:30:23,894 (trainer:816) INFO: 15epoch:train:1401-1500batch: iter_time=1.210e-04, forward_time=0.271, class_loss=1.646, geo_loss_downstream=0.224, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.019, inter_geo_loss_mean=0.019, geo_loss_all=0.142, loss=0.336, accuracy=0.935, backward_time=1.039, grad_norm=77.077, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.634e-06, train_time=5.300 +[gpue05] 2025-06-01 09:32:27,100 (trainer:816) INFO: 15epoch:train:1501-1600batch: iter_time=1.299e-04, forward_time=0.255, class_loss=2.126, geo_loss_downstream=0.225, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.019, inter_geo_loss_mean=0.019, geo_loss_all=0.142, loss=0.432, accuracy=0.913, backward_time=0.961, grad_norm=84.567, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.573e-06, train_time=4.927 +[gpue05] 2025-06-01 09:34:20,356 (trainer:816) INFO: 15epoch:train:1601-1700batch: iter_time=1.168e-04, forward_time=0.244, class_loss=1.372, geo_loss_downstream=0.222, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.141, loss=0.281, accuracy=0.946, backward_time=0.873, grad_norm=63.313, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.513e-06, train_time=4.529 +[gpue05] 2025-06-01 09:36:10,485 (trainer:816) INFO: 15epoch:train:1701-1800batch: iter_time=1.396e-04, forward_time=0.242, class_loss=1.757, geo_loss_downstream=0.222, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.019, inter_geo_loss_mean=0.019, geo_loss_all=0.141, loss=0.358, accuracy=0.931, backward_time=0.841, grad_norm=60.569, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=6.453e-06, train_time=4.404 +[gpue05] 2025-06-01 09:38:11,203 (trainer:816) INFO: 15epoch:train:1801-1900batch: iter_time=1.294e-04, forward_time=0.276, class_loss=1.841, geo_loss_downstream=0.221, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.140, loss=0.375, accuracy=0.918, backward_time=0.914, grad_norm=78.743, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.394e-06, train_time=4.828 +[gpue05] 2025-06-01 09:39:55,017 (trainer:816) INFO: 15epoch:train:1901-2000batch: iter_time=9.861e-05, forward_time=0.242, class_loss=1.731, geo_loss_downstream=0.222, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.019, geo_loss_all=0.141, loss=0.353, accuracy=0.927, backward_time=0.779, grad_norm=60.525, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.335e-06, train_time=4.152 +[gpue05] 2025-06-01 10:03:15,241 (trainer:401) INFO: 15epoch results: [train] iter_time=1.803e-04, forward_time=0.277, class_loss=1.643, geo_loss_downstream=0.226, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.019, inter_geo_loss_mean=0.019, geo_loss_all=0.144, loss=0.336, accuracy=0.934, backward_time=0.888, grad_norm=70.701, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.924e-06, train_time=4.726, time=41 minutes and 43.15 seconds, total_count=30000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=3.309, geo_loss_downstream=0.264, inter_geo_loss_layer32=0.027, inter_geo_loss_layer36=0.031, inter_geo_loss_layer40=0.032, inter_geo_loss_layer44=0.033, inter_geo_loss_mean=0.031, geo_loss_all=0.171, loss=2.681, accuracy=0.859, time=23 minutes and 19.91 seconds, total_count=70830, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 10:03:29,716 (trainer:467) INFO: There are no improvements in this epoch +[gpue05] 2025-06-01 10:03:29,727 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/14epoch.pth +[gpue05] 2025-06-01 10:03:29,728 (trainer:335) INFO: 16/50epoch started. Estimated time to finish: 1 day, 14 hours and 26 minutes +[gpue05] 2025-06-01 10:07:49,375 (trainer:816) INFO: 16epoch:train:1-100batch: iter_time=0.003, forward_time=0.384, class_loss=1.573, geo_loss_downstream=0.222, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.020, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.141, loss=0.322, accuracy=0.940, backward_time=0.784, grad_norm=70.369, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.277e-06, train_time=4.776 +[gpue05] 2025-06-01 10:09:57,193 (trainer:816) INFO: 16epoch:train:101-200batch: iter_time=9.379e-05, forward_time=0.356, class_loss=1.680, geo_loss_downstream=0.220, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.019, inter_geo_loss_mean=0.019, geo_loss_all=0.139, loss=0.343, accuracy=0.933, backward_time=0.907, grad_norm=81.207, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.220e-06, train_time=5.112 +[gpue05] 2025-06-01 10:11:57,498 (trainer:816) INFO: 16epoch:train:201-300batch: iter_time=9.261e-05, forward_time=0.331, class_loss=1.495, geo_loss_downstream=0.220, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.139, loss=0.306, accuracy=0.935, backward_time=0.856, grad_norm=50.749, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=6.163e-06, train_time=4.811 +[gpue05] 2025-06-01 10:14:14,706 (trainer:816) INFO: 16epoch:train:301-400batch: iter_time=1.005e-04, forward_time=0.346, class_loss=1.527, geo_loss_downstream=0.219, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.139, loss=0.312, accuracy=0.937, backward_time=1.012, grad_norm=74.229, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.106e-06, train_time=5.488 +[gpue05] 2025-06-01 10:16:28,302 (trainer:816) INFO: 16epoch:train:401-500batch: iter_time=9.490e-05, forward_time=0.316, class_loss=1.778, geo_loss_downstream=0.220, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.019, geo_loss_all=0.140, loss=0.363, accuracy=0.923, backward_time=1.006, grad_norm=87.793, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=6.050e-06, train_time=5.343 +[gpue05] 2025-06-01 10:18:31,032 (trainer:816) INFO: 16epoch:train:501-600batch: iter_time=9.826e-05, forward_time=0.294, class_loss=1.234, geo_loss_downstream=0.219, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.139, loss=0.254, accuracy=0.952, backward_time=0.919, grad_norm=66.782, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=5.995e-06, train_time=4.908 +[gpue05] 2025-06-01 10:20:38,280 (trainer:816) INFO: 16epoch:train:601-700batch: iter_time=9.515e-05, forward_time=0.264, class_loss=1.542, geo_loss_downstream=0.217, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.138, loss=0.315, accuracy=0.928, backward_time=0.993, grad_norm=70.824, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.011, optim0_lr0=5.940e-06, train_time=5.089 +[gpue05] 2025-06-01 10:22:35,916 (trainer:816) INFO: 16epoch:train:701-800batch: iter_time=8.910e-05, forward_time=0.265, class_loss=1.999, geo_loss_downstream=0.217, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.138, loss=0.407, accuracy=0.920, backward_time=0.896, grad_norm=69.975, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=5.885e-06, train_time=4.705 +[gpue05] 2025-06-01 10:24:41,702 (trainer:816) INFO: 16epoch:train:801-900batch: iter_time=9.683e-05, forward_time=0.242, class_loss=1.896, geo_loss_downstream=0.216, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.137, loss=0.386, accuracy=0.925, backward_time=1.001, grad_norm=90.875, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=5.831e-06, train_time=5.031 +[gpue05] 2025-06-01 10:26:35,841 (trainer:816) INFO: 16epoch:train:901-1000batch: iter_time=9.546e-05, forward_time=0.250, class_loss=1.422, geo_loss_downstream=0.217, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.137, loss=0.291, accuracy=0.946, backward_time=0.875, grad_norm=73.923, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=5.778e-06, train_time=4.565 +[gpue05] 2025-06-01 10:28:34,672 (trainer:816) INFO: 16epoch:train:1001-1100batch: iter_time=9.856e-05, forward_time=0.248, class_loss=1.984, geo_loss_downstream=0.217, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.138, loss=0.404, accuracy=0.925, backward_time=0.925, grad_norm=68.927, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=5.725e-06, train_time=4.753 +[gpue05] 2025-06-01 10:30:37,868 (trainer:816) INFO: 16epoch:train:1101-1200batch: iter_time=9.836e-05, forward_time=0.249, class_loss=1.586, geo_loss_downstream=0.216, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.137, loss=0.324, accuracy=0.932, backward_time=0.968, grad_norm=92.719, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=5.672e-06, train_time=4.927 +[gpue05] 2025-06-01 10:32:34,461 (trainer:816) INFO: 16epoch:train:1201-1300batch: iter_time=9.189e-05, forward_time=0.235, class_loss=1.591, geo_loss_downstream=0.216, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.137, loss=0.325, accuracy=0.932, backward_time=0.915, grad_norm=70.163, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=5.620e-06, train_time=4.663 +[gpue05] 2025-06-01 10:34:17,638 (trainer:816) INFO: 16epoch:train:1301-1400batch: iter_time=1.002e-04, forward_time=0.232, class_loss=1.728, geo_loss_downstream=0.216, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.019, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.019, inter_geo_loss_mean=0.019, geo_loss_all=0.137, loss=0.352, accuracy=0.928, backward_time=0.783, grad_norm=122.556, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=5.569e-06, train_time=4.126 +[gpue05] 2025-06-01 10:36:12,368 (trainer:816) INFO: 16epoch:train:1401-1500batch: iter_time=9.291e-05, forward_time=0.229, class_loss=1.790, geo_loss_downstream=0.214, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.135, loss=0.365, accuracy=0.925, backward_time=0.903, grad_norm=79.412, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=5.518e-06, train_time=4.588 +[gpue05] 2025-06-01 10:38:15,267 (trainer:816) INFO: 16epoch:train:1501-1600batch: iter_time=9.512e-05, forward_time=0.247, class_loss=1.087, geo_loss_downstream=0.214, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.135, loss=0.224, accuracy=0.960, backward_time=0.965, grad_norm=55.074, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=5.467e-06, train_time=4.915 +[gpue05] 2025-06-01 10:40:12,494 (trainer:816) INFO: 16epoch:train:1601-1700batch: iter_time=9.114e-05, forward_time=0.255, class_loss=1.818, geo_loss_downstream=0.215, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.019, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.137, loss=0.370, accuracy=0.925, backward_time=0.900, grad_norm=77.919, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=5.417e-06, train_time=4.688 +[gpue05] 2025-06-01 10:42:12,528 (trainer:816) INFO: 16epoch:train:1701-1800batch: iter_time=9.642e-05, forward_time=0.266, class_loss=1.605, geo_loss_downstream=0.212, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.135, loss=0.328, accuracy=0.930, backward_time=0.918, grad_norm=47.224, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=5.367e-06, train_time=4.801 +[gpue05] 2025-06-01 10:44:13,219 (trainer:816) INFO: 16epoch:train:1801-1900batch: iter_time=9.777e-05, forward_time=0.229, class_loss=1.603, geo_loss_downstream=0.214, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.135, loss=0.327, accuracy=0.935, backward_time=0.963, grad_norm=96.660, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=5.318e-06, train_time=4.827 +[gpue05] 2025-06-01 10:46:04,441 (trainer:816) INFO: 16epoch:train:1901-2000batch: iter_time=8.660e-05, forward_time=0.228, class_loss=1.622, geo_loss_downstream=0.212, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.134, loss=0.331, accuracy=0.942, backward_time=0.868, grad_norm=72.055, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.009, optim0_lr0=5.269e-06, train_time=4.448 +[gpue05] 2025-06-01 11:09:28,110 (trainer:401) INFO: 16epoch results: [train] iter_time=2.620e-04, forward_time=0.273, class_loss=1.628, geo_loss_downstream=0.217, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.137, loss=0.332, accuracy=0.934, backward_time=0.918, grad_norm=75.972, clip=0.000e+00, loss_scale=5.243e+05, optim_step_time=0.010, optim0_lr0=5.759e-06, train_time=4.828, time=42 minutes and 34.98 seconds, total_count=32000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=3.324, geo_loss_downstream=0.253, inter_geo_loss_layer32=0.026, inter_geo_loss_layer36=0.030, inter_geo_loss_layer40=0.032, inter_geo_loss_layer44=0.035, inter_geo_loss_mean=0.031, geo_loss_all=0.164, loss=2.692, accuracy=0.860, time=23 minutes and 23.37 seconds, total_count=75552, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 11:09:42,048 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 11:09:42,085 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/15epoch.pth +[gpue05] 2025-06-01 11:09:42,086 (trainer:335) INFO: 17/50epoch started. Estimated time to finish: 1 day, 13 hours and 21 minutes +[gpue05] 2025-06-01 11:14:09,552 (trainer:816) INFO: 17epoch:train:1-100batch: iter_time=0.002, forward_time=0.384, class_loss=1.492, geo_loss_downstream=0.211, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.133, loss=0.305, accuracy=0.945, backward_time=0.849, grad_norm=62.227, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=5.221e-06, train_time=5.024 +[gpue05] 2025-06-01 11:16:17,247 (trainer:816) INFO: 17epoch:train:101-200batch: iter_time=1.118e-04, forward_time=0.365, class_loss=1.606, geo_loss_downstream=0.211, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.134, loss=0.328, accuracy=0.933, backward_time=0.896, grad_norm=89.208, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=5.173e-06, train_time=5.107 +[gpue05] 2025-06-01 11:18:19,202 (trainer:816) INFO: 17epoch:train:201-300batch: iter_time=1.049e-04, forward_time=0.339, class_loss=1.425, geo_loss_downstream=0.211, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.134, loss=0.292, accuracy=0.947, backward_time=0.864, grad_norm=57.676, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=5.126e-06, train_time=4.877 +[gpue05] 2025-06-01 11:20:13,975 (trainer:816) INFO: 17epoch:train:301-400batch: iter_time=1.171e-04, forward_time=0.302, class_loss=1.341, geo_loss_downstream=0.211, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.134, loss=0.275, accuracy=0.942, backward_time=0.829, grad_norm=49.883, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=5.079e-06, train_time=4.590 +[gpue05] 2025-06-01 11:22:13,558 (trainer:816) INFO: 17epoch:train:401-500batch: iter_time=1.589e-04, forward_time=0.293, class_loss=1.417, geo_loss_downstream=0.208, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.132, loss=0.290, accuracy=0.937, backward_time=0.886, grad_norm=65.766, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=5.032e-06, train_time=4.783 +[gpue05] 2025-06-01 11:24:05,769 (trainer:816) INFO: 17epoch:train:501-600batch: iter_time=1.126e-04, forward_time=0.277, class_loss=1.121, geo_loss_downstream=0.209, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.132, loss=0.231, accuracy=0.953, backward_time=0.827, grad_norm=104.961, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.986e-06, train_time=4.488 +[gpue05] 2025-06-01 11:26:07,507 (trainer:816) INFO: 17epoch:train:601-700batch: iter_time=1.050e-04, forward_time=0.278, class_loss=1.779, geo_loss_downstream=0.211, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.134, loss=0.362, accuracy=0.930, backward_time=0.922, grad_norm=78.955, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=4.940e-06, train_time=4.869 +[gpue05] 2025-06-01 11:28:16,776 (trainer:816) INFO: 17epoch:train:701-800batch: iter_time=1.284e-04, forward_time=0.274, class_loss=1.964, geo_loss_downstream=0.209, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.132, loss=0.399, accuracy=0.917, backward_time=1.002, grad_norm=83.350, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.895e-06, train_time=5.170 +[gpue05] 2025-06-01 11:30:09,936 (trainer:816) INFO: 17epoch:train:801-900batch: iter_time=1.205e-04, forward_time=0.261, class_loss=1.670, geo_loss_downstream=0.208, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.132, loss=0.341, accuracy=0.933, backward_time=0.855, grad_norm=69.469, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.850e-06, train_time=4.526 +[gpue05] 2025-06-01 11:32:13,395 (trainer:816) INFO: 17epoch:train:901-1000batch: iter_time=1.245e-04, forward_time=0.248, class_loss=1.527, geo_loss_downstream=0.208, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.131, loss=0.312, accuracy=0.942, backward_time=0.971, grad_norm=56.235, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.806e-06, train_time=4.937 +[gpue05] 2025-06-01 11:34:04,026 (trainer:816) INFO: 17epoch:train:1001-1100batch: iter_time=1.360e-04, forward_time=0.222, class_loss=1.472, geo_loss_downstream=0.208, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.131, loss=0.301, accuracy=0.935, backward_time=0.869, grad_norm=59.081, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.762e-06, train_time=4.424 +[gpue05] 2025-06-01 11:35:54,577 (trainer:816) INFO: 17epoch:train:1101-1200batch: iter_time=1.285e-04, forward_time=0.235, class_loss=1.431, geo_loss_downstream=0.207, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.131, loss=0.293, accuracy=0.942, backward_time=0.854, grad_norm=60.284, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=4.718e-06, train_time=4.421 +[gpue05] 2025-06-01 11:37:44,807 (trainer:816) INFO: 17epoch:train:1201-1300batch: iter_time=1.140e-04, forward_time=0.227, class_loss=1.530, geo_loss_downstream=0.207, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.131, loss=0.313, accuracy=0.935, backward_time=0.858, grad_norm=62.123, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.675e-06, train_time=4.408 +[gpue05] 2025-06-01 11:39:41,943 (trainer:816) INFO: 17epoch:train:1301-1400batch: iter_time=1.201e-04, forward_time=0.259, class_loss=1.928, geo_loss_downstream=0.207, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.018, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.018, geo_loss_all=0.131, loss=0.392, accuracy=0.911, backward_time=0.896, grad_norm=69.540, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=4.632e-06, train_time=4.685 +[gpue05] 2025-06-01 11:41:47,088 (trainer:816) INFO: 17epoch:train:1401-1500batch: iter_time=1.260e-04, forward_time=0.275, class_loss=1.642, geo_loss_downstream=0.206, inter_geo_loss_layer32=0.018, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.018, inter_geo_loss_layer44=0.018, inter_geo_loss_mean=0.018, geo_loss_all=0.131, loss=0.335, accuracy=0.933, backward_time=0.961, grad_norm=60.255, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.589e-06, train_time=5.005 +[gpue05] 2025-06-01 11:43:46,436 (trainer:816) INFO: 17epoch:train:1501-1600batch: iter_time=1.325e-04, forward_time=0.248, class_loss=1.621, geo_loss_downstream=0.205, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.130, loss=0.331, accuracy=0.937, backward_time=0.929, grad_norm=77.639, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.547e-06, train_time=4.773 +[gpue05] 2025-06-01 11:45:52,383 (trainer:816) INFO: 17epoch:train:1601-1700batch: iter_time=1.642e-04, forward_time=0.282, class_loss=1.443, geo_loss_downstream=0.205, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.017, geo_loss_all=0.130, loss=0.295, accuracy=0.936, backward_time=0.960, grad_norm=59.332, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.012, optim0_lr0=4.506e-06, train_time=5.037 +[gpue05] 2025-06-01 11:47:45,807 (trainer:816) INFO: 17epoch:train:1701-1800batch: iter_time=1.312e-04, forward_time=0.247, class_loss=1.291, geo_loss_downstream=0.206, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.131, loss=0.265, accuracy=0.953, backward_time=0.870, grad_norm=53.183, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.464e-06, train_time=4.536 +[gpue05] 2025-06-01 11:49:56,381 (trainer:816) INFO: 17epoch:train:1801-1900batch: iter_time=1.381e-04, forward_time=0.286, class_loss=1.407, geo_loss_downstream=0.204, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.129, loss=0.288, accuracy=0.945, backward_time=1.005, grad_norm=70.793, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.423e-06, train_time=5.222 +[gpue05] 2025-06-01 11:52:04,748 (trainer:816) INFO: 17epoch:train:1901-2000batch: iter_time=1.329e-04, forward_time=0.273, class_loss=1.689, geo_loss_downstream=0.204, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.129, loss=0.344, accuracy=0.927, backward_time=0.995, grad_norm=70.371, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.383e-06, train_time=5.134 +[gpue05] 2025-06-01 12:15:31,476 (trainer:401) INFO: 17epoch results: [train] iter_time=2.371e-04, forward_time=0.279, class_loss=1.540, geo_loss_downstream=0.208, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.132, loss=0.315, accuracy=0.937, backward_time=0.905, grad_norm=68.016, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.790e-06, train_time=4.801, time=42 minutes and 22.96 seconds, total_count=34000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.892, geo_loss_downstream=0.271, inter_geo_loss_layer32=0.024, inter_geo_loss_layer36=0.026, inter_geo_loss_layer40=0.027, inter_geo_loss_layer44=0.028, inter_geo_loss_mean=0.026, geo_loss_all=0.173, loss=2.348, accuracy=0.881, time=23 minutes and 26.43 seconds, total_count=80274, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 12:15:45,597 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 12:15:45,609 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/13epoch.pth +[gpue05] 2025-06-01 12:15:45,609 (trainer:335) INFO: 18/50epoch started. Estimated time to finish: 1 day, 12 hours and 15 minutes +[gpue05] 2025-06-01 12:20:20,672 (trainer:816) INFO: 18epoch:train:1-100batch: iter_time=0.003, forward_time=0.409, class_loss=1.187, geo_loss_downstream=0.206, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.130, loss=0.244, accuracy=0.953, backward_time=0.911, grad_norm=55.740, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.343e-06, train_time=5.379 +[gpue05] 2025-06-01 12:22:36,288 (trainer:816) INFO: 18epoch:train:101-200batch: iter_time=1.303e-04, forward_time=0.378, class_loss=1.340, geo_loss_downstream=0.204, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.129, loss=0.274, accuracy=0.948, backward_time=0.962, grad_norm=53.023, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=4.303e-06, train_time=5.424 +[gpue05] 2025-06-01 12:24:49,218 (trainer:816) INFO: 18epoch:train:201-300batch: iter_time=1.129e-04, forward_time=0.352, class_loss=1.333, geo_loss_downstream=0.204, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.129, loss=0.273, accuracy=0.942, backward_time=0.956, grad_norm=59.748, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.028, optim0_lr0=4.263e-06, train_time=5.316 +[gpue05] 2025-06-01 12:26:51,742 (trainer:816) INFO: 18epoch:train:301-400batch: iter_time=1.232e-04, forward_time=0.315, class_loss=1.473, geo_loss_downstream=0.204, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.129, loss=0.301, accuracy=0.945, backward_time=0.894, grad_norm=67.914, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.224e-06, train_time=4.900 +[gpue05] 2025-06-01 12:29:13,170 (trainer:816) INFO: 18epoch:train:401-500batch: iter_time=1.402e-04, forward_time=0.336, class_loss=1.625, geo_loss_downstream=0.202, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.128, loss=0.331, accuracy=0.933, backward_time=1.064, grad_norm=75.256, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.186e-06, train_time=5.656 +[gpue05] 2025-06-01 12:31:09,510 (trainer:816) INFO: 18epoch:train:501-600batch: iter_time=1.134e-04, forward_time=0.276, class_loss=1.544, geo_loss_downstream=0.200, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.126, loss=0.315, accuracy=0.933, backward_time=0.871, grad_norm=66.815, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.147e-06, train_time=4.653 +[gpue05] 2025-06-01 12:33:16,904 (trainer:816) INFO: 18epoch:train:601-700batch: iter_time=1.121e-04, forward_time=0.271, class_loss=1.468, geo_loss_downstream=0.201, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.127, loss=0.300, accuracy=0.942, backward_time=0.988, grad_norm=57.211, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=4.109e-06, train_time=5.095 +[gpue05] 2025-06-01 12:35:20,324 (trainer:816) INFO: 18epoch:train:701-800batch: iter_time=1.186e-04, forward_time=0.269, class_loss=1.208, geo_loss_downstream=0.201, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.128, loss=0.248, accuracy=0.948, backward_time=0.950, grad_norm=60.458, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=4.072e-06, train_time=4.936 +[gpue05] 2025-06-01 12:37:30,118 (trainer:816) INFO: 18epoch:train:801-900batch: iter_time=1.354e-04, forward_time=0.266, class_loss=1.460, geo_loss_downstream=0.201, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.127, loss=0.298, accuracy=0.938, backward_time=1.016, grad_norm=61.067, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=4.034e-06, train_time=5.191 +[gpue05] 2025-06-01 12:39:23,735 (trainer:816) INFO: 18epoch:train:901-1000batch: iter_time=1.432e-04, forward_time=0.251, class_loss=1.873, geo_loss_downstream=0.202, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.128, loss=0.381, accuracy=0.928, backward_time=0.868, grad_norm=74.149, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=3.997e-06, train_time=4.544 +[gpue05] 2025-06-01 12:41:16,398 (trainer:816) INFO: 18epoch:train:1001-1100batch: iter_time=1.350e-04, forward_time=0.236, class_loss=1.400, geo_loss_downstream=0.200, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.126, loss=0.286, accuracy=0.943, backward_time=0.873, grad_norm=64.242, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.961e-06, train_time=4.506 +[gpue05] 2025-06-01 12:43:08,636 (trainer:816) INFO: 18epoch:train:1101-1200batch: iter_time=1.119e-04, forward_time=0.221, class_loss=1.600, geo_loss_downstream=0.198, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.125, loss=0.326, accuracy=0.943, backward_time=0.886, grad_norm=62.542, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.924e-06, train_time=4.489 +[gpue05] 2025-06-01 12:45:15,899 (trainer:816) INFO: 18epoch:train:1201-1300batch: iter_time=1.138e-04, forward_time=0.259, class_loss=1.810, geo_loss_downstream=0.200, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.127, loss=0.368, accuracy=0.927, backward_time=1.000, grad_norm=65.996, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.888e-06, train_time=5.090 +[gpue05] 2025-06-01 12:47:29,755 (trainer:816) INFO: 18epoch:train:1301-1400batch: iter_time=1.288e-04, forward_time=0.278, class_loss=1.299, geo_loss_downstream=0.198, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.125, loss=0.266, accuracy=0.952, backward_time=1.046, grad_norm=60.451, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.853e-06, train_time=5.353 +[gpue05] 2025-06-01 12:49:47,095 (trainer:816) INFO: 18epoch:train:1401-1500batch: iter_time=1.287e-04, forward_time=0.291, class_loss=1.739, geo_loss_downstream=0.198, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.126, loss=0.354, accuracy=0.927, backward_time=1.068, grad_norm=53.518, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.817e-06, train_time=5.493 +[gpue05] 2025-06-01 12:51:37,147 (trainer:816) INFO: 18epoch:train:1501-1600batch: iter_time=1.426e-04, forward_time=0.235, class_loss=1.806, geo_loss_downstream=0.198, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.126, loss=0.368, accuracy=0.922, backward_time=0.851, grad_norm=67.710, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.782e-06, train_time=4.401 +[gpue05] 2025-06-01 12:53:47,293 (trainer:816) INFO: 18epoch:train:1601-1700batch: iter_time=1.355e-04, forward_time=0.283, class_loss=1.510, geo_loss_downstream=0.198, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.125, loss=0.308, accuracy=0.942, backward_time=1.003, grad_norm=75.602, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.748e-06, train_time=5.205 +[gpue05] 2025-06-01 12:55:54,166 (trainer:816) INFO: 18epoch:train:1701-1800batch: iter_time=1.554e-04, forward_time=0.253, class_loss=1.253, geo_loss_downstream=0.197, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.124, loss=0.257, accuracy=0.948, backward_time=0.999, grad_norm=62.729, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.713e-06, train_time=5.074 +[gpue05] 2025-06-01 12:57:47,450 (trainer:816) INFO: 18epoch:train:1801-1900batch: iter_time=1.443e-04, forward_time=0.245, class_loss=1.494, geo_loss_downstream=0.199, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.126, loss=0.305, accuracy=0.942, backward_time=0.872, grad_norm=58.385, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.679e-06, train_time=4.531 +[gpue05] 2025-06-01 12:59:46,537 (trainer:816) INFO: 18epoch:train:1901-2000batch: iter_time=1.131e-04, forward_time=0.253, class_loss=1.678, geo_loss_downstream=0.197, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.017, geo_loss_all=0.125, loss=0.342, accuracy=0.932, backward_time=0.923, grad_norm=54.955, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.646e-06, train_time=4.763 +[gpue05] 2025-06-01 13:23:07,713 (trainer:401) INFO: 18epoch results: [train] iter_time=2.591e-04, forward_time=0.284, class_loss=1.505, geo_loss_downstream=0.200, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.017, geo_loss_all=0.127, loss=0.307, accuracy=0.939, backward_time=0.950, grad_norm=62.875, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=3.985e-06, train_time=5.000, time=44 minutes and 1.2 seconds, total_count=36000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.834, geo_loss_downstream=0.235, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.024, inter_geo_loss_layer40=0.024, inter_geo_loss_layer44=0.025, inter_geo_loss_mean=0.024, geo_loss_all=0.151, loss=2.297, accuracy=0.883, time=23 minutes and 20.9 seconds, total_count=84996, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 13:23:21,637 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 13:23:21,648 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/16epoch.pth +[gpue05] 2025-06-01 13:23:21,648 (trainer:335) INFO: 19/50epoch started. Estimated time to finish: 1 day, 11 hours and 12 minutes +[gpue05] 2025-06-01 13:27:52,764 (trainer:816) INFO: 19epoch:train:1-100batch: iter_time=0.003, forward_time=0.387, class_loss=1.616, geo_loss_downstream=0.198, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.125, loss=0.329, accuracy=0.940, backward_time=0.909, grad_norm=62.879, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.612e-06, train_time=5.272 +[gpue05] 2025-06-01 13:30:11,381 (trainer:816) INFO: 19epoch:train:101-200batch: iter_time=1.068e-04, forward_time=0.387, class_loss=1.322, geo_loss_downstream=0.195, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.123, loss=0.271, accuracy=0.945, backward_time=0.985, grad_norm=60.112, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.579e-06, train_time=5.544 +[gpue05] 2025-06-01 13:32:24,235 (trainer:816) INFO: 19epoch:train:201-300batch: iter_time=9.916e-05, forward_time=0.348, class_loss=1.629, geo_loss_downstream=0.197, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.125, loss=0.332, accuracy=0.935, backward_time=0.966, grad_norm=81.370, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.546e-06, train_time=5.313 +[gpue05] 2025-06-01 13:34:22,727 (trainer:816) INFO: 19epoch:train:301-400batch: iter_time=1.123e-04, forward_time=0.305, class_loss=1.435, geo_loss_downstream=0.194, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.123, loss=0.293, accuracy=0.942, backward_time=0.863, grad_norm=77.350, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.514e-06, train_time=4.739 +[gpue05] 2025-06-01 13:36:25,227 (trainer:816) INFO: 19epoch:train:401-500batch: iter_time=1.048e-04, forward_time=0.302, class_loss=1.108, geo_loss_downstream=0.196, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.124, loss=0.228, accuracy=0.960, backward_time=0.906, grad_norm=56.034, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.481e-06, train_time=4.899 +[gpue05] 2025-06-01 13:38:14,071 (trainer:816) INFO: 19epoch:train:501-600batch: iter_time=1.039e-04, forward_time=0.259, class_loss=1.484, geo_loss_downstream=0.196, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.124, loss=0.303, accuracy=0.940, backward_time=0.813, grad_norm=83.559, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.450e-06, train_time=4.353 +[gpue05] 2025-06-01 13:40:23,050 (trainer:816) INFO: 19epoch:train:601-700batch: iter_time=9.902e-05, forward_time=0.286, class_loss=1.085, geo_loss_downstream=0.195, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.124, loss=0.223, accuracy=0.962, backward_time=0.989, grad_norm=57.987, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.418e-06, train_time=5.158 +[gpue05] 2025-06-01 13:42:21,470 (trainer:816) INFO: 19epoch:train:701-800batch: iter_time=9.752e-05, forward_time=0.260, class_loss=1.616, geo_loss_downstream=0.195, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.123, loss=0.329, accuracy=0.932, backward_time=0.908, grad_norm=77.642, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.387e-06, train_time=4.736 +[gpue05] 2025-06-01 13:44:18,746 (trainer:816) INFO: 19epoch:train:801-900batch: iter_time=1.038e-04, forward_time=0.244, class_loss=1.687, geo_loss_downstream=0.193, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.017, inter_geo_loss_layer44=0.017, inter_geo_loss_mean=0.017, geo_loss_all=0.123, loss=0.344, accuracy=0.933, backward_time=0.912, grad_norm=59.114, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.356e-06, train_time=4.690 +[gpue05] 2025-06-01 13:46:07,327 (trainer:816) INFO: 19epoch:train:901-1000batch: iter_time=1.091e-04, forward_time=0.238, class_loss=1.537, geo_loss_downstream=0.194, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.122, loss=0.313, accuracy=0.937, backward_time=0.831, grad_norm=60.245, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.325e-06, train_time=4.342 +[gpue05] 2025-06-01 13:48:05,659 (trainer:816) INFO: 19epoch:train:1001-1100batch: iter_time=1.011e-04, forward_time=0.244, class_loss=1.422, geo_loss_downstream=0.193, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.016, geo_loss_all=0.122, loss=0.290, accuracy=0.945, backward_time=0.924, grad_norm=69.774, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.294e-06, train_time=4.733 +[gpue05] 2025-06-01 13:49:59,107 (trainer:816) INFO: 19epoch:train:1101-1200batch: iter_time=1.075e-04, forward_time=0.237, class_loss=1.372, geo_loss_downstream=0.194, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.017, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.123, loss=0.281, accuracy=0.943, backward_time=0.880, grad_norm=95.309, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.264e-06, train_time=4.537 +[gpue05] 2025-06-01 13:51:58,719 (trainer:816) INFO: 19epoch:train:1201-1300batch: iter_time=9.577e-05, forward_time=0.263, class_loss=1.392, geo_loss_downstream=0.192, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.121, loss=0.284, accuracy=0.943, backward_time=0.919, grad_norm=74.176, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.234e-06, train_time=4.784 +[gpue05] 2025-06-01 13:54:02,983 (trainer:816) INFO: 19epoch:train:1301-1400batch: iter_time=1.081e-04, forward_time=0.260, class_loss=1.026, geo_loss_downstream=0.192, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.121, loss=0.211, accuracy=0.957, backward_time=0.967, grad_norm=62.589, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.205e-06, train_time=4.970 +[gpue05] 2025-06-01 13:56:04,877 (trainer:816) INFO: 19epoch:train:1401-1500batch: iter_time=1.131e-04, forward_time=0.235, class_loss=1.365, geo_loss_downstream=0.193, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.122, loss=0.279, accuracy=0.946, backward_time=0.969, grad_norm=64.603, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.175e-06, train_time=4.875 +[gpue05] 2025-06-01 13:57:58,169 (trainer:816) INFO: 19epoch:train:1501-1600batch: iter_time=1.013e-04, forward_time=0.232, class_loss=1.426, geo_loss_downstream=0.194, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.123, loss=0.291, accuracy=0.938, backward_time=0.885, grad_norm=75.328, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.146e-06, train_time=4.531 +[gpue05] 2025-06-01 13:59:51,812 (trainer:816) INFO: 19epoch:train:1601-1700batch: iter_time=1.006e-04, forward_time=0.222, class_loss=1.352, geo_loss_downstream=0.193, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.122, loss=0.276, accuracy=0.948, backward_time=0.899, grad_norm=59.212, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.117e-06, train_time=4.545 +[gpue05] 2025-06-01 14:02:01,969 (trainer:816) INFO: 19epoch:train:1701-1800batch: iter_time=1.038e-04, forward_time=0.259, class_loss=1.502, geo_loss_downstream=0.191, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.121, loss=0.306, accuracy=0.940, backward_time=1.029, grad_norm=61.281, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.089e-06, train_time=5.206 +[gpue05] 2025-06-01 14:04:03,971 (trainer:816) INFO: 19epoch:train:1801-1900batch: iter_time=9.433e-05, forward_time=0.257, class_loss=1.563, geo_loss_downstream=0.190, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.120, loss=0.319, accuracy=0.930, backward_time=0.948, grad_norm=67.793, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.060e-06, train_time=4.879 +[gpue05] 2025-06-01 14:05:55,595 (trainer:816) INFO: 19epoch:train:1901-2000batch: iter_time=1.043e-04, forward_time=0.237, class_loss=1.333, geo_loss_downstream=0.191, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.121, loss=0.273, accuracy=0.953, backward_time=0.864, grad_norm=75.040, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.032e-06, train_time=4.464 +[gpue05] 2025-06-01 14:29:17,179 (trainer:401) INFO: 19epoch results: [train] iter_time=2.543e-04, forward_time=0.273, class_loss=1.414, geo_loss_downstream=0.194, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.123, loss=0.289, accuracy=0.943, backward_time=0.918, grad_norm=69.070, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=3.314e-06, train_time=4.829, time=42 minutes and 34.22 seconds, total_count=38000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.851, geo_loss_downstream=0.222, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.024, inter_geo_loss_layer40=0.025, inter_geo_loss_layer44=0.026, inter_geo_loss_mean=0.024, geo_loss_all=0.143, loss=2.309, accuracy=0.883, time=23 minutes and 21.31 seconds, total_count=89718, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 14:29:31,070 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 14:29:31,082 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/17epoch.pth +[gpue05] 2025-06-01 14:29:31,082 (trainer:335) INFO: 20/50epoch started. Estimated time to finish: 1 day, 10 hours and 6 minutes +[gpue05] 2025-06-01 14:34:08,363 (trainer:816) INFO: 20epoch:train:1-100batch: iter_time=0.004, forward_time=0.401, class_loss=1.276, geo_loss_downstream=0.191, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.120, loss=0.261, accuracy=0.950, backward_time=0.951, grad_norm=52.635, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=3.004e-06, train_time=5.502 +[gpue05] 2025-06-01 14:36:23,985 (trainer:816) INFO: 20epoch:train:101-200batch: iter_time=1.067e-04, forward_time=0.385, class_loss=1.302, geo_loss_downstream=0.191, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.120, loss=0.266, accuracy=0.952, backward_time=0.954, grad_norm=57.286, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.014, optim0_lr0=2.977e-06, train_time=5.424 +[gpue05] 2025-06-01 14:38:41,114 (trainer:816) INFO: 20epoch:train:201-300batch: iter_time=1.012e-04, forward_time=0.360, class_loss=1.562, geo_loss_downstream=0.191, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.015, geo_loss_all=0.121, loss=0.318, accuracy=0.933, backward_time=0.993, grad_norm=75.044, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.021, optim0_lr0=2.950e-06, train_time=5.484 +[gpue05] 2025-06-01 14:41:04,346 (trainer:816) INFO: 20epoch:train:301-400batch: iter_time=1.109e-04, forward_time=0.388, class_loss=1.755, geo_loss_downstream=0.189, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.016, geo_loss_all=0.120, loss=0.357, accuracy=0.923, backward_time=1.030, grad_norm=60.296, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=2.923e-06, train_time=5.729 +[gpue05] 2025-06-01 14:42:58,517 (trainer:816) INFO: 20epoch:train:401-500batch: iter_time=9.612e-05, forward_time=0.281, class_loss=1.132, geo_loss_downstream=0.190, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.016, geo_loss_all=0.120, loss=0.233, accuracy=0.955, backward_time=0.844, grad_norm=54.084, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=2.896e-06, train_time=4.566 +[gpue05] 2025-06-01 14:44:57,615 (trainer:816) INFO: 20epoch:train:501-600batch: iter_time=1.014e-04, forward_time=0.282, class_loss=1.310, geo_loss_downstream=0.189, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.120, loss=0.268, accuracy=0.945, backward_time=0.893, grad_norm=67.969, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=2.869e-06, train_time=4.763 +[gpue05] 2025-06-01 14:47:03,110 (trainer:816) INFO: 20epoch:train:601-700batch: iter_time=9.488e-05, forward_time=0.291, class_loss=1.307, geo_loss_downstream=0.188, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.119, loss=0.267, accuracy=0.950, backward_time=0.949, grad_norm=54.078, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=2.843e-06, train_time=5.019 +[gpue05] 2025-06-01 14:49:05,156 (trainer:816) INFO: 20epoch:train:701-800batch: iter_time=9.570e-05, forward_time=0.266, class_loss=1.330, geo_loss_downstream=0.187, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.118, loss=0.272, accuracy=0.945, backward_time=0.939, grad_norm=58.285, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=2.817e-06, train_time=4.881 +[gpue05] 2025-06-01 14:51:01,858 (trainer:816) INFO: 20epoch:train:801-900batch: iter_time=1.008e-04, forward_time=0.261, class_loss=1.083, geo_loss_downstream=0.187, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.118, loss=0.222, accuracy=0.957, backward_time=0.891, grad_norm=76.659, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=2.791e-06, train_time=4.667 +[gpue05] 2025-06-01 14:52:54,788 (trainer:816) INFO: 20epoch:train:901-1000batch: iter_time=1.041e-04, forward_time=0.250, class_loss=1.412, geo_loss_downstream=0.189, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.120, loss=0.288, accuracy=0.945, backward_time=0.864, grad_norm=58.419, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=2.765e-06, train_time=4.517 +[gpue05] 2025-06-01 14:54:48,996 (trainer:816) INFO: 20epoch:train:1001-1100batch: iter_time=1.044e-04, forward_time=0.255, class_loss=1.123, geo_loss_downstream=0.188, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.119, loss=0.231, accuracy=0.960, backward_time=0.872, grad_norm=77.930, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=2.740e-06, train_time=4.568 +[gpue05] 2025-06-01 14:56:46,490 (trainer:816) INFO: 20epoch:train:1101-1200batch: iter_time=9.973e-05, forward_time=0.247, class_loss=1.353, geo_loss_downstream=0.188, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.119, loss=0.276, accuracy=0.947, backward_time=0.912, grad_norm=51.530, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=2.715e-06, train_time=4.699 +[gpue05] 2025-06-01 14:58:45,993 (trainer:816) INFO: 20epoch:train:1201-1300batch: iter_time=1.107e-04, forward_time=0.239, class_loss=1.200, geo_loss_downstream=0.186, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.118, loss=0.246, accuracy=0.955, backward_time=0.940, grad_norm=47.534, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=2.690e-06, train_time=4.779 +[gpue05] 2025-06-01 15:00:41,967 (trainer:816) INFO: 20epoch:train:1301-1400batch: iter_time=9.831e-05, forward_time=0.246, class_loss=1.472, geo_loss_downstream=0.187, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.119, loss=0.300, accuracy=0.938, backward_time=0.898, grad_norm=78.109, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=2.665e-06, train_time=4.638 +[gpue05] 2025-06-01 15:02:37,817 (trainer:816) INFO: 20epoch:train:1401-1500batch: iter_time=9.947e-05, forward_time=0.231, class_loss=1.452, geo_loss_downstream=0.188, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.119, loss=0.296, accuracy=0.947, backward_time=0.912, grad_norm=63.763, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.010, optim0_lr0=2.641e-06, train_time=4.633 +[gpue05] 2025-06-01 15:04:29,712 (trainer:816) INFO: 20epoch:train:1501-1600batch: iter_time=1.027e-04, forward_time=0.229, class_loss=1.226, geo_loss_downstream=0.186, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.118, loss=0.251, accuracy=0.953, backward_time=0.874, grad_norm=83.169, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=2.617e-06, train_time=4.475 +[gpue05] 2025-06-01 15:06:23,894 (trainer:816) INFO: 20epoch:train:1601-1700batch: iter_time=9.527e-05, forward_time=0.237, class_loss=1.107, geo_loss_downstream=0.185, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.117, loss=0.227, accuracy=0.955, backward_time=0.890, grad_norm=53.946, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=2.593e-06, train_time=4.567 +[gpue05] 2025-06-01 15:08:14,463 (trainer:816) INFO: 20epoch:train:1701-1800batch: iter_time=9.892e-05, forward_time=0.229, class_loss=1.541, geo_loss_downstream=0.188, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.119, loss=0.314, accuracy=0.940, backward_time=0.860, grad_norm=64.984, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=2.569e-06, train_time=4.422 +[gpue05] 2025-06-01 15:10:00,882 (trainer:816) INFO: 20epoch:train:1801-1900batch: iter_time=1.055e-04, forward_time=0.230, class_loss=1.452, geo_loss_downstream=0.188, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.119, loss=0.296, accuracy=0.937, backward_time=0.812, grad_norm=65.902, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.028, optim0_lr0=2.545e-06, train_time=4.256 +[gpue05] 2025-06-01 15:11:59,014 (trainer:816) INFO: 20epoch:train:1901-2000batch: iter_time=9.095e-05, forward_time=0.240, class_loss=1.598, geo_loss_downstream=0.185, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.117, loss=0.325, accuracy=0.930, backward_time=0.926, grad_norm=61.957, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.009, optim0_lr0=2.522e-06, train_time=4.725 +[gpue05] 2025-06-01 15:35:35,316 (trainer:401) INFO: 20epoch results: [train] iter_time=2.906e-04, forward_time=0.277, class_loss=1.349, geo_loss_downstream=0.188, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.119, loss=0.276, accuracy=0.946, backward_time=0.910, grad_norm=63.179, clip=0.000e+00, loss_scale=1.049e+06, optim_step_time=0.011, optim0_lr0=2.757e-06, train_time=4.816, time=42 minutes and 28.2 seconds, total_count=40000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.686, geo_loss_downstream=0.221, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.025, inter_geo_loss_mean=0.023, geo_loss_all=0.142, loss=2.177, accuracy=0.888, time=23 minutes and 36.02 seconds, total_count=94440, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 15:35:48,853 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 15:35:48,868 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/18epoch.pth +[gpue05] 2025-06-01 15:35:48,868 (trainer:335) INFO: 21/50epoch started. Estimated time to finish: 1 day, 9 hours and 1 minute +[gpue05] 2025-06-01 15:40:27,068 (trainer:816) INFO: 21epoch:train:1-100batch: iter_time=0.002, forward_time=0.405, class_loss=1.489, geo_loss_downstream=0.187, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.118, loss=0.304, accuracy=0.940, backward_time=0.955, grad_norm=66.705, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.499e-06, train_time=5.519 +[gpue05] 2025-06-01 15:42:36,037 (trainer:816) INFO: 21epoch:train:101-200batch: iter_time=9.761e-05, forward_time=0.364, class_loss=1.407, geo_loss_downstream=0.184, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.116, loss=0.287, accuracy=0.940, backward_time=0.910, grad_norm=58.265, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.476e-06, train_time=5.158 +[gpue05] 2025-06-01 15:44:39,844 (trainer:816) INFO: 21epoch:train:201-300batch: iter_time=1.067e-04, forward_time=0.340, class_loss=1.745, geo_loss_downstream=0.186, inter_geo_loss_layer32=0.017, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.118, loss=0.355, accuracy=0.927, backward_time=0.882, grad_norm=80.532, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.453e-06, train_time=4.952 +[gpue05] 2025-06-01 15:46:45,330 (trainer:816) INFO: 21epoch:train:301-400batch: iter_time=1.027e-04, forward_time=0.318, class_loss=1.173, geo_loss_downstream=0.183, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.116, loss=0.240, accuracy=0.948, backward_time=0.923, grad_norm=47.056, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.431e-06, train_time=5.019 +[gpue05] 2025-06-01 15:48:41,053 (trainer:816) INFO: 21epoch:train:401-500batch: iter_time=1.081e-04, forward_time=0.290, class_loss=1.191, geo_loss_downstream=0.185, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.117, loss=0.244, accuracy=0.952, backward_time=0.852, grad_norm=67.763, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.409e-06, train_time=4.628 +[gpue05] 2025-06-01 15:50:53,797 (trainer:816) INFO: 21epoch:train:501-600batch: iter_time=1.257e-04, forward_time=0.305, class_loss=1.450, geo_loss_downstream=0.185, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.117, loss=0.296, accuracy=0.935, backward_time=1.008, grad_norm=64.789, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.387e-06, train_time=5.309 +[gpue05] 2025-06-01 15:52:44,806 (trainer:816) INFO: 21epoch:train:601-700batch: iter_time=9.668e-05, forward_time=0.257, class_loss=1.389, geo_loss_downstream=0.183, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.116, loss=0.284, accuracy=0.947, backward_time=0.837, grad_norm=57.858, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.365e-06, train_time=4.440 +[gpue05] 2025-06-01 15:54:51,766 (trainer:816) INFO: 21epoch:train:701-800batch: iter_time=1.021e-04, forward_time=0.276, class_loss=1.426, geo_loss_downstream=0.183, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.116, loss=0.291, accuracy=0.943, backward_time=0.977, grad_norm=79.643, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.343e-06, train_time=5.078 +[gpue05] 2025-06-01 15:56:53,726 (trainer:816) INFO: 21epoch:train:801-900batch: iter_time=1.230e-04, forward_time=0.260, class_loss=1.159, geo_loss_downstream=0.184, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.116, loss=0.238, accuracy=0.952, backward_time=0.943, grad_norm=60.100, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.321e-06, train_time=4.878 +[gpue05] 2025-06-01 15:58:41,269 (trainer:816) INFO: 21epoch:train:901-1000batch: iter_time=1.057e-04, forward_time=0.219, class_loss=1.584, geo_loss_downstream=0.183, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.116, loss=0.323, accuracy=0.932, backward_time=0.840, grad_norm=71.272, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=2.300e-06, train_time=4.301 +[gpue05] 2025-06-01 16:00:42,850 (trainer:816) INFO: 21epoch:train:1001-1100batch: iter_time=1.087e-04, forward_time=0.257, class_loss=1.266, geo_loss_downstream=0.184, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.116, loss=0.259, accuracy=0.945, backward_time=0.943, grad_norm=50.755, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.279e-06, train_time=4.863 +[gpue05] 2025-06-01 16:02:41,219 (trainer:816) INFO: 21epoch:train:1101-1200batch: iter_time=1.081e-04, forward_time=0.233, class_loss=1.127, geo_loss_downstream=0.183, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.115, loss=0.231, accuracy=0.957, backward_time=0.934, grad_norm=54.602, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=2.258e-06, train_time=4.734 +[gpue05] 2025-06-01 16:04:34,318 (trainer:816) INFO: 21epoch:train:1201-1300batch: iter_time=1.006e-04, forward_time=0.254, class_loss=1.132, geo_loss_downstream=0.182, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.116, loss=0.232, accuracy=0.955, backward_time=0.862, grad_norm=47.500, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.237e-06, train_time=4.523 +[gpue05] 2025-06-01 16:06:30,036 (trainer:816) INFO: 21epoch:train:1301-1400batch: iter_time=1.168e-04, forward_time=0.255, class_loss=1.456, geo_loss_downstream=0.181, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.115, loss=0.297, accuracy=0.933, backward_time=0.887, grad_norm=63.905, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.217e-06, train_time=4.628 +[gpue05] 2025-06-01 16:08:24,593 (trainer:816) INFO: 21epoch:train:1401-1500batch: iter_time=1.409e-04, forward_time=0.236, class_loss=1.225, geo_loss_downstream=0.183, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.116, loss=0.251, accuracy=0.953, backward_time=0.892, grad_norm=56.246, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.197e-06, train_time=4.582 +[gpue05] 2025-06-01 16:10:21,359 (trainer:816) INFO: 21epoch:train:1501-1600batch: iter_time=9.871e-05, forward_time=0.253, class_loss=1.804, geo_loss_downstream=0.182, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.016, geo_loss_all=0.116, loss=0.367, accuracy=0.918, backward_time=0.898, grad_norm=69.364, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=2.177e-06, train_time=4.670 +[gpue05] 2025-06-01 16:12:19,950 (trainer:816) INFO: 21epoch:train:1601-1700batch: iter_time=1.081e-04, forward_time=0.254, class_loss=1.513, geo_loss_downstream=0.181, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.115, loss=0.308, accuracy=0.935, backward_time=0.916, grad_norm=81.877, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.157e-06, train_time=4.743 +[gpue05] 2025-06-01 16:14:27,492 (trainer:816) INFO: 21epoch:train:1701-1800batch: iter_time=1.385e-04, forward_time=0.266, class_loss=1.161, geo_loss_downstream=0.182, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.115, loss=0.238, accuracy=0.960, backward_time=0.993, grad_norm=63.865, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.137e-06, train_time=5.101 +[gpue05] 2025-06-01 16:16:30,029 (trainer:816) INFO: 21epoch:train:1801-1900batch: iter_time=1.323e-04, forward_time=0.273, class_loss=1.405, geo_loss_downstream=0.181, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.114, loss=0.287, accuracy=0.937, backward_time=0.935, grad_norm=71.245, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.012, optim0_lr0=2.117e-06, train_time=4.901 +[gpue05] 2025-06-01 16:18:34,684 (trainer:816) INFO: 21epoch:train:1901-2000batch: iter_time=1.134e-04, forward_time=0.248, class_loss=1.249, geo_loss_downstream=0.181, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.114, loss=0.255, accuracy=0.950, backward_time=0.983, grad_norm=38.101, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.098e-06, train_time=4.985 +[gpue05] 2025-06-01 16:41:58,773 (trainer:401) INFO: 21epoch results: [train] iter_time=2.229e-04, forward_time=0.278, class_loss=1.368, geo_loss_downstream=0.183, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.116, loss=0.279, accuracy=0.943, backward_time=0.919, grad_norm=62.572, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.293e-06, train_time=4.850, time=42 minutes and 46.09 seconds, total_count=42000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.777, geo_loss_downstream=0.232, inter_geo_loss_layer32=0.023, inter_geo_loss_layer36=0.026, inter_geo_loss_layer40=0.026, inter_geo_loss_layer44=0.026, inter_geo_loss_mean=0.025, geo_loss_all=0.149, loss=2.252, accuracy=0.886, time=23 minutes and 23.81 seconds, total_count=99162, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 16:42:12,952 (trainer:467) INFO: There are no improvements in this epoch +[gpue05] 2025-06-01 16:42:12,966 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/19epoch.pth +[gpue05] 2025-06-01 16:42:12,966 (trainer:335) INFO: 22/50epoch started. Estimated time to finish: 1 day, 7 hours and 55 minutes +[gpue05] 2025-06-01 16:46:31,410 (trainer:816) INFO: 22epoch:train:1-100batch: iter_time=0.003, forward_time=0.388, class_loss=1.625, geo_loss_downstream=0.182, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.115, loss=0.331, accuracy=0.933, backward_time=0.764, grad_norm=81.904, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=2.079e-06, train_time=4.704 +[gpue05] 2025-06-01 16:48:38,426 (trainer:816) INFO: 22epoch:train:101-200batch: iter_time=1.229e-04, forward_time=0.354, class_loss=1.339, geo_loss_downstream=0.181, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.114, loss=0.274, accuracy=0.943, backward_time=0.901, grad_norm=77.509, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.059e-06, train_time=5.080 +[gpue05] 2025-06-01 16:50:42,439 (trainer:816) INFO: 22epoch:train:201-300batch: iter_time=9.759e-05, forward_time=0.332, class_loss=1.393, geo_loss_downstream=0.181, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.114, loss=0.284, accuracy=0.943, backward_time=0.893, grad_norm=69.637, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=2.041e-06, train_time=4.960 +[gpue05] 2025-06-01 16:52:37,444 (trainer:816) INFO: 22epoch:train:301-400batch: iter_time=9.445e-05, forward_time=0.307, class_loss=1.526, geo_loss_downstream=0.180, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.016, geo_loss_all=0.114, loss=0.311, accuracy=0.932, backward_time=0.826, grad_norm=66.184, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=2.022e-06, train_time=4.600 +[gpue05] 2025-06-01 16:54:41,568 (trainer:816) INFO: 22epoch:train:401-500batch: iter_time=1.035e-04, forward_time=0.309, class_loss=0.883, geo_loss_downstream=0.180, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.114, loss=0.182, accuracy=0.967, backward_time=0.916, grad_norm=47.546, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=2.003e-06, train_time=4.964 +[gpue05] 2025-06-01 16:56:44,268 (trainer:816) INFO: 22epoch:train:501-600batch: iter_time=1.052e-04, forward_time=0.294, class_loss=1.345, geo_loss_downstream=0.180, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.114, loss=0.275, accuracy=0.947, backward_time=0.917, grad_norm=49.200, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.985e-06, train_time=4.907 +[gpue05] 2025-06-01 16:58:33,721 (trainer:816) INFO: 22epoch:train:601-700batch: iter_time=1.013e-04, forward_time=0.257, class_loss=1.356, geo_loss_downstream=0.181, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.114, loss=0.277, accuracy=0.947, backward_time=0.822, grad_norm=54.983, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.967e-06, train_time=4.377 +[gpue05] 2025-06-01 17:00:39,136 (trainer:816) INFO: 22epoch:train:701-800batch: iter_time=9.918e-05, forward_time=0.272, class_loss=1.370, geo_loss_downstream=0.182, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.115, loss=0.280, accuracy=0.945, backward_time=0.966, grad_norm=79.491, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.949e-06, train_time=5.016 +[gpue05] 2025-06-01 17:02:43,691 (trainer:816) INFO: 22epoch:train:801-900batch: iter_time=1.087e-04, forward_time=0.263, class_loss=1.321, geo_loss_downstream=0.179, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.114, loss=0.270, accuracy=0.950, backward_time=0.968, grad_norm=52.062, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.931e-06, train_time=4.982 +[gpue05] 2025-06-01 17:04:42,632 (trainer:816) INFO: 22epoch:train:901-1000batch: iter_time=1.051e-04, forward_time=0.253, class_loss=1.223, geo_loss_downstream=0.179, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.113, loss=0.250, accuracy=0.948, backward_time=0.921, grad_norm=61.123, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.913e-06, train_time=4.757 +[gpue05] 2025-06-01 17:06:38,633 (trainer:816) INFO: 22epoch:train:1001-1100batch: iter_time=1.025e-04, forward_time=0.245, class_loss=1.394, geo_loss_downstream=0.180, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.114, loss=0.284, accuracy=0.943, backward_time=0.900, grad_norm=57.503, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.896e-06, train_time=4.639 +[gpue05] 2025-06-01 17:08:39,332 (trainer:816) INFO: 22epoch:train:1101-1200batch: iter_time=1.067e-04, forward_time=0.277, class_loss=1.179, geo_loss_downstream=0.178, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.113, loss=0.241, accuracy=0.953, backward_time=0.914, grad_norm=61.730, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.878e-06, train_time=4.827 +[gpue05] 2025-06-01 17:10:31,181 (trainer:816) INFO: 22epoch:train:1201-1300batch: iter_time=9.773e-05, forward_time=0.253, class_loss=1.186, geo_loss_downstream=0.180, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.114, loss=0.243, accuracy=0.947, backward_time=0.850, grad_norm=71.871, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.861e-06, train_time=4.473 +[gpue05] 2025-06-01 17:12:27,776 (trainer:816) INFO: 22epoch:train:1301-1400batch: iter_time=1.115e-04, forward_time=0.250, class_loss=1.180, geo_loss_downstream=0.178, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.113, loss=0.242, accuracy=0.953, backward_time=0.901, grad_norm=52.762, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.844e-06, train_time=4.663 +[gpue05] 2025-06-01 17:14:23,004 (trainer:816) INFO: 22epoch:train:1401-1500batch: iter_time=1.128e-04, forward_time=0.250, class_loss=1.604, geo_loss_downstream=0.179, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.113, loss=0.326, accuracy=0.940, backward_time=0.886, grad_norm=88.669, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.827e-06, train_time=4.608 +[gpue05] 2025-06-01 17:16:14,770 (trainer:816) INFO: 22epoch:train:1501-1600batch: iter_time=9.846e-05, forward_time=0.233, class_loss=1.165, geo_loss_downstream=0.177, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.112, loss=0.239, accuracy=0.952, backward_time=0.869, grad_norm=49.623, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.810e-06, train_time=4.470 +[gpue05] 2025-06-01 17:18:13,367 (trainer:816) INFO: 22epoch:train:1601-1700batch: iter_time=9.560e-05, forward_time=0.236, class_loss=1.432, geo_loss_downstream=0.180, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.114, loss=0.292, accuracy=0.937, backward_time=0.935, grad_norm=61.891, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.794e-06, train_time=4.743 +[gpue05] 2025-06-01 17:20:13,977 (trainer:816) INFO: 22epoch:train:1701-1800batch: iter_time=1.043e-04, forward_time=0.260, class_loss=1.604, geo_loss_downstream=0.178, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.113, loss=0.326, accuracy=0.933, backward_time=0.931, grad_norm=80.730, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.777e-06, train_time=4.824 +[gpue05] 2025-06-01 17:22:25,435 (trainer:816) INFO: 22epoch:train:1801-1900batch: iter_time=1.024e-04, forward_time=0.280, class_loss=1.485, geo_loss_downstream=0.179, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.113, loss=0.303, accuracy=0.937, backward_time=1.019, grad_norm=60.540, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.761e-06, train_time=5.258 +[gpue05] 2025-06-01 17:24:25,669 (trainer:816) INFO: 22epoch:train:1901-2000batch: iter_time=9.303e-05, forward_time=0.253, class_loss=1.385, geo_loss_downstream=0.178, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.112, loss=0.283, accuracy=0.942, backward_time=0.934, grad_norm=59.371, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.745e-06, train_time=4.809 +[gpue05] 2025-06-01 17:47:48,314 (trainer:401) INFO: 22epoch results: [train] iter_time=2.448e-04, forward_time=0.278, class_loss=1.350, geo_loss_downstream=0.180, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.114, loss=0.276, accuracy=0.945, backward_time=0.902, grad_norm=64.216, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.907e-06, train_time=4.783, time=42 minutes and 12.94 seconds, total_count=44000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.743, geo_loss_downstream=0.219, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.024, inter_geo_loss_layer40=0.024, inter_geo_loss_layer44=0.023, inter_geo_loss_mean=0.023, geo_loss_all=0.141, loss=2.223, accuracy=0.887, time=23 minutes and 22.4 seconds, total_count=103884, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 17:48:02,296 (trainer:467) INFO: There are no improvements in this epoch +[gpue05] 2025-06-01 17:48:02,308 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/21epoch.pth +[gpue05] 2025-06-01 17:48:02,309 (trainer:335) INFO: 23/50epoch started. Estimated time to finish: 1 day, 6 hours and 49 minutes +[gpue05] 2025-06-01 17:52:40,799 (trainer:816) INFO: 23epoch:train:1-100batch: iter_time=6.849e-04, forward_time=0.403, class_loss=1.779, geo_loss_downstream=0.179, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.016, inter_geo_loss_layer44=0.016, inter_geo_loss_mean=0.016, geo_loss_all=0.114, loss=0.362, accuracy=0.928, backward_time=0.945, grad_norm=73.739, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.729e-06, train_time=5.482 +[gpue05] 2025-06-01 17:54:46,750 (trainer:816) INFO: 23epoch:train:101-200batch: iter_time=1.038e-04, forward_time=0.368, class_loss=1.256, geo_loss_downstream=0.177, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.112, loss=0.257, accuracy=0.948, backward_time=0.876, grad_norm=46.298, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.713e-06, train_time=5.037 +[gpue05] 2025-06-01 17:56:59,001 (trainer:816) INFO: 23epoch:train:201-300batch: iter_time=9.348e-05, forward_time=0.356, class_loss=1.009, geo_loss_downstream=0.179, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.113, loss=0.207, accuracy=0.960, backward_time=0.951, grad_norm=45.906, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.697e-06, train_time=5.289 +[gpue05] 2025-06-01 17:59:12,982 (trainer:816) INFO: 23epoch:train:301-400batch: iter_time=1.028e-04, forward_time=0.336, class_loss=1.458, geo_loss_downstream=0.176, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.111, loss=0.297, accuracy=0.938, backward_time=0.990, grad_norm=82.373, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.682e-06, train_time=5.359 +[gpue05] 2025-06-01 18:01:13,136 (trainer:816) INFO: 23epoch:train:401-500batch: iter_time=1.105e-04, forward_time=0.294, class_loss=1.073, geo_loss_downstream=0.179, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.113, loss=0.220, accuracy=0.958, backward_time=0.893, grad_norm=54.864, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.666e-06, train_time=4.805 +[gpue05] 2025-06-01 18:03:20,164 (trainer:816) INFO: 23epoch:train:501-600batch: iter_time=1.110e-04, forward_time=0.302, class_loss=1.225, geo_loss_downstream=0.176, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.112, loss=0.251, accuracy=0.948, backward_time=0.953, grad_norm=62.207, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.651e-06, train_time=5.080 +[gpue05] 2025-06-01 18:05:09,176 (trainer:816) INFO: 23epoch:train:601-700batch: iter_time=1.183e-04, forward_time=0.262, class_loss=1.136, geo_loss_downstream=0.175, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.111, loss=0.233, accuracy=0.952, backward_time=0.811, grad_norm=45.442, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.636e-06, train_time=4.360 +[gpue05] 2025-06-01 18:07:12,816 (trainer:816) INFO: 23epoch:train:701-800batch: iter_time=1.033e-04, forward_time=0.264, class_loss=1.090, geo_loss_downstream=0.177, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.112, loss=0.224, accuracy=0.953, backward_time=0.956, grad_norm=52.278, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.621e-06, train_time=4.945 +[gpue05] 2025-06-01 18:08:55,421 (trainer:816) INFO: 23epoch:train:801-900batch: iter_time=1.028e-04, forward_time=0.239, class_loss=1.343, geo_loss_downstream=0.177, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.112, loss=0.274, accuracy=0.945, backward_time=0.770, grad_norm=72.926, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.606e-06, train_time=4.103 +[gpue05] 2025-06-01 18:10:48,524 (trainer:816) INFO: 23epoch:train:901-1000batch: iter_time=1.105e-04, forward_time=0.248, class_loss=1.184, geo_loss_downstream=0.177, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.112, loss=0.242, accuracy=0.957, backward_time=0.867, grad_norm=64.965, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.591e-06, train_time=4.523 +[gpue05] 2025-06-01 18:12:52,706 (trainer:816) INFO: 23epoch:train:1001-1100batch: iter_time=1.058e-04, forward_time=0.265, class_loss=1.121, geo_loss_downstream=0.175, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.111, loss=0.230, accuracy=0.945, backward_time=0.960, grad_norm=63.275, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.577e-06, train_time=4.967 +[gpue05] 2025-06-01 18:14:54,335 (trainer:816) INFO: 23epoch:train:1101-1200batch: iter_time=1.138e-04, forward_time=0.253, class_loss=1.266, geo_loss_downstream=0.176, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.111, loss=0.259, accuracy=0.953, backward_time=0.947, grad_norm=53.992, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.562e-06, train_time=4.864 +[gpue05] 2025-06-01 18:16:48,602 (trainer:816) INFO: 23epoch:train:1201-1300batch: iter_time=1.089e-04, forward_time=0.243, class_loss=1.768, geo_loss_downstream=0.176, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.111, loss=0.359, accuracy=0.925, backward_time=0.885, grad_norm=78.143, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.548e-06, train_time=4.570 +[gpue05] 2025-06-01 18:18:48,056 (trainer:816) INFO: 23epoch:train:1301-1400batch: iter_time=9.868e-05, forward_time=0.242, class_loss=1.496, geo_loss_downstream=0.176, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.016, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.112, loss=0.305, accuracy=0.935, backward_time=0.937, grad_norm=79.867, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.534e-06, train_time=4.777 +[gpue05] 2025-06-01 18:20:37,774 (trainer:816) INFO: 23epoch:train:1401-1500batch: iter_time=1.037e-04, forward_time=0.223, class_loss=0.905, geo_loss_downstream=0.177, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.112, loss=0.187, accuracy=0.965, backward_time=0.858, grad_norm=59.416, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.520e-06, train_time=4.388 +[gpue05] 2025-06-01 18:22:38,461 (trainer:816) INFO: 23epoch:train:1501-1600batch: iter_time=1.098e-04, forward_time=0.256, class_loss=1.268, geo_loss_downstream=0.174, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.110, loss=0.259, accuracy=0.958, backward_time=0.936, grad_norm=54.722, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.506e-06, train_time=4.827 +[gpue05] 2025-06-01 18:24:27,796 (trainer:816) INFO: 23epoch:train:1601-1700batch: iter_time=1.056e-04, forward_time=0.241, class_loss=1.029, geo_loss_downstream=0.174, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.110, loss=0.211, accuracy=0.962, backward_time=0.836, grad_norm=49.837, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.492e-06, train_time=4.373 +[gpue05] 2025-06-01 18:26:31,021 (trainer:816) INFO: 23epoch:train:1701-1800batch: iter_time=1.125e-04, forward_time=0.250, class_loss=1.438, geo_loss_downstream=0.175, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.110, loss=0.293, accuracy=0.935, backward_time=0.967, grad_norm=64.024, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.478e-06, train_time=4.928 +[gpue05] 2025-06-01 18:28:32,454 (trainer:816) INFO: 23epoch:train:1801-1900batch: iter_time=1.108e-04, forward_time=0.249, class_loss=1.393, geo_loss_downstream=0.175, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.111, loss=0.284, accuracy=0.945, backward_time=0.950, grad_norm=63.263, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.465e-06, train_time=4.857 +[gpue05] 2025-06-01 18:30:24,429 (trainer:816) INFO: 23epoch:train:1901-2000batch: iter_time=1.033e-04, forward_time=0.247, class_loss=1.181, geo_loss_downstream=0.175, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.111, loss=0.242, accuracy=0.955, backward_time=0.856, grad_norm=54.179, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.451e-06, train_time=4.478 +[gpue05] 2025-06-01 18:53:49,470 (trainer:401) INFO: 23epoch results: [train] iter_time=1.357e-04, forward_time=0.277, class_loss=1.271, geo_loss_downstream=0.176, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.112, loss=0.260, accuracy=0.948, backward_time=0.907, grad_norm=61.086, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.586e-06, train_time=4.801, time=42 minutes and 22.42 seconds, total_count=46000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.672, geo_loss_downstream=0.212, inter_geo_loss_layer32=0.022, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.136, loss=2.165, accuracy=0.890, time=23 minutes and 24.73 seconds, total_count=108606, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 18:54:03,935 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 18:54:03,947 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/22epoch.pth +[gpue05] 2025-06-01 18:54:03,948 (trainer:335) INFO: 24/50epoch started. Estimated time to finish: 1 day, 5 hours and 43 minutes +[gpue05] 2025-06-01 18:58:42,008 (trainer:816) INFO: 24epoch:train:1-100batch: iter_time=0.002, forward_time=0.402, class_loss=1.541, geo_loss_downstream=0.173, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.109, loss=0.314, accuracy=0.937, backward_time=0.938, grad_norm=80.542, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.011, optim0_lr0=1.438e-06, train_time=5.449 +[gpue05] 2025-06-01 19:00:56,001 (trainer:816) INFO: 24epoch:train:101-200batch: iter_time=1.160e-04, forward_time=0.379, class_loss=1.278, geo_loss_downstream=0.175, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.111, loss=0.261, accuracy=0.950, backward_time=0.946, grad_norm=50.869, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.425e-06, train_time=5.359 +[gpue05] 2025-06-01 19:02:55,023 (trainer:816) INFO: 24epoch:train:201-300batch: iter_time=1.057e-04, forward_time=0.338, class_loss=1.270, geo_loss_downstream=0.174, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.110, loss=0.260, accuracy=0.948, backward_time=0.836, grad_norm=75.693, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.412e-06, train_time=4.760 +[gpue05] 2025-06-01 19:04:54,625 (trainer:816) INFO: 24epoch:train:301-400batch: iter_time=1.252e-04, forward_time=0.310, class_loss=1.346, geo_loss_downstream=0.174, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.110, loss=0.275, accuracy=0.943, backward_time=0.871, grad_norm=67.897, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.399e-06, train_time=4.783 +[gpue05] 2025-06-01 19:07:10,258 (trainer:816) INFO: 24epoch:train:401-500batch: iter_time=1.376e-04, forward_time=0.331, class_loss=1.128, geo_loss_downstream=0.174, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.110, loss=0.231, accuracy=0.957, backward_time=1.010, grad_norm=49.178, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.386e-06, train_time=5.425 +[gpue05] 2025-06-01 19:09:08,101 (trainer:816) INFO: 24epoch:train:501-600batch: iter_time=1.061e-04, forward_time=0.278, class_loss=1.262, geo_loss_downstream=0.174, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.110, loss=0.258, accuracy=0.950, backward_time=0.884, grad_norm=62.022, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.373e-06, train_time=4.713 +[gpue05] 2025-06-01 19:11:10,236 (trainer:816) INFO: 24epoch:train:601-700batch: iter_time=1.056e-04, forward_time=0.290, class_loss=1.455, geo_loss_downstream=0.174, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.110, loss=0.296, accuracy=0.938, backward_time=0.914, grad_norm=75.102, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.361e-06, train_time=4.885 +[gpue05] 2025-06-01 19:13:10,051 (trainer:816) INFO: 24epoch:train:701-800batch: iter_time=1.054e-04, forward_time=0.271, class_loss=0.822, geo_loss_downstream=0.173, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.109, loss=0.170, accuracy=0.968, backward_time=0.911, grad_norm=49.439, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.011, optim0_lr0=1.348e-06, train_time=4.792 +[gpue05] 2025-06-01 19:15:16,813 (trainer:816) INFO: 24epoch:train:801-900batch: iter_time=1.156e-04, forward_time=0.271, class_loss=1.328, geo_loss_downstream=0.175, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.110, loss=0.271, accuracy=0.943, backward_time=0.981, grad_norm=42.333, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.336e-06, train_time=5.070 +[gpue05] 2025-06-01 19:17:08,950 (trainer:816) INFO: 24epoch:train:901-1000batch: iter_time=1.080e-04, forward_time=0.242, class_loss=1.106, geo_loss_downstream=0.173, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.109, loss=0.227, accuracy=0.955, backward_time=0.862, grad_norm=58.749, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.324e-06, train_time=4.485 +[gpue05] 2025-06-01 19:19:20,934 (trainer:816) INFO: 24epoch:train:1001-1100batch: iter_time=1.153e-04, forward_time=0.282, class_loss=1.552, geo_loss_downstream=0.176, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.111, loss=0.316, accuracy=0.937, backward_time=1.023, grad_norm=69.752, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.012, optim0_lr0=1.311e-06, train_time=5.279 +[gpue05] 2025-06-01 19:21:26,434 (trainer:816) INFO: 24epoch:train:1101-1200batch: iter_time=1.268e-04, forward_time=0.270, class_loss=1.467, geo_loss_downstream=0.172, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.109, loss=0.299, accuracy=0.940, backward_time=0.969, grad_norm=85.795, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.299e-06, train_time=5.019 +[gpue05] 2025-06-01 19:23:13,457 (trainer:816) INFO: 24epoch:train:1201-1300batch: iter_time=1.112e-04, forward_time=0.245, class_loss=1.236, geo_loss_downstream=0.176, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.111, loss=0.253, accuracy=0.947, backward_time=0.808, grad_norm=61.498, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.288e-06, train_time=4.280 +[gpue05] 2025-06-01 19:25:11,563 (trainer:816) INFO: 24epoch:train:1301-1400batch: iter_time=1.171e-04, forward_time=0.259, class_loss=0.907, geo_loss_downstream=0.173, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.110, loss=0.187, accuracy=0.960, backward_time=0.906, grad_norm=55.931, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.276e-06, train_time=4.724 +[gpue05] 2025-06-01 19:27:04,931 (trainer:816) INFO: 24epoch:train:1401-1500batch: iter_time=1.365e-04, forward_time=0.256, class_loss=1.122, geo_loss_downstream=0.173, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.110, loss=0.230, accuracy=0.953, backward_time=0.860, grad_norm=27.817, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.264e-06, train_time=4.534 +[gpue05] 2025-06-01 19:29:18,838 (trainer:816) INFO: 24epoch:train:1501-1600batch: iter_time=1.306e-04, forward_time=0.277, class_loss=1.406, geo_loss_downstream=0.172, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.109, loss=0.287, accuracy=0.945, backward_time=1.047, grad_norm=85.806, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.252e-06, train_time=5.356 +[gpue05] 2025-06-01 19:31:26,847 (trainer:816) INFO: 24epoch:train:1601-1700batch: iter_time=1.094e-04, forward_time=0.277, class_loss=1.659, geo_loss_downstream=0.173, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.109, loss=0.337, accuracy=0.931, backward_time=0.987, grad_norm=68.361, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.011, optim0_lr0=1.241e-06, train_time=5.120 +[gpue05] 2025-06-01 19:33:24,543 (trainer:816) INFO: 24epoch:train:1701-1800batch: iter_time=1.044e-04, forward_time=0.246, class_loss=1.147, geo_loss_downstream=0.172, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.109, loss=0.235, accuracy=0.950, backward_time=0.915, grad_norm=49.839, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.009, optim0_lr0=1.230e-06, train_time=4.707 +[gpue05] 2025-06-01 19:35:30,326 (trainer:816) INFO: 24epoch:train:1801-1900batch: iter_time=1.257e-04, forward_time=0.279, class_loss=1.241, geo_loss_downstream=0.170, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.107, loss=0.254, accuracy=0.950, backward_time=0.963, grad_norm=66.427, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.218e-06, train_time=5.031 +[gpue05] 2025-06-01 19:37:28,556 (trainer:816) INFO: 24epoch:train:1901-2000batch: iter_time=1.112e-04, forward_time=0.238, class_loss=1.224, geo_loss_downstream=0.172, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.109, loss=0.250, accuracy=0.952, backward_time=0.929, grad_norm=56.180, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.207e-06, train_time=4.728 +[gpue05] 2025-06-01 20:00:59,925 (trainer:401) INFO: 24epoch results: [train] iter_time=2.269e-04, forward_time=0.287, class_loss=1.275, geo_loss_downstream=0.173, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.110, loss=0.260, accuracy=0.948, backward_time=0.928, grad_norm=61.962, clip=0.000e+00, loss_scale=2.097e+06, optim_step_time=0.010, optim0_lr0=1.319e-06, train_time=4.925, time=43 minutes and 24.84 seconds, total_count=48000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.594, geo_loss_downstream=0.206, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.132, loss=2.102, accuracy=0.895, time=23 minutes and 31.13 seconds, total_count=113328, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 20:01:13,881 (trainer:469) INFO: The best model has been updated: valid.accuracy +[gpue05] 2025-06-01 20:01:13,897 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/20epoch.pth +[gpue05] 2025-06-01 20:01:13,897 (trainer:335) INFO: 25/50epoch started. Estimated time to finish: 1 day, 4 hours and 38 minutes +[gpue05] 2025-06-01 20:05:41,349 (trainer:816) INFO: 25epoch:train:1-100batch: iter_time=0.003, forward_time=0.409, class_loss=1.232, geo_loss_downstream=0.172, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.109, loss=0.252, accuracy=0.950, backward_time=0.822, grad_norm=47.071, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.196e-06, train_time=5.024 +[gpue05] 2025-06-01 20:07:44,800 (trainer:816) INFO: 25epoch:train:101-200batch: iter_time=1.031e-04, forward_time=0.363, class_loss=1.042, geo_loss_downstream=0.173, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.109, loss=0.214, accuracy=0.957, backward_time=0.855, grad_norm=38.648, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.185e-06, train_time=4.937 +[gpue05] 2025-06-01 20:09:58,527 (trainer:816) INFO: 25epoch:train:201-300batch: iter_time=9.621e-05, forward_time=0.345, class_loss=1.151, geo_loss_downstream=0.171, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.236, accuracy=0.958, backward_time=0.976, grad_norm=45.409, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.174e-06, train_time=5.348 +[gpue05] 2025-06-01 20:12:06,383 (trainer:816) INFO: 25epoch:train:301-400batch: iter_time=1.056e-04, forward_time=0.330, class_loss=1.077, geo_loss_downstream=0.172, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.109, loss=0.221, accuracy=0.963, backward_time=0.932, grad_norm=46.653, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.009, optim0_lr0=1.163e-06, train_time=5.114 +[gpue05] 2025-06-01 20:14:13,698 (trainer:816) INFO: 25epoch:train:401-500batch: iter_time=1.030e-04, forward_time=0.310, class_loss=1.029, geo_loss_downstream=0.171, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.108, loss=0.211, accuracy=0.962, backward_time=0.948, grad_norm=55.222, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.153e-06, train_time=5.092 +[gpue05] 2025-06-01 20:16:07,773 (trainer:816) INFO: 25epoch:train:501-600batch: iter_time=1.077e-04, forward_time=0.284, class_loss=1.061, geo_loss_downstream=0.171, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.218, accuracy=0.958, backward_time=0.839, grad_norm=60.218, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.142e-06, train_time=4.562 +[gpue05] 2025-06-01 20:18:02,609 (trainer:816) INFO: 25epoch:train:601-700batch: iter_time=1.005e-04, forward_time=0.280, class_loss=1.021, geo_loss_downstream=0.171, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.108, loss=0.210, accuracy=0.958, backward_time=0.852, grad_norm=45.171, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.132e-06, train_time=4.593 +[gpue05] 2025-06-01 20:20:08,497 (trainer:816) INFO: 25epoch:train:701-800batch: iter_time=1.004e-04, forward_time=0.296, class_loss=1.145, geo_loss_downstream=0.172, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.109, loss=0.235, accuracy=0.957, backward_time=0.948, grad_norm=53.336, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.121e-06, train_time=5.035 +[gpue05] 2025-06-01 20:22:16,440 (trainer:816) INFO: 25epoch:train:801-900batch: iter_time=1.023e-04, forward_time=0.265, class_loss=1.053, geo_loss_downstream=0.173, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.109, loss=0.216, accuracy=0.957, backward_time=1.000, grad_norm=58.079, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.111e-06, train_time=5.117 +[gpue05] 2025-06-01 20:24:16,518 (trainer:816) INFO: 25epoch:train:901-1000batch: iter_time=1.031e-04, forward_time=0.254, class_loss=1.127, geo_loss_downstream=0.171, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.109, loss=0.231, accuracy=0.958, backward_time=0.931, grad_norm=60.299, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.101e-06, train_time=4.802 +[gpue05] 2025-06-01 20:26:15,077 (trainer:816) INFO: 25epoch:train:1001-1100batch: iter_time=1.200e-04, forward_time=0.291, class_loss=1.107, geo_loss_downstream=0.171, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.227, accuracy=0.953, backward_time=0.873, grad_norm=47.653, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.025, optim0_lr0=1.091e-06, train_time=4.742 +[gpue05] 2025-06-01 20:28:18,105 (trainer:816) INFO: 25epoch:train:1101-1200batch: iter_time=1.551e-04, forward_time=0.267, class_loss=1.382, geo_loss_downstream=0.171, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.108, loss=0.282, accuracy=0.945, backward_time=0.947, grad_norm=62.848, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.081e-06, train_time=4.920 +[gpue05] 2025-06-01 20:30:22,440 (trainer:816) INFO: 25epoch:train:1201-1300batch: iter_time=1.561e-04, forward_time=0.277, class_loss=1.172, geo_loss_downstream=0.170, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.240, accuracy=0.948, backward_time=0.950, grad_norm=56.899, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=1.071e-06, train_time=4.973 +[gpue05] 2025-06-01 20:32:24,108 (trainer:816) INFO: 25epoch:train:1301-1400batch: iter_time=1.454e-04, forward_time=0.274, class_loss=1.005, geo_loss_downstream=0.170, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.206, accuracy=0.953, backward_time=0.924, grad_norm=52.490, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.061e-06, train_time=4.866 +[gpue05] 2025-06-01 20:34:33,876 (trainer:816) INFO: 25epoch:train:1401-1500batch: iter_time=1.618e-04, forward_time=0.279, class_loss=1.372, geo_loss_downstream=0.170, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.280, accuracy=0.946, backward_time=1.003, grad_norm=85.939, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=1.051e-06, train_time=5.190 +[gpue05] 2025-06-01 20:36:29,065 (trainer:816) INFO: 25epoch:train:1501-1600batch: iter_time=1.575e-04, forward_time=0.230, class_loss=1.404, geo_loss_downstream=0.171, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.286, accuracy=0.938, backward_time=0.905, grad_norm=63.676, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.042e-06, train_time=4.607 +[gpue05] 2025-06-01 20:38:30,807 (trainer:816) INFO: 25epoch:train:1601-1700batch: iter_time=1.510e-04, forward_time=0.254, class_loss=1.233, geo_loss_downstream=0.170, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.252, accuracy=0.950, backward_time=0.948, grad_norm=54.208, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=1.032e-06, train_time=4.869 +[gpue05] 2025-06-01 20:40:32,916 (trainer:816) INFO: 25epoch:train:1701-1800batch: iter_time=1.462e-04, forward_time=0.259, class_loss=0.905, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.106, loss=0.186, accuracy=0.960, backward_time=0.944, grad_norm=47.485, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=1.023e-06, train_time=4.884 +[gpue05] 2025-06-01 20:42:29,840 (trainer:816) INFO: 25epoch:train:1801-1900batch: iter_time=1.611e-04, forward_time=0.246, class_loss=1.286, geo_loss_downstream=0.170, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.263, accuracy=0.947, backward_time=0.907, grad_norm=65.606, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=1.013e-06, train_time=4.676 +[gpue05] 2025-06-01 20:44:43,965 (trainer:816) INFO: 25epoch:train:1901-2000batch: iter_time=1.469e-04, forward_time=0.271, class_loss=1.186, geo_loss_downstream=0.170, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.108, loss=0.243, accuracy=0.953, backward_time=1.055, grad_norm=55.769, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=1.004e-06, train_time=5.364 +[gpue05] 2025-06-01 21:08:24,010 (trainer:401) INFO: 25epoch results: [train] iter_time=2.704e-04, forward_time=0.289, class_loss=1.149, geo_loss_downstream=0.171, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.235, accuracy=0.954, backward_time=0.928, grad_norm=55.134, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=1.097e-06, train_time=4.936, time=43 minutes and 30.35 seconds, total_count=50000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.617, geo_loss_downstream=0.211, inter_geo_loss_layer32=0.021, inter_geo_loss_layer36=0.023, inter_geo_loss_layer40=0.023, inter_geo_loss_layer44=0.022, inter_geo_loss_mean=0.022, geo_loss_all=0.136, loss=2.120, accuracy=0.892, time=23 minutes and 39.76 seconds, total_count=118050, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 21:08:38,051 (trainer:467) INFO: There are no improvements in this epoch +[gpue05] 2025-06-01 21:08:38,068 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/23epoch.pth +[gpue05] 2025-06-01 21:08:38,068 (trainer:335) INFO: 26/50epoch started. Estimated time to finish: 1 day, 3 hours and 33 minutes +[gpue05] 2025-06-01 21:13:13,466 (trainer:816) INFO: 26epoch:train:1-100batch: iter_time=0.034, forward_time=0.417, class_loss=1.608, geo_loss_downstream=0.172, inter_geo_loss_layer32=0.016, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.015, inter_geo_loss_mean=0.015, geo_loss_all=0.109, loss=0.327, accuracy=0.928, backward_time=0.873, grad_norm=67.754, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=9.949e-07, train_time=5.385 +[gpue05] 2025-06-01 21:15:26,167 (trainer:816) INFO: 26epoch:train:101-200batch: iter_time=1.398e-04, forward_time=0.399, class_loss=1.095, geo_loss_downstream=0.171, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.224, accuracy=0.958, backward_time=0.911, grad_norm=46.591, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=9.857e-07, train_time=5.307 +[gpue05] 2025-06-01 21:17:35,395 (trainer:816) INFO: 26epoch:train:201-300batch: iter_time=1.146e-04, forward_time=0.355, class_loss=1.206, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.247, accuracy=0.947, backward_time=0.922, grad_norm=70.092, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=9.767e-07, train_time=5.168 +[gpue05] 2025-06-01 21:19:59,041 (trainer:816) INFO: 26epoch:train:301-400batch: iter_time=1.334e-04, forward_time=0.365, class_loss=1.212, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.248, accuracy=0.950, backward_time=1.057, grad_norm=68.907, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=9.677e-07, train_time=5.745 +[gpue05] 2025-06-01 21:22:09,569 (trainer:816) INFO: 26epoch:train:401-500batch: iter_time=1.348e-04, forward_time=0.321, class_loss=1.374, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.280, accuracy=0.942, backward_time=0.968, grad_norm=55.736, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=9.589e-07, train_time=5.220 +[gpue05] 2025-06-01 21:24:16,101 (trainer:816) INFO: 26epoch:train:501-600batch: iter_time=1.264e-04, forward_time=0.294, class_loss=1.316, geo_loss_downstream=0.170, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.269, accuracy=0.942, backward_time=0.956, grad_norm=65.583, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=9.501e-07, train_time=5.061 +[gpue05] 2025-06-01 21:26:26,655 (trainer:816) INFO: 26epoch:train:601-700batch: iter_time=1.084e-04, forward_time=0.350, class_loss=1.405, geo_loss_downstream=0.171, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.108, loss=0.286, accuracy=0.945, backward_time=0.940, grad_norm=55.622, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=9.414e-07, train_time=5.221 +[gpue05] 2025-06-01 21:28:31,396 (trainer:816) INFO: 26epoch:train:701-800batch: iter_time=1.720e-04, forward_time=0.276, class_loss=1.131, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.231, accuracy=0.960, backward_time=0.955, grad_norm=68.246, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=9.327e-07, train_time=4.989 +[gpue05] 2025-06-01 21:30:22,848 (trainer:816) INFO: 26epoch:train:801-900batch: iter_time=1.631e-04, forward_time=0.251, class_loss=1.527, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.311, accuracy=0.940, backward_time=0.846, grad_norm=62.673, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=9.242e-07, train_time=4.457 +[gpue05] 2025-06-01 21:32:20,803 (trainer:816) INFO: 26epoch:train:901-1000batch: iter_time=1.653e-04, forward_time=0.273, class_loss=1.227, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.107, loss=0.251, accuracy=0.950, backward_time=0.891, grad_norm=64.613, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=9.157e-07, train_time=4.717 +[gpue05] 2025-06-01 21:34:19,949 (trainer:816) INFO: 26epoch:train:1001-1100batch: iter_time=1.580e-04, forward_time=0.274, class_loss=1.181, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.242, accuracy=0.953, backward_time=0.901, grad_norm=48.057, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=9.073e-07, train_time=4.765 +[gpue05] 2025-06-01 21:36:25,152 (trainer:816) INFO: 26epoch:train:1101-1200batch: iter_time=1.670e-04, forward_time=0.261, class_loss=1.047, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.215, accuracy=0.955, backward_time=0.976, grad_norm=58.977, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=8.990e-07, train_time=5.007 +[gpue05] 2025-06-01 21:38:23,739 (trainer:816) INFO: 26epoch:train:1201-1300batch: iter_time=1.598e-04, forward_time=0.258, class_loss=1.573, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.320, accuracy=0.933, backward_time=0.911, grad_norm=82.680, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=8.908e-07, train_time=4.743 +[gpue05] 2025-06-01 21:40:19,716 (trainer:816) INFO: 26epoch:train:1301-1400batch: iter_time=1.377e-04, forward_time=0.236, class_loss=1.183, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.242, accuracy=0.958, backward_time=0.909, grad_norm=52.755, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=8.826e-07, train_time=4.638 +[gpue05] 2025-06-01 21:42:08,073 (trainer:816) INFO: 26epoch:train:1401-1500batch: iter_time=9.788e-05, forward_time=0.238, class_loss=1.140, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.106, loss=0.233, accuracy=0.950, backward_time=0.831, grad_norm=49.054, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=8.745e-07, train_time=4.334 +[gpue05] 2025-06-01 21:44:22,982 (trainer:816) INFO: 26epoch:train:1501-1600batch: iter_time=1.173e-04, forward_time=0.298, class_loss=1.288, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.106, loss=0.263, accuracy=0.950, backward_time=1.037, grad_norm=56.145, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=8.665e-07, train_time=5.396 +[gpue05] 2025-06-01 21:46:21,203 (trainer:816) INFO: 26epoch:train:1601-1700batch: iter_time=1.066e-04, forward_time=0.257, class_loss=1.352, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.276, accuracy=0.942, backward_time=0.909, grad_norm=77.199, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=8.585e-07, train_time=4.728 +[gpue05] 2025-06-01 21:48:23,272 (trainer:816) INFO: 26epoch:train:1701-1800batch: iter_time=1.119e-04, forward_time=0.258, class_loss=1.318, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.269, accuracy=0.943, backward_time=0.946, grad_norm=67.036, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=8.507e-07, train_time=4.882 +[gpue05] 2025-06-01 21:50:40,857 (trainer:816) INFO: 26epoch:train:1801-1900batch: iter_time=1.167e-04, forward_time=0.288, class_loss=1.342, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.106, loss=0.274, accuracy=0.945, backward_time=1.073, grad_norm=69.248, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=8.429e-07, train_time=5.503 +[gpue05] 2025-06-01 21:52:49,010 (trainer:816) INFO: 26epoch:train:1901-2000batch: iter_time=1.081e-04, forward_time=0.289, class_loss=1.094, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.224, accuracy=0.957, backward_time=0.977, grad_norm=52.852, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=8.351e-07, train_time=5.125 +[gpue05] 2025-06-01 22:16:14,111 (trainer:401) INFO: 26epoch results: [train] iter_time=0.002, forward_time=0.298, class_loss=1.281, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.262, accuracy=0.947, backward_time=0.939, grad_norm=61.991, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=9.128e-07, train_time=5.020, time=44 minutes and 11.17 seconds, total_count=52000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.652, geo_loss_downstream=0.209, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.023, inter_geo_loss_mean=0.022, geo_loss_all=0.134, loss=2.148, accuracy=0.890, time=23 minutes and 24.86 seconds, total_count=122772, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 22:16:28,263 (trainer:467) INFO: There are no improvements in this epoch +[gpue05] 2025-06-01 22:16:28,279 (trainer:335) INFO: 27/50epoch started. Estimated time to finish: 1 day, 2 hours and 29 minutes +[gpue05] 2025-06-01 22:21:02,199 (trainer:816) INFO: 27epoch:train:1-100batch: iter_time=0.005, forward_time=0.411, class_loss=1.130, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.106, loss=0.231, accuracy=0.957, backward_time=0.901, grad_norm=48.414, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=8.275e-07, train_time=5.350 +[gpue05] 2025-06-01 22:23:19,080 (trainer:816) INFO: 27epoch:train:101-200batch: iter_time=1.198e-04, forward_time=0.371, class_loss=1.189, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.243, accuracy=0.955, backward_time=0.983, grad_norm=73.052, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=8.199e-07, train_time=5.474 +[gpue05] 2025-06-01 22:25:19,313 (trainer:816) INFO: 27epoch:train:201-300batch: iter_time=1.119e-04, forward_time=0.327, class_loss=1.442, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.294, accuracy=0.937, backward_time=0.859, grad_norm=68.429, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=8.124e-07, train_time=4.809 +[gpue05] 2025-06-01 22:27:06,262 (trainer:816) INFO: 27epoch:train:301-400batch: iter_time=1.298e-04, forward_time=0.291, class_loss=1.452, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.105, loss=0.296, accuracy=0.937, backward_time=0.761, grad_norm=60.558, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=8.049e-07, train_time=4.277 +[gpue05] 2025-06-01 22:29:13,760 (trainer:816) INFO: 27epoch:train:401-500batch: iter_time=1.185e-04, forward_time=0.303, class_loss=0.911, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.187, accuracy=0.970, backward_time=0.957, grad_norm=47.270, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=7.976e-07, train_time=5.099 +[gpue05] 2025-06-01 22:31:21,042 (trainer:816) INFO: 27epoch:train:501-600batch: iter_time=1.230e-04, forward_time=0.301, class_loss=1.331, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.272, accuracy=0.947, backward_time=0.955, grad_norm=55.049, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=7.902e-07, train_time=5.090 +[gpue05] 2025-06-01 22:33:22,208 (trainer:816) INFO: 27epoch:train:601-700batch: iter_time=1.276e-04, forward_time=0.274, class_loss=1.096, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.015, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.107, loss=0.225, accuracy=0.958, backward_time=0.922, grad_norm=68.600, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=7.830e-07, train_time=4.846 +[gpue05] 2025-06-01 22:35:23,939 (trainer:816) INFO: 27epoch:train:701-800batch: iter_time=1.209e-04, forward_time=0.306, class_loss=1.237, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.253, accuracy=0.947, backward_time=0.894, grad_norm=43.230, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=7.758e-07, train_time=4.868 +[gpue05] 2025-06-01 22:37:25,119 (trainer:816) INFO: 27epoch:train:801-900batch: iter_time=1.223e-04, forward_time=0.262, class_loss=1.050, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.215, accuracy=0.958, backward_time=0.933, grad_norm=73.696, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=7.687e-07, train_time=4.846 +[gpue05] 2025-06-01 22:39:34,353 (trainer:816) INFO: 27epoch:train:901-1000batch: iter_time=1.364e-04, forward_time=0.293, class_loss=1.185, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.105, loss=0.242, accuracy=0.958, backward_time=0.983, grad_norm=64.179, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=7.617e-07, train_time=5.169 +[gpue05] 2025-06-01 22:41:27,218 (trainer:816) INFO: 27epoch:train:1001-1100batch: iter_time=1.165e-04, forward_time=0.251, class_loss=1.165, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.105, loss=0.238, accuracy=0.945, backward_time=0.861, grad_norm=51.214, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=7.547e-07, train_time=4.514 +[gpue05] 2025-06-01 22:43:23,734 (trainer:816) INFO: 27epoch:train:1101-1200batch: iter_time=1.185e-04, forward_time=0.266, class_loss=1.350, geo_loss_downstream=0.169, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.107, loss=0.275, accuracy=0.942, backward_time=0.882, grad_norm=55.894, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=7.478e-07, train_time=4.660 +[gpue05] 2025-06-01 22:45:32,973 (trainer:816) INFO: 27epoch:train:1201-1300batch: iter_time=1.129e-04, forward_time=0.295, class_loss=1.039, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.213, accuracy=0.960, backward_time=0.981, grad_norm=59.198, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=7.409e-07, train_time=5.169 +[gpue05] 2025-06-01 22:47:34,545 (trainer:816) INFO: 27epoch:train:1301-1400batch: iter_time=1.130e-04, forward_time=0.294, class_loss=1.362, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.278, accuracy=0.943, backward_time=0.905, grad_norm=74.231, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=7.341e-07, train_time=4.862 +[gpue05] 2025-06-01 22:49:23,784 (trainer:816) INFO: 27epoch:train:1401-1500batch: iter_time=1.074e-04, forward_time=0.262, class_loss=1.072, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.220, accuracy=0.953, backward_time=0.813, grad_norm=44.766, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=7.274e-07, train_time=4.369 +[gpue05] 2025-06-01 22:51:14,881 (trainer:816) INFO: 27epoch:train:1501-1600batch: iter_time=1.187e-04, forward_time=0.270, class_loss=1.072, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.220, accuracy=0.957, backward_time=0.825, grad_norm=51.742, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=7.207e-07, train_time=4.443 +[gpue05] 2025-06-01 22:53:06,722 (trainer:816) INFO: 27epoch:train:1601-1700batch: iter_time=1.135e-04, forward_time=0.261, class_loss=1.498, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.106, loss=0.305, accuracy=0.935, backward_time=0.840, grad_norm=54.605, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=7.141e-07, train_time=4.473 +[gpue05] 2025-06-01 22:54:51,228 (trainer:816) INFO: 27epoch:train:1701-1800batch: iter_time=1.230e-04, forward_time=0.231, class_loss=1.034, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.212, accuracy=0.962, backward_time=0.796, grad_norm=74.533, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=7.076e-07, train_time=4.179 +[gpue05] 2025-06-01 22:56:40,128 (trainer:816) INFO: 27epoch:train:1801-1900batch: iter_time=1.232e-04, forward_time=0.235, class_loss=1.469, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.299, accuracy=0.943, backward_time=0.838, grad_norm=70.892, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=7.011e-07, train_time=4.355 +[gpue05] 2025-06-01 22:58:46,034 (trainer:816) INFO: 27epoch:train:1901-2000batch: iter_time=1.180e-04, forward_time=0.281, class_loss=1.157, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.237, accuracy=0.957, backward_time=0.963, grad_norm=44.000, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=6.946e-07, train_time=5.035 +[gpue05] 2025-06-01 23:22:14,887 (trainer:401) INFO: 27epoch results: [train] iter_time=3.753e-04, forward_time=0.289, class_loss=1.212, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.248, accuracy=0.951, backward_time=0.893, grad_norm=59.178, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=7.592e-07, train_time=4.794, time=42 minutes and 18.04 seconds, total_count=54000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.588, geo_loss_downstream=0.210, inter_geo_loss_layer32=0.020, inter_geo_loss_layer36=0.022, inter_geo_loss_layer40=0.022, inter_geo_loss_layer44=0.021, inter_geo_loss_mean=0.021, geo_loss_all=0.135, loss=2.097, accuracy=0.894, time=23 minutes and 28.56 seconds, total_count=127494, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-01 23:22:28,906 (trainer:467) INFO: There are no improvements in this epoch +[gpue05] 2025-06-01 23:22:28,928 (trainer:523) INFO: The model files were removed: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/25epoch.pth, exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_raw/26epoch.pth +[gpue05] 2025-06-01 23:22:28,929 (trainer:335) INFO: 28/50epoch started. Estimated time to finish: 1 day, 1 hour and 22 minutes +[gpue05] 2025-06-01 23:26:57,540 (trainer:816) INFO: 28epoch:train:1-100batch: iter_time=0.002, forward_time=0.435, class_loss=1.125, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.230, accuracy=0.958, backward_time=0.825, grad_norm=60.938, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.012, optim0_lr0=6.883e-07, train_time=5.132 +[gpue05] 2025-06-01 23:29:03,447 (trainer:816) INFO: 28epoch:train:101-200batch: iter_time=1.170e-04, forward_time=0.390, class_loss=1.267, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.259, accuracy=0.950, backward_time=0.852, grad_norm=64.084, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=6.820e-07, train_time=5.035 +[gpue05] 2025-06-01 23:31:25,656 (trainer:816) INFO: 28epoch:train:201-300batch: iter_time=1.350e-04, forward_time=0.393, class_loss=1.204, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.246, accuracy=0.953, backward_time=1.014, grad_norm=70.260, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.012, optim0_lr0=6.757e-07, train_time=5.688 +[gpue05] 2025-06-01 23:33:37,025 (trainer:816) INFO: 28epoch:train:301-400batch: iter_time=1.239e-04, forward_time=0.365, class_loss=1.009, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.207, accuracy=0.953, backward_time=0.932, grad_norm=70.884, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.012, optim0_lr0=6.695e-07, train_time=5.254 +[gpue05] 2025-06-01 23:35:29,743 (trainer:816) INFO: 28epoch:train:401-500batch: iter_time=1.157e-04, forward_time=0.318, class_loss=1.045, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.214, accuracy=0.957, backward_time=0.792, grad_norm=44.688, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=6.634e-07, train_time=4.508 +[gpue05] 2025-06-01 23:37:39,746 (trainer:816) INFO: 28epoch:train:501-600batch: iter_time=1.212e-04, forward_time=0.328, class_loss=1.361, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.277, accuracy=0.942, backward_time=0.957, grad_norm=81.026, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=6.573e-07, train_time=5.199 +[gpue05] 2025-06-01 23:39:42,093 (trainer:816) INFO: 28epoch:train:601-700batch: iter_time=1.154e-04, forward_time=0.316, class_loss=0.993, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.105, loss=0.204, accuracy=0.963, backward_time=0.892, grad_norm=41.351, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=6.513e-07, train_time=4.893 +[gpue05] 2025-06-01 23:41:40,218 (trainer:816) INFO: 28epoch:train:701-800batch: iter_time=1.149e-04, forward_time=0.272, class_loss=1.148, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.235, accuracy=0.955, backward_time=0.893, grad_norm=47.948, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.011, optim0_lr0=6.453e-07, train_time=4.724 +[gpue05] 2025-06-01 23:43:47,795 (trainer:816) INFO: 28epoch:train:801-900batch: iter_time=1.240e-04, forward_time=0.292, class_loss=1.390, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.105, loss=0.283, accuracy=0.941, backward_time=0.969, grad_norm=61.755, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=6.394e-07, train_time=5.102 +[gpue05] 2025-06-01 23:45:46,338 (trainer:816) INFO: 28epoch:train:901-1000batch: iter_time=1.165e-04, forward_time=0.257, class_loss=1.285, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.015, geo_loss_all=0.106, loss=0.262, accuracy=0.945, backward_time=0.913, grad_norm=60.256, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=6.335e-07, train_time=4.741 +[gpue05] 2025-06-01 23:47:51,812 (trainer:816) INFO: 28epoch:train:1001-1100batch: iter_time=1.217e-04, forward_time=0.253, class_loss=1.163, geo_loss_downstream=0.168, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.238, accuracy=0.948, backward_time=0.987, grad_norm=67.737, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=6.277e-07, train_time=5.018 +[gpue05] 2025-06-01 23:49:49,241 (trainer:816) INFO: 28epoch:train:1101-1200batch: iter_time=1.209e-04, forward_time=0.270, class_loss=1.454, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.296, accuracy=0.935, backward_time=0.888, grad_norm=48.631, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=6.220e-07, train_time=4.696 +[gpue05] 2025-06-01 23:51:49,439 (trainer:816) INFO: 28epoch:train:1201-1300batch: iter_time=1.183e-04, forward_time=0.259, class_loss=1.120, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.229, accuracy=0.952, backward_time=0.926, grad_norm=49.451, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=6.163e-07, train_time=4.807 +[gpue05] 2025-06-01 23:53:37,474 (trainer:816) INFO: 28epoch:train:1301-1400batch: iter_time=1.173e-04, forward_time=0.242, class_loss=1.108, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.227, accuracy=0.953, backward_time=0.822, grad_norm=57.233, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=6.106e-07, train_time=4.321 +[gpue05] 2025-06-01 23:55:45,524 (trainer:816) INFO: 28epoch:train:1401-1500batch: iter_time=1.188e-04, forward_time=0.254, class_loss=0.969, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.105, loss=0.199, accuracy=0.962, backward_time=1.011, grad_norm=70.811, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=6.050e-07, train_time=5.121 +[gpue05] 2025-06-01 23:57:55,264 (trainer:816) INFO: 28epoch:train:1501-1600batch: iter_time=1.255e-04, forward_time=0.282, class_loss=0.991, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.105, loss=0.203, accuracy=0.958, backward_time=0.999, grad_norm=48.682, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=5.995e-07, train_time=5.189 +[gpue05] 2025-06-01 23:59:59,675 (trainer:816) INFO: 28epoch:train:1601-1700batch: iter_time=1.173e-04, forward_time=0.262, class_loss=1.113, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.105, loss=0.228, accuracy=0.955, backward_time=0.967, grad_norm=50.583, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=5.940e-07, train_time=4.976 +[gpue05] 2025-06-02 00:01:58,300 (trainer:816) INFO: 28epoch:train:1701-1800batch: iter_time=1.260e-04, forward_time=0.252, class_loss=0.975, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.200, accuracy=0.960, backward_time=0.918, grad_norm=60.931, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=5.885e-07, train_time=4.744 +[gpue05] 2025-06-02 00:04:00,760 (trainer:816) INFO: 28epoch:train:1801-1900batch: iter_time=1.141e-04, forward_time=0.251, class_loss=1.399, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.285, accuracy=0.945, backward_time=0.957, grad_norm=47.189, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=5.831e-07, train_time=4.898 +[gpue05] 2025-06-02 00:05:53,330 (trainer:816) INFO: 28epoch:train:1901-2000batch: iter_time=1.119e-04, forward_time=0.236, class_loss=1.328, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.271, accuracy=0.948, backward_time=0.874, grad_norm=58.546, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=5.778e-07, train_time=4.502 +[gpue05] 2025-06-02 00:29:20,672 (trainer:401) INFO: 28epoch results: [train] iter_time=1.956e-04, forward_time=0.296, class_loss=1.172, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.240, accuracy=0.952, backward_time=0.919, grad_norm=58.149, clip=0.000e+00, loss_scale=4.194e+06, optim_step_time=0.010, optim0_lr0=6.315e-07, train_time=4.927, time=43 minutes and 24.68 seconds, total_count=56000, gpu_max_cached_mem_GB=137.758, [valid] class_loss=2.609, geo_loss_downstream=0.200, inter_geo_loss_layer32=0.019, inter_geo_loss_layer36=0.021, inter_geo_loss_layer40=0.020, inter_geo_loss_layer44=0.020, inter_geo_loss_mean=0.020, geo_loss_all=0.128, loss=2.113, accuracy=0.894, time=23 minutes and 27.06 seconds, total_count=132216, gpu_max_cached_mem_GB=137.758 +[gpue05] 2025-06-02 00:29:34,724 (trainer:467) INFO: There are no improvements in this epoch +[gpue05] 2025-06-02 00:29:34,741 (trainer:335) INFO: 29/50epoch started. Estimated time to finish: 1 day, 17 minutes and 14.03 seconds +[gpue05] 2025-06-02 00:34:19,871 (trainer:816) INFO: 29epoch:train:1-100batch: iter_time=0.002, forward_time=0.418, class_loss=1.274, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.260, accuracy=0.953, backward_time=0.996, grad_norm=65.526, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.011, optim0_lr0=5.725e-07, train_time=5.747 +[gpue05] 2025-06-02 00:36:28,468 (trainer:816) INFO: 29epoch:train:101-200batch: iter_time=1.279e-04, forward_time=0.374, class_loss=1.608, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.327, accuracy=0.932, backward_time=0.891, grad_norm=64.421, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.030, optim0_lr0=5.672e-07, train_time=5.143 +[gpue05] 2025-06-02 00:38:36,305 (trainer:816) INFO: 29epoch:train:201-300batch: iter_time=1.120e-04, forward_time=0.354, class_loss=1.173, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.240, accuracy=0.958, backward_time=0.908, grad_norm=44.881, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.011, optim0_lr0=5.620e-07, train_time=5.113 +[gpue05] 2025-06-02 00:40:48,715 (trainer:816) INFO: 29epoch:train:301-400batch: iter_time=1.227e-04, forward_time=0.365, class_loss=1.148, geo_loss_downstream=0.167, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.106, loss=0.235, accuracy=0.957, backward_time=0.943, grad_norm=83.514, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.010, optim0_lr0=5.569e-07, train_time=5.296 +[gpue05] 2025-06-02 00:42:38,394 (trainer:816) INFO: 29epoch:train:401-500batch: iter_time=1.152e-04, forward_time=0.286, class_loss=1.517, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.015, inter_geo_loss_layer36=0.015, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.014, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.309, accuracy=0.945, backward_time=0.793, grad_norm=55.713, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.010, optim0_lr0=5.518e-07, train_time=4.386 +[gpue05] 2025-06-02 00:44:29,748 (trainer:816) INFO: 29epoch:train:501-600batch: iter_time=1.149e-04, forward_time=0.284, class_loss=1.212, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.248, accuracy=0.950, backward_time=0.813, grad_norm=59.296, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.010, optim0_lr0=5.467e-07, train_time=4.453 +[gpue05] 2025-06-02 00:46:32,067 (trainer:816) INFO: 29epoch:train:601-700batch: iter_time=1.526e-04, forward_time=0.295, class_loss=1.219, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.249, accuracy=0.947, backward_time=0.912, grad_norm=66.957, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.011, optim0_lr0=5.417e-07, train_time=4.892 +[gpue05] 2025-06-02 00:48:39,427 (trainer:816) INFO: 29epoch:train:701-800batch: iter_time=1.169e-04, forward_time=0.300, class_loss=1.067, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.219, accuracy=0.953, backward_time=0.958, grad_norm=49.440, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.010, optim0_lr0=5.367e-07, train_time=5.094 +[gpue05] 2025-06-02 00:50:41,943 (trainer:816) INFO: 29epoch:train:801-900batch: iter_time=1.242e-04, forward_time=0.275, class_loss=0.948, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.195, accuracy=0.953, backward_time=0.935, grad_norm=53.450, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.010, optim0_lr0=5.318e-07, train_time=4.900 +[gpue05] 2025-06-02 00:52:42,131 (trainer:816) INFO: 29epoch:train:901-1000batch: iter_time=1.168e-04, forward_time=0.264, class_loss=1.318, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.269, accuracy=0.945, backward_time=0.920, grad_norm=57.451, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.010, optim0_lr0=5.269e-07, train_time=4.807 +[gpue05] 2025-06-02 00:54:51,296 (trainer:816) INFO: 29epoch:train:1001-1100batch: iter_time=1.350e-04, forward_time=0.268, class_loss=1.131, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.104, loss=0.231, accuracy=0.955, backward_time=1.009, grad_norm=53.761, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.011, optim0_lr0=5.221e-07, train_time=5.166 +[gpue05] 2025-06-02 00:56:49,200 (trainer:816) INFO: 29epoch:train:1101-1200batch: iter_time=1.218e-04, forward_time=0.274, class_loss=0.888, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.183, accuracy=0.967, backward_time=0.889, grad_norm=45.040, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.011, optim0_lr0=5.173e-07, train_time=4.715 +[gpue05] 2025-06-02 00:58:51,070 (trainer:816) INFO: 29epoch:train:1201-1300batch: iter_time=1.192e-04, forward_time=0.276, class_loss=0.957, geo_loss_downstream=0.164, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.197, accuracy=0.963, backward_time=0.927, grad_norm=57.245, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.010, optim0_lr0=5.126e-07, train_time=4.874 +[gpue05] 2025-06-02 01:00:55,832 (trainer:816) INFO: 29epoch:train:1301-1400batch: iter_time=1.290e-04, forward_time=0.291, class_loss=0.841, geo_loss_downstream=0.163, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.103, loss=0.173, accuracy=0.968, backward_time=0.941, grad_norm=58.849, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.010, optim0_lr0=5.079e-07, train_time=4.990 +[gpue05] 2025-06-02 01:03:03,504 (trainer:816) INFO: 29epoch:train:1401-1500batch: iter_time=1.440e-04, forward_time=0.287, class_loss=1.401, geo_loss_downstream=0.166, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.014, inter_geo_loss_layer40=0.014, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.014, geo_loss_all=0.105, loss=0.285, accuracy=0.932, backward_time=0.974, grad_norm=61.210, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.012, optim0_lr0=5.032e-07, train_time=5.106 +[gpue05] 2025-06-02 01:05:03,400 (trainer:816) INFO: 29epoch:train:1501-1600batch: iter_time=1.333e-04, forward_time=0.270, class_loss=1.157, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.013, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.012, inter_geo_loss_layer44=0.012, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.237, accuracy=0.955, backward_time=0.913, grad_norm=47.130, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.011, optim0_lr0=4.986e-07, train_time=4.795 +[gpue05] 2025-06-02 01:06:57,011 (trainer:816) INFO: 29epoch:train:1601-1700batch: iter_time=1.110e-04, forward_time=0.275, class_loss=1.265, geo_loss_downstream=0.165, inter_geo_loss_layer32=0.014, inter_geo_loss_layer36=0.013, inter_geo_loss_layer40=0.013, inter_geo_loss_layer44=0.013, inter_geo_loss_mean=0.013, geo_loss_all=0.104, loss=0.258, accuracy=0.948, backward_time=0.844, grad_norm=54.364, clip=0.000e+00, loss_scale=8.389e+06, optim_step_time=0.011, optim0_lr0=4.940e-07, train_time=4.544 +Traceback (most recent call last): + File "", line 198, in _run_module_as_main + File "", line 88, in _run_code + File "/work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_train.py", line 18, in + main() + File "/work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_train.py", line 14, in main + LIDTask.main(cmd=cmd) + File "/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py", line 1247, in main + cls.main_worker(args) + File "/work/nvme/bbjs/qwang20/espnet/espnet2/tasks/abs_task.py", line 1603, in main_worker + cls.trainer.run( + File "/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py", line 353, in run + all_steps_are_invalid = cls.train_one_epoch( + ^^^^^^^^^^^^^^^^^^^^ + File "/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py", line 650, in train_one_epoch + retval = model(**batch) + ^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl + return self._call_impl(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl + return forward_call(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/work/nvme/bbjs/qwang20/espnet/espnet2/lid/espnet_model_upstream_condition.py", line 247, in forward + feats, feat_lengths, intermediate_lang2vec_preds, intermediate_lid_logits = self.extract_feats(speech, speech_lengths, lid_labels) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/work/nvme/bbjs/qwang20/espnet/espnet2/lid/espnet_model_upstream_condition.py", line 372, in extract_feats + feats, feat_lengths, intermediate_lang2vec_preds, intermediate_lid_logits = self.frontend( + ^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl + return self._call_impl(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl + return forward_call(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/work/nvme/bbjs/qwang20/espnet/espnet2/asr/frontend/s3prl.py", line 358, in forward + feats, feats_lens, intermediate_lang2vec_preds, intermediate_lid_logits = self.upstream(input, input_lengths, labels) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl + return self._call_impl(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl + return forward_call(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/work/nvme/bbjs/qwang20/s3prl/s3prl/nn/upstream.py", line 365, in forward + upstream_output = self.upstream(wavs_list, labels) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl + return self._call_impl(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl + return forward_call(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/hf_wav2vec2/expert.py", line 77, in forward + output_values = self.model(**input_values, output_hidden_states=True, labels=labels) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl + return self._call_impl(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl + return forward_call(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/work/nvme/bbjs/qwang20/transformers/src/transformers/models/wav2vec2/modeling_wav2vec2.py", line 2454, in forward + encoder_outputs = self.encoder( + ^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl + return self._call_impl(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl + return forward_call(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/work/nvme/bbjs/qwang20/transformers/src/transformers/models/wav2vec2/modeling_wav2vec2.py", line 1575, in forward + lang2vec_pred = self._ecapa_lang2vec_pred(hidden_states, feat_lengths, i) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/work/nvme/bbjs/qwang20/transformers/src/transformers/models/wav2vec2/modeling_wav2vec2.py", line 1434, in _ecapa_lang2vec_pred + lang_embd = self._embedding_extract(hidden_states, feat_lengths, layer_idx) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/work/nvme/bbjs/qwang20/transformers/src/transformers/models/wav2vec2/modeling_wav2vec2.py", line 1427, in _embedding_extract + lang_embd = projector(utt_level_feat) + ^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl + return self._call_impl(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl + return forward_call(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/work/nvme/bbjs/qwang20/espnet/espnet2/spk/projector/rawnet3_projector.py", line 18, in forward + return self.fc(self.bn(x)) + ^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl + return self._call_impl(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl + return forward_call(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/modules/batchnorm.py", line 176, in forward + return F.batch_norm( + ^^^^^^^^^^^^^ + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/functional.py", line 2510, in batch_norm + _verify_batch_size(input.size()) + File "/u/qwang20/miniconda3/envs/espnet2/lib/python3.11/site-packages/torch/nn/functional.py", line 2478, in _verify_batch_size + raise ValueError(f"Expected more than 1 value per channel when training, got input size {size}") +ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 2560]) +wandb: +wandb: 🚀 View run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3 at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/0zfdmaq1 +# Accounting: time=113834 threads=1 +# Ended (code 1) at Mon Jun 2 01:08:30 CDT 2025, elapsed time 113834 seconds diff --git a/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.log b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.log new file mode 100644 index 0000000000000000000000000000000000000000..4656f3d80e02d68fcc1b5bcdfd86b20f1603c51c --- /dev/null +++ b/exp_combined/lid_mms_ecapa_upcon_32_44_it0.4_shared_trainable_raw/train.log @@ -0,0 +1,388 @@ +# python3 -m espnet2.bin.lid_train --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True +# Started at Wed Jun 4 20:37:36 CDT 2025 +# +/u/qwang20/miniconda3/envs/espnet2/bin/python3 /work/nvme/bbjs/qwang20/espnet/espnet2/bin/lid_train.py --use_preprocessor true --resume true --ignore_init_mismatch false --output_dir exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/wav.scp,speech,sound --train_data_path_and_name_and_type dump/raw/train_all_no_filter_lang/utt2spk,lid_labels,text --train_shape_file exp_all_no_filter_raw/spk_stats_16k/train/speech_shape --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/wav.scp,speech,sound --valid_data_path_and_name_and_type dump/raw/dev_ml_superb2_lang/utt2spk,lid_labels,text --spk2utt dump/raw/train_all_no_filter_lang/spk2utt --spk_num 157 --fold_length 120000 --valid_shape_file exp_all_no_filter_raw/spk_stats_16k/valid/speech_shape --config /work/nvme/bbjs/qwang20/espnet/egs2/lid_delta/lid1/conf/mms_1b_all_no_filter_balanced_dataset/mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch.yaml --use_wandb true --wandb_project lid --wandb_entity qingzhew-carnegie-mellon-university --ngpu 1 --multiprocessing_distributed True +/work/nvme/bbjs/qwang20/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call. + torchaudio.set_audio_backend("sox_io") +[gpue03] 2025-06-04 20:38:25,336 (abs_task:1420) INFO: pytorch.version=2.4.0+cu118, cuda.available=True, cudnn.version=90100, cudnn.benchmark=True, cudnn.deterministic=False +[gpue03] 2025-06-04 20:38:25,343 (abs_task:1421) INFO: Model structure: +ESPnetLIDUpstreamConditionModel( + (frontend): S3prlFrontendCondition( + (upstream): S3PRLUpstreamCondition( + (upstream): UpstreamExpertCondition( + (model): Wav2Vec2ModelCondition( + (feature_extractor): Wav2Vec2FeatureEncoder( + (conv_layers): ModuleList( + (0): Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (1-4): 4 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + (5-6): 2 x Wav2Vec2LayerNormConvLayer( + (conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,)) + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (activation): GELUActivation() + ) + ) + ) + (feature_projection): Wav2Vec2FeatureProjection( + (layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True) + (projection): Linear(in_features=512, out_features=1280, bias=True) + (dropout): Dropout(p=0.1, inplace=False) + ) + (encoder): Wav2Vec2EncoderCondition( + (pos_conv_embed): Wav2Vec2PositionalConvEmbedding( + (conv): ParametrizedConv1d( + 1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16 + (parametrizations): ModuleDict( + (weight): ParametrizationList( + (0): _WeightNorm() + ) + ) + ) + (padding): Wav2Vec2SamePadLayer() + (activation): GELUActivation() + ) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (dropout): Dropout(p=0.1, inplace=False) + (layers): ModuleList( + (0-47): 48 x Wav2Vec2EncoderLayerStableLayerNorm( + (attention): Wav2Vec2SdpaAttention( + (k_proj): Linear(in_features=1280, out_features=1280, bias=True) + (v_proj): Linear(in_features=1280, out_features=1280, bias=True) + (q_proj): Linear(in_features=1280, out_features=1280, bias=True) + (out_proj): Linear(in_features=1280, out_features=1280, bias=True) + ) + (dropout): Dropout(p=0.1, inplace=False) + (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + (feed_forward): Wav2Vec2FeedForward( + (intermediate_dropout): Dropout(p=0.0, inplace=False) + (intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True) + (intermediate_act_fn): GELUActivation() + (output_dense): Linear(in_features=5120, out_features=1280, bias=True) + (output_dropout): Dropout(p=0.1, inplace=False) + ) + (final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True) + ) + ) + (ecapa_encoder): ModuleDict( + (32): IdentityEncoder() + (36): IdentityEncoder() + (40): IdentityEncoder() + (44): IdentityEncoder() + ) + (pooling): ModuleDict( + (32): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (36): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (40): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (44): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + ) + (projector): ModuleDict( + (32): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (36): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (40): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + (44): RawNet3Projector( + (bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=2560, out_features=192, bias=True) + ) + ) + (lang2vec_head): ModuleDict( + (32): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (36): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (40): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (44): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + ) + (aamsoftmax_weight): ParameterDict() + (lang2vec_conditioning_projs): Linear(in_features=299, out_features=1280, bias=True) + (aamsoftmax_loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) + ) + ) + ) + ) + (featurizer): Featurizer() + ) + (normalize): UtteranceMVN(norm_means=True, norm_vars=False) + (encoder): EcapaTdnnEncoder( + (conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,)) + (relu): ReLU() + (bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (layer1): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer2): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer3): EcapaBlock( + (conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (convs): ModuleList( + (0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,)) + ) + (bns): ModuleList( + (0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + ) + (conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,)) + (bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (relu): ReLU() + (se): SEModule( + (se): Sequential( + (0): AdaptiveAvgPool1d(output_size=1) + (1): Conv1d(512, 128, kernel_size=(1,), stride=(1,)) + (2): ReLU() + (3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (4): Conv1d(128, 512, kernel_size=(1,), stride=(1,)) + (5): Sigmoid() + ) + ) + ) + (layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,)) + (mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False) + ) + (pooling): ChnAttnStatPooling( + (attention): Sequential( + (0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,)) + (1): ReLU() + (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,)) + ) + (softmax): Softmax(dim=2) + ) + (projector): RawNet3Projector( + (bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) + (fc): Linear(in_features=3072, out_features=192, bias=True) + ) + (loss): AAMSoftmaxSCTopKLang2Vec( + (ce): CrossEntropyLoss() + (lang2vec_head): Sequential( + (0): Linear(in_features=192, out_features=299, bias=True) + ) + (lang2vec_loss): MSELoss() + ) +) + +Model summary: + Class Name: ESPnetLIDUpstreamConditionModel + Total Number of model parameters: 977.14 M + Number of trainable parameters: 977.14 M (100.0%) + Size: 3.91 GB + Type: torch.float32 +[gpue03] 2025-06-04 20:38:25,343 (abs_task:1424) INFO: Optimizer: +Adam ( +Parameter Group 0 + amsgrad: False + betas: [0.9, 0.98] + capturable: False + differentiable: False + eps: 1e-08 + foreach: None + fused: None + initial_lr: 1e-05 + lr: 6.0032e-06 + maximize: False + weight_decay: 0 +) +[gpue03] 2025-06-04 20:38:25,343 (abs_task:1425) INFO: Scheduler: TristageLR(warmup_steps=1250)(hold_steps=5000)(decay_steps=6250)(init_lr_scale=0.6)(final_lr_scale=0.1)(decay_factor=0.00036841361487904725) +[gpue03] 2025-06-04 20:38:25,349 (abs_task:1434) INFO: Saving the configuration in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/config.yaml +[gpue03] 2025-06-04 20:38:25,625 (preprocessor:2245) INFO: Using lang2vec geo +[gpue03] 2025-06-04 20:38:41,537 (abs_task:1899) WARNING: Reading dump/raw/train_all_no_filter_lang/category2utt +[gpue03] 2025-06-04 20:38:41,539 (abs_task:1946) WARNING: Reading dump/raw/train_all_no_filter_lang/dataset2utt +[gpue03] 2025-06-04 20:38:41,540 (abs_task:1962) WARNING: Reading dump/raw/train_all_no_filter_lang/utt2dataset +[gpue03] 2025-06-04 20:40:59,199 (abs_task:1997) INFO: [train] dataset: +ESPnetDataset( + speech: {"path": "dump/raw/train_all_no_filter_lang/wav.scp", "type": "sound"} + lid_labels: {"path": "dump/raw/train_all_no_filter_lang/utt2spk", "type": "text"} + preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)) +[gpue03] 2025-06-04 20:40:59,199 (abs_task:1998) INFO: [train] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=True, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False) +[gpue03] 2025-06-04 20:40:59,200 (abs_task:1999) INFO: [train] collate_fn: (float_pad_value=0.0, int_pad_value=0.0) +[gpue03] 2025-06-04 20:40:59,200 (abs_task:2000) INFO: [train] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=727460, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3) +[gpue03] 2025-06-04 20:40:59,266 (abs_task:2001) INFO: [train] mini-batch sizes summary: N-batch=727460, mean=6.0, min=1, max=6 +[gpue03] 2025-06-04 20:40:59,684 (preprocessor:2245) INFO: Using lang2vec geo +[gpue03] 2025-06-04 20:41:12,217 (abs_task:1899) WARNING: Reading dump/raw/dev_ml_superb2_lang/category2utt +[gpue03] 2025-06-04 20:41:12,219 (abs_task:1946) WARNING: Reading dump/raw/dev_ml_superb2_lang/dataset2utt +[gpue03] 2025-06-04 20:41:12,221 (abs_task:1962) WARNING: Reading dump/raw/dev_ml_superb2_lang/utt2dataset +[gpue03] 2025-06-04 20:41:13,249 (abs_task:1997) INFO: [valid] dataset: +ESPnetDataset( + speech: {"path": "dump/raw/dev_ml_superb2_lang/wav.scp", "type": "sound"} + lid_labels: {"path": "dump/raw/dev_ml_superb2_lang/utt2spk", "type": "text"} + preprocess: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False)) +[gpue03] 2025-06-04 20:41:13,249 (abs_task:1998) INFO: [valid] process_fn: espnet2.train.preprocessor.LIDPreprocessor(train=False, spk2utt=dump/raw/train_all_no_filter_lang/spk2utt, len(spk2label)=157, fix_duration=False) +[gpue03] 2025-06-04 20:41:13,249 (abs_task:1999) INFO: [valid] collate_fn: (float_pad_value=0.0, int_pad_value=0.0) +[gpue03] 2025-06-04 20:41:13,249 (abs_task:2000) INFO: [valid] Batch sampler: CategoryPowerSamplerBalancedDataset(N-batch=4722, batch_bins=1440000, language_upsampling_factor=0.5, dataset_upsampling_factor=0.3) +[gpue03] 2025-06-04 20:41:13,250 (abs_task:2001) INFO: [valid] mini-batch sizes summary: N-batch=4722, mean=6.0, min=4, max=6 +wandb: Currently logged in as: qingzhew (qingzhew-carnegie-mellon-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin +wandb: Tracking run with wandb version 0.19.10 +wandb: Run data is saved locally in exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_204114-htm68ys8 +wandb: Run `wandb offline` to turn off syncing. +wandb: Syncing run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch +wandb: ⭐️ View project at https://wandb.ai/qingzhew-carnegie-mellon-university/lid +wandb: 🚀 View run at https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/htm68ys8 +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:218: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead. + scaler = GradScaler() +/work/nvme/bbjs/qwang20/espnet/espnet2/train/trainer.py:159: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + states = torch.load( +[gpue03] 2025-06-04 20:41:24,303 (trainer:176) INFO: The training was resumed using exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/checkpoint.pth +[gpue03] 2025-06-04 20:41:24,647 (trainer:251) INFO: Frontend featurizer weights for each layer: +Parameter containing: +tensor([-0.0056, -0.0141, -0.0168, -0.0187, -0.0203, -0.0225, -0.0231, -0.0246, + -0.0253, -0.0252, -0.0254, -0.0241, -0.0226, -0.0200, -0.0162, -0.0120, + -0.0095, -0.0059, -0.0017, 0.0058, 0.0097, 0.0142, 0.0175, 0.0196, + 0.0211, 0.0224, 0.0228, 0.0230, 0.0226, 0.0224, 0.0215, 0.0210, + 0.0196, 0.0176, 0.0157, 0.0126, 0.0095, 0.0070, 0.0051, 0.0037, + 0.0020, -0.0003, -0.0030, -0.0056, -0.0076, -0.0090, -0.0096, -0.0102, + -0.0102], device='cuda:0', requires_grad=True) +[gpue03] 2025-06-04 20:41:24,648 (trainer:267) INFO: Error: 'Linear' object is not subscriptable +[gpue03] 2025-06-04 20:41:24,648 (trainer:272) INFO: cos_mp: 1.0 +[gpue03] 2025-06-04 20:41:24,648 (trainer:273) INFO: easy_margin: False +[gpue03] 2025-06-04 20:41:24,648 (trainer:281) WARNING: The training has already reached at max_epoch: 34 +[gpue03] 2025-06-04 20:41:24,659 (trainer:541) INFO: The training was finished at 33 epochs +[gpue03] 2025-06-04 20:41:24,660 (average_nbest_models:69) INFO: Averaging 2best models: criterion="valid.accuracy": exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/valid.accuracy.ave_2best.pth +/work/nvme/bbjs/qwang20/espnet/espnet2/main_funcs/average_nbest_models.py:77: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. + _loaded[e] = torch.load( +[gpue03] 2025-06-04 20:41:30,224 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.32.attention.2.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,225 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.36.attention.2.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,225 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.40.attention.2.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,226 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.pooling.44.attention.2.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,227 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.32.bn.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,227 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.36.bn.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,227 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.40.bn.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,228 (average_nbest_models:96) INFO: Accumulating frontend.upstream.upstream.model.encoder.projector.44.bn.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,230 (average_nbest_models:96) INFO: Accumulating encoder.bn.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn1.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.0.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.1.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.2.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.3.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.4.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,231 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.5.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,232 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bns.6.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,232 (average_nbest_models:96) INFO: Accumulating encoder.layer1.bn3.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,232 (average_nbest_models:96) INFO: Accumulating encoder.layer1.se.se.3.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,232 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn1.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.0.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.1.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.2.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.3.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.4.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.5.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bns.6.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.bn3.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,233 (average_nbest_models:96) INFO: Accumulating encoder.layer2.se.se.3.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn1.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.0.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.1.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.2.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.3.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.4.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.5.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,234 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bns.6.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,235 (average_nbest_models:96) INFO: Accumulating encoder.layer3.bn3.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,235 (average_nbest_models:96) INFO: Accumulating encoder.layer3.se.se.3.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,237 (average_nbest_models:96) INFO: Accumulating pooling.attention.2.num_batches_tracked instead of averaging +[gpue03] 2025-06-04 20:41:30,237 (average_nbest_models:96) INFO: Accumulating projector.bn.num_batches_tracked instead of averaging +wandb: +wandb: 🚀 View run mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid/runs/htm68ys8 +wandb: ⭐️ View project at: https://wandb.ai/qingzhew-carnegie-mellon-university/lid +wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) +wandb: Find logs at: exp_all_no_filter_raw/spk_mms_ecapa_upcon_32_44_it0.4_sharedCondProj_butUpdate_50k_lr1e-5_datasetup0.3_backup_33epoch_raw/wandb/run-20250604_204114-htm68ys8/logs +# Accounting: time=240 threads=1 +# Ended (code 0) at Wed Jun 4 20:41:36 CDT 2025, elapsed time 240 seconds