| Namespace(no_progress_bar=False, log_interval=100, log_format=None, tensorboard_logdir=None, seed=42, cpu=False, tpu=False, bf16=False, memory_efficient_bf16=False, fp16=False, memory_efficient_fp16=False, fp16_no_flatten_grads=False, fp16_init_scale=128, fp16_scale_window=None, fp16_scale_tolerance=0.0, min_loss_scale=0.0001, threshold_loss_scale=None, user_dir=None, empty_cache_freq=0, all_gather_list_size=16384, model_parallel_size=1, checkpoint_suffix='', checkpoint_shard_count=1, quantization_config_path=None, profile=False, criterion='cross_entropy', tokenizer=None, bpe=None, optimizer=None, lr_scheduler='fixed', scoring='bleu', task='translation', source_lang='en', target_lang='de', trainpref=None, validpref=None, testpref='/mnt/congmh/luoyf/xzq-fairseq/data/test/tokenized/en2de/bpe.test.en2de.wmt23', align_suffix=None, destdir='/mnt/congmh/luoyf/xzq-fairseq/data/test/trainable_data/en2de', thresholdtgt=0, thresholdsrc=0, tgtdict='/mnt/congmh/luoyf/xzq-fairseq/data/de-en/wmt23/trainable_data/dict.de.txt', srcdict='/mnt/congmh/luoyf/xzq-fairseq/data/de-en/wmt23/trainable_data/dict.en.txt', nwordstgt=-1, nwordssrc=-1, alignfile=None, dataset_impl='mmap', joined_dictionary=False, only_source=False, padding_factor=8, workers=32) | |
| [en] Dictionary: 47776 types | |
| [en] /mnt/congmh/luoyf/xzq-fairseq/data/test/tokenized/en2de/bpe.test.en2de.wmt23.en: 557 sents, 47662 tokens, 0.0% replaced by <unk> | |
| [de] Dictionary: 47776 types | |
| [de] /mnt/congmh/luoyf/xzq-fairseq/data/test/tokenized/en2de/bpe.test.en2de.wmt23.de: 557 sents, 51978 tokens, 0.0% replaced by <unk> | |
| Wrote preprocessed data to /mnt/congmh/luoyf/xzq-fairseq/data/test/trainable_data/en2de | |
| Namespace(no_progress_bar=False, log_interval=100, log_format=None, tensorboard_logdir=None, seed=42, cpu=False, tpu=False, bf16=False, memory_efficient_bf16=False, fp16=False, memory_efficient_fp16=False, fp16_no_flatten_grads=False, fp16_init_scale=128, fp16_scale_window=None, fp16_scale_tolerance=0.0, min_loss_scale=0.0001, threshold_loss_scale=None, user_dir=None, empty_cache_freq=0, all_gather_list_size=16384, model_parallel_size=1, checkpoint_suffix='', checkpoint_shard_count=1, quantization_config_path=None, profile=False, criterion='cross_entropy', tokenizer=None, bpe=None, optimizer=None, lr_scheduler='fixed', scoring='bleu', task='translation', source_lang='en', target_lang='de', trainpref=None, validpref=None, testpref='/mnt/congmh/luoyf/xzq-fairseq/data/test/tokenized/en2de/bpe.test.en2de.wmt22', align_suffix=None, destdir='/mnt/congmh/luoyf/xzq-fairseq/data/test/trainable_data/en2de', thresholdtgt=0, thresholdsrc=0, tgtdict='/mnt/congmh/luoyf/xzq-fairseq/data/de-en/wmt23/trainable_data/dict.de.txt', srcdict='/mnt/congmh/luoyf/xzq-fairseq/data/de-en/wmt23/trainable_data/dict.en.txt', nwordstgt=-1, nwordssrc=-1, alignfile=None, dataset_impl='mmap', joined_dictionary=False, only_source=False, padding_factor=8, workers=32) | |
| [en] Dictionary: 47776 types | |
| [en] /mnt/congmh/luoyf/xzq-fairseq/data/test/tokenized/en2de/bpe.test.en2de.wmt22.en: 2037 sents, 46796 tokens, 0.00641% replaced by <unk> | |
| [de] Dictionary: 47776 types | |
| [de] /mnt/congmh/luoyf/xzq-fairseq/data/test/tokenized/en2de/bpe.test.en2de.wmt22.de: 2037 sents, 51454 tokens, 0.00389% replaced by <unk> | |
| Wrote preprocessed data to /mnt/congmh/luoyf/xzq-fairseq/data/test/trainable_data/en2de | |