| /workspace/bina02-ppocr/.venv/lib/python3.12/site-packages/paddle/utils/cpp_extension/extension_utils.py:712: UserWarning: No ccache found. Please be aware that recompiling all source files may be required. You can download and install ccache from: https://github.com/ccache/ccache/blob/master/doc/INSTALL.md |
| warnings.warn(warning_message) |
| LAUNCH INFO 2026-07-25 19:23:46,992 ----------- Configuration ---------------------- |
| LAUNCH INFO 2026-07-25 19:23:46,992 auto_cluster_config: 0 |
| LAUNCH INFO 2026-07-25 19:23:46,992 auto_parallel_config: None |
| LAUNCH INFO 2026-07-25 19:23:46,992 auto_tuner_json: None |
| LAUNCH INFO 2026-07-25 19:23:46,992 devices: 0,1,2,3,4,5,6,7 |
| LAUNCH INFO 2026-07-25 19:23:46,992 elastic_level: -1 |
| LAUNCH INFO 2026-07-25 19:23:46,992 elastic_timeout: 30 |
| LAUNCH INFO 2026-07-25 19:23:46,992 enable_gpu_log: True |
| LAUNCH INFO 2026-07-25 19:23:46,992 gloo_port: 6767 |
| LAUNCH INFO 2026-07-25 19:23:46,992 host: None |
| LAUNCH INFO 2026-07-25 19:23:46,992 ips: None |
| LAUNCH INFO 2026-07-25 19:23:46,992 job_id: default |
| LAUNCH INFO 2026-07-25 19:23:46,992 legacy: False |
| LAUNCH INFO 2026-07-25 19:23:46,992 log_dir: /workspace/bina02-ppocr/training/stage-010/distributed-logs |
| LAUNCH INFO 2026-07-25 19:23:46,992 log_level: INFO |
| LAUNCH INFO 2026-07-25 19:23:46,992 log_overwrite: False |
| LAUNCH INFO 2026-07-25 19:23:46,992 master: None |
| LAUNCH INFO 2026-07-25 19:23:46,992 max_restart: 3 |
| LAUNCH INFO 2026-07-25 19:23:46,992 nnodes: 1 |
| LAUNCH INFO 2026-07-25 19:23:46,992 nproc_per_node: None |
| LAUNCH INFO 2026-07-25 19:23:46,992 rank: -1 |
| LAUNCH INFO 2026-07-25 19:23:46,993 run_mode: collective |
| LAUNCH INFO 2026-07-25 19:23:46,993 server_num: None |
| LAUNCH INFO 2026-07-25 19:23:46,993 servers: |
| LAUNCH INFO 2026-07-25 19:23:46,993 sort_ip: False |
| LAUNCH INFO 2026-07-25 19:23:46,993 start_port: 6070 |
| LAUNCH INFO 2026-07-25 19:23:46,993 trainer_num: None |
| LAUNCH INFO 2026-07-25 19:23:46,993 trainers: |
| LAUNCH INFO 2026-07-25 19:23:46,993 training_script: /workspace/bina02-ppocr/PaddleOCR/tools/train.py |
| LAUNCH INFO 2026-07-25 19:23:46,993 training_script_args: ['-c', '/workspace/bina02-ppocr/training/stage-010/input_config.yml'] |
| LAUNCH INFO 2026-07-25 19:23:46,993 with_gloo: 1 |
| LAUNCH INFO 2026-07-25 19:23:46,993 -------------------------------------------------- |
| LAUNCH INFO 2026-07-25 19:23:46,994 Job: default, mode collective, replicas 1[1:1], elastic False |
| LAUNCH INFO 2026-07-25 19:23:46,995 Run Pod: thqxut, replicas 8, status ready |
| LAUNCH INFO 2026-07-25 19:23:47,264 Watching Pod: thqxut, replicas 8, status running |
| /workspace/bina02-ppocr/.venv/lib/python3.12/site-packages/paddle/utils/cpp_extension/extension_utils.py:712: UserWarning: No ccache found. Please be aware that recompiling all source files may be required. You can download and install ccache from: https://github.com/ccache/ccache/blob/master/doc/INSTALL.md |
| warnings.warn(warning_message) |
| Skipping import of the encryption module. |
| [2026/07/25 19:23:54] ppocr INFO: Architecture : |
| [2026/07/25 19:23:54] ppocr INFO: Backbone : |
| [2026/07/25 19:23:54] ppocr INFO: force_pool_width : True |
| [2026/07/25 19:23:54] ppocr INFO: model_size : medium |
| [2026/07/25 19:23:54] ppocr INFO: name : PPLCNetV4 |
| [2026/07/25 19:23:54] ppocr INFO: train_pool_width : 96 |
| [2026/07/25 19:23:54] ppocr INFO: Head : |
| [2026/07/25 19:23:54] ppocr INFO: head_list : |
| [2026/07/25 19:23:54] ppocr INFO: CTCHead : |
| [2026/07/25 19:23:54] ppocr INFO: Head : |
| [2026/07/25 19:23:54] ppocr INFO: fc_decay : 1e-05 |
| [2026/07/25 19:23:54] ppocr INFO: Neck : |
| [2026/07/25 19:23:54] ppocr INFO: depth : 2 |
| [2026/07/25 19:23:54] ppocr INFO: dims : 192 |
| [2026/07/25 19:23:54] ppocr INFO: local_kernel : 7 |
| [2026/07/25 19:23:54] ppocr INFO: mlp_ratio : 4.0 |
| [2026/07/25 19:23:54] ppocr INFO: name : lightsvtr |
| [2026/07/25 19:23:54] ppocr INFO: use_guide : False |
| [2026/07/25 19:23:54] ppocr INFO: NRTRHead : |
| [2026/07/25 19:23:54] ppocr INFO: max_text_length : 128 |
| [2026/07/25 19:23:54] ppocr INFO: nrtr_dim : 512 |
| [2026/07/25 19:23:54] ppocr INFO: name : MultiHead |
| [2026/07/25 19:23:54] ppocr INFO: Transform : None |
| [2026/07/25 19:23:54] ppocr INFO: algorithm : SVTR_LCNet |
| [2026/07/25 19:23:54] ppocr INFO: model_type : rec |
| [2026/07/25 19:23:54] ppocr INFO: Eval : |
| [2026/07/25 19:23:54] ppocr INFO: dataset : |
| [2026/07/25 19:23:54] ppocr INFO: data_dir : / |
| [2026/07/25 19:23:54] ppocr INFO: label_file_list : ['/workspace/bina02-ppocr/stream/datasets/chunk-010/eval_ctc.txt'] |
| [2026/07/25 19:23:54] ppocr INFO: name : SimpleDataSet |
| [2026/07/25 19:23:54] ppocr INFO: transforms : |
| [2026/07/25 19:23:54] ppocr INFO: DecodeImage : |
| [2026/07/25 19:23:54] ppocr INFO: channel_first : False |
| [2026/07/25 19:23:54] ppocr INFO: img_mode : BGR |
| [2026/07/25 19:23:54] ppocr INFO: MultiLabelEncode : |
| [2026/07/25 19:23:54] ppocr INFO: gtc_encode : NRTRLabelEncode |
| [2026/07/25 19:23:54] ppocr INFO: RecResizeImg : |
| [2026/07/25 19:23:54] ppocr INFO: image_shape : [3, 48, 768] |
| [2026/07/25 19:23:54] ppocr INFO: padding : False |
| [2026/07/25 19:23:54] ppocr INFO: KeepKeys : |
| [2026/07/25 19:23:54] ppocr INFO: keep_keys : ['image', 'label_ctc', 'label_gtc', 'length', 'valid_ratio'] |
| [2026/07/25 19:23:54] ppocr INFO: loader : |
| [2026/07/25 19:23:54] ppocr INFO: batch_size_per_card : 32 |
| [2026/07/25 19:23:54] ppocr INFO: drop_last : False |
| [2026/07/25 19:23:54] ppocr INFO: num_workers : 4 |
| [2026/07/25 19:23:54] ppocr INFO: shuffle : False |
| [2026/07/25 19:23:54] ppocr INFO: Global : |
| [2026/07/25 19:23:54] ppocr INFO: cal_metric_during_train : False |
| [2026/07/25 19:23:54] ppocr INFO: character_dict_path : /workspace/bina02-ppocr/stream/datasets/chunk-010/persian_arabic_bina02_dict.txt |
| [2026/07/25 19:23:54] ppocr INFO: checkpoints : None |
| [2026/07/25 19:23:54] ppocr INFO: d2s_train_image_shape : [3, 48, 768] |
| [2026/07/25 19:23:54] ppocr INFO: debug : False |
| [2026/07/25 19:23:54] ppocr INFO: distributed : True |
| [2026/07/25 19:23:54] ppocr INFO: epoch_num : 1 |
| [2026/07/25 19:23:54] ppocr INFO: eval_batch_step : [0, 100] |
| [2026/07/25 19:23:54] ppocr INFO: freeze_batch_norm : True |
| [2026/07/25 19:23:54] ppocr INFO: infer_img : doc/imgs_words/ch/word_1.jpg |
| [2026/07/25 19:23:54] ppocr INFO: infer_mode : False |
| [2026/07/25 19:23:54] ppocr INFO: log_smooth_window : 20 |
| [2026/07/25 19:23:54] ppocr INFO: max_text_length : 128 |
| [2026/07/25 19:23:54] ppocr INFO: model_name : PP-OCRv6_medium_rec |
| [2026/07/25 19:23:54] ppocr INFO: pretrained_model : /workspace/bina02-ppocr/training/stage-009/best_accuracy |
| [2026/07/25 19:23:54] ppocr INFO: print_batch_step : 20 |
| [2026/07/25 19:23:54] ppocr INFO: save_epoch_step : 1 |
| [2026/07/25 19:23:54] ppocr INFO: save_inference_dir : None |
| [2026/07/25 19:23:54] ppocr INFO: save_model_dir : /workspace/bina02-ppocr/training/stage-010 |
| [2026/07/25 19:23:54] ppocr INFO: save_res_path : /workspace/bina02-ppocr/training/stage-010/predicts.txt |
| [2026/07/25 19:23:54] ppocr INFO: trainable_parameter_prefixes : ['head.ctc_head.fc.'] |
| [2026/07/25 19:23:54] ppocr INFO: use_gpu : True |
| [2026/07/25 19:23:54] ppocr INFO: use_space_char : True |
| [2026/07/25 19:23:54] ppocr INFO: use_visualdl : False |
| [2026/07/25 19:23:54] ppocr INFO: Loss : |
| [2026/07/25 19:23:54] ppocr INFO: loss_config_list : |
| [2026/07/25 19:23:54] ppocr INFO: CTCLoss : None |
| [2026/07/25 19:23:54] ppocr INFO: NRTRLoss : None |
| [2026/07/25 19:23:54] ppocr INFO: name : MultiLoss |
| [2026/07/25 19:23:54] ppocr INFO: weight_1 : 1.0 |
| [2026/07/25 19:23:54] ppocr INFO: weight_2 : 0.0 |
| [2026/07/25 19:23:54] ppocr INFO: Metric : |
| [2026/07/25 19:23:54] ppocr INFO: ignore_space : False |
| [2026/07/25 19:23:54] ppocr INFO: main_indicator : norm_edit_dis |
| [2026/07/25 19:23:54] ppocr INFO: name : RecMetric |
| [2026/07/25 19:23:54] ppocr INFO: Optimizer : |
| [2026/07/25 19:23:54] ppocr INFO: beta1 : 0.9 |
| [2026/07/25 19:23:54] ppocr INFO: beta2 : 0.999 |
| [2026/07/25 19:23:54] ppocr INFO: lr : |
| [2026/07/25 19:23:54] ppocr INFO: learning_rate : 1e-05 |
| [2026/07/25 19:23:54] ppocr INFO: name : Cosine |
| [2026/07/25 19:23:54] ppocr INFO: warmup_epoch : 0 |
| [2026/07/25 19:23:54] ppocr INFO: name : Adam |
| [2026/07/25 19:23:54] ppocr INFO: regularizer : |
| [2026/07/25 19:23:54] ppocr INFO: factor : 0.0 |
| [2026/07/25 19:23:54] ppocr INFO: name : L2 |
| [2026/07/25 19:23:54] ppocr INFO: PostProcess : |
| [2026/07/25 19:23:54] ppocr INFO: name : CTCLabelDecode |
| [2026/07/25 19:23:54] ppocr INFO: Train : |
| [2026/07/25 19:23:54] ppocr INFO: dataset : |
| [2026/07/25 19:23:54] ppocr INFO: data_dir : / |
| [2026/07/25 19:23:54] ppocr INFO: ds_width : False |
| [2026/07/25 19:23:54] ppocr INFO: ext_op_transform_idx : 1 |
| [2026/07/25 19:23:54] ppocr INFO: label_file_list : ['/workspace/bina02-ppocr/stream/datasets/chunk-010/train_ctc.txt'] |
| [2026/07/25 19:23:54] ppocr INFO: name : MultiScaleDataSet |
| [2026/07/25 19:23:54] ppocr INFO: padding : False |
| [2026/07/25 19:23:54] ppocr INFO: ratio_list : [1.0] |
| [2026/07/25 19:23:54] ppocr INFO: transforms : |
| [2026/07/25 19:23:54] ppocr INFO: DecodeImage : |
| [2026/07/25 19:23:54] ppocr INFO: channel_first : False |
| [2026/07/25 19:23:54] ppocr INFO: img_mode : BGR |
| [2026/07/25 19:23:54] ppocr INFO: RecConAug : |
| [2026/07/25 19:23:54] ppocr INFO: ext_data_num : 0 |
| [2026/07/25 19:23:54] ppocr INFO: image_shape : [48, 768, 3] |
| [2026/07/25 19:23:54] ppocr INFO: max_text_length : 128 |
| [2026/07/25 19:23:54] ppocr INFO: prob : 0.0 |
| [2026/07/25 19:23:54] ppocr INFO: RecAug : None |
| [2026/07/25 19:23:54] ppocr INFO: MultiLabelEncode : |
| [2026/07/25 19:23:54] ppocr INFO: gtc_encode : NRTRLabelEncode |
| [2026/07/25 19:23:54] ppocr INFO: KeepKeys : |
| [2026/07/25 19:23:54] ppocr INFO: keep_keys : ['image', 'label_ctc', 'label_gtc', 'length', 'valid_ratio'] |
| [2026/07/25 19:23:54] ppocr INFO: loader : |
| [2026/07/25 19:23:54] ppocr INFO: batch_size_per_card : 32 |
| [2026/07/25 19:23:54] ppocr INFO: drop_last : False |
| [2026/07/25 19:23:54] ppocr INFO: num_workers : 4 |
| [2026/07/25 19:23:54] ppocr INFO: shuffle : True |
| [2026/07/25 19:23:54] ppocr INFO: sampler : |
| [2026/07/25 19:23:54] ppocr INFO: divided_factor : [8, 16] |
| [2026/07/25 19:23:54] ppocr INFO: first_bs : 32 |
| [2026/07/25 19:23:54] ppocr INFO: fix_bs : True |
| [2026/07/25 19:23:54] ppocr INFO: is_training : True |
| [2026/07/25 19:23:54] ppocr INFO: name : MultiScaleSampler |
| [2026/07/25 19:23:54] ppocr INFO: scales : [[768, 48]] |
| [2026/07/25 19:23:54] ppocr INFO: profiler_options : None |
| [2026/07/25 19:23:54] ppocr INFO: train with paddle 3.3.0 and device Place(gpu:0) |
| ======================= Modified FLAGS detected ======================= |
| FLAGS(name='FLAGS_enable_pir_in_executor', current_value=True, default_value=False) |
| FLAGS(name='FLAGS_nvidia_package_dir', current_value='/workspace/bina02-ppocr/.venv/lib/python3.12/site-packages/paddle/../nvidia', default_value='') |
| FLAGS(name='FLAGS_cupti_dir', current_value='/workspace/bina02-ppocr/.venv/lib/python3.12/site-packages/paddle/../nvidia/cuda_cupti/lib', default_value='') |
| FLAGS(name='FLAGS_cublas_dir', current_value='/workspace/bina02-ppocr/.venv/lib/python3.12/site-packages/paddle/../nvidia/cublas/lib', default_value='') |
| FLAGS(name='FLAGS_cuda_cccl_dir', current_value='/workspace/bina02-ppocr/.venv/lib/python3.12/site-packages/paddle/../nvidia/cuda_cccl/include/', default_value='') |
| FLAGS(name='FLAGS_cudnn_deterministic', current_value=True, default_value=False) |
| FLAGS(name='FLAGS_curand_dir', current_value='/workspace/bina02-ppocr/.venv/lib/python3.12/site-packages/paddle/../nvidia/curand/lib', default_value='') |
| FLAGS(name='FLAGS_selected_gpus', current_value='0', default_value='') |
| FLAGS(name='FLAGS_cudnn_dir', current_value='/workspace/bina02-ppocr/.venv/lib/python3.12/site-packages/paddle/../nvidia/cudnn/lib', default_value='') |
| FLAGS(name='FLAGS_cusolver_dir', current_value='/workspace/bina02-ppocr/.venv/lib/python3.12/site-packages/paddle/../nvidia/cusolver/lib', default_value='') |
| FLAGS(name='FLAGS_cusparse_dir', current_value='/workspace/bina02-ppocr/.venv/lib/python3.12/site-packages/paddle/../nvidia/cusparse/lib', default_value='') |
| FLAGS(name='FLAGS_nccl_dir', current_value='/workspace/bina02-ppocr/.venv/lib/python3.12/site-packages/paddle/../nvidia/nccl/lib', default_value='') |
| ======================================================================= |
| I0725 19:23:54.243121 14638 tcp_store.cc:336] create libuv server at port: 41218 |
| I0725 19:23:54.243770 14638 tcp_utils.cc:132] Successfully connected to 172.17.0.2:41218 |
| I0725 19:23:57.441540 14638 process_group_nccl.cc:155] ProcessGroupNCCL pg_timeout_ 1800000 |
| I0725 19:23:57.441627 14638 process_group_nccl.cc:156] ProcessGroupNCCL nccl_comm_init_option_ 0 |
| [2026/07/25 19:23:57] ppocr INFO: Initialize indexs of datasets:['/workspace/bina02-ppocr/stream/datasets/chunk-010/train_ctc.txt'] |
| [2026/07/25 19:23:57] ppocr INFO: Initialize indexs of datasets:['/workspace/bina02-ppocr/stream/datasets/chunk-010/eval_ctc.txt'] |
| W0725 19:23:57.638904 14638 gpu_resources.cc:116] Please NOTE: device: 0, GPU Compute Capability: 12.0, Driver API Version: 13.0, Runtime API Version: 13.0 |
| [2026/07/25 19:23:58] ppocr INFO: froze 61 BatchNorm layers with pretrained global statistics |
| [2026/07/25 19:23:58] ppocr INFO: selective training enabled: 2 trainable tensors, 438 frozen tensors; head.ctc_head.fc.weight, head.ctc_head.fc.bias |
| [2026-07-25 19:23:58,013] [ INFO] optimizer.py:254 - If regularizer of a Parameter has been set by 'paddle.ParamAttr' or 'static.WeightNormParamAttr' already. The weight_decay[0.0] in Optimizer will not take effect, and it will only be applied to other Parameters! |
| [2026/07/25 19:23:58] ppocr INFO: train dataloader has 391 iters |
| [2026/07/25 19:23:58] ppocr INFO: valid dataloader has 128 iters |
| [2026/07/25 19:23:58] ppocr INFO: load pretrain successful from /workspace/bina02-ppocr/training/stage-009/best_accuracy |
| [2026/07/25 19:23:59] ppocr INFO: During the training process, after the 0th iteration, an evaluation is run every 100 iterations |
| [2026/07/25 19:24:03] ppocr INFO: epoch: [1/1], global_step: 20, lr: 0.000010, CTCLoss: 58.009033, NRTRLoss: 0.000000, loss: 58.009033, avg_reader_cost: 0.08819 s, avg_batch_cost: 0.20218 s, avg_samples: 32.0, ips: 158.27464 samples/s, eta: 0:01:15, max_mem_reserved: 1045 MB, max_mem_allocated: 656 MB |
| [2026/07/25 19:24:06] ppocr INFO: epoch: [1/1], global_step: 40, lr: 0.000010, CTCLoss: 50.958549, NRTRLoss: 0.000000, loss: 50.958549, avg_reader_cost: 0.00171 s, avg_batch_cost: 0.07987 s, avg_samples: 32.0, ips: 400.66931 samples/s, eta: 0:00:49, max_mem_reserved: 1045 MB, max_mem_allocated: 656 MB |
| [2026/07/25 19:24:09] ppocr INFO: epoch: [1/1], global_step: 60, lr: 0.000010, CTCLoss: 49.933956, NRTRLoss: 0.000000, loss: 49.933956, avg_reader_cost: 0.00154 s, avg_batch_cost: 0.07706 s, avg_samples: 32.0, ips: 415.27236 samples/s, eta: 0:00:39, max_mem_reserved: 1045 MB, max_mem_allocated: 656 MB |
| [2026/07/25 19:24:11] ppocr INFO: epoch: [1/1], global_step: 80, lr: 0.000009, CTCLoss: 56.557682, NRTRLoss: 0.000000, loss: 56.557682, avg_reader_cost: 0.00172 s, avg_batch_cost: 0.08214 s, avg_samples: 32.0, ips: 389.58344 samples/s, eta: 0:00:34, max_mem_reserved: 1045 MB, max_mem_allocated: 656 MB |
| [2026/07/25 19:24:14] ppocr INFO: epoch: [1/1], global_step: 100, lr: 0.000009, CTCLoss: 57.385790, NRTRLoss: 0.000000, loss: 57.385790, avg_reader_cost: 0.01860 s, avg_batch_cost: 0.09830 s, avg_samples: 32.0, ips: 325.52996 samples/s, eta: 0:00:31, max_mem_reserved: 1045 MB, max_mem_allocated: 656 MB |
|
|
| eval model:: 0%| | 0/128 [00:00<?, ?it/s] |
| eval model:: 1%| | 1/128 [00:00<00:50, 2.51it/s] |
| eval model:: 2%|▏ | 2/128 [00:00<00:28, 4.43it/s] |
| eval model:: 3%|▎ | 4/128 [00:00<00:15, 7.76it/s] |
| eval model:: 5%|▍ | 6/128 [00:00<00:12, 9.42it/s] |
| eval model:: 6%|▋ | 8/128 [00:00<00:11, 10.75it/s] |
| eval model:: 8%|▊ | 10/128 [00:01<00:10, 11.62it/s] |
| eval model:: 9%|▉ | 12/128 [00:01<00:09, 12.41it/s] |
| eval model:: 11%|█ | 14/128 [00:01<00:09, 12.14it/s] |
| eval model:: 12%|█▎ | 16/128 [00:01<00:08, 12.54it/s] |
| eval model:: 14%|█▍ | 18/128 [00:01<00:09, 12.15it/s] |
| eval model:: 16%|█▌ | 20/128 [00:01<00:08, 12.51it/s] |
| eval model:: 17%|█▋ | 22/128 [00:02<00:08, 13.03it/s] |
| eval model:: 19%|█▉ | 24/128 [00:02<00:08, 12.16it/s] |
| eval model:: 20%|██ | 26/128 [00:02<00:08, 11.63it/s] |
| eval model:: 22%|██▏ | 28/128 [00:02<00:08, 12.32it/s] |
| eval model:: 23%|██▎ | 30/128 [00:02<00:07, 12.82it/s] |
| eval model:: 25%|██▌ | 32/128 [00:02<00:07, 13.23it/s] |
| eval model:: 27%|██▋ | 34/128 [00:02<00:06, 13.52it/s] |
| eval model:: 28%|██▊ | 36/128 [00:03<00:06, 13.66it/s] |
| eval model:: 30%|██▉ | 38/128 [00:03<00:06, 13.85it/s] |
| eval model:: 31%|███▏ | 40/128 [00:03<00:06, 13.96it/s] |
| eval model:: 33%|███▎ | 42/128 [00:03<00:06, 14.03it/s] |
| eval model:: 34%|███▍ | 44/128 [00:03<00:05, 14.10it/s] |
| eval model:: 36%|███▌ | 46/128 [00:03<00:05, 14.18it/s] |
| eval model:: 38%|███▊ | 48/128 [00:03<00:05, 14.20it/s] |
| eval model:: 39%|███▉ | 50/128 [00:04<00:05, 14.24it/s] |
| eval model:: 41%|████ | 52/128 [00:04<00:05, 14.27it/s] |
| eval model:: 42%|████▏ | 54/128 [00:04<00:05, 14.27it/s] |
| eval model:: 44%|████▍ | 56/128 [00:04<00:05, 14.21it/s] |
| eval model:: 45%|████▌ | 58/128 [00:04<00:04, 14.18it/s] |
| eval model:: 47%|████▋ | 60/128 [00:04<00:04, 14.16it/s] |
| eval model:: 48%|████▊ | 62/128 [00:04<00:04, 14.21it/s] |
| eval model:: 50%|█████ | 64/128 [00:05<00:04, 14.20it/s] |
| eval model:: 52%|█████▏ | 66/128 [00:05<00:04, 14.18it/s] |
| eval model:: 53%|█████▎ | 68/128 [00:05<00:04, 14.20it/s] |
| eval model:: 55%|█████▍ | 70/128 [00:05<00:04, 12.78it/s] |
| eval model:: 56%|█████▋ | 72/128 [00:05<00:04, 12.89it/s] |
| eval model:: 58%|█████▊ | 74/128 [00:05<00:04, 13.28it/s] |
| eval model:: 59%|█████▉ | 76/128 [00:05<00:03, 13.36it/s] |
| eval model:: 61%|██████ | 78/128 [00:06<00:03, 13.64it/s] |
| eval model:: 62%|██████▎ | 80/128 [00:06<00:03, 13.62it/s] |
| eval model:: 64%|██████▍ | 82/128 [00:06<00:03, 12.96it/s] |
| eval model:: 66%|██████▌ | 84/128 [00:06<00:03, 13.31it/s] |
| eval model:: 67%|██████▋ | 86/128 [00:06<00:03, 13.61it/s] |
| eval model:: 69%|██████▉ | 88/128 [00:06<00:02, 13.52it/s] |
| eval model:: 70%|███████ | 90/128 [00:07<00:02, 13.53it/s] |
| eval model:: 72%|███████▏ | 92/128 [00:07<00:02, 13.17it/s] |
| eval model:: 73%|███████▎ | 94/128 [00:07<00:02, 13.49it/s] |
| eval model:: 75%|███████▌ | 96/128 [00:07<00:02, 13.34it/s] |
| eval model:: 77%|███████▋ | 98/128 [00:07<00:02, 13.40it/s] |
| eval model:: 78%|███████▊ | 100/128 [00:07<00:02, 13.45it/s] |
| eval model:: 80%|███████▉ | 102/128 [00:07<00:01, 13.71it/s] |
| eval model:: 81%|████████▏ | 104/128 [00:08<00:01, 13.64it/s] |
| eval model:: 83%|████████▎ | 106/128 [00:08<00:01, 13.84it/s] |
| eval model:: 84%|████████▍ | 108/128 [00:08<00:01, 13.71it/s] |
| eval model:: 86%|████████▌ | 110/128 [00:08<00:01, 13.90it/s] |
| eval model:: 88%|████████▊ | 112/128 [00:08<00:01, 13.78it/s] |
| eval model:: 89%|████████▉ | 114/128 [00:08<00:01, 13.90it/s] |
| eval model:: 91%|█████████ | 116/128 [00:08<00:00, 14.00it/s] |
| eval model:: 92%|█████████▏| 118/128 [00:09<00:00, 14.08it/s] |
| eval model:: 94%|█████████▍| 120/128 [00:09<00:00, 13.26it/s] |
| eval model:: 95%|█████████▌| 122/128 [00:09<00:00, 12.60it/s] |
| eval model:: 97%|█████████▋| 124/128 [00:09<00:00, 12.17it/s] |
| eval model:: 98%|█████████▊| 126/128 [00:09<00:00, 12.38it/s] |
| eval model:: 100%|██████████| 128/128 [00:09<00:00, 12.92it/s] |
| eval model:: 100%|██████████| 128/128 [00:09<00:00, 12.80it/s] |
| [2026/07/25 19:24:24] ppocr INFO: cur metric, acc: 0.8459472635596991, norm_edit_dis: 0.9878096886547653, fps: 451.91959000322055 |
| [2026/07/25 19:24:25] ppocr INFO: save best model is to /workspace/bina02-ppocr/training/stage-010/best_accuracy |
| [2026/07/25 19:24:25] ppocr INFO: best metric, norm_edit_dis: 0.9878096886547653, is_float16: False, acc: 0.8459472635596991, fps: 451.91959000322055, best_epoch: 1 |
| [2026/07/25 19:24:27] ppocr INFO: epoch: [1/1], global_step: 120, lr: 0.000008, CTCLoss: 53.492466, NRTRLoss: 0.000000, loss: 53.492466, avg_reader_cost: 0.00151 s, avg_batch_cost: 0.07594 s, avg_samples: 32.0, ips: 421.39562 samples/s, eta: 0:00:27, max_mem_reserved: 1045 MB, max_mem_allocated: 672 MB |
| [2026/07/25 19:24:30] ppocr INFO: epoch: [1/1], global_step: 140, lr: 0.000008, CTCLoss: 51.919825, NRTRLoss: 0.000000, loss: 51.919825, avg_reader_cost: 0.00961 s, avg_batch_cost: 0.08736 s, avg_samples: 32.0, ips: 366.31136 samples/s, eta: 0:00:25, max_mem_reserved: 1045 MB, max_mem_allocated: 672 MB |
| [2026/07/25 19:24:32] ppocr INFO: epoch: [1/1], global_step: 160, lr: 0.000007, CTCLoss: 54.856768, NRTRLoss: 0.000000, loss: 54.856768, avg_reader_cost: 0.00157 s, avg_batch_cost: 0.07820 s, avg_samples: 32.0, ips: 409.21698 samples/s, eta: 0:00:22, max_mem_reserved: 1045 MB, max_mem_allocated: 672 MB |
| [2026/07/25 19:24:35] ppocr INFO: epoch: [1/1], global_step: 180, lr: 0.000006, CTCLoss: 49.938061, NRTRLoss: 0.000000, loss: 49.938061, avg_reader_cost: 0.00169 s, avg_batch_cost: 0.07811 s, avg_samples: 32.0, ips: 409.66564 samples/s, eta: 0:00:20, max_mem_reserved: 1045 MB, max_mem_allocated: 672 MB |
| [2026/07/25 19:24:37] ppocr INFO: epoch: [1/1], global_step: 200, lr: 0.000005, CTCLoss: 53.406744, NRTRLoss: 0.000000, loss: 53.406744, avg_reader_cost: 0.00164 s, avg_batch_cost: 0.07753 s, avg_samples: 32.0, ips: 412.76318 samples/s, eta: 0:00:17, max_mem_reserved: 1045 MB, max_mem_allocated: 672 MB |
|
|
| eval model:: 0%| | 0/128 [00:00<?, ?it/s] |
| eval model:: 1%| | 1/128 [00:00<00:55, 2.30it/s] |
| eval model:: 2%|▏ | 2/128 [00:00<00:31, 3.94it/s] |
| eval model:: 3%|▎ | 4/128 [00:00<00:17, 7.03it/s] |
| eval model:: 5%|▍ | 6/128 [00:00<00:13, 9.01it/s] |
| eval model:: 6%|▋ | 8/128 [00:01<00:11, 10.56it/s] |
| eval model:: 8%|▊ | 10/128 [00:01<00:10, 11.22it/s] |
| eval model:: 9%|▉ | 12/128 [00:01<00:10, 11.50it/s] |
| eval model:: 11%|█ | 14/128 [00:01<00:09, 12.14it/s] |
| eval model:: 12%|█▎ | 16/128 [00:01<00:08, 12.72it/s] |
| eval model:: 14%|█▍ | 18/128 [00:01<00:08, 12.88it/s] |
| eval model:: 16%|█▌ | 20/128 [00:01<00:08, 12.83it/s] |
| eval model:: 17%|█▋ | 22/128 [00:02<00:08, 12.75it/s] |
| eval model:: 19%|█▉ | 24/128 [00:02<00:08, 12.88it/s] |
| eval model:: 20%|██ | 26/128 [00:02<00:07, 13.23it/s] |
| eval model:: 22%|██▏ | 28/128 [00:02<00:07, 13.25it/s] |
| eval model:: 23%|██▎ | 30/128 [00:02<00:07, 12.90it/s] |
| eval model:: 25%|██▌ | 32/128 [00:02<00:07, 13.19it/s] |
| eval model:: 27%|██▋ | 34/128 [00:02<00:06, 13.43it/s] |
| eval model:: 28%|██▊ | 36/128 [00:03<00:06, 13.47it/s] |
| eval model:: 30%|██▉ | 38/128 [00:03<00:06, 13.41it/s] |
| eval model:: 31%|███▏ | 40/128 [00:03<00:06, 13.58it/s] |
| eval model:: 33%|███▎ | 42/128 [00:03<00:06, 13.72it/s] |
| eval model:: 34%|███▍ | 44/128 [00:03<00:06, 13.86it/s] |
| eval model:: 36%|███▌ | 46/128 [00:03<00:05, 13.86it/s] |
| eval model:: 38%|███▊ | 48/128 [00:03<00:05, 13.96it/s] |
| eval model:: 39%|███▉ | 50/128 [00:04<00:05, 13.70it/s] |
| eval model:: 41%|████ | 52/128 [00:04<00:05, 13.46it/s] |
| eval model:: 42%|████▏ | 54/128 [00:04<00:05, 13.23it/s] |
| eval model:: 44%|████▍ | 56/128 [00:04<00:05, 12.93it/s] |
| eval model:: 45%|████▌ | 58/128 [00:04<00:05, 13.33it/s] |
| eval model:: 47%|████▋ | 60/128 [00:04<00:05, 13.12it/s] |
| eval model:: 48%|████▊ | 62/128 [00:05<00:04, 13.46it/s] |
| eval model:: 50%|█████ | 64/128 [00:05<00:04, 12.97it/s] |
| eval model:: 52%|█████▏ | 66/128 [00:05<00:04, 13.36it/s] |
| eval model:: 53%|█████▎ | 68/128 [00:05<00:04, 13.39it/s] |
| eval model:: 55%|█████▍ | 70/128 [00:05<00:04, 13.65it/s] |
| eval model:: 56%|█████▋ | 72/128 [00:05<00:04, 13.63it/s] |
| eval model:: 58%|█████▊ | 74/128 [00:05<00:03, 13.83it/s] |
| eval model:: 59%|█████▉ | 76/128 [00:06<00:03, 13.96it/s] |
| eval model:: 61%|██████ | 78/128 [00:06<00:03, 13.84it/s] |
| eval model:: 62%|██████▎ | 80/128 [00:06<00:03, 13.12it/s] |
| eval model:: 64%|██████▍ | 82/128 [00:06<00:03, 13.49it/s] |
| eval model:: 66%|██████▌ | 84/128 [00:06<00:03, 13.72it/s] |
| eval model:: 67%|██████▋ | 86/128 [00:06<00:03, 13.66it/s] |
| eval model:: 69%|██████▉ | 88/128 [00:06<00:02, 13.84it/s] |
| eval model:: 70%|███████ | 90/128 [00:07<00:02, 13.95it/s] |
| eval model:: 72%|███████▏ | 92/128 [00:07<00:02, 13.03it/s] |
| eval model:: 73%|███████▎ | 94/128 [00:07<00:02, 13.18it/s] |
| eval model:: 75%|███████▌ | 96/128 [00:07<00:02, 12.91it/s] |
| eval model:: 77%|███████▋ | 98/128 [00:07<00:02, 13.04it/s] |
| eval model:: 78%|███████▊ | 100/128 [00:07<00:02, 13.40it/s] |
| eval model:: 80%|███████▉ | 102/128 [00:08<00:01, 13.63it/s] |
| eval model:: 81%|████████▏ | 104/128 [00:08<00:01, 13.81it/s] |
| eval model:: 83%|████████▎ | 106/128 [00:08<00:01, 13.95it/s] |
| eval model:: 84%|████████▍ | 108/128 [00:08<00:01, 14.04it/s] |
| eval model:: 86%|████████▌ | 110/128 [00:08<00:01, 14.06it/s] |
| eval model:: 88%|████████▊ | 112/128 [00:08<00:01, 14.11it/s] |
| eval model:: 89%|████████▉ | 114/128 [00:08<00:00, 14.17it/s] |
| eval model:: 91%|█████████ | 116/128 [00:09<00:00, 13.75it/s] |
| eval model:: 92%|█████████▏| 118/128 [00:09<00:00, 13.38it/s] |
| eval model:: 94%|█████████▍| 120/128 [00:09<00:00, 13.63it/s] |
| eval model:: 95%|█████████▌| 122/128 [00:09<00:00, 13.79it/s] |
| eval model:: 97%|█████████▋| 124/128 [00:09<00:00, 13.64it/s] |
| eval model:: 98%|█████████▊| 126/128 [00:09<00:00, 13.22it/s] |
| eval model:: 100%|██████████| 128/128 [00:09<00:00, 13.28it/s] |
| eval model:: 100%|██████████| 128/128 [00:10<00:00, 12.72it/s] |
| [2026/07/25 19:24:47] ppocr INFO: cur metric, acc: 0.8457031229352951, norm_edit_dis: 0.9877865336005216, fps: 454.11770889368813 |
| [2026/07/25 19:24:47] ppocr INFO: best metric, norm_edit_dis: 0.9878096886547653, is_float16: False, acc: 0.8459472635596991, fps: 451.91959000322055, best_epoch: 1 |
| [2026/07/25 19:24:49] ppocr INFO: epoch: [1/1], global_step: 220, lr: 0.000004, CTCLoss: 48.392014, NRTRLoss: 0.000000, loss: 48.392014, avg_reader_cost: 0.01909 s, avg_batch_cost: 0.09783 s, avg_samples: 32.0, ips: 327.11123 samples/s, eta: 0:00:16, max_mem_reserved: 1045 MB, max_mem_allocated: 672 MB |
| [2026/07/25 19:24:52] ppocr INFO: epoch: [1/1], global_step: 240, lr: 0.000004, CTCLoss: 52.109823, NRTRLoss: 0.000000, loss: 52.109823, avg_reader_cost: 0.02025 s, avg_batch_cost: 0.09756 s, avg_samples: 32.0, ips: 328.01346 samples/s, eta: 0:00:14, max_mem_reserved: 1045 MB, max_mem_allocated: 672 MB |
| [2026/07/25 19:24:54] ppocr INFO: epoch: [1/1], global_step: 260, lr: 0.000003, CTCLoss: 53.730009, NRTRLoss: 0.000000, loss: 53.730009, avg_reader_cost: 0.00170 s, avg_batch_cost: 0.08012 s, avg_samples: 32.0, ips: 399.38068 samples/s, eta: 0:00:12, max_mem_reserved: 1045 MB, max_mem_allocated: 672 MB |
| [2026/07/25 19:24:57] ppocr INFO: epoch: [1/1], global_step: 280, lr: 0.000002, CTCLoss: 53.973894, NRTRLoss: 0.000000, loss: 53.973894, avg_reader_cost: 0.00168 s, avg_batch_cost: 0.07663 s, avg_samples: 32.0, ips: 417.59260 samples/s, eta: 0:00:10, max_mem_reserved: 1045 MB, max_mem_allocated: 672 MB |
| [2026/07/25 19:24:59] ppocr INFO: epoch: [1/1], global_step: 300, lr: 0.000002, CTCLoss: 48.643612, NRTRLoss: 0.000000, loss: 48.643612, avg_reader_cost: 0.00159 s, avg_batch_cost: 0.07643 s, avg_samples: 32.0, ips: 418.68024 samples/s, eta: 0:00:08, max_mem_reserved: 1045 MB, max_mem_allocated: 672 MB |
|
|
| eval model:: 0%| | 0/128 [00:00<?, ?it/s] |
| eval model:: 1%| | 1/128 [00:00<00:52, 2.41it/s] |
| eval model:: 2%|▏ | 2/128 [00:00<00:29, 4.34it/s] |
| eval model:: 3%|▎ | 4/128 [00:00<00:17, 7.21it/s] |
| eval model:: 5%|▍ | 6/128 [00:00<00:13, 9.29it/s] |
| eval model:: 6%|▋ | 8/128 [00:00<00:11, 10.79it/s] |
| eval model:: 8%|▊ | 10/128 [00:01<00:10, 11.66it/s] |
| eval model:: 9%|▉ | 12/128 [00:01<00:09, 12.40it/s] |
| eval model:: 11%|█ | 14/128 [00:01<00:09, 12.61it/s] |
| eval model:: 12%|█▎ | 16/128 [00:01<00:09, 12.26it/s] |
| eval model:: 14%|█▍ | 18/128 [00:01<00:08, 12.73it/s] |
| eval model:: 16%|█▌ | 20/128 [00:01<00:08, 13.05it/s] |
| eval model:: 17%|█▋ | 22/128 [00:02<00:08, 12.79it/s] |
| eval model:: 19%|█▉ | 24/128 [00:02<00:08, 12.74it/s] |
| eval model:: 20%|██ | 26/128 [00:02<00:07, 12.80it/s] |
| eval model:: 22%|██▏ | 28/128 [00:02<00:07, 13.01it/s] |
| eval model:: 23%|██▎ | 30/128 [00:02<00:07, 12.98it/s] |
| eval model:: 25%|██▌ | 32/128 [00:02<00:07, 13.18it/s] |
| eval model:: 27%|██▋ | 34/128 [00:02<00:07, 13.27it/s] |
| eval model:: 28%|██▊ | 36/128 [00:03<00:07, 13.07it/s] |
| eval model:: 30%|██▉ | 38/128 [00:03<00:08, 10.81it/s] |
| eval model:: 31%|███▏ | 40/128 [00:03<00:07, 11.30it/s] |
| eval model:: 33%|███▎ | 42/128 [00:03<00:07, 11.94it/s] |
| eval model:: 34%|███▍ | 44/128 [00:03<00:06, 12.42it/s] |
| eval model:: 36%|███▌ | 46/128 [00:04<00:06, 11.86it/s] |
| eval model:: 38%|███▊ | 48/128 [00:04<00:06, 12.37it/s] |
| eval model:: 39%|███▉ | 50/128 [00:04<00:06, 12.55it/s] |
| eval model:: 41%|████ | 52/128 [00:04<00:05, 12.93it/s] |
| eval model:: 42%|████▏ | 54/128 [00:04<00:05, 13.16it/s] |
| eval model:: 44%|████▍ | 56/128 [00:04<00:05, 13.14it/s] |
| eval model:: 45%|████▌ | 58/128 [00:04<00:05, 13.15it/s] |
| eval model:: 47%|████▋ | 60/128 [00:05<00:05, 13.21it/s] |
| eval model:: 48%|████▊ | 62/128 [00:05<00:04, 13.35it/s] |
| eval model:: 50%|█████ | 64/128 [00:05<00:04, 12.89it/s] |
| eval model:: 52%|█████▏ | 66/128 [00:05<00:04, 13.08it/s] |
| eval model:: 53%|█████▎ | 68/128 [00:05<00:04, 13.29it/s] |
| eval model:: 55%|█████▍ | 70/128 [00:05<00:04, 12.82it/s] |
| eval model:: 56%|█████▋ | 72/128 [00:05<00:04, 12.54it/s] |
| eval model:: 58%|█████▊ | 74/128 [00:06<00:04, 12.61it/s] |
| eval model:: 59%|█████▉ | 76/128 [00:06<00:04, 12.26it/s] |
| eval model:: 61%|██████ | 78/128 [00:06<00:04, 12.48it/s] |
| eval model:: 62%|██████▎ | 80/128 [00:06<00:03, 12.93it/s] |
| eval model:: 64%|██████▍ | 82/128 [00:06<00:03, 13.01it/s] |
| eval model:: 66%|██████▌ | 84/128 [00:06<00:03, 12.98it/s] |
| eval model:: 67%|██████▋ | 86/128 [00:07<00:03, 13.28it/s] |
| eval model:: 69%|██████▉ | 88/128 [00:07<00:02, 13.48it/s] |
| eval model:: 70%|███████ | 90/128 [00:07<00:02, 13.64it/s] |
| eval model:: 72%|███████▏ | 92/128 [00:07<00:02, 12.81it/s] |
| eval model:: 73%|███████▎ | 94/128 [00:07<00:02, 13.16it/s] |
| eval model:: 75%|███████▌ | 96/128 [00:07<00:02, 13.44it/s] |
| eval model:: 77%|███████▋ | 98/128 [00:07<00:02, 13.61it/s] |
| eval model:: 78%|███████▊ | 100/128 [00:08<00:02, 13.74it/s] |
| eval model:: 80%|███████▉ | 102/128 [00:08<00:01, 13.22it/s] |
| eval model:: 81%|████████▏ | 104/128 [00:08<00:01, 13.52it/s] |
| eval model:: 83%|████████▎ | 106/128 [00:08<00:01, 13.73it/s] |
| eval model:: 84%|████████▍ | 108/128 [00:08<00:01, 13.88it/s] |
| eval model:: 86%|████████▌ | 110/128 [00:08<00:01, 13.92it/s] |
| eval model:: 88%|████████▊ | 112/128 [00:08<00:01, 14.00it/s] |
| eval model:: 89%|████████▉ | 114/128 [00:09<00:01, 13.69it/s] |
| eval model:: 91%|█████████ | 116/128 [00:09<00:00, 13.52it/s] |
| eval model:: 92%|█████████▏| 118/128 [00:09<00:00, 13.46it/s] |
| eval model:: 94%|█████████▍| 120/128 [00:09<00:00, 13.66it/s] |
| eval model:: 95%|█████████▌| 122/128 [00:09<00:00, 13.78it/s] |
| eval model:: 97%|█████████▋| 124/128 [00:09<00:00, 13.68it/s] |
| eval model:: 98%|█████████▊| 126/128 [00:10<00:00, 13.23it/s] |
| eval model:: 100%|██████████| 128/128 [00:10<00:00, 13.56it/s] |
| eval model:: 100%|██████████| 128/128 [00:10<00:00, 12.34it/s] |
| [2026/07/25 19:25:10] ppocr INFO: cur metric, acc: 0.8459472635596991, norm_edit_dis: 0.9878475793419357, fps: 440.65064394348906 |
| [2026/07/25 19:25:10] ppocr INFO: save best model is to /workspace/bina02-ppocr/training/stage-010/best_accuracy |
| [2026/07/25 19:25:10] ppocr INFO: best metric, norm_edit_dis: 0.9878475793419357, is_float16: False, acc: 0.8459472635596991, fps: 440.65064394348906, best_epoch: 1 |
| LAUNCH INFO 2026-07-25 19:25:11,341 Terminating with signal 2 |
| LAUNCH INFO 2026-07-25 19:25:11,428 Force quit in 10 seconds... |
|
|
