eval_lighteval|math_500|0.log

INFO 10-26 08:02:52 [__init__.py:235] Automatically detected platform cuda.
[2025-10-26 08:02:53,703] [    INFO]: --- INIT SEEDS --- (pipeline.py:249)
[2025-10-26 08:02:53,704] [    INFO]: --- LOADING TASKS --- (pipeline.py:210)
[2025-10-26 08:02:53,707] [ WARNING]: Careful, the task math_500 is using evaluation data to build the few shot examples. (lighteval_task.py:269)
[2025-10-26 08:02:58,277] [    INFO]: --- LOADING MODEL --- (pipeline.py:177)
`torch_dtype` is deprecated! Use `dtype` instead!
[2025-10-26 08:03:06,028] [    INFO]: Using max model len 32768 (config.py:1604)
[2025-10-26 08:03:06,860] [    INFO]: Chunked prefill is enabled with max_num_batched_tokens=2048. (config.py:2434)
INFO 10-26 08:03:11 [__init__.py:235] Automatically detected platform cuda.
INFO 10-26 08:03:13 [core.py:572] Waiting for init message from front-end.
INFO 10-26 08:03:13 [core.py:71] Initializing a V1 LLM engine (v0.10.0) with config: model='/mnt/public/wucanhui/outputs/Qwen3-4B-math-reasoning/checkpoint-2562', speculative_config=None, tokenizer='/mnt/public/wucanhui/outputs/Qwen3-4B-math-reasoning/checkpoint-2562', skip_tokenizer_init=False, tokenizer_mode=auto, revision=main, override_neuron_config={}, tokenizer_revision=main, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=1234, served_model_name=/mnt/public/wucanhui/outputs/Qwen3-4B-math-reasoning/checkpoint-2562, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, pooler_config=None, compilation_config={"level":0,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":[],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":0,"cudagraph_capture_sizes":[],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":0,"local_cache_dir":null}
INFO 10-26 08:03:17 [parallel_state.py:1102] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
WARNING 10-26 08:03:17 [topk_topp_sampler.py:59] FlashInfer is not available. Falling back to the PyTorch-native implementation of top-p & top-k sampling. For the best performance, please install FlashInfer.
INFO 10-26 08:03:17 [gpu_model_runner.py:1843] Starting to load model /mnt/public/wucanhui/outputs/Qwen3-4B-math-reasoning/checkpoint-2562...
INFO 10-26 08:03:18 [gpu_model_runner.py:1875] Loading model from scratch...
INFO 10-26 08:03:18 [cuda.py:290] Using Flash Attention backend on V1 engine.

Loading safetensors checkpoint shards:   0% Completed | 0/2 [00:00<?, ?it/s]

Loading safetensors checkpoint shards:  50% Completed | 1/2 [00:31<00:31, 31.36s/it]

Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:52<00:00, 25.46s/it]

Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:52<00:00, 26.35s/it]

INFO 10-26 08:04:11 [default_loader.py:262] Loading weights took 53.12 seconds
INFO 10-26 08:04:11 [gpu_model_runner.py:1892] Model loading took 7.5552 GiB and 53.239761 seconds
INFO 10-26 08:04:12 [gpu_worker.py:255] Available KV cache memory: 117.60 GiB
INFO 10-26 08:04:12 [kv_cache_utils.py:833] GPU KV cache size: 856,336 tokens
INFO 10-26 08:04:12 [kv_cache_utils.py:837] Maximum concurrency for 32,768 tokens per request: 26.13x
INFO 10-26 08:04:13 [core.py:193] init engine (profile, create kv cache, warmup model) took 1.36 seconds
[2025-10-26 08:04:13,818] [    INFO]: [CACHING] Initializing data cache (cache_management.py:105)
[2025-10-26 08:04:13,838] [    INFO]: --- RUNNING MODEL --- (pipeline.py:330)
[2025-10-26 08:04:13,839] [    INFO]: Running SamplingMethod.GENERATIVE requests (pipeline.py:313)
[2025-10-26 08:04:30,144] [    INFO]: Cache: Starting to process 500/500 samples (not found in cache) for tasks lighteval|math_500|0 (3aecc7facae3926c, GENERATIVE) (cache_management.py:399)
[2025-10-26 08:04:30,147] [ WARNING]: You cannot select the number of dataset splits for a generative evaluation at the moment. Automatically inferring. (data.py:206)

Splits:   0%|          | 0/1 [00:00<?, ?it/s][2025-10-26 08:04:30,290] [ WARNING]: context_size + max_new_tokens=33645 which is greater than self.max_length=32768. Truncating context to 0 tokens. (vllm_model.py:367)


Adding requests:   0%|          | 0/500 [00:00<?, ?it/s]

Adding requests:  19%|█▉        | 94/500 [00:00<00:01, 337.68it/s]
Adding requests: 100%|██████████| 500/500 [00:00<00:00, 1518.38it/s]


Processed prompts:   0%|          | 0/2000 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]

Processed prompts:   0%|          | 4/2000 [00:06<51:43,  1.55s/it, est. speed input: 257.90 toks/s, output: 199.68 toks/s]

Processed prompts:   0%|          | 8/2000 [00:07<29:34,  1.12it/s, est. speed input: 645.31 toks/s, output: 408.86 toks/s]

Processed prompts:   1%|          | 12/2000 [00:08<19:01,  1.74it/s, est. speed input: 687.81 toks/s, output: 609.39 toks/s]

Processed prompts:   1%|          | 16/2000 [00:09<13:17,  2.49it/s, est. speed input: 759.39 toks/s, output: 824.45 toks/s]

Processed prompts:   1%|          | 20/2000 [00:09<08:52,  3.72it/s, est. speed input: 859.93 toks/s, output: 1031.57 toks/s]

Processed prompts:   1%|▏         | 28/2000 [00:11<08:50,  3.72it/s, est. speed input: 963.35 toks/s, output: 1287.00 toks/s]

Processed prompts:   2%|▏         | 36/2000 [00:13<07:51,  4.17it/s, est. speed input: 1043.30 toks/s, output: 1524.83 toks/s]

Processed prompts:   2%|▏         | 40/2000 [00:14<08:36,  3.79it/s, est. speed input: 1033.96 toks/s, output: 1575.02 toks/s]

Processed prompts:   2%|▏         | 44/2000 [00:16<09:50,  3.31it/s, est. speed input: 992.49 toks/s, output: 1570.79 toks/s] 

Processed prompts:   2%|▏         | 48/2000 [00:18<11:41,  2.78it/s, est. speed input: 926.33 toks/s, output: 1508.05 toks/s]

Processed prompts:   3%|▎         | 52/2000 [00:19<11:46,  2.76it/s, est. speed input: 899.63 toks/s, output: 1534.66 toks/s]

Processed prompts:   3%|▎         | 56/2000 [00:19<08:59,  3.60it/s, est. speed input: 929.94 toks/s, output: 1604.13 toks/s]

Processed prompts:   3%|▎         | 60/2000 [00:20<06:52,  4.70it/s, est. speed input: 977.94 toks/s, output: 1708.55 toks/s]

Processed prompts:   3%|▎         | 68/2000 [00:20<05:02,  6.38it/s, est. speed input: 1018.71 toks/s, output: 1852.37 toks/s]

Processed prompts:   4%|▎         | 72/2000 [00:21<04:32,  7.08it/s, est. speed input: 1037.30 toks/s, output: 1975.24 toks/s]

Processed prompts:   4%|▍         | 76/2000 [00:22<06:38,  4.83it/s, est. speed input: 1011.00 toks/s, output: 1882.17 toks/s]

Processed prompts:   4%|▍         | 80/2000 [00:23<07:08,  4.48it/s, est. speed input: 1037.33 toks/s, output: 1961.76 toks/s]

Processed prompts:   4%|▍         | 84/2000 [00:24<06:37,  4.82it/s, est. speed input: 1046.38 toks/s, output: 1990.89 toks/s]

Processed prompts:   5%|▍         | 92/2000 [00:26<06:14,  5.09it/s, est. speed input: 1060.70 toks/s, output: 2220.66 toks/s]

Processed prompts:   5%|▍         | 96/2000 [00:26<06:05,  5.22it/s, est. speed input: 1069.79 toks/s, output: 2222.24 toks/s]

Processed prompts:   5%|▌         | 100/2000 [00:27<05:28,  5.79it/s, est. speed input: 1078.72 toks/s, output: 2325.20 toks/s]

Processed prompts:   5%|▌         | 104/2000 [00:28<06:28,  4.88it/s, est. speed input: 1057.61 toks/s, output: 2258.86 toks/s]

Processed prompts:   5%|▌         | 108/2000 [00:29<06:15,  5.04it/s, est. speed input: 1054.02 toks/s, output: 2231.88 toks/s]

Processed prompts:   6%|▌         | 112/2000 [00:29<05:19,  5.91it/s, est. speed input: 1077.29 toks/s, output: 2255.28 toks/s]

Processed prompts:   6%|▌         | 116/2000 [00:31<07:26,  4.22it/s, est. speed input: 1052.62 toks/s, output: 2181.19 toks/s]

Processed prompts:   6%|▌         | 120/2000 [00:32<08:38,  3.63it/s, est. speed input: 1024.93 toks/s, output: 2126.65 toks/s]

Processed prompts:   6%|▌         | 124/2000 [00:33<09:03,  3.45it/s, est. speed input: 1007.53 toks/s, output: 2117.89 toks/s]

Processed prompts:   6%|▋         | 128/2000 [00:34<08:44,  3.57it/s, est. speed input: 1007.46 toks/s, output: 2164.47 toks/s]

Processed prompts:   7%|▋         | 132/2000 [00:35<07:36,  4.09it/s, est. speed input: 1016.24 toks/s, output: 2280.76 toks/s]

Processed prompts:   7%|▋         | 140/2000 [00:35<04:23,  7.07it/s, est. speed input: 1055.48 toks/s, output: 2362.42 toks/s]

Processed prompts:   7%|▋         | 144/2000 [00:35<03:44,  8.28it/s, est. speed input: 1080.94 toks/s, output: 2537.78 toks/s]

Processed prompts:   7%|▋         | 148/2000 [00:36<03:53,  7.94it/s, est. speed input: 1093.18 toks/s, output: 2594.10 toks/s]

Processed prompts:   8%|▊         | 152/2000 [00:37<05:18,  5.80it/s, est. speed input: 1075.86 toks/s, output: 2540.40 toks/s]

Processed prompts:   8%|▊         | 156/2000 [00:38<06:02,  5.09it/s, est. speed input: 1064.38 toks/s, output: 2548.32 toks/s]

Processed prompts:   8%|▊         | 160/2000 [00:40<08:29,  3.61it/s, est. speed input: 1031.26 toks/s, output: 2496.48 toks/s]

Processed prompts:   8%|▊         | 164/2000 [00:41<08:00,  3.82it/s, est. speed input: 1023.37 toks/s, output: 2476.21 toks/s]

Processed prompts:   8%|▊         | 168/2000 [00:43<09:01,  3.38it/s, est. speed input: 1002.11 toks/s, output: 2416.49 toks/s]

Processed prompts:   9%|▊         | 172/2000 [00:43<07:18,  4.16it/s, est. speed input: 1008.78 toks/s, output: 2468.73 toks/s]

Processed prompts:   9%|▉         | 176/2000 [00:44<06:22,  4.76it/s, est. speed input: 1011.28 toks/s, output: 2475.89 toks/s]

Processed prompts:   9%|▉         | 180/2000 [00:46<10:13,  2.96it/s, est. speed input: 971.06 toks/s, output: 2414.53 toks/s] 

Processed prompts:   9%|▉         | 184/2000 [00:46<07:39,  3.95it/s, est. speed input: 992.94 toks/s, output: 2546.82 toks/s]

Processed prompts:   9%|▉         | 188/2000 [00:47<06:44,  4.48it/s, est. speed input: 992.89 toks/s, output: 2541.58 toks/s]

Processed prompts:  10%|▉         | 192/2000 [00:49<10:15,  2.94it/s, est. speed input: 956.75 toks/s, output: 2456.80 toks/s]

Processed prompts:  10%|▉         | 196/2000 [00:50<08:10,  3.68it/s, est. speed input: 961.47 toks/s, output: 2461.25 toks/s]

Processed prompts:  10%|█         | 200/2000 [00:50<06:35,  4.55it/s, est. speed input: 969.62 toks/s, output: 2461.41 toks/s]

Processed prompts:  10%|█         | 204/2000 [00:52<07:51,  3.81it/s, est. speed input: 955.30 toks/s, output: 2454.84 toks/s]

Processed prompts:  10%|█         | 208/2000 [00:53<07:49,  3.81it/s, est. speed input: 959.69 toks/s, output: 2525.49 toks/s]

Processed prompts:  11%|█         | 212/2000 [00:54<07:52,  3.78it/s, est. speed input: 954.72 toks/s, output: 2530.46 toks/s]

Processed prompts:  11%|█         | 220/2000 [00:55<06:30,  4.56it/s, est. speed input: 958.57 toks/s, output: 2580.04 toks/s]

Processed prompts:  11%|█         | 224/2000 [00:56<06:00,  4.93it/s, est. speed input: 960.43 toks/s, output: 2584.55 toks/s]

Processed prompts:  11%|█▏        | 228/2000 [00:56<04:56,  5.98it/s, est. speed input: 966.43 toks/s, output: 2598.12 toks/s]

Processed prompts:  12%|█▏        | 232/2000 [00:59<09:11,  3.21it/s, est. speed input: 932.18 toks/s, output: 2494.43 toks/s]

Processed prompts:  12%|█▏        | 236/2000 [01:00<09:28,  3.10it/s, est. speed input: 920.43 toks/s, output: 2460.64 toks/s]

Processed prompts:  12%|█▏        | 240/2000 [01:03<13:02,  2.25it/s, est. speed input: 895.64 toks/s, output: 2444.55 toks/s]

Processed prompts:  12%|█▏        | 244/2000 [01:04<09:58,  2.93it/s, est. speed input: 900.52 toks/s, output: 2456.64 toks/s]

Processed prompts:  12%|█▏        | 248/2000 [01:05<09:30,  3.07it/s, est. speed input: 896.37 toks/s, output: 2442.08 toks/s]

Processed prompts:  13%|█▎        | 252/2000 [01:05<07:00,  4.16it/s, est. speed input: 910.26 toks/s, output: 2490.29 toks/s]

Processed prompts:  13%|█▎        | 256/2000 [01:06<06:48,  4.27it/s, est. speed input: 910.91 toks/s, output: 2486.92 toks/s]

Processed prompts:  13%|█▎        | 264/2000 [01:07<05:28,  5.29it/s, est. speed input: 918.61 toks/s, output: 2523.27 toks/s]

Processed prompts:  13%|█▎        | 268/2000 [01:08<06:20,  4.55it/s, est. speed input: 912.03 toks/s, output: 2537.33 toks/s]

Processed prompts:  14%|█▎        | 272/2000 [01:09<06:49,  4.22it/s, est. speed input: 905.46 toks/s, output: 2521.56 toks/s]

Processed prompts:  14%|█▍        | 276/2000 [01:10<05:52,  4.89it/s, est. speed input: 911.33 toks/s, output: 2556.72 toks/s]

Processed prompts:  14%|█▍        | 280/2000 [01:11<06:42,  4.27it/s, est. speed input: 913.40 toks/s, output: 2667.63 toks/s]

Processed prompts:  14%|█▍        | 284/2000 [01:12<07:16,  3.93it/s, est. speed input: 907.07 toks/s, output: 2634.15 toks/s]

Processed prompts:  14%|█▍        | 288/2000 [01:12<05:34,  5.12it/s, est. speed input: 912.70 toks/s, output: 2641.77 toks/s]

Processed prompts:  15%|█▍        | 292/2000 [01:14<07:39,  3.72it/s, est. speed input: 898.70 toks/s, output: 2611.08 toks/s]

Processed prompts:  15%|█▍        | 296/2000 [01:14<05:38,  5.03it/s, est. speed input: 905.74 toks/s, output: 2640.43 toks/s]

Processed prompts:  15%|█▌        | 300/2000 [01:16<07:02,  4.02it/s, est. speed input: 896.05 toks/s, output: 2608.58 toks/s]

Processed prompts:  15%|█▌        | 304/2000 [01:16<05:35,  5.05it/s, est. speed input: 902.79 toks/s, output: 2617.12 toks/s]

Processed prompts:  15%|█▌        | 308/2000 [01:18<07:31,  3.75it/s, est. speed input: 891.58 toks/s, output: 2608.95 toks/s]

Processed prompts:  16%|█▌        | 312/2000 [01:18<05:31,  5.09it/s, est. speed input: 898.21 toks/s, output: 2627.01 toks/s]

Processed prompts:  16%|█▌        | 316/2000 [01:18<04:43,  5.95it/s, est. speed input: 902.48 toks/s, output: 2633.42 toks/s]

Processed prompts:  16%|█▌        | 320/2000 [01:21<08:45,  3.20it/s, est. speed input: 881.50 toks/s, output: 2565.39 toks/s]

Processed prompts:  16%|█▌        | 324/2000 [01:22<09:06,  3.07it/s, est. speed input: 882.37 toks/s, output: 2624.62 toks/s]

Processed prompts:  16%|█▋        | 328/2000 [01:24<10:29,  2.66it/s, est. speed input: 868.42 toks/s, output: 2576.18 toks/s]

Processed prompts:  17%|█▋        | 332/2000 [01:26<11:52,  2.34it/s, est. speed input: 853.68 toks/s, output: 2523.02 toks/s]

Processed prompts:  17%|█▋        | 336/2000 [01:27<10:03,  2.76it/s, est. speed input: 854.61 toks/s, output: 2529.46 toks/s]

Processed prompts:  17%|█▋        | 340/2000 [01:28<09:09,  3.02it/s, est. speed input: 853.96 toks/s, output: 2568.46 toks/s]

Processed prompts:  17%|█▋        | 344/2000 [01:33<14:59,  1.84it/s, est. speed input: 822.27 toks/s, output: 2482.56 toks/s]

Processed prompts:  17%|█▋        | 348/2000 [01:34<12:48,  2.15it/s, est. speed input: 818.29 toks/s, output: 2464.90 toks/s]

Processed prompts:  18%|█▊        | 352/2000 [01:38<17:22,  1.58it/s, est. speed input: 791.20 toks/s, output: 2379.53 toks/s]

Processed prompts:  18%|█▊        | 356/2000 [01:41<17:47,  1.54it/s, est. speed input: 775.83 toks/s, output: 2327.73 toks/s]

Processed prompts:  18%|█▊        | 360/2000 [01:41<13:52,  1.97it/s, est. speed input: 776.36 toks/s, output: 2328.95 toks/s]

Processed prompts:  18%|█▊        | 364/2000 [01:42<10:39,  2.56it/s, est. speed input: 780.07 toks/s, output: 2354.29 toks/s]

Processed prompts:  18%|█▊        | 368/2000 [01:43<09:17,  2.92it/s, est. speed input: 781.98 toks/s, output: 2401.93 toks/s]

Processed prompts:  19%|█▊        | 372/2000 [01:46<14:21,  1.89it/s, est. speed input: 759.88 toks/s, output: 2346.50 toks/s]

Processed prompts:  19%|█▉        | 380/2000 [01:48<09:47,  2.76it/s, est. speed input: 762.73 toks/s, output: 2380.49 toks/s]

Processed prompts:  19%|█▉        | 384/2000 [01:53<16:46,  1.60it/s, est. speed input: 730.74 toks/s, output: 2327.57 toks/s]

Processed prompts:  19%|█▉        | 388/2000 [01:54<12:37,  2.13it/s, est. speed input: 735.91 toks/s, output: 2351.78 toks/s]

Processed prompts:  20%|█▉        | 392/2000 [01:55<10:53,  2.46it/s, est. speed input: 740.90 toks/s, output: 2406.08 toks/s]

Processed prompts:  20%|█▉        | 396/2000 [01:55<08:48,  3.04it/s, est. speed input: 743.76 toks/s, output: 2417.40 toks/s]

Processed prompts:  20%|██        | 400/2000 [01:57<09:52,  2.70it/s, est. speed input: 742.32 toks/s, output: 2489.13 toks/s]

Processed prompts:  20%|██        | 404/2000 [01:58<08:52,  3.00it/s, est. speed input: 741.18 toks/s, output: 2479.79 toks/s]

Processed prompts:  20%|██        | 408/2000 [01:59<07:36,  3.49it/s, est. speed input: 742.17 toks/s, output: 2486.94 toks/s]

Processed prompts:  21%|██        | 412/2000 [01:59<06:00,  4.41it/s, est. speed input: 746.49 toks/s, output: 2528.48 toks/s]

Processed prompts:  21%|██        | 416/2000 [02:04<14:28,  1.82it/s, est. speed input: 721.40 toks/s, output: 2443.85 toks/s]

Processed prompts:  21%|██        | 420/2000 [02:06<14:16,  1.85it/s, est. speed input: 713.75 toks/s, output: 2412.95 toks/s]

Processed prompts:  21%|██        | 424/2000 [02:07<10:35,  2.48it/s, est. speed input: 716.26 toks/s, output: 2420.90 toks/s]

Processed prompts:  21%|██▏       | 428/2000 [02:09<12:08,  2.16it/s, est. speed input: 710.37 toks/s, output: 2480.25 toks/s]

Processed prompts:  22%|██▏       | 432/2000 [02:09<09:15,  2.82it/s, est. speed input: 712.86 toks/s, output: 2501.71 toks/s]

Processed prompts:  22%|██▏       | 436/2000 [02:10<07:28,  3.49it/s, est. speed input: 714.33 toks/s, output: 2512.19 toks/s]

Processed prompts:  22%|██▏       | 440/2000 [02:11<07:50,  3.32it/s, est. speed input: 712.57 toks/s, output: 2519.80 toks/s]

Processed prompts:  22%|██▏       | 444/2000 [02:11<05:42,  4.54it/s, est. speed input: 716.27 toks/s, output: 2579.57 toks/s]

Processed prompts:  22%|██▏       | 448/2000 [02:17<14:07,  1.83it/s, est. speed input: 701.64 toks/s, output: 2567.66 toks/s]

Processed prompts:  23%|██▎       | 452/2000 [02:17<10:20,  2.50it/s, est. speed input: 704.76 toks/s, output: 2582.03 toks/s]

Processed prompts:  23%|██▎       | 456/2000 [02:19<11:01,  2.33it/s, est. speed input: 699.23 toks/s, output: 2562.63 toks/s]

Processed prompts:  23%|██▎       | 460/2000 [02:20<10:48,  2.37it/s, est. speed input: 696.12 toks/s, output: 2595.51 toks/s]

Processed prompts:  23%|██▎       | 464/2000 [02:23<11:25,  2.24it/s, est. speed input: 690.09 toks/s, output: 2570.50 toks/s]

Processed prompts:  23%|██▎       | 468/2000 [02:24<10:06,  2.53it/s, est. speed input: 689.12 toks/s, output: 2574.14 toks/s]

Processed prompts:  24%|██▎       | 472/2000 [02:24<07:58,  3.20it/s, est. speed input: 691.41 toks/s, output: 2581.51 toks/s]

Processed prompts:  24%|██▍       | 480/2000 [02:25<05:27,  4.64it/s, est. speed input: 701.85 toks/s, output: 2710.05 toks/s]

Processed prompts:  24%|██▍       | 484/2000 [02:25<04:24,  5.74it/s, est. speed input: 704.65 toks/s, output: 2715.82 toks/s]

Processed prompts:  24%|██▍       | 488/2000 [02:26<05:01,  5.02it/s, est. speed input: 705.90 toks/s, output: 2825.83 toks/s]

Processed prompts:  25%|██▍       | 492/2000 [02:29<08:11,  3.07it/s, est. speed input: 696.97 toks/s, output: 2783.87 toks/s]

Processed prompts:  25%|██▍       | 496/2000 [02:33<12:47,  1.96it/s, est. speed input: 682.53 toks/s, output: 2718.39 toks/s]

Processed prompts:  25%|██▌       | 500/2000 [02:36<14:03,  1.78it/s, est. speed input: 674.25 toks/s, output: 2685.08 toks/s]

Processed prompts:  25%|██▌       | 504/2000 [02:36<11:18,  2.21it/s, est. speed input: 674.57 toks/s, output: 2692.16 toks/s]

Processed prompts:  25%|██▌       | 508/2000 [02:37<10:07,  2.45it/s, est. speed input: 674.17 toks/s, output: 2684.69 toks/s]

Processed prompts:  26%|██▌       | 512/2000 [02:38<07:58,  3.11it/s, est. speed input: 676.01 toks/s, output: 2690.49 toks/s]

Processed prompts:  26%|██▌       | 516/2000 [02:39<06:40,  3.71it/s, est. speed input: 678.38 toks/s, output: 2790.41 toks/s]

Processed prompts:  26%|██▌       | 520/2000 [02:40<08:07,  3.03it/s, est. speed input: 673.63 toks/s, output: 2766.73 toks/s]

Processed prompts:  26%|██▌       | 524/2000 [02:44<12:30,  1.97it/s, est. speed input: 661.69 toks/s, output: 2713.06 toks/s]

Processed prompts:  26%|██▋       | 528/2000 [02:44<08:56,  2.74it/s, est. speed input: 664.34 toks/s, output: 2718.94 toks/s]

Processed prompts:  27%|██▋       | 532/2000 [02:46<09:04,  2.70it/s, est. speed input: 661.99 toks/s, output: 2727.12 toks/s]

Processed prompts:  27%|██▋       | 536/2000 [02:47<07:52,  3.10it/s, est. speed input: 662.38 toks/s, output: 2725.09 toks/s]

Processed prompts:  27%|██▋       | 540/2000 [02:48<08:45,  2.78it/s, est. speed input: 661.98 toks/s, output: 2832.95 toks/s]

Processed prompts:  27%|██▋       | 544/2000 [02:50<08:36,  2.82it/s, est. speed input: 660.37 toks/s, output: 2818.73 toks/s]

Processed prompts:  27%|██▋       | 548/2000 [02:51<08:24,  2.88it/s, est. speed input: 658.21 toks/s, output: 2803.08 toks/s]

Processed prompts:  28%|██▊       | 552/2000 [02:52<07:58,  3.03it/s, est. speed input: 658.01 toks/s, output: 2809.46 toks/s]

Processed prompts:  28%|██▊       | 556/2000 [02:53<07:03,  3.41it/s, est. speed input: 658.61 toks/s, output: 2808.46 toks/s]

Processed prompts:  28%|██▊       | 560/2000 [02:57<11:54,  2.02it/s, est. speed input: 647.37 toks/s, output: 2754.07 toks/s]

Processed prompts:  28%|██▊       | 564/2000 [02:58<10:01,  2.39it/s, est. speed input: 647.22 toks/s, output: 2750.00 toks/s]

Processed prompts:  28%|██▊       | 568/2000 [02:59<08:16,  2.89it/s, est. speed input: 647.46 toks/s, output: 2744.96 toks/s]

Processed prompts:  29%|██▊       | 572/2000 [03:00<08:05,  2.94it/s, est. speed input: 647.83 toks/s, output: 2794.23 toks/s]

Processed prompts:  29%|██▉       | 576/2000 [03:00<05:56,  3.99it/s, est. speed input: 650.23 toks/s, output: 2799.46 toks/s]

Processed prompts:  29%|██▉       | 580/2000 [03:01<05:31,  4.28it/s, est. speed input: 654.33 toks/s, output: 2927.13 toks/s]

Processed prompts:  29%|██▉       | 584/2000 [03:05<11:30,  2.05it/s, est. speed input: 641.84 toks/s, output: 2865.93 toks/s]

Processed prompts:  29%|██▉       | 588/2000 [03:07<11:17,  2.09it/s, est. speed input: 652.30 toks/s, output: 2924.53 toks/s]

Processed prompts:  30%|██▉       | 592/2000 [03:11<14:52,  1.58it/s, est. speed input: 642.20 toks/s, output: 2876.55 toks/s]

Processed prompts:  30%|██▉       | 596/2000 [03:11<10:35,  2.21it/s, est. speed input: 644.69 toks/s, output: 2905.43 toks/s]

Processed prompts:  30%|███       | 600/2000 [03:17<17:17,  1.35it/s, est. speed input: 630.24 toks/s, output: 2879.58 toks/s]

Processed prompts:  30%|███       | 604/2000 [03:18<14:00,  1.66it/s, est. speed input: 629.28 toks/s, output: 2875.76 toks/s]

Processed prompts:  30%|███       | 608/2000 [03:20<13:34,  1.71it/s, est. speed input: 625.12 toks/s, output: 2851.24 toks/s]

Processed prompts:  31%|███       | 616/2000 [03:23<11:15,  2.05it/s, est. speed input: 621.98 toks/s, output: 2839.97 toks/s]

Processed prompts:  31%|███       | 620/2000 [03:24<09:03,  2.54it/s, est. speed input: 623.24 toks/s, output: 2840.24 toks/s]

Processed prompts:  31%|███       | 624/2000 [03:24<07:15,  3.16it/s, est. speed input: 625.11 toks/s, output: 2865.52 toks/s]

Processed prompts:  31%|███▏      | 628/2000 [03:24<06:08,  3.73it/s, est. speed input: 626.68 toks/s, output: 2876.67 toks/s]

Processed prompts:  32%|███▏      | 632/2000 [03:25<05:58,  3.82it/s, est. speed input: 626.43 toks/s, output: 2867.36 toks/s]

Processed prompts:  32%|███▏      | 636/2000 [03:27<06:04,  3.74it/s, est. speed input: 626.38 toks/s, output: 2869.97 toks/s]

Processed prompts:  32%|███▏      | 640/2000 [03:31<10:46,  2.10it/s, est. speed input: 618.18 toks/s, output: 2889.11 toks/s]

Processed prompts:  32%|███▏      | 644/2000 [03:31<08:04,  2.80it/s, est. speed input: 619.76 toks/s, output: 2890.17 toks/s]

Processed prompts:  32%|███▏      | 648/2000 [03:33<08:34,  2.63it/s, est. speed input: 617.89 toks/s, output: 2935.85 toks/s]

Processed prompts:  33%|███▎      | 652/2000 [03:36<11:36,  1.94it/s, est. speed input: 613.42 toks/s, output: 2961.05 toks/s]

Processed prompts:  33%|███▎      | 656/2000 [03:38<10:51,  2.06it/s, est. speed input: 611.33 toks/s, output: 2949.99 toks/s]

Processed prompts:  33%|███▎      | 660/2000 [03:38<07:52,  2.83it/s, est. speed input: 613.64 toks/s, output: 2964.23 toks/s]

Processed prompts:  33%|███▎      | 664/2000 [03:39<08:23,  2.65it/s, est. speed input: 611.92 toks/s, output: 2985.83 toks/s]

Processed prompts:  33%|███▎      | 668/2000 [03:40<06:12,  3.58it/s, est. speed input: 613.71 toks/s, output: 2989.86 toks/s]

Processed prompts:  34%|███▎      | 672/2000 [03:43<09:08,  2.42it/s, est. speed input: 608.51 toks/s, output: 2958.31 toks/s]

Processed prompts:  34%|███▍      | 676/2000 [03:44<08:25,  2.62it/s, est. speed input: 607.63 toks/s, output: 2947.55 toks/s]

Processed prompts:  34%|███▍      | 680/2000 [03:44<06:52,  3.20it/s, est. speed input: 608.14 toks/s, output: 2943.10 toks/s]

Processed prompts:  34%|███▍      | 684/2000 [03:47<09:40,  2.27it/s, est. speed input: 603.18 toks/s, output: 2924.69 toks/s]

Processed prompts:  34%|███▍      | 688/2000 [03:49<09:52,  2.21it/s, est. speed input: 600.64 toks/s, output: 2916.06 toks/s]

Processed prompts:  35%|███▍      | 692/2000 [03:50<07:56,  2.74it/s, est. speed input: 601.18 toks/s, output: 2910.97 toks/s]

Processed prompts:  35%|███▍      | 696/2000 [03:52<08:13,  2.64it/s, est. speed input: 599.34 toks/s, output: 2899.24 toks/s]

Processed prompts:  35%|███▌      | 700/2000 [03:52<05:55,  3.65it/s, est. speed input: 602.12 toks/s, output: 2938.91 toks/s]

Processed prompts:  35%|███▌      | 708/2000 [03:53<05:10,  4.16it/s, est. speed input: 602.82 toks/s, output: 2937.99 toks/s]

Processed prompts:  36%|███▌      | 712/2000 [03:59<11:13,  1.91it/s, est. speed input: 592.04 toks/s, output: 2929.66 toks/s]

Processed prompts:  36%|███▌      | 720/2000 [03:59<07:02,  3.03it/s, est. speed input: 596.10 toks/s, output: 2950.49 toks/s]

Processed prompts:  36%|███▌      | 724/2000 [04:00<06:53,  3.08it/s, est. speed input: 595.13 toks/s, output: 2943.49 toks/s]

Processed prompts:  36%|███▋      | 728/2000 [04:01<05:40,  3.74it/s, est. speed input: 596.24 toks/s, output: 2944.82 toks/s]

Processed prompts:  37%|███▋      | 732/2000 [04:03<06:38,  3.18it/s, est. speed input: 594.49 toks/s, output: 2945.35 toks/s]

Processed prompts:  37%|███▋      | 736/2000 [04:04<06:17,  3.35it/s, est. speed input: 594.35 toks/s, output: 2947.90 toks/s]

Processed prompts:  37%|███▋      | 740/2000 [04:07<09:26,  2.23it/s, est. speed input: 588.60 toks/s, output: 2930.28 toks/s]

Processed prompts:  37%|███▋      | 748/2000 [04:07<05:23,  3.87it/s, est. speed input: 593.11 toks/s, output: 2942.40 toks/s]

Processed prompts:  38%|███▊      | 752/2000 [04:08<04:49,  4.32it/s, est. speed input: 595.49 toks/s, output: 3009.39 toks/s]

Processed prompts:  38%|███▊      | 756/2000 [04:08<04:09,  4.98it/s, est. speed input: 596.65 toks/s, output: 3013.89 toks/s]

Processed prompts:  38%|███▊      | 760/2000 [04:08<03:14,  6.38it/s, est. speed input: 598.27 toks/s, output: 3017.24 toks/s]

Processed prompts:  38%|███▊      | 764/2000 [04:13<08:25,  2.45it/s, est. speed input: 590.27 toks/s, output: 2970.57 toks/s]

Processed prompts:  38%|███▊      | 768/2000 [04:13<06:38,  3.09it/s, est. speed input: 591.46 toks/s, output: 2972.87 toks/s]

Processed prompts:  39%|███▊      | 772/2000 [04:15<06:59,  2.93it/s, est. speed input: 590.22 toks/s, output: 3026.64 toks/s]

Processed prompts:  39%|███▉      | 776/2000 [04:19<11:48,  1.73it/s, est. speed input: 581.77 toks/s, output: 2979.64 toks/s]

Processed prompts:  39%|███▉      | 780/2000 [04:20<09:28,  2.14it/s, est. speed input: 582.50 toks/s, output: 2979.30 toks/s]

Processed prompts:  39%|███▉      | 784/2000 [04:21<07:36,  2.67it/s, est. speed input: 586.66 toks/s, output: 3064.31 toks/s]

Processed prompts:  40%|███▉      | 792/2000 [04:29<14:01,  1.44it/s, est. speed input: 571.74 toks/s, output: 2981.88 toks/s]

Processed prompts:  40%|███▉      | 796/2000 [04:30<10:54,  1.84it/s, est. speed input: 572.87 toks/s, output: 2982.93 toks/s]

Processed prompts:  40%|████      | 800/2000 [04:30<08:40,  2.30it/s, est. speed input: 573.67 toks/s, output: 2982.86 toks/s]

Processed prompts:  40%|████      | 804/2000 [04:31<08:01,  2.48it/s, est. speed input: 572.96 toks/s, output: 2972.97 toks/s]

Processed prompts:  40%|████      | 808/2000 [04:32<06:30,  3.05it/s, est. speed input: 573.99 toks/s, output: 2976.25 toks/s]

Processed prompts:  41%|████      | 812/2000 [04:33<06:18,  3.14it/s, est. speed input: 573.40 toks/s, output: 2981.48 toks/s]

Processed prompts:  41%|████      | 816/2000 [04:35<07:18,  2.70it/s, est. speed input: 571.05 toks/s, output: 2967.78 toks/s]

Processed prompts:  41%|████      | 820/2000 [04:35<05:45,  3.41it/s, est. speed input: 571.89 toks/s, output: 2966.55 toks/s]

Processed prompts:  41%|████      | 824/2000 [04:36<05:23,  3.63it/s, est. speed input: 571.80 toks/s, output: 2962.97 toks/s]

Processed prompts:  41%|████▏     | 828/2000 [04:43<13:21,  1.46it/s, est. speed input: 560.63 toks/s, output: 2902.23 toks/s]

Processed prompts:  42%|████▏     | 832/2000 [04:44<11:23,  1.71it/s, est. speed input: 559.82 toks/s, output: 2899.00 toks/s]

Processed prompts:  42%|████▏     | 836/2000 [04:47<11:03,  1.75it/s, est. speed input: 563.27 toks/s, output: 2967.50 toks/s]

Processed prompts:  42%|████▏     | 840/2000 [04:48<10:12,  1.89it/s, est. speed input: 561.82 toks/s, output: 2959.86 toks/s]

Processed prompts:  42%|████▏     | 844/2000 [04:56<18:36,  1.04it/s, est. speed input: 548.70 toks/s, output: 2896.05 toks/s]

Processed prompts:  42%|████▏     | 848/2000 [04:57<13:35,  1.41it/s, est. speed input: 549.55 toks/s, output: 2897.21 toks/s]

Processed prompts:  43%|████▎     | 852/2000 [04:57<10:13,  1.87it/s, est. speed input: 553.06 toks/s, output: 3043.43 toks/s]

Processed prompts:  43%|████▎     | 856/2000 [04:59<10:27,  1.82it/s, est. speed input: 550.32 toks/s, output: 3024.83 toks/s]

Processed prompts:  43%|████▎     | 860/2000 [05:05<14:44,  1.29it/s, est. speed input: 542.56 toks/s, output: 2978.51 toks/s]

Processed prompts:  43%|████▎     | 864/2000 [05:05<10:42,  1.77it/s, est. speed input: 543.76 toks/s, output: 2994.87 toks/s]

Processed prompts:  43%|████▎     | 868/2000 [05:11<15:25,  1.22it/s, est. speed input: 535.58 toks/s, output: 2953.36 toks/s]

Processed prompts:  44%|████▎     | 872/2000 [05:13<13:35,  1.38it/s, est. speed input: 533.63 toks/s, output: 2937.24 toks/s]

Processed prompts:  44%|████▍     | 876/2000 [05:13<09:42,  1.93it/s, est. speed input: 535.20 toks/s, output: 2952.54 toks/s]

Processed prompts:  44%|████▍     | 880/2000 [05:14<08:41,  2.15it/s, est. speed input: 534.45 toks/s, output: 2944.92 toks/s]

Processed prompts:  44%|████▍     | 884/2000 [05:16<09:10,  2.03it/s, est. speed input: 532.17 toks/s, output: 2930.46 toks/s]

Processed prompts:  44%|████▍     | 888/2000 [05:18<09:07,  2.03it/s, est. speed input: 530.47 toks/s, output: 2922.78 toks/s]

Processed prompts:  45%|████▍     | 892/2000 [05:21<10:29,  1.76it/s, est. speed input: 527.36 toks/s, output: 2901.28 toks/s]

Processed prompts:  45%|████▍     | 896/2000 [05:22<08:52,  2.07it/s, est. speed input: 527.37 toks/s, output: 2894.65 toks/s]

Processed prompts:  45%|████▌     | 900/2000 [05:23<07:33,  2.43it/s, est. speed input: 527.32 toks/s, output: 2889.91 toks/s]

Processed prompts:  45%|████▌     | 904/2000 [05:25<06:52,  2.65it/s, est. speed input: 527.00 toks/s, output: 2882.51 toks/s]

Processed prompts:  45%|████▌     | 908/2000 [05:28<09:02,  2.01it/s, est. speed input: 531.19 toks/s, output: 3001.45 toks/s]

Processed prompts:  46%|████▌     | 912/2000 [05:32<11:42,  1.55it/s, est. speed input: 526.54 toks/s, output: 2975.53 toks/s]

Processed prompts:  46%|████▌     | 916/2000 [05:32<08:50,  2.04it/s, est. speed input: 527.54 toks/s, output: 2981.82 toks/s]

Processed prompts:  46%|████▌     | 920/2000 [05:33<06:40,  2.70it/s, est. speed input: 528.37 toks/s, output: 2982.28 toks/s]

Processed prompts:  46%|████▌     | 924/2000 [05:34<06:41,  2.68it/s, est. speed input: 527.53 toks/s, output: 2974.72 toks/s]

Processed prompts:  47%|████▋     | 932/2000 [05:38<07:25,  2.40it/s, est. speed input: 525.11 toks/s, output: 2988.86 toks/s]

Processed prompts:  47%|████▋     | 936/2000 [05:38<05:54,  3.00it/s, est. speed input: 526.43 toks/s, output: 2991.12 toks/s]

Processed prompts:  47%|████▋     | 944/2000 [05:40<05:13,  3.37it/s, est. speed input: 526.52 toks/s, output: 2990.26 toks/s]

Processed prompts:  47%|████▋     | 948/2000 [05:41<05:16,  3.33it/s, est. speed input: 526.00 toks/s, output: 2982.41 toks/s]

Processed prompts:  48%|████▊     | 952/2000 [05:42<04:17,  4.07it/s, est. speed input: 526.97 toks/s, output: 2981.84 toks/s]

Processed prompts:  48%|████▊     | 956/2000 [05:46<07:38,  2.28it/s, est. speed input: 522.70 toks/s, output: 2962.83 toks/s]

Processed prompts:  48%|████▊     | 960/2000 [05:47<07:40,  2.26it/s, est. speed input: 521.42 toks/s, output: 2950.10 toks/s]

Processed prompts:  48%|████▊     | 964/2000 [05:51<10:08,  1.70it/s, est. speed input: 517.27 toks/s, output: 2922.08 toks/s]

Processed prompts:  48%|████▊     | 968/2000 [05:53<09:30,  1.81it/s, est. speed input: 516.12 toks/s, output: 2922.37 toks/s]

Processed prompts:  49%|████▊     | 972/2000 [05:54<07:21,  2.33it/s, est. speed input: 517.00 toks/s, output: 2923.48 toks/s]

Processed prompts:  49%|████▉     | 976/2000 [05:55<07:17,  2.34it/s, est. speed input: 516.07 toks/s, output: 2938.36 toks/s]

Processed prompts:  49%|████▉     | 980/2000 [05:56<05:57,  2.85it/s, est. speed input: 516.55 toks/s, output: 2937.13 toks/s]

Processed prompts:  49%|████▉     | 984/2000 [05:58<06:35,  2.57it/s, est. speed input: 515.09 toks/s, output: 2925.31 toks/s]

Processed prompts:  49%|████▉     | 988/2000 [06:00<07:36,  2.22it/s, est. speed input: 513.35 toks/s, output: 2915.42 toks/s]

Processed prompts:  50%|████▉     | 992/2000 [06:03<08:07,  2.07it/s, est. speed input: 511.78 toks/s, output: 2901.73 toks/s]

Processed prompts:  50%|████▉     | 996/2000 [06:06<09:57,  1.68it/s, est. speed input: 508.50 toks/s, output: 2879.52 toks/s]

Processed prompts:  50%|█████     | 1000/2000 [06:07<08:26,  1.97it/s, est. speed input: 508.51 toks/s, output: 2883.53 toks/s]

Processed prompts:  50%|█████     | 1004/2000 [06:11<10:05,  1.65it/s, est. speed input: 505.08 toks/s, output: 2859.79 toks/s]

Processed prompts:  50%|█████     | 1008/2000 [06:12<09:07,  1.81it/s, est. speed input: 504.11 toks/s, output: 2852.85 toks/s]

Processed prompts:  51%|█████     | 1012/2000 [06:13<07:38,  2.16it/s, est. speed input: 503.88 toks/s, output: 2849.40 toks/s]

Processed prompts:  51%|█████     | 1016/2000 [06:14<05:51,  2.80it/s, est. speed input: 504.93 toks/s, output: 2860.58 toks/s]

Processed prompts:  51%|█████     | 1020/2000 [06:17<07:47,  2.10it/s, est. speed input: 502.21 toks/s, output: 2843.06 toks/s]

Processed prompts:  51%|█████     | 1024/2000 [06:21<10:45,  1.51it/s, est. speed input: 498.01 toks/s, output: 2819.06 toks/s]

Processed prompts:  51%|█████▏    | 1028/2000 [06:23<09:15,  1.75it/s, est. speed input: 497.47 toks/s, output: 2816.43 toks/s]

Processed prompts:  52%|█████▏    | 1032/2000 [06:29<13:58,  1.15it/s, est. speed input: 491.06 toks/s, output: 2783.01 toks/s]

Processed prompts:  52%|█████▏    | 1036/2000 [06:30<11:10,  1.44it/s, est. speed input: 490.78 toks/s, output: 2777.76 toks/s]

Processed prompts:  52%|█████▏    | 1040/2000 [06:30<08:11,  1.95it/s, est. speed input: 491.94 toks/s, output: 2795.72 toks/s]

Processed prompts:  52%|█████▏    | 1044/2000 [06:31<05:57,  2.68it/s, est. speed input: 492.92 toks/s, output: 2798.69 toks/s]

Processed prompts:  52%|█████▏    | 1048/2000 [06:37<12:06,  1.31it/s, est. speed input: 486.28 toks/s, output: 2761.81 toks/s]

Processed prompts:  53%|█████▎    | 1052/2000 [06:38<09:34,  1.65it/s, est. speed input: 486.29 toks/s, output: 2757.81 toks/s]

Processed prompts:  53%|█████▎    | 1056/2000 [06:38<06:57,  2.26it/s, est. speed input: 487.20 toks/s, output: 2759.40 toks/s]

Processed prompts:  53%|█████▎    | 1060/2000 [06:41<07:55,  1.98it/s, est. speed input: 485.64 toks/s, output: 2748.04 toks/s]

Processed prompts:  53%|█████▎    | 1064/2000 [06:42<06:40,  2.34it/s, est. speed input: 485.80 toks/s, output: 2744.03 toks/s]

Processed prompts:  53%|█████▎    | 1068/2000 [06:48<11:29,  1.35it/s, est. speed input: 480.56 toks/s, output: 2754.57 toks/s]

Processed prompts:  54%|█████▎    | 1072/2000 [06:48<08:29,  1.82it/s, est. speed input: 481.25 toks/s, output: 2754.39 toks/s]

Processed prompts:  54%|█████▍    | 1076/2000 [06:51<08:26,  1.82it/s, est. speed input: 479.98 toks/s, output: 2742.26 toks/s]

Processed prompts:  54%|█████▍    | 1080/2000 [06:52<07:34,  2.02it/s, est. speed input: 479.58 toks/s, output: 2734.62 toks/s]

Processed prompts:  54%|█████▍    | 1084/2000 [06:57<10:39,  1.43it/s, est. speed input: 475.39 toks/s, output: 2709.87 toks/s]

Processed prompts:  54%|█████▍    | 1088/2000 [06:57<08:05,  1.88it/s, est. speed input: 475.87 toks/s, output: 2708.33 toks/s]

Processed prompts:  55%|█████▍    | 1092/2000 [07:03<11:53,  1.27it/s, est. speed input: 471.00 toks/s, output: 2675.68 toks/s]

Processed prompts:  55%|█████▍    | 1096/2000 [07:07<13:36,  1.11it/s, est. speed input: 467.14 toks/s, output: 2650.38 toks/s]

Processed prompts:  55%|█████▌    | 1100/2000 [07:10<12:02,  1.25it/s, est. speed input: 466.15 toks/s, output: 2658.25 toks/s]

Processed prompts:  55%|█████▌    | 1104/2000 [07:10<08:30,  1.75it/s, est. speed input: 467.22 toks/s, output: 2664.64 toks/s]

Processed prompts:  55%|█████▌    | 1108/2000 [07:11<07:15,  2.05it/s, est. speed input: 467.40 toks/s, output: 2708.12 toks/s]

Processed prompts:  56%|█████▌    | 1112/2000 [07:13<07:25,  1.99it/s, est. speed input: 466.20 toks/s, output: 2699.59 toks/s]

Processed prompts:  56%|█████▌    | 1116/2000 [07:15<07:44,  1.90it/s, est. speed input: 464.94 toks/s, output: 2688.81 toks/s]

Processed prompts:  56%|█████▌    | 1120/2000 [07:18<08:08,  1.80it/s, est. speed input: 463.52 toks/s, output: 2676.33 toks/s]

Processed prompts:  56%|█████▌    | 1124/2000 [07:18<06:06,  2.39it/s, est. speed input: 464.14 toks/s, output: 2677.06 toks/s]

Processed prompts:  56%|█████▋    | 1128/2000 [07:19<04:56,  2.94it/s, est. speed input: 464.70 toks/s, output: 2680.51 toks/s]

Processed prompts:  57%|█████▋    | 1132/2000 [07:19<03:49,  3.78it/s, est. speed input: 465.64 toks/s, output: 2689.95 toks/s]

Processed prompts:  57%|█████▋    | 1136/2000 [07:21<04:37,  3.11it/s, est. speed input: 464.80 toks/s, output: 2686.19 toks/s]

Processed prompts:  57%|█████▋    | 1140/2000 [07:22<04:15,  3.36it/s, est. speed input: 464.83 toks/s, output: 2682.68 toks/s]

Processed prompts:  57%|█████▋    | 1144/2000 [07:23<03:25,  4.17it/s, est. speed input: 465.49 toks/s, output: 2685.35 toks/s]

Processed prompts:  57%|█████▋    | 1148/2000 [07:24<03:36,  3.93it/s, est. speed input: 465.37 toks/s, output: 2682.56 toks/s]

Processed prompts:  58%|█████▊    | 1152/2000 [07:24<03:05,  4.57it/s, est. speed input: 466.05 toks/s, output: 2681.47 toks/s]

Processed prompts:  58%|█████▊    | 1156/2000 [07:27<04:43,  2.97it/s, est. speed input: 464.69 toks/s, output: 2690.64 toks/s]

Processed prompts:  58%|█████▊    | 1160/2000 [07:27<03:53,  3.59it/s, est. speed input: 465.25 toks/s, output: 2689.05 toks/s]

Processed prompts:  58%|█████▊    | 1164/2000 [07:30<05:36,  2.48it/s, est. speed input: 463.54 toks/s, output: 2680.70 toks/s]

Processed prompts:  58%|█████▊    | 1168/2000 [07:37<11:34,  1.20it/s, est. speed input: 457.13 toks/s, output: 2641.70 toks/s]

Processed prompts:  59%|█████▊    | 1172/2000 [07:38<08:42,  1.58it/s, est. speed input: 457.51 toks/s, output: 2641.86 toks/s]

Processed prompts:  59%|█████▉    | 1176/2000 [07:39<06:44,  2.04it/s, est. speed input: 457.98 toks/s, output: 2642.92 toks/s]

Processed prompts:  59%|█████▉    | 1180/2000 [07:39<05:17,  2.58it/s, est. speed input: 458.45 toks/s, output: 2643.08 toks/s]

Processed prompts:  59%|█████▉    | 1184/2000 [07:41<05:21,  2.54it/s, est. speed input: 457.97 toks/s, output: 2638.96 toks/s]

Processed prompts:  59%|█████▉    | 1188/2000 [07:44<06:30,  2.08it/s, est. speed input: 456.27 toks/s, output: 2627.03 toks/s]

Processed prompts:  60%|█████▉    | 1192/2000 [07:45<05:56,  2.27it/s, est. speed input: 456.13 toks/s, output: 2625.37 toks/s]

Processed prompts:  60%|█████▉    | 1196/2000 [07:45<04:23,  3.05it/s, est. speed input: 457.28 toks/s, output: 2650.07 toks/s]

Processed prompts:  60%|██████    | 1200/2000 [07:49<06:45,  1.97it/s, est. speed input: 454.70 toks/s, output: 2634.69 toks/s]

Processed prompts:  60%|██████    | 1204/2000 [07:50<05:36,  2.36it/s, est. speed input: 454.92 toks/s, output: 2642.85 toks/s]

Processed prompts:  60%|██████    | 1208/2000 [07:51<05:28,  2.41it/s, est. speed input: 454.33 toks/s, output: 2636.37 toks/s]

Processed prompts:  61%|██████    | 1212/2000 [07:52<04:18,  3.05it/s, est. speed input: 454.82 toks/s, output: 2636.45 toks/s]

Processed prompts:  61%|██████    | 1216/2000 [07:53<03:56,  3.31it/s, est. speed input: 455.27 toks/s, output: 2683.19 toks/s]

Processed prompts:  61%|██████    | 1220/2000 [07:54<03:59,  3.25it/s, est. speed input: 456.45 toks/s, output: 2787.91 toks/s]

Processed prompts:  61%|██████    | 1224/2000 [07:57<05:34,  2.32it/s, est. speed input: 456.10 toks/s, output: 2856.71 toks/s]

Processed prompts:  61%|██████▏   | 1228/2000 [07:58<05:02,  2.55it/s, est. speed input: 455.84 toks/s, output: 2853.73 toks/s]

Processed prompts:  62%|██████▏   | 1232/2000 [07:59<03:50,  3.33it/s, est. speed input: 456.69 toks/s, output: 2855.20 toks/s]

Processed prompts:  62%|██████▏   | 1236/2000 [08:00<03:35,  3.54it/s, est. speed input: 456.69 toks/s, output: 2852.59 toks/s]

Processed prompts:  62%|██████▏   | 1240/2000 [08:03<05:44,  2.21it/s, est. speed input: 454.48 toks/s, output: 2836.22 toks/s]

Processed prompts:  62%|██████▏   | 1244/2000 [08:07<07:47,  1.62it/s, est. speed input: 451.77 toks/s, output: 2814.77 toks/s]

Processed prompts:  62%|██████▏   | 1248/2000 [08:08<06:38,  1.89it/s, est. speed input: 451.59 toks/s, output: 2810.54 toks/s]

Processed prompts:  63%|██████▎   | 1252/2000 [08:10<06:03,  2.06it/s, est. speed input: 451.07 toks/s, output: 2804.49 toks/s]

Processed prompts:  63%|██████▎   | 1256/2000 [08:11<04:57,  2.50it/s, est. speed input: 451.32 toks/s, output: 2805.06 toks/s]

Processed prompts:  63%|██████▎   | 1264/2000 [08:11<02:56,  4.17it/s, est. speed input: 452.95 toks/s, output: 2809.01 toks/s]

Processed prompts:  63%|██████▎   | 1268/2000 [08:18<07:41,  1.59it/s, est. speed input: 447.35 toks/s, output: 2772.57 toks/s]

Processed prompts:  64%|██████▎   | 1272/2000 [08:19<05:59,  2.02it/s, est. speed input: 447.86 toks/s, output: 2772.41 toks/s]

Processed prompts:  64%|██████▍   | 1276/2000 [08:20<05:21,  2.25it/s, est. speed input: 447.74 toks/s, output: 2768.46 toks/s]

Processed prompts:  64%|██████▍   | 1280/2000 [08:21<04:30,  2.66it/s, est. speed input: 447.96 toks/s, output: 2771.25 toks/s]

Processed prompts:  64%|██████▍   | 1284/2000 [08:21<03:29,  3.42it/s, est. speed input: 448.61 toks/s, output: 2773.84 toks/s]

Processed prompts:  64%|██████▍   | 1288/2000 [08:24<04:33,  2.61it/s, est. speed input: 447.49 toks/s, output: 2762.90 toks/s]

Processed prompts:  65%|██████▍   | 1292/2000 [08:24<03:40,  3.21it/s, est. speed input: 448.00 toks/s, output: 2769.49 toks/s]

Processed prompts:  65%|██████▍   | 1296/2000 [08:25<02:53,  4.06it/s, est. speed input: 448.63 toks/s, output: 2770.67 toks/s]

Processed prompts:  65%|██████▌   | 1300/2000 [08:28<04:37,  2.53it/s, est. speed input: 446.98 toks/s, output: 2757.08 toks/s]

Processed prompts:  65%|██████▌   | 1304/2000 [08:28<04:03,  2.86it/s, est. speed input: 447.20 toks/s, output: 2758.09 toks/s]

Processed prompts:  65%|██████▌   | 1308/2000 [08:29<03:00,  3.82it/s, est. speed input: 447.95 toks/s, output: 2761.18 toks/s]

Processed prompts:  66%|██████▌   | 1312/2000 [08:30<03:17,  3.48it/s, est. speed input: 447.72 toks/s, output: 2756.70 toks/s]

Processed prompts:  66%|██████▌   | 1316/2000 [08:31<02:40,  4.26it/s, est. speed input: 448.20 toks/s, output: 2759.77 toks/s]

Processed prompts:  66%|██████▌   | 1320/2000 [08:34<04:51,  2.34it/s, est. speed input: 446.15 toks/s, output: 2744.99 toks/s]

Processed prompts:  66%|██████▌   | 1324/2000 [08:34<03:43,  3.02it/s, est. speed input: 446.75 toks/s, output: 2745.50 toks/s]

Processed prompts:  66%|██████▋   | 1328/2000 [08:36<03:57,  2.84it/s, est. speed input: 446.32 toks/s, output: 2740.42 toks/s]

Processed prompts:  67%|██████▋   | 1332/2000 [08:37<03:21,  3.31it/s, est. speed input: 446.64 toks/s, output: 2739.68 toks/s]

Processed prompts:  67%|██████▋   | 1336/2000 [08:37<02:35,  4.27it/s, est. speed input: 447.23 toks/s, output: 2740.52 toks/s]

Processed prompts:  67%|██████▋   | 1340/2000 [08:38<02:40,  4.10it/s, est. speed input: 447.13 toks/s, output: 2736.82 toks/s]

Processed prompts:  67%|██████▋   | 1344/2000 [08:38<02:04,  5.25it/s, est. speed input: 447.74 toks/s, output: 2736.94 toks/s]

Processed prompts:  67%|██████▋   | 1348/2000 [08:46<07:16,  1.49it/s, est. speed input: 442.55 toks/s, output: 2703.60 toks/s]

Processed prompts:  68%|██████▊   | 1352/2000 [08:47<06:14,  1.73it/s, est. speed input: 442.30 toks/s, output: 2715.78 toks/s]

Processed prompts:  68%|██████▊   | 1356/2000 [08:48<05:00,  2.14it/s, est. speed input: 442.59 toks/s, output: 2715.99 toks/s]

Processed prompts:  68%|██████▊   | 1360/2000 [08:50<05:13,  2.04it/s, est. speed input: 441.60 toks/s, output: 2707.22 toks/s]

Processed prompts:  68%|██████▊   | 1364/2000 [08:52<04:58,  2.13it/s, est. speed input: 441.05 toks/s, output: 2702.58 toks/s]

Processed prompts:  68%|██████▊   | 1368/2000 [08:56<06:52,  1.53it/s, est. speed input: 438.41 toks/s, output: 2683.76 toks/s]

Processed prompts:  69%|██████▊   | 1372/2000 [08:57<05:19,  1.97it/s, est. speed input: 438.71 toks/s, output: 2682.10 toks/s]

Processed prompts:  69%|██████▉   | 1376/2000 [09:00<05:55,  1.76it/s, est. speed input: 437.30 toks/s, output: 2673.25 toks/s]

Processed prompts:  69%|██████▉   | 1384/2000 [09:02<04:23,  2.34it/s, est. speed input: 437.84 toks/s, output: 2725.49 toks/s]

Processed prompts:  69%|██████▉   | 1388/2000 [09:04<04:31,  2.25it/s, est. speed input: 437.07 toks/s, output: 2716.85 toks/s]

Processed prompts:  70%|██████▉   | 1392/2000 [09:06<04:41,  2.16it/s, est. speed input: 436.29 toks/s, output: 2711.56 toks/s]

Processed prompts:  70%|██████▉   | 1396/2000 [09:06<03:40,  2.74it/s, est. speed input: 436.88 toks/s, output: 2712.26 toks/s]

Processed prompts:  70%|███████   | 1400/2000 [09:07<03:27,  2.89it/s, est. speed input: 436.89 toks/s, output: 2711.15 toks/s]

Processed prompts:  70%|███████   | 1404/2000 [09:08<02:54,  3.41it/s, est. speed input: 437.42 toks/s, output: 2714.14 toks/s]

Processed prompts:  70%|███████   | 1408/2000 [09:08<02:24,  4.10it/s, est. speed input: 437.79 toks/s, output: 2713.20 toks/s]

Processed prompts:  71%|███████   | 1412/2000 [09:10<02:34,  3.80it/s, est. speed input: 437.60 toks/s, output: 2722.28 toks/s]

Processed prompts:  71%|███████   | 1416/2000 [09:10<02:14,  4.34it/s, est. speed input: 437.94 toks/s, output: 2721.01 toks/s]

Processed prompts:  71%|███████   | 1420/2000 [09:11<02:20,  4.12it/s, est. speed input: 437.93 toks/s, output: 2717.99 toks/s]

Processed prompts:  71%|███████   | 1424/2000 [09:13<02:40,  3.59it/s, est. speed input: 438.17 toks/s, output: 2772.36 toks/s]

Processed prompts:  71%|███████▏  | 1428/2000 [09:13<01:59,  4.79it/s, est. speed input: 438.90 toks/s, output: 2775.74 toks/s]

Processed prompts:  72%|███████▏  | 1432/2000 [09:13<01:41,  5.58it/s, est. speed input: 439.37 toks/s, output: 2775.96 toks/s]

Processed prompts:  72%|███████▏  | 1436/2000 [09:15<02:25,  3.88it/s, est. speed input: 438.98 toks/s, output: 2787.08 toks/s]

Processed prompts:  72%|███████▏  | 1440/2000 [09:17<02:34,  3.62it/s, est. speed input: 438.82 toks/s, output: 2783.31 toks/s]

Processed prompts:  72%|███████▏  | 1444/2000 [09:18<02:38,  3.52it/s, est. speed input: 438.71 toks/s, output: 2779.13 toks/s]

Processed prompts:  72%|███████▏  | 1448/2000 [09:18<02:02,  4.52it/s, est. speed input: 439.34 toks/s, output: 2779.12 toks/s]

Processed prompts:  73%|███████▎  | 1452/2000 [09:18<01:29,  6.13it/s, est. speed input: 440.09 toks/s, output: 2781.10 toks/s]

Processed prompts:  73%|███████▎  | 1456/2000 [09:19<01:26,  6.32it/s, est. speed input: 440.47 toks/s, output: 2779.96 toks/s]

Processed prompts:  73%|███████▎  | 1460/2000 [09:19<01:20,  6.70it/s, est. speed input: 440.92 toks/s, output: 2779.98 toks/s]

Processed prompts:  73%|███████▎  | 1464/2000 [09:23<03:33,  2.52it/s, est. speed input: 438.69 toks/s, output: 2761.90 toks/s]

Processed prompts:  74%|███████▎  | 1472/2000 [09:24<02:30,  3.52it/s, est. speed input: 439.30 toks/s, output: 2761.79 toks/s]

Processed prompts:  74%|███████▍  | 1476/2000 [09:27<03:04,  2.84it/s, est. speed input: 438.32 toks/s, output: 2756.71 toks/s]

Processed prompts:  74%|███████▍  | 1480/2000 [09:27<02:25,  3.58it/s, est. speed input: 438.83 toks/s, output: 2758.37 toks/s]

Processed prompts:  74%|███████▍  | 1484/2000 [09:28<02:20,  3.68it/s, est. speed input: 438.79 toks/s, output: 2755.21 toks/s]

Processed prompts:  74%|███████▍  | 1488/2000 [09:29<02:35,  3.30it/s, est. speed input: 438.43 toks/s, output: 2753.48 toks/s]

Processed prompts:  75%|███████▍  | 1492/2000 [09:30<02:08,  3.94it/s, est. speed input: 438.80 toks/s, output: 2752.74 toks/s]

Processed prompts:  75%|███████▍  | 1496/2000 [09:31<01:52,  4.48it/s, est. speed input: 439.38 toks/s, output: 2777.26 toks/s]

Processed prompts:  75%|███████▌  | 1500/2000 [09:31<01:51,  4.47it/s, est. speed input: 439.49 toks/s, output: 2776.18 toks/s]

Processed prompts:  75%|███████▌  | 1504/2000 [09:32<01:34,  5.24it/s, est. speed input: 439.87 toks/s, output: 2776.19 toks/s]

Processed prompts:  75%|███████▌  | 1508/2000 [09:32<01:16,  6.39it/s, est. speed input: 440.43 toks/s, output: 2778.52 toks/s]

Processed prompts:  76%|███████▌  | 1512/2000 [09:33<01:31,  5.36it/s, est. speed input: 440.41 toks/s, output: 2781.01 toks/s]

Processed prompts:  76%|███████▌  | 1516/2000 [09:34<01:31,  5.27it/s, est. speed input: 440.53 toks/s, output: 2779.49 toks/s]

Processed prompts:  76%|███████▌  | 1524/2000 [09:36<01:56,  4.09it/s, est. speed input: 440.18 toks/s, output: 2771.42 toks/s]

Processed prompts:  76%|███████▋  | 1528/2000 [09:37<01:45,  4.48it/s, est. speed input: 440.61 toks/s, output: 2775.04 toks/s]

Processed prompts:  77%|███████▋  | 1532/2000 [09:37<01:21,  5.73it/s, est. speed input: 441.19 toks/s, output: 2777.45 toks/s]

Processed prompts:  77%|███████▋  | 1536/2000 [09:38<01:09,  6.68it/s, est. speed input: 441.91 toks/s, output: 2846.87 toks/s]

Processed prompts:  77%|███████▋  | 1540/2000 [09:38<01:00,  7.65it/s, est. speed input: 442.52 toks/s, output: 2847.02 toks/s]

Processed prompts:  77%|███████▋  | 1544/2000 [09:40<01:47,  4.25it/s, est. speed input: 441.77 toks/s, output: 2839.19 toks/s]

Processed prompts:  77%|███████▋  | 1548/2000 [09:44<03:17,  2.28it/s, est. speed input: 439.69 toks/s, output: 2823.10 toks/s]

Processed prompts:  78%|███████▊  | 1552/2000 [09:45<03:13,  2.32it/s, est. speed input: 439.20 toks/s, output: 2815.90 toks/s]

Processed prompts:  78%|███████▊  | 1556/2000 [09:46<02:38,  2.81it/s, est. speed input: 439.40 toks/s, output: 2814.96 toks/s]

Processed prompts:  78%|███████▊  | 1560/2000 [09:46<02:01,  3.63it/s, est. speed input: 439.92 toks/s, output: 2814.84 toks/s]

Processed prompts:  78%|███████▊  | 1564/2000 [09:47<01:31,  4.76it/s, est. speed input: 440.57 toks/s, output: 2815.86 toks/s]

Processed prompts:  78%|███████▊  | 1568/2000 [09:47<01:08,  6.32it/s, est. speed input: 441.37 toks/s, output: 2835.28 toks/s]

Processed prompts:  79%|███████▊  | 1572/2000 [09:48<01:26,  4.96it/s, est. speed input: 441.20 toks/s, output: 2832.73 toks/s]

Processed prompts:  79%|███████▉  | 1576/2000 [09:48<01:08,  6.23it/s, est. speed input: 441.84 toks/s, output: 2833.31 toks/s]

Processed prompts:  79%|███████▉  | 1580/2000 [09:49<01:01,  6.87it/s, est. speed input: 442.32 toks/s, output: 2833.72 toks/s]

Processed prompts:  79%|███████▉  | 1584/2000 [09:53<02:50,  2.44it/s, est. speed input: 440.21 toks/s, output: 2841.01 toks/s]

Processed prompts:  79%|███████▉  | 1588/2000 [09:54<02:27,  2.79it/s, est. speed input: 440.33 toks/s, output: 2841.57 toks/s]

Processed prompts:  80%|███████▉  | 1592/2000 [09:55<02:22,  2.86it/s, est. speed input: 440.06 toks/s, output: 2838.82 toks/s]

Processed prompts:  80%|███████▉  | 1596/2000 [09:56<01:56,  3.46it/s, est. speed input: 440.38 toks/s, output: 2837.77 toks/s]

Processed prompts:  80%|████████  | 1600/2000 [09:56<01:29,  4.49it/s, est. speed input: 440.90 toks/s, output: 2838.57 toks/s]

Processed prompts:  80%|████████  | 1604/2000 [09:56<01:05,  6.01it/s, est. speed input: 441.60 toks/s, output: 2842.50 toks/s]

Processed prompts:  80%|████████  | 1608/2000 [09:56<00:59,  6.57it/s, est. speed input: 442.07 toks/s, output: 2844.87 toks/s]

Processed prompts:  81%|████████  | 1612/2000 [09:57<00:50,  7.71it/s, est. speed input: 442.69 toks/s, output: 2853.16 toks/s]

Processed prompts:  81%|████████  | 1616/2000 [09:58<01:21,  4.72it/s, est. speed input: 442.30 toks/s, output: 2849.33 toks/s]

Processed prompts:  81%|████████  | 1620/2000 [09:59<01:00,  6.31it/s, est. speed input: 443.02 toks/s, output: 2850.92 toks/s]

Processed prompts:  81%|████████  | 1624/2000 [09:59<01:07,  5.55it/s, est. speed input: 443.15 toks/s, output: 2850.60 toks/s]

Processed prompts:  81%|████████▏ | 1628/2000 [10:00<00:49,  7.45it/s, est. speed input: 443.84 toks/s, output: 2851.97 toks/s]

Processed prompts:  82%|████████▏ | 1636/2000 [10:00<00:33, 10.86it/s, est. speed input: 445.25 toks/s, output: 2880.27 toks/s]

Processed prompts:  82%|████████▏ | 1640/2000 [10:00<00:32, 11.03it/s, est. speed input: 445.73 toks/s, output: 2882.43 toks/s]

Processed prompts:  82%|████████▏ | 1644/2000 [10:01<00:35, 10.14it/s, est. speed input: 446.20 toks/s, output: 2885.87 toks/s]

Processed prompts:  82%|████████▏ | 1648/2000 [10:01<00:35, 10.00it/s, est. speed input: 446.66 toks/s, output: 2886.20 toks/s]

Processed prompts:  83%|████████▎ | 1652/2000 [10:01<00:29, 11.86it/s, est. speed input: 447.43 toks/s, output: 2906.94 toks/s]

Processed prompts:  83%|████████▎ | 1656/2000 [10:03<01:06,  5.17it/s, est. speed input: 446.82 toks/s, output: 2899.77 toks/s]

Processed prompts:  83%|████████▎ | 1660/2000 [10:04<00:55,  6.14it/s, est. speed input: 447.31 toks/s, output: 2901.00 toks/s]

Processed prompts:  83%|████████▎ | 1664/2000 [10:05<01:22,  4.05it/s, est. speed input: 446.73 toks/s, output: 2894.74 toks/s]

Processed prompts:  83%|████████▎ | 1668/2000 [10:06<01:23,  4.00it/s, est. speed input: 446.66 toks/s, output: 2891.52 toks/s]

Processed prompts:  84%|████████▎ | 1672/2000 [10:07<01:10,  4.67it/s, est. speed input: 446.99 toks/s, output: 2891.08 toks/s]

Processed prompts:  84%|████████▍ | 1676/2000 [10:08<01:05,  4.96it/s, est. speed input: 447.22 toks/s, output: 2889.10 toks/s]

Processed prompts:  84%|████████▍ | 1680/2000 [10:08<00:53,  5.99it/s, est. speed input: 447.77 toks/s, output: 2899.45 toks/s]

Processed prompts:  84%|████████▍ | 1688/2000 [10:08<00:31,  9.99it/s, est. speed input: 449.15 toks/s, output: 2905.20 toks/s]

Processed prompts:  85%|████████▍ | 1696/2000 [10:09<00:29, 10.44it/s, est. speed input: 450.10 toks/s, output: 2911.46 toks/s]

Processed prompts:  85%|████████▌ | 1700/2000 [10:09<00:27, 10.72it/s, est. speed input: 450.51 toks/s, output: 2912.04 toks/s]

Processed prompts:  85%|████████▌ | 1708/2000 [10:09<00:21, 13.89it/s, est. speed input: 451.79 toks/s, output: 2914.03 toks/s]

Processed prompts:  86%|████████▌ | 1712/2000 [10:10<00:25, 11.23it/s, est. speed input: 452.00 toks/s, output: 2911.67 toks/s]

Processed prompts:  86%|████████▌ | 1716/2000 [10:10<00:22, 12.79it/s, est. speed input: 452.74 toks/s, output: 2941.26 toks/s]

Processed prompts:  86%|████████▌ | 1720/2000 [10:12<00:45,  6.16it/s, est. speed input: 453.26 toks/s, output: 2995.23 toks/s]

Processed prompts:  86%|████████▌ | 1724/2000 [10:13<00:46,  5.95it/s, est. speed input: 453.48 toks/s, output: 2994.56 toks/s]

Processed prompts:  87%|████████▋ | 1732/2000 [10:14<00:40,  6.63it/s, est. speed input: 454.16 toks/s, output: 2992.47 toks/s]

Processed prompts:  87%|████████▋ | 1736/2000 [10:14<00:42,  6.23it/s, est. speed input: 454.24 toks/s, output: 2989.88 toks/s]

Processed prompts:  87%|████████▋ | 1740/2000 [10:16<00:49,  5.29it/s, est. speed input: 454.11 toks/s, output: 2986.62 toks/s]

Processed prompts:  87%|████████▋ | 1748/2000 [10:17<00:40,  6.17it/s, est. speed input: 454.79 toks/s, output: 2985.59 toks/s]

Processed prompts:  88%|████████▊ | 1760/2000 [10:17<00:23, 10.38it/s, est. speed input: 456.59 toks/s, output: 2987.48 toks/s]

Processed prompts:  88%|████████▊ | 1764/2000 [10:17<00:26,  9.02it/s, est. speed input: 456.75 toks/s, output: 2985.39 toks/s]

Processed prompts:  88%|████████▊ | 1768/2000 [10:18<00:22, 10.19it/s, est. speed input: 457.29 toks/s, output: 2986.05 toks/s]

Processed prompts:  89%|████████▊ | 1772/2000 [10:18<00:22, 10.07it/s, est. speed input: 457.71 toks/s, output: 2985.78 toks/s]

Processed prompts:  89%|████████▉ | 1776/2000 [10:19<00:25,  8.91it/s, est. speed input: 457.94 toks/s, output: 2984.04 toks/s]

Processed prompts:  89%|████████▉ | 1780/2000 [10:19<00:23,  9.54it/s, est. speed input: 458.39 toks/s, output: 2985.37 toks/s]

Processed prompts:  89%|████████▉ | 1784/2000 [10:20<00:32,  6.61it/s, est. speed input: 458.27 toks/s, output: 2981.51 toks/s]

Processed prompts:  90%|████████▉ | 1792/2000 [10:21<00:26,  7.86it/s, est. speed input: 459.33 toks/s, output: 3002.74 toks/s]

Processed prompts:  90%|████████▉ | 1796/2000 [10:22<00:30,  6.62it/s, est. speed input: 459.31 toks/s, output: 2999.79 toks/s]

Processed prompts:  90%|█████████ | 1800/2000 [10:23<00:40,  5.00it/s, est. speed input: 458.93 toks/s, output: 2994.26 toks/s]

Processed prompts:  90%|█████████ | 1804/2000 [10:24<00:32,  5.97it/s, est. speed input: 459.40 toks/s, output: 2994.37 toks/s]

Processed prompts:  90%|█████████ | 1808/2000 [10:24<00:34,  5.52it/s, est. speed input: 459.41 toks/s, output: 2991.42 toks/s]

Processed prompts:  91%|█████████ | 1812/2000 [10:25<00:26,  7.01it/s, est. speed input: 459.95 toks/s, output: 2991.66 toks/s]

Processed prompts:  91%|█████████ | 1816/2000 [10:25<00:24,  7.59it/s, est. speed input: 460.34 toks/s, output: 2991.21 toks/s]

Processed prompts:  91%|█████████ | 1820/2000 [10:25<00:20,  8.63it/s, est. speed input: 460.79 toks/s, output: 2990.93 toks/s]

Processed prompts:  91%|█████████▏| 1828/2000 [10:27<00:24,  7.05it/s, est. speed input: 461.12 toks/s, output: 2987.49 toks/s]

Processed prompts:  92%|█████████▏| 1832/2000 [10:28<00:33,  5.01it/s, est. speed input: 460.61 toks/s, output: 2981.10 toks/s]

Processed prompts:  92%|█████████▏| 1836/2000 [10:28<00:25,  6.47it/s, est. speed input: 461.18 toks/s, output: 2983.48 toks/s]

Processed prompts:  92%|█████████▏| 1840/2000 [10:28<00:19,  8.14it/s, est. speed input: 461.68 toks/s, output: 2984.34 toks/s]

Processed prompts:  92%|█████████▏| 1844/2000 [10:29<00:17,  8.72it/s, est. speed input: 462.05 toks/s, output: 2984.36 toks/s]

Processed prompts:  92%|█████████▏| 1848/2000 [10:29<00:19,  7.71it/s, est. speed input: 462.24 toks/s, output: 2986.68 toks/s]

Processed prompts:  93%|█████████▎| 1852/2000 [10:30<00:17,  8.25it/s, est. speed input: 462.56 toks/s, output: 2986.72 toks/s]

Processed prompts:  93%|█████████▎| 1856/2000 [10:31<00:22,  6.33it/s, est. speed input: 462.82 toks/s, output: 3014.60 toks/s]

Processed prompts:  93%|█████████▎| 1860/2000 [10:32<00:22,  6.10it/s, est. speed input: 462.92 toks/s, output: 3012.64 toks/s]

Processed prompts:  93%|█████████▎| 1864/2000 [10:32<00:19,  6.80it/s, est. speed input: 463.52 toks/s, output: 3035.26 toks/s]

Processed prompts:  93%|█████████▎| 1868/2000 [10:33<00:26,  4.96it/s, est. speed input: 463.29 toks/s, output: 3033.04 toks/s]

Processed prompts:  94%|█████████▎| 1872/2000 [10:35<00:35,  3.59it/s, est. speed input: 462.67 toks/s, output: 3030.43 toks/s]

Processed prompts:  94%|█████████▍| 1876/2000 [10:36<00:31,  3.92it/s, est. speed input: 462.71 toks/s, output: 3029.38 toks/s]

Processed prompts:  94%|█████████▍| 1880/2000 [10:36<00:23,  5.04it/s, est. speed input: 463.20 toks/s, output: 3032.90 toks/s]

Processed prompts:  94%|█████████▍| 1884/2000 [10:37<00:23,  4.96it/s, est. speed input: 463.22 toks/s, output: 3031.11 toks/s]

Processed prompts:  94%|█████████▍| 1888/2000 [10:39<00:31,  3.54it/s, est. speed input: 462.52 toks/s, output: 3027.09 toks/s]

Processed prompts:  95%|█████████▍| 1892/2000 [10:49<01:45,  1.02it/s, est. speed input: 455.76 toks/s, output: 2983.52 toks/s]

Processed prompts:  95%|█████████▍| 1896/2000 [10:54<01:45,  1.01s/it, est. speed input: 453.40 toks/s, output: 2969.43 toks/s]

Processed prompts:  95%|█████████▌| 1900/2000 [11:01<02:06,  1.26s/it, est. speed input: 449.15 toks/s, output: 2952.86 toks/s]

Processed prompts:  95%|█████████▌| 1904/2000 [11:01<01:27,  1.10it/s, est. speed input: 449.92 toks/s, output: 3003.88 toks/s]

Processed prompts:  95%|█████████▌| 1908/2000 [11:06<01:31,  1.00it/s, est. speed input: 447.33 toks/s, output: 2990.62 toks/s]

Processed prompts:  96%|█████████▌| 1912/2000 [11:25<03:06,  2.12s/it, est. speed input: 435.64 toks/s, output: 2923.33 toks/s]

Processed prompts:  96%|█████████▌| 1916/2000 [11:36<03:13,  2.31s/it, est. speed input: 429.63 toks/s, output: 2914.71 toks/s]

Processed prompts:  96%|█████████▌| 1920/2000 [11:45<03:03,  2.30s/it, est. speed input: 425.08 toks/s, output: 2945.14 toks/s]

Processed prompts:  96%|█████████▌| 1924/2000 [12:02<03:35,  2.83s/it, est. speed input: 416.34 toks/s, output: 2914.67 toks/s]

Processed prompts:  96%|█████████▋| 1928/2000 [12:02<02:24,  2.00s/it, est. speed input: 417.13 toks/s, output: 2954.54 toks/s]

Processed prompts:  97%|█████████▋| 1932/2000 [12:02<01:36,  1.42s/it, est. speed input: 417.89 toks/s, output: 3035.83 toks/s]

Processed prompts:  97%|█████████▋| 1936/2000 [12:05<01:17,  1.22s/it, est. speed input: 416.95 toks/s, output: 3077.94 toks/s]

Processed prompts:  97%|█████████▋| 1940/2000 [12:15<01:35,  1.58s/it, est. speed input: 412.24 toks/s, output: 3126.51 toks/s]

Processed prompts:  97%|█████████▋| 1944/2000 [12:28<01:55,  2.06s/it, est. speed input: 405.98 toks/s, output: 3098.96 toks/s]

Processed prompts:  97%|█████████▋| 1948/2000 [13:48<06:29,  7.50s/it, est. speed input: 366.95 toks/s, output: 2810.49 toks/s]

Processed prompts:  98%|█████████▊| 1952/2000 [14:51<07:56,  9.93s/it, est. speed input: 342.36 toks/s, output: 2660.98 toks/s]

Processed prompts:  98%|█████████▊| 1956/2000 [14:51<05:06,  6.98s/it, est. speed input: 343.25 toks/s, output: 2719.42 toks/s]

Processed prompts:  98%|█████████▊| 1960/2000 [15:35<05:27,  8.18s/it, est. speed input: 327.99 toks/s, output: 2694.69 toks/s]

Processed prompts:  98%|█████████▊| 1964/2000 [15:44<03:49,  6.39s/it, est. speed input: 325.54 toks/s, output: 2719.53 toks/s]

Processed prompts:  98%|█████████▊| 1968/2000 [15:49<02:34,  4.83s/it, est. speed input: 324.47 toks/s, output: 2745.56 toks/s]

Processed prompts:  99%|█████████▊| 1972/2000 [15:49<01:36,  3.44s/it, est. speed input: 325.03 toks/s, output: 2840.25 toks/s]

Processed prompts:  99%|█████████▉| 1976/2000 [15:51<01:00,  2.52s/it, est. speed input: 325.19 toks/s, output: 2894.27 toks/s]

Processed prompts:  99%|█████████▉| 1980/2000 [15:54<00:40,  2.01s/it, est. speed input: 324.91 toks/s, output: 2923.94 toks/s]

Processed prompts:  99%|█████████▉| 1984/2000 [16:38<01:15,  4.69s/it, est. speed input: 311.51 toks/s, output: 2852.28 toks/s]

Processed prompts:  99%|█████████▉| 1988/2000 [16:39<00:40,  3.35s/it, est. speed input: 311.97 toks/s, output: 2959.92 toks/s]

Processed prompts: 100%|█████████▉| 1992/2000 [16:39<00:19,  2.38s/it, est. speed input: 312.60 toks/s, output: 3017.59 toks/s]

Processed prompts: 100%|█████████▉| 1996/2000 [17:12<00:16,  4.10s/it, est. speed input: 303.41 toks/s, output: 3003.85 toks/s]

Processed prompts: 100%|██████████| 2000/2000 [17:28<00:00,  4.06s/it, est. speed input: 299.45 toks/s, output: 3038.39 toks/s]

Processed prompts: 100%|██████████| 2000/2000 [17:28<00:00,  4.06s/it, est. speed input: 299.45 toks/s, output: 3038.39 toks/s]
Processed prompts: 100%|██████████| 2000/2000 [17:28<00:00,  1.91it/s, est. speed input: 299.45 toks/s, output: 3038.39 toks/s]

Splits: 100%|██████████| 1/1 [17:28<00:00, 1048.68s/it]
Splits: 100%|██████████| 1/1 [17:28<00:00, 1048.68s/it]

Creating parquet from Arrow format:   0%|          | 0/1 [00:00<?, ?ba/s]
Creating parquet from Arrow format: 100%|██████████| 1/1 [00:00<00:00,  5.18ba/s]
Creating parquet from Arrow format: 100%|██████████| 1/1 [00:00<00:00,  5.16ba/s]
[2025-10-26 08:22:08,366] [    INFO]: Cached 500 samples of lighteval|math_500|0 (3aecc7facae3926c, GENERATIVE) at /mnt/public/wucanhui/outputs/Qwen3-4B-math-reasoning/checkpoint-2562/0619260e1176b049/lighteval|math_500|0/3aecc7facae3926c/GENERATIVE.parquet. (cache_management.py:345)

Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 500 examples [00:00, 4560.92 examples/s]
Generating train split: 500 examples [00:00, 4476.59 examples/s]
[rank0]:[W1026 08:22:18.930475912 ProcessGroupNCCL.cpp:1479] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
[2025-10-26 08:22:19,869] [    INFO]: --- POST-PROCESSING MODEL RESPONSES --- (pipeline.py:344)
[2025-10-26 08:22:19,876] [    INFO]: --- COMPUTING METRICS --- (pipeline.py:371)
[2025-10-26 08:22:19,919] [ WARNING]: n undefined in the pass@k. We assume it's the same as the sample's number of predictions. (metrics_sample.py:1302)
[2025-10-26 08:22:25,021] [    INFO]: --- DISPLAYING RESULTS --- (pipeline.py:432)
[2025-10-26 08:22:25,033] [    INFO]: --- SAVING AND PUSHING RESULTS --- (pipeline.py:422)
[2025-10-26 08:22:25,034] [    INFO]: Saving experiment tracker (evaluation_tracker.py:246)
[2025-10-26 08:22:26,957] [    INFO]: Saving results to /mnt/public/wucanhui/lighteval/results/results/mnt/public/wucanhui/outputs/Qwen3-4B-math-reasoning/checkpoint-2562/results_2025-10-26T08-22-25.035867.json (evaluation_tracker.py:310)
|        Task        |Version|   Metric    |Value |   |Stderr|
|--------------------|-------|-------------|-----:|---|-----:|
|all                 |       |avg@k_with_k |0.8785|±  |0.0118|
|                    |       |pass@k_with_k|0.9560|±  |0.0092|
|lighteval:math_500:0|       |avg@k_with_k |0.8785|±  |0.0118|
|                    |       |pass@k_with_k|0.9560|±  |0.0092|