eval_lighteval|math_500|0.log

INFO 10-26 02:40:28 [__init__.py:235] Automatically detected platform cuda.
[2025-10-26 02:40:30,504] [    INFO]: --- INIT SEEDS --- (pipeline.py:249)
[2025-10-26 02:40:30,505] [    INFO]: --- LOADING TASKS --- (pipeline.py:210)
[2025-10-26 02:40:30,508] [ WARNING]: Careful, the task math_500 is using evaluation data to build the few shot examples. (lighteval_task.py:269)
[2025-10-26 02:40:34,976] [    INFO]: --- LOADING MODEL --- (pipeline.py:177)
`torch_dtype` is deprecated! Use `dtype` instead!
[2025-10-26 02:40:41,447] [    INFO]: Using max model len 32768 (config.py:1604)
[2025-10-26 02:40:41,951] [    INFO]: Chunked prefill is enabled with max_num_batched_tokens=2048. (config.py:2434)
INFO 10-26 02:40:46 [__init__.py:235] Automatically detected platform cuda.
INFO 10-26 02:40:47 [core.py:572] Waiting for init message from front-end.
INFO 10-26 02:40:47 [core.py:71] Initializing a V1 LLM engine (v0.10.0) with config: model='/mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562', speculative_config=None, tokenizer='/mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562', skip_tokenizer_init=False, tokenizer_mode=auto, revision=main, override_neuron_config={}, tokenizer_revision=main, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=1234, served_model_name=/mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, pooler_config=None, compilation_config={"level":0,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":[],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":0,"cudagraph_capture_sizes":[],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":0,"local_cache_dir":null}
INFO 10-26 02:40:50 [parallel_state.py:1102] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
WARNING 10-26 02:40:50 [topk_topp_sampler.py:59] FlashInfer is not available. Falling back to the PyTorch-native implementation of top-p & top-k sampling. For the best performance, please install FlashInfer.
INFO 10-26 02:40:50 [gpu_model_runner.py:1843] Starting to load model /mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562...
INFO 10-26 02:40:50 [gpu_model_runner.py:1875] Loading model from scratch...
INFO 10-26 02:40:50 [cuda.py:290] Using Flash Attention backend on V1 engine.

Loading safetensors checkpoint shards:   0% Completed | 0/6 [00:00<?, ?it/s]

Loading safetensors checkpoint shards:  17% Completed | 1/6 [00:36<03:01, 36.20s/it]

Loading safetensors checkpoint shards:  33% Completed | 2/6 [01:14<02:30, 37.72s/it]

Loading safetensors checkpoint shards:  50% Completed | 3/6 [01:50<01:49, 36.62s/it]

Loading safetensors checkpoint shards:  67% Completed | 4/6 [02:24<01:11, 35.85s/it]

Loading safetensors checkpoint shards:  83% Completed | 5/6 [03:01<00:35, 35.95s/it]

Loading safetensors checkpoint shards: 100% Completed | 6/6 [03:35<00:00, 35.39s/it]

Loading safetensors checkpoint shards: 100% Completed | 6/6 [03:35<00:00, 35.90s/it]

INFO 10-26 02:44:27 [default_loader.py:262] Loading weights took 216.40 seconds
INFO 10-26 02:44:27 [gpu_model_runner.py:1892] Model loading took 27.5185 GiB and 217.035510 seconds
INFO 10-26 02:44:28 [gpu_worker.py:255] Available KV cache memory: 97.63 GiB
INFO 10-26 02:44:28 [kv_cache_utils.py:833] GPU KV cache size: 639,792 tokens
INFO 10-26 02:44:28 [kv_cache_utils.py:837] Maximum concurrency for 32,768 tokens per request: 19.52x
INFO 10-26 02:44:29 [core.py:193] init engine (profile, create kv cache, warmup model) took 1.19 seconds
[2025-10-26 02:44:29,515] [    INFO]: [CACHING] Initializing data cache (cache_management.py:105)
[2025-10-26 02:44:29,521] [    INFO]: --- RUNNING MODEL --- (pipeline.py:330)
[2025-10-26 02:44:29,522] [    INFO]: Running SamplingMethod.GENERATIVE requests (pipeline.py:313)
[2025-10-26 02:44:45,633] [    INFO]: Cache: Starting to process 500/500 samples (not found in cache) for tasks lighteval|math_500|0 (3aecc7facae3926c, GENERATIVE) (cache_management.py:399)
[2025-10-26 02:44:45,636] [ WARNING]: You cannot select the number of dataset splits for a generative evaluation at the moment. Automatically inferring. (data.py:206)

Splits:   0%|          | 0/1 [00:00<?, ?it/s][2025-10-26 02:44:45,773] [ WARNING]: context_size + max_new_tokens=33645 which is greater than self.max_length=32768. Truncating context to 0 tokens. (vllm_model.py:367)


Adding requests:   0%|          | 0/500 [00:00<?, ?it/s]

Adding requests:  19%|█▉        | 94/500 [00:00<00:01, 338.54it/s]
Adding requests: 100%|██████████| 500/500 [00:00<00:00, 1544.03it/s]


Processed prompts:   0%|          | 0/2000 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]

Processed prompts:   0%|          | 4/2000 [00:06<53:11,  1.60s/it, est. speed input: 250.78 toks/s, output: 152.12 toks/s]

Processed prompts:   0%|          | 8/2000 [00:12<49:12,  1.48s/it, est. speed input: 207.02 toks/s, output: 235.60 toks/s]

Processed prompts:   1%|          | 12/2000 [00:16<42:09,  1.27s/it, est. speed input: 202.85 toks/s, output: 291.03 toks/s]

Processed prompts:   1%|          | 16/2000 [00:19<37:15,  1.13s/it, est. speed input: 231.24 toks/s, output: 399.58 toks/s]

Processed prompts:   1%|          | 20/2000 [00:24<37:46,  1.14s/it, est. speed input: 238.78 toks/s, output: 473.08 toks/s]

Processed prompts:   1%|          | 24/2000 [00:26<29:28,  1.12it/s, est. speed input: 263.79 toks/s, output: 560.59 toks/s]

Processed prompts:   1%|▏         | 28/2000 [00:27<22:31,  1.46it/s, est. speed input: 300.14 toks/s, output: 694.04 toks/s]

Processed prompts:   2%|▏         | 32/2000 [00:35<36:08,  1.10s/it, est. speed input: 256.17 toks/s, output: 619.32 toks/s]

Processed prompts:   2%|▏         | 36/2000 [00:41<40:15,  1.23s/it, est. speed input: 303.83 toks/s, output: 608.55 toks/s]

Processed prompts:   2%|▏         | 40/2000 [00:44<36:20,  1.11s/it, est. speed input: 318.83 toks/s, output: 636.93 toks/s]

Processed prompts:   2%|▏         | 44/2000 [00:44<25:27,  1.28it/s, est. speed input: 336.32 toks/s, output: 677.24 toks/s]

Processed prompts:   2%|▏         | 48/2000 [00:45<19:28,  1.67it/s, est. speed input: 349.82 toks/s, output: 727.46 toks/s]

Processed prompts:   3%|▎         | 52/2000 [00:47<19:04,  1.70it/s, est. speed input: 349.49 toks/s, output: 823.31 toks/s]

Processed prompts:   3%|▎         | 56/2000 [00:53<27:52,  1.16it/s, est. speed input: 330.08 toks/s, output: 752.69 toks/s]

Processed prompts:   3%|▎         | 60/2000 [00:53<20:00,  1.62it/s, est. speed input: 342.68 toks/s, output: 824.11 toks/s]

Processed prompts:   3%|▎         | 64/2000 [01:02<35:06,  1.09s/it, est. speed input: 309.49 toks/s, output: 750.27 toks/s]

Processed prompts:   3%|▎         | 68/2000 [01:03<26:56,  1.20it/s, est. speed input: 322.67 toks/s, output: 866.73 toks/s]

Processed prompts:   4%|▎         | 72/2000 [01:05<22:26,  1.43it/s, est. speed input: 335.83 toks/s, output: 919.70 toks/s]

Processed prompts:   4%|▍         | 76/2000 [01:06<20:00,  1.60it/s, est. speed input: 337.89 toks/s, output: 979.54 toks/s]

Processed prompts:   4%|▍         | 80/2000 [01:08<17:24,  1.84it/s, est. speed input: 348.23 toks/s, output: 1067.83 toks/s]

Processed prompts:   4%|▍         | 84/2000 [01:10<16:37,  1.92it/s, est. speed input: 358.19 toks/s, output: 1170.19 toks/s]

Processed prompts:   4%|▍         | 88/2000 [01:13<19:33,  1.63it/s, est. speed input: 356.18 toks/s, output: 1193.16 toks/s]

Processed prompts:   5%|▍         | 92/2000 [01:14<15:14,  2.09it/s, est. speed input: 376.11 toks/s, output: 1305.41 toks/s]

Processed prompts:   5%|▍         | 96/2000 [01:14<11:26,  2.78it/s, est. speed input: 388.68 toks/s, output: 1396.95 toks/s]

Processed prompts:   5%|▌         | 100/2000 [01:14<08:55,  3.55it/s, est. speed input: 400.54 toks/s, output: 1444.53 toks/s]

Processed prompts:   5%|▌         | 104/2000 [01:14<06:33,  4.82it/s, est. speed input: 408.72 toks/s, output: 1454.87 toks/s]

Processed prompts:   5%|▌         | 108/2000 [01:16<07:42,  4.09it/s, est. speed input: 415.90 toks/s, output: 1461.20 toks/s]

Processed prompts:   6%|▌         | 112/2000 [01:16<07:04,  4.45it/s, est. speed input: 440.41 toks/s, output: 1582.36 toks/s]

Processed prompts:   6%|▌         | 116/2000 [01:17<05:31,  5.69it/s, est. speed input: 454.95 toks/s, output: 1661.43 toks/s]

Processed prompts:   6%|▌         | 120/2000 [01:19<08:53,  3.52it/s, est. speed input: 451.88 toks/s, output: 1646.44 toks/s]

Processed prompts:   6%|▌         | 124/2000 [01:19<06:37,  4.72it/s, est. speed input: 466.41 toks/s, output: 1737.67 toks/s]

Processed prompts:   6%|▋         | 128/2000 [01:21<09:30,  3.28it/s, est. speed input: 462.53 toks/s, output: 1710.89 toks/s]

Processed prompts:   7%|▋         | 132/2000 [01:31<30:11,  1.03it/s, est. speed input: 418.92 toks/s, output: 1538.05 toks/s]

Processed prompts:   7%|▋         | 136/2000 [01:35<29:02,  1.07it/s, est. speed input: 410.90 toks/s, output: 1498.45 toks/s]

Processed prompts:   7%|▋         | 140/2000 [01:39<30:57,  1.00it/s, est. speed input: 399.22 toks/s, output: 1476.93 toks/s]

Processed prompts:   7%|▋         | 144/2000 [01:41<24:39,  1.25it/s, est. speed input: 400.70 toks/s, output: 1483.85 toks/s]

Processed prompts:   7%|▋         | 148/2000 [01:41<19:24,  1.59it/s, est. speed input: 403.31 toks/s, output: 1481.55 toks/s]

Processed prompts:   8%|▊         | 152/2000 [01:42<14:01,  2.19it/s, est. speed input: 411.85 toks/s, output: 1593.00 toks/s]

Processed prompts:   8%|▊         | 156/2000 [01:44<15:01,  2.05it/s, est. speed input: 408.67 toks/s, output: 1570.19 toks/s]

Processed prompts:   8%|▊         | 160/2000 [01:44<10:51,  2.83it/s, est. speed input: 417.00 toks/s, output: 1599.96 toks/s]

Processed prompts:   8%|▊         | 164/2000 [01:51<23:35,  1.30it/s, est. speed input: 402.41 toks/s, output: 1628.30 toks/s]

Processed prompts:   8%|▊         | 168/2000 [01:56<27:02,  1.13it/s, est. speed input: 392.10 toks/s, output: 1589.13 toks/s]

Processed prompts:   9%|▊         | 172/2000 [01:57<21:19,  1.43it/s, est. speed input: 398.39 toks/s, output: 1658.06 toks/s]

Processed prompts:   9%|▉         | 176/2000 [02:04<30:42,  1.01s/it, est. speed input: 381.21 toks/s, output: 1583.34 toks/s]

Processed prompts:   9%|▉         | 180/2000 [02:04<22:09,  1.37it/s, est. speed input: 388.60 toks/s, output: 1599.25 toks/s]

Processed prompts:   9%|▉         | 184/2000 [02:12<34:15,  1.13s/it, est. speed input: 369.72 toks/s, output: 1535.08 toks/s]

Processed prompts:   9%|▉         | 188/2000 [02:15<30:13,  1.00s/it, est. speed input: 366.60 toks/s, output: 1515.81 toks/s]

Processed prompts:  10%|▉         | 192/2000 [02:16<22:31,  1.34it/s, est. speed input: 372.11 toks/s, output: 1632.66 toks/s]

Processed prompts:  10%|▉         | 196/2000 [02:18<20:00,  1.50it/s, est. speed input: 371.81 toks/s, output: 1618.05 toks/s]

Processed prompts:  10%|█         | 200/2000 [02:20<19:22,  1.55it/s, est. speed input: 371.06 toks/s, output: 1597.18 toks/s]

Processed prompts:  10%|█         | 204/2000 [02:21<15:04,  1.99it/s, est. speed input: 376.74 toks/s, output: 1643.41 toks/s]

Processed prompts:  10%|█         | 208/2000 [02:35<42:54,  1.44s/it, est. speed input: 346.66 toks/s, output: 1524.14 toks/s]

Processed prompts:  11%|█         | 212/2000 [02:36<31:10,  1.05s/it, est. speed input: 349.65 toks/s, output: 1535.51 toks/s]

Processed prompts:  11%|█         | 216/2000 [02:41<34:01,  1.14s/it, est. speed input: 342.84 toks/s, output: 1523.42 toks/s]

Processed prompts:  11%|█         | 220/2000 [02:47<36:34,  1.23s/it, est. speed input: 336.86 toks/s, output: 1535.15 toks/s]

Processed prompts:  11%|█         | 224/2000 [02:48<28:30,  1.04it/s, est. speed input: 339.15 toks/s, output: 1543.70 toks/s]

Processed prompts:  11%|█▏        | 228/2000 [02:50<23:51,  1.24it/s, est. speed input: 339.53 toks/s, output: 1534.09 toks/s]

Processed prompts:  12%|█▏        | 232/2000 [02:55<27:37,  1.07it/s, est. speed input: 337.50 toks/s, output: 1582.00 toks/s]

Processed prompts:  12%|█▏        | 236/2000 [02:56<22:15,  1.32it/s, est. speed input: 338.30 toks/s, output: 1581.45 toks/s]

Processed prompts:  12%|█▏        | 240/2000 [03:00<22:34,  1.30it/s, est. speed input: 336.74 toks/s, output: 1610.18 toks/s]

Processed prompts:  12%|█▏        | 244/2000 [03:04<25:44,  1.14it/s, est. speed input: 331.63 toks/s, output: 1583.08 toks/s]

Processed prompts:  12%|█▏        | 248/2000 [03:06<21:25,  1.36it/s, est. speed input: 331.96 toks/s, output: 1585.75 toks/s]

Processed prompts:  13%|█▎        | 252/2000 [03:06<16:44,  1.74it/s, est. speed input: 335.01 toks/s, output: 1591.92 toks/s]

Processed prompts:  13%|█▎        | 256/2000 [03:09<17:52,  1.63it/s, est. speed input: 334.39 toks/s, output: 1610.18 toks/s]

Processed prompts:  13%|█▎        | 260/2000 [03:11<16:57,  1.71it/s, est. speed input: 334.03 toks/s, output: 1604.18 toks/s]

Processed prompts:  13%|█▎        | 264/2000 [03:14<18:04,  1.60it/s, est. speed input: 333.05 toks/s, output: 1592.07 toks/s]

Processed prompts:  13%|█▎        | 268/2000 [03:15<14:06,  2.05it/s, est. speed input: 334.84 toks/s, output: 1591.62 toks/s]

Processed prompts:  14%|█▎        | 272/2000 [03:32<47:24,  1.65s/it, est. speed input: 311.07 toks/s, output: 1498.52 toks/s]

Processed prompts:  14%|█▍        | 276/2000 [03:32<33:30,  1.17s/it, est. speed input: 316.06 toks/s, output: 1579.83 toks/s]

Processed prompts:  14%|█▍        | 284/2000 [03:38<26:25,  1.08it/s, est. speed input: 314.33 toks/s, output: 1554.96 toks/s]

Processed prompts:  14%|█▍        | 288/2000 [03:43<28:43,  1.01s/it, est. speed input: 309.91 toks/s, output: 1525.92 toks/s]

Processed prompts:  15%|█▍        | 292/2000 [03:45<25:25,  1.12it/s, est. speed input: 309.32 toks/s, output: 1545.25 toks/s]

Processed prompts:  15%|█▍        | 296/2000 [03:50<27:35,  1.03it/s, est. speed input: 306.03 toks/s, output: 1521.19 toks/s]

Processed prompts:  15%|█▌        | 300/2000 [03:50<20:28,  1.38it/s, est. speed input: 309.89 toks/s, output: 1579.88 toks/s]

Processed prompts:  15%|█▌        | 304/2000 [03:50<15:07,  1.87it/s, est. speed input: 312.54 toks/s, output: 1599.35 toks/s]

Processed prompts:  15%|█▌        | 308/2000 [03:51<13:10,  2.14it/s, est. speed input: 313.64 toks/s, output: 1599.43 toks/s]

Processed prompts:  16%|█▌        | 312/2000 [03:55<16:18,  1.73it/s, est. speed input: 312.49 toks/s, output: 1584.71 toks/s]

Processed prompts:  16%|█▌        | 316/2000 [03:58<16:58,  1.65it/s, est. speed input: 311.68 toks/s, output: 1583.66 toks/s]

Processed prompts:  16%|█▌        | 320/2000 [04:02<21:07,  1.33it/s, est. speed input: 308.63 toks/s, output: 1559.88 toks/s]

Processed prompts:  16%|█▌        | 324/2000 [04:02<15:53,  1.76it/s, est. speed input: 320.91 toks/s, output: 1656.12 toks/s]

Processed prompts:  16%|█▋        | 328/2000 [04:06<17:39,  1.58it/s, est. speed input: 319.08 toks/s, output: 1640.77 toks/s]

Processed prompts:  17%|█▋        | 332/2000 [04:08<17:57,  1.55it/s, est. speed input: 318.85 toks/s, output: 1652.23 toks/s]

Processed prompts:  17%|█▋        | 336/2000 [04:09<13:37,  2.04it/s, est. speed input: 322.07 toks/s, output: 1733.10 toks/s]

Processed prompts:  17%|█▋        | 340/2000 [04:12<16:05,  1.72it/s, est. speed input: 329.96 toks/s, output: 1826.91 toks/s]

Processed prompts:  17%|█▋        | 344/2000 [04:13<13:22,  2.06it/s, est. speed input: 331.03 toks/s, output: 1824.01 toks/s]

Processed prompts:  17%|█▋        | 348/2000 [04:21<25:19,  1.09it/s, est. speed input: 325.96 toks/s, output: 1858.60 toks/s]

Processed prompts:  18%|█▊        | 352/2000 [04:24<24:49,  1.11it/s, est. speed input: 323.80 toks/s, output: 1838.34 toks/s]

Processed prompts:  18%|█▊        | 356/2000 [04:28<25:42,  1.07it/s, est. speed input: 321.38 toks/s, output: 1841.31 toks/s]

Processed prompts:  18%|█▊        | 360/2000 [04:34<29:27,  1.08s/it, est. speed input: 316.99 toks/s, output: 1808.49 toks/s]

Processed prompts:  18%|█▊        | 364/2000 [04:35<21:50,  1.25it/s, est. speed input: 318.56 toks/s, output: 1809.25 toks/s]

Processed prompts:  18%|█▊        | 368/2000 [04:36<19:09,  1.42it/s, est. speed input: 319.27 toks/s, output: 1810.90 toks/s]

Processed prompts:  19%|█▊        | 372/2000 [04:38<16:50,  1.61it/s, est. speed input: 321.81 toks/s, output: 1876.38 toks/s]

Processed prompts:  19%|█▉        | 376/2000 [04:39<14:05,  1.92it/s, est. speed input: 322.93 toks/s, output: 1876.93 toks/s]

Processed prompts:  19%|█▉        | 380/2000 [04:46<23:56,  1.13it/s, est. speed input: 317.18 toks/s, output: 1844.00 toks/s]

Processed prompts:  19%|█▉        | 384/2000 [04:47<19:03,  1.41it/s, est. speed input: 317.96 toks/s, output: 1840.32 toks/s]

Processed prompts:  19%|█▉        | 388/2000 [04:53<25:13,  1.07it/s, est. speed input: 314.67 toks/s, output: 1846.02 toks/s]

Processed prompts:  20%|█▉        | 392/2000 [04:59<29:01,  1.08s/it, est. speed input: 310.65 toks/s, output: 1834.56 toks/s]

Processed prompts:  20%|█▉        | 396/2000 [05:04<30:35,  1.14s/it, est. speed input: 311.10 toks/s, output: 1890.47 toks/s]

Processed prompts:  20%|██        | 400/2000 [05:10<33:34,  1.26s/it, est. speed input: 307.11 toks/s, output: 1872.53 toks/s]

Processed prompts:  20%|██        | 404/2000 [05:18<37:58,  1.43s/it, est. speed input: 301.92 toks/s, output: 1832.08 toks/s]

Processed prompts:  20%|██        | 408/2000 [05:22<34:45,  1.31s/it, est. speed input: 300.81 toks/s, output: 1864.38 toks/s]

Processed prompts:  21%|██        | 412/2000 [05:33<47:15,  1.79s/it, est. speed input: 292.61 toks/s, output: 1818.59 toks/s]

Processed prompts:  21%|██        | 416/2000 [05:36<39:03,  1.48s/it, est. speed input: 293.88 toks/s, output: 1903.04 toks/s]

Processed prompts:  21%|██        | 420/2000 [05:46<47:07,  1.79s/it, est. speed input: 286.92 toks/s, output: 1852.69 toks/s]

Processed prompts:  21%|██        | 424/2000 [05:47<34:42,  1.32s/it, est. speed input: 287.74 toks/s, output: 1851.39 toks/s]

Processed prompts:  21%|██▏       | 428/2000 [05:50<30:17,  1.16s/it, est. speed input: 287.26 toks/s, output: 1853.75 toks/s]

Processed prompts:  22%|██▏       | 432/2000 [06:00<39:08,  1.50s/it, est. speed input: 281.99 toks/s, output: 1869.26 toks/s]

Processed prompts:  22%|██▏       | 436/2000 [06:02<31:42,  1.22s/it, est. speed input: 282.85 toks/s, output: 1919.26 toks/s]

Processed prompts:  22%|██▏       | 440/2000 [06:04<26:55,  1.04s/it, est. speed input: 284.13 toks/s, output: 1994.20 toks/s]

Processed prompts:  22%|██▏       | 444/2000 [06:09<27:43,  1.07s/it, est. speed input: 282.57 toks/s, output: 1984.79 toks/s]

Processed prompts:  22%|██▏       | 448/2000 [06:11<23:44,  1.09it/s, est. speed input: 282.47 toks/s, output: 1981.85 toks/s]

Processed prompts:  23%|██▎       | 452/2000 [06:16<25:09,  1.03it/s, est. speed input: 280.78 toks/s, output: 1965.89 toks/s]

Processed prompts:  23%|██▎       | 456/2000 [06:16<17:48,  1.45it/s, est. speed input: 282.55 toks/s, output: 1986.99 toks/s]

Processed prompts:  23%|██▎       | 460/2000 [06:22<25:25,  1.01it/s, est. speed input: 279.24 toks/s, output: 1959.77 toks/s]

Processed prompts:  23%|██▎       | 464/2000 [06:26<24:13,  1.06it/s, est. speed input: 278.44 toks/s, output: 1961.43 toks/s]

Processed prompts:  23%|██▎       | 468/2000 [06:27<18:15,  1.40it/s, est. speed input: 279.34 toks/s, output: 1961.63 toks/s]

Processed prompts:  24%|██▎       | 472/2000 [06:27<14:11,  1.79it/s, est. speed input: 280.60 toks/s, output: 1998.22 toks/s]

Processed prompts:  24%|██▍       | 476/2000 [06:34<23:27,  1.08it/s, est. speed input: 277.30 toks/s, output: 2007.39 toks/s]

Processed prompts:  24%|██▍       | 484/2000 [06:36<14:26,  1.75it/s, est. speed input: 279.67 toks/s, output: 2056.68 toks/s]

Processed prompts:  24%|██▍       | 488/2000 [06:39<15:41,  1.61it/s, est. speed input: 279.15 toks/s, output: 2048.23 toks/s]

Processed prompts:  25%|██▍       | 492/2000 [06:41<15:33,  1.62it/s, est. speed input: 279.16 toks/s, output: 2060.78 toks/s]

Processed prompts:  25%|██▍       | 496/2000 [06:42<13:14,  1.89it/s, est. speed input: 281.13 toks/s, output: 2145.14 toks/s]

Processed prompts:  25%|██▌       | 500/2000 [06:44<12:36,  1.98it/s, est. speed input: 283.45 toks/s, output: 2245.18 toks/s]

Processed prompts:  25%|██▌       | 504/2000 [06:46<12:34,  1.98it/s, est. speed input: 283.38 toks/s, output: 2238.29 toks/s]

Processed prompts:  25%|██▌       | 508/2000 [06:49<13:35,  1.83it/s, est. speed input: 283.38 toks/s, output: 2229.21 toks/s]

Processed prompts:  26%|██▌       | 516/2000 [06:52<12:10,  2.03it/s, est. speed input: 283.85 toks/s, output: 2226.62 toks/s]

Processed prompts:  26%|██▌       | 520/2000 [07:01<22:28,  1.10it/s, est. speed input: 279.13 toks/s, output: 2183.85 toks/s]

Processed prompts:  26%|██▌       | 524/2000 [07:02<18:22,  1.34it/s, est. speed input: 279.82 toks/s, output: 2185.82 toks/s]

Processed prompts:  26%|██▋       | 528/2000 [07:03<14:26,  1.70it/s, est. speed input: 280.59 toks/s, output: 2185.06 toks/s]

Processed prompts:  27%|██▋       | 536/2000 [07:11<19:41,  1.24it/s, est. speed input: 277.44 toks/s, output: 2149.81 toks/s]

Processed prompts:  27%|██▋       | 540/2000 [07:13<17:49,  1.37it/s, est. speed input: 277.99 toks/s, output: 2151.35 toks/s]

Processed prompts:  27%|██▋       | 544/2000 [07:15<16:00,  1.52it/s, est. speed input: 278.52 toks/s, output: 2152.48 toks/s]

Processed prompts:  27%|██▋       | 548/2000 [07:17<14:51,  1.63it/s, est. speed input: 279.90 toks/s, output: 2215.88 toks/s]

Processed prompts:  28%|██▊       | 552/2000 [07:20<15:19,  1.58it/s, est. speed input: 279.56 toks/s, output: 2207.50 toks/s]

Processed prompts:  28%|██▊       | 556/2000 [07:23<16:39,  1.45it/s, est. speed input: 278.58 toks/s, output: 2193.54 toks/s]

Processed prompts:  28%|██▊       | 560/2000 [07:24<12:26,  1.93it/s, est. speed input: 279.73 toks/s, output: 2196.94 toks/s]

Processed prompts:  28%|██▊       | 564/2000 [07:37<31:13,  1.30s/it, est. speed input: 273.13 toks/s, output: 2139.13 toks/s]

Processed prompts:  28%|██▊       | 568/2000 [07:54<51:56,  2.18s/it, est. speed input: 264.46 toks/s, output: 2067.31 toks/s]

Processed prompts:  29%|██▊       | 572/2000 [07:55<38:03,  1.60s/it, est. speed input: 265.58 toks/s, output: 2110.88 toks/s]

Processed prompts:  29%|██▉       | 576/2000 [07:55<28:03,  1.18s/it, est. speed input: 266.28 toks/s, output: 2111.63 toks/s]

Processed prompts:  29%|██▉       | 580/2000 [07:56<20:29,  1.15it/s, est. speed input: 267.08 toks/s, output: 2113.36 toks/s]

Processed prompts:  29%|██▉       | 584/2000 [07:57<16:45,  1.41it/s, est. speed input: 267.81 toks/s, output: 2132.53 toks/s]

Processed prompts:  29%|██▉       | 588/2000 [08:00<16:37,  1.42it/s, est. speed input: 267.52 toks/s, output: 2129.85 toks/s]

Processed prompts:  30%|██▉       | 592/2000 [08:06<22:30,  1.04it/s, est. speed input: 265.18 toks/s, output: 2104.46 toks/s]

Processed prompts:  30%|██▉       | 596/2000 [08:07<16:55,  1.38it/s, est. speed input: 266.21 toks/s, output: 2132.50 toks/s]

Processed prompts:  30%|███       | 600/2000 [08:08<14:01,  1.66it/s, est. speed input: 266.80 toks/s, output: 2130.26 toks/s]

Processed prompts:  30%|███       | 604/2000 [08:09<11:13,  2.07it/s, est. speed input: 267.49 toks/s, output: 2139.76 toks/s]

Processed prompts:  30%|███       | 608/2000 [08:14<16:22,  1.42it/s, est. speed input: 265.84 toks/s, output: 2124.85 toks/s]

Processed prompts:  31%|███       | 612/2000 [08:15<13:13,  1.75it/s, est. speed input: 266.57 toks/s, output: 2126.43 toks/s]

Processed prompts:  31%|███       | 616/2000 [08:15<09:41,  2.38it/s, est. speed input: 267.56 toks/s, output: 2127.09 toks/s]

Processed prompts:  31%|███       | 620/2000 [08:17<09:52,  2.33it/s, est. speed input: 267.94 toks/s, output: 2127.73 toks/s]

Processed prompts:  31%|███       | 624/2000 [08:18<08:48,  2.60it/s, est. speed input: 268.84 toks/s, output: 2145.69 toks/s]

Processed prompts:  31%|███▏      | 628/2000 [08:19<08:03,  2.84it/s, est. speed input: 270.06 toks/s, output: 2173.91 toks/s]

Processed prompts:  32%|███▏      | 632/2000 [08:21<08:00,  2.85it/s, est. speed input: 270.32 toks/s, output: 2169.91 toks/s]

Processed prompts:  32%|███▏      | 636/2000 [08:29<20:05,  1.13it/s, est. speed input: 267.08 toks/s, output: 2144.95 toks/s]

Processed prompts:  32%|███▏      | 640/2000 [08:29<14:35,  1.55it/s, est. speed input: 268.08 toks/s, output: 2191.59 toks/s]

Processed prompts:  32%|███▏      | 644/2000 [08:33<16:32,  1.37it/s, est. speed input: 267.41 toks/s, output: 2183.48 toks/s]

Processed prompts:  32%|███▏      | 648/2000 [08:34<12:51,  1.75it/s, est. speed input: 268.30 toks/s, output: 2219.46 toks/s]

Processed prompts:  33%|███▎      | 652/2000 [08:35<10:57,  2.05it/s, est. speed input: 268.89 toks/s, output: 2218.43 toks/s]

Processed prompts:  33%|███▎      | 656/2000 [08:36<09:14,  2.42it/s, est. speed input: 269.74 toks/s, output: 2251.91 toks/s]

Processed prompts:  33%|███▎      | 660/2000 [08:38<09:51,  2.27it/s, est. speed input: 269.65 toks/s, output: 2252.20 toks/s]

Processed prompts:  33%|███▎      | 664/2000 [08:39<08:15,  2.69it/s, est. speed input: 270.14 toks/s, output: 2250.18 toks/s]

Processed prompts:  33%|███▎      | 668/2000 [08:41<09:27,  2.35it/s, est. speed input: 270.76 toks/s, output: 2345.83 toks/s]

Processed prompts:  34%|███▎      | 672/2000 [08:43<09:02,  2.45it/s, est. speed input: 271.61 toks/s, output: 2363.30 toks/s]

Processed prompts:  34%|███▍      | 676/2000 [08:47<12:54,  1.71it/s, est. speed input: 270.53 toks/s, output: 2346.96 toks/s]

Processed prompts:  34%|███▍      | 684/2000 [08:47<07:05,  3.09it/s, est. speed input: 272.52 toks/s, output: 2357.43 toks/s]

Processed prompts:  34%|███▍      | 688/2000 [08:48<06:50,  3.20it/s, est. speed input: 272.97 toks/s, output: 2354.65 toks/s]

Processed prompts:  35%|███▍      | 692/2000 [08:49<07:04,  3.08it/s, est. speed input: 273.54 toks/s, output: 2360.98 toks/s]

Processed prompts:  35%|███▍      | 696/2000 [08:49<05:17,  4.11it/s, est. speed input: 274.67 toks/s, output: 2382.42 toks/s]

Processed prompts:  35%|███▌      | 700/2000 [08:52<08:01,  2.70it/s, est. speed input: 274.27 toks/s, output: 2373.53 toks/s]

Processed prompts:  35%|███▌      | 704/2000 [08:54<07:58,  2.71it/s, est. speed input: 274.48 toks/s, output: 2371.75 toks/s]

Processed prompts:  35%|███▌      | 708/2000 [08:56<09:20,  2.30it/s, est. speed input: 275.98 toks/s, output: 2454.46 toks/s]

Processed prompts:  36%|███▌      | 712/2000 [08:57<08:53,  2.42it/s, est. speed input: 276.28 toks/s, output: 2475.03 toks/s]

Processed prompts:  36%|███▌      | 716/2000 [09:00<09:28,  2.26it/s, est. speed input: 276.27 toks/s, output: 2471.43 toks/s]

Processed prompts:  36%|███▌      | 720/2000 [09:06<17:05,  1.25it/s, est. speed input: 273.83 toks/s, output: 2444.82 toks/s]

Processed prompts:  36%|███▌      | 724/2000 [09:07<13:09,  1.62it/s, est. speed input: 274.50 toks/s, output: 2447.98 toks/s]

Processed prompts:  36%|███▋      | 728/2000 [09:09<12:11,  1.74it/s, est. speed input: 274.59 toks/s, output: 2442.26 toks/s]

Processed prompts:  37%|███▋      | 732/2000 [09:09<09:18,  2.27it/s, est. speed input: 275.37 toks/s, output: 2443.12 toks/s]

Processed prompts:  37%|███▋      | 736/2000 [09:12<11:21,  1.85it/s, est. speed input: 275.17 toks/s, output: 2470.61 toks/s]

Processed prompts:  37%|███▋      | 740/2000 [09:15<12:24,  1.69it/s, est. speed input: 274.96 toks/s, output: 2504.64 toks/s]

Processed prompts:  37%|███▋      | 744/2000 [09:16<10:04,  2.08it/s, est. speed input: 275.47 toks/s, output: 2502.28 toks/s]

Processed prompts:  37%|███▋      | 748/2000 [09:17<08:42,  2.40it/s, est. speed input: 275.82 toks/s, output: 2500.60 toks/s]

Processed prompts:  38%|███▊      | 752/2000 [09:34<32:52,  1.58s/it, est. speed input: 268.69 toks/s, output: 2431.04 toks/s]

Processed prompts:  38%|███▊      | 756/2000 [09:36<25:27,  1.23s/it, est. speed input: 268.90 toks/s, output: 2431.00 toks/s]

Processed prompts:  38%|███▊      | 760/2000 [09:36<18:14,  1.13it/s, est. speed input: 269.80 toks/s, output: 2445.49 toks/s]

Processed prompts:  38%|███▊      | 764/2000 [09:41<20:42,  1.00s/it, est. speed input: 268.33 toks/s, output: 2428.43 toks/s]

Processed prompts:  38%|███▊      | 768/2000 [09:42<14:47,  1.39it/s, est. speed input: 269.26 toks/s, output: 2442.78 toks/s]

Processed prompts:  39%|███▊      | 772/2000 [09:47<17:53,  1.14it/s, est. speed input: 267.85 toks/s, output: 2424.21 toks/s]

Processed prompts:  39%|███▉      | 776/2000 [09:49<16:01,  1.27it/s, est. speed input: 267.70 toks/s, output: 2416.20 toks/s]

Processed prompts:  39%|███▉      | 780/2000 [09:49<11:55,  1.71it/s, est. speed input: 268.39 toks/s, output: 2419.50 toks/s]

Processed prompts:  39%|███▉      | 784/2000 [09:50<08:42,  2.33it/s, est. speed input: 270.21 toks/s, output: 2493.58 toks/s]

Processed prompts:  39%|███▉      | 788/2000 [09:50<07:13,  2.80it/s, est. speed input: 271.03 toks/s, output: 2502.96 toks/s]

Processed prompts:  40%|███▉      | 792/2000 [09:51<06:27,  3.11it/s, est. speed input: 271.43 toks/s, output: 2502.81 toks/s]

Processed prompts:  40%|███▉      | 796/2000 [09:57<13:37,  1.47it/s, est. speed input: 269.48 toks/s, output: 2478.97 toks/s]

Processed prompts:  40%|████      | 800/2000 [10:01<14:50,  1.35it/s, est. speed input: 268.68 toks/s, output: 2467.45 toks/s]

Processed prompts:  40%|████      | 804/2000 [10:02<11:30,  1.73it/s, est. speed input: 269.18 toks/s, output: 2466.95 toks/s]

Processed prompts:  40%|████      | 808/2000 [10:02<08:58,  2.22it/s, est. speed input: 269.83 toks/s, output: 2471.15 toks/s]

Processed prompts:  41%|████      | 812/2000 [10:05<10:38,  1.86it/s, est. speed input: 269.44 toks/s, output: 2464.69 toks/s]

Processed prompts:  41%|████      | 816/2000 [10:06<08:01,  2.46it/s, est. speed input: 270.31 toks/s, output: 2473.93 toks/s]

Processed prompts:  41%|████      | 820/2000 [10:15<18:47,  1.05it/s, est. speed input: 267.41 toks/s, output: 2448.04 toks/s]

Processed prompts:  41%|████      | 824/2000 [10:16<15:31,  1.26it/s, est. speed input: 267.66 toks/s, output: 2447.16 toks/s]

Processed prompts:  41%|████▏     | 828/2000 [10:21<18:06,  1.08it/s, est. speed input: 266.54 toks/s, output: 2458.90 toks/s]

Processed prompts:  42%|████▏     | 832/2000 [10:22<12:57,  1.50it/s, est. speed input: 267.37 toks/s, output: 2470.31 toks/s]

Processed prompts:  42%|████▏     | 836/2000 [10:25<14:40,  1.32it/s, est. speed input: 266.80 toks/s, output: 2478.04 toks/s]

Processed prompts:  42%|████▏     | 840/2000 [10:30<16:15,  1.19it/s, est. speed input: 265.81 toks/s, output: 2463.99 toks/s]

Processed prompts:  42%|████▏     | 844/2000 [10:36<20:50,  1.08s/it, est. speed input: 263.83 toks/s, output: 2446.94 toks/s]

Processed prompts:  42%|████▏     | 848/2000 [10:39<18:37,  1.03it/s, est. speed input: 263.66 toks/s, output: 2452.50 toks/s]

Processed prompts:  43%|████▎     | 852/2000 [10:42<16:51,  1.14it/s, est. speed input: 263.48 toks/s, output: 2454.39 toks/s]

Processed prompts:  43%|████▎     | 856/2000 [10:48<20:18,  1.06s/it, est. speed input: 262.02 toks/s, output: 2484.81 toks/s]

Processed prompts:  43%|████▎     | 860/2000 [10:55<23:58,  1.26s/it, est. speed input: 260.70 toks/s, output: 2557.21 toks/s]

Processed prompts:  43%|████▎     | 864/2000 [10:55<17:45,  1.07it/s, est. speed input: 261.26 toks/s, output: 2567.66 toks/s]

Processed prompts:  43%|████▎     | 868/2000 [10:59<17:42,  1.07it/s, est. speed input: 260.94 toks/s, output: 2619.34 toks/s]

Processed prompts:  44%|████▍     | 876/2000 [10:59<09:47,  1.91it/s, est. speed input: 262.62 toks/s, output: 2647.32 toks/s]

Processed prompts:  44%|████▍     | 880/2000 [10:59<07:31,  2.48it/s, est. speed input: 263.56 toks/s, output: 2672.45 toks/s]

Processed prompts:  44%|████▍     | 884/2000 [11:00<06:47,  2.74it/s, est. speed input: 264.13 toks/s, output: 2687.94 toks/s]

Processed prompts:  44%|████▍     | 888/2000 [11:04<09:41,  1.91it/s, est. speed input: 263.80 toks/s, output: 2695.26 toks/s]

Processed prompts:  45%|████▍     | 892/2000 [11:04<07:08,  2.59it/s, est. speed input: 264.57 toks/s, output: 2737.39 toks/s]

Processed prompts:  45%|████▍     | 896/2000 [11:05<06:10,  2.98it/s, est. speed input: 265.32 toks/s, output: 2789.01 toks/s]

Processed prompts:  45%|████▌     | 900/2000 [11:06<05:03,  3.63it/s, est. speed input: 265.81 toks/s, output: 2791.24 toks/s]

Processed prompts:  45%|████▌     | 904/2000 [11:08<06:23,  2.86it/s, est. speed input: 265.74 toks/s, output: 2783.25 toks/s]

Processed prompts:  45%|████▌     | 908/2000 [11:11<08:20,  2.18it/s, est. speed input: 265.35 toks/s, output: 2773.81 toks/s]

Processed prompts:  46%|████▌     | 912/2000 [11:17<14:54,  1.22it/s, est. speed input: 263.40 toks/s, output: 2748.74 toks/s]

Processed prompts:  46%|████▌     | 916/2000 [11:20<14:25,  1.25it/s, est. speed input: 263.14 toks/s, output: 2738.55 toks/s]

Processed prompts:  46%|████▌     | 920/2000 [11:21<10:20,  1.74it/s, est. speed input: 263.81 toks/s, output: 2747.91 toks/s]

Processed prompts:  46%|████▌     | 924/2000 [11:21<07:56,  2.26it/s, est. speed input: 264.37 toks/s, output: 2748.25 toks/s]

Processed prompts:  46%|████▋     | 928/2000 [11:25<11:01,  1.62it/s, est. speed input: 263.47 toks/s, output: 2734.76 toks/s]

Processed prompts:  47%|████▋     | 932/2000 [11:27<09:25,  1.89it/s, est. speed input: 263.77 toks/s, output: 2743.14 toks/s]

Processed prompts:  47%|████▋     | 936/2000 [11:44<29:56,  1.69s/it, est. speed input: 257.91 toks/s, output: 2680.47 toks/s]

Processed prompts:  47%|████▋     | 940/2000 [11:45<21:25,  1.21s/it, est. speed input: 258.62 toks/s, output: 2681.41 toks/s]

Processed prompts:  47%|████▋     | 944/2000 [11:49<20:49,  1.18s/it, est. speed input: 257.66 toks/s, output: 2667.95 toks/s]

Processed prompts:  48%|████▊     | 952/2000 [11:49<11:24,  1.53it/s, est. speed input: 258.97 toks/s, output: 2693.08 toks/s]

Processed prompts:  48%|████▊     | 956/2000 [11:53<12:37,  1.38it/s, est. speed input: 258.30 toks/s, output: 2679.86 toks/s]

Processed prompts:  48%|████▊     | 960/2000 [11:55<11:32,  1.50it/s, est. speed input: 258.38 toks/s, output: 2676.37 toks/s]

Processed prompts:  48%|████▊     | 964/2000 [12:01<15:40,  1.10it/s, est. speed input: 256.94 toks/s, output: 2681.22 toks/s]

Processed prompts:  48%|████▊     | 968/2000 [12:04<14:52,  1.16it/s, est. speed input: 256.93 toks/s, output: 2726.92 toks/s]

Processed prompts:  49%|████▊     | 972/2000 [12:06<12:53,  1.33it/s, est. speed input: 256.96 toks/s, output: 2724.26 toks/s]

Processed prompts:  49%|████▉     | 976/2000 [12:09<12:14,  1.39it/s, est. speed input: 256.76 toks/s, output: 2717.76 toks/s]

Processed prompts:  49%|████▉     | 980/2000 [12:09<09:18,  1.83it/s, est. speed input: 257.31 toks/s, output: 2718.01 toks/s]

Processed prompts:  49%|████▉     | 984/2000 [12:20<19:55,  1.18s/it, est. speed input: 254.22 toks/s, output: 2679.93 toks/s]

Processed prompts:  49%|████▉     | 988/2000 [12:23<17:12,  1.02s/it, est. speed input: 254.10 toks/s, output: 2673.63 toks/s]

Processed prompts:  50%|████▉     | 992/2000 [12:24<13:36,  1.23it/s, est. speed input: 254.45 toks/s, output: 2671.30 toks/s]

Processed prompts:  50%|████▉     | 996/2000 [12:27<13:34,  1.23it/s, est. speed input: 254.08 toks/s, output: 2665.73 toks/s]

Processed prompts:  50%|█████     | 1000/2000 [12:31<13:53,  1.20it/s, est. speed input: 253.57 toks/s, output: 2655.93 toks/s]

Processed prompts:  50%|█████     | 1004/2000 [12:31<10:02,  1.65it/s, est. speed input: 254.35 toks/s, output: 2687.51 toks/s]

Processed prompts:  50%|█████     | 1008/2000 [12:33<09:04,  1.82it/s, est. speed input: 254.61 toks/s, output: 2691.20 toks/s]

Processed prompts:  51%|█████     | 1012/2000 [12:33<07:01,  2.34it/s, est. speed input: 255.14 toks/s, output: 2699.64 toks/s]

Processed prompts:  51%|█████     | 1016/2000 [12:36<08:29,  1.93it/s, est. speed input: 254.96 toks/s, output: 2699.55 toks/s]

Processed prompts:  51%|█████     | 1020/2000 [12:37<06:35,  2.48it/s, est. speed input: 255.47 toks/s, output: 2703.58 toks/s]

Processed prompts:  51%|█████     | 1024/2000 [12:37<05:24,  3.01it/s, est. speed input: 255.97 toks/s, output: 2704.47 toks/s]

Processed prompts:  51%|█████▏    | 1028/2000 [12:41<07:52,  2.06it/s, est. speed input: 256.23 toks/s, output: 2776.43 toks/s]

Processed prompts:  52%|█████▏    | 1032/2000 [12:44<09:25,  1.71it/s, est. speed input: 255.78 toks/s, output: 2768.83 toks/s]

Processed prompts:  52%|█████▏    | 1036/2000 [12:47<10:26,  1.54it/s, est. speed input: 255.52 toks/s, output: 2769.60 toks/s]

Processed prompts:  52%|█████▏    | 1040/2000 [12:50<10:44,  1.49it/s, est. speed input: 255.27 toks/s, output: 2801.33 toks/s]

Processed prompts:  52%|█████▏    | 1044/2000 [12:51<09:03,  1.76it/s, est. speed input: 255.40 toks/s, output: 2799.15 toks/s]

Processed prompts:  52%|█████▏    | 1048/2000 [12:52<06:44,  2.35it/s, est. speed input: 256.12 toks/s, output: 2843.69 toks/s]

Processed prompts:  53%|█████▎    | 1052/2000 [12:57<10:38,  1.48it/s, est. speed input: 255.09 toks/s, output: 2827.18 toks/s]

Processed prompts:  53%|█████▎    | 1056/2000 [12:57<07:36,  2.07it/s, est. speed input: 255.69 toks/s, output: 2829.63 toks/s]

Processed prompts:  53%|█████▎    | 1060/2000 [13:01<09:58,  1.57it/s, est. speed input: 255.04 toks/s, output: 2819.58 toks/s]

Processed prompts:  53%|█████▎    | 1064/2000 [13:08<15:29,  1.01it/s, est. speed input: 253.63 toks/s, output: 2841.62 toks/s]

Processed prompts:  53%|█████▎    | 1068/2000 [13:09<11:22,  1.37it/s, est. speed input: 254.13 toks/s, output: 2845.81 toks/s]

Processed prompts:  54%|█████▎    | 1072/2000 [13:09<08:06,  1.91it/s, est. speed input: 254.68 toks/s, output: 2847.04 toks/s]

Processed prompts:  54%|█████▍    | 1076/2000 [13:10<07:06,  2.17it/s, est. speed input: 254.96 toks/s, output: 2843.83 toks/s]

Processed prompts:  54%|█████▍    | 1080/2000 [13:10<05:07,  2.99it/s, est. speed input: 255.52 toks/s, output: 2845.21 toks/s]

Processed prompts:  54%|█████▍    | 1080/2000 [13:20<05:07,  2.99it/s, est. speed input: 255.52 toks/s, output: 2845.21 toks/s]

Processed prompts:  54%|█████▍    | 1084/2000 [13:27<22:46,  1.49s/it, est. speed input: 250.96 toks/s, output: 2796.96 toks/s]

Processed prompts:  54%|█████▍    | 1088/2000 [13:29<18:10,  1.20s/it, est. speed input: 251.10 toks/s, output: 2800.23 toks/s]

Processed prompts:  55%|█████▍    | 1092/2000 [13:33<16:53,  1.12s/it, est. speed input: 250.61 toks/s, output: 2789.10 toks/s]

Processed prompts:  55%|█████▍    | 1096/2000 [13:33<12:18,  1.22it/s, est. speed input: 251.04 toks/s, output: 2789.80 toks/s]

Processed prompts:  55%|█████▌    | 1100/2000 [13:35<10:56,  1.37it/s, est. speed input: 250.94 toks/s, output: 2786.43 toks/s]

Processed prompts:  55%|█████▌    | 1104/2000 [13:36<08:20,  1.79it/s, est. speed input: 251.40 toks/s, output: 2785.45 toks/s]

Processed prompts:  55%|█████▌    | 1108/2000 [13:43<13:09,  1.13it/s, est. speed input: 250.11 toks/s, output: 2771.79 toks/s]

Processed prompts:  56%|█████▌    | 1112/2000 [13:43<09:46,  1.51it/s, est. speed input: 250.53 toks/s, output: 2771.12 toks/s]

Processed prompts:  56%|█████▌    | 1116/2000 [13:51<15:10,  1.03s/it, est. speed input: 248.91 toks/s, output: 2768.07 toks/s]

Processed prompts:  56%|█████▌    | 1120/2000 [13:54<13:56,  1.05it/s, est. speed input: 248.77 toks/s, output: 2787.06 toks/s]

Processed prompts:  56%|█████▌    | 1124/2000 [13:59<15:07,  1.04s/it, est. speed input: 248.03 toks/s, output: 2789.38 toks/s]

Processed prompts:  56%|█████▋    | 1128/2000 [14:03<14:47,  1.02s/it, est. speed input: 247.45 toks/s, output: 2779.66 toks/s]

Processed prompts:  57%|█████▋    | 1132/2000 [14:12<20:45,  1.43s/it, est. speed input: 245.32 toks/s, output: 2751.09 toks/s]

Processed prompts:  57%|█████▋    | 1136/2000 [14:13<14:50,  1.03s/it, est. speed input: 245.87 toks/s, output: 2751.99 toks/s]

Processed prompts:  57%|█████▋    | 1140/2000 [14:15<13:19,  1.08it/s, est. speed input: 245.70 toks/s, output: 2745.07 toks/s]

Processed prompts:  57%|█████▋    | 1144/2000 [14:17<11:25,  1.25it/s, est. speed input: 245.65 toks/s, output: 2740.07 toks/s]

Processed prompts:  57%|█████▋    | 1148/2000 [14:19<10:08,  1.40it/s, est. speed input: 245.70 toks/s, output: 2735.05 toks/s]

Processed prompts:  58%|█████▊    | 1152/2000 [14:22<10:05,  1.40it/s, est. speed input: 245.47 toks/s, output: 2733.35 toks/s]

Processed prompts:  58%|█████▊    | 1156/2000 [14:24<08:30,  1.65it/s, est. speed input: 245.72 toks/s, output: 2743.16 toks/s]

Processed prompts:  58%|█████▊    | 1160/2000 [14:28<10:24,  1.35it/s, est. speed input: 245.10 toks/s, output: 2738.24 toks/s]

Processed prompts:  58%|█████▊    | 1164/2000 [14:28<07:29,  1.86it/s, est. speed input: 245.64 toks/s, output: 2741.35 toks/s]

Processed prompts:  58%|█████▊    | 1168/2000 [14:29<06:38,  2.09it/s, est. speed input: 245.89 toks/s, output: 2738.29 toks/s]

Processed prompts:  59%|█████▊    | 1172/2000 [14:33<08:16,  1.67it/s, est. speed input: 245.48 toks/s, output: 2728.18 toks/s]

Processed prompts:  59%|█████▉    | 1180/2000 [14:35<05:52,  2.32it/s, est. speed input: 246.16 toks/s, output: 2741.17 toks/s]

Processed prompts:  59%|█████▉    | 1184/2000 [14:40<08:41,  1.56it/s, est. speed input: 245.33 toks/s, output: 2727.98 toks/s]

Processed prompts:  59%|█████▉    | 1188/2000 [14:43<09:18,  1.45it/s, est. speed input: 245.10 toks/s, output: 2730.58 toks/s]

Processed prompts:  60%|█████▉    | 1192/2000 [14:44<07:07,  1.89it/s, est. speed input: 245.50 toks/s, output: 2731.23 toks/s]

Processed prompts:  60%|█████▉    | 1196/2000 [14:44<05:21,  2.50it/s, est. speed input: 246.15 toks/s, output: 2739.61 toks/s]

Processed prompts:  60%|██████    | 1200/2000 [14:44<03:55,  3.40it/s, est. speed input: 246.77 toks/s, output: 2741.64 toks/s]

Processed prompts:  60%|██████    | 1204/2000 [14:50<08:30,  1.56it/s, est. speed input: 245.69 toks/s, output: 2725.99 toks/s]

Processed prompts:  60%|██████    | 1208/2000 [14:50<06:24,  2.06it/s, est. speed input: 246.09 toks/s, output: 2726.71 toks/s]

Processed prompts:  61%|██████    | 1212/2000 [14:51<05:01,  2.62it/s, est. speed input: 246.50 toks/s, output: 2737.27 toks/s]

Processed prompts:  61%|██████    | 1216/2000 [14:52<04:02,  3.24it/s, est. speed input: 246.89 toks/s, output: 2738.85 toks/s]

Processed prompts:  61%|██████    | 1224/2000 [14:52<02:37,  4.94it/s, est. speed input: 247.77 toks/s, output: 2744.74 toks/s]

Processed prompts:  61%|██████▏   | 1228/2000 [14:57<06:01,  2.14it/s, est. speed input: 246.87 toks/s, output: 2730.52 toks/s]

Processed prompts:  62%|██████▏   | 1232/2000 [14:58<04:41,  2.73it/s, est. speed input: 247.27 toks/s, output: 2730.87 toks/s]

Processed prompts:  62%|██████▏   | 1236/2000 [14:59<04:21,  2.92it/s, est. speed input: 247.51 toks/s, output: 2730.13 toks/s]

Processed prompts:  62%|██████▏   | 1240/2000 [15:01<04:51,  2.61it/s, est. speed input: 247.57 toks/s, output: 2731.89 toks/s]

Processed prompts:  62%|██████▏   | 1244/2000 [15:02<04:59,  2.53it/s, est. speed input: 247.76 toks/s, output: 2740.08 toks/s]

Processed prompts:  62%|██████▏   | 1248/2000 [15:09<09:20,  1.34it/s, est. speed input: 246.49 toks/s, output: 2722.53 toks/s]

Processed prompts:  63%|██████▎   | 1252/2000 [15:12<09:09,  1.36it/s, est. speed input: 246.26 toks/s, output: 2716.35 toks/s]

Processed prompts:  63%|██████▎   | 1256/2000 [15:13<08:06,  1.53it/s, est. speed input: 246.42 toks/s, output: 2731.95 toks/s]

Processed prompts:  63%|██████▎   | 1260/2000 [15:15<07:08,  1.73it/s, est. speed input: 246.57 toks/s, output: 2740.60 toks/s]

Processed prompts:  63%|██████▎   | 1264/2000 [15:19<08:20,  1.47it/s, est. speed input: 246.13 toks/s, output: 2730.88 toks/s]

Processed prompts:  63%|██████▎   | 1268/2000 [15:20<07:25,  1.64it/s, est. speed input: 246.20 toks/s, output: 2727.85 toks/s]

Processed prompts:  64%|██████▎   | 1272/2000 [15:21<05:53,  2.06it/s, est. speed input: 246.63 toks/s, output: 2741.02 toks/s]

Processed prompts:  64%|██████▍   | 1276/2000 [15:22<04:45,  2.53it/s, est. speed input: 247.06 toks/s, output: 2741.68 toks/s]

Processed prompts:  64%|██████▍   | 1280/2000 [15:26<06:59,  1.72it/s, est. speed input: 246.56 toks/s, output: 2742.16 toks/s]

Processed prompts:  64%|██████▍   | 1284/2000 [15:26<05:02,  2.37it/s, est. speed input: 247.05 toks/s, output: 2743.10 toks/s]

Processed prompts:  64%|██████▍   | 1288/2000 [15:27<04:04,  2.91it/s, est. speed input: 247.39 toks/s, output: 2745.41 toks/s]

Processed prompts:  65%|██████▍   | 1292/2000 [15:28<03:38,  3.24it/s, est. speed input: 247.68 toks/s, output: 2743.86 toks/s]

Processed prompts:  65%|██████▍   | 1296/2000 [15:30<04:05,  2.87it/s, est. speed input: 247.72 toks/s, output: 2740.64 toks/s]

Processed prompts:  65%|██████▌   | 1300/2000 [15:31<03:46,  3.10it/s, est. speed input: 247.89 toks/s, output: 2741.09 toks/s]

Processed prompts:  65%|██████▌   | 1304/2000 [15:31<02:59,  3.88it/s, est. speed input: 248.26 toks/s, output: 2741.23 toks/s]

Processed prompts:  66%|██████▌   | 1312/2000 [15:34<03:19,  3.45it/s, est. speed input: 248.74 toks/s, output: 2749.57 toks/s]

Processed prompts:  66%|██████▌   | 1316/2000 [15:34<02:33,  4.45it/s, est. speed input: 249.18 toks/s, output: 2755.13 toks/s]

Processed prompts:  66%|██████▌   | 1320/2000 [15:38<05:16,  2.15it/s, est. speed input: 248.47 toks/s, output: 2746.01 toks/s]

Processed prompts:  66%|██████▌   | 1324/2000 [15:39<03:56,  2.86it/s, est. speed input: 248.98 toks/s, output: 2747.08 toks/s]

Processed prompts:  66%|██████▋   | 1328/2000 [15:42<05:55,  1.89it/s, est. speed input: 248.45 toks/s, output: 2737.23 toks/s]

Processed prompts:  67%|██████▋   | 1332/2000 [15:44<05:33,  2.00it/s, est. speed input: 248.54 toks/s, output: 2735.67 toks/s]

Processed prompts:  67%|██████▋   | 1336/2000 [15:47<06:08,  1.80it/s, est. speed input: 248.41 toks/s, output: 2750.11 toks/s]

Processed prompts:  67%|██████▋   | 1340/2000 [15:48<05:01,  2.19it/s, est. speed input: 248.71 toks/s, output: 2754.20 toks/s]

Processed prompts:  67%|██████▋   | 1344/2000 [15:49<04:27,  2.45it/s, est. speed input: 248.86 toks/s, output: 2751.98 toks/s]

Processed prompts:  67%|██████▋   | 1348/2000 [15:50<03:44,  2.91it/s, est. speed input: 249.19 toks/s, output: 2751.91 toks/s]

Processed prompts:  68%|██████▊   | 1356/2000 [15:51<02:32,  4.22it/s, est. speed input: 249.96 toks/s, output: 2757.27 toks/s]

Processed prompts:  68%|██████▊   | 1360/2000 [15:51<02:06,  5.08it/s, est. speed input: 250.33 toks/s, output: 2757.37 toks/s]

Processed prompts:  68%|██████▊   | 1364/2000 [15:51<01:50,  5.78it/s, est. speed input: 250.77 toks/s, output: 2772.78 toks/s]

Processed prompts:  68%|██████▊   | 1368/2000 [15:55<03:42,  2.84it/s, est. speed input: 250.45 toks/s, output: 2765.33 toks/s]

Processed prompts:  69%|██████▊   | 1372/2000 [15:56<03:46,  2.78it/s, est. speed input: 250.57 toks/s, output: 2763.65 toks/s]

Processed prompts:  69%|██████▉   | 1376/2000 [16:01<06:09,  1.69it/s, est. speed input: 249.80 toks/s, output: 2752.38 toks/s]

Processed prompts:  69%|██████▉   | 1380/2000 [16:04<06:34,  1.57it/s, est. speed input: 249.50 toks/s, output: 2745.99 toks/s]

Processed prompts:  69%|██████▉   | 1384/2000 [16:06<06:05,  1.69it/s, est. speed input: 249.54 toks/s, output: 2743.12 toks/s]

Processed prompts:  69%|██████▉   | 1388/2000 [16:07<04:56,  2.07it/s, est. speed input: 249.76 toks/s, output: 2742.15 toks/s]

Processed prompts:  70%|██████▉   | 1392/2000 [16:07<03:53,  2.61it/s, est. speed input: 250.14 toks/s, output: 2744.00 toks/s]

Processed prompts:  70%|██████▉   | 1396/2000 [16:08<02:51,  3.52it/s, est. speed input: 250.60 toks/s, output: 2744.99 toks/s]

Processed prompts:  70%|███████   | 1404/2000 [16:13<04:34,  2.17it/s, est. speed input: 250.22 toks/s, output: 2735.64 toks/s]

Processed prompts:  70%|███████   | 1408/2000 [16:13<03:46,  2.61it/s, est. speed input: 250.54 toks/s, output: 2735.19 toks/s]

Processed prompts:  71%|███████   | 1412/2000 [16:16<04:21,  2.25it/s, est. speed input: 250.41 toks/s, output: 2735.15 toks/s]

Processed prompts:  71%|███████   | 1416/2000 [16:18<04:31,  2.15it/s, est. speed input: 250.36 toks/s, output: 2732.09 toks/s]

Processed prompts:  71%|███████   | 1420/2000 [16:19<03:44,  2.59it/s, est. speed input: 250.60 toks/s, output: 2730.89 toks/s]

Processed prompts:  71%|███████   | 1424/2000 [16:20<03:42,  2.59it/s, est. speed input: 250.87 toks/s, output: 2743.51 toks/s]

Processed prompts:  71%|███████▏  | 1428/2000 [16:21<03:27,  2.76it/s, est. speed input: 251.04 toks/s, output: 2741.46 toks/s]

Processed prompts:  72%|███████▏  | 1432/2000 [16:22<02:42,  3.50it/s, est. speed input: 251.40 toks/s, output: 2741.35 toks/s]

Processed prompts:  72%|███████▏  | 1440/2000 [16:23<02:02,  4.58it/s, est. speed input: 252.17 toks/s, output: 2771.99 toks/s]

Processed prompts:  72%|███████▏  | 1444/2000 [16:24<01:49,  5.06it/s, est. speed input: 252.53 toks/s, output: 2773.96 toks/s]

Processed prompts:  72%|███████▏  | 1448/2000 [16:24<01:31,  6.01it/s, est. speed input: 252.94 toks/s, output: 2783.17 toks/s]

Processed prompts:  73%|███████▎  | 1452/2000 [16:29<04:05,  2.24it/s, est. speed input: 252.17 toks/s, output: 2771.10 toks/s]

Processed prompts:  73%|███████▎  | 1468/2000 [16:30<02:02,  4.34it/s, est. speed input: 253.86 toks/s, output: 2814.77 toks/s]

Processed prompts:  74%|███████▎  | 1472/2000 [16:31<01:53,  4.63it/s, est. speed input: 254.29 toks/s, output: 2830.51 toks/s]

Processed prompts:  74%|███████▍  | 1476/2000 [16:32<02:15,  3.88it/s, est. speed input: 254.33 toks/s, output: 2826.88 toks/s]

Processed prompts:  74%|███████▍  | 1480/2000 [16:33<02:03,  4.21it/s, est. speed input: 254.63 toks/s, output: 2826.53 toks/s]

Processed prompts:  74%|███████▍  | 1484/2000 [16:34<02:00,  4.27it/s, est. speed input: 254.83 toks/s, output: 2826.23 toks/s]

Processed prompts:  74%|███████▍  | 1488/2000 [16:35<01:52,  4.56it/s, est. speed input: 255.12 toks/s, output: 2826.05 toks/s]

Processed prompts:  75%|███████▍  | 1492/2000 [16:35<01:32,  5.48it/s, est. speed input: 255.57 toks/s, output: 2829.23 toks/s]

Processed prompts:  75%|███████▍  | 1496/2000 [16:37<02:04,  4.05it/s, est. speed input: 255.59 toks/s, output: 2825.27 toks/s]

Processed prompts:  75%|███████▌  | 1500/2000 [16:37<01:48,  4.62it/s, est. speed input: 255.92 toks/s, output: 2825.38 toks/s]

Processed prompts:  75%|███████▌  | 1504/2000 [16:38<01:45,  4.71it/s, est. speed input: 256.20 toks/s, output: 2824.18 toks/s]

Processed prompts:  75%|███████▌  | 1508/2000 [16:40<02:13,  3.69it/s, est. speed input: 256.19 toks/s, output: 2820.59 toks/s]

Processed prompts:  76%|███████▌  | 1512/2000 [16:40<01:49,  4.44it/s, est. speed input: 256.54 toks/s, output: 2820.19 toks/s]

Processed prompts:  76%|███████▌  | 1516/2000 [16:41<01:36,  5.02it/s, est. speed input: 256.97 toks/s, output: 2823.77 toks/s]

Processed prompts:  76%|███████▌  | 1520/2000 [16:42<01:48,  4.44it/s, est. speed input: 257.10 toks/s, output: 2821.43 toks/s]

Processed prompts:  76%|███████▌  | 1524/2000 [16:44<02:15,  3.51it/s, est. speed input: 257.13 toks/s, output: 2817.97 toks/s]

Processed prompts:  76%|███████▋  | 1528/2000 [16:46<02:47,  2.82it/s, est. speed input: 257.05 toks/s, output: 2813.27 toks/s]

Processed prompts:  77%|███████▋  | 1536/2000 [16:46<01:42,  4.53it/s, est. speed input: 257.89 toks/s, output: 2848.25 toks/s]

Processed prompts:  77%|███████▋  | 1540/2000 [16:47<01:48,  4.22it/s, est. speed input: 258.01 toks/s, output: 2846.19 toks/s]

Processed prompts:  77%|███████▋  | 1544/2000 [16:48<01:41,  4.48it/s, est. speed input: 258.28 toks/s, output: 2845.14 toks/s]

Processed prompts:  77%|███████▋  | 1548/2000 [16:53<03:35,  2.09it/s, est. speed input: 257.49 toks/s, output: 2834.16 toks/s]

Processed prompts:  78%|███████▊  | 1552/2000 [16:53<02:38,  2.82it/s, est. speed input: 257.87 toks/s, output: 2835.35 toks/s]

Processed prompts:  78%|███████▊  | 1556/2000 [16:55<02:51,  2.60it/s, est. speed input: 257.85 toks/s, output: 2832.58 toks/s]

Processed prompts:  78%|███████▊  | 1560/2000 [16:55<02:07,  3.45it/s, est. speed input: 258.27 toks/s, output: 2832.77 toks/s]

Processed prompts:  78%|███████▊  | 1564/2000 [16:56<02:04,  3.49it/s, est. speed input: 258.43 toks/s, output: 2832.02 toks/s]

Processed prompts:  78%|███████▊  | 1568/2000 [16:57<02:06,  3.42it/s, est. speed input: 258.55 toks/s, output: 2830.47 toks/s]

Processed prompts:  79%|███████▊  | 1572/2000 [16:57<01:34,  4.52it/s, est. speed input: 258.99 toks/s, output: 2831.24 toks/s]

Processed prompts:  79%|███████▉  | 1576/2000 [17:01<02:46,  2.54it/s, est. speed input: 258.64 toks/s, output: 2823.57 toks/s]

Processed prompts:  79%|███████▉  | 1580/2000 [17:01<02:11,  3.19it/s, est. speed input: 258.98 toks/s, output: 2825.93 toks/s]

Processed prompts:  79%|███████▉  | 1588/2000 [17:02<01:22,  4.98it/s, est. speed input: 259.74 toks/s, output: 2830.27 toks/s]

Processed prompts:  80%|███████▉  | 1592/2000 [17:02<01:09,  5.89it/s, est. speed input: 260.07 toks/s, output: 2831.20 toks/s]

Processed prompts:  80%|███████▉  | 1596/2000 [17:04<01:48,  3.72it/s, est. speed input: 259.95 toks/s, output: 2825.70 toks/s]

Processed prompts:  80%|████████  | 1600/2000 [17:05<01:29,  4.45it/s, est. speed input: 260.36 toks/s, output: 2846.05 toks/s]

Processed prompts:  80%|████████  | 1604/2000 [17:06<01:43,  3.82it/s, est. speed input: 260.44 toks/s, output: 2842.98 toks/s]

Processed prompts:  80%|████████  | 1608/2000 [17:06<01:16,  5.13it/s, est. speed input: 260.88 toks/s, output: 2844.41 toks/s]

Processed prompts:  81%|████████  | 1612/2000 [17:07<01:30,  4.31it/s, est. speed input: 261.03 toks/s, output: 2845.66 toks/s]

Processed prompts:  81%|████████  | 1616/2000 [17:09<01:50,  3.46it/s, est. speed input: 261.06 toks/s, output: 2854.54 toks/s]

Processed prompts:  81%|████████  | 1620/2000 [17:11<02:06,  3.02it/s, est. speed input: 261.15 toks/s, output: 2864.88 toks/s]

Processed prompts:  81%|████████  | 1624/2000 [17:14<02:56,  2.13it/s, est. speed input: 260.78 toks/s, output: 2857.11 toks/s]

Processed prompts:  81%|████████▏ | 1628/2000 [17:14<02:06,  2.93it/s, est. speed input: 261.38 toks/s, output: 2895.90 toks/s]

Processed prompts:  82%|████████▏ | 1632/2000 [17:16<02:07,  2.90it/s, est. speed input: 261.47 toks/s, output: 2893.01 toks/s]

Processed prompts:  82%|████████▏ | 1636/2000 [17:17<02:12,  2.74it/s, est. speed input: 261.52 toks/s, output: 2889.72 toks/s]

Processed prompts:  82%|████████▏ | 1640/2000 [17:18<01:49,  3.28it/s, est. speed input: 261.80 toks/s, output: 2888.91 toks/s]

Processed prompts:  82%|████████▏ | 1644/2000 [17:19<01:52,  3.17it/s, est. speed input: 261.88 toks/s, output: 2892.40 toks/s]

Processed prompts:  82%|████████▏ | 1648/2000 [17:20<01:34,  3.73it/s, est. speed input: 262.14 toks/s, output: 2892.07 toks/s]

Processed prompts:  83%|████████▎ | 1652/2000 [17:20<01:10,  4.97it/s, est. speed input: 262.55 toks/s, output: 2892.80 toks/s]

Processed prompts:  83%|████████▎ | 1656/2000 [17:21<01:04,  5.30it/s, est. speed input: 262.81 toks/s, output: 2893.42 toks/s]

Processed prompts:  83%|████████▎ | 1660/2000 [17:21<01:03,  5.32it/s, est. speed input: 263.03 toks/s, output: 2892.99 toks/s]

Processed prompts:  83%|████████▎ | 1664/2000 [17:22<00:48,  6.92it/s, est. speed input: 263.55 toks/s, output: 2928.69 toks/s]

Processed prompts:  83%|████████▎ | 1668/2000 [17:22<00:37,  8.94it/s, est. speed input: 263.95 toks/s, output: 2929.85 toks/s]

Processed prompts:  84%|████████▎ | 1672/2000 [17:22<00:39,  8.38it/s, est. speed input: 264.25 toks/s, output: 2929.45 toks/s]

Processed prompts:  84%|████████▍ | 1676/2000 [17:25<01:29,  3.62it/s, est. speed input: 264.04 toks/s, output: 2923.76 toks/s]

Processed prompts:  84%|████████▍ | 1680/2000 [17:25<01:15,  4.27it/s, est. speed input: 264.37 toks/s, output: 2925.93 toks/s]

Processed prompts:  84%|████████▍ | 1684/2000 [17:27<01:29,  3.53it/s, est. speed input: 264.38 toks/s, output: 2922.16 toks/s]

Processed prompts:  84%|████████▍ | 1688/2000 [17:28<01:35,  3.28it/s, est. speed input: 264.42 toks/s, output: 2919.26 toks/s]

Processed prompts:  85%|████████▍ | 1692/2000 [17:29<01:11,  4.31it/s, est. speed input: 264.74 toks/s, output: 2919.83 toks/s]

Processed prompts:  85%|████████▍ | 1696/2000 [17:29<01:03,  4.81it/s, est. speed input: 265.02 toks/s, output: 2918.94 toks/s]

Processed prompts:  85%|████████▌ | 1700/2000 [17:31<01:20,  3.74it/s, est. speed input: 265.04 toks/s, output: 2915.67 toks/s]

Processed prompts:  85%|████████▌ | 1704/2000 [17:31<00:57,  5.13it/s, est. speed input: 265.45 toks/s, output: 2916.50 toks/s]

Processed prompts:  85%|████████▌ | 1708/2000 [17:33<01:16,  3.84it/s, est. speed input: 265.44 toks/s, output: 2913.49 toks/s]

Processed prompts:  86%|████████▌ | 1716/2000 [17:34<00:53,  5.31it/s, est. speed input: 266.06 toks/s, output: 2916.13 toks/s]

Processed prompts:  86%|████████▌ | 1720/2000 [17:34<00:46,  6.04it/s, est. speed input: 266.35 toks/s, output: 2915.64 toks/s]

Processed prompts:  86%|████████▌ | 1724/2000 [17:36<01:13,  3.73it/s, est. speed input: 266.16 toks/s, output: 2910.24 toks/s]

Processed prompts:  86%|████████▋ | 1728/2000 [17:37<01:02,  4.38it/s, est. speed input: 266.42 toks/s, output: 2909.50 toks/s]

Processed prompts:  87%|████████▋ | 1736/2000 [17:37<00:40,  6.58it/s, est. speed input: 267.20 toks/s, output: 2913.43 toks/s]

Processed prompts:  87%|████████▋ | 1740/2000 [17:37<00:33,  7.74it/s, est. speed input: 267.51 toks/s, output: 2913.15 toks/s]

Processed prompts:  87%|████████▋ | 1744/2000 [17:38<00:33,  7.65it/s, est. speed input: 267.94 toks/s, output: 2947.12 toks/s]

Processed prompts:  87%|████████▋ | 1748/2000 [17:38<00:26,  9.56it/s, est. speed input: 268.30 toks/s, output: 2947.69 toks/s]

Processed prompts:  88%|████████▊ | 1752/2000 [17:38<00:24, 10.29it/s, est. speed input: 268.62 toks/s, output: 2949.05 toks/s]

Processed prompts:  88%|████████▊ | 1756/2000 [17:38<00:19, 12.73it/s, est. speed input: 269.03 toks/s, output: 2950.13 toks/s]

Processed prompts:  88%|████████▊ | 1764/2000 [17:39<00:19, 11.95it/s, est. speed input: 269.62 toks/s, output: 2950.86 toks/s]

Processed prompts:  88%|████████▊ | 1768/2000 [17:40<00:34,  6.78it/s, est. speed input: 269.72 toks/s, output: 2951.72 toks/s]

Processed prompts:  89%|████████▊ | 1772/2000 [17:41<00:26,  8.49it/s, est. speed input: 270.09 toks/s, output: 2952.16 toks/s]

Processed prompts:  89%|████████▉ | 1776/2000 [17:41<00:23,  9.54it/s, est. speed input: 270.42 toks/s, output: 2952.23 toks/s]

Processed prompts:  89%|████████▉ | 1780/2000 [17:42<00:26,  8.38it/s, est. speed input: 270.66 toks/s, output: 2951.46 toks/s]

Processed prompts:  89%|████████▉ | 1784/2000 [17:42<00:29,  7.20it/s, est. speed input: 270.89 toks/s, output: 2957.50 toks/s]

Processed prompts:  89%|████████▉ | 1788/2000 [17:43<00:28,  7.52it/s, est. speed input: 271.18 toks/s, output: 2957.11 toks/s]

Processed prompts:  90%|████████▉ | 1792/2000 [17:43<00:22,  9.28it/s, est. speed input: 271.53 toks/s, output: 2957.38 toks/s]

Processed prompts:  90%|████████▉ | 1796/2000 [17:44<00:26,  7.59it/s, est. speed input: 271.75 toks/s, output: 2956.46 toks/s]

Processed prompts:  90%|█████████ | 1800/2000 [17:45<00:33,  6.00it/s, est. speed input: 271.96 toks/s, output: 2958.67 toks/s]

Processed prompts:  90%|█████████ | 1804/2000 [17:45<00:31,  6.23it/s, est. speed input: 272.18 toks/s, output: 2957.89 toks/s]

Processed prompts:  90%|█████████ | 1808/2000 [17:47<00:40,  4.77it/s, est. speed input: 272.24 toks/s, output: 2954.96 toks/s]

Processed prompts:  91%|█████████ | 1816/2000 [17:47<00:23,  7.81it/s, est. speed input: 272.96 toks/s, output: 2958.40 toks/s]

Processed prompts:  91%|█████████ | 1820/2000 [17:47<00:19,  9.27it/s, est. speed input: 273.49 toks/s, output: 3002.45 toks/s]

Processed prompts:  91%|█████████▏| 1828/2000 [17:48<00:16, 10.38it/s, est. speed input: 274.15 toks/s, output: 3004.04 toks/s]

Processed prompts:  92%|█████████▏| 1832/2000 [17:49<00:24,  6.78it/s, est. speed input: 274.21 toks/s, output: 3001.19 toks/s]

Processed prompts:  92%|█████████▏| 1836/2000 [17:49<00:22,  7.34it/s, est. speed input: 274.50 toks/s, output: 3000.83 toks/s]

Processed prompts:  92%|█████████▏| 1840/2000 [17:50<00:22,  6.97it/s, est. speed input: 274.70 toks/s, output: 2999.58 toks/s]

Processed prompts:  92%|█████████▏| 1848/2000 [17:51<00:16,  9.03it/s, est. speed input: 275.40 toks/s, output: 3012.58 toks/s]

Processed prompts:  93%|█████████▎| 1856/2000 [17:51<00:10, 13.32it/s, est. speed input: 276.12 toks/s, output: 3014.19 toks/s]

Processed prompts:  93%|█████████▎| 1864/2000 [17:52<00:12, 10.88it/s, est. speed input: 276.72 toks/s, output: 3019.42 toks/s]

Processed prompts:  94%|█████████▎| 1872/2000 [17:52<00:10, 12.10it/s, est. speed input: 277.33 toks/s, output: 3020.38 toks/s]

Processed prompts:  94%|█████████▍| 1876/2000 [17:53<00:11, 10.98it/s, est. speed input: 277.61 toks/s, output: 3022.19 toks/s]

Processed prompts:  94%|█████████▍| 1880/2000 [17:53<00:13,  9.10it/s, est. speed input: 277.78 toks/s, output: 3021.22 toks/s]

Processed prompts:  94%|█████████▍| 1884/2000 [17:54<00:13,  8.58it/s, est. speed input: 278.05 toks/s, output: 3020.95 toks/s]

Processed prompts:  94%|█████████▍| 1888/2000 [17:58<00:37,  3.01it/s, est. speed input: 277.51 toks/s, output: 3021.06 toks/s]

Processed prompts:  95%|█████████▍| 1892/2000 [17:58<00:29,  3.67it/s, est. speed input: 277.77 toks/s, output: 3021.53 toks/s]

Processed prompts:  95%|█████████▍| 1896/2000 [17:59<00:24,  4.24it/s, est. speed input: 277.99 toks/s, output: 3021.64 toks/s]

Processed prompts:  95%|█████████▌| 1900/2000 [17:59<00:18,  5.41it/s, est. speed input: 278.34 toks/s, output: 3023.17 toks/s]

Processed prompts:  95%|█████████▌| 1904/2000 [18:01<00:25,  3.74it/s, est. speed input: 278.30 toks/s, output: 3022.17 toks/s]

Processed prompts:  95%|█████████▌| 1908/2000 [18:03<00:30,  3.03it/s, est. speed input: 278.20 toks/s, output: 3020.70 toks/s]

Processed prompts:  96%|█████████▌| 1912/2000 [18:06<00:42,  2.07it/s, est. speed input: 277.78 toks/s, output: 3024.82 toks/s]

Processed prompts:  96%|█████████▌| 1916/2000 [18:09<00:44,  1.87it/s, est. speed input: 277.49 toks/s, output: 3022.39 toks/s]

Processed prompts:  96%|█████████▌| 1920/2000 [18:10<00:39,  2.05it/s, est. speed input: 277.61 toks/s, output: 3040.21 toks/s]

Processed prompts:  96%|█████████▌| 1924/2000 [18:20<01:22,  1.09s/it, est. speed input: 275.52 toks/s, output: 3017.25 toks/s]

Processed prompts:  96%|█████████▋| 1928/2000 [18:22<01:01,  1.17it/s, est. speed input: 275.70 toks/s, output: 3047.42 toks/s]

Processed prompts:  97%|█████████▋| 1932/2000 [18:40<02:13,  1.96s/it, est. speed input: 271.64 toks/s, output: 3008.64 toks/s]

Processed prompts:  97%|█████████▋| 1936/2000 [18:44<01:46,  1.67s/it, est. speed input: 271.29 toks/s, output: 3053.36 toks/s]

Processed prompts:  97%|█████████▋| 1940/2000 [18:44<01:13,  1.22s/it, est. speed input: 271.52 toks/s, output: 3058.19 toks/s]

Processed prompts:  97%|█████████▋| 1944/2000 [18:45<00:50,  1.12it/s, est. speed input: 271.87 toks/s, output: 3072.38 toks/s]

Processed prompts:  97%|█████████▋| 1948/2000 [19:02<01:38,  1.89s/it, est. speed input: 268.33 toks/s, output: 3044.08 toks/s]

Processed prompts:  98%|█████████▊| 1952/2000 [19:12<01:40,  2.09s/it, est. speed input: 266.63 toks/s, output: 3070.06 toks/s]

Processed prompts:  98%|█████████▊| 1956/2000 [19:22<01:35,  2.17s/it, est. speed input: 264.90 toks/s, output: 3067.99 toks/s]

Processed prompts:  98%|█████████▊| 1960/2000 [19:24<01:09,  1.73s/it, est. speed input: 264.81 toks/s, output: 3094.60 toks/s]

Processed prompts:  98%|█████████▊| 1964/2000 [19:28<00:53,  1.49s/it, est. speed input: 264.32 toks/s, output: 3100.84 toks/s]

Processed prompts:  98%|█████████▊| 1968/2000 [19:29<00:36,  1.15s/it, est. speed input: 264.71 toks/s, output: 3152.24 toks/s]

Processed prompts:  99%|█████████▊| 1972/2000 [19:45<00:54,  1.93s/it, est. speed input: 261.71 toks/s, output: 3124.91 toks/s]

Processed prompts:  99%|█████████▉| 1976/2000 [20:48<02:25,  6.08s/it, est. speed input: 248.96 toks/s, output: 2998.85 toks/s]

Processed prompts:  99%|█████████▉| 1980/2000 [20:54<01:34,  4.74s/it, est. speed input: 248.23 toks/s, output: 3034.21 toks/s]

Processed prompts:  99%|█████████▉| 1984/2000 [21:32<01:38,  6.18s/it, est. speed input: 241.24 toks/s, output: 2967.21 toks/s]

Processed prompts:  99%|█████████▉| 1988/2000 [21:51<01:08,  5.73s/it, est. speed input: 238.23 toks/s, output: 2960.15 toks/s]

Processed prompts: 100%|█████████▉| 1992/2000 [22:23<00:51,  6.41s/it, est. speed input: 232.97 toks/s, output: 2949.91 toks/s]

Processed prompts: 100%|█████████▉| 1996/2000 [23:34<00:39,  9.82s/it, est. speed input: 221.61 toks/s, output: 2830.03 toks/s]

Processed prompts: 100%|██████████| 2000/2000 [25:56<00:00, 17.49s/it, est. speed input: 201.72 toks/s, output: 2603.08 toks/s]

Processed prompts: 100%|██████████| 2000/2000 [25:56<00:00, 17.49s/it, est. speed input: 201.72 toks/s, output: 2603.08 toks/s]
Processed prompts: 100%|██████████| 2000/2000 [25:56<00:00,  1.29it/s, est. speed input: 201.72 toks/s, output: 2603.08 toks/s]

Splits: 100%|██████████| 1/1 [25:56<00:00, 1556.49s/it]
Splits: 100%|██████████| 1/1 [25:56<00:00, 1556.49s/it]

Creating parquet from Arrow format:   0%|          | 0/1 [00:00<?, ?ba/s]
Creating parquet from Arrow format: 100%|██████████| 1/1 [00:00<00:00,  3.90ba/s]
Creating parquet from Arrow format: 100%|██████████| 1/1 [00:00<00:00,  3.89ba/s]
[2025-10-26 03:10:53,817] [    INFO]: Cached 500 samples of lighteval|math_500|0 (3aecc7facae3926c, GENERATIVE) at /mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562/081b5a149587018c/lighteval|math_500|0/3aecc7facae3926c/GENERATIVE.parquet. (cache_management.py:345)

Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 500 examples [00:00, 3409.17 examples/s]
Generating train split: 500 examples [00:00, 3367.58 examples/s]
[rank0]:[W1026 03:11:03.786488099 ProcessGroupNCCL.cpp:1479] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
[2025-10-26 03:11:04,702] [    INFO]: --- POST-PROCESSING MODEL RESPONSES --- (pipeline.py:344)
[2025-10-26 03:11:04,711] [    INFO]: --- COMPUTING METRICS --- (pipeline.py:371)
[2025-10-26 03:11:04,755] [ WARNING]: n undefined in the pass@k. We assume it's the same as the sample's number of predictions. (metrics_sample.py:1302)
[2025-10-26 03:11:09,287] [    INFO]: --- DISPLAYING RESULTS --- (pipeline.py:432)
[2025-10-26 03:11:09,300] [    INFO]: --- SAVING AND PUSHING RESULTS --- (pipeline.py:422)
[2025-10-26 03:11:09,301] [    INFO]: Saving experiment tracker (evaluation_tracker.py:246)
[2025-10-26 03:11:11,741] [    INFO]: Saving results to /mnt/public/wucanhui/lighteval/results/results/mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562/results_2025-10-26T03-11-09.302144.json (evaluation_tracker.py:310)
|        Task        |Version|   Metric    |Value |   |Stderr|
|--------------------|-------|-------------|-----:|---|-----:|
|all                 |       |avg@k_with_k |0.9425|±  |0.0076|
|                    |       |pass@k_with_k|0.9900|±  |0.0045|
|lighteval:math_500:0|       |avg@k_with_k |0.9425|±  |0.0076|
|                    |       |pass@k_with_k|0.9900|±  |0.0045|