INFO 09-26 12:57:52 [__init__.py:239] Automatically detected platform cuda.

=== Processing model: /home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_750M_chinese_wikipedia_corpus_2e_240925/checkpoint-2944, type: qwen ===
INFO 09-26 12:57:55 [config.py:2700] Downcasting torch.float32 to torch.float16.
INFO 09-26 12:58:10 [config.py:600] This model supports multiple tasks: {'classify', 'embed', 'reward', 'score', 'generate'}. Defaulting to 'generate'.
WARNING 09-26 12:58:10 [cuda.py:96] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
INFO 09-26 12:58:10 [llm_engine.py:242] Initializing a V0 LLM engine (v0.8.3) with config: model='/home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_750M_chinese_wikipedia_corpus_2e_240925/checkpoint-2944', speculative_config=None, tokenizer='/home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_750M_chinese_wikipedia_corpus_2e_240925/checkpoint-2944', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=32768, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar', reasoning_backend=None), observability_config=ObservabilityConfig(show_hidden_metrics=False, otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=None, served_model_name=/home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_750M_chinese_wikipedia_corpus_2e_240925/checkpoint-2944, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=None, chunked_prefill_enabled=False, use_async_output_proc=False, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"splitting_ops":[],"compile_sizes":[],"cudagraph_capture_sizes":[],"max_capture_size":0}, use_cached_outputs=False, 
INFO 09-26 12:58:12 [cuda.py:292] Using Flash Attention backend.
INFO 09-26 12:58:13 [parallel_state.py:957] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0
INFO 09-26 12:58:13 [model_runner.py:1110] Starting to load model /home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_750M_chinese_wikipedia_corpus_2e_240925/checkpoint-2944...

Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.06s/it]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.06s/it]

INFO 09-26 12:58:14 [loader.py:447] Loading weights took 1.13 seconds
INFO 09-26 12:58:15 [model_runner.py:1146] Model loading took 0.9277 GiB and 1.303196 seconds
INFO 09-26 12:58:16 [worker.py:267] Memory profiling takes 1.00 seconds
INFO 09-26 12:58:16 [worker.py:267] the current vLLM instance can use total_gpu_memory (79.25GiB) x gpu_memory_utilization (0.90) = 71.33GiB
INFO 09-26 12:58:16 [worker.py:267] model weights take 0.93GiB; non_torch_memory takes 0.09GiB; PyTorch activation peak memory takes 1.44GiB; the rest of the memory reserved for KV Cache is 68.87GiB.
INFO 09-26 12:58:17 [executor_base.py:112] # cuda blocks: 376119, # CPU blocks: 21845
INFO 09-26 12:58:17 [executor_base.py:117] Maximum concurrency for 32768 tokens per request: 183.65x
INFO 09-26 12:58:20 [llm_engine.py:448] init engine (profile, create kv cache, warmup model) took 4.71 seconds
Layers: 24, Intermediate size: 4864, Max length: 32768

Loading IDs from ./ids/q2.5/id.zh.train.qwen2.5-0.5 (lang: zh)...
ID shape: torch.Size([100000000])
Processing 3051 sequences of length 32768
Running inference...

Processed prompts:   0%|          | 0/3051 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts:   0%|          | 1/3051 [00:00<25:26,  2.00it/s, est. speed input: 65461.71 toks/s, output: 2.00 toks/s]
Processed prompts:   0%|          | 2/3051 [00:00<24:29,  2.08it/s, est. speed input: 67608.98 toks/s, output: 2.06 toks/s]
Processed prompts:   0%|          | 3/3051 [00:01<24:09,  2.10it/s, est. speed input: 68384.23 toks/s, output: 2.09 toks/s]
Processed prompts:   0%|          | 4/3051 [00:01<23:59,  2.12it/s, est. speed input: 68809.17 toks/s, output: 2.10 toks/s]
Processed prompts:   0%|          | 5/3051 [00:02<26:12,  1.94it/s, est. speed input: 65561.69 toks/s, output: 2.00 toks/s]
Processed prompts:   0%|          | 6/3051 [00:02<25:22,  2.00it/s, est. speed input: 66254.36 toks/s, output: 2.02 toks/s]
Processed prompts:   0%|          | 7/3051 [00:03<24:50,  2.04it/s, est. speed input: 66766.99 toks/s, output: 2.04 toks/s]
Processed prompts:   0%|          | 8/3051 [00:03<24:28,  2.07it/s, est. speed input: 67164.51 toks/s, output: 2.05 toks/s]
Processed prompts:   0%|          | 9/3051 [00:04<24:14,  2.09it/s, est. speed input: 67464.13 toks/s, output: 2.06 toks/s]
Processed prompts:   0%|          | 10/3051 [00:04<24:04,  2.11it/s, est. speed input: 67710.69 toks/s, output: 2.07 toks/s]
Processed prompts:   0%|          | 11/3051 [00:05<23:57,  2.12it/s, est. speed input: 67917.31 toks/s, output: 2.07 toks/s]
Processed prompts:   0%|          | 12/3051 [00:05<23:52,  2.12it/s, est. speed input: 68088.47 toks/s, output: 2.08 toks/s]
Processed prompts:   0%|          | 13/3051 [00:06<23:48,  2.13it/s, est. speed input: 68235.01 toks/s, output: 2.08 toks/s]
Processed prompts:   0%|          | 14/3051 [00:06<23:46,  2.13it/s, est. speed input: 68358.20 toks/s, output: 2.09 toks/s]
Processed prompts:   0%|          | 15/3051 [00:07<23:44,  2.13it/s, est. speed input: 68466.04 toks/s, output: 2.09 toks/s]
Processed prompts:   1%|          | 16/3051 [00:07<23:43,  2.13it/s, est. speed input: 68558.13 toks/s, output: 2.09 toks/s]
Processed prompts:   1%|          | 17/3051 [00:08<23:42,  2.13it/s, est. speed input: 68636.33 toks/s, output: 2.09 toks/s]
Processed prompts:   1%|          | 18/3051 [00:08<23:41,  2.13it/s, est. speed input: 68709.82 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 19/3051 [00:09<23:40,  2.13it/s, est. speed input: 68775.37 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 20/3051 [00:09<23:39,  2.13it/s, est. speed input: 68833.60 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 21/3051 [00:09<23:39,  2.14it/s, est. speed input: 68889.39 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 22/3051 [00:10<23:38,  2.13it/s, est. speed input: 68936.09 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 23/3051 [00:10<23:37,  2.14it/s, est. speed input: 68982.39 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 24/3051 [00:11<23:37,  2.14it/s, est. speed input: 69021.69 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 25/3051 [00:11<23:37,  2.13it/s, est. speed input: 69057.20 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 26/3051 [00:12<23:37,  2.13it/s, est. speed input: 69091.34 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 27/3051 [00:12<23:35,  2.14it/s, est. speed input: 69126.93 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 28/3051 [00:13<23:36,  2.13it/s, est. speed input: 69153.31 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 29/3051 [00:13<23:35,  2.13it/s, est. speed input: 69180.85 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 30/3051 [00:14<23:35,  2.13it/s, est. speed input: 69203.07 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 31/3051 [00:14<23:35,  2.13it/s, est. speed input: 69226.63 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 32/3051 [00:15<23:34,  2.13it/s, est. speed input: 69248.99 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 33/3051 [00:15<23:34,  2.13it/s, est. speed input: 69268.83 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 34/3051 [00:16<23:34,  2.13it/s, est. speed input: 69285.72 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 35/3051 [00:16<23:33,  2.13it/s, est. speed input: 69304.25 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 36/3051 [00:17<23:32,  2.13it/s, est. speed input: 69322.42 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 37/3051 [00:17<23:32,  2.13it/s, est. speed input: 69338.61 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 38/3051 [00:17<23:32,  2.13it/s, est. speed input: 69353.36 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 39/3051 [00:18<23:31,  2.13it/s, est. speed input: 69367.73 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 40/3051 [00:18<23:31,  2.13it/s, est. speed input: 69380.33 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 41/3051 [00:19<23:31,  2.13it/s, est. speed input: 69392.45 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 42/3051 [00:19<23:30,  2.13it/s, est. speed input: 69406.61 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 43/3051 [00:20<23:29,  2.13it/s, est. speed input: 69419.27 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 44/3051 [00:20<23:29,  2.13it/s, est. speed input: 69429.18 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 45/3051 [00:21<23:29,  2.13it/s, est. speed input: 69439.15 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 46/3051 [00:21<23:28,  2.13it/s, est. speed input: 69449.67 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 47/3051 [00:22<23:27,  2.13it/s, est. speed input: 69460.38 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 48/3051 [00:22<23:27,  2.13it/s, est. speed input: 69470.52 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 49/3051 [00:23<23:26,  2.13it/s, est. speed input: 69478.71 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 50/3051 [00:23<23:27,  2.13it/s, est. speed input: 69485.09 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 51/3051 [00:24<23:26,  2.13it/s, est. speed input: 69492.24 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 52/3051 [00:24<23:26,  2.13it/s, est. speed input: 69500.85 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 53/3051 [00:24<23:25,  2.13it/s, est. speed input: 69507.70 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 54/3051 [00:25<23:25,  2.13it/s, est. speed input: 69515.15 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 55/3051 [00:25<23:24,  2.13it/s, est. speed input: 69522.71 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 56/3051 [00:26<23:24,  2.13it/s, est. speed input: 69528.64 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 57/3051 [00:26<23:23,  2.13it/s, est. speed input: 69534.57 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 58/3051 [00:27<23:23,  2.13it/s, est. speed input: 69539.99 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 59/3051 [00:27<23:23,  2.13it/s, est. speed input: 69545.31 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 60/3051 [00:28<23:23,  2.13it/s, est. speed input: 69549.74 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 61/3051 [00:28<23:22,  2.13it/s, est. speed input: 69555.33 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 62/3051 [00:29<23:21,  2.13it/s, est. speed input: 69561.20 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 63/3051 [00:29<23:21,  2.13it/s, est. speed input: 69565.50 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 64/3051 [00:30<23:21,  2.13it/s, est. speed input: 69568.91 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 65/3051 [00:30<23:20,  2.13it/s, est. speed input: 69573.37 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 66/3051 [00:31<23:20,  2.13it/s, est. speed input: 69577.30 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 67/3051 [00:31<23:19,  2.13it/s, est. speed input: 69581.73 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 68/3051 [00:32<23:19,  2.13it/s, est. speed input: 69585.47 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 69/3051 [00:32<23:19,  2.13it/s, est. speed input: 69589.00 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 70/3051 [00:32<23:18,  2.13it/s, est. speed input: 69593.72 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 71/3051 [00:33<23:17,  2.13it/s, est. speed input: 69598.46 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 72/3051 [00:33<23:17,  2.13it/s, est. speed input: 69601.50 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 73/3051 [00:34<23:16,  2.13it/s, est. speed input: 69604.74 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 74/3051 [00:34<23:16,  2.13it/s, est. speed input: 69608.33 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 75/3051 [00:35<23:16,  2.13it/s, est. speed input: 69610.26 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 76/3051 [00:35<23:15,  2.13it/s, est. speed input: 69613.30 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 77/3051 [00:36<23:15,  2.13it/s, est. speed input: 69615.81 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 78/3051 [00:36<23:14,  2.13it/s, est. speed input: 69619.11 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 79/3051 [00:37<23:14,  2.13it/s, est. speed input: 69621.71 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 80/3051 [00:37<23:14,  2.13it/s, est. speed input: 69623.54 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 81/3051 [00:38<23:13,  2.13it/s, est. speed input: 69627.21 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 82/3051 [00:38<23:12,  2.13it/s, est. speed input: 69629.69 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 83/3051 [00:39<23:12,  2.13it/s, est. speed input: 69631.48 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 84/3051 [00:39<23:12,  2.13it/s, est. speed input: 69634.13 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 85/3051 [00:39<23:11,  2.13it/s, est. speed input: 69636.39 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 86/3051 [00:40<23:11,  2.13it/s, est. speed input: 69638.72 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 87/3051 [00:40<23:10,  2.13it/s, est. speed input: 69641.06 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 88/3051 [00:41<23:10,  2.13it/s, est. speed input: 69642.85 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 89/3051 [00:41<23:10,  2.13it/s, est. speed input: 69644.43 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 90/3051 [00:42<23:09,  2.13it/s, est. speed input: 69647.16 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 91/3051 [00:42<23:09,  2.13it/s, est. speed input: 69648.21 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 92/3051 [00:43<23:09,  2.13it/s, est. speed input: 69649.40 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 93/3051 [00:43<23:08,  2.13it/s, est. speed input: 69650.96 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 94/3051 [00:44<23:08,  2.13it/s, est. speed input: 69652.91 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 95/3051 [00:44<23:07,  2.13it/s, est. speed input: 69654.34 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 96/3051 [00:45<23:07,  2.13it/s, est. speed input: 69655.38 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 97/3051 [00:45<23:07,  2.13it/s, est. speed input: 69656.46 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 98/3051 [00:46<23:06,  2.13it/s, est. speed input: 69658.27 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 99/3051 [00:46<23:06,  2.13it/s, est. speed input: 69659.10 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 100/3051 [00:47<23:05,  2.13it/s, est. speed input: 69660.84 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 101/3051 [00:47<23:05,  2.13it/s, est. speed input: 69661.27 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 102/3051 [00:47<23:05,  2.13it/s, est. speed input: 69662.36 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 103/3051 [00:48<23:04,  2.13it/s, est. speed input: 69663.94 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 104/3051 [00:48<23:03,  2.13it/s, est. speed input: 69665.15 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 105/3051 [00:49<23:03,  2.13it/s, est. speed input: 69666.37 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 106/3051 [00:49<23:02,  2.13it/s, est. speed input: 69667.83 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 107/3051 [00:50<23:02,  2.13it/s, est. speed input: 69668.97 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 108/3051 [00:50<23:02,  2.13it/s, est. speed input: 69669.21 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 109/3051 [00:51<23:01,  2.13it/s, est. speed input: 69669.86 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 110/3051 [00:51<23:01,  2.13it/s, est. speed input: 69670.25 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 111/3051 [00:52<23:01,  2.13it/s, est. speed input: 69670.96 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 112/3051 [00:52<23:00,  2.13it/s, est. speed input: 69671.64 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 113/3051 [00:53<23:00,  2.13it/s, est. speed input: 69672.10 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 114/3051 [00:53<22:59,  2.13it/s, est. speed input: 69673.17 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 115/3051 [00:54<22:59,  2.13it/s, est. speed input: 69673.37 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 116/3051 [00:54<22:59,  2.13it/s, est. speed input: 69673.73 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 117/3051 [00:55<22:58,  2.13it/s, est. speed input: 69674.85 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 118/3051 [00:55<22:58,  2.13it/s, est. speed input: 69675.07 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 119/3051 [00:55<22:57,  2.13it/s, est. speed input: 69675.67 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 120/3051 [00:56<22:57,  2.13it/s, est. speed input: 69675.61 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 121/3051 [00:56<22:56,  2.13it/s, est. speed input: 69676.59 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 122/3051 [00:57<22:56,  2.13it/s, est. speed input: 69677.37 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 123/3051 [00:57<22:55,  2.13it/s, est. speed input: 69677.85 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 124/3051 [00:58<22:54,  2.13it/s, est. speed input: 69678.68 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 125/3051 [00:58<22:54,  2.13it/s, est. speed input: 69679.43 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 126/3051 [00:59<22:53,  2.13it/s, est. speed input: 69680.21 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 127/3051 [00:59<22:53,  2.13it/s, est. speed input: 69680.65 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 128/3051 [01:00<22:53,  2.13it/s, est. speed input: 69680.96 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 129/3051 [01:00<22:52,  2.13it/s, est. speed input: 69681.28 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 130/3051 [01:01<22:53,  2.13it/s, est. speed input: 69680.95 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 131/3051 [01:01<22:52,  2.13it/s, est. speed input: 69681.55 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 132/3051 [01:02<22:51,  2.13it/s, est. speed input: 69681.81 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 133/3051 [01:02<22:51,  2.13it/s, est. speed input: 69682.05 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 134/3051 [01:03<22:50,  2.13it/s, est. speed input: 69682.74 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 135/3051 [01:03<22:50,  2.13it/s, est. speed input: 69682.94 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 136/3051 [01:03<22:50,  2.13it/s, est. speed input: 69683.05 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 137/3051 [01:04<22:49,  2.13it/s, est. speed input: 69683.50 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 138/3051 [01:04<22:48,  2.13it/s, est. speed input: 69683.98 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 139/3051 [01:05<22:48,  2.13it/s, est. speed input: 69684.34 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 140/3051 [01:05<22:48,  2.13it/s, est. speed input: 69684.55 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 141/3051 [01:06<22:47,  2.13it/s, est. speed input: 69685.01 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 142/3051 [01:06<22:47,  2.13it/s, est. speed input: 69685.17 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 143/3051 [01:07<22:46,  2.13it/s, est. speed input: 69685.17 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 144/3051 [01:07<22:46,  2.13it/s, est. speed input: 69685.29 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 145/3051 [01:08<22:45,  2.13it/s, est. speed input: 69685.62 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 146/3051 [01:08<22:45,  2.13it/s, est. speed input: 69685.70 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 147/3051 [01:09<22:45,  2.13it/s, est. speed input: 69685.64 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 148/3051 [01:09<22:44,  2.13it/s, est. speed input: 69685.62 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 149/3051 [01:10<22:43,  2.13it/s, est. speed input: 69686.31 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 150/3051 [01:10<22:43,  2.13it/s, est. speed input: 69686.65 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 151/3051 [01:11<22:42,  2.13it/s, est. speed input: 69686.94 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 152/3051 [01:11<22:42,  2.13it/s, est. speed input: 69686.94 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 153/3051 [01:11<22:42,  2.13it/s, est. speed input: 69687.06 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 154/3051 [01:12<22:41,  2.13it/s, est. speed input: 69686.98 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 155/3051 [01:12<22:41,  2.13it/s, est. speed input: 69687.00 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 156/3051 [01:13<22:41,  2.13it/s, est. speed input: 69686.89 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 157/3051 [01:13<22:40,  2.13it/s, est. speed input: 69687.28 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 158/3051 [01:14<22:39,  2.13it/s, est. speed input: 69687.61 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 159/3051 [01:14<22:39,  2.13it/s, est. speed input: 69687.11 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 160/3051 [01:15<22:39,  2.13it/s, est. speed input: 69687.08 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 161/3051 [01:15<22:39,  2.13it/s, est. speed input: 69686.99 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 162/3051 [01:16<22:38,  2.13it/s, est. speed input: 69687.03 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 163/3051 [01:16<22:37,  2.13it/s, est. speed input: 69687.64 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 164/3051 [01:17<22:37,  2.13it/s, est. speed input: 69686.91 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 165/3051 [01:17<22:37,  2.13it/s, est. speed input: 69687.12 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 166/3051 [01:18<22:36,  2.13it/s, est. speed input: 69686.78 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 167/3051 [01:18<22:36,  2.13it/s, est. speed input: 69686.20 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 168/3051 [01:18<22:37,  2.12it/s, est. speed input: 69684.99 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 169/3051 [01:19<22:37,  2.12it/s, est. speed input: 69684.26 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 170/3051 [01:19<22:36,  2.12it/s, est. speed input: 69683.46 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 171/3051 [01:20<22:37,  2.12it/s, est. speed input: 69682.02 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 172/3051 [01:20<22:37,  2.12it/s, est. speed input: 69680.48 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 173/3051 [01:21<22:36,  2.12it/s, est. speed input: 69679.38 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 174/3051 [01:21<22:36,  2.12it/s, est. speed input: 69678.21 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 175/3051 [01:22<22:36,  2.12it/s, est. speed input: 69676.52 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 176/3051 [01:22<22:36,  2.12it/s, est. speed input: 69675.58 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 177/3051 [01:23<22:34,  2.12it/s, est. speed input: 69675.09 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 178/3051 [01:23<22:34,  2.12it/s, est. speed input: 69674.25 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 179/3051 [01:24<22:34,  2.12it/s, est. speed input: 69672.73 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 180/3051 [01:24<22:34,  2.12it/s, est. speed input: 69671.60 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 181/3051 [01:25<22:33,  2.12it/s, est. speed input: 69670.66 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 182/3051 [01:25<22:32,  2.12it/s, est. speed input: 69670.07 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 183/3051 [01:26<22:32,  2.12it/s, est. speed input: 69669.13 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 184/3051 [01:26<22:31,  2.12it/s, est. speed input: 69668.06 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 185/3051 [01:27<22:31,  2.12it/s, est. speed input: 69666.92 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 186/3051 [01:27<22:31,  2.12it/s, est. speed input: 69665.89 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 187/3051 [01:27<22:31,  2.12it/s, est. speed input: 69664.50 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 188/3051 [01:28<22:30,  2.12it/s, est. speed input: 69663.25 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 189/3051 [01:28<22:30,  2.12it/s, est. speed input: 69662.09 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 190/3051 [01:29<22:29,  2.12it/s, est. speed input: 69661.30 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 191/3051 [01:29<22:29,  2.12it/s, est. speed input: 69660.32 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 192/3051 [01:30<22:28,  2.12it/s, est. speed input: 69659.60 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 193/3051 [01:30<22:28,  2.12it/s, est. speed input: 69658.35 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 194/3051 [01:31<22:29,  2.12it/s, est. speed input: 69656.15 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 195/3051 [01:31<22:28,  2.12it/s, est. speed input: 69655.26 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 196/3051 [01:32<22:27,  2.12it/s, est. speed input: 69654.40 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 197/3051 [01:32<22:26,  2.12it/s, est. speed input: 69653.44 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 198/3051 [01:33<22:26,  2.12it/s, est. speed input: 69652.22 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 199/3051 [01:33<22:25,  2.12it/s, est. speed input: 69651.30 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 200/3051 [01:34<22:25,  2.12it/s, est. speed input: 69650.42 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 201/3051 [01:34<22:25,  2.12it/s, est. speed input: 69648.81 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 202/3051 [01:35<22:25,  2.12it/s, est. speed input: 69647.48 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 203/3051 [01:35<22:25,  2.12it/s, est. speed input: 69645.69 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 204/3051 [01:35<22:23,  2.12it/s, est. speed input: 69645.15 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 205/3051 [01:36<22:23,  2.12it/s, est. speed input: 69644.25 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 206/3051 [01:36<22:22,  2.12it/s, est. speed input: 69643.25 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 207/3051 [01:37<22:22,  2.12it/s, est. speed input: 69642.24 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 208/3051 [01:37<22:21,  2.12it/s, est. speed input: 69641.09 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 209/3051 [01:38<22:21,  2.12it/s, est. speed input: 69640.25 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 210/3051 [01:38<22:21,  2.12it/s, est. speed input: 69638.81 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 211/3051 [01:39<22:21,  2.12it/s, est. speed input: 69637.49 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 212/3051 [01:39<22:19,  2.12it/s, est. speed input: 69637.10 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 213/3051 [01:40<22:19,  2.12it/s, est. speed input: 69636.11 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 214/3051 [01:40<22:19,  2.12it/s, est. speed input: 69634.98 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 215/3051 [01:41<22:18,  2.12it/s, est. speed input: 69633.80 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 216/3051 [01:41<22:17,  2.12it/s, est. speed input: 69633.09 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 217/3051 [01:42<22:17,  2.12it/s, est. speed input: 69632.04 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 218/3051 [01:42<22:17,  2.12it/s, est. speed input: 69630.51 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 219/3051 [01:43<22:17,  2.12it/s, est. speed input: 69629.58 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 220/3051 [01:43<22:15,  2.12it/s, est. speed input: 69629.32 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 221/3051 [01:44<22:15,  2.12it/s, est. speed input: 69628.07 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 222/3051 [01:44<22:15,  2.12it/s, est. speed input: 69627.05 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 223/3051 [01:44<22:15,  2.12it/s, est. speed input: 69625.76 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 224/3051 [01:45<22:14,  2.12it/s, est. speed input: 69624.99 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 225/3051 [01:45<22:14,  2.12it/s, est. speed input: 69623.88 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 226/3051 [01:46<22:14,  2.12it/s, est. speed input: 69622.69 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 227/3051 [01:46<22:13,  2.12it/s, est. speed input: 69621.90 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 228/3051 [01:47<22:12,  2.12it/s, est. speed input: 69621.36 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 229/3051 [01:47<22:11,  2.12it/s, est. speed input: 69620.59 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 230/3051 [01:48<22:11,  2.12it/s, est. speed input: 69619.65 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 231/3051 [01:48<22:11,  2.12it/s, est. speed input: 69618.59 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 232/3051 [01:49<22:11,  2.12it/s, est. speed input: 69617.50 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 233/3051 [01:49<22:10,  2.12it/s, est. speed input: 69616.45 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 234/3051 [01:50<22:10,  2.12it/s, est. speed input: 69615.51 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 235/3051 [01:50<22:09,  2.12it/s, est. speed input: 69614.71 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 236/3051 [01:51<22:08,  2.12it/s, est. speed input: 69614.20 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 237/3051 [01:51<22:08,  2.12it/s, est. speed input: 69613.12 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 238/3051 [01:52<22:08,  2.12it/s, est. speed input: 69612.10 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 239/3051 [01:52<22:07,  2.12it/s, est. speed input: 69611.23 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 240/3051 [01:52<22:07,  2.12it/s, est. speed input: 69610.45 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 241/3051 [01:53<22:06,  2.12it/s, est. speed input: 69609.53 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 242/3051 [01:53<22:06,  2.12it/s, est. speed input: 69608.25 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 243/3051 [01:54<22:06,  2.12it/s, est. speed input: 69607.23 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 244/3051 [01:54<22:06,  2.12it/s, est. speed input: 69606.00 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 245/3051 [01:55<22:05,  2.12it/s, est. speed input: 69605.11 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 246/3051 [01:55<22:04,  2.12it/s, est. speed input: 69604.57 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 247/3051 [01:56<22:03,  2.12it/s, est. speed input: 69603.80 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 248/3051 [01:56<22:03,  2.12it/s, est. speed input: 69602.78 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 249/3051 [01:57<22:03,  2.12it/s, est. speed input: 69601.75 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 250/3051 [01:57<22:02,  2.12it/s, est. speed input: 69601.24 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 251/3051 [01:58<22:01,  2.12it/s, est. speed input: 69600.84 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 252/3051 [01:58<22:00,  2.12it/s, est. speed input: 69600.27 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 253/3051 [01:59<22:01,  2.12it/s, est. speed input: 69598.78 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 254/3051 [01:59<22:01,  2.12it/s, est. speed input: 69597.69 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 255/3051 [02:00<22:00,  2.12it/s, est. speed input: 69596.94 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 256/3051 [02:00<22:00,  2.12it/s, est. speed input: 69595.89 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 257/3051 [02:01<21:59,  2.12it/s, est. speed input: 69595.13 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 258/3051 [02:01<21:58,  2.12it/s, est. speed input: 69594.46 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 259/3051 [02:01<21:57,  2.12it/s, est. speed input: 69594.11 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 260/3051 [02:02<21:57,  2.12it/s, est. speed input: 69593.55 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 261/3051 [02:02<21:57,  2.12it/s, est. speed input: 69592.53 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 262/3051 [02:03<21:56,  2.12it/s, est. speed input: 69591.80 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 263/3051 [02:03<21:56,  2.12it/s, est. speed input: 69591.06 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 264/3051 [02:04<21:56,  2.12it/s, est. speed input: 69590.23 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 265/3051 [02:04<21:55,  2.12it/s, est. speed input: 69589.29 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 266/3051 [02:05<21:55,  2.12it/s, est. speed input: 69588.27 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 267/3051 [02:05<21:55,  2.12it/s, est. speed input: 69587.42 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 268/3051 [02:06<21:54,  2.12it/s, est. speed input: 69586.64 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 269/3051 [02:06<21:54,  2.12it/s, est. speed input: 69585.89 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 270/3051 [02:07<21:53,  2.12it/s, est. speed input: 69585.04 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 271/3051 [02:07<21:52,  2.12it/s, est. speed input: 69584.69 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 272/3051 [02:08<21:52,  2.12it/s, est. speed input: 69583.62 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 273/3051 [02:08<21:52,  2.12it/s, est. speed input: 69582.45 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 274/3051 [02:09<21:51,  2.12it/s, est. speed input: 69581.99 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 275/3051 [02:09<21:51,  2.12it/s, est. speed input: 69581.31 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 276/3051 [02:09<21:50,  2.12it/s, est. speed input: 69580.59 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 277/3051 [02:10<21:50,  2.12it/s, est. speed input: 69579.76 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 278/3051 [02:10<21:50,  2.12it/s, est. speed input: 69578.74 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 279/3051 [02:11<21:49,  2.12it/s, est. speed input: 69578.03 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 280/3051 [02:11<21:49,  2.12it/s, est. speed input: 69577.22 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 281/3051 [02:12<21:48,  2.12it/s, est. speed input: 69576.61 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 282/3051 [02:12<21:48,  2.12it/s, est. speed input: 69575.71 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 283/3051 [02:13<21:47,  2.12it/s, est. speed input: 69575.15 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 284/3051 [02:13<21:46,  2.12it/s, est. speed input: 69574.54 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 285/3051 [02:14<21:46,  2.12it/s, est. speed input: 69573.78 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 286/3051 [02:14<21:46,  2.12it/s, est. speed input: 69573.02 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 287/3051 [02:15<21:45,  2.12it/s, est. speed input: 69572.30 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 288/3051 [02:15<21:45,  2.12it/s, est. speed input: 69571.58 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 289/3051 [02:16<21:44,  2.12it/s, est. speed input: 69570.85 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 290/3051 [02:16<21:44,  2.12it/s, est. speed input: 69570.23 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 291/3051 [02:17<21:43,  2.12it/s, est. speed input: 69569.59 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 292/3051 [02:17<21:43,  2.12it/s, est. speed input: 69568.76 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 293/3051 [02:18<21:43,  2.12it/s, est. speed input: 69567.92 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 294/3051 [02:18<21:42,  2.12it/s, est. speed input: 69567.17 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 295/3051 [02:18<21:42,  2.12it/s, est. speed input: 69566.28 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 296/3051 [02:19<21:42,  2.11it/s, est. speed input: 69565.02 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 297/3051 [02:19<21:41,  2.12it/s, est. speed input: 69564.46 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 298/3051 [02:20<21:40,  2.12it/s, est. speed input: 69563.88 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 299/3051 [02:20<21:39,  2.12it/s, est. speed input: 69563.43 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 300/3051 [02:21<21:39,  2.12it/s, est. speed input: 69562.96 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 301/3051 [02:21<21:38,  2.12it/s, est. speed input: 69562.55 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 302/3051 [02:22<21:38,  2.12it/s, est. speed input: 69561.96 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 303/3051 [02:22<21:38,  2.12it/s, est. speed input: 69560.90 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 304/3051 [02:23<21:37,  2.12it/s, est. speed input: 69560.31 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 305/3051 [02:23<21:37,  2.12it/s, est. speed input: 69559.47 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 306/3051 [02:24<21:37,  2.12it/s, est. speed input: 69558.78 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 307/3051 [02:24<21:36,  2.12it/s, est. speed input: 69558.32 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 308/3051 [02:25<21:35,  2.12it/s, est. speed input: 69557.89 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 309/3051 [02:25<21:35,  2.12it/s, est. speed input: 69556.88 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 310/3051 [02:26<21:35,  2.12it/s, est. speed input: 69556.10 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 311/3051 [02:26<21:34,  2.12it/s, est. speed input: 69555.74 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 312/3051 [02:26<21:34,  2.12it/s, est. speed input: 69555.01 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 313/3051 [02:27<21:34,  2.12it/s, est. speed input: 69553.98 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 314/3051 [02:27<21:33,  2.12it/s, est. speed input: 69553.16 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 315/3051 [02:28<21:33,  2.12it/s, est. speed input: 69552.38 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 316/3051 [02:28<21:32,  2.12it/s, est. speed input: 69551.69 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 317/3051 [02:29<21:32,  2.12it/s, est. speed input: 69551.07 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 318/3051 [02:29<21:31,  2.12it/s, est. speed input: 69550.77 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 319/3051 [02:30<21:32,  2.11it/s, est. speed input: 69549.07 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 320/3051 [02:30<21:31,  2.11it/s, est. speed input: 69548.49 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 321/3051 [02:31<21:30,  2.11it/s, est. speed input: 69547.78 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 322/3051 [02:31<21:29,  2.12it/s, est. speed input: 69547.50 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 323/3051 [02:32<21:29,  2.12it/s, est. speed input: 69546.76 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 324/3051 [02:32<21:29,  2.12it/s, est. speed input: 69545.97 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 325/3051 [02:33<21:29,  2.11it/s, est. speed input: 69544.84 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 326/3051 [02:33<21:28,  2.11it/s, est. speed input: 69544.24 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 327/3051 [02:34<21:27,  2.12it/s, est. speed input: 69543.76 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 328/3051 [02:34<21:26,  2.12it/s, est. speed input: 69543.23 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 329/3051 [02:35<21:26,  2.12it/s, est. speed input: 69542.52 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 330/3051 [02:35<21:25,  2.12it/s, est. speed input: 69542.09 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 331/3051 [02:35<21:25,  2.12it/s, est. speed input: 69541.47 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 332/3051 [02:36<21:24,  2.12it/s, est. speed input: 69541.09 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 333/3051 [02:36<21:23,  2.12it/s, est. speed input: 69540.71 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 334/3051 [02:37<21:22,  2.12it/s, est. speed input: 69540.45 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 335/3051 [02:37<21:25,  2.11it/s, est. speed input: 69538.14 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 336/3051 [02:38<21:26,  2.11it/s, est. speed input: 69536.37 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 337/3051 [02:38<21:25,  2.11it/s, est. speed input: 69535.87 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 338/3051 [02:39<21:23,  2.11it/s, est. speed input: 69535.43 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 339/3051 [02:39<21:23,  2.11it/s, est. speed input: 69534.47 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 340/3051 [02:40<21:22,  2.11it/s, est. speed input: 69533.89 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 341/3051 [02:40<21:21,  2.11it/s, est. speed input: 69533.39 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 342/3051 [02:41<21:20,  2.12it/s, est. speed input: 69533.08 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 343/3051 [02:41<21:19,  2.12it/s, est. speed input: 69532.61 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 344/3051 [02:42<21:18,  2.12it/s, est. speed input: 69532.40 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 345/3051 [02:42<21:18,  2.12it/s, est. speed input: 69531.89 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 346/3051 [02:43<21:17,  2.12it/s, est. speed input: 69531.28 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 347/3051 [02:43<21:17,  2.12it/s, est. speed input: 69530.56 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 348/3051 [02:44<21:18,  2.11it/s, est. speed input: 69529.53 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 349/3051 [02:44<21:17,  2.12it/s, est. speed input: 69529.11 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 350/3051 [02:44<21:16,  2.12it/s, est. speed input: 69528.63 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 351/3051 [02:45<21:16,  2.12it/s, est. speed input: 69527.92 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 352/3051 [02:45<21:15,  2.12it/s, est. speed input: 69527.46 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 353/3051 [02:46<21:15,  2.12it/s, est. speed input: 69526.98 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 354/3051 [02:46<21:14,  2.12it/s, est. speed input: 69526.37 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 355/3051 [02:47<21:14,  2.12it/s, est. speed input: 69525.86 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 356/3051 [02:47<21:13,  2.12it/s, est. speed input: 69525.50 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 357/3051 [02:48<21:12,  2.12it/s, est. speed input: 69524.98 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 358/3051 [02:48<21:12,  2.12it/s, est. speed input: 69524.69 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 359/3051 [02:49<21:11,  2.12it/s, est. speed input: 69524.14 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 360/3051 [02:49<21:11,  2.12it/s, est. speed input: 69523.76 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 361/3051 [02:50<21:10,  2.12it/s, est. speed input: 69523.18 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 362/3051 [02:50<21:10,  2.12it/s, est. speed input: 69522.75 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 363/3051 [02:51<21:10,  2.12it/s, est. speed input: 69522.22 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 364/3051 [02:51<21:09,  2.12it/s, est. speed input: 69521.89 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 365/3051 [02:52<21:08,  2.12it/s, est. speed input: 69521.43 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 366/3051 [02:52<21:09,  2.12it/s, est. speed input: 69520.63 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 367/3051 [02:52<21:08,  2.12it/s, est. speed input: 69520.06 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 368/3051 [02:53<21:08,  2.11it/s, est. speed input: 69519.29 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 369/3051 [02:53<21:08,  2.12it/s, est. speed input: 69518.81 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 370/3051 [02:54<21:07,  2.11it/s, est. speed input: 69518.19 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 371/3051 [02:54<21:07,  2.12it/s, est. speed input: 69517.68 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 372/3051 [02:55<21:06,  2.11it/s, est. speed input: 69516.97 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 373/3051 [02:55<21:05,  2.12it/s, est. speed input: 69516.65 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 374/3051 [02:56<21:05,  2.12it/s, est. speed input: 69516.24 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 375/3051 [02:56<21:04,  2.12it/s, est. speed input: 69515.83 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 376/3051 [02:57<21:03,  2.12it/s, est. speed input: 69515.40 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 377/3051 [02:57<21:03,  2.12it/s, est. speed input: 69514.88 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 378/3051 [02:58<21:03,  2.12it/s, est. speed input: 69514.46 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 379/3051 [02:58<21:02,  2.12it/s, est. speed input: 69513.98 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 380/3051 [02:59<21:01,  2.12it/s, est. speed input: 69513.79 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 381/3051 [02:59<21:01,  2.12it/s, est. speed input: 69513.03 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 382/3051 [03:00<21:01,  2.12it/s, est. speed input: 69512.41 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 383/3051 [03:00<21:01,  2.12it/s, est. speed input: 69511.86 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 384/3051 [03:01<21:04,  2.11it/s, est. speed input: 69509.63 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 385/3051 [03:01<21:02,  2.11it/s, est. speed input: 69509.29 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 386/3051 [03:01<21:01,  2.11it/s, est. speed input: 69508.93 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 387/3051 [03:02<21:00,  2.11it/s, est. speed input: 69508.22 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 388/3051 [03:02<20:59,  2.11it/s, est. speed input: 69507.70 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 389/3051 [03:03<20:59,  2.11it/s, est. speed input: 69507.06 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 390/3051 [03:03<20:58,  2.11it/s, est. speed input: 69506.68 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 391/3051 [03:04<20:57,  2.12it/s, est. speed input: 69506.32 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 392/3051 [03:04<20:56,  2.12it/s, est. speed input: 69506.05 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 393/3051 [03:05<20:56,  2.12it/s, est. speed input: 69505.53 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 394/3051 [03:05<20:55,  2.12it/s, est. speed input: 69505.01 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 395/3051 [03:06<20:54,  2.12it/s, est. speed input: 69504.80 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 396/3051 [03:06<20:54,  2.12it/s, est. speed input: 69504.17 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 397/3051 [03:07<20:57,  2.11it/s, est. speed input: 69502.09 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 398/3051 [03:07<20:57,  2.11it/s, est. speed input: 69501.34 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 399/3051 [03:08<20:55,  2.11it/s, est. speed input: 69500.98 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 400/3051 [03:08<20:54,  2.11it/s, est. speed input: 69500.69 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 401/3051 [03:09<20:53,  2.11it/s, est. speed input: 69500.26 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 402/3051 [03:09<20:52,  2.12it/s, est. speed input: 69500.02 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 403/3051 [03:10<20:51,  2.12it/s, est. speed input: 69499.52 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 404/3051 [03:10<20:51,  2.12it/s, est. speed input: 69499.16 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 405/3051 [03:10<20:50,  2.12it/s, est. speed input: 69498.92 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 406/3051 [03:11<20:49,  2.12it/s, est. speed input: 69498.47 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 407/3051 [03:11<20:49,  2.12it/s, est. speed input: 69498.28 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 408/3051 [03:12<20:49,  2.12it/s, est. speed input: 69497.48 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 409/3051 [03:12<20:49,  2.11it/s, est. speed input: 69496.88 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 410/3051 [03:13<20:48,  2.11it/s, est. speed input: 69496.37 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 411/3051 [03:13<20:48,  2.11it/s, est. speed input: 69495.84 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 412/3051 [03:14<20:47,  2.11it/s, est. speed input: 69495.47 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 413/3051 [03:14<20:47,  2.11it/s, est. speed input: 69494.88 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 414/3051 [03:15<20:47,  2.11it/s, est. speed input: 69494.31 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 415/3051 [03:15<20:46,  2.11it/s, est. speed input: 69493.97 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 416/3051 [03:16<20:45,  2.12it/s, est. speed input: 69493.59 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 417/3051 [03:16<20:45,  2.12it/s, est. speed input: 69493.19 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 418/3051 [03:17<20:45,  2.11it/s, est. speed input: 69492.59 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 419/3051 [03:17<20:44,  2.12it/s, est. speed input: 69492.28 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 420/3051 [03:18<20:43,  2.12it/s, est. speed input: 69491.91 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 421/3051 [03:18<20:42,  2.12it/s, est. speed input: 69491.77 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 422/3051 [03:18<20:42,  2.12it/s, est. speed input: 69491.16 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 423/3051 [03:19<20:41,  2.12it/s, est. speed input: 69490.89 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 424/3051 [03:19<20:41,  2.12it/s, est. speed input: 69490.33 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 425/3051 [03:20<20:41,  2.12it/s, est. speed input: 69490.02 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 426/3051 [03:20<20:41,  2.11it/s, est. speed input: 69489.34 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 427/3051 [03:21<20:41,  2.11it/s, est. speed input: 69488.49 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 428/3051 [03:21<20:40,  2.11it/s, est. speed input: 69488.19 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 429/3051 [03:22<20:41,  2.11it/s, est. speed input: 69487.04 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 430/3051 [03:22<20:40,  2.11it/s, est. speed input: 69486.48 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 431/3051 [03:23<20:39,  2.11it/s, est. speed input: 69486.15 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 432/3051 [03:23<20:38,  2.11it/s, est. speed input: 69485.89 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 433/3051 [03:24<20:37,  2.11it/s, est. speed input: 69485.56 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 434/3051 [03:24<20:36,  2.12it/s, est. speed input: 69485.37 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 435/3051 [03:25<20:36,  2.12it/s, est. speed input: 69485.09 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 436/3051 [03:25<20:36,  2.11it/s, est. speed input: 69484.43 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 437/3051 [03:26<20:38,  2.11it/s, est. speed input: 69483.12 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 438/3051 [03:26<20:38,  2.11it/s, est. speed input: 69482.04 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 439/3051 [03:27<20:38,  2.11it/s, est. speed input: 69480.87 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 440/3051 [03:27<20:37,  2.11it/s, est. speed input: 69480.42 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 441/3051 [03:27<20:36,  2.11it/s, est. speed input: 69479.93 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 442/3051 [03:28<20:34,  2.11it/s, est. speed input: 69479.64 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 443/3051 [03:28<20:33,  2.11it/s, est. speed input: 69479.36 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 444/3051 [03:29<20:33,  2.11it/s, est. speed input: 69478.90 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 445/3051 [03:29<20:32,  2.11it/s, est. speed input: 69478.54 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 446/3051 [03:30<20:31,  2.12it/s, est. speed input: 69478.26 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 447/3051 [03:30<20:31,  2.12it/s, est. speed input: 69477.91 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 448/3051 [03:31<20:45,  2.09it/s, est. speed input: 69471.24 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 449/3051 [03:31<20:40,  2.10it/s, est. speed input: 69470.92 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 450/3051 [03:32<20:36,  2.10it/s, est. speed input: 69470.64 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 451/3051 [03:32<20:33,  2.11it/s, est. speed input: 69470.38 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 452/3051 [03:33<20:32,  2.11it/s, est. speed input: 69469.84 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 453/3051 [03:33<20:30,  2.11it/s, est. speed input: 69469.65 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 454/3051 [03:34<20:29,  2.11it/s, est. speed input: 69469.12 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 455/3051 [03:34<20:30,  2.11it/s, est. speed input: 69467.97 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 456/3051 [03:35<20:29,  2.11it/s, est. speed input: 69467.66 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 457/3051 [03:35<20:27,  2.11it/s, est. speed input: 69467.39 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 458/3051 [03:36<20:27,  2.11it/s, est. speed input: 69466.65 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 459/3051 [03:36<20:27,  2.11it/s, est. speed input: 69466.20 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 460/3051 [03:36<20:26,  2.11it/s, est. speed input: 69465.87 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 461/3051 [03:37<20:25,  2.11it/s, est. speed input: 69465.57 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 462/3051 [03:37<20:24,  2.12it/s, est. speed input: 69465.41 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 463/3051 [03:38<20:23,  2.12it/s, est. speed input: 69465.15 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 464/3051 [03:38<20:23,  2.12it/s, est. speed input: 69464.77 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 465/3051 [03:39<20:22,  2.11it/s, est. speed input: 69464.35 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 466/3051 [03:39<20:22,  2.12it/s, est. speed input: 69464.11 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 467/3051 [03:40<20:43,  2.08it/s, est. speed input: 69454.73 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 468/3051 [03:40<20:36,  2.09it/s, est. speed input: 69454.58 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 469/3051 [03:41<20:31,  2.10it/s, est. speed input: 69454.29 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 470/3051 [03:41<20:27,  2.10it/s, est. speed input: 69454.00 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 471/3051 [03:42<20:24,  2.11it/s, est. speed input: 69453.82 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 472/3051 [03:42<20:22,  2.11it/s, est. speed input: 69453.56 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 473/3051 [03:43<20:23,  2.11it/s, est. speed input: 69452.42 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 474/3051 [03:43<20:22,  2.11it/s, est. speed input: 69451.85 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 475/3051 [03:44<20:20,  2.11it/s, est. speed input: 69451.72 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 476/3051 [03:44<20:18,  2.11it/s, est. speed input: 69451.46 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 477/3051 [03:45<20:17,  2.11it/s, est. speed input: 69451.24 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 478/3051 [03:45<20:16,  2.11it/s, est. speed input: 69451.02 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 479/3051 [03:45<20:15,  2.12it/s, est. speed input: 69450.92 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 480/3051 [03:46<20:19,  2.11it/s, est. speed input: 69448.87 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 481/3051 [03:46<20:17,  2.11it/s, est. speed input: 69448.62 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 482/3051 [03:47<20:16,  2.11it/s, est. speed input: 69448.52 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 483/3051 [03:47<20:14,  2.11it/s, est. speed input: 69448.33 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 484/3051 [03:48<20:13,  2.11it/s, est. speed input: 69448.14 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 485/3051 [03:48<20:12,  2.12it/s, est. speed input: 69448.11 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 486/3051 [03:49<20:12,  2.12it/s, est. speed input: 69447.89 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 487/3051 [03:49<20:11,  2.12it/s, est. speed input: 69447.56 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 488/3051 [03:50<20:11,  2.12it/s, est. speed input: 69447.23 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 489/3051 [03:50<20:11,  2.12it/s, est. speed input: 69446.94 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 490/3051 [03:51<20:10,  2.12it/s, est. speed input: 69446.63 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 491/3051 [03:51<20:11,  2.11it/s, est. speed input: 69446.07 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 492/3051 [03:52<20:10,  2.11it/s, est. speed input: 69445.85 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 493/3051 [03:52<20:09,  2.11it/s, est. speed input: 69445.49 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 494/3051 [03:53<20:10,  2.11it/s, est. speed input: 69444.59 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 495/3051 [03:53<20:09,  2.11it/s, est. speed input: 69444.50 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 496/3051 [03:54<20:08,  2.11it/s, est. speed input: 69444.29 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 497/3051 [03:54<20:08,  2.11it/s, est. speed input: 69443.91 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 498/3051 [03:54<20:07,  2.11it/s, est. speed input: 69443.60 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 499/3051 [03:55<20:06,  2.12it/s, est. speed input: 69443.48 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 500/3051 [03:55<20:05,  2.12it/s, est. speed input: 69443.34 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 501/3051 [03:56<20:06,  2.11it/s, est. speed input: 69442.81 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 502/3051 [03:56<20:05,  2.11it/s, est. speed input: 69442.41 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 503/3051 [03:57<20:05,  2.11it/s, est. speed input: 69442.13 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 504/3051 [03:57<20:04,  2.11it/s, est. speed input: 69441.75 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 505/3051 [03:58<20:03,  2.11it/s, est. speed input: 69441.59 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 506/3051 [03:58<20:03,  2.11it/s, est. speed input: 69441.31 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 507/3051 [03:59<20:02,  2.12it/s, est. speed input: 69441.17 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 508/3051 [03:59<20:02,  2.11it/s, est. speed input: 69440.75 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 509/3051 [04:00<20:06,  2.11it/s, est. speed input: 69438.81 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 510/3051 [04:00<20:04,  2.11it/s, est. speed input: 69438.67 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 511/3051 [04:01<20:02,  2.11it/s, est. speed input: 69438.69 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 512/3051 [04:01<20:01,  2.11it/s, est. speed input: 69438.44 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 513/3051 [04:02<20:00,  2.11it/s, est. speed input: 69438.22 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 514/3051 [04:02<19:59,  2.11it/s, est. speed input: 69437.95 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 515/3051 [04:03<19:59,  2.11it/s, est. speed input: 69437.70 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 516/3051 [04:03<19:58,  2.12it/s, est. speed input: 69437.66 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 517/3051 [04:03<19:58,  2.11it/s, est. speed input: 69437.20 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 518/3051 [04:04<19:58,  2.11it/s, est. speed input: 69436.80 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 519/3051 [04:04<19:56,  2.12it/s, est. speed input: 69436.83 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 520/3051 [04:05<19:56,  2.12it/s, est. speed input: 69436.56 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 521/3051 [04:05<19:55,  2.12it/s, est. speed input: 69436.33 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 522/3051 [04:06<19:55,  2.12it/s, est. speed input: 69436.24 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 523/3051 [04:06<19:54,  2.12it/s, est. speed input: 69435.95 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 524/3051 [04:07<19:54,  2.12it/s, est. speed input: 69435.90 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 525/3051 [04:07<19:53,  2.12it/s, est. speed input: 69435.64 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 526/3051 [04:08<19:53,  2.12it/s, est. speed input: 69435.38 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 527/3051 [04:08<19:53,  2.11it/s, est. speed input: 69434.91 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 528/3051 [04:09<19:52,  2.11it/s, est. speed input: 69434.73 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 529/3051 [04:09<19:54,  2.11it/s, est. speed input: 69433.89 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 530/3051 [04:10<19:53,  2.11it/s, est. speed input: 69433.63 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 531/3051 [04:10<19:52,  2.11it/s, est. speed input: 69433.50 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 532/3051 [04:11<19:50,  2.12it/s, est. speed input: 69433.40 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 533/3051 [04:11<19:50,  2.12it/s, est. speed input: 69433.30 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 534/3051 [04:12<19:49,  2.12it/s, est. speed input: 69433.30 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 535/3051 [04:12<19:49,  2.12it/s, est. speed input: 69432.99 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 536/3051 [04:12<19:49,  2.11it/s, est. speed input: 69432.37 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 537/3051 [04:13<19:49,  2.11it/s, est. speed input: 69432.17 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 538/3051 [04:13<19:48,  2.11it/s, est. speed input: 69431.91 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 539/3051 [04:14<19:47,  2.11it/s, est. speed input: 69431.66 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 540/3051 [04:14<19:47,  2.11it/s, est. speed input: 69431.43 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 541/3051 [04:15<19:46,  2.11it/s, est. speed input: 69431.23 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 542/3051 [04:15<19:47,  2.11it/s, est. speed input: 69430.40 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 543/3051 [04:16<19:46,  2.11it/s, est. speed input: 69430.26 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 544/3051 [04:16<19:45,  2.11it/s, est. speed input: 69430.04 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 545/3051 [04:17<19:45,  2.11it/s, est. speed input: 69429.91 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 546/3051 [04:17<19:44,  2.11it/s, est. speed input: 69429.62 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 547/3051 [04:18<19:43,  2.12it/s, est. speed input: 69429.52 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 548/3051 [04:18<19:43,  2.12it/s, est. speed input: 69429.33 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 549/3051 [04:19<19:43,  2.11it/s, est. speed input: 69428.88 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 550/3051 [04:19<19:42,  2.11it/s, est. speed input: 69428.73 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 551/3051 [04:20<19:41,  2.12it/s, est. speed input: 69428.58 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 552/3051 [04:20<19:42,  2.11it/s, est. speed input: 69428.13 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 553/3051 [04:20<19:41,  2.12it/s, est. speed input: 69428.06 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 554/3051 [04:21<19:40,  2.12it/s, est. speed input: 69427.90 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 555/3051 [04:21<19:40,  2.12it/s, est. speed input: 69427.65 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 556/3051 [04:22<19:40,  2.11it/s, est. speed input: 69427.17 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 557/3051 [04:22<19:41,  2.11it/s, est. speed input: 69426.29 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 558/3051 [04:23<19:40,  2.11it/s, est. speed input: 69425.94 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 559/3051 [04:23<19:39,  2.11it/s, est. speed input: 69425.71 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 560/3051 [04:24<19:38,  2.11it/s, est. speed input: 69425.69 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 561/3051 [04:24<19:37,  2.11it/s, est. speed input: 69425.34 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 562/3051 [04:25<19:38,  2.11it/s, est. speed input: 69424.79 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 563/3051 [04:25<19:37,  2.11it/s, est. speed input: 69424.50 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 564/3051 [04:26<19:36,  2.11it/s, est. speed input: 69424.23 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 565/3051 [04:26<19:37,  2.11it/s, est. speed input: 69423.63 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 566/3051 [04:27<19:36,  2.11it/s, est. speed input: 69423.29 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 567/3051 [04:27<19:36,  2.11it/s, est. speed input: 69422.93 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 568/3051 [04:28<19:34,  2.11it/s, est. speed input: 69422.84 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 569/3051 [04:28<19:34,  2.11it/s, est. speed input: 69422.44 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 570/3051 [04:29<19:35,  2.11it/s, est. speed input: 69421.79 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 571/3051 [04:29<19:34,  2.11it/s, est. speed input: 69421.53 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 572/3051 [04:29<19:33,  2.11it/s, est. speed input: 69421.21 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 573/3051 [04:30<19:32,  2.11it/s, est. speed input: 69420.92 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 574/3051 [04:30<19:32,  2.11it/s, est. speed input: 69420.60 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 575/3051 [04:31<19:31,  2.11it/s, est. speed input: 69420.31 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 576/3051 [04:31<19:30,  2.11it/s, est. speed input: 69420.16 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 577/3051 [04:32<19:31,  2.11it/s, est. speed input: 69419.57 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 578/3051 [04:32<19:30,  2.11it/s, est. speed input: 69419.21 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 579/3051 [04:33<19:29,  2.11it/s, est. speed input: 69419.02 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 580/3051 [04:33<19:32,  2.11it/s, est. speed input: 69417.78 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 581/3051 [04:34<19:32,  2.11it/s, est. speed input: 69416.99 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 582/3051 [04:34<19:30,  2.11it/s, est. speed input: 69416.90 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 583/3051 [04:35<19:28,  2.11it/s, est. speed input: 69416.74 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 584/3051 [04:35<19:28,  2.11it/s, est. speed input: 69416.16 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 585/3051 [04:36<19:27,  2.11it/s, est. speed input: 69415.94 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 586/3051 [04:36<19:26,  2.11it/s, est. speed input: 69415.89 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 587/3051 [04:37<19:25,  2.11it/s, est. speed input: 69415.62 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 588/3051 [04:37<19:24,  2.11it/s, est. speed input: 69415.56 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 589/3051 [04:38<19:24,  2.11it/s, est. speed input: 69415.38 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 590/3051 [04:38<19:24,  2.11it/s, est. speed input: 69415.03 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 591/3051 [04:38<19:24,  2.11it/s, est. speed input: 69414.58 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 592/3051 [04:39<19:23,  2.11it/s, est. speed input: 69414.42 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 593/3051 [04:39<19:23,  2.11it/s, est. speed input: 69414.10 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 594/3051 [04:40<19:23,  2.11it/s, est. speed input: 69413.55 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 595/3051 [04:40<19:22,  2.11it/s, est. speed input: 69413.37 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 596/3051 [04:41<19:23,  2.11it/s, est. speed input: 69412.71 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 597/3051 [04:41<19:23,  2.11it/s, est. speed input: 69412.13 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 598/3051 [04:42<19:22,  2.11it/s, est. speed input: 69411.85 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 599/3051 [04:42<19:20,  2.11it/s, est. speed input: 69411.66 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 600/3051 [04:43<19:24,  2.11it/s, est. speed input: 69410.11 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 601/3051 [04:43<19:21,  2.11it/s, est. speed input: 69410.09 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 602/3051 [04:44<19:24,  2.10it/s, est. speed input: 69408.58 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 603/3051 [04:44<19:21,  2.11it/s, est. speed input: 69408.44 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 604/3051 [04:45<19:20,  2.11it/s, est. speed input: 69408.25 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 605/3051 [04:45<19:18,  2.11it/s, est. speed input: 69408.10 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 606/3051 [04:46<19:18,  2.11it/s, est. speed input: 69407.58 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 607/3051 [04:46<19:17,  2.11it/s, est. speed input: 69407.23 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 608/3051 [04:47<19:16,  2.11it/s, est. speed input: 69407.02 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 609/3051 [04:47<19:16,  2.11it/s, est. speed input: 69406.49 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 610/3051 [04:47<19:15,  2.11it/s, est. speed input: 69406.35 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 611/3051 [04:48<19:15,  2.11it/s, est. speed input: 69406.06 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 612/3051 [04:48<19:15,  2.11it/s, est. speed input: 69405.48 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 613/3051 [04:49<19:14,  2.11it/s, est. speed input: 69405.17 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 614/3051 [04:49<19:13,  2.11it/s, est. speed input: 69405.02 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 615/3051 [04:50<19:12,  2.11it/s, est. speed input: 69404.96 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 616/3051 [04:50<19:12,  2.11it/s, est. speed input: 69404.46 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 617/3051 [04:51<19:11,  2.11it/s, est. speed input: 69404.26 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 618/3051 [04:51<19:10,  2.11it/s, est. speed input: 69404.17 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 619/3051 [04:52<19:11,  2.11it/s, est. speed input: 69403.60 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 620/3051 [04:52<19:10,  2.11it/s, est. speed input: 69403.55 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 621/3051 [04:53<19:09,  2.11it/s, est. speed input: 69403.40 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 622/3051 [04:53<19:12,  2.11it/s, est. speed input: 69401.90 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 623/3051 [04:54<19:10,  2.11it/s, est. speed input: 69401.89 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 624/3051 [04:54<19:09,  2.11it/s, est. speed input: 69401.80 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 625/3051 [04:55<19:08,  2.11it/s, est. speed input: 69401.71 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 626/3051 [04:55<19:06,  2.11it/s, est. speed input: 69401.67 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 627/3051 [04:56<19:06,  2.11it/s, est. speed input: 69401.56 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 628/3051 [04:56<19:05,  2.11it/s, est. speed input: 69401.41 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 629/3051 [04:56<19:04,  2.12it/s, est. speed input: 69401.38 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 630/3051 [04:57<19:04,  2.12it/s, est. speed input: 69401.34 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 631/3051 [04:57<19:03,  2.12it/s, est. speed input: 69401.26 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 632/3051 [04:58<19:07,  2.11it/s, est. speed input: 69399.88 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 633/3051 [04:58<19:05,  2.11it/s, est. speed input: 69399.78 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 634/3051 [04:59<19:03,  2.11it/s, est. speed input: 69399.77 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 635/3051 [04:59<19:03,  2.11it/s, est. speed input: 69399.54 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 636/3051 [05:00<19:02,  2.11it/s, est. speed input: 69399.39 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 637/3051 [05:00<19:01,  2.11it/s, est. speed input: 69399.23 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 638/3051 [05:01<19:00,  2.11it/s, est. speed input: 69399.17 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 639/3051 [05:01<19:00,  2.11it/s, est. speed input: 69399.04 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 640/3051 [05:02<19:00,  2.11it/s, est. speed input: 69398.80 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 641/3051 [05:02<19:00,  2.11it/s, est. speed input: 69398.51 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 642/3051 [05:03<19:01,  2.11it/s, est. speed input: 69397.86 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 643/3051 [05:03<18:59,  2.11it/s, est. speed input: 69397.89 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 644/3051 [05:04<18:58,  2.11it/s, est. speed input: 69397.69 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 645/3051 [05:04<18:59,  2.11it/s, est. speed input: 69397.23 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 646/3051 [05:05<18:58,  2.11it/s, est. speed input: 69397.14 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 647/3051 [05:05<18:57,  2.11it/s, est. speed input: 69396.94 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 648/3051 [05:05<18:59,  2.11it/s, est. speed input: 69395.83 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 649/3051 [05:06<18:58,  2.11it/s, est. speed input: 69395.67 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 650/3051 [05:06<18:57,  2.11it/s, est. speed input: 69395.46 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 651/3051 [05:07<18:56,  2.11it/s, est. speed input: 69395.23 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 652/3051 [05:07<18:55,  2.11it/s, est. speed input: 69395.11 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 653/3051 [05:08<18:54,  2.11it/s, est. speed input: 69394.86 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 654/3051 [05:08<18:53,  2.11it/s, est. speed input: 69394.80 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 655/3051 [05:09<18:53,  2.11it/s, est. speed input: 69394.61 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 656/3051 [05:09<18:53,  2.11it/s, est. speed input: 69394.40 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 657/3051 [05:10<18:52,  2.11it/s, est. speed input: 69394.33 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 658/3051 [05:10<18:51,  2.11it/s, est. speed input: 69394.18 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 659/3051 [05:11<18:50,  2.12it/s, est. speed input: 69394.13 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 660/3051 [05:11<18:52,  2.11it/s, est. speed input: 69393.44 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 661/3051 [05:12<18:52,  2.11it/s, est. speed input: 69392.94 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 662/3051 [05:12<18:50,  2.11it/s, est. speed input: 69393.04 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 663/3051 [05:13<18:49,  2.11it/s, est. speed input: 69392.91 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 664/3051 [05:13<18:50,  2.11it/s, est. speed input: 69392.34 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 665/3051 [05:14<18:49,  2.11it/s, est. speed input: 69392.25 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 666/3051 [05:14<18:48,  2.11it/s, est. speed input: 69392.20 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 667/3051 [05:14<18:47,  2.11it/s, est. speed input: 69392.05 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 668/3051 [05:15<18:47,  2.11it/s, est. speed input: 69391.79 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 669/3051 [05:15<18:46,  2.11it/s, est. speed input: 69391.78 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 670/3051 [05:16<18:45,  2.11it/s, est. speed input: 69391.65 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 671/3051 [05:16<18:46,  2.11it/s, est. speed input: 69391.13 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 672/3051 [05:17<18:45,  2.11it/s, est. speed input: 69391.06 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 673/3051 [05:17<18:44,  2.11it/s, est. speed input: 69390.95 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 674/3051 [05:18<18:45,  2.11it/s, est. speed input: 69390.42 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 675/3051 [05:18<18:44,  2.11it/s, est. speed input: 69390.34 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 676/3051 [05:19<18:43,  2.11it/s, est. speed input: 69390.19 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 677/3051 [05:19<18:42,  2.11it/s, est. speed input: 69390.09 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 678/3051 [05:20<18:42,  2.11it/s, est. speed input: 69390.05 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 679/3051 [05:20<19:05,  2.07it/s, est. speed input: 69382.52 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 680/3051 [05:21<18:57,  2.08it/s, est. speed input: 69382.52 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 681/3051 [05:21<18:51,  2.09it/s, est. speed input: 69382.51 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 682/3051 [05:22<18:48,  2.10it/s, est. speed input: 69382.37 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 683/3051 [05:22<18:49,  2.10it/s, est. speed input: 69380.94 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 684/3051 [05:23<18:46,  2.10it/s, est. speed input: 69380.84 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 685/3051 [05:23<18:43,  2.11it/s, est. speed input: 69380.79 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 686/3051 [05:23<18:41,  2.11it/s, est. speed input: 69380.73 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 687/3051 [05:24<18:40,  2.11it/s, est. speed input: 69380.57 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 688/3051 [05:24<18:38,  2.11it/s, est. speed input: 69380.60 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 689/3051 [05:25<18:37,  2.11it/s, est. speed input: 69380.55 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 690/3051 [05:25<18:36,  2.11it/s, est. speed input: 69380.40 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 691/3051 [05:26<18:36,  2.11it/s, est. speed input: 69380.23 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 692/3051 [05:26<18:35,  2.12it/s, est. speed input: 69380.25 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 693/3051 [05:27<18:35,  2.11it/s, est. speed input: 69380.05 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 694/3051 [05:27<18:34,  2.12it/s, est. speed input: 69380.04 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 695/3051 [05:28<18:33,  2.12it/s, est. speed input: 69380.00 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 696/3051 [05:28<18:34,  2.11it/s, est. speed input: 69379.56 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 697/3051 [05:29<18:33,  2.11it/s, est. speed input: 69379.47 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 698/3051 [05:29<18:32,  2.11it/s, est. speed input: 69379.45 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 699/3051 [05:30<18:32,  2.11it/s, est. speed input: 69379.20 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 700/3051 [05:30<18:31,  2.11it/s, est. speed input: 69379.09 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 701/3051 [05:31<18:31,  2.11it/s, est. speed input: 69378.96 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 702/3051 [05:31<18:31,  2.11it/s, est. speed input: 69378.80 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 703/3051 [05:32<18:30,  2.11it/s, est. speed input: 69378.71 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 704/3051 [05:32<18:30,  2.11it/s, est. speed input: 69378.30 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 705/3051 [05:32<18:31,  2.11it/s, est. speed input: 69377.86 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 706/3051 [05:33<18:30,  2.11it/s, est. speed input: 69377.56 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 707/3051 [05:33<18:29,  2.11it/s, est. speed input: 69377.41 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 708/3051 [05:34<18:29,  2.11it/s, est. speed input: 69377.20 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 709/3051 [05:34<18:28,  2.11it/s, est. speed input: 69377.18 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 710/3051 [05:35<18:27,  2.11it/s, est. speed input: 69377.03 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 711/3051 [05:35<18:27,  2.11it/s, est. speed input: 69376.85 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 712/3051 [05:36<18:26,  2.11it/s, est. speed input: 69376.60 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 713/3051 [05:36<18:26,  2.11it/s, est. speed input: 69376.52 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 714/3051 [05:37<18:25,  2.11it/s, est. speed input: 69376.43 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 715/3051 [05:37<18:24,  2.11it/s, est. speed input: 69376.31 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 716/3051 [05:38<18:23,  2.12it/s, est. speed input: 69376.32 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 717/3051 [05:38<18:23,  2.12it/s, est. speed input: 69376.31 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 718/3051 [05:39<18:23,  2.11it/s, est. speed input: 69376.07 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 719/3051 [05:39<18:24,  2.11it/s, est. speed input: 69375.41 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 720/3051 [05:40<18:23,  2.11it/s, est. speed input: 69375.29 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 721/3051 [05:40<18:22,  2.11it/s, est. speed input: 69375.28 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 722/3051 [05:41<18:21,  2.11it/s, est. speed input: 69375.19 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 723/3051 [05:41<18:20,  2.11it/s, est. speed input: 69375.11 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 724/3051 [05:41<18:24,  2.11it/s, est. speed input: 69373.81 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 725/3051 [05:42<18:23,  2.11it/s, est. speed input: 69373.52 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 726/3051 [05:42<18:21,  2.11it/s, est. speed input: 69373.46 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 727/3051 [05:43<18:26,  2.10it/s, est. speed input: 69371.56 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 728/3051 [05:43<18:23,  2.10it/s, est. speed input: 69371.49 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 729/3051 [05:44<18:21,  2.11it/s, est. speed input: 69371.49 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 730/3051 [05:44<18:19,  2.11it/s, est. speed input: 69371.40 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 731/3051 [05:45<18:18,  2.11it/s, est. speed input: 69371.40 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 732/3051 [05:45<18:17,  2.11it/s, est. speed input: 69371.18 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 733/3051 [05:46<18:16,  2.11it/s, est. speed input: 69371.23 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 734/3051 [05:46<18:16,  2.11it/s, est. speed input: 69371.06 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 735/3051 [05:47<18:15,  2.11it/s, est. speed input: 69371.05 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 736/3051 [05:47<18:14,  2.11it/s, est. speed input: 69370.88 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 737/3051 [05:48<18:13,  2.12it/s, est. speed input: 69370.90 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 738/3051 [05:48<18:13,  2.11it/s, est. speed input: 69370.68 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 739/3051 [05:49<18:13,  2.11it/s, est. speed input: 69370.52 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 740/3051 [05:49<18:13,  2.11it/s, est. speed input: 69370.32 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 741/3051 [05:50<18:16,  2.11it/s, est. speed input: 69369.20 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 742/3051 [05:50<18:13,  2.11it/s, est. speed input: 69369.33 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 743/3051 [05:50<18:12,  2.11it/s, est. speed input: 69369.23 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 744/3051 [05:51<18:11,  2.11it/s, est. speed input: 69369.23 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 745/3051 [05:51<18:10,  2.11it/s, est. speed input: 69369.18 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 746/3051 [05:52<18:10,  2.11it/s, est. speed input: 69369.05 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 747/3051 [05:52<18:14,  2.11it/s, est. speed input: 69367.71 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 748/3051 [05:53<18:12,  2.11it/s, est. speed input: 69367.47 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 749/3051 [05:53<18:11,  2.11it/s, est. speed input: 69367.40 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 750/3051 [05:54<18:11,  2.11it/s, est. speed input: 69366.93 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 751/3051 [05:54<18:09,  2.11it/s, est. speed input: 69366.83 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 752/3051 [05:55<18:08,  2.11it/s, est. speed input: 69366.77 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 753/3051 [05:55<18:10,  2.11it/s, est. speed input: 69365.83 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 754/3051 [05:56<18:08,  2.11it/s, est. speed input: 69365.79 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 755/3051 [05:56<18:06,  2.11it/s, est. speed input: 69365.87 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 756/3051 [05:57<18:06,  2.11it/s, est. speed input: 69365.55 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 757/3051 [05:57<18:05,  2.11it/s, est. speed input: 69365.45 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 758/3051 [05:58<18:05,  2.11it/s, est. speed input: 69365.27 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 759/3051 [05:58<18:04,  2.11it/s, est. speed input: 69365.22 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 760/3051 [05:59<18:03,  2.11it/s, est. speed input: 69365.24 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 761/3051 [05:59<18:02,  2.11it/s, est. speed input: 69365.14 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 762/3051 [05:59<18:02,  2.11it/s, est. speed input: 69365.02 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 763/3051 [06:00<18:01,  2.11it/s, est. speed input: 69364.99 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 764/3051 [06:00<18:01,  2.11it/s, est. speed input: 69364.84 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 765/3051 [06:01<18:00,  2.12it/s, est. speed input: 69364.93 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 766/3051 [06:01<18:01,  2.11it/s, est. speed input: 69364.57 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 767/3051 [06:02<18:00,  2.11it/s, est. speed input: 69364.52 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 768/3051 [06:02<17:59,  2.11it/s, est. speed input: 69364.38 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 769/3051 [06:03<17:58,  2.12it/s, est. speed input: 69364.45 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 770/3051 [06:03<17:58,  2.12it/s, est. speed input: 69364.40 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 771/3051 [06:04<17:58,  2.11it/s, est. speed input: 69364.15 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 772/3051 [06:04<17:58,  2.11it/s, est. speed input: 69363.96 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 773/3051 [06:05<17:57,  2.11it/s, est. speed input: 69363.79 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 774/3051 [06:05<17:57,  2.11it/s, est. speed input: 69363.64 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 775/3051 [06:06<17:56,  2.11it/s, est. speed input: 69363.51 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 776/3051 [06:06<17:56,  2.11it/s, est. speed input: 69363.30 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 777/3051 [06:07<17:55,  2.11it/s, est. speed input: 69363.27 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 778/3051 [06:07<17:55,  2.11it/s, est. speed input: 69362.98 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 779/3051 [06:08<17:55,  2.11it/s, est. speed input: 69362.84 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 780/3051 [06:08<17:54,  2.11it/s, est. speed input: 69362.71 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 781/3051 [06:08<17:53,  2.11it/s, est. speed input: 69362.63 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 782/3051 [06:09<17:53,  2.11it/s, est. speed input: 69362.55 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 783/3051 [06:09<17:54,  2.11it/s, est. speed input: 69361.86 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 784/3051 [06:10<17:53,  2.11it/s, est. speed input: 69361.82 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 785/3051 [06:10<17:52,  2.11it/s, est. speed input: 69361.72 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 786/3051 [06:11<17:52,  2.11it/s, est. speed input: 69361.58 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 787/3051 [06:11<17:52,  2.11it/s, est. speed input: 69361.29 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 788/3051 [06:12<17:51,  2.11it/s, est. speed input: 69361.17 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 789/3051 [06:12<17:50,  2.11it/s, est. speed input: 69361.13 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 790/3051 [06:13<17:48,  2.12it/s, est. speed input: 69361.26 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 791/3051 [06:13<17:48,  2.11it/s, est. speed input: 69361.14 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 792/3051 [06:14<17:47,  2.12it/s, est. speed input: 69361.19 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 793/3051 [06:14<17:47,  2.11it/s, est. speed input: 69361.00 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 794/3051 [06:15<17:46,  2.12it/s, est. speed input: 69361.00 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 795/3051 [06:15<17:46,  2.12it/s, est. speed input: 69360.98 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 796/3051 [06:16<17:46,  2.12it/s, est. speed input: 69360.87 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 797/3051 [06:16<17:45,  2.11it/s, est. speed input: 69360.77 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 798/3051 [06:16<17:45,  2.11it/s, est. speed input: 69360.69 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 799/3051 [06:17<17:47,  2.11it/s, est. speed input: 69359.99 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 800/3051 [06:17<17:45,  2.11it/s, est. speed input: 69359.92 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 801/3051 [06:18<17:45,  2.11it/s, est. speed input: 69359.83 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 802/3051 [06:18<17:44,  2.11it/s, est. speed input: 69359.82 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 803/3051 [06:19<17:43,  2.11it/s, est. speed input: 69359.63 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 804/3051 [06:19<17:47,  2.10it/s, est. speed input: 69358.26 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 805/3051 [06:20<17:46,  2.11it/s, est. speed input: 69358.08 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 806/3051 [06:20<17:43,  2.11it/s, est. speed input: 69358.13 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 807/3051 [06:21<17:42,  2.11it/s, est. speed input: 69358.17 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 808/3051 [06:21<17:40,  2.11it/s, est. speed input: 69358.25 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 809/3051 [06:22<17:40,  2.11it/s, est. speed input: 69358.07 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 810/3051 [06:22<17:39,  2.11it/s, est. speed input: 69358.05 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 811/3051 [06:23<17:40,  2.11it/s, est. speed input: 69357.58 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 812/3051 [06:23<17:40,  2.11it/s, est. speed input: 69357.45 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 813/3051 [06:24<17:39,  2.11it/s, est. speed input: 69357.38 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 814/3051 [06:24<17:38,  2.11it/s, est. speed input: 69357.24 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 815/3051 [06:25<17:40,  2.11it/s, est. speed input: 69356.45 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 816/3051 [06:25<17:39,  2.11it/s, est. speed input: 69356.42 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 817/3051 [06:25<17:37,  2.11it/s, est. speed input: 69356.32 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 818/3051 [06:26<17:37,  2.11it/s, est. speed input: 69356.18 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 819/3051 [06:26<17:36,  2.11it/s, est. speed input: 69356.13 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 820/3051 [06:27<17:35,  2.11it/s, est. speed input: 69356.18 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 821/3051 [06:27<17:34,  2.11it/s, est. speed input: 69356.07 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 822/3051 [06:28<17:33,  2.12it/s, est. speed input: 69356.14 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 823/3051 [06:28<17:33,  2.11it/s, est. speed input: 69355.91 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 824/3051 [06:29<17:33,  2.11it/s, est. speed input: 69355.72 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 825/3051 [06:29<17:32,  2.11it/s, est. speed input: 69355.74 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 826/3051 [06:30<17:32,  2.11it/s, est. speed input: 69355.61 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 827/3051 [06:30<17:31,  2.11it/s, est. speed input: 69355.54 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 828/3051 [06:31<17:32,  2.11it/s, est. speed input: 69355.11 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 829/3051 [06:31<17:31,  2.11it/s, est. speed input: 69355.04 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 830/3051 [06:32<17:30,  2.11it/s, est. speed input: 69355.10 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 831/3051 [06:32<17:29,  2.12it/s, est. speed input: 69355.12 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 832/3051 [06:33<17:29,  2.12it/s, est. speed input: 69355.07 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 833/3051 [06:33<17:29,  2.11it/s, est. speed input: 69354.83 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 834/3051 [06:34<17:28,  2.11it/s, est. speed input: 69354.72 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 835/3051 [06:34<17:27,  2.12it/s, est. speed input: 69354.80 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 836/3051 [06:34<17:27,  2.12it/s, est. speed input: 69354.78 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 837/3051 [06:35<17:27,  2.11it/s, est. speed input: 69354.63 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 838/3051 [06:35<17:27,  2.11it/s, est. speed input: 69354.29 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 839/3051 [06:36<17:26,  2.11it/s, est. speed input: 69354.30 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 840/3051 [06:36<17:25,  2.11it/s, est. speed input: 69354.20 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 841/3051 [06:37<17:26,  2.11it/s, est. speed input: 69353.85 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 842/3051 [06:37<17:25,  2.11it/s, est. speed input: 69353.83 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 843/3051 [06:38<17:24,  2.11it/s, est. speed input: 69353.82 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 844/3051 [06:38<17:23,  2.11it/s, est. speed input: 69353.74 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 845/3051 [06:39<17:23,  2.11it/s, est. speed input: 69353.64 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 846/3051 [06:39<17:22,  2.12it/s, est. speed input: 69353.66 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 847/3051 [06:40<17:22,  2.11it/s, est. speed input: 69353.58 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 848/3051 [06:40<17:21,  2.12it/s, est. speed input: 69353.61 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 849/3051 [06:41<17:20,  2.12it/s, est. speed input: 69353.59 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 850/3051 [06:41<17:20,  2.12it/s, est. speed input: 69353.52 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 851/3051 [06:42<17:21,  2.11it/s, est. speed input: 69353.20 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 852/3051 [06:42<17:19,  2.11it/s, est. speed input: 69353.25 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 853/3051 [06:43<17:20,  2.11it/s, est. speed input: 69352.86 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 854/3051 [06:43<17:20,  2.11it/s, est. speed input: 69352.73 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 855/3051 [06:43<17:19,  2.11it/s, est. speed input: 69352.64 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 856/3051 [06:44<17:18,  2.11it/s, est. speed input: 69352.69 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 857/3051 [06:44<17:17,  2.12it/s, est. speed input: 69352.75 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 858/3051 [06:45<17:16,  2.12it/s, est. speed input: 69352.69 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 859/3051 [06:45<17:16,  2.12it/s, est. speed input: 69352.69 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 860/3051 [06:46<17:15,  2.12it/s, est. speed input: 69352.61 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 861/3051 [06:46<17:15,  2.11it/s, est. speed input: 69352.44 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 862/3051 [06:47<17:15,  2.11it/s, est. speed input: 69352.26 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 863/3051 [06:47<17:15,  2.11it/s, est. speed input: 69352.18 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 864/3051 [06:48<17:15,  2.11it/s, est. speed input: 69351.96 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 865/3051 [06:48<17:14,  2.11it/s, est. speed input: 69351.85 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 866/3051 [06:49<17:13,  2.11it/s, est. speed input: 69351.85 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 867/3051 [06:49<17:12,  2.11it/s, est. speed input: 69351.86 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 868/3051 [06:50<17:12,  2.11it/s, est. speed input: 69351.76 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 869/3051 [06:50<17:12,  2.11it/s, est. speed input: 69351.44 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 870/3051 [06:51<17:12,  2.11it/s, est. speed input: 69351.28 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 871/3051 [06:51<17:11,  2.11it/s, est. speed input: 69351.34 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 872/3051 [06:52<17:10,  2.11it/s, est. speed input: 69351.33 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 873/3051 [06:52<17:10,  2.11it/s, est. speed input: 69351.07 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 874/3051 [06:52<17:10,  2.11it/s, est. speed input: 69350.97 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 875/3051 [06:53<17:08,  2.12it/s, est. speed input: 69351.07 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 876/3051 [06:53<17:09,  2.11it/s, est. speed input: 69350.77 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 877/3051 [06:54<17:08,  2.11it/s, est. speed input: 69350.64 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 878/3051 [06:54<17:08,  2.11it/s, est. speed input: 69350.55 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 879/3051 [06:55<17:07,  2.11it/s, est. speed input: 69350.53 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 880/3051 [06:55<17:06,  2.11it/s, est. speed input: 69350.47 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 881/3051 [06:56<17:05,  2.12it/s, est. speed input: 69350.52 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 882/3051 [06:56<17:06,  2.11it/s, est. speed input: 69350.29 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 883/3051 [06:57<17:06,  2.11it/s, est. speed input: 69349.94 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 884/3051 [06:57<17:05,  2.11it/s, est. speed input: 69349.83 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 885/3051 [06:58<17:08,  2.11it/s, est. speed input: 69348.92 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 886/3051 [06:58<17:07,  2.11it/s, est. speed input: 69348.73 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 887/3051 [06:59<17:05,  2.11it/s, est. speed input: 69348.73 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 888/3051 [06:59<17:04,  2.11it/s, est. speed input: 69348.67 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 889/3051 [07:00<17:04,  2.11it/s, est. speed input: 69348.30 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 890/3051 [07:00<17:04,  2.11it/s, est. speed input: 69348.05 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 891/3051 [07:01<17:03,  2.11it/s, est. speed input: 69347.98 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 892/3051 [07:01<17:02,  2.11it/s, est. speed input: 69347.73 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 893/3051 [07:01<17:02,  2.11it/s, est. speed input: 69347.52 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 894/3051 [07:02<17:01,  2.11it/s, est. speed input: 69347.45 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 895/3051 [07:02<17:00,  2.11it/s, est. speed input: 69347.35 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 896/3051 [07:03<16:59,  2.11it/s, est. speed input: 69347.33 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 897/3051 [07:03<16:58,  2.11it/s, est. speed input: 69347.31 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 898/3051 [07:04<16:58,  2.11it/s, est. speed input: 69347.26 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 899/3051 [07:04<16:58,  2.11it/s, est. speed input: 69347.13 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 900/3051 [07:05<16:58,  2.11it/s, est. speed input: 69346.88 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 901/3051 [07:05<16:58,  2.11it/s, est. speed input: 69346.57 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 902/3051 [07:06<16:57,  2.11it/s, est. speed input: 69346.49 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 903/3051 [07:06<16:56,  2.11it/s, est. speed input: 69346.46 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 904/3051 [07:07<16:55,  2.11it/s, est. speed input: 69346.37 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 905/3051 [07:07<16:55,  2.11it/s, est. speed input: 69346.25 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 906/3051 [07:08<16:54,  2.11it/s, est. speed input: 69346.32 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 907/3051 [07:08<16:53,  2.11it/s, est. speed input: 69346.28 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 908/3051 [07:09<16:53,  2.11it/s, est. speed input: 69346.08 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 909/3051 [07:09<16:53,  2.11it/s, est. speed input: 69345.96 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 910/3051 [07:10<16:52,  2.12it/s, est. speed input: 69346.07 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 911/3051 [07:10<16:53,  2.11it/s, est. speed input: 69345.70 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 912/3051 [07:10<16:52,  2.11it/s, est. speed input: 69345.62 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 913/3051 [07:11<16:51,  2.11it/s, est. speed input: 69345.54 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 914/3051 [07:11<16:51,  2.11it/s, est. speed input: 69345.29 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 915/3051 [07:12<16:51,  2.11it/s, est. speed input: 69345.22 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 916/3051 [07:12<16:50,  2.11it/s, est. speed input: 69345.18 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 917/3051 [07:13<16:49,  2.11it/s, est. speed input: 69345.16 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 918/3051 [07:13<16:48,  2.12it/s, est. speed input: 69345.20 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 919/3051 [07:14<16:48,  2.12it/s, est. speed input: 69345.15 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 920/3051 [07:14<16:47,  2.11it/s, est. speed input: 69345.01 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 921/3051 [07:15<16:48,  2.11it/s, est. speed input: 69344.59 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 922/3051 [07:15<16:47,  2.11it/s, est. speed input: 69344.60 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 923/3051 [07:16<16:49,  2.11it/s, est. speed input: 69344.01 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 924/3051 [07:16<16:49,  2.11it/s, est. speed input: 69343.57 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 925/3051 [07:17<16:49,  2.11it/s, est. speed input: 69343.13 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 926/3051 [07:17<16:47,  2.11it/s, est. speed input: 69343.03 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 927/3051 [07:18<16:46,  2.11it/s, est. speed input: 69342.95 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 928/3051 [07:18<16:46,  2.11it/s, est. speed input: 69342.65 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 929/3051 [07:19<16:45,  2.11it/s, est. speed input: 69342.65 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 930/3051 [07:19<16:44,  2.11it/s, est. speed input: 69342.41 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 931/3051 [07:19<16:44,  2.11it/s, est. speed input: 69342.26 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 932/3051 [07:20<16:43,  2.11it/s, est. speed input: 69342.20 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 933/3051 [07:20<16:43,  2.11it/s, est. speed input: 69341.89 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 934/3051 [07:21<16:42,  2.11it/s, est. speed input: 69341.74 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 935/3051 [07:21<16:41,  2.11it/s, est. speed input: 69341.69 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 936/3051 [07:22<16:40,  2.11it/s, est. speed input: 69341.70 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 937/3051 [07:22<16:39,  2.11it/s, est. speed input: 69341.70 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 938/3051 [07:23<16:39,  2.11it/s, est. speed input: 69341.66 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 939/3051 [07:23<16:39,  2.11it/s, est. speed input: 69341.50 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 940/3051 [07:24<16:39,  2.11it/s, est. speed input: 69341.32 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 941/3051 [07:24<16:41,  2.11it/s, est. speed input: 69340.48 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 942/3051 [07:25<16:40,  2.11it/s, est. speed input: 69340.43 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 943/3051 [07:25<16:38,  2.11it/s, est. speed input: 69340.41 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 944/3051 [07:26<16:37,  2.11it/s, est. speed input: 69340.46 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 945/3051 [07:26<16:36,  2.11it/s, est. speed input: 69340.41 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 946/3051 [07:27<16:36,  2.11it/s, est. speed input: 69340.21 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 947/3051 [07:27<16:36,  2.11it/s, est. speed input: 69340.07 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 948/3051 [07:27<16:35,  2.11it/s, est. speed input: 69339.98 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 949/3051 [07:28<16:35,  2.11it/s, est. speed input: 69339.67 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 950/3051 [07:28<16:34,  2.11it/s, est. speed input: 69339.59 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 951/3051 [07:29<16:34,  2.11it/s, est. speed input: 69339.48 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 952/3051 [07:29<16:33,  2.11it/s, est. speed input: 69339.47 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 953/3051 [07:30<16:32,  2.11it/s, est. speed input: 69339.46 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 954/3051 [07:30<16:32,  2.11it/s, est. speed input: 69339.31 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 955/3051 [07:31<16:31,  2.11it/s, est. speed input: 69339.30 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 956/3051 [07:31<16:31,  2.11it/s, est. speed input: 69339.21 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 957/3051 [07:32<16:31,  2.11it/s, est. speed input: 69338.81 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 958/3051 [07:32<16:30,  2.11it/s, est. speed input: 69338.83 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 959/3051 [07:33<16:30,  2.11it/s, est. speed input: 69338.57 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 960/3051 [07:33<16:30,  2.11it/s, est. speed input: 69338.44 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 961/3051 [07:34<16:29,  2.11it/s, est. speed input: 69338.24 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 962/3051 [07:34<16:32,  2.10it/s, est. speed input: 69337.27 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 963/3051 [07:35<16:30,  2.11it/s, est. speed input: 69337.19 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 964/3051 [07:35<16:29,  2.11it/s, est. speed input: 69337.23 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 965/3051 [07:36<16:28,  2.11it/s, est. speed input: 69337.04 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 966/3051 [07:36<16:27,  2.11it/s, est. speed input: 69336.97 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 967/3051 [07:36<16:26,  2.11it/s, est. speed input: 69336.91 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 968/3051 [07:37<16:25,  2.11it/s, est. speed input: 69336.92 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 969/3051 [07:37<16:24,  2.11it/s, est. speed input: 69336.89 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 970/3051 [07:38<16:24,  2.11it/s, est. speed input: 69336.92 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 971/3051 [07:38<16:23,  2.11it/s, est. speed input: 69336.82 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 972/3051 [07:39<16:23,  2.11it/s, est. speed input: 69336.78 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 973/3051 [07:39<16:25,  2.11it/s, est. speed input: 69335.98 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 974/3051 [07:40<16:24,  2.11it/s, est. speed input: 69335.93 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 975/3051 [07:40<16:23,  2.11it/s, est. speed input: 69335.88 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 976/3051 [07:41<16:22,  2.11it/s, est. speed input: 69335.88 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 977/3051 [07:41<16:21,  2.11it/s, est. speed input: 69335.87 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 978/3051 [07:42<16:20,  2.11it/s, est. speed input: 69335.88 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 979/3051 [07:42<16:20,  2.11it/s, est. speed input: 69335.77 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 980/3051 [07:43<16:19,  2.11it/s, est. speed input: 69335.72 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 981/3051 [07:43<16:18,  2.11it/s, est. speed input: 69335.72 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 982/3051 [07:44<16:18,  2.12it/s, est. speed input: 69335.73 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 983/3051 [07:44<16:17,  2.12it/s, est. speed input: 69335.74 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 984/3051 [07:45<16:50,  2.04it/s, est. speed input: 69327.62 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 985/3051 [07:45<16:40,  2.07it/s, est. speed input: 69327.61 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 986/3051 [07:46<16:32,  2.08it/s, est. speed input: 69327.58 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 987/3051 [07:46<16:26,  2.09it/s, est. speed input: 69327.66 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 988/3051 [07:46<16:22,  2.10it/s, est. speed input: 69327.67 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 989/3051 [07:47<16:20,  2.10it/s, est. speed input: 69327.69 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 990/3051 [07:47<16:19,  2.11it/s, est. speed input: 69327.43 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 991/3051 [07:48<16:17,  2.11it/s, est. speed input: 69327.39 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 992/3051 [07:48<16:15,  2.11it/s, est. speed input: 69327.38 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 993/3051 [07:49<16:14,  2.11it/s, est. speed input: 69327.38 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 994/3051 [07:49<16:13,  2.11it/s, est. speed input: 69327.39 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 995/3051 [07:50<16:12,  2.11it/s, est. speed input: 69327.41 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 996/3051 [07:50<16:11,  2.11it/s, est. speed input: 69327.44 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 997/3051 [07:51<16:11,  2.11it/s, est. speed input: 69327.26 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 998/3051 [07:51<16:11,  2.11it/s, est. speed input: 69327.12 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 999/3051 [07:52<16:10,  2.11it/s, est. speed input: 69327.09 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1000/3051 [07:52<16:10,  2.11it/s, est. speed input: 69327.05 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1001/3051 [07:53<16:09,  2.11it/s, est. speed input: 69327.00 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1002/3051 [07:53<16:09,  2.11it/s, est. speed input: 69326.99 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1003/3051 [07:54<16:09,  2.11it/s, est. speed input: 69326.77 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1004/3051 [07:54<16:08,  2.11it/s, est. speed input: 69326.69 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1005/3051 [07:55<16:08,  2.11it/s, est. speed input: 69326.67 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1006/3051 [07:55<16:10,  2.11it/s, est. speed input: 69325.91 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1007/3051 [07:55<16:08,  2.11it/s, est. speed input: 69325.93 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1008/3051 [07:56<16:07,  2.11it/s, est. speed input: 69325.89 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1009/3051 [07:56<16:08,  2.11it/s, est. speed input: 69325.51 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1010/3051 [07:57<16:07,  2.11it/s, est. speed input: 69325.43 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1011/3051 [07:57<16:06,  2.11it/s, est. speed input: 69325.34 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1012/3051 [07:58<16:05,  2.11it/s, est. speed input: 69325.39 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1013/3051 [07:58<16:04,  2.11it/s, est. speed input: 69325.34 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1014/3051 [07:59<16:03,  2.11it/s, est. speed input: 69325.31 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1015/3051 [07:59<16:03,  2.11it/s, est. speed input: 69325.29 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1016/3051 [08:00<16:03,  2.11it/s, est. speed input: 69324.99 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1017/3051 [08:00<16:04,  2.11it/s, est. speed input: 69324.50 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1018/3051 [08:01<16:02,  2.11it/s, est. speed input: 69324.57 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1019/3051 [08:01<16:02,  2.11it/s, est. speed input: 69324.53 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1020/3051 [08:02<16:02,  2.11it/s, est. speed input: 69324.18 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1021/3051 [08:02<16:01,  2.11it/s, est. speed input: 69324.14 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1022/3051 [08:03<16:00,  2.11it/s, est. speed input: 69324.10 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1023/3051 [08:03<16:01,  2.11it/s, est. speed input: 69323.70 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1024/3051 [08:04<16:00,  2.11it/s, est. speed input: 69323.62 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1025/3051 [08:04<15:59,  2.11it/s, est. speed input: 69323.56 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1026/3051 [08:04<15:58,  2.11it/s, est. speed input: 69323.65 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1027/3051 [08:05<15:57,  2.11it/s, est. speed input: 69323.50 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1028/3051 [08:05<15:57,  2.11it/s, est. speed input: 69323.49 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1029/3051 [08:06<15:56,  2.11it/s, est. speed input: 69323.41 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1030/3051 [08:06<15:56,  2.11it/s, est. speed input: 69323.34 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1031/3051 [08:07<15:55,  2.11it/s, est. speed input: 69323.30 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1032/3051 [08:07<15:54,  2.11it/s, est. speed input: 69323.31 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1033/3051 [08:08<15:55,  2.11it/s, est. speed input: 69322.96 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1034/3051 [08:08<15:54,  2.11it/s, est. speed input: 69323.00 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1035/3051 [08:09<15:54,  2.11it/s, est. speed input: 69322.87 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1036/3051 [08:09<15:53,  2.11it/s, est. speed input: 69322.96 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1037/3051 [08:10<15:52,  2.11it/s, est. speed input: 69322.95 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1038/3051 [08:10<15:51,  2.11it/s, est. speed input: 69322.91 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1039/3051 [08:11<15:51,  2.11it/s, est. speed input: 69322.81 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1040/3051 [08:11<15:50,  2.11it/s, est. speed input: 69322.86 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1041/3051 [08:12<15:50,  2.12it/s, est. speed input: 69322.87 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1042/3051 [08:12<15:49,  2.11it/s, est. speed input: 69322.82 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1043/3051 [08:13<15:50,  2.11it/s, est. speed input: 69322.59 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1044/3051 [08:13<15:49,  2.11it/s, est. speed input: 69322.54 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1045/3051 [08:13<15:48,  2.11it/s, est. speed input: 69322.58 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1046/3051 [08:14<15:47,  2.12it/s, est. speed input: 69322.66 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1047/3051 [08:14<15:47,  2.12it/s, est. speed input: 69322.64 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1048/3051 [08:15<15:48,  2.11it/s, est. speed input: 69322.29 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1049/3051 [08:15<15:48,  2.11it/s, est. speed input: 69322.13 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1050/3051 [08:16<15:46,  2.11it/s, est. speed input: 69322.24 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1051/3051 [08:16<15:47,  2.11it/s, est. speed input: 69321.95 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1052/3051 [08:17<15:47,  2.11it/s, est. speed input: 69321.70 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1053/3051 [08:17<15:46,  2.11it/s, est. speed input: 69321.70 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1054/3051 [08:18<15:53,  2.10it/s, est. speed input: 69319.83 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1055/3051 [08:18<15:49,  2.10it/s, est. speed input: 69319.88 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1056/3051 [08:19<15:47,  2.11it/s, est. speed input: 69319.92 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1057/3051 [08:19<15:45,  2.11it/s, est. speed input: 69319.90 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1058/3051 [08:20<15:44,  2.11it/s, est. speed input: 69319.86 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1059/3051 [08:20<15:43,  2.11it/s, est. speed input: 69319.77 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1060/3051 [08:21<15:42,  2.11it/s, est. speed input: 69319.82 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1061/3051 [08:21<15:41,  2.11it/s, est. speed input: 69319.83 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1062/3051 [08:22<15:40,  2.11it/s, est. speed input: 69319.85 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1063/3051 [08:22<15:40,  2.11it/s, est. speed input: 69319.79 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1064/3051 [08:22<15:40,  2.11it/s, est. speed input: 69319.61 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1065/3051 [08:23<15:40,  2.11it/s, est. speed input: 69319.30 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1066/3051 [08:23<15:39,  2.11it/s, est. speed input: 69319.40 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1067/3051 [08:24<15:39,  2.11it/s, est. speed input: 69319.29 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1068/3051 [08:24<15:39,  2.11it/s, est. speed input: 69319.00 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1069/3051 [08:25<15:38,  2.11it/s, est. speed input: 69318.92 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1070/3051 [08:25<15:37,  2.11it/s, est. speed input: 69318.91 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1071/3051 [08:26<15:37,  2.11it/s, est. speed input: 69318.88 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1072/3051 [08:26<15:36,  2.11it/s, est. speed input: 69318.81 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1073/3051 [08:27<15:36,  2.11it/s, est. speed input: 69318.70 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1074/3051 [08:27<15:36,  2.11it/s, est. speed input: 69318.50 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1075/3051 [08:28<15:35,  2.11it/s, est. speed input: 69318.43 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1076/3051 [08:28<15:34,  2.11it/s, est. speed input: 69318.45 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1077/3051 [08:29<15:34,  2.11it/s, est. speed input: 69318.20 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1078/3051 [08:29<15:34,  2.11it/s, est. speed input: 69318.15 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1079/3051 [08:30<15:33,  2.11it/s, est. speed input: 69318.18 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1080/3051 [08:30<15:32,  2.11it/s, est. speed input: 69318.12 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1081/3051 [08:31<15:33,  2.11it/s, est. speed input: 69317.85 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1082/3051 [08:31<15:31,  2.11it/s, est. speed input: 69317.89 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1083/3051 [08:31<15:31,  2.11it/s, est. speed input: 69317.84 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1084/3051 [08:32<15:31,  2.11it/s, est. speed input: 69317.51 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1085/3051 [08:32<15:31,  2.11it/s, est. speed input: 69317.29 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1086/3051 [08:33<15:31,  2.11it/s, est. speed input: 69317.20 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1087/3051 [08:33<15:31,  2.11it/s, est. speed input: 69316.83 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1088/3051 [08:34<15:29,  2.11it/s, est. speed input: 69316.90 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1089/3051 [08:34<15:28,  2.11it/s, est. speed input: 69316.97 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1090/3051 [08:35<15:28,  2.11it/s, est. speed input: 69316.74 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1091/3051 [08:35<15:27,  2.11it/s, est. speed input: 69316.68 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1092/3051 [08:36<15:26,  2.11it/s, est. speed input: 69316.71 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1093/3051 [08:36<15:26,  2.11it/s, est. speed input: 69316.57 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1094/3051 [08:37<15:26,  2.11it/s, est. speed input: 69316.54 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1095/3051 [08:37<15:25,  2.11it/s, est. speed input: 69316.61 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1096/3051 [08:38<15:24,  2.11it/s, est. speed input: 69316.60 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1097/3051 [08:38<15:24,  2.11it/s, est. speed input: 69316.44 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1098/3051 [08:39<15:24,  2.11it/s, est. speed input: 69316.29 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1099/3051 [08:39<15:23,  2.11it/s, est. speed input: 69316.33 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1100/3051 [08:40<15:23,  2.11it/s, est. speed input: 69316.13 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1101/3051 [08:40<15:22,  2.11it/s, est. speed input: 69316.13 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1102/3051 [08:40<15:22,  2.11it/s, est. speed input: 69316.12 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1103/3051 [08:41<15:21,  2.11it/s, est. speed input: 69316.06 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1104/3051 [08:41<15:20,  2.11it/s, est. speed input: 69316.10 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1105/3051 [08:42<15:20,  2.11it/s, est. speed input: 69316.10 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1106/3051 [08:42<15:19,  2.11it/s, est. speed input: 69316.10 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1107/3051 [08:43<15:18,  2.12it/s, est. speed input: 69316.19 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1108/3051 [08:43<15:18,  2.12it/s, est. speed input: 69316.25 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1109/3051 [08:44<15:19,  2.11it/s, est. speed input: 69315.92 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1110/3051 [08:44<15:19,  2.11it/s, est. speed input: 69315.70 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1111/3051 [08:45<15:18,  2.11it/s, est. speed input: 69315.63 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1112/3051 [08:45<15:17,  2.11it/s, est. speed input: 69315.61 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1113/3051 [08:46<15:17,  2.11it/s, est. speed input: 69315.45 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1114/3051 [08:46<15:16,  2.11it/s, est. speed input: 69315.46 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1115/3051 [08:47<15:16,  2.11it/s, est. speed input: 69315.31 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1116/3051 [08:47<15:17,  2.11it/s, est. speed input: 69314.94 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1117/3051 [08:48<15:15,  2.11it/s, est. speed input: 69315.01 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1118/3051 [08:48<15:35,  2.07it/s, est. speed input: 69310.28 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1119/3051 [08:49<15:28,  2.08it/s, est. speed input: 69310.31 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1120/3051 [08:49<15:23,  2.09it/s, est. speed input: 69310.37 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1121/3051 [08:49<15:19,  2.10it/s, est. speed input: 69310.36 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1122/3051 [08:50<15:17,  2.10it/s, est. speed input: 69310.30 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1123/3051 [08:50<15:15,  2.11it/s, est. speed input: 69310.28 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1124/3051 [08:51<15:13,  2.11it/s, est. speed input: 69310.24 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1125/3051 [08:51<15:12,  2.11it/s, est. speed input: 69310.27 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1126/3051 [08:52<15:11,  2.11it/s, est. speed input: 69310.22 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1127/3051 [08:52<15:10,  2.11it/s, est. speed input: 69310.28 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1128/3051 [08:53<15:11,  2.11it/s, est. speed input: 69309.89 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1129/3051 [08:53<15:10,  2.11it/s, est. speed input: 69309.82 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1130/3051 [08:54<15:09,  2.11it/s, est. speed input: 69309.77 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1131/3051 [08:54<15:09,  2.11it/s, est. speed input: 69309.51 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1132/3051 [08:55<15:09,  2.11it/s, est. speed input: 69309.40 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1133/3051 [08:55<15:08,  2.11it/s, est. speed input: 69309.45 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1134/3051 [08:56<15:07,  2.11it/s, est. speed input: 69309.47 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1135/3051 [08:56<15:07,  2.11it/s, est. speed input: 69309.30 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1136/3051 [08:57<15:06,  2.11it/s, est. speed input: 69309.17 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1137/3051 [08:57<15:05,  2.11it/s, est. speed input: 69309.18 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1138/3051 [08:58<15:06,  2.11it/s, est. speed input: 69308.84 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1139/3051 [08:58<15:05,  2.11it/s, est. speed input: 69308.86 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1140/3051 [08:58<15:04,  2.11it/s, est. speed input: 69308.82 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1141/3051 [08:59<15:04,  2.11it/s, est. speed input: 69308.59 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1142/3051 [08:59<15:04,  2.11it/s, est. speed input: 69308.49 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1143/3051 [09:00<15:03,  2.11it/s, est. speed input: 69308.42 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1144/3051 [09:00<15:02,  2.11it/s, est. speed input: 69308.39 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1145/3051 [09:01<15:01,  2.11it/s, est. speed input: 69308.42 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1146/3051 [09:01<15:01,  2.11it/s, est. speed input: 69308.37 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1147/3051 [09:02<15:03,  2.11it/s, est. speed input: 69307.68 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1148/3051 [09:02<15:02,  2.11it/s, est. speed input: 69307.64 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1149/3051 [09:03<15:00,  2.11it/s, est. speed input: 69307.75 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1150/3051 [09:03<15:01,  2.11it/s, est. speed input: 69307.46 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1151/3051 [09:04<15:00,  2.11it/s, est. speed input: 69307.36 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1152/3051 [09:04<14:59,  2.11it/s, est. speed input: 69307.34 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1153/3051 [09:05<14:57,  2.11it/s, est. speed input: 69307.46 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1154/3051 [09:05<14:57,  2.11it/s, est. speed input: 69307.48 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1155/3051 [09:06<14:56,  2.11it/s, est. speed input: 69307.43 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1156/3051 [09:06<14:56,  2.11it/s, est. speed input: 69307.46 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1157/3051 [09:07<14:55,  2.12it/s, est. speed input: 69307.50 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1158/3051 [09:07<14:54,  2.12it/s, est. speed input: 69307.52 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1159/3051 [09:07<14:54,  2.12it/s, est. speed input: 69307.58 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1160/3051 [09:08<14:54,  2.11it/s, est. speed input: 69307.38 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1161/3051 [09:08<14:54,  2.11it/s, est. speed input: 69307.33 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1162/3051 [09:09<14:54,  2.11it/s, est. speed input: 69307.20 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1163/3051 [09:09<14:53,  2.11it/s, est. speed input: 69307.18 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1164/3051 [09:10<14:52,  2.11it/s, est. speed input: 69307.18 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1165/3051 [09:10<14:51,  2.11it/s, est. speed input: 69307.22 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1166/3051 [09:11<14:51,  2.11it/s, est. speed input: 69307.18 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1167/3051 [09:11<14:50,  2.11it/s, est. speed input: 69307.17 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1168/3051 [09:12<14:51,  2.11it/s, est. speed input: 69307.02 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1169/3051 [09:12<14:51,  2.11it/s, est. speed input: 69306.85 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1170/3051 [09:13<14:50,  2.11it/s, est. speed input: 69306.81 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1171/3051 [09:13<14:49,  2.11it/s, est. speed input: 69306.79 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1172/3051 [09:14<14:48,  2.11it/s, est. speed input: 69306.82 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1173/3051 [09:14<14:52,  2.11it/s, est. speed input: 69305.95 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1174/3051 [09:15<14:50,  2.11it/s, est. speed input: 69305.98 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1175/3051 [09:15<14:48,  2.11it/s, est. speed input: 69306.02 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1176/3051 [09:16<14:48,  2.11it/s, est. speed input: 69305.80 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1177/3051 [09:16<14:48,  2.11it/s, est. speed input: 69305.58 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1178/3051 [09:16<14:47,  2.11it/s, est. speed input: 69305.54 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1179/3051 [09:17<14:47,  2.11it/s, est. speed input: 69305.33 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1180/3051 [09:17<14:46,  2.11it/s, est. speed input: 69305.28 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1181/3051 [09:18<14:45,  2.11it/s, est. speed input: 69305.32 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1182/3051 [09:18<14:45,  2.11it/s, est. speed input: 69305.04 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1183/3051 [09:19<14:44,  2.11it/s, est. speed input: 69305.06 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1184/3051 [09:19<14:43,  2.11it/s, est. speed input: 69305.08 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1185/3051 [09:20<14:42,  2.11it/s, est. speed input: 69305.07 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1186/3051 [09:20<14:42,  2.11it/s, est. speed input: 69305.04 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1187/3051 [09:21<14:41,  2.11it/s, est. speed input: 69305.11 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1188/3051 [09:21<14:41,  2.11it/s, est. speed input: 69305.06 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1189/3051 [09:22<14:40,  2.11it/s, est. speed input: 69304.97 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1190/3051 [09:22<14:42,  2.11it/s, est. speed input: 69304.59 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1191/3051 [09:23<14:40,  2.11it/s, est. speed input: 69304.58 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1192/3051 [09:23<14:40,  2.11it/s, est. speed input: 69304.54 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1193/3051 [09:24<14:39,  2.11it/s, est. speed input: 69304.53 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1194/3051 [09:24<14:38,  2.11it/s, est. speed input: 69304.48 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1195/3051 [09:25<14:38,  2.11it/s, est. speed input: 69304.44 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1196/3051 [09:25<14:39,  2.11it/s, est. speed input: 69303.97 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1197/3051 [09:25<14:38,  2.11it/s, est. speed input: 69303.97 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1198/3051 [09:26<14:38,  2.11it/s, est. speed input: 69303.75 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1199/3051 [09:26<14:37,  2.11it/s, est. speed input: 69303.64 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1200/3051 [09:27<14:37,  2.11it/s, est. speed input: 69303.55 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1201/3051 [09:27<14:37,  2.11it/s, est. speed input: 69303.23 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1202/3051 [09:28<14:37,  2.11it/s, est. speed input: 69303.03 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1203/3051 [09:28<14:35,  2.11it/s, est. speed input: 69303.02 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1204/3051 [09:29<14:34,  2.11it/s, est. speed input: 69303.06 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1205/3051 [09:29<14:33,  2.11it/s, est. speed input: 69303.10 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1206/3051 [09:30<14:33,  2.11it/s, est. speed input: 69302.87 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1207/3051 [09:30<14:33,  2.11it/s, est. speed input: 69302.84 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1208/3051 [09:31<14:32,  2.11it/s, est. speed input: 69302.82 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1209/3051 [09:31<14:31,  2.11it/s, est. speed input: 69302.80 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1210/3051 [09:32<14:30,  2.11it/s, est. speed input: 69302.87 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1211/3051 [09:32<14:30,  2.11it/s, est. speed input: 69302.85 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1212/3051 [09:33<14:32,  2.11it/s, est. speed input: 69302.28 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1213/3051 [09:33<14:30,  2.11it/s, est. speed input: 69302.29 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1214/3051 [09:34<14:29,  2.11it/s, est. speed input: 69302.33 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1215/3051 [09:34<14:28,  2.11it/s, est. speed input: 69302.35 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1216/3051 [09:34<14:28,  2.11it/s, est. speed input: 69302.37 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1217/3051 [09:35<14:27,  2.11it/s, est. speed input: 69302.43 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1218/3051 [09:35<14:26,  2.11it/s, est. speed input: 69302.40 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1219/3051 [09:36<14:26,  2.12it/s, est. speed input: 69302.43 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1220/3051 [09:36<14:25,  2.12it/s, est. speed input: 69302.44 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1221/3051 [09:37<14:25,  2.11it/s, est. speed input: 69302.40 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1222/3051 [09:37<14:27,  2.11it/s, est. speed input: 69301.89 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1223/3051 [09:38<14:25,  2.11it/s, est. speed input: 69301.99 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1224/3051 [09:38<14:25,  2.11it/s, est. speed input: 69301.89 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1225/3051 [09:39<14:25,  2.11it/s, est. speed input: 69301.65 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1226/3051 [09:39<14:24,  2.11it/s, est. speed input: 69301.60 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1227/3051 [09:40<14:23,  2.11it/s, est. speed input: 69301.59 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1228/3051 [09:40<14:24,  2.11it/s, est. speed input: 69301.21 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1229/3051 [09:41<14:23,  2.11it/s, est. speed input: 69301.18 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1230/3051 [09:41<14:22,  2.11it/s, est. speed input: 69301.21 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1231/3051 [09:42<14:22,  2.11it/s, est. speed input: 69300.99 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1232/3051 [09:42<14:21,  2.11it/s, est. speed input: 69300.91 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1233/3051 [09:43<14:21,  2.11it/s, est. speed input: 69300.85 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1234/3051 [09:43<14:22,  2.11it/s, est. speed input: 69300.42 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1235/3051 [09:43<14:20,  2.11it/s, est. speed input: 69300.40 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1236/3051 [09:44<14:19,  2.11it/s, est. speed input: 69300.38 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1237/3051 [09:44<14:19,  2.11it/s, est. speed input: 69300.14 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1238/3051 [09:45<14:18,  2.11it/s, est. speed input: 69300.11 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1239/3051 [09:45<14:17,  2.11it/s, est. speed input: 69300.11 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1240/3051 [09:46<14:18,  2.11it/s, est. speed input: 69299.83 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1241/3051 [09:46<14:17,  2.11it/s, est. speed input: 69299.87 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1242/3051 [09:47<14:15,  2.11it/s, est. speed input: 69299.94 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1243/3051 [09:47<14:15,  2.11it/s, est. speed input: 69299.94 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1244/3051 [09:48<14:15,  2.11it/s, est. speed input: 69299.77 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1245/3051 [09:48<14:14,  2.11it/s, est. speed input: 69299.76 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1246/3051 [09:49<14:13,  2.11it/s, est. speed input: 69299.84 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1247/3051 [09:49<14:14,  2.11it/s, est. speed input: 69299.60 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1248/3051 [09:50<14:13,  2.11it/s, est. speed input: 69299.55 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1249/3051 [09:50<14:12,  2.11it/s, est. speed input: 69299.56 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1250/3051 [09:51<14:12,  2.11it/s, est. speed input: 69299.56 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1251/3051 [09:51<14:11,  2.11it/s, est. speed input: 69299.50 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1252/3051 [09:52<14:10,  2.11it/s, est. speed input: 69299.52 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1253/3051 [09:52<14:10,  2.11it/s, est. speed input: 69299.50 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1254/3051 [09:52<14:09,  2.11it/s, est. speed input: 69299.53 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1255/3051 [09:53<14:09,  2.12it/s, est. speed input: 69299.58 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1256/3051 [09:53<14:08,  2.12it/s, est. speed input: 69299.62 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1257/3051 [09:54<14:08,  2.11it/s, est. speed input: 69299.51 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1258/3051 [09:54<14:07,  2.11it/s, est. speed input: 69299.51 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1259/3051 [09:55<14:07,  2.11it/s, est. speed input: 69299.49 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1260/3051 [09:55<14:08,  2.11it/s, est. speed input: 69299.20 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1261/3051 [09:56<14:07,  2.11it/s, est. speed input: 69299.19 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1262/3051 [09:56<14:06,  2.11it/s, est. speed input: 69299.18 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1263/3051 [09:57<14:06,  2.11it/s, est. speed input: 69299.16 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1264/3051 [09:57<14:05,  2.11it/s, est. speed input: 69299.20 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1265/3051 [09:58<14:04,  2.11it/s, est. speed input: 69299.15 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1266/3051 [09:58<14:05,  2.11it/s, est. speed input: 69298.91 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1267/3051 [09:59<14:05,  2.11it/s, est. speed input: 69298.79 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1268/3051 [09:59<14:04,  2.11it/s, est. speed input: 69298.80 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1269/3051 [10:00<14:04,  2.11it/s, est. speed input: 69298.50 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1270/3051 [10:00<14:04,  2.11it/s, est. speed input: 69298.37 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1271/3051 [10:00<14:03,  2.11it/s, est. speed input: 69298.28 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1272/3051 [10:01<14:02,  2.11it/s, est. speed input: 69298.23 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1273/3051 [10:01<14:02,  2.11it/s, est. speed input: 69298.15 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1274/3051 [10:02<14:01,  2.11it/s, est. speed input: 69298.09 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1275/3051 [10:02<14:02,  2.11it/s, est. speed input: 69297.76 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1276/3051 [10:03<14:01,  2.11it/s, est. speed input: 69297.58 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1277/3051 [10:03<14:00,  2.11it/s, est. speed input: 69297.54 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1278/3051 [10:04<13:59,  2.11it/s, est. speed input: 69297.51 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1279/3051 [10:04<13:59,  2.11it/s, est. speed input: 69297.40 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1280/3051 [10:05<13:58,  2.11it/s, est. speed input: 69297.34 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1281/3051 [10:05<13:57,  2.11it/s, est. speed input: 69297.35 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1282/3051 [10:06<13:57,  2.11it/s, est. speed input: 69297.34 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1283/3051 [10:06<13:56,  2.11it/s, est. speed input: 69297.31 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1284/3051 [10:07<13:55,  2.11it/s, est. speed input: 69297.36 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1285/3051 [10:07<13:56,  2.11it/s, est. speed input: 69297.06 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1286/3051 [10:08<13:55,  2.11it/s, est. speed input: 69297.04 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1287/3051 [10:08<13:55,  2.11it/s, est. speed input: 69297.02 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1288/3051 [10:09<13:55,  2.11it/s, est. speed input: 69296.73 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1289/3051 [10:09<13:54,  2.11it/s, est. speed input: 69296.73 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1290/3051 [10:09<13:53,  2.11it/s, est. speed input: 69296.78 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1291/3051 [10:10<13:54,  2.11it/s, est. speed input: 69296.35 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1292/3051 [10:10<13:54,  2.11it/s, est. speed input: 69296.19 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1293/3051 [10:11<13:53,  2.11it/s, est. speed input: 69296.12 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1294/3051 [10:11<13:53,  2.11it/s, est. speed input: 69295.77 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1295/3051 [10:12<13:53,  2.11it/s, est. speed input: 69295.61 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1296/3051 [10:12<13:51,  2.11it/s, est. speed input: 69295.61 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1297/3051 [10:13<13:50,  2.11it/s, est. speed input: 69295.63 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1298/3051 [10:13<13:50,  2.11it/s, est. speed input: 69295.53 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1299/3051 [10:14<13:50,  2.11it/s, est. speed input: 69295.37 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1300/3051 [10:14<13:49,  2.11it/s, est. speed input: 69295.36 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1301/3051 [10:15<13:48,  2.11it/s, est. speed input: 69295.38 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1302/3051 [10:15<13:47,  2.11it/s, est. speed input: 69295.34 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1303/3051 [10:16<13:46,  2.11it/s, est. speed input: 69295.41 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1304/3051 [10:16<13:46,  2.11it/s, est. speed input: 69295.25 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1305/3051 [10:17<13:46,  2.11it/s, est. speed input: 69295.24 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1306/3051 [10:17<13:45,  2.11it/s, est. speed input: 69295.23 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1307/3051 [10:18<13:46,  2.11it/s, est. speed input: 69294.94 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1308/3051 [10:18<13:45,  2.11it/s, est. speed input: 69294.84 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1309/3051 [10:18<13:44,  2.11it/s, est. speed input: 69294.84 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1310/3051 [10:19<13:45,  2.11it/s, est. speed input: 69294.62 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1311/3051 [10:19<13:44,  2.11it/s, est. speed input: 69294.62 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1312/3051 [10:20<13:43,  2.11it/s, est. speed input: 69294.63 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1313/3051 [10:20<13:42,  2.11it/s, est. speed input: 69294.63 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1314/3051 [10:21<13:41,  2.11it/s, est. speed input: 69294.62 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1315/3051 [10:21<13:41,  2.11it/s, est. speed input: 69294.64 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1316/3051 [10:22<13:40,  2.12it/s, est. speed input: 69294.70 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1317/3051 [10:22<13:41,  2.11it/s, est. speed input: 69294.45 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1318/3051 [10:23<13:41,  2.11it/s, est. speed input: 69294.28 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1319/3051 [10:23<13:40,  2.11it/s, est. speed input: 69294.26 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1320/3051 [10:24<13:41,  2.11it/s, est. speed input: 69293.77 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1321/3051 [10:24<13:41,  2.11it/s, est. speed input: 69293.53 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1322/3051 [10:25<13:40,  2.11it/s, est. speed input: 69293.49 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1323/3051 [10:25<13:39,  2.11it/s, est. speed input: 69293.42 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1324/3051 [10:26<13:39,  2.11it/s, est. speed input: 69293.21 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1325/3051 [10:26<13:37,  2.11it/s, est. speed input: 69293.21 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1326/3051 [10:27<13:36,  2.11it/s, est. speed input: 69293.29 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1327/3051 [10:27<13:36,  2.11it/s, est. speed input: 69293.15 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1328/3051 [10:27<13:35,  2.11it/s, est. speed input: 69293.20 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1329/3051 [10:28<13:34,  2.11it/s, est. speed input: 69293.15 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1330/3051 [10:28<13:34,  2.11it/s, est. speed input: 69293.16 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1331/3051 [10:29<13:33,  2.11it/s, est. speed input: 69293.17 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1332/3051 [10:29<13:32,  2.11it/s, est. speed input: 69293.22 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1333/3051 [10:30<13:32,  2.11it/s, est. speed input: 69293.16 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1334/3051 [10:30<13:33,  2.11it/s, est. speed input: 69292.95 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1335/3051 [10:31<13:32,  2.11it/s, est. speed input: 69292.96 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1336/3051 [10:31<13:31,  2.11it/s, est. speed input: 69292.94 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1337/3051 [10:32<13:31,  2.11it/s, est. speed input: 69292.82 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1338/3051 [10:32<13:30,  2.11it/s, est. speed input: 69292.83 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1339/3051 [10:33<13:30,  2.11it/s, est. speed input: 69292.82 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1340/3051 [10:33<13:32,  2.11it/s, est. speed input: 69292.27 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1341/3051 [10:34<13:30,  2.11it/s, est. speed input: 69292.27 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1342/3051 [10:34<13:29,  2.11it/s, est. speed input: 69292.27 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1343/3051 [10:35<13:28,  2.11it/s, est. speed input: 69292.23 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1344/3051 [10:35<13:28,  2.11it/s, est. speed input: 69292.18 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1345/3051 [10:36<13:50,  2.05it/s, est. speed input: 69287.29 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1346/3051 [10:36<13:42,  2.07it/s, est. speed input: 69287.33 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1347/3051 [10:37<13:36,  2.09it/s, est. speed input: 69287.44 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1348/3051 [10:37<13:33,  2.09it/s, est. speed input: 69287.39 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1349/3051 [10:37<13:30,  2.10it/s, est. speed input: 69287.31 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1350/3051 [10:38<13:28,  2.10it/s, est. speed input: 69287.33 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1351/3051 [10:38<13:27,  2.10it/s, est. speed input: 69287.13 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1352/3051 [10:39<13:26,  2.11it/s, est. speed input: 69287.09 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1353/3051 [10:39<13:24,  2.11it/s, est. speed input: 69287.10 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1354/3051 [10:40<13:23,  2.11it/s, est. speed input: 69287.15 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1355/3051 [10:40<13:22,  2.11it/s, est. speed input: 69287.22 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1356/3051 [10:41<13:22,  2.11it/s, est. speed input: 69287.19 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1357/3051 [10:41<13:21,  2.11it/s, est. speed input: 69287.25 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1358/3051 [10:42<13:21,  2.11it/s, est. speed input: 69287.14 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1359/3051 [10:42<13:20,  2.11it/s, est. speed input: 69287.08 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1360/3051 [10:43<13:20,  2.11it/s, est. speed input: 69287.01 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1361/3051 [10:43<13:19,  2.11it/s, est. speed input: 69287.03 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1362/3051 [10:44<13:19,  2.11it/s, est. speed input: 69286.96 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1363/3051 [10:44<13:18,  2.11it/s, est. speed input: 69287.01 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1364/3051 [10:45<13:18,  2.11it/s, est. speed input: 69286.96 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1365/3051 [10:45<13:17,  2.11it/s, est. speed input: 69286.99 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1366/3051 [10:46<13:17,  2.11it/s, est. speed input: 69286.95 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1367/3051 [10:46<13:16,  2.11it/s, est. speed input: 69286.99 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1368/3051 [10:46<13:15,  2.11it/s, est. speed input: 69287.00 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1369/3051 [10:47<13:16,  2.11it/s, est. speed input: 69286.70 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1370/3051 [10:47<13:16,  2.11it/s, est. speed input: 69286.66 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1371/3051 [10:48<13:15,  2.11it/s, est. speed input: 69286.65 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1372/3051 [10:48<13:14,  2.11it/s, est. speed input: 69286.63 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1373/3051 [10:49<13:13,  2.11it/s, est. speed input: 69286.66 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1374/3051 [10:49<13:13,  2.11it/s, est. speed input: 69286.71 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1375/3051 [10:50<13:12,  2.11it/s, est. speed input: 69286.75 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1376/3051 [10:50<13:11,  2.12it/s, est. speed input: 69286.77 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1377/3051 [10:51<13:12,  2.11it/s, est. speed input: 69286.49 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1378/3051 [10:51<13:11,  2.11it/s, est. speed input: 69286.52 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1379/3051 [10:52<13:11,  2.11it/s, est. speed input: 69286.53 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1380/3051 [10:52<13:11,  2.11it/s, est. speed input: 69286.43 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1381/3051 [10:53<13:10,  2.11it/s, est. speed input: 69286.32 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1382/3051 [10:53<13:09,  2.11it/s, est. speed input: 69286.34 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1383/3051 [10:54<13:09,  2.11it/s, est. speed input: 69286.38 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1384/3051 [10:54<13:08,  2.11it/s, est. speed input: 69286.44 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1385/3051 [10:55<13:07,  2.11it/s, est. speed input: 69286.43 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1386/3051 [10:55<13:07,  2.12it/s, est. speed input: 69286.49 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1387/3051 [10:55<13:07,  2.11it/s, est. speed input: 69286.35 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1388/3051 [10:56<13:06,  2.11it/s, est. speed input: 69286.32 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1389/3051 [10:56<13:06,  2.11it/s, est. speed input: 69286.37 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1390/3051 [10:57<13:05,  2.11it/s, est. speed input: 69286.41 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1391/3051 [10:57<13:04,  2.11it/s, est. speed input: 69286.41 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1392/3051 [10:58<13:04,  2.11it/s, est. speed input: 69286.42 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1393/3051 [10:58<13:03,  2.11it/s, est. speed input: 69286.44 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1394/3051 [10:59<13:03,  2.11it/s, est. speed input: 69286.35 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1395/3051 [10:59<13:03,  2.11it/s, est. speed input: 69286.33 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1396/3051 [11:00<13:04,  2.11it/s, est. speed input: 69286.08 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1397/3051 [11:00<13:03,  2.11it/s, est. speed input: 69286.12 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1398/3051 [11:01<13:02,  2.11it/s, est. speed input: 69286.15 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1399/3051 [11:01<13:01,  2.11it/s, est. speed input: 69286.14 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1400/3051 [11:02<13:01,  2.11it/s, est. speed input: 69285.95 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1401/3051 [11:02<13:01,  2.11it/s, est. speed input: 69285.90 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1402/3051 [11:03<13:00,  2.11it/s, est. speed input: 69285.91 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1403/3051 [11:03<13:01,  2.11it/s, est. speed input: 69285.56 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1404/3051 [11:04<13:00,  2.11it/s, est. speed input: 69285.47 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1405/3051 [11:04<13:00,  2.11it/s, est. speed input: 69285.41 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1406/3051 [11:04<12:59,  2.11it/s, est. speed input: 69285.40 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1407/3051 [11:05<13:00,  2.11it/s, est. speed input: 69284.94 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1408/3051 [11:05<12:59,  2.11it/s, est. speed input: 69284.92 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1409/3051 [11:06<12:58,  2.11it/s, est. speed input: 69284.92 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1410/3051 [11:06<12:56,  2.11it/s, est. speed input: 69284.96 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1411/3051 [11:07<12:56,  2.11it/s, est. speed input: 69284.91 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1412/3051 [11:07<12:55,  2.11it/s, est. speed input: 69284.98 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1413/3051 [11:08<12:54,  2.11it/s, est. speed input: 69285.04 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1414/3051 [11:08<12:55,  2.11it/s, est. speed input: 69284.82 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1415/3051 [11:09<12:54,  2.11it/s, est. speed input: 69284.72 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1416/3051 [11:09<12:54,  2.11it/s, est. speed input: 69284.73 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1417/3051 [11:10<12:53,  2.11it/s, est. speed input: 69284.74 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1418/3051 [11:10<12:52,  2.11it/s, est. speed input: 69284.79 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1419/3051 [11:11<12:52,  2.11it/s, est. speed input: 69284.59 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1420/3051 [11:11<12:52,  2.11it/s, est. speed input: 69284.57 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1421/3051 [11:12<12:51,  2.11it/s, est. speed input: 69284.61 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1422/3051 [11:12<12:52,  2.11it/s, est. speed input: 69284.27 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1423/3051 [11:13<12:51,  2.11it/s, est. speed input: 69284.13 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1424/3051 [11:13<12:50,  2.11it/s, est. speed input: 69284.20 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1425/3051 [11:13<12:49,  2.11it/s, est. speed input: 69284.18 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1426/3051 [11:14<12:48,  2.11it/s, est. speed input: 69284.21 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1427/3051 [11:14<12:48,  2.11it/s, est. speed input: 69284.11 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1428/3051 [11:15<12:48,  2.11it/s, est. speed input: 69283.97 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1429/3051 [11:15<12:48,  2.11it/s, est. speed input: 69283.96 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1430/3051 [11:16<12:47,  2.11it/s, est. speed input: 69283.97 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1431/3051 [11:16<12:46,  2.11it/s, est. speed input: 69283.98 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1432/3051 [11:17<12:45,  2.11it/s, est. speed input: 69283.99 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1433/3051 [11:17<12:46,  2.11it/s, est. speed input: 69283.86 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1434/3051 [11:18<12:44,  2.11it/s, est. speed input: 69283.95 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1435/3051 [11:18<12:44,  2.11it/s, est. speed input: 69283.86 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1436/3051 [11:19<12:45,  2.11it/s, est. speed input: 69283.68 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1437/3051 [11:19<12:44,  2.11it/s, est. speed input: 69283.67 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1438/3051 [11:20<12:46,  2.11it/s, est. speed input: 69283.13 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1439/3051 [11:20<12:44,  2.11it/s, est. speed input: 69283.15 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1440/3051 [11:21<12:43,  2.11it/s, est. speed input: 69283.18 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1441/3051 [11:21<12:42,  2.11it/s, est. speed input: 69283.09 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1442/3051 [11:22<12:42,  2.11it/s, est. speed input: 69283.05 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1443/3051 [11:22<12:41,  2.11it/s, est. speed input: 69283.05 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1444/3051 [11:22<12:40,  2.11it/s, est. speed input: 69283.15 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1445/3051 [11:23<12:39,  2.11it/s, est. speed input: 69283.17 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1446/3051 [11:23<12:39,  2.11it/s, est. speed input: 69283.17 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1447/3051 [11:24<12:38,  2.11it/s, est. speed input: 69283.21 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1448/3051 [11:24<12:38,  2.11it/s, est. speed input: 69283.20 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1449/3051 [11:25<12:38,  2.11it/s, est. speed input: 69283.07 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1450/3051 [11:25<12:37,  2.11it/s, est. speed input: 69283.09 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1451/3051 [11:26<12:37,  2.11it/s, est. speed input: 69283.07 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1452/3051 [11:26<12:36,  2.11it/s, est. speed input: 69282.97 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1453/3051 [11:27<12:35,  2.11it/s, est. speed input: 69283.06 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1454/3051 [11:27<12:37,  2.11it/s, est. speed input: 69282.65 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1455/3051 [11:28<12:37,  2.11it/s, est. speed input: 69282.46 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1456/3051 [11:28<12:35,  2.11it/s, est. speed input: 69282.55 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1457/3051 [11:29<12:34,  2.11it/s, est. speed input: 69282.57 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1458/3051 [11:29<12:35,  2.11it/s, est. speed input: 69282.34 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1459/3051 [11:30<12:34,  2.11it/s, est. speed input: 69282.31 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1460/3051 [11:30<12:36,  2.10it/s, est. speed input: 69281.67 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1461/3051 [11:31<12:34,  2.11it/s, est. speed input: 69281.71 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1462/3051 [11:31<12:33,  2.11it/s, est. speed input: 69281.76 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1463/3051 [11:31<12:31,  2.11it/s, est. speed input: 69281.83 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1464/3051 [11:32<12:31,  2.11it/s, est. speed input: 69281.70 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1465/3051 [11:32<12:31,  2.11it/s, est. speed input: 69281.67 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1466/3051 [11:33<12:30,  2.11it/s, est. speed input: 69281.68 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1467/3051 [11:33<12:30,  2.11it/s, est. speed input: 69281.59 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1468/3051 [11:34<12:29,  2.11it/s, est. speed input: 69281.62 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1469/3051 [11:34<12:28,  2.11it/s, est. speed input: 69281.65 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1470/3051 [11:35<12:27,  2.11it/s, est. speed input: 69281.63 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1471/3051 [11:35<12:27,  2.11it/s, est. speed input: 69281.56 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1472/3051 [11:36<12:26,  2.11it/s, est. speed input: 69281.63 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1473/3051 [11:36<12:27,  2.11it/s, est. speed input: 69281.49 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1474/3051 [11:37<12:26,  2.11it/s, est. speed input: 69281.40 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1475/3051 [11:37<12:26,  2.11it/s, est. speed input: 69281.38 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1476/3051 [11:38<12:25,  2.11it/s, est. speed input: 69281.46 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1477/3051 [11:38<12:25,  2.11it/s, est. speed input: 69281.21 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1478/3051 [11:39<12:24,  2.11it/s, est. speed input: 69281.25 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1479/3051 [11:39<12:23,  2.11it/s, est. speed input: 69281.26 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1480/3051 [11:39<12:22,  2.11it/s, est. speed input: 69281.34 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1481/3051 [11:40<12:21,  2.12it/s, est. speed input: 69281.45 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1482/3051 [11:40<12:21,  2.12it/s, est. speed input: 69281.46 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1483/3051 [11:41<12:21,  2.11it/s, est. speed input: 69281.33 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1484/3051 [11:41<12:21,  2.11it/s, est. speed input: 69281.27 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1485/3051 [11:42<12:20,  2.11it/s, est. speed input: 69281.30 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1486/3051 [11:42<12:20,  2.11it/s, est. speed input: 69281.35 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1487/3051 [11:43<12:20,  2.11it/s, est. speed input: 69281.24 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1488/3051 [11:43<12:19,  2.11it/s, est. speed input: 69281.29 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1489/3051 [11:44<12:18,  2.11it/s, est. speed input: 69281.35 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1490/3051 [11:44<12:18,  2.12it/s, est. speed input: 69281.38 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1491/3051 [11:45<12:17,  2.11it/s, est. speed input: 69281.35 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1492/3051 [11:45<12:17,  2.11it/s, est. speed input: 69281.30 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1493/3051 [11:46<12:16,  2.11it/s, est. speed input: 69281.35 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1494/3051 [11:46<12:16,  2.11it/s, est. speed input: 69281.37 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1495/3051 [11:47<12:15,  2.12it/s, est. speed input: 69281.40 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1496/3051 [11:47<12:16,  2.11it/s, est. speed input: 69281.07 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1497/3051 [11:48<12:16,  2.11it/s, est. speed input: 69281.03 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1498/3051 [11:48<12:15,  2.11it/s, est. speed input: 69281.09 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1499/3051 [11:48<12:14,  2.11it/s, est. speed input: 69281.10 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1500/3051 [11:49<12:13,  2.11it/s, est. speed input: 69281.10 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1501/3051 [11:49<12:13,  2.11it/s, est. speed input: 69281.14 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1502/3051 [11:50<12:13,  2.11it/s, est. speed input: 69280.93 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1503/3051 [11:50<12:12,  2.11it/s, est. speed input: 69280.98 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1504/3051 [11:51<12:12,  2.11it/s, est. speed input: 69280.95 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1505/3051 [11:51<12:48,  2.01it/s, est. speed input: 69273.15 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1506/3051 [11:52<12:36,  2.04it/s, est. speed input: 69273.26 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1507/3051 [11:52<12:27,  2.07it/s, est. speed input: 69273.40 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1508/3051 [11:53<12:21,  2.08it/s, est. speed input: 69273.57 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1509/3051 [11:53<12:16,  2.09it/s, est. speed input: 69273.78 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1510/3051 [11:54<12:12,  2.10it/s, est. speed input: 69273.95 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1511/3051 [11:54<12:10,  2.11it/s, est. speed input: 69274.04 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1512/3051 [11:55<12:10,  2.11it/s, est. speed input: 69273.89 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1513/3051 [11:55<12:08,  2.11it/s, est. speed input: 69274.01 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1514/3051 [11:56<12:09,  2.11it/s, est. speed input: 69273.60 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1515/3051 [11:56<12:08,  2.11it/s, est. speed input: 69273.56 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1516/3051 [11:57<12:06,  2.11it/s, est. speed input: 69273.73 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1517/3051 [11:57<12:05,  2.11it/s, est. speed input: 69273.80 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1518/3051 [11:58<12:05,  2.11it/s, est. speed input: 69273.81 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1519/3051 [11:58<12:04,  2.11it/s, est. speed input: 69273.81 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1520/3051 [11:59<12:21,  2.07it/s, est. speed input: 69270.27 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1521/3051 [11:59<12:14,  2.08it/s, est. speed input: 69270.50 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1522/3051 [11:59<12:09,  2.10it/s, est. speed input: 69270.71 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1523/3051 [12:00<12:07,  2.10it/s, est. speed input: 69270.71 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1524/3051 [12:00<12:07,  2.10it/s, est. speed input: 69270.24 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1525/3051 [12:01<12:05,  2.10it/s, est. speed input: 69270.25 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1526/3051 [12:01<12:03,  2.11it/s, est. speed input: 69270.37 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1527/3051 [12:02<12:01,  2.11it/s, est. speed input: 69270.46 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1528/3051 [12:02<12:00,  2.11it/s, est. speed input: 69270.66 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1529/3051 [12:03<11:59,  2.12it/s, est. speed input: 69270.81 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1530/3051 [12:03<11:59,  2.11it/s, est. speed input: 69270.67 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1531/3051 [12:04<11:59,  2.11it/s, est. speed input: 69270.64 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1532/3051 [12:04<11:58,  2.11it/s, est. speed input: 69270.70 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1533/3051 [12:05<11:57,  2.12it/s, est. speed input: 69270.89 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1534/3051 [12:05<11:56,  2.12it/s, est. speed input: 69270.93 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1535/3051 [12:06<11:56,  2.12it/s, est. speed input: 69270.89 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1536/3051 [12:06<11:57,  2.11it/s, est. speed input: 69270.67 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1537/3051 [12:07<11:57,  2.11it/s, est. speed input: 69270.59 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1538/3051 [12:07<11:55,  2.11it/s, est. speed input: 69270.69 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1539/3051 [12:08<11:54,  2.12it/s, est. speed input: 69270.83 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1540/3051 [12:08<11:53,  2.12it/s, est. speed input: 69270.98 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1541/3051 [12:08<11:53,  2.12it/s, est. speed input: 69270.97 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1542/3051 [12:09<11:53,  2.12it/s, est. speed input: 69270.97 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1543/3051 [12:09<11:53,  2.11it/s, est. speed input: 69270.94 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1544/3051 [12:10<11:52,  2.11it/s, est. speed input: 69270.88 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1545/3051 [12:10<11:51,  2.12it/s, est. speed input: 69271.06 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1546/3051 [12:11<11:53,  2.11it/s, est. speed input: 69270.57 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1547/3051 [12:11<11:52,  2.11it/s, est. speed input: 69270.54 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1548/3051 [12:12<11:51,  2.11it/s, est. speed input: 69270.68 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1549/3051 [12:12<11:50,  2.11it/s, est. speed input: 69270.78 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1550/3051 [12:13<11:49,  2.11it/s, est. speed input: 69270.79 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1551/3051 [12:13<11:48,  2.12it/s, est. speed input: 69270.90 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1552/3051 [12:14<11:48,  2.12it/s, est. speed input: 69270.88 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1553/3051 [12:14<11:47,  2.12it/s, est. speed input: 69270.98 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1554/3051 [12:15<11:47,  2.12it/s, est. speed input: 69271.02 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1555/3051 [12:15<11:46,  2.12it/s, est. speed input: 69271.16 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1556/3051 [12:16<11:46,  2.12it/s, est. speed input: 69271.17 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1557/3051 [12:16<11:45,  2.12it/s, est. speed input: 69271.22 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1558/3051 [12:16<11:45,  2.12it/s, est. speed input: 69271.28 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1559/3051 [12:17<11:46,  2.11it/s, est. speed input: 69271.03 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1560/3051 [12:17<11:45,  2.11it/s, est. speed input: 69271.05 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1561/3051 [12:18<11:45,  2.11it/s, est. speed input: 69271.03 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1562/3051 [12:18<11:45,  2.11it/s, est. speed input: 69270.78 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1563/3051 [12:19<11:44,  2.11it/s, est. speed input: 69270.83 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1564/3051 [12:19<11:43,  2.11it/s, est. speed input: 69270.90 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1565/3051 [12:20<11:43,  2.11it/s, est. speed input: 69270.79 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1566/3051 [12:20<11:42,  2.11it/s, est. speed input: 69270.84 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1567/3051 [12:21<11:41,  2.12it/s, est. speed input: 69271.00 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1568/3051 [12:21<11:40,  2.12it/s, est. speed input: 69271.06 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1569/3051 [12:22<11:41,  2.11it/s, est. speed input: 69270.84 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1570/3051 [12:22<11:40,  2.11it/s, est. speed input: 69270.91 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1571/3051 [12:23<11:39,  2.11it/s, est. speed input: 69270.98 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1572/3051 [12:23<11:42,  2.11it/s, est. speed input: 69270.40 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1573/3051 [12:24<11:41,  2.11it/s, est. speed input: 69270.39 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1574/3051 [12:24<11:39,  2.11it/s, est. speed input: 69270.51 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1575/3051 [12:25<11:38,  2.11it/s, est. speed input: 69270.48 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1576/3051 [12:25<11:37,  2.11it/s, est. speed input: 69270.53 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1577/3051 [12:25<11:36,  2.12it/s, est. speed input: 69270.68 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1578/3051 [12:26<11:37,  2.11it/s, est. speed input: 69270.48 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1579/3051 [12:26<11:36,  2.11it/s, est. speed input: 69270.52 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1580/3051 [12:27<11:36,  2.11it/s, est. speed input: 69270.47 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1581/3051 [12:27<11:35,  2.11it/s, est. speed input: 69270.53 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1582/3051 [12:28<11:36,  2.11it/s, est. speed input: 69270.20 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1583/3051 [12:28<11:34,  2.11it/s, est. speed input: 69270.33 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1584/3051 [12:29<11:33,  2.12it/s, est. speed input: 69270.49 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1585/3051 [12:29<11:33,  2.11it/s, est. speed input: 69270.32 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1586/3051 [12:30<11:34,  2.11it/s, est. speed input: 69270.13 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1587/3051 [12:30<11:33,  2.11it/s, est. speed input: 69270.19 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1588/3051 [12:31<11:33,  2.11it/s, est. speed input: 69269.99 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1589/3051 [12:31<11:31,  2.11it/s, est. speed input: 69270.15 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1590/3051 [12:32<11:31,  2.11it/s, est. speed input: 69270.07 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1591/3051 [12:32<11:31,  2.11it/s, est. speed input: 69269.89 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1592/3051 [12:33<11:30,  2.11it/s, est. speed input: 69269.93 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1593/3051 [12:33<11:29,  2.11it/s, est. speed input: 69270.10 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1594/3051 [12:34<11:28,  2.11it/s, est. speed input: 69270.12 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1595/3051 [12:34<11:29,  2.11it/s, est. speed input: 69269.91 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1596/3051 [12:34<11:28,  2.11it/s, est. speed input: 69269.98 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1597/3051 [12:35<11:27,  2.11it/s, est. speed input: 69270.03 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1598/3051 [12:35<11:27,  2.11it/s, est. speed input: 69270.05 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1599/3051 [12:36<11:27,  2.11it/s, est. speed input: 69269.89 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1600/3051 [12:36<11:26,  2.11it/s, est. speed input: 69269.87 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1601/3051 [12:37<11:26,  2.11it/s, est. speed input: 69269.91 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1602/3051 [12:37<11:24,  2.12it/s, est. speed input: 69270.10 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1603/3051 [12:38<11:25,  2.11it/s, est. speed input: 69269.97 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1604/3051 [12:38<11:24,  2.11it/s, est. speed input: 69270.02 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1605/3051 [12:39<11:23,  2.12it/s, est. speed input: 69270.12 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1606/3051 [12:39<11:22,  2.12it/s, est. speed input: 69270.17 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1607/3051 [12:40<11:23,  2.11it/s, est. speed input: 69270.09 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1608/3051 [12:40<11:22,  2.11it/s, est. speed input: 69270.15 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1609/3051 [12:41<11:21,  2.12it/s, est. speed input: 69270.22 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1610/3051 [12:41<11:22,  2.11it/s, est. speed input: 69270.03 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1611/3051 [12:42<11:21,  2.11it/s, est. speed input: 69269.98 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1612/3051 [12:42<11:20,  2.11it/s, est. speed input: 69270.11 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1613/3051 [12:43<11:19,  2.12it/s, est. speed input: 69270.23 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1614/3051 [12:43<11:20,  2.11it/s, est. speed input: 69270.00 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1615/3051 [12:43<11:19,  2.11it/s, est. speed input: 69270.12 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1616/3051 [12:44<11:19,  2.11it/s, est. speed input: 69270.04 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1617/3051 [12:44<11:19,  2.11it/s, est. speed input: 69269.89 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1618/3051 [12:45<11:18,  2.11it/s, est. speed input: 69269.93 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1619/3051 [12:45<11:17,  2.11it/s, est. speed input: 69270.05 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1620/3051 [12:46<11:16,  2.12it/s, est. speed input: 69270.12 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1621/3051 [12:46<11:16,  2.11it/s, est. speed input: 69270.10 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1622/3051 [12:47<11:15,  2.12it/s, est. speed input: 69270.19 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1623/3051 [12:47<11:17,  2.11it/s, est. speed input: 69269.74 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1624/3051 [12:48<11:15,  2.11it/s, est. speed input: 69269.90 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1625/3051 [12:48<11:14,  2.11it/s, est. speed input: 69270.01 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1626/3051 [12:49<11:15,  2.11it/s, est. speed input: 69269.75 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1627/3051 [12:49<11:15,  2.11it/s, est. speed input: 69269.53 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1628/3051 [12:50<11:14,  2.11it/s, est. speed input: 69269.44 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1629/3051 [12:50<11:13,  2.11it/s, est. speed input: 69269.51 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1630/3051 [12:51<11:12,  2.11it/s, est. speed input: 69269.57 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1631/3051 [12:51<11:11,  2.11it/s, est. speed input: 69269.66 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1632/3051 [12:52<11:10,  2.12it/s, est. speed input: 69269.80 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1633/3051 [12:52<11:09,  2.12it/s, est. speed input: 69269.85 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1634/3051 [12:52<11:10,  2.11it/s, est. speed input: 69269.61 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1635/3051 [12:53<11:09,  2.11it/s, est. speed input: 69269.70 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1636/3051 [12:53<11:08,  2.12it/s, est. speed input: 69269.82 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1637/3051 [12:54<11:07,  2.12it/s, est. speed input: 69269.94 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1638/3051 [12:54<11:07,  2.12it/s, est. speed input: 69269.96 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1639/3051 [12:55<11:07,  2.12it/s, est. speed input: 69270.03 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1640/3051 [12:55<11:06,  2.12it/s, est. speed input: 69270.05 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1641/3051 [12:56<11:05,  2.12it/s, est. speed input: 69270.16 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1642/3051 [12:56<11:05,  2.12it/s, est. speed input: 69270.27 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1643/3051 [12:57<11:06,  2.11it/s, est. speed input: 69269.91 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1644/3051 [12:57<11:06,  2.11it/s, est. speed input: 69269.89 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1645/3051 [12:58<11:05,  2.11it/s, est. speed input: 69270.00 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1646/3051 [12:58<11:04,  2.11it/s, est. speed input: 69270.06 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1647/3051 [12:59<11:05,  2.11it/s, est. speed input: 69269.68 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1648/3051 [12:59<11:04,  2.11it/s, est. speed input: 69269.78 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1649/3051 [13:00<11:03,  2.11it/s, est. speed input: 69269.90 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1650/3051 [13:00<11:02,  2.11it/s, est. speed input: 69269.89 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1651/3051 [13:01<11:02,  2.11it/s, est. speed input: 69269.88 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1652/3051 [13:01<11:02,  2.11it/s, est. speed input: 69269.68 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1653/3051 [13:01<11:02,  2.11it/s, est. speed input: 69269.61 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1654/3051 [13:02<11:01,  2.11it/s, est. speed input: 69269.72 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1655/3051 [13:02<11:06,  2.09it/s, est. speed input: 69268.48 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1656/3051 [13:03<11:02,  2.10it/s, est. speed input: 69268.73 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1657/3051 [13:03<11:00,  2.11it/s, est. speed input: 69268.92 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1658/3051 [13:04<11:00,  2.11it/s, est. speed input: 69268.77 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1659/3051 [13:04<11:00,  2.11it/s, est. speed input: 69268.55 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1660/3051 [13:05<10:59,  2.11it/s, est. speed input: 69268.61 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1661/3051 [13:05<10:57,  2.11it/s, est. speed input: 69268.80 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1662/3051 [13:06<10:56,  2.12it/s, est. speed input: 69268.90 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1663/3051 [13:06<10:56,  2.12it/s, est. speed input: 69268.93 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1664/3051 [13:07<10:55,  2.12it/s, est. speed input: 69269.00 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1665/3051 [13:07<10:55,  2.12it/s, est. speed input: 69269.04 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1666/3051 [13:08<10:56,  2.11it/s, est. speed input: 69268.75 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1667/3051 [13:08<10:54,  2.11it/s, est. speed input: 69268.83 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1668/3051 [13:09<10:54,  2.11it/s, est. speed input: 69268.91 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1669/3051 [13:09<10:54,  2.11it/s, est. speed input: 69268.81 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1670/3051 [13:10<10:53,  2.11it/s, est. speed input: 69268.88 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1671/3051 [13:10<10:52,  2.12it/s, est. speed input: 69268.99 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1672/3051 [13:10<10:51,  2.12it/s, est. speed input: 69269.17 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1673/3051 [13:11<10:51,  2.11it/s, est. speed input: 69269.01 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1674/3051 [13:11<10:51,  2.11it/s, est. speed input: 69269.04 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1675/3051 [13:12<10:50,  2.12it/s, est. speed input: 69269.15 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1676/3051 [13:12<10:49,  2.12it/s, est. speed input: 69269.16 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1677/3051 [13:13<10:49,  2.12it/s, est. speed input: 69269.20 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1678/3051 [13:13<10:51,  2.11it/s, est. speed input: 69268.64 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1679/3051 [13:14<10:50,  2.11it/s, est. speed input: 69268.67 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1680/3051 [13:14<10:49,  2.11it/s, est. speed input: 69268.75 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1681/3051 [13:15<10:48,  2.11it/s, est. speed input: 69268.76 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1682/3051 [13:15<10:49,  2.11it/s, est. speed input: 69268.49 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1683/3051 [13:16<10:47,  2.11it/s, est. speed input: 69268.57 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1684/3051 [13:16<10:47,  2.11it/s, est. speed input: 69268.60 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1685/3051 [13:17<10:46,  2.11it/s, est. speed input: 69268.58 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1686/3051 [13:17<10:45,  2.11it/s, est. speed input: 69268.66 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1687/3051 [13:18<10:50,  2.10it/s, est. speed input: 69267.57 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1688/3051 [13:18<10:47,  2.10it/s, est. speed input: 69267.70 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1689/3051 [13:19<10:45,  2.11it/s, est. speed input: 69267.77 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1690/3051 [13:19<10:45,  2.11it/s, est. speed input: 69267.71 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1691/3051 [13:19<10:45,  2.11it/s, est. speed input: 69267.47 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1692/3051 [13:20<10:44,  2.11it/s, est. speed input: 69267.49 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1693/3051 [13:20<10:42,  2.11it/s, est. speed input: 69267.64 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1694/3051 [13:21<10:42,  2.11it/s, est. speed input: 69267.65 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1695/3051 [13:21<10:41,  2.12it/s, est. speed input: 69267.77 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1696/3051 [13:22<10:40,  2.12it/s, est. speed input: 69267.91 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1697/3051 [13:22<10:40,  2.11it/s, est. speed input: 69267.79 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1698/3051 [13:23<10:40,  2.11it/s, est. speed input: 69267.75 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1699/3051 [13:23<10:38,  2.12it/s, est. speed input: 69267.91 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1700/3051 [13:24<10:39,  2.11it/s, est. speed input: 69267.79 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1701/3051 [13:24<10:39,  2.11it/s, est. speed input: 69267.68 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1702/3051 [13:25<10:37,  2.11it/s, est. speed input: 69267.80 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1703/3051 [13:25<10:37,  2.11it/s, est. speed input: 69267.79 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1704/3051 [13:26<10:37,  2.11it/s, est. speed input: 69267.72 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1705/3051 [13:26<10:36,  2.11it/s, est. speed input: 69267.74 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1706/3051 [13:27<10:35,  2.12it/s, est. speed input: 69267.89 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1707/3051 [13:27<10:34,  2.12it/s, est. speed input: 69268.02 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1708/3051 [13:27<10:35,  2.11it/s, est. speed input: 69267.83 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1709/3051 [13:28<10:34,  2.11it/s, est. speed input: 69267.90 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1710/3051 [13:28<10:33,  2.12it/s, est. speed input: 69268.02 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1711/3051 [13:29<10:33,  2.12it/s, est. speed input: 69268.05 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1712/3051 [13:29<10:32,  2.12it/s, est. speed input: 69268.12 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1713/3051 [13:30<10:33,  2.11it/s, est. speed input: 69267.92 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1714/3051 [13:30<10:32,  2.11it/s, est. speed input: 69267.92 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1715/3051 [13:31<10:31,  2.12it/s, est. speed input: 69268.07 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1716/3051 [13:31<10:30,  2.12it/s, est. speed input: 69268.18 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1717/3051 [13:32<10:31,  2.11it/s, est. speed input: 69267.89 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1718/3051 [13:32<10:30,  2.11it/s, est. speed input: 69267.92 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1719/3051 [13:33<10:30,  2.11it/s, est. speed input: 69268.00 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1720/3051 [13:33<10:28,  2.12it/s, est. speed input: 69268.14 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1721/3051 [13:34<10:28,  2.11it/s, est. speed input: 69268.08 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1722/3051 [13:34<10:28,  2.12it/s, est. speed input: 69268.18 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1723/3051 [13:35<10:29,  2.11it/s, est. speed input: 69267.78 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1724/3051 [13:35<10:28,  2.11it/s, est. speed input: 69267.79 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1725/3051 [13:36<10:27,  2.11it/s, est. speed input: 69267.88 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1726/3051 [13:36<10:26,  2.11it/s, est. speed input: 69267.89 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1727/3051 [13:36<10:26,  2.11it/s, est. speed input: 69267.97 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1728/3051 [13:37<10:25,  2.12it/s, est. speed input: 69268.02 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1729/3051 [13:37<10:24,  2.12it/s, est. speed input: 69268.07 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1730/3051 [13:38<10:24,  2.12it/s, est. speed input: 69268.12 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1731/3051 [13:38<10:23,  2.12it/s, est. speed input: 69268.23 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1732/3051 [13:39<10:22,  2.12it/s, est. speed input: 69268.34 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1733/3051 [13:39<10:23,  2.11it/s, est. speed input: 69268.11 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1734/3051 [13:40<10:22,  2.12it/s, est. speed input: 69268.23 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1735/3051 [13:40<10:21,  2.12it/s, est. speed input: 69268.39 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1736/3051 [13:41<10:21,  2.12it/s, est. speed input: 69268.39 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1737/3051 [13:41<10:21,  2.11it/s, est. speed input: 69268.35 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1738/3051 [13:42<10:20,  2.11it/s, est. speed input: 69268.35 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1739/3051 [13:42<10:20,  2.11it/s, est. speed input: 69268.37 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1740/3051 [13:43<10:19,  2.11it/s, est. speed input: 69268.40 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1741/3051 [13:43<10:19,  2.11it/s, est. speed input: 69268.40 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1742/3051 [13:44<10:18,  2.12it/s, est. speed input: 69268.47 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1743/3051 [13:44<10:18,  2.12it/s, est. speed input: 69268.50 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1744/3051 [13:45<10:17,  2.12it/s, est. speed input: 69268.55 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1745/3051 [13:45<10:16,  2.12it/s, est. speed input: 69268.65 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1746/3051 [13:45<10:18,  2.11it/s, est. speed input: 69268.33 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1747/3051 [13:46<10:17,  2.11it/s, est. speed input: 69268.39 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1748/3051 [13:46<10:16,  2.12it/s, est. speed input: 69268.51 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1749/3051 [13:47<10:15,  2.12it/s, est. speed input: 69268.56 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1750/3051 [13:47<10:15,  2.12it/s, est. speed input: 69268.58 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1751/3051 [13:48<10:13,  2.12it/s, est. speed input: 69268.74 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1752/3051 [13:48<10:13,  2.12it/s, est. speed input: 69268.74 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1753/3051 [13:49<10:13,  2.11it/s, est. speed input: 69268.65 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1754/3051 [13:49<10:12,  2.12it/s, est. speed input: 69268.77 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1755/3051 [13:50<10:14,  2.11it/s, est. speed input: 69268.28 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1756/3051 [13:50<10:13,  2.11it/s, est. speed input: 69268.46 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1757/3051 [13:51<10:11,  2.11it/s, est. speed input: 69268.59 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1758/3051 [13:51<10:11,  2.11it/s, est. speed input: 69268.56 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1759/3051 [13:52<10:11,  2.11it/s, est. speed input: 69268.40 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1760/3051 [13:52<10:10,  2.11it/s, est. speed input: 69268.44 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1761/3051 [13:53<10:09,  2.12it/s, est. speed input: 69268.58 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1762/3051 [13:53<10:09,  2.12it/s, est. speed input: 69268.61 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1763/3051 [13:54<10:10,  2.11it/s, est. speed input: 69268.30 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1764/3051 [13:54<10:09,  2.11it/s, est. speed input: 69268.39 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1765/3051 [13:54<10:08,  2.11it/s, est. speed input: 69268.48 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1766/3051 [13:55<10:08,  2.11it/s, est. speed input: 69268.39 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1767/3051 [13:55<10:06,  2.12it/s, est. speed input: 69268.56 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1768/3051 [13:56<10:07,  2.11it/s, est. speed input: 69268.42 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1769/3051 [13:56<10:06,  2.11it/s, est. speed input: 69268.46 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1770/3051 [13:57<10:05,  2.12it/s, est. speed input: 69268.53 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1771/3051 [13:57<10:04,  2.12it/s, est. speed input: 69268.64 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1772/3051 [13:58<10:03,  2.12it/s, est. speed input: 69268.75 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1773/3051 [13:58<10:03,  2.12it/s, est. speed input: 69268.77 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1774/3051 [13:59<10:03,  2.12it/s, est. speed input: 69268.79 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1775/3051 [13:59<10:03,  2.12it/s, est. speed input: 69268.78 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1776/3051 [14:00<10:02,  2.12it/s, est. speed input: 69268.86 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1777/3051 [14:00<10:02,  2.11it/s, est. speed input: 69268.79 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1778/3051 [14:01<10:02,  2.11it/s, est. speed input: 69268.64 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1779/3051 [14:01<10:02,  2.11it/s, est. speed input: 69268.63 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1780/3051 [14:02<10:00,  2.12it/s, est. speed input: 69268.79 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1781/3051 [14:02<10:00,  2.12it/s, est. speed input: 69268.83 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1782/3051 [14:02<09:59,  2.12it/s, est. speed input: 69268.89 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1783/3051 [14:03<09:59,  2.12it/s, est. speed input: 69268.92 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1784/3051 [14:03<09:58,  2.12it/s, est. speed input: 69268.93 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1785/3051 [14:04<09:58,  2.12it/s, est. speed input: 69269.05 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1786/3051 [14:04<09:57,  2.12it/s, est. speed input: 69269.04 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1787/3051 [14:05<09:57,  2.12it/s, est. speed input: 69269.04 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1788/3051 [14:05<09:58,  2.11it/s, est. speed input: 69268.68 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1789/3051 [14:06<09:57,  2.11it/s, est. speed input: 69268.78 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1790/3051 [14:06<09:56,  2.11it/s, est. speed input: 69268.86 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1791/3051 [14:07<09:56,  2.11it/s, est. speed input: 69268.70 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1792/3051 [14:07<09:56,  2.11it/s, est. speed input: 69268.72 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1793/3051 [14:08<09:55,  2.11it/s, est. speed input: 69268.78 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1794/3051 [14:08<09:55,  2.11it/s, est. speed input: 69268.63 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1795/3051 [14:09<09:54,  2.11it/s, est. speed input: 69268.66 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1796/3051 [14:09<09:53,  2.11it/s, est. speed input: 69268.67 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1797/3051 [14:10<09:52,  2.12it/s, est. speed input: 69268.77 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1798/3051 [14:10<09:51,  2.12it/s, est. speed input: 69268.91 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1799/3051 [14:11<09:51,  2.12it/s, est. speed input: 69268.95 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1800/3051 [14:11<09:51,  2.12it/s, est. speed input: 69268.99 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1801/3051 [14:11<09:51,  2.11it/s, est. speed input: 69268.85 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1802/3051 [14:12<09:50,  2.11it/s, est. speed input: 69268.88 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1803/3051 [14:12<09:49,  2.12it/s, est. speed input: 69269.03 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1804/3051 [14:13<09:49,  2.12it/s, est. speed input: 69269.08 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1805/3051 [14:13<09:49,  2.11it/s, est. speed input: 69268.94 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1806/3051 [14:14<09:48,  2.12it/s, est. speed input: 69269.03 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1807/3051 [14:14<09:47,  2.12it/s, est. speed input: 69269.15 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1808/3051 [14:15<09:47,  2.11it/s, est. speed input: 69269.05 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1809/3051 [14:15<09:47,  2.11it/s, est. speed input: 69269.04 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1810/3051 [14:16<09:46,  2.11it/s, est. speed input: 69269.08 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1811/3051 [14:16<09:47,  2.11it/s, est. speed input: 69268.91 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1812/3051 [14:17<09:46,  2.11it/s, est. speed input: 69268.98 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1813/3051 [14:17<09:45,  2.11it/s, est. speed input: 69268.94 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1814/3051 [14:18<09:46,  2.11it/s, est. speed input: 69268.74 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1815/3051 [14:18<09:45,  2.11it/s, est. speed input: 69268.78 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1816/3051 [14:19<09:44,  2.11it/s, est. speed input: 69268.90 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1817/3051 [14:19<09:43,  2.11it/s, est. speed input: 69268.89 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1818/3051 [14:20<09:43,  2.11it/s, est. speed input: 69268.93 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1819/3051 [14:20<09:42,  2.12it/s, est. speed input: 69269.05 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1820/3051 [14:20<09:41,  2.12it/s, est. speed input: 69269.13 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1821/3051 [14:21<09:41,  2.11it/s, est. speed input: 69269.00 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1822/3051 [14:21<09:41,  2.11it/s, est. speed input: 69269.06 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1823/3051 [14:22<09:42,  2.11it/s, est. speed input: 69268.74 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1824/3051 [14:22<09:42,  2.11it/s, est. speed input: 69268.54 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1825/3051 [14:23<09:40,  2.11it/s, est. speed input: 69268.63 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1826/3051 [14:23<09:39,  2.11it/s, est. speed input: 69268.73 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1827/3051 [14:24<09:38,  2.12it/s, est. speed input: 69268.87 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1828/3051 [14:24<09:37,  2.12it/s, est. speed input: 69269.01 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1829/3051 [14:25<09:54,  2.06it/s, est. speed input: 69265.35 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1830/3051 [14:25<09:48,  2.08it/s, est. speed input: 69265.49 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1831/3051 [14:26<09:44,  2.09it/s, est. speed input: 69265.61 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1832/3051 [14:26<09:41,  2.10it/s, est. speed input: 69265.60 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1833/3051 [14:27<09:39,  2.10it/s, est. speed input: 69265.61 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1834/3051 [14:27<09:37,  2.11it/s, est. speed input: 69265.72 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1835/3051 [14:28<09:42,  2.09it/s, est. speed input: 69264.39 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1836/3051 [14:28<09:39,  2.10it/s, est. speed input: 69264.59 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1837/3051 [14:29<09:36,  2.11it/s, est. speed input: 69264.75 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1838/3051 [14:29<09:35,  2.11it/s, est. speed input: 69264.66 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1839/3051 [14:30<09:35,  2.10it/s, est. speed input: 69264.41 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1840/3051 [14:30<09:34,  2.11it/s, est. speed input: 69264.47 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1841/3051 [14:30<09:33,  2.11it/s, est. speed input: 69264.49 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1842/3051 [14:31<09:34,  2.11it/s, est. speed input: 69264.16 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1843/3051 [14:31<09:32,  2.11it/s, est. speed input: 69264.19 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1844/3051 [14:32<09:31,  2.11it/s, est. speed input: 69264.30 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1845/3051 [14:32<09:30,  2.11it/s, est. speed input: 69264.32 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1846/3051 [14:33<09:30,  2.11it/s, est. speed input: 69264.29 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1847/3051 [14:33<09:29,  2.11it/s, est. speed input: 69264.35 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1848/3051 [14:34<09:28,  2.12it/s, est. speed input: 69264.46 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1849/3051 [14:34<09:28,  2.11it/s, est. speed input: 69264.36 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1850/3051 [14:35<09:27,  2.12it/s, est. speed input: 69264.44 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1851/3051 [14:35<09:27,  2.12it/s, est. speed input: 69264.51 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1852/3051 [14:36<09:27,  2.11it/s, est. speed input: 69264.43 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1853/3051 [14:36<09:26,  2.12it/s, est. speed input: 69264.53 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1854/3051 [14:37<09:44,  2.05it/s, est. speed input: 69260.43 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1855/3051 [14:37<09:37,  2.07it/s, est. speed input: 69260.58 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1856/3051 [14:38<09:33,  2.09it/s, est. speed input: 69260.76 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1857/3051 [14:38<09:30,  2.09it/s, est. speed input: 69260.81 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1858/3051 [14:39<09:28,  2.10it/s, est. speed input: 69260.69 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1859/3051 [14:39<09:26,  2.11it/s, est. speed input: 69260.86 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1860/3051 [14:39<09:25,  2.11it/s, est. speed input: 69260.78 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1861/3051 [14:40<09:24,  2.11it/s, est. speed input: 69260.84 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1862/3051 [14:40<09:22,  2.11it/s, est. speed input: 69260.93 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1863/3051 [14:41<09:21,  2.11it/s, est. speed input: 69261.00 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1864/3051 [14:41<09:21,  2.11it/s, est. speed input: 69261.04 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1865/3051 [14:42<09:22,  2.11it/s, est. speed input: 69260.66 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1866/3051 [14:42<09:22,  2.11it/s, est. speed input: 69260.44 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1867/3051 [14:43<09:21,  2.11it/s, est. speed input: 69260.43 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1868/3051 [14:43<09:20,  2.11it/s, est. speed input: 69260.53 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1869/3051 [14:44<09:19,  2.11it/s, est. speed input: 69260.63 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1870/3051 [14:44<09:18,  2.11it/s, est. speed input: 69260.68 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1871/3051 [14:45<09:19,  2.11it/s, est. speed input: 69260.47 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1872/3051 [14:45<09:17,  2.11it/s, est. speed input: 69260.59 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1873/3051 [14:46<09:16,  2.12it/s, est. speed input: 69260.72 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1874/3051 [14:46<09:16,  2.12it/s, est. speed input: 69260.74 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1875/3051 [14:47<09:15,  2.12it/s, est. speed input: 69260.82 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1876/3051 [14:47<09:20,  2.10it/s, est. speed input: 69259.72 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1877/3051 [14:48<09:18,  2.10it/s, est. speed input: 69259.79 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1878/3051 [14:48<09:16,  2.11it/s, est. speed input: 69259.81 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1879/3051 [14:48<09:15,  2.11it/s, est. speed input: 69259.80 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1880/3051 [14:49<09:15,  2.11it/s, est. speed input: 69259.79 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1881/3051 [14:49<09:14,  2.11it/s, est. speed input: 69259.81 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1882/3051 [14:50<09:13,  2.11it/s, est. speed input: 69259.85 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1883/3051 [14:50<09:13,  2.11it/s, est. speed input: 69259.62 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1884/3051 [14:51<09:12,  2.11it/s, est. speed input: 69259.69 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1885/3051 [14:51<09:11,  2.11it/s, est. speed input: 69259.82 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1886/3051 [14:52<09:11,  2.11it/s, est. speed input: 69259.82 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1887/3051 [14:52<09:12,  2.11it/s, est. speed input: 69259.46 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1888/3051 [14:53<09:10,  2.11it/s, est. speed input: 69259.54 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1889/3051 [14:53<09:09,  2.11it/s, est. speed input: 69259.63 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1890/3051 [14:54<09:09,  2.11it/s, est. speed input: 69259.66 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1891/3051 [14:54<09:08,  2.11it/s, est. speed input: 69259.71 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1892/3051 [14:55<09:07,  2.12it/s, est. speed input: 69259.83 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1893/3051 [14:55<09:07,  2.11it/s, est. speed input: 69259.69 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1894/3051 [14:56<09:07,  2.11it/s, est. speed input: 69259.78 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1895/3051 [14:56<09:07,  2.11it/s, est. speed input: 69259.70 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1896/3051 [14:57<09:07,  2.11it/s, est. speed input: 69259.57 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1897/3051 [14:57<09:06,  2.11it/s, est. speed input: 69259.58 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1898/3051 [14:57<09:05,  2.11it/s, est. speed input: 69259.72 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1899/3051 [14:58<09:04,  2.12it/s, est. speed input: 69259.80 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1900/3051 [14:58<09:04,  2.11it/s, est. speed input: 69259.78 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1901/3051 [14:59<09:03,  2.11it/s, est. speed input: 69259.79 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1902/3051 [14:59<09:03,  2.11it/s, est. speed input: 69259.82 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1903/3051 [15:00<09:03,  2.11it/s, est. speed input: 69259.75 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1904/3051 [15:00<09:02,  2.12it/s, est. speed input: 69259.85 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1905/3051 [15:01<09:01,  2.12it/s, est. speed input: 69259.90 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1906/3051 [15:01<09:02,  2.11it/s, est. speed input: 69259.56 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1907/3051 [15:02<09:01,  2.11it/s, est. speed input: 69259.74 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1908/3051 [15:02<09:00,  2.11it/s, est. speed input: 69259.79 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1909/3051 [15:03<09:00,  2.11it/s, est. speed input: 69259.81 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1910/3051 [15:03<08:59,  2.12it/s, est. speed input: 69259.90 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1911/3051 [15:04<09:18,  2.04it/s, est. speed input: 69255.43 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1912/3051 [15:04<09:12,  2.06it/s, est. speed input: 69255.53 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1913/3051 [15:05<09:07,  2.08it/s, est. speed input: 69255.56 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1914/3051 [15:05<09:04,  2.09it/s, est. speed input: 69255.58 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1915/3051 [15:06<09:01,  2.10it/s, est. speed input: 69255.64 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1916/3051 [15:06<08:59,  2.10it/s, est. speed input: 69255.76 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1917/3051 [15:07<08:58,  2.11it/s, est. speed input: 69255.77 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1918/3051 [15:07<08:57,  2.11it/s, est. speed input: 69255.80 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1919/3051 [15:07<08:55,  2.11it/s, est. speed input: 69255.87 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1920/3051 [15:08<08:55,  2.11it/s, est. speed input: 69255.88 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1921/3051 [15:08<08:54,  2.11it/s, est. speed input: 69255.87 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1922/3051 [15:09<08:54,  2.11it/s, est. speed input: 69255.88 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1923/3051 [15:09<08:53,  2.11it/s, est. speed input: 69255.91 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1924/3051 [15:10<08:53,  2.11it/s, est. speed input: 69255.92 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1925/3051 [15:10<08:52,  2.11it/s, est. speed input: 69255.93 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1926/3051 [15:11<08:52,  2.11it/s, est. speed input: 69255.97 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1927/3051 [15:11<08:51,  2.11it/s, est. speed input: 69255.99 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1928/3051 [15:12<08:52,  2.11it/s, est. speed input: 69255.71 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1929/3051 [15:12<08:51,  2.11it/s, est. speed input: 69255.75 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1930/3051 [15:13<08:50,  2.11it/s, est. speed input: 69255.80 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1931/3051 [15:13<08:49,  2.11it/s, est. speed input: 69255.87 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1932/3051 [15:14<08:49,  2.11it/s, est. speed input: 69255.87 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1933/3051 [15:14<08:48,  2.11it/s, est. speed input: 69255.88 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1934/3051 [15:15<08:48,  2.11it/s, est. speed input: 69255.86 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1935/3051 [15:15<08:47,  2.11it/s, est. speed input: 69255.92 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1936/3051 [15:16<08:59,  2.07it/s, est. speed input: 69253.27 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1937/3051 [15:16<08:54,  2.08it/s, est. speed input: 69253.36 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1938/3051 [15:16<08:51,  2.09it/s, est. speed input: 69253.43 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1939/3051 [15:17<08:49,  2.10it/s, est. speed input: 69253.44 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1940/3051 [15:17<08:48,  2.10it/s, est. speed input: 69253.45 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1941/3051 [15:18<08:46,  2.11it/s, est. speed input: 69253.50 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1942/3051 [15:18<08:45,  2.11it/s, est. speed input: 69253.56 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1943/3051 [15:19<08:45,  2.11it/s, est. speed input: 69253.38 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1944/3051 [15:19<08:44,  2.11it/s, est. speed input: 69253.37 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1945/3051 [15:20<08:43,  2.11it/s, est. speed input: 69253.40 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1946/3051 [15:20<08:43,  2.11it/s, est. speed input: 69253.43 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1947/3051 [15:21<08:42,  2.11it/s, est. speed input: 69253.46 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1948/3051 [15:21<08:41,  2.11it/s, est. speed input: 69253.46 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1949/3051 [15:22<08:41,  2.11it/s, est. speed input: 69253.45 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1950/3051 [15:22<08:41,  2.11it/s, est. speed input: 69253.40 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1951/3051 [15:23<08:40,  2.11it/s, est. speed input: 69253.42 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1952/3051 [15:23<08:39,  2.11it/s, est. speed input: 69253.46 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1953/3051 [15:24<08:39,  2.12it/s, est. speed input: 69253.53 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1954/3051 [15:24<08:38,  2.11it/s, est. speed input: 69253.52 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1955/3051 [15:25<08:38,  2.12it/s, est. speed input: 69253.57 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1956/3051 [15:25<08:37,  2.12it/s, est. speed input: 69253.60 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1957/3051 [15:25<08:37,  2.12it/s, est. speed input: 69253.65 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1958/3051 [15:26<08:36,  2.12it/s, est. speed input: 69253.74 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1959/3051 [15:26<08:36,  2.12it/s, est. speed input: 69253.72 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1960/3051 [15:27<08:36,  2.11it/s, est. speed input: 69253.67 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1961/3051 [15:27<08:35,  2.11it/s, est. speed input: 69253.72 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1962/3051 [15:28<08:35,  2.11it/s, est. speed input: 69253.74 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1963/3051 [15:28<08:34,  2.11it/s, est. speed input: 69253.75 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1964/3051 [15:29<08:34,  2.11it/s, est. speed input: 69253.78 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1965/3051 [15:29<08:35,  2.11it/s, est. speed input: 69253.40 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1966/3051 [15:30<08:34,  2.11it/s, est. speed input: 69253.40 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1967/3051 [15:30<08:33,  2.11it/s, est. speed input: 69253.44 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1968/3051 [15:31<08:32,  2.11it/s, est. speed input: 69253.40 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1969/3051 [15:31<08:31,  2.11it/s, est. speed input: 69253.48 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1970/3051 [15:32<08:31,  2.11it/s, est. speed input: 69253.50 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1971/3051 [15:32<08:37,  2.09it/s, est. speed input: 69251.97 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1972/3051 [15:33<08:35,  2.09it/s, est. speed input: 69251.99 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1973/3051 [15:33<08:33,  2.10it/s, est. speed input: 69251.95 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1974/3051 [15:34<08:32,  2.10it/s, est. speed input: 69251.85 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1975/3051 [15:34<08:31,  2.10it/s, est. speed input: 69251.79 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1976/3051 [15:34<08:30,  2.11it/s, est. speed input: 69251.78 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1977/3051 [15:35<08:29,  2.11it/s, est. speed input: 69251.82 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1978/3051 [15:35<08:28,  2.11it/s, est. speed input: 69251.84 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1979/3051 [15:36<08:28,  2.11it/s, est. speed input: 69251.73 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1980/3051 [15:36<08:27,  2.11it/s, est. speed input: 69251.81 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1981/3051 [15:37<08:26,  2.11it/s, est. speed input: 69251.78 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1982/3051 [15:37<08:27,  2.11it/s, est. speed input: 69251.42 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1983/3051 [15:38<08:26,  2.11it/s, est. speed input: 69251.40 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1984/3051 [15:38<08:25,  2.11it/s, est. speed input: 69251.43 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1985/3051 [15:39<08:24,  2.11it/s, est. speed input: 69251.43 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1986/3051 [15:39<08:24,  2.11it/s, est. speed input: 69251.44 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1987/3051 [15:40<08:23,  2.11it/s, est. speed input: 69251.46 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1988/3051 [15:40<08:23,  2.11it/s, est. speed input: 69251.33 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1989/3051 [15:41<08:22,  2.11it/s, est. speed input: 69251.37 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1990/3051 [15:41<08:22,  2.11it/s, est. speed input: 69251.39 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1991/3051 [15:42<08:21,  2.11it/s, est. speed input: 69251.41 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1992/3051 [15:42<08:21,  2.11it/s, est. speed input: 69251.41 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1993/3051 [15:43<08:45,  2.01it/s, est. speed input: 69245.61 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1994/3051 [15:43<08:37,  2.04it/s, est. speed input: 69245.75 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1995/3051 [15:44<08:30,  2.07it/s, est. speed input: 69245.90 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1996/3051 [15:44<08:26,  2.08it/s, est. speed input: 69246.05 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1997/3051 [15:45<08:23,  2.09it/s, est. speed input: 69246.18 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1998/3051 [15:45<08:21,  2.10it/s, est. speed input: 69246.23 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 1999/3051 [15:45<08:19,  2.10it/s, est. speed input: 69246.25 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2000/3051 [15:46<08:20,  2.10it/s, est. speed input: 69245.93 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2001/3051 [15:46<08:18,  2.11it/s, est. speed input: 69246.07 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2002/3051 [15:47<08:17,  2.11it/s, est. speed input: 69246.14 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2003/3051 [15:47<08:15,  2.11it/s, est. speed input: 69246.23 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2004/3051 [15:48<08:15,  2.11it/s, est. speed input: 69246.29 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2005/3051 [15:48<08:14,  2.12it/s, est. speed input: 69246.41 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2006/3051 [15:49<08:13,  2.12it/s, est. speed input: 69246.42 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2007/3051 [15:49<08:13,  2.11it/s, est. speed input: 69246.41 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2008/3051 [15:50<08:12,  2.12it/s, est. speed input: 69246.51 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2009/3051 [15:50<08:12,  2.12it/s, est. speed input: 69246.55 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2010/3051 [15:51<08:13,  2.11it/s, est. speed input: 69246.31 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2011/3051 [15:51<08:12,  2.11it/s, est. speed input: 69246.43 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2012/3051 [15:52<08:11,  2.12it/s, est. speed input: 69246.54 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2013/3051 [15:52<08:10,  2.11it/s, est. speed input: 69246.55 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2014/3051 [15:53<08:10,  2.12it/s, est. speed input: 69246.64 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2015/3051 [15:53<08:09,  2.11it/s, est. speed input: 69246.59 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2016/3051 [15:53<08:09,  2.12it/s, est. speed input: 69246.73 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2017/3051 [15:54<08:09,  2.11it/s, est. speed input: 69246.56 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2018/3051 [15:54<08:08,  2.11it/s, est. speed input: 69246.56 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2019/3051 [15:55<08:08,  2.11it/s, est. speed input: 69246.59 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2020/3051 [15:55<08:07,  2.12it/s, est. speed input: 69246.69 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2021/3051 [15:56<08:06,  2.12it/s, est. speed input: 69246.75 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2022/3051 [15:56<08:07,  2.11it/s, est. speed input: 69246.47 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2023/3051 [15:57<08:06,  2.11it/s, est. speed input: 69246.51 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2024/3051 [15:57<08:05,  2.12it/s, est. speed input: 69246.67 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2025/3051 [15:58<08:05,  2.11it/s, est. speed input: 69246.67 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2026/3051 [15:58<08:04,  2.12it/s, est. speed input: 69246.72 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2027/3051 [15:59<08:04,  2.11it/s, est. speed input: 69246.61 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2028/3051 [15:59<08:05,  2.11it/s, est. speed input: 69246.38 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2029/3051 [16:00<08:03,  2.11it/s, est. speed input: 69246.50 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2030/3051 [16:00<08:03,  2.11it/s, est. speed input: 69246.49 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2031/3051 [16:01<08:03,  2.11it/s, est. speed input: 69246.39 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2032/3051 [16:01<08:02,  2.11it/s, est. speed input: 69246.39 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2033/3051 [16:02<08:01,  2.11it/s, est. speed input: 69246.45 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2034/3051 [16:02<08:00,  2.12it/s, est. speed input: 69246.55 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2035/3051 [16:02<07:59,  2.12it/s, est. speed input: 69246.67 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2036/3051 [16:03<08:00,  2.11it/s, est. speed input: 69246.49 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2037/3051 [16:03<07:59,  2.11it/s, est. speed input: 69246.59 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2038/3051 [16:04<07:58,  2.12it/s, est. speed input: 69246.73 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2039/3051 [16:04<07:58,  2.12it/s, est. speed input: 69246.73 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2040/3051 [16:05<07:57,  2.12it/s, est. speed input: 69246.81 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2041/3051 [16:05<07:57,  2.12it/s, est. speed input: 69246.89 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2042/3051 [16:06<07:56,  2.12it/s, est. speed input: 69246.90 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2043/3051 [16:06<07:56,  2.12it/s, est. speed input: 69247.00 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2044/3051 [16:07<07:55,  2.12it/s, est. speed input: 69247.13 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2045/3051 [16:07<07:55,  2.12it/s, est. speed input: 69247.05 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2046/3051 [16:08<07:54,  2.12it/s, est. speed input: 69247.21 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2047/3051 [16:08<07:54,  2.12it/s, est. speed input: 69247.21 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2048/3051 [16:09<07:54,  2.12it/s, est. speed input: 69247.20 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2049/3051 [16:09<07:54,  2.11it/s, est. speed input: 69247.04 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2050/3051 [16:10<07:53,  2.11it/s, est. speed input: 69247.13 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2051/3051 [16:10<07:52,  2.12it/s, est. speed input: 69247.25 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2052/3051 [16:11<07:52,  2.11it/s, est. speed input: 69247.16 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2053/3051 [16:11<07:51,  2.11it/s, est. speed input: 69247.21 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2054/3051 [16:11<07:51,  2.12it/s, est. speed input: 69247.28 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2055/3051 [16:12<07:51,  2.11it/s, est. speed input: 69247.23 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2056/3051 [16:12<07:50,  2.12it/s, est. speed input: 69247.31 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2057/3051 [16:13<07:49,  2.12it/s, est. speed input: 69247.35 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2058/3051 [16:13<07:48,  2.12it/s, est. speed input: 69247.51 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2059/3051 [16:14<07:48,  2.12it/s, est. speed input: 69247.62 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2060/3051 [16:14<07:47,  2.12it/s, est. speed input: 69247.67 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2061/3051 [16:15<07:48,  2.12it/s, est. speed input: 69247.59 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2062/3051 [16:15<07:47,  2.11it/s, est. speed input: 69247.59 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2063/3051 [16:16<07:46,  2.12it/s, est. speed input: 69247.69 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2064/3051 [16:16<07:47,  2.11it/s, est. speed input: 69247.60 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2065/3051 [16:17<07:46,  2.11it/s, est. speed input: 69247.54 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2066/3051 [16:17<07:45,  2.11it/s, est. speed input: 69247.64 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2067/3051 [16:18<07:46,  2.11it/s, est. speed input: 69247.33 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2068/3051 [16:18<07:45,  2.11it/s, est. speed input: 69247.33 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2069/3051 [16:19<07:44,  2.11it/s, est. speed input: 69247.48 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2070/3051 [16:19<07:44,  2.11it/s, est. speed input: 69247.40 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2071/3051 [16:20<07:44,  2.11it/s, est. speed input: 69247.29 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2072/3051 [16:20<07:43,  2.11it/s, est. speed input: 69247.38 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2073/3051 [16:20<07:44,  2.10it/s, est. speed input: 69246.92 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2074/3051 [16:21<07:43,  2.11it/s, est. speed input: 69247.04 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2075/3051 [16:21<07:42,  2.11it/s, est. speed input: 69247.11 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2076/3051 [16:22<07:51,  2.07it/s, est. speed input: 69244.64 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2077/3051 [16:22<07:47,  2.08it/s, est. speed input: 69244.83 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2078/3051 [16:23<07:44,  2.10it/s, est. speed input: 69244.98 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2079/3051 [16:23<07:42,  2.10it/s, est. speed input: 69245.09 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2080/3051 [16:24<07:42,  2.10it/s, est. speed input: 69244.79 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2081/3051 [16:24<07:40,  2.10it/s, est. speed input: 69244.82 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2082/3051 [16:26<15:39,  1.03it/s, est. speed input: 69129.06 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2083/3051 [16:27<13:13,  1.22it/s, est. speed input: 69129.35 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2084/3051 [16:27<11:31,  1.40it/s, est. speed input: 69129.56 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2085/3051 [16:28<10:19,  1.56it/s, est. speed input: 69129.79 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2086/3051 [16:28<09:30,  1.69it/s, est. speed input: 69129.91 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2087/3051 [16:29<08:55,  1.80it/s, est. speed input: 69130.01 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2088/3051 [16:29<08:30,  1.88it/s, est. speed input: 69130.12 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2089/3051 [16:30<08:13,  1.95it/s, est. speed input: 69130.25 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2090/3051 [16:30<08:01,  2.00it/s, est. speed input: 69130.39 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2091/3051 [16:31<07:52,  2.03it/s, est. speed input: 69130.49 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2092/3051 [16:31<07:46,  2.06it/s, est. speed input: 69130.59 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2093/3051 [16:32<07:41,  2.07it/s, est. speed input: 69130.72 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2094/3051 [16:32<07:38,  2.09it/s, est. speed input: 69130.80 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2095/3051 [16:33<07:36,  2.10it/s, est. speed input: 69130.90 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2096/3051 [16:33<07:34,  2.10it/s, est. speed input: 69130.99 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2097/3051 [16:33<07:33,  2.11it/s, est. speed input: 69131.09 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2098/3051 [16:34<07:31,  2.11it/s, est. speed input: 69131.21 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2099/3051 [16:34<07:30,  2.11it/s, est. speed input: 69131.35 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2100/3051 [16:35<07:30,  2.11it/s, est. speed input: 69131.45 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2101/3051 [16:35<07:29,  2.11it/s, est. speed input: 69131.54 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2102/3051 [16:36<07:28,  2.11it/s, est. speed input: 69131.64 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2103/3051 [16:36<07:28,  2.12it/s, est. speed input: 69131.75 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2104/3051 [16:37<07:27,  2.12it/s, est. speed input: 69131.90 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2105/3051 [16:37<07:26,  2.12it/s, est. speed input: 69132.03 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2106/3051 [16:38<07:26,  2.12it/s, est. speed input: 69132.11 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2107/3051 [16:38<07:26,  2.12it/s, est. speed input: 69132.21 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2108/3051 [16:39<07:25,  2.11it/s, est. speed input: 69132.23 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2109/3051 [16:39<07:26,  2.11it/s, est. speed input: 69132.17 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2110/3051 [16:40<07:25,  2.11it/s, est. speed input: 69132.25 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2111/3051 [16:40<07:24,  2.11it/s, est. speed input: 69132.32 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2112/3051 [16:41<07:24,  2.11it/s, est. speed input: 69132.41 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2113/3051 [16:41<07:23,  2.11it/s, est. speed input: 69132.51 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2114/3051 [16:42<07:22,  2.12it/s, est. speed input: 69132.63 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2115/3051 [16:42<07:22,  2.12it/s, est. speed input: 69132.74 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2116/3051 [16:42<07:21,  2.12it/s, est. speed input: 69132.86 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2117/3051 [16:43<07:21,  2.12it/s, est. speed input: 69133.01 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2118/3051 [16:43<07:20,  2.12it/s, est. speed input: 69133.11 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2119/3051 [16:44<07:20,  2.12it/s, est. speed input: 69133.21 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2120/3051 [16:44<07:19,  2.12it/s, est. speed input: 69133.35 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2121/3051 [16:45<07:19,  2.12it/s, est. speed input: 69133.43 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2122/3051 [16:45<07:19,  2.11it/s, est. speed input: 69133.45 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2123/3051 [16:46<07:19,  2.11it/s, est. speed input: 69133.46 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2124/3051 [16:46<07:18,  2.11it/s, est. speed input: 69133.56 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2125/3051 [16:47<07:18,  2.11it/s, est. speed input: 69133.49 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2126/3051 [16:47<07:17,  2.11it/s, est. speed input: 69133.60 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2127/3051 [16:48<07:17,  2.11it/s, est. speed input: 69133.69 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2128/3051 [16:48<07:16,  2.12it/s, est. speed input: 69133.81 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2129/3051 [16:49<07:15,  2.12it/s, est. speed input: 69133.89 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2130/3051 [16:49<07:15,  2.12it/s, est. speed input: 69133.97 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2131/3051 [16:50<07:15,  2.11it/s, est. speed input: 69134.04 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2132/3051 [16:50<07:14,  2.11it/s, est. speed input: 69134.06 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2133/3051 [16:50<07:14,  2.11it/s, est. speed input: 69134.11 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2134/3051 [16:51<07:13,  2.11it/s, est. speed input: 69134.19 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2135/3051 [16:51<07:13,  2.11it/s, est. speed input: 69134.20 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2136/3051 [16:52<07:13,  2.11it/s, est. speed input: 69134.25 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2137/3051 [16:52<07:12,  2.11it/s, est. speed input: 69134.29 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2138/3051 [16:53<07:12,  2.11it/s, est. speed input: 69134.33 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2139/3051 [16:53<07:11,  2.11it/s, est. speed input: 69134.38 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2140/3051 [16:54<07:11,  2.11it/s, est. speed input: 69134.44 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2141/3051 [16:54<07:10,  2.11it/s, est. speed input: 69134.51 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2142/3051 [16:55<07:10,  2.11it/s, est. speed input: 69134.58 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2143/3051 [16:55<07:09,  2.12it/s, est. speed input: 69134.72 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2144/3051 [16:56<07:08,  2.12it/s, est. speed input: 69134.80 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2145/3051 [16:56<07:08,  2.11it/s, est. speed input: 69134.87 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2146/3051 [16:57<07:07,  2.11it/s, est. speed input: 69134.94 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2147/3051 [16:57<07:07,  2.12it/s, est. speed input: 69135.05 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2148/3051 [16:58<07:07,  2.11it/s, est. speed input: 69135.08 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2149/3051 [16:58<07:06,  2.11it/s, est. speed input: 69135.16 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2150/3051 [16:59<07:13,  2.08it/s, est. speed input: 69133.26 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2151/3051 [16:59<07:11,  2.09it/s, est. speed input: 69133.33 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2152/3051 [17:00<07:09,  2.10it/s, est. speed input: 69133.38 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2153/3051 [17:00<07:07,  2.10it/s, est. speed input: 69133.49 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2154/3051 [17:00<07:06,  2.11it/s, est. speed input: 69133.55 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2155/3051 [17:01<07:04,  2.11it/s, est. speed input: 69133.64 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2156/3051 [17:01<07:04,  2.11it/s, est. speed input: 69133.71 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2157/3051 [17:02<07:03,  2.11it/s, est. speed input: 69133.76 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2158/3051 [17:02<07:02,  2.11it/s, est. speed input: 69133.86 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2159/3051 [17:03<07:01,  2.11it/s, est. speed input: 69133.98 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2160/3051 [17:03<07:01,  2.11it/s, est. speed input: 69134.04 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2161/3051 [17:04<07:00,  2.12it/s, est. speed input: 69134.16 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2162/3051 [17:04<07:00,  2.11it/s, est. speed input: 69134.21 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2163/3051 [17:05<06:59,  2.12it/s, est. speed input: 69134.31 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2164/3051 [17:05<06:59,  2.12it/s, est. speed input: 69134.40 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2165/3051 [17:06<06:58,  2.12it/s, est. speed input: 69134.49 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2166/3051 [17:06<06:58,  2.11it/s, est. speed input: 69134.50 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2167/3051 [17:07<06:58,  2.11it/s, est. speed input: 69134.58 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2168/3051 [17:07<06:57,  2.11it/s, est. speed input: 69134.60 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2169/3051 [17:08<06:57,  2.11it/s, est. speed input: 69134.73 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2170/3051 [17:08<06:56,  2.11it/s, est. speed input: 69134.76 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2171/3051 [17:08<06:56,  2.11it/s, est. speed input: 69134.84 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2172/3051 [17:09<06:55,  2.11it/s, est. speed input: 69134.90 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2173/3051 [17:09<06:55,  2.11it/s, est. speed input: 69134.98 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2174/3051 [17:10<06:54,  2.11it/s, est. speed input: 69135.02 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2175/3051 [17:10<06:54,  2.11it/s, est. speed input: 69135.09 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2176/3051 [17:11<06:53,  2.11it/s, est. speed input: 69135.18 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2177/3051 [17:11<06:53,  2.12it/s, est. speed input: 69135.29 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2178/3051 [17:12<06:52,  2.12it/s, est. speed input: 69135.38 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2179/3051 [17:12<06:52,  2.11it/s, est. speed input: 69135.31 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2180/3051 [17:13<06:52,  2.11it/s, est. speed input: 69135.38 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2181/3051 [17:13<06:51,  2.11it/s, est. speed input: 69135.46 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2182/3051 [17:14<06:51,  2.11it/s, est. speed input: 69135.47 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2183/3051 [17:14<06:50,  2.11it/s, est. speed input: 69135.52 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2184/3051 [17:15<06:50,  2.11it/s, est. speed input: 69135.63 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2185/3051 [17:15<06:49,  2.11it/s, est. speed input: 69135.68 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2186/3051 [17:16<06:49,  2.11it/s, est. speed input: 69135.76 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2187/3051 [17:16<06:48,  2.12it/s, est. speed input: 69135.86 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2188/3051 [17:17<06:48,  2.12it/s, est. speed input: 69135.95 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2189/3051 [17:17<06:48,  2.11it/s, est. speed input: 69135.82 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2190/3051 [17:17<06:47,  2.11it/s, est. speed input: 69135.80 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2191/3051 [17:18<06:47,  2.11it/s, est. speed input: 69135.92 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2192/3051 [17:18<06:46,  2.11it/s, est. speed input: 69135.88 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2193/3051 [17:19<06:46,  2.11it/s, est. speed input: 69135.87 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2194/3051 [17:19<06:45,  2.11it/s, est. speed input: 69135.90 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2195/3051 [17:20<06:46,  2.10it/s, est. speed input: 69135.55 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2196/3051 [17:20<06:46,  2.11it/s, est. speed input: 69135.55 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2197/3051 [17:21<06:45,  2.11it/s, est. speed input: 69135.62 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2198/3051 [17:21<06:44,  2.11it/s, est. speed input: 69135.67 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2199/3051 [17:22<06:43,  2.11it/s, est. speed input: 69135.76 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2200/3051 [17:22<06:43,  2.11it/s, est. speed input: 69135.79 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2201/3051 [17:23<06:42,  2.11it/s, est. speed input: 69135.87 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2202/3051 [17:23<06:41,  2.11it/s, est. speed input: 69135.95 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2203/3051 [17:24<06:41,  2.11it/s, est. speed input: 69136.03 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2204/3051 [17:24<06:40,  2.11it/s, est. speed input: 69136.06 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2205/3051 [17:25<06:40,  2.11it/s, est. speed input: 69136.14 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2206/3051 [17:25<06:39,  2.11it/s, est. speed input: 69136.17 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2207/3051 [17:26<06:39,  2.11it/s, est. speed input: 69136.29 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2208/3051 [17:26<06:38,  2.11it/s, est. speed input: 69136.28 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2209/3051 [17:26<06:38,  2.11it/s, est. speed input: 69136.34 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2210/3051 [17:27<06:37,  2.11it/s, est. speed input: 69136.38 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2211/3051 [17:27<06:37,  2.11it/s, est. speed input: 69136.48 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2212/3051 [17:28<06:36,  2.11it/s, est. speed input: 69136.55 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2213/3051 [17:28<06:36,  2.11it/s, est. speed input: 69136.62 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2214/3051 [17:29<06:36,  2.11it/s, est. speed input: 69136.65 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2215/3051 [17:29<06:35,  2.11it/s, est. speed input: 69136.69 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2216/3051 [17:30<06:35,  2.11it/s, est. speed input: 69136.71 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2217/3051 [17:30<06:34,  2.11it/s, est. speed input: 69136.77 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2218/3051 [17:31<06:34,  2.11it/s, est. speed input: 69136.80 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2219/3051 [17:31<06:33,  2.11it/s, est. speed input: 69136.87 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2220/3051 [17:32<06:33,  2.11it/s, est. speed input: 69136.90 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2221/3051 [17:32<06:32,  2.11it/s, est. speed input: 69136.97 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2222/3051 [17:33<06:32,  2.11it/s, est. speed input: 69137.02 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2223/3051 [17:33<06:31,  2.11it/s, est. speed input: 69137.12 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2224/3051 [17:34<06:31,  2.11it/s, est. speed input: 69137.11 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2225/3051 [17:34<06:30,  2.11it/s, est. speed input: 69137.18 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2226/3051 [17:35<06:30,  2.11it/s, est. speed input: 69137.27 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2227/3051 [17:35<06:30,  2.11it/s, est. speed input: 69137.07 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2228/3051 [17:35<06:30,  2.11it/s, est. speed input: 69136.97 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2229/3051 [17:36<06:29,  2.11it/s, est. speed input: 69137.04 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2230/3051 [17:36<06:29,  2.11it/s, est. speed input: 69137.04 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2231/3051 [17:37<06:28,  2.11it/s, est. speed input: 69137.09 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2232/3051 [17:37<06:27,  2.11it/s, est. speed input: 69137.11 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2233/3051 [17:38<06:27,  2.11it/s, est. speed input: 69137.20 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2234/3051 [17:38<06:27,  2.11it/s, est. speed input: 69137.13 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2235/3051 [17:39<06:26,  2.11it/s, est. speed input: 69137.15 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2236/3051 [17:39<06:26,  2.11it/s, est. speed input: 69137.12 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2237/3051 [17:40<06:25,  2.11it/s, est. speed input: 69137.17 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2238/3051 [17:40<06:25,  2.11it/s, est. speed input: 69136.98 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2239/3051 [17:41<06:24,  2.11it/s, est. speed input: 69137.06 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2240/3051 [17:41<06:24,  2.11it/s, est. speed input: 69137.14 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2241/3051 [17:42<06:23,  2.11it/s, est. speed input: 69137.13 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2242/3051 [17:42<06:22,  2.11it/s, est. speed input: 69137.25 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2243/3051 [17:43<06:22,  2.11it/s, est. speed input: 69137.31 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2244/3051 [17:43<06:21,  2.11it/s, est. speed input: 69137.39 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2245/3051 [17:44<06:21,  2.11it/s, est. speed input: 69137.43 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2246/3051 [17:44<06:20,  2.11it/s, est. speed input: 69137.50 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2247/3051 [17:44<06:20,  2.11it/s, est. speed input: 69137.50 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2248/3051 [17:45<06:19,  2.11it/s, est. speed input: 69137.58 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2249/3051 [17:45<06:35,  2.03it/s, est. speed input: 69133.30 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2250/3051 [17:46<06:29,  2.06it/s, est. speed input: 69133.51 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2251/3051 [17:46<06:25,  2.08it/s, est. speed input: 69133.69 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2252/3051 [17:47<06:22,  2.09it/s, est. speed input: 69133.74 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2253/3051 [17:47<06:20,  2.10it/s, est. speed input: 69133.86 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2254/3051 [17:48<06:19,  2.10it/s, est. speed input: 69133.95 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2255/3051 [17:48<06:18,  2.10it/s, est. speed input: 69133.90 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2256/3051 [17:49<06:17,  2.11it/s, est. speed input: 69134.02 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2257/3051 [17:49<06:16,  2.11it/s, est. speed input: 69134.01 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2258/3051 [17:50<06:16,  2.11it/s, est. speed input: 69134.02 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2259/3051 [17:50<06:15,  2.11it/s, est. speed input: 69134.06 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2260/3051 [17:51<06:14,  2.11it/s, est. speed input: 69134.16 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2261/3051 [17:51<06:13,  2.11it/s, est. speed input: 69134.32 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2262/3051 [17:52<06:13,  2.11it/s, est. speed input: 69134.22 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2263/3051 [17:52<06:13,  2.11it/s, est. speed input: 69134.11 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2264/3051 [17:53<06:12,  2.11it/s, est. speed input: 69134.23 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2265/3051 [17:53<06:11,  2.12it/s, est. speed input: 69134.45 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2266/3051 [17:54<06:11,  2.12it/s, est. speed input: 69134.51 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2267/3051 [17:54<06:10,  2.12it/s, est. speed input: 69134.60 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2268/3051 [17:54<06:11,  2.11it/s, est. speed input: 69134.43 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2269/3051 [17:55<06:10,  2.11it/s, est. speed input: 69134.54 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2270/3051 [17:55<06:09,  2.12it/s, est. speed input: 69134.72 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2271/3051 [17:56<06:08,  2.12it/s, est. speed input: 69134.80 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2272/3051 [17:56<06:08,  2.11it/s, est. speed input: 69134.80 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2273/3051 [17:57<06:07,  2.11it/s, est. speed input: 69134.91 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2274/3051 [17:57<06:07,  2.12it/s, est. speed input: 69135.07 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2275/3051 [17:58<06:06,  2.12it/s, est. speed input: 69135.10 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2276/3051 [17:58<06:06,  2.12it/s, est. speed input: 69135.19 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2277/3051 [17:59<06:05,  2.12it/s, est. speed input: 69135.25 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2278/3051 [17:59<06:05,  2.11it/s, est. speed input: 69135.19 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2279/3051 [18:00<06:05,  2.11it/s, est. speed input: 69135.33 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2280/3051 [18:00<06:04,  2.11it/s, est. speed input: 69135.31 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2281/3051 [18:01<06:04,  2.11it/s, est. speed input: 69135.31 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2282/3051 [18:01<06:04,  2.11it/s, est. speed input: 69135.36 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2283/3051 [18:02<06:03,  2.11it/s, est. speed input: 69135.46 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2284/3051 [18:02<06:02,  2.11it/s, est. speed input: 69135.52 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2285/3051 [18:03<06:02,  2.11it/s, est. speed input: 69135.60 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2286/3051 [18:03<06:01,  2.12it/s, est. speed input: 69135.70 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2287/3051 [18:03<06:01,  2.12it/s, est. speed input: 69135.80 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2288/3051 [18:04<06:00,  2.12it/s, est. speed input: 69135.87 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2289/3051 [18:04<05:59,  2.12it/s, est. speed input: 69136.03 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2290/3051 [18:05<05:59,  2.12it/s, est. speed input: 69136.18 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2291/3051 [18:05<05:59,  2.11it/s, est. speed input: 69136.03 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2292/3051 [18:06<05:58,  2.11it/s, est. speed input: 69136.17 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2293/3051 [18:06<05:58,  2.12it/s, est. speed input: 69136.32 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2294/3051 [18:07<05:57,  2.11it/s, est. speed input: 69136.34 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2295/3051 [18:07<05:57,  2.11it/s, est. speed input: 69136.38 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2296/3051 [18:08<05:56,  2.11it/s, est. speed input: 69136.48 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2297/3051 [18:08<05:56,  2.11it/s, est. speed input: 69136.47 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2298/3051 [18:09<05:56,  2.11it/s, est. speed input: 69136.47 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2299/3051 [18:09<05:56,  2.11it/s, est. speed input: 69136.45 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2300/3051 [18:10<05:56,  2.11it/s, est. speed input: 69136.26 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2301/3051 [18:10<05:55,  2.11it/s, est. speed input: 69136.32 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2302/3051 [18:11<05:54,  2.11it/s, est. speed input: 69136.48 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2303/3051 [18:11<05:54,  2.11it/s, est. speed input: 69136.43 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2304/3051 [18:12<05:54,  2.11it/s, est. speed input: 69136.36 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2305/3051 [18:12<05:53,  2.11it/s, est. speed input: 69136.45 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2306/3051 [18:12<05:52,  2.11it/s, est. speed input: 69136.60 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2307/3051 [18:13<05:52,  2.11it/s, est. speed input: 69136.50 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2308/3051 [18:13<05:51,  2.11it/s, est. speed input: 69136.65 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2309/3051 [18:14<05:50,  2.12it/s, est. speed input: 69136.79 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2310/3051 [18:14<05:50,  2.12it/s, est. speed input: 69136.89 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2311/3051 [18:15<05:50,  2.11it/s, est. speed input: 69136.76 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2312/3051 [18:15<05:49,  2.11it/s, est. speed input: 69136.93 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2313/3051 [18:16<05:48,  2.12it/s, est. speed input: 69137.07 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2314/3051 [18:16<05:48,  2.11it/s, est. speed input: 69137.10 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2315/3051 [18:17<05:47,  2.12it/s, est. speed input: 69137.19 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2316/3051 [18:17<05:47,  2.12it/s, est. speed input: 69137.39 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2317/3051 [18:18<05:46,  2.12it/s, est. speed input: 69137.54 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2318/3051 [18:18<05:46,  2.12it/s, est. speed input: 69137.59 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2319/3051 [18:19<05:45,  2.12it/s, est. speed input: 69137.64 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2320/3051 [18:19<05:45,  2.12it/s, est. speed input: 69137.68 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2321/3051 [18:20<05:45,  2.12it/s, est. speed input: 69137.79 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2322/3051 [18:20<05:44,  2.12it/s, est. speed input: 69137.96 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2323/3051 [18:20<05:44,  2.11it/s, est. speed input: 69137.93 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2324/3051 [18:21<05:43,  2.12it/s, est. speed input: 69138.02 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2325/3051 [18:21<05:43,  2.12it/s, est. speed input: 69138.16 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2326/3051 [18:22<05:42,  2.12it/s, est. speed input: 69138.26 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2327/3051 [18:22<05:42,  2.12it/s, est. speed input: 69138.35 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2328/3051 [18:23<05:41,  2.12it/s, est. speed input: 69138.37 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2329/3051 [18:23<05:41,  2.12it/s, est. speed input: 69138.47 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2330/3051 [18:24<05:40,  2.11it/s, est. speed input: 69138.52 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2331/3051 [18:24<05:40,  2.12it/s, est. speed input: 69138.68 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2332/3051 [18:25<05:39,  2.12it/s, est. speed input: 69138.78 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2333/3051 [18:25<05:40,  2.11it/s, est. speed input: 69138.62 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2334/3051 [18:26<05:39,  2.11it/s, est. speed input: 69138.78 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2335/3051 [18:26<05:38,  2.11it/s, est. speed input: 69138.74 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2336/3051 [18:27<05:39,  2.11it/s, est. speed input: 69138.59 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2337/3051 [18:27<05:38,  2.11it/s, est. speed input: 69138.70 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2338/3051 [18:28<05:37,  2.11it/s, est. speed input: 69138.83 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2339/3051 [18:28<05:37,  2.11it/s, est. speed input: 69138.83 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2340/3051 [18:29<05:36,  2.11it/s, est. speed input: 69138.94 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2341/3051 [18:29<05:35,  2.11it/s, est. speed input: 69139.02 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2342/3051 [18:29<05:35,  2.11it/s, est. speed input: 69139.09 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2343/3051 [18:30<05:35,  2.11it/s, est. speed input: 69138.87 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2344/3051 [18:30<05:34,  2.11it/s, est. speed input: 69138.96 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2345/3051 [18:31<05:34,  2.11it/s, est. speed input: 69139.08 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2346/3051 [18:31<05:33,  2.12it/s, est. speed input: 69139.22 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2347/3051 [18:32<05:33,  2.11it/s, est. speed input: 69139.12 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2348/3051 [18:32<05:32,  2.12it/s, est. speed input: 69139.31 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2349/3051 [18:33<05:32,  2.11it/s, est. speed input: 69139.21 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2350/3051 [18:33<05:32,  2.11it/s, est. speed input: 69139.18 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2351/3051 [18:34<05:31,  2.11it/s, est. speed input: 69139.37 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2352/3051 [18:34<05:30,  2.12it/s, est. speed input: 69139.58 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2353/3051 [18:35<05:29,  2.12it/s, est. speed input: 69139.71 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2354/3051 [18:35<05:29,  2.12it/s, est. speed input: 69139.85 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2355/3051 [18:36<05:28,  2.12it/s, est. speed input: 69139.91 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2356/3051 [18:36<05:29,  2.11it/s, est. speed input: 69139.71 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2357/3051 [18:37<05:28,  2.11it/s, est. speed input: 69139.77 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2358/3051 [18:37<05:27,  2.11it/s, est. speed input: 69139.90 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2359/3051 [18:38<05:27,  2.11it/s, est. speed input: 69139.93 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2360/3051 [18:38<05:26,  2.11it/s, est. speed input: 69140.05 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2361/3051 [18:38<05:25,  2.12it/s, est. speed input: 69140.23 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2362/3051 [18:39<05:25,  2.12it/s, est. speed input: 69140.30 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2363/3051 [18:39<05:25,  2.12it/s, est. speed input: 69140.35 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2364/3051 [18:40<05:24,  2.12it/s, est. speed input: 69140.43 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2365/3051 [18:40<05:24,  2.12it/s, est. speed input: 69140.50 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2366/3051 [18:41<05:24,  2.11it/s, est. speed input: 69140.49 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2367/3051 [18:41<05:23,  2.11it/s, est. speed input: 69140.60 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2368/3051 [18:42<05:23,  2.11it/s, est. speed input: 69140.63 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2369/3051 [18:42<05:22,  2.11it/s, est. speed input: 69140.70 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2370/3051 [18:43<05:21,  2.12it/s, est. speed input: 69140.80 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2371/3051 [18:43<05:21,  2.12it/s, est. speed input: 69140.95 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2372/3051 [18:44<05:20,  2.12it/s, est. speed input: 69141.03 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2373/3051 [18:44<05:21,  2.11it/s, est. speed input: 69140.86 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2374/3051 [18:45<05:20,  2.11it/s, est. speed input: 69140.81 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2375/3051 [18:45<05:20,  2.11it/s, est. speed input: 69140.80 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2376/3051 [18:46<05:19,  2.11it/s, est. speed input: 69140.93 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2377/3051 [18:46<05:18,  2.11it/s, est. speed input: 69141.03 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2378/3051 [18:47<05:18,  2.12it/s, est. speed input: 69141.17 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2379/3051 [18:47<05:17,  2.12it/s, est. speed input: 69141.34 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2380/3051 [18:47<05:17,  2.12it/s, est. speed input: 69141.33 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2381/3051 [18:48<05:17,  2.11it/s, est. speed input: 69141.26 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2382/3051 [18:48<05:16,  2.11it/s, est. speed input: 69141.35 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2383/3051 [18:49<05:15,  2.12it/s, est. speed input: 69141.49 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2384/3051 [18:49<05:15,  2.12it/s, est. speed input: 69141.59 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2385/3051 [18:50<05:14,  2.12it/s, est. speed input: 69141.69 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2386/3051 [18:50<05:14,  2.12it/s, est. speed input: 69141.81 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2387/3051 [18:51<05:13,  2.12it/s, est. speed input: 69141.94 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2388/3051 [18:51<05:14,  2.11it/s, est. speed input: 69141.74 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2389/3051 [18:52<05:13,  2.11it/s, est. speed input: 69141.79 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2390/3051 [18:52<05:12,  2.11it/s, est. speed input: 69141.93 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2391/3051 [18:53<05:12,  2.11it/s, est. speed input: 69141.99 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2392/3051 [18:53<05:12,  2.11it/s, est. speed input: 69141.70 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2393/3051 [18:54<05:11,  2.11it/s, est. speed input: 69141.78 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2394/3051 [18:54<05:11,  2.11it/s, est. speed input: 69141.90 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2395/3051 [18:55<05:10,  2.11it/s, est. speed input: 69142.01 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2396/3051 [18:55<05:09,  2.11it/s, est. speed input: 69142.03 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2397/3051 [18:55<05:09,  2.11it/s, est. speed input: 69141.97 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2398/3051 [18:56<05:09,  2.11it/s, est. speed input: 69141.96 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2399/3051 [18:56<05:08,  2.11it/s, est. speed input: 69142.13 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2400/3051 [18:57<05:07,  2.11it/s, est. speed input: 69142.21 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2401/3051 [18:57<05:07,  2.11it/s, est. speed input: 69142.26 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2402/3051 [18:58<05:07,  2.11it/s, est. speed input: 69142.19 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2403/3051 [18:58<05:06,  2.11it/s, est. speed input: 69142.34 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2404/3051 [18:59<05:06,  2.11it/s, est. speed input: 69142.36 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2405/3051 [18:59<05:05,  2.11it/s, est. speed input: 69142.35 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2406/3051 [19:00<05:05,  2.11it/s, est. speed input: 69142.47 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2407/3051 [19:00<05:05,  2.11it/s, est. speed input: 69142.36 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2408/3051 [19:01<05:04,  2.11it/s, est. speed input: 69142.30 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2409/3051 [19:01<05:04,  2.11it/s, est. speed input: 69142.40 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2410/3051 [19:02<05:03,  2.11it/s, est. speed input: 69142.45 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2411/3051 [19:02<05:03,  2.11it/s, est. speed input: 69142.46 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2412/3051 [19:03<05:02,  2.11it/s, est. speed input: 69142.58 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2413/3051 [19:03<05:02,  2.11it/s, est. speed input: 69142.46 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2414/3051 [19:04<05:01,  2.11it/s, est. speed input: 69142.52 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2415/3051 [19:04<05:00,  2.12it/s, est. speed input: 69142.71 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2416/3051 [19:04<04:59,  2.12it/s, est. speed input: 69142.86 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2417/3051 [19:05<05:00,  2.11it/s, est. speed input: 69142.74 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2418/3051 [19:05<04:59,  2.11it/s, est. speed input: 69142.84 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2419/3051 [19:06<04:58,  2.12it/s, est. speed input: 69142.98 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2420/3051 [19:06<04:58,  2.11it/s, est. speed input: 69142.90 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2421/3051 [19:07<04:58,  2.11it/s, est. speed input: 69142.97 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2422/3051 [19:07<04:57,  2.11it/s, est. speed input: 69143.06 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2423/3051 [19:08<04:57,  2.11it/s, est. speed input: 69143.04 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2424/3051 [19:08<04:56,  2.12it/s, est. speed input: 69143.23 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2425/3051 [19:09<04:55,  2.12it/s, est. speed input: 69143.32 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2426/3051 [19:09<04:55,  2.11it/s, est. speed input: 69143.22 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2427/3051 [19:10<04:55,  2.11it/s, est. speed input: 69143.35 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2428/3051 [19:10<04:54,  2.12it/s, est. speed input: 69143.45 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2429/3051 [19:11<04:53,  2.12it/s, est. speed input: 69143.54 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2430/3051 [19:11<04:53,  2.12it/s, est. speed input: 69143.59 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2431/3051 [19:12<04:53,  2.12it/s, est. speed input: 69143.68 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2432/3051 [19:12<04:52,  2.12it/s, est. speed input: 69143.81 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2433/3051 [19:13<04:51,  2.12it/s, est. speed input: 69143.90 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2434/3051 [19:13<04:51,  2.12it/s, est. speed input: 69143.96 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2435/3051 [19:13<04:50,  2.12it/s, est. speed input: 69144.10 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2436/3051 [19:14<04:50,  2.11it/s, est. speed input: 69144.03 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2437/3051 [19:14<04:50,  2.11it/s, est. speed input: 69144.10 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2438/3051 [19:15<04:49,  2.12it/s, est. speed input: 69144.27 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2439/3051 [19:15<04:49,  2.11it/s, est. speed input: 69144.14 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2440/3051 [19:16<04:49,  2.11it/s, est. speed input: 69144.17 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2441/3051 [19:16<04:48,  2.11it/s, est. speed input: 69144.31 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2442/3051 [19:17<04:48,  2.11it/s, est. speed input: 69144.32 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2443/3051 [19:17<04:47,  2.11it/s, est. speed input: 69144.37 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2444/3051 [19:18<04:47,  2.11it/s, est. speed input: 69144.46 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2445/3051 [19:18<04:46,  2.12it/s, est. speed input: 69144.57 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2446/3051 [19:19<04:46,  2.11it/s, est. speed input: 69144.59 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2447/3051 [19:19<04:45,  2.11it/s, est. speed input: 69144.55 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2448/3051 [19:20<04:45,  2.11it/s, est. speed input: 69144.62 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2449/3051 [19:20<04:44,  2.11it/s, est. speed input: 69144.71 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2450/3051 [19:21<04:44,  2.11it/s, est. speed input: 69144.70 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2451/3051 [19:21<04:43,  2.12it/s, est. speed input: 69144.87 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2452/3051 [19:22<04:43,  2.11it/s, est. speed input: 69144.90 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2453/3051 [19:22<04:43,  2.11it/s, est. speed input: 69144.68 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2454/3051 [19:22<04:42,  2.11it/s, est. speed input: 69144.88 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2455/3051 [19:23<04:42,  2.11it/s, est. speed input: 69144.69 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2456/3051 [19:23<04:42,  2.11it/s, est. speed input: 69144.72 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2457/3051 [19:24<04:41,  2.11it/s, est. speed input: 69144.90 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2458/3051 [19:24<04:40,  2.12it/s, est. speed input: 69145.10 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2459/3051 [19:25<04:39,  2.12it/s, est. speed input: 69145.21 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2460/3051 [19:25<04:39,  2.12it/s, est. speed input: 69145.27 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2461/3051 [19:26<04:38,  2.12it/s, est. speed input: 69145.29 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2462/3051 [19:26<04:38,  2.12it/s, est. speed input: 69145.36 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2463/3051 [19:27<04:37,  2.12it/s, est. speed input: 69145.45 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2464/3051 [19:27<04:37,  2.12it/s, est. speed input: 69145.52 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2465/3051 [19:28<04:37,  2.11it/s, est. speed input: 69145.50 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2466/3051 [19:28<04:36,  2.11it/s, est. speed input: 69145.55 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2467/3051 [19:29<04:36,  2.12it/s, est. speed input: 69145.68 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2468/3051 [19:29<04:36,  2.11it/s, est. speed input: 69145.37 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2469/3051 [19:30<04:35,  2.11it/s, est. speed input: 69145.47 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2470/3051 [19:30<04:35,  2.11it/s, est. speed input: 69145.56 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2471/3051 [19:31<04:34,  2.11it/s, est. speed input: 69145.51 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2472/3051 [19:31<04:34,  2.11it/s, est. speed input: 69145.57 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2473/3051 [19:31<04:33,  2.11it/s, est. speed input: 69145.57 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2474/3051 [19:32<04:33,  2.11it/s, est. speed input: 69145.49 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2475/3051 [19:32<04:33,  2.11it/s, est. speed input: 69145.48 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2476/3051 [19:33<04:32,  2.11it/s, est. speed input: 69145.56 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2477/3051 [19:33<04:31,  2.12it/s, est. speed input: 69145.76 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2478/3051 [19:34<04:30,  2.12it/s, est. speed input: 69145.85 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2479/3051 [19:34<04:31,  2.11it/s, est. speed input: 69145.62 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2480/3051 [19:35<04:30,  2.11it/s, est. speed input: 69145.77 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2481/3051 [19:35<04:29,  2.11it/s, est. speed input: 69145.89 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2482/3051 [19:36<04:29,  2.11it/s, est. speed input: 69145.94 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2483/3051 [19:36<04:28,  2.11it/s, est. speed input: 69146.02 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2484/3051 [19:37<04:27,  2.12it/s, est. speed input: 69146.13 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2485/3051 [19:37<04:27,  2.12it/s, est. speed input: 69146.18 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2486/3051 [19:38<04:26,  2.12it/s, est. speed input: 69146.29 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2487/3051 [19:38<04:34,  2.06it/s, est. speed input: 69143.73 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2488/3051 [19:39<04:31,  2.08it/s, est. speed input: 69143.89 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2489/3051 [19:39<04:28,  2.09it/s, est. speed input: 69144.01 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2490/3051 [19:40<04:28,  2.09it/s, est. speed input: 69143.90 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2491/3051 [19:40<04:26,  2.10it/s, est. speed input: 69143.97 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2492/3051 [19:40<04:25,  2.10it/s, est. speed input: 69144.06 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2493/3051 [19:41<04:24,  2.11it/s, est. speed input: 69144.19 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2494/3051 [19:41<04:23,  2.11it/s, est. speed input: 69144.32 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2495/3051 [19:42<04:23,  2.11it/s, est. speed input: 69144.39 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2496/3051 [19:42<04:22,  2.11it/s, est. speed input: 69144.47 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2497/3051 [19:43<04:22,  2.11it/s, est. speed input: 69144.42 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2498/3051 [19:43<04:21,  2.11it/s, est. speed input: 69144.50 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2499/3051 [19:44<04:20,  2.12it/s, est. speed input: 69144.65 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2500/3051 [19:44<04:20,  2.12it/s, est. speed input: 69144.71 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2501/3051 [19:45<04:19,  2.12it/s, est. speed input: 69144.78 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2502/3051 [19:45<04:19,  2.12it/s, est. speed input: 69144.86 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2503/3051 [19:46<04:19,  2.11it/s, est. speed input: 69144.65 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2504/3051 [19:46<04:19,  2.11it/s, est. speed input: 69144.66 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2505/3051 [19:47<04:18,  2.11it/s, est. speed input: 69144.82 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2506/3051 [19:47<04:18,  2.11it/s, est. speed input: 69144.78 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2507/3051 [19:48<04:17,  2.11it/s, est. speed input: 69144.84 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2508/3051 [19:48<04:16,  2.11it/s, est. speed input: 69144.94 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2509/3051 [19:49<04:16,  2.11it/s, est. speed input: 69145.02 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2510/3051 [19:49<04:16,  2.11it/s, est. speed input: 69144.94 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2511/3051 [19:49<04:15,  2.11it/s, est. speed input: 69145.09 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2512/3051 [19:50<04:14,  2.12it/s, est. speed input: 69145.24 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2513/3051 [19:50<04:14,  2.11it/s, est. speed input: 69145.15 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2514/3051 [19:51<04:14,  2.11it/s, est. speed input: 69145.23 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2515/3051 [19:51<04:13,  2.11it/s, est. speed input: 69145.25 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2516/3051 [19:52<04:13,  2.11it/s, est. speed input: 69145.21 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2517/3051 [19:52<04:12,  2.11it/s, est. speed input: 69145.29 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2518/3051 [19:53<04:12,  2.11it/s, est. speed input: 69145.41 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2519/3051 [19:53<04:11,  2.12it/s, est. speed input: 69145.56 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2520/3051 [19:54<04:11,  2.11it/s, est. speed input: 69145.37 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2521/3051 [19:54<04:10,  2.11it/s, est. speed input: 69145.48 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2522/3051 [19:55<04:10,  2.11it/s, est. speed input: 69145.60 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2523/3051 [19:55<04:09,  2.11it/s, est. speed input: 69145.61 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2524/3051 [19:56<04:09,  2.12it/s, est. speed input: 69145.71 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2525/3051 [19:56<04:08,  2.12it/s, est. speed input: 69145.85 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2526/3051 [19:57<04:07,  2.12it/s, est. speed input: 69145.97 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2527/3051 [19:57<04:08,  2.11it/s, est. speed input: 69145.80 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2528/3051 [19:58<04:07,  2.11it/s, est. speed input: 69145.89 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2529/3051 [19:58<04:06,  2.12it/s, est. speed input: 69146.02 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2530/3051 [19:58<04:06,  2.12it/s, est. speed input: 69146.11 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2531/3051 [19:59<04:05,  2.12it/s, est. speed input: 69146.23 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2532/3051 [19:59<04:05,  2.11it/s, est. speed input: 69146.16 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2533/3051 [20:00<04:05,  2.11it/s, est. speed input: 69146.20 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2534/3051 [20:00<04:04,  2.11it/s, est. speed input: 69146.30 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2535/3051 [20:01<04:04,  2.11it/s, est. speed input: 69146.31 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2536/3051 [20:01<04:03,  2.11it/s, est. speed input: 69146.37 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2537/3051 [20:02<04:02,  2.12it/s, est. speed input: 69146.54 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2538/3051 [20:02<04:02,  2.12it/s, est. speed input: 69146.68 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2539/3051 [20:03<04:02,  2.11it/s, est. speed input: 69146.61 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2540/3051 [20:03<04:01,  2.12it/s, est. speed input: 69146.73 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2541/3051 [20:04<04:01,  2.11it/s, est. speed input: 69146.75 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2542/3051 [20:04<04:00,  2.11it/s, est. speed input: 69146.80 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2543/3051 [20:05<04:00,  2.11it/s, est. speed input: 69146.87 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2544/3051 [20:05<03:59,  2.12it/s, est. speed input: 69147.03 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2545/3051 [20:06<03:59,  2.12it/s, est. speed input: 69147.08 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2546/3051 [20:06<03:58,  2.12it/s, est. speed input: 69147.15 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2547/3051 [20:06<03:58,  2.12it/s, est. speed input: 69147.24 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2548/3051 [20:07<03:57,  2.12it/s, est. speed input: 69147.31 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2549/3051 [20:07<03:57,  2.12it/s, est. speed input: 69147.37 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2550/3051 [20:08<03:57,  2.11it/s, est. speed input: 69147.29 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2551/3051 [20:08<03:56,  2.11it/s, est. speed input: 69147.32 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2552/3051 [20:09<03:56,  2.11it/s, est. speed input: 69147.35 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2553/3051 [20:09<03:55,  2.11it/s, est. speed input: 69147.38 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2554/3051 [20:10<03:55,  2.11it/s, est. speed input: 69147.51 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2555/3051 [20:10<03:54,  2.11it/s, est. speed input: 69147.51 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2556/3051 [20:11<03:54,  2.11it/s, est. speed input: 69147.55 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2557/3051 [20:11<03:53,  2.12it/s, est. speed input: 69147.69 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2558/3051 [20:12<03:53,  2.11it/s, est. speed input: 69147.62 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2559/3051 [20:12<03:52,  2.11it/s, est. speed input: 69147.69 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2560/3051 [20:13<03:52,  2.11it/s, est. speed input: 69147.73 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2561/3051 [20:13<03:51,  2.11it/s, est. speed input: 69147.77 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2562/3051 [20:14<03:51,  2.11it/s, est. speed input: 69147.78 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2563/3051 [20:14<03:50,  2.11it/s, est. speed input: 69147.89 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2564/3051 [20:15<03:50,  2.12it/s, est. speed input: 69148.01 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2565/3051 [20:15<03:50,  2.11it/s, est. speed input: 69147.91 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2566/3051 [20:15<03:49,  2.11it/s, est. speed input: 69147.97 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2567/3051 [20:16<03:48,  2.12it/s, est. speed input: 69148.10 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2568/3051 [20:16<03:48,  2.11it/s, est. speed input: 69148.09 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2569/3051 [20:17<03:47,  2.12it/s, est. speed input: 69148.24 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2570/3051 [20:17<03:47,  2.12it/s, est. speed input: 69148.38 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2571/3051 [20:18<03:46,  2.12it/s, est. speed input: 69148.49 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2572/3051 [20:18<03:46,  2.12it/s, est. speed input: 69148.56 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2573/3051 [20:19<03:46,  2.11it/s, est. speed input: 69148.43 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2574/3051 [20:19<03:45,  2.11it/s, est. speed input: 69148.49 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2575/3051 [20:20<03:45,  2.11it/s, est. speed input: 69148.52 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2576/3051 [20:20<03:44,  2.11it/s, est. speed input: 69148.64 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2577/3051 [20:21<03:44,  2.11it/s, est. speed input: 69148.66 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2578/3051 [20:21<03:43,  2.11it/s, est. speed input: 69148.73 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2579/3051 [20:22<03:43,  2.12it/s, est. speed input: 69148.82 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2580/3051 [20:22<03:42,  2.12it/s, est. speed input: 69148.95 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2581/3051 [20:23<03:41,  2.12it/s, est. speed input: 69149.05 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2582/3051 [20:23<03:41,  2.11it/s, est. speed input: 69149.04 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2583/3051 [20:24<03:41,  2.12it/s, est. speed input: 69149.14 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2584/3051 [20:24<03:41,  2.11it/s, est. speed input: 69148.90 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2585/3051 [20:24<03:40,  2.11it/s, est. speed input: 69148.96 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2586/3051 [20:25<03:40,  2.11it/s, est. speed input: 69149.10 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2587/3051 [20:25<03:39,  2.12it/s, est. speed input: 69149.28 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2588/3051 [20:26<03:39,  2.11it/s, est. speed input: 69149.23 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2589/3051 [20:26<03:38,  2.12it/s, est. speed input: 69149.38 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2590/3051 [20:27<03:38,  2.11it/s, est. speed input: 69149.34 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2591/3051 [20:27<03:37,  2.11it/s, est. speed input: 69149.42 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2592/3051 [20:28<03:37,  2.11it/s, est. speed input: 69149.47 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2593/3051 [20:28<03:36,  2.12it/s, est. speed input: 69149.59 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2594/3051 [20:29<03:35,  2.12it/s, est. speed input: 69149.68 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2595/3051 [20:29<03:35,  2.11it/s, est. speed input: 69149.58 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2596/3051 [20:30<03:35,  2.11it/s, est. speed input: 69149.59 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2597/3051 [20:30<03:34,  2.11it/s, est. speed input: 69149.67 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2598/3051 [20:31<03:34,  2.11it/s, est. speed input: 69149.44 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2599/3051 [20:31<03:34,  2.11it/s, est. speed input: 69149.54 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2600/3051 [20:32<03:33,  2.11it/s, est. speed input: 69149.64 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2601/3051 [20:32<03:32,  2.12it/s, est. speed input: 69149.77 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2602/3051 [20:33<03:32,  2.12it/s, est. speed input: 69149.94 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2603/3051 [20:33<03:31,  2.12it/s, est. speed input: 69149.96 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2604/3051 [20:33<03:31,  2.12it/s, est. speed input: 69150.01 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2605/3051 [20:34<03:30,  2.12it/s, est. speed input: 69150.08 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2606/3051 [20:34<03:30,  2.12it/s, est. speed input: 69150.23 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2607/3051 [20:35<03:30,  2.11it/s, est. speed input: 69150.16 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2608/3051 [20:35<03:29,  2.11it/s, est. speed input: 69150.23 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2609/3051 [20:36<03:28,  2.12it/s, est. speed input: 69150.35 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2610/3051 [20:36<03:28,  2.11it/s, est. speed input: 69150.35 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2611/3051 [20:37<03:28,  2.12it/s, est. speed input: 69150.43 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2612/3051 [20:37<03:27,  2.12it/s, est. speed input: 69150.55 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2613/3051 [20:38<03:27,  2.11it/s, est. speed input: 69150.36 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2614/3051 [20:38<03:26,  2.11it/s, est. speed input: 69150.42 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2615/3051 [20:39<03:26,  2.11it/s, est. speed input: 69150.52 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2616/3051 [20:39<03:26,  2.11it/s, est. speed input: 69150.48 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2617/3051 [20:40<03:25,  2.11it/s, est. speed input: 69150.55 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2618/3051 [20:40<03:24,  2.11it/s, est. speed input: 69150.65 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2619/3051 [20:41<03:24,  2.12it/s, est. speed input: 69150.73 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2620/3051 [20:41<03:23,  2.11it/s, est. speed input: 69150.78 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2621/3051 [20:41<03:23,  2.11it/s, est. speed input: 69150.82 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2622/3051 [20:42<03:22,  2.12it/s, est. speed input: 69150.94 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2623/3051 [20:42<03:22,  2.11it/s, est. speed input: 69150.82 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2624/3051 [20:43<03:21,  2.11it/s, est. speed input: 69150.96 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2625/3051 [20:43<03:21,  2.11it/s, est. speed input: 69150.88 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2626/3051 [20:44<03:21,  2.11it/s, est. speed input: 69150.80 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2627/3051 [20:44<03:21,  2.11it/s, est. speed input: 69150.75 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2628/3051 [20:45<03:20,  2.11it/s, est. speed input: 69150.85 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2629/3051 [20:45<03:20,  2.11it/s, est. speed input: 69150.64 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2630/3051 [20:46<03:19,  2.11it/s, est. speed input: 69150.59 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2631/3051 [20:46<03:19,  2.11it/s, est. speed input: 69150.68 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2632/3051 [20:47<03:18,  2.11it/s, est. speed input: 69150.82 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2633/3051 [20:47<03:18,  2.11it/s, est. speed input: 69150.73 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2634/3051 [20:48<03:17,  2.11it/s, est. speed input: 69150.90 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2635/3051 [20:48<03:17,  2.11it/s, est. speed input: 69150.84 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2636/3051 [20:49<03:16,  2.11it/s, est. speed input: 69150.89 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2637/3051 [20:49<03:15,  2.11it/s, est. speed input: 69150.92 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2638/3051 [20:50<03:15,  2.11it/s, est. speed input: 69151.03 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2639/3051 [20:50<03:14,  2.12it/s, est. speed input: 69151.12 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2640/3051 [20:50<03:14,  2.12it/s, est. speed input: 69151.17 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2641/3051 [20:51<03:13,  2.12it/s, est. speed input: 69151.24 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2642/3051 [20:51<03:13,  2.11it/s, est. speed input: 69151.24 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2643/3051 [20:52<03:12,  2.11it/s, est. speed input: 69151.33 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2644/3051 [20:52<03:12,  2.12it/s, est. speed input: 69151.43 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2645/3051 [20:53<03:11,  2.11it/s, est. speed input: 69151.45 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2646/3051 [20:53<03:11,  2.11it/s, est. speed input: 69151.51 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2647/3051 [20:54<03:11,  2.11it/s, est. speed input: 69151.53 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2648/3051 [20:54<03:10,  2.11it/s, est. speed input: 69151.59 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2649/3051 [20:55<03:10,  2.11it/s, est. speed input: 69151.64 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2650/3051 [20:55<03:09,  2.12it/s, est. speed input: 69151.73 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2651/3051 [20:56<03:09,  2.12it/s, est. speed input: 69151.80 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2652/3051 [20:56<03:09,  2.11it/s, est. speed input: 69151.61 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2653/3051 [20:57<03:08,  2.11it/s, est. speed input: 69151.66 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2654/3051 [20:57<03:07,  2.11it/s, est. speed input: 69151.80 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2655/3051 [20:58<03:07,  2.11it/s, est. speed input: 69151.76 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2656/3051 [20:58<03:06,  2.11it/s, est. speed input: 69151.86 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2657/3051 [20:59<03:06,  2.12it/s, est. speed input: 69151.96 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2658/3051 [20:59<03:05,  2.11it/s, est. speed input: 69151.93 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2659/3051 [20:59<03:05,  2.11it/s, est. speed input: 69151.99 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2660/3051 [21:00<03:04,  2.11it/s, est. speed input: 69152.03 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2661/3051 [21:00<03:04,  2.11it/s, est. speed input: 69152.08 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2662/3051 [21:01<03:04,  2.11it/s, est. speed input: 69152.12 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2663/3051 [21:01<03:03,  2.12it/s, est. speed input: 69152.24 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2664/3051 [21:02<03:02,  2.12it/s, est. speed input: 69152.34 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2665/3051 [21:02<03:02,  2.12it/s, est. speed input: 69152.41 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2666/3051 [21:03<03:01,  2.12it/s, est. speed input: 69152.52 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2667/3051 [21:03<03:01,  2.11it/s, est. speed input: 69152.51 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2668/3051 [21:04<03:01,  2.11it/s, est. speed input: 69152.51 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2669/3051 [21:04<03:00,  2.11it/s, est. speed input: 69152.52 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2670/3051 [21:05<03:00,  2.11it/s, est. speed input: 69152.43 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2671/3051 [21:05<03:00,  2.11it/s, est. speed input: 69152.41 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2672/3051 [21:06<02:59,  2.11it/s, est. speed input: 69152.54 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2673/3051 [21:06<02:58,  2.12it/s, est. speed input: 69152.66 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2674/3051 [21:07<02:58,  2.11it/s, est. speed input: 69152.56 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2675/3051 [21:07<02:58,  2.11it/s, est. speed input: 69152.59 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2676/3051 [21:08<02:57,  2.11it/s, est. speed input: 69152.68 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2677/3051 [21:08<02:56,  2.12it/s, est. speed input: 69152.80 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2678/3051 [21:08<02:56,  2.11it/s, est. speed input: 69152.84 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2679/3051 [21:09<02:55,  2.12it/s, est. speed input: 69152.91 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2680/3051 [21:09<02:55,  2.11it/s, est. speed input: 69152.95 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2681/3051 [21:10<02:54,  2.11it/s, est. speed input: 69153.00 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2682/3051 [21:10<02:54,  2.12it/s, est. speed input: 69153.07 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2683/3051 [21:11<02:53,  2.12it/s, est. speed input: 69153.16 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2684/3051 [21:11<02:53,  2.12it/s, est. speed input: 69153.23 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2685/3051 [21:12<02:52,  2.12it/s, est. speed input: 69153.30 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2686/3051 [21:12<02:52,  2.12it/s, est. speed input: 69153.42 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2687/3051 [21:13<02:52,  2.12it/s, est. speed input: 69153.45 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2688/3051 [21:13<02:51,  2.11it/s, est. speed input: 69153.44 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2689/3051 [21:14<02:51,  2.12it/s, est. speed input: 69153.57 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2690/3051 [21:14<02:50,  2.12it/s, est. speed input: 69153.60 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2691/3051 [21:15<02:50,  2.11it/s, est. speed input: 69153.64 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2692/3051 [21:15<02:49,  2.11it/s, est. speed input: 69153.68 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2693/3051 [21:16<02:49,  2.12it/s, est. speed input: 69153.83 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2694/3051 [21:16<02:48,  2.11it/s, est. speed input: 69153.83 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2695/3051 [21:17<02:48,  2.11it/s, est. speed input: 69153.81 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2696/3051 [21:17<02:47,  2.11it/s, est. speed input: 69153.87 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2697/3051 [21:17<02:47,  2.11it/s, est. speed input: 69153.88 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2698/3051 [21:18<02:47,  2.11it/s, est. speed input: 69153.94 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2699/3051 [21:18<02:46,  2.11it/s, est. speed input: 69154.02 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2700/3051 [21:19<02:46,  2.11it/s, est. speed input: 69153.99 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2701/3051 [21:19<02:45,  2.11it/s, est. speed input: 69154.05 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2702/3051 [21:20<02:45,  2.11it/s, est. speed input: 69154.13 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2703/3051 [21:20<02:44,  2.11it/s, est. speed input: 69154.07 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2704/3051 [21:21<02:44,  2.11it/s, est. speed input: 69154.15 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2705/3051 [21:21<02:43,  2.11it/s, est. speed input: 69154.19 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2706/3051 [21:22<02:43,  2.11it/s, est. speed input: 69154.26 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2707/3051 [21:22<02:42,  2.12it/s, est. speed input: 69154.36 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2708/3051 [21:23<02:42,  2.12it/s, est. speed input: 69154.47 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2709/3051 [21:23<02:41,  2.12it/s, est. speed input: 69154.58 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2710/3051 [21:24<02:41,  2.11it/s, est. speed input: 69154.53 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2711/3051 [21:24<02:40,  2.11it/s, est. speed input: 69154.57 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2712/3051 [21:25<02:40,  2.12it/s, est. speed input: 69154.68 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2713/3051 [21:25<02:39,  2.12it/s, est. speed input: 69154.75 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2714/3051 [21:25<02:39,  2.12it/s, est. speed input: 69154.83 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2715/3051 [21:26<02:38,  2.12it/s, est. speed input: 69154.85 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2716/3051 [21:26<02:38,  2.12it/s, est. speed input: 69154.92 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2717/3051 [21:27<02:37,  2.12it/s, est. speed input: 69154.97 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2718/3051 [21:27<02:37,  2.12it/s, est. speed input: 69155.03 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2719/3051 [21:28<02:37,  2.11it/s, est. speed input: 69155.06 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2720/3051 [21:28<02:36,  2.11it/s, est. speed input: 69155.05 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2721/3051 [21:29<02:36,  2.11it/s, est. speed input: 69155.12 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2722/3051 [21:29<02:35,  2.11it/s, est. speed input: 69155.16 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2723/3051 [21:30<02:35,  2.11it/s, est. speed input: 69155.06 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2724/3051 [21:30<02:34,  2.11it/s, est. speed input: 69155.16 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2725/3051 [21:31<02:34,  2.11it/s, est. speed input: 69155.25 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2726/3051 [21:31<02:33,  2.11it/s, est. speed input: 69155.31 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2727/3051 [21:32<02:33,  2.11it/s, est. speed input: 69155.35 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2728/3051 [21:32<02:32,  2.12it/s, est. speed input: 69155.45 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2729/3051 [21:33<02:32,  2.12it/s, est. speed input: 69155.53 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2730/3051 [21:33<02:31,  2.11it/s, est. speed input: 69155.43 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2731/3051 [21:34<02:31,  2.11it/s, est. speed input: 69155.50 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2732/3051 [21:34<02:30,  2.12it/s, est. speed input: 69155.61 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2733/3051 [21:34<02:30,  2.11it/s, est. speed input: 69155.59 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2734/3051 [21:35<02:29,  2.11it/s, est. speed input: 69155.63 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2735/3051 [21:35<02:29,  2.12it/s, est. speed input: 69155.74 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2736/3051 [21:36<02:28,  2.12it/s, est. speed input: 69155.88 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2737/3051 [21:36<02:28,  2.12it/s, est. speed input: 69155.90 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2738/3051 [21:37<02:28,  2.11it/s, est. speed input: 69155.90 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2739/3051 [21:37<02:27,  2.11it/s, est. speed input: 69155.96 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2740/3051 [21:38<02:27,  2.11it/s, est. speed input: 69156.02 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2741/3051 [21:38<02:26,  2.12it/s, est. speed input: 69156.18 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2742/3051 [21:39<02:26,  2.11it/s, est. speed input: 69156.10 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2743/3051 [21:39<02:26,  2.11it/s, est. speed input: 69155.97 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2744/3051 [21:40<02:25,  2.11it/s, est. speed input: 69156.08 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2745/3051 [21:40<02:25,  2.11it/s, est. speed input: 69155.94 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2746/3051 [21:41<02:24,  2.11it/s, est. speed input: 69155.96 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2747/3051 [21:41<02:23,  2.11it/s, est. speed input: 69156.11 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2748/3051 [21:42<02:23,  2.11it/s, est. speed input: 69156.18 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2749/3051 [21:42<02:22,  2.12it/s, est. speed input: 69156.27 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2750/3051 [21:43<02:22,  2.12it/s, est. speed input: 69156.35 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2751/3051 [21:43<02:21,  2.12it/s, est. speed input: 69156.43 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2752/3051 [21:43<02:21,  2.12it/s, est. speed input: 69156.54 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2753/3051 [21:44<02:20,  2.12it/s, est. speed input: 69156.59 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2754/3051 [21:44<02:20,  2.12it/s, est. speed input: 69156.72 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2755/3051 [21:45<02:19,  2.12it/s, est. speed input: 69156.72 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2756/3051 [21:45<02:19,  2.12it/s, est. speed input: 69156.82 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2757/3051 [21:46<02:18,  2.12it/s, est. speed input: 69156.87 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2758/3051 [21:46<02:18,  2.12it/s, est. speed input: 69156.92 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2759/3051 [21:47<02:18,  2.11it/s, est. speed input: 69156.93 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2760/3051 [21:47<02:17,  2.12it/s, est. speed input: 69157.02 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2761/3051 [21:48<02:17,  2.12it/s, est. speed input: 69157.06 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2762/3051 [21:48<02:17,  2.11it/s, est. speed input: 69156.84 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2763/3051 [21:49<02:16,  2.11it/s, est. speed input: 69156.94 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2764/3051 [21:49<02:15,  2.11it/s, est. speed input: 69156.94 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2765/3051 [21:50<02:15,  2.11it/s, est. speed input: 69157.02 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2766/3051 [21:50<02:14,  2.11it/s, est. speed input: 69157.01 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2767/3051 [21:51<02:14,  2.11it/s, est. speed input: 69157.07 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2768/3051 [21:51<02:13,  2.12it/s, est. speed input: 69157.22 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2769/3051 [21:52<02:13,  2.12it/s, est. speed input: 69157.30 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2770/3051 [21:52<02:12,  2.11it/s, est. speed input: 69157.25 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2771/3051 [21:52<02:12,  2.11it/s, est. speed input: 69157.14 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2772/3051 [21:53<02:12,  2.11it/s, est. speed input: 69157.27 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2773/3051 [21:53<02:11,  2.12it/s, est. speed input: 69157.37 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2774/3051 [21:54<02:11,  2.11it/s, est. speed input: 69157.37 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2775/3051 [21:54<02:10,  2.11it/s, est. speed input: 69157.45 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2776/3051 [21:55<02:10,  2.12it/s, est. speed input: 69157.52 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2777/3051 [21:55<02:09,  2.12it/s, est. speed input: 69157.60 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2778/3051 [21:56<02:09,  2.11it/s, est. speed input: 69157.59 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2779/3051 [21:56<02:08,  2.12it/s, est. speed input: 69157.69 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2780/3051 [21:57<02:08,  2.12it/s, est. speed input: 69157.80 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2781/3051 [21:57<02:07,  2.12it/s, est. speed input: 69157.91 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2782/3051 [21:58<02:07,  2.12it/s, est. speed input: 69157.97 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2783/3051 [21:58<02:06,  2.12it/s, est. speed input: 69158.08 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2784/3051 [21:59<02:06,  2.11it/s, est. speed input: 69158.00 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2785/3051 [21:59<02:05,  2.11it/s, est. speed input: 69157.99 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2786/3051 [22:00<02:05,  2.11it/s, est. speed input: 69158.08 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2787/3051 [22:00<02:04,  2.12it/s, est. speed input: 69158.17 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2788/3051 [22:00<02:04,  2.11it/s, est. speed input: 69158.18 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2789/3051 [22:01<02:03,  2.11it/s, est. speed input: 69158.24 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2790/3051 [22:01<02:03,  2.12it/s, est. speed input: 69158.33 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2791/3051 [22:02<02:02,  2.12it/s, est. speed input: 69158.44 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2792/3051 [22:02<02:02,  2.12it/s, est. speed input: 69158.49 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2793/3051 [22:03<02:01,  2.12it/s, est. speed input: 69158.55 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2794/3051 [22:03<02:01,  2.12it/s, est. speed input: 69158.58 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2795/3051 [22:04<02:01,  2.11it/s, est. speed input: 69158.52 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2796/3051 [22:04<02:00,  2.12it/s, est. speed input: 69158.68 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2797/3051 [22:05<02:00,  2.11it/s, est. speed input: 69158.68 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2798/3051 [22:05<01:59,  2.11it/s, est. speed input: 69158.70 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2799/3051 [22:06<01:59,  2.12it/s, est. speed input: 69158.83 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2800/3051 [22:06<01:58,  2.11it/s, est. speed input: 69158.78 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2801/3051 [22:07<01:58,  2.12it/s, est. speed input: 69158.89 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2802/3051 [22:07<01:57,  2.11it/s, est. speed input: 69158.91 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2803/3051 [22:08<01:57,  2.11it/s, est. speed input: 69158.90 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2804/3051 [22:08<01:56,  2.11it/s, est. speed input: 69158.96 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2805/3051 [22:09<01:56,  2.11it/s, est. speed input: 69159.04 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2806/3051 [22:09<01:55,  2.11it/s, est. speed input: 69159.07 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2807/3051 [22:09<01:55,  2.11it/s, est. speed input: 69159.12 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2808/3051 [22:10<01:54,  2.12it/s, est. speed input: 69159.19 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2809/3051 [22:10<01:54,  2.12it/s, est. speed input: 69159.33 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2810/3051 [22:11<01:53,  2.12it/s, est. speed input: 69159.33 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2811/3051 [22:11<01:53,  2.11it/s, est. speed input: 69159.35 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2812/3051 [22:12<01:52,  2.12it/s, est. speed input: 69159.44 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2813/3051 [22:12<01:52,  2.12it/s, est. speed input: 69159.54 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2814/3051 [22:13<01:51,  2.12it/s, est. speed input: 69159.61 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2815/3051 [22:13<01:51,  2.12it/s, est. speed input: 69159.71 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2816/3051 [22:14<01:51,  2.11it/s, est. speed input: 69159.68 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2817/3051 [22:14<01:50,  2.11it/s, est. speed input: 69159.68 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2818/3051 [22:15<01:50,  2.12it/s, est. speed input: 69159.80 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2819/3051 [22:15<01:49,  2.12it/s, est. speed input: 69159.89 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2820/3051 [22:16<01:49,  2.11it/s, est. speed input: 69159.85 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2821/3051 [22:16<01:48,  2.11it/s, est. speed input: 69159.90 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2822/3051 [22:17<01:48,  2.11it/s, est. speed input: 69159.97 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2823/3051 [22:17<01:47,  2.11it/s, est. speed input: 69160.01 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2824/3051 [22:18<01:47,  2.11it/s, est. speed input: 69160.05 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2825/3051 [22:18<01:46,  2.12it/s, est. speed input: 69160.17 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2826/3051 [22:18<01:46,  2.11it/s, est. speed input: 69160.12 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2827/3051 [22:19<01:46,  2.11it/s, est. speed input: 69160.14 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2828/3051 [22:19<01:45,  2.11it/s, est. speed input: 69160.18 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2829/3051 [22:20<01:45,  2.11it/s, est. speed input: 69160.14 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2830/3051 [22:20<01:44,  2.11it/s, est. speed input: 69160.17 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2831/3051 [22:21<01:44,  2.11it/s, est. speed input: 69160.28 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2832/3051 [22:21<01:43,  2.12it/s, est. speed input: 69160.40 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2833/3051 [22:22<01:43,  2.11it/s, est. speed input: 69160.41 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2834/3051 [22:22<01:42,  2.11it/s, est. speed input: 69160.45 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2835/3051 [22:23<01:42,  2.12it/s, est. speed input: 69160.54 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2836/3051 [22:23<01:41,  2.12it/s, est. speed input: 69160.64 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2837/3051 [22:24<01:41,  2.12it/s, est. speed input: 69160.71 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2838/3051 [22:24<01:40,  2.12it/s, est. speed input: 69160.84 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2839/3051 [22:25<01:40,  2.11it/s, est. speed input: 69160.70 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2840/3051 [22:25<01:39,  2.11it/s, est. speed input: 69160.73 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2841/3051 [22:26<01:39,  2.11it/s, est. speed input: 69160.83 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2842/3051 [22:26<01:38,  2.11it/s, est. speed input: 69160.78 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2843/3051 [22:26<01:38,  2.11it/s, est. speed input: 69160.82 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2844/3051 [22:27<01:37,  2.11it/s, est. speed input: 69160.89 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2845/3051 [22:27<01:37,  2.11it/s, est. speed input: 69160.97 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2846/3051 [22:28<01:36,  2.12it/s, est. speed input: 69161.05 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2847/3051 [22:28<01:36,  2.11it/s, est. speed input: 69160.87 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2848/3051 [22:29<01:36,  2.11it/s, est. speed input: 69160.97 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2849/3051 [22:29<01:35,  2.11it/s, est. speed input: 69160.99 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2850/3051 [22:30<01:35,  2.11it/s, est. speed input: 69160.97 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2851/3051 [22:30<01:34,  2.12it/s, est. speed input: 69161.13 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2852/3051 [22:31<01:34,  2.12it/s, est. speed input: 69161.17 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2853/3051 [22:31<01:33,  2.11it/s, est. speed input: 69161.19 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2854/3051 [22:32<01:33,  2.12it/s, est. speed input: 69161.31 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2855/3051 [22:32<01:32,  2.11it/s, est. speed input: 69161.31 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2856/3051 [22:33<01:32,  2.11it/s, est. speed input: 69161.36 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2857/3051 [22:33<01:31,  2.11it/s, est. speed input: 69161.41 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2858/3051 [22:34<01:31,  2.12it/s, est. speed input: 69161.49 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2859/3051 [22:34<01:30,  2.12it/s, est. speed input: 69161.60 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2860/3051 [22:35<01:30,  2.12it/s, est. speed input: 69161.61 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2861/3051 [22:35<01:29,  2.12it/s, est. speed input: 69161.66 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2862/3051 [22:35<01:29,  2.11it/s, est. speed input: 69161.68 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2863/3051 [22:36<01:28,  2.11it/s, est. speed input: 69161.71 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2864/3051 [22:36<01:28,  2.12it/s, est. speed input: 69161.83 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2865/3051 [22:37<01:28,  2.11it/s, est. speed input: 69161.76 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2866/3051 [22:37<01:27,  2.11it/s, est. speed input: 69161.83 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2867/3051 [22:38<01:27,  2.11it/s, est. speed input: 69161.89 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2868/3051 [22:38<01:26,  2.12it/s, est. speed input: 69161.99 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2869/3051 [22:39<01:26,  2.12it/s, est. speed input: 69162.02 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2870/3051 [22:39<01:25,  2.11it/s, est. speed input: 69162.05 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2871/3051 [22:40<01:25,  2.12it/s, est. speed input: 69162.13 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2872/3051 [22:40<01:24,  2.11it/s, est. speed input: 69162.09 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2873/3051 [22:41<01:24,  2.11it/s, est. speed input: 69162.19 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2874/3051 [22:41<01:23,  2.11it/s, est. speed input: 69162.14 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2875/3051 [22:42<01:23,  2.11it/s, est. speed input: 69162.07 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2876/3051 [22:42<01:22,  2.11it/s, est. speed input: 69162.17 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2877/3051 [22:43<01:22,  2.12it/s, est. speed input: 69162.29 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2878/3051 [22:43<01:21,  2.12it/s, est. speed input: 69162.43 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2879/3051 [22:44<01:21,  2.12it/s, est. speed input: 69162.45 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2880/3051 [22:44<01:20,  2.12it/s, est. speed input: 69162.51 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2881/3051 [22:44<01:20,  2.12it/s, est. speed input: 69162.64 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2882/3051 [22:45<01:19,  2.12it/s, est. speed input: 69162.63 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2883/3051 [22:45<01:19,  2.12it/s, est. speed input: 69162.71 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2884/3051 [22:46<01:19,  2.11it/s, est. speed input: 69162.66 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2885/3051 [22:46<01:18,  2.11it/s, est. speed input: 69162.73 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2886/3051 [22:47<01:18,  2.12it/s, est. speed input: 69162.80 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2887/3051 [22:47<01:17,  2.12it/s, est. speed input: 69162.94 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2888/3051 [22:48<01:16,  2.12it/s, est. speed input: 69163.03 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2889/3051 [22:48<01:16,  2.11it/s, est. speed input: 69162.90 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2890/3051 [22:49<01:16,  2.11it/s, est. speed input: 69162.99 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2891/3051 [22:49<01:15,  2.11it/s, est. speed input: 69163.00 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2892/3051 [22:50<01:15,  2.11it/s, est. speed input: 69163.01 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2893/3051 [22:50<01:14,  2.12it/s, est. speed input: 69163.13 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2894/3051 [22:51<01:14,  2.11it/s, est. speed input: 69163.15 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2895/3051 [22:51<01:13,  2.11it/s, est. speed input: 69163.14 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2896/3051 [22:52<01:13,  2.12it/s, est. speed input: 69163.26 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2897/3051 [22:52<01:12,  2.11it/s, est. speed input: 69163.29 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2898/3051 [22:53<01:12,  2.11it/s, est. speed input: 69163.34 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2899/3051 [22:53<01:11,  2.12it/s, est. speed input: 69163.41 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2900/3051 [22:53<01:11,  2.12it/s, est. speed input: 69163.51 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2901/3051 [22:54<01:10,  2.12it/s, est. speed input: 69163.61 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2902/3051 [22:54<01:10,  2.12it/s, est. speed input: 69163.67 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2903/3051 [22:55<01:09,  2.12it/s, est. speed input: 69163.70 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2904/3051 [22:55<01:09,  2.11it/s, est. speed input: 69163.48 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2905/3051 [22:56<01:09,  2.11it/s, est. speed input: 69163.56 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2906/3051 [22:56<01:08,  2.11it/s, est. speed input: 69163.69 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2907/3051 [22:57<01:08,  2.12it/s, est. speed input: 69163.77 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2908/3051 [22:57<01:07,  2.11it/s, est. speed input: 69163.71 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2909/3051 [22:58<01:07,  2.11it/s, est. speed input: 69163.81 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2910/3051 [22:58<01:06,  2.11it/s, est. speed input: 69163.79 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2911/3051 [22:59<01:06,  2.11it/s, est. speed input: 69163.86 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2912/3051 [22:59<01:05,  2.11it/s, est. speed input: 69163.80 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2913/3051 [23:00<01:05,  2.11it/s, est. speed input: 69163.64 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2914/3051 [23:00<01:04,  2.11it/s, est. speed input: 69163.74 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2915/3051 [23:01<01:04,  2.11it/s, est. speed input: 69163.83 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2916/3051 [23:01<01:03,  2.11it/s, est. speed input: 69163.88 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2917/3051 [23:01<01:03,  2.11it/s, est. speed input: 69163.91 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2918/3051 [23:02<01:02,  2.12it/s, est. speed input: 69164.01 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2919/3051 [23:02<01:02,  2.12it/s, est. speed input: 69164.16 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2920/3051 [23:03<01:01,  2.11it/s, est. speed input: 69164.08 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2921/3051 [23:03<01:01,  2.12it/s, est. speed input: 69164.18 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2922/3051 [23:04<01:01,  2.11it/s, est. speed input: 69164.18 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2923/3051 [23:04<01:00,  2.11it/s, est. speed input: 69164.06 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2924/3051 [23:05<01:00,  2.11it/s, est. speed input: 69164.11 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2925/3051 [23:05<00:59,  2.11it/s, est. speed input: 69164.22 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2926/3051 [23:06<00:59,  2.12it/s, est. speed input: 69164.31 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2927/3051 [23:06<00:58,  2.12it/s, est. speed input: 69164.41 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2928/3051 [23:07<00:58,  2.12it/s, est. speed input: 69164.50 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2929/3051 [23:07<00:57,  2.12it/s, est. speed input: 69164.59 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2930/3051 [23:08<00:57,  2.11it/s, est. speed input: 69164.53 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2931/3051 [23:08<00:56,  2.12it/s, est. speed input: 69164.64 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2932/3051 [23:09<00:56,  2.12it/s, est. speed input: 69164.74 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2933/3051 [23:09<00:55,  2.12it/s, est. speed input: 69164.77 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2934/3051 [23:10<00:55,  2.11it/s, est. speed input: 69164.69 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2935/3051 [23:10<00:54,  2.12it/s, est. speed input: 69164.82 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2936/3051 [23:10<00:54,  2.12it/s, est. speed input: 69164.94 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2937/3051 [23:11<00:53,  2.11it/s, est. speed input: 69164.85 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2938/3051 [23:11<00:53,  2.11it/s, est. speed input: 69164.93 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2939/3051 [23:12<00:52,  2.12it/s, est. speed input: 69165.06 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2940/3051 [23:12<00:52,  2.12it/s, est. speed input: 69165.15 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2941/3051 [23:13<00:51,  2.12it/s, est. speed input: 69165.21 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2942/3051 [23:13<00:51,  2.12it/s, est. speed input: 69165.30 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2943/3051 [23:14<00:51,  2.12it/s, est. speed input: 69165.32 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2944/3051 [23:14<00:50,  2.12it/s, est. speed input: 69165.41 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2945/3051 [23:15<00:50,  2.12it/s, est. speed input: 69165.50 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2946/3051 [23:15<00:49,  2.11it/s, est. speed input: 69165.34 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2947/3051 [23:16<00:49,  2.11it/s, est. speed input: 69165.38 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2948/3051 [23:16<00:48,  2.12it/s, est. speed input: 69165.50 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2949/3051 [23:17<00:48,  2.11it/s, est. speed input: 69165.50 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2950/3051 [23:17<00:47,  2.11it/s, est. speed input: 69165.53 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2951/3051 [23:18<00:47,  2.12it/s, est. speed input: 69165.63 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2952/3051 [23:18<00:46,  2.12it/s, est. speed input: 69165.68 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2953/3051 [23:19<00:46,  2.12it/s, est. speed input: 69165.76 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2954/3051 [23:19<00:45,  2.11it/s, est. speed input: 69165.73 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2955/3051 [23:19<00:45,  2.11it/s, est. speed input: 69165.80 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2956/3051 [23:20<00:44,  2.11it/s, est. speed input: 69165.83 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2957/3051 [23:20<00:44,  2.11it/s, est. speed input: 69165.65 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2958/3051 [23:21<00:44,  2.11it/s, est. speed input: 69165.73 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2959/3051 [23:21<00:43,  2.11it/s, est. speed input: 69165.81 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2960/3051 [23:22<00:43,  2.11it/s, est. speed input: 69165.76 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2961/3051 [23:22<00:42,  2.11it/s, est. speed input: 69165.87 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2962/3051 [23:23<00:42,  2.12it/s, est. speed input: 69165.98 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2963/3051 [23:23<00:41,  2.12it/s, est. speed input: 69166.08 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2964/3051 [23:24<00:41,  2.12it/s, est. speed input: 69166.17 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2965/3051 [23:24<00:40,  2.12it/s, est. speed input: 69166.15 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2966/3051 [23:25<00:40,  2.12it/s, est. speed input: 69166.23 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2967/3051 [23:25<00:39,  2.11it/s, est. speed input: 69166.11 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2968/3051 [23:26<00:39,  2.11it/s, est. speed input: 69166.17 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2969/3051 [23:26<00:38,  2.11it/s, est. speed input: 69166.22 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2970/3051 [23:27<00:38,  2.11it/s, est. speed input: 69166.22 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2971/3051 [23:27<00:37,  2.11it/s, est. speed input: 69166.28 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2972/3051 [23:28<00:37,  2.11it/s, est. speed input: 69166.35 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2973/3051 [23:28<00:36,  2.12it/s, est. speed input: 69166.43 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2974/3051 [23:28<00:36,  2.12it/s, est. speed input: 69166.52 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2975/3051 [23:29<00:35,  2.11it/s, est. speed input: 69166.50 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2976/3051 [23:29<00:35,  2.11it/s, est. speed input: 69166.54 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2977/3051 [23:30<00:35,  2.11it/s, est. speed input: 69166.57 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2978/3051 [23:30<00:34,  2.11it/s, est. speed input: 69166.56 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2979/3051 [23:31<00:34,  2.11it/s, est. speed input: 69166.60 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2980/3051 [23:31<00:33,  2.11it/s, est. speed input: 69166.69 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2981/3051 [23:32<00:33,  2.12it/s, est. speed input: 69166.77 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2982/3051 [23:32<00:32,  2.11it/s, est. speed input: 69166.64 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2983/3051 [23:33<00:32,  2.11it/s, est. speed input: 69166.70 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2984/3051 [23:33<00:31,  2.12it/s, est. speed input: 69166.85 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2985/3051 [23:34<00:31,  2.12it/s, est. speed input: 69166.91 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2986/3051 [23:34<00:30,  2.12it/s, est. speed input: 69166.98 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2987/3051 [23:35<00:30,  2.12it/s, est. speed input: 69167.04 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2988/3051 [23:35<00:29,  2.12it/s, est. speed input: 69167.10 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2989/3051 [23:36<00:29,  2.12it/s, est. speed input: 69167.19 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2990/3051 [23:36<00:28,  2.12it/s, est. speed input: 69167.31 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2991/3051 [23:36<00:28,  2.11it/s, est. speed input: 69167.13 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2992/3051 [23:37<00:27,  2.11it/s, est. speed input: 69167.20 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2993/3051 [23:37<00:27,  2.12it/s, est. speed input: 69167.36 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2994/3051 [23:38<00:26,  2.12it/s, est. speed input: 69167.39 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2995/3051 [23:38<00:26,  2.11it/s, est. speed input: 69167.37 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2996/3051 [23:39<00:26,  2.11it/s, est. speed input: 69167.39 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2997/3051 [23:39<00:25,  2.11it/s, est. speed input: 69167.47 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2998/3051 [23:40<00:25,  2.12it/s, est. speed input: 69167.55 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2999/3051 [23:40<00:24,  2.11it/s, est. speed input: 69167.55 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3000/3051 [23:41<00:24,  2.12it/s, est. speed input: 69167.63 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3001/3051 [23:41<00:23,  2.12it/s, est. speed input: 69167.68 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3002/3051 [23:42<00:23,  2.11it/s, est. speed input: 69167.66 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3003/3051 [23:42<00:22,  2.12it/s, est. speed input: 69167.78 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3004/3051 [23:43<00:22,  2.12it/s, est. speed input: 69167.84 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3005/3051 [23:43<00:21,  2.11it/s, est. speed input: 69167.75 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3006/3051 [23:44<00:21,  2.12it/s, est. speed input: 69167.91 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3007/3051 [23:44<00:20,  2.11it/s, est. speed input: 69167.84 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3008/3051 [23:45<00:20,  2.11it/s, est. speed input: 69167.91 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3009/3051 [23:45<00:19,  2.12it/s, est. speed input: 69168.03 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3010/3051 [23:45<00:19,  2.12it/s, est. speed input: 69168.11 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3011/3051 [23:46<00:18,  2.11it/s, est. speed input: 69167.96 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3012/3051 [23:46<00:18,  2.11it/s, est. speed input: 69168.02 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3013/3051 [23:47<00:17,  2.11it/s, est. speed input: 69168.12 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3014/3051 [23:47<00:17,  2.11it/s, est. speed input: 69168.13 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3015/3051 [23:48<00:17,  2.11it/s, est. speed input: 69168.16 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3016/3051 [23:48<00:16,  2.12it/s, est. speed input: 69168.29 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3017/3051 [23:49<00:16,  2.11it/s, est. speed input: 69168.18 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3018/3051 [23:49<00:15,  2.11it/s, est. speed input: 69168.24 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3019/3051 [23:50<00:15,  2.12it/s, est. speed input: 69168.35 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3020/3051 [23:50<00:14,  2.11it/s, est. speed input: 69168.33 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3021/3051 [23:51<00:14,  2.11it/s, est. speed input: 69168.35 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3022/3051 [23:51<00:13,  2.12it/s, est. speed input: 69168.46 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3023/3051 [23:52<00:13,  2.11it/s, est. speed input: 69168.46 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3024/3051 [23:52<00:12,  2.11it/s, est. speed input: 69168.51 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3025/3051 [23:53<00:12,  2.11it/s, est. speed input: 69168.54 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3026/3051 [23:53<00:11,  2.12it/s, est. speed input: 69168.68 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3027/3051 [23:54<00:11,  2.12it/s, est. speed input: 69168.71 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3028/3051 [23:54<00:10,  2.12it/s, est. speed input: 69168.83 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3029/3051 [23:54<00:10,  2.12it/s, est. speed input: 69168.89 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3030/3051 [23:55<00:09,  2.11it/s, est. speed input: 69168.83 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3031/3051 [23:55<00:09,  2.11it/s, est. speed input: 69168.88 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3032/3051 [23:56<00:08,  2.11it/s, est. speed input: 69168.86 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3033/3051 [23:56<00:08,  2.11it/s, est. speed input: 69168.82 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3034/3051 [23:57<00:08,  2.11it/s, est. speed input: 69168.94 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3035/3051 [23:57<00:07,  2.12it/s, est. speed input: 69169.01 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3036/3051 [23:58<00:07,  2.12it/s, est. speed input: 69169.13 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3037/3051 [23:58<00:06,  2.12it/s, est. speed input: 69169.18 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3038/3051 [23:59<00:06,  2.12it/s, est. speed input: 69169.23 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3039/3051 [23:59<00:05,  2.11it/s, est. speed input: 69169.19 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3040/3051 [24:00<00:05,  2.11it/s, est. speed input: 69169.14 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3041/3051 [24:00<00:04,  2.11it/s, est. speed input: 69169.21 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3042/3051 [24:01<00:04,  2.11it/s, est. speed input: 69169.16 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3043/3051 [24:01<00:03,  2.11it/s, est. speed input: 69169.14 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3044/3051 [24:02<00:03,  2.12it/s, est. speed input: 69169.30 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3045/3051 [24:02<00:02,  2.12it/s, est. speed input: 69169.42 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3046/3051 [24:02<00:02,  2.11it/s, est. speed input: 69169.38 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3047/3051 [24:03<00:01,  2.11it/s, est. speed input: 69169.43 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3048/3051 [24:03<00:01,  2.12it/s, est. speed input: 69169.55 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3049/3051 [24:04<00:00,  2.11it/s, est. speed input: 69169.42 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3050/3051 [24:04<00:00,  2.11it/s, est. speed input: 69169.32 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 3051/3051 [24:05<00:00,  2.11it/s, est. speed input: 69169.40 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 3051/3051 [24:05<00:00,  2.11it/s, est. speed input: 69169.40 toks/s, output: 2.11 toks/s]
Saved activation counts to new_activations/activation.zh.train.q2.5-zh.pt
Processed 99975168 tokens total

Loading IDs from ./ids/q2.5/id.en.train.qwen2.5-0.5 (lang: en)...
ID shape: torch.Size([100000000])
Processing 3051 sequences of length 32768
Running inference...

Processed prompts:   0%|          | 0/3051 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts:   0%|          | 1/3051 [00:00<24:34,  2.07it/s, est. speed input: 67785.59 toks/s, output: 2.07 toks/s]
Processed prompts:   0%|          | 2/3051 [00:00<24:07,  2.11it/s, est. speed input: 68839.57 toks/s, output: 2.10 toks/s]
Processed prompts:   0%|          | 3/3051 [00:01<23:58,  2.12it/s, est. speed input: 69198.35 toks/s, output: 2.11 toks/s]
Processed prompts:   0%|          | 4/3051 [00:01<23:53,  2.13it/s, est. speed input: 69382.04 toks/s, output: 2.12 toks/s]
Processed prompts:   0%|          | 5/3051 [00:02<23:51,  2.13it/s, est. speed input: 69494.72 toks/s, output: 2.12 toks/s]
Processed prompts:   0%|          | 6/3051 [00:02<23:50,  2.13it/s, est. speed input: 69547.13 toks/s, output: 2.12 toks/s]
Processed prompts:   0%|          | 7/3051 [00:03<23:48,  2.13it/s, est. speed input: 69598.05 toks/s, output: 2.12 toks/s]
Processed prompts:   0%|          | 8/3051 [00:03<23:47,  2.13it/s, est. speed input: 69641.96 toks/s, output: 2.13 toks/s]
Processed prompts:   0%|          | 9/3051 [00:04<23:46,  2.13it/s, est. speed input: 69676.17 toks/s, output: 2.13 toks/s]
Processed prompts:   0%|          | 10/3051 [00:04<24:23,  2.08it/s, est. speed input: 69112.72 toks/s, output: 2.11 toks/s]
Processed prompts:   0%|          | 11/3051 [00:05<24:12,  2.09it/s, est. speed input: 69171.79 toks/s, output: 2.11 toks/s]
Processed prompts:   0%|          | 12/3051 [00:05<24:03,  2.11it/s, est. speed input: 69231.81 toks/s, output: 2.11 toks/s]
Processed prompts:   0%|          | 13/3051 [00:06<23:57,  2.11it/s, est. speed input: 69282.16 toks/s, output: 2.11 toks/s]
Processed prompts:   0%|          | 14/3051 [00:06<23:53,  2.12it/s, est. speed input: 69319.96 toks/s, output: 2.12 toks/s]
Processed prompts:   0%|          | 15/3051 [00:07<23:50,  2.12it/s, est. speed input: 69355.95 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 16/3051 [00:07<23:48,  2.13it/s, est. speed input: 69385.83 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 17/3051 [00:08<23:46,  2.13it/s, est. speed input: 69414.38 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 18/3051 [00:08<23:45,  2.13it/s, est. speed input: 69435.79 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 19/3051 [00:08<23:44,  2.13it/s, est. speed input: 69452.90 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 20/3051 [00:09<23:42,  2.13it/s, est. speed input: 69475.77 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 21/3051 [00:09<23:42,  2.13it/s, est. speed input: 69493.73 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 22/3051 [00:10<23:41,  2.13it/s, est. speed input: 69508.75 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 23/3051 [00:10<23:41,  2.13it/s, est. speed input: 69520.61 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 24/3051 [00:11<23:40,  2.13it/s, est. speed input: 69534.97 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 25/3051 [00:11<23:40,  2.13it/s, est. speed input: 69546.60 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 26/3051 [00:12<23:39,  2.13it/s, est. speed input: 69559.21 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 27/3051 [00:12<23:38,  2.13it/s, est. speed input: 69569.41 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 28/3051 [00:13<23:38,  2.13it/s, est. speed input: 69579.36 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 29/3051 [00:13<23:37,  2.13it/s, est. speed input: 69588.85 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 30/3051 [00:14<23:37,  2.13it/s, est. speed input: 69596.85 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 31/3051 [00:14<23:36,  2.13it/s, est. speed input: 69605.13 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 32/3051 [00:15<23:36,  2.13it/s, est. speed input: 69611.46 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 33/3051 [00:15<23:36,  2.13it/s, est. speed input: 69615.95 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 34/3051 [00:16<23:36,  2.13it/s, est. speed input: 69621.06 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 35/3051 [00:16<23:35,  2.13it/s, est. speed input: 69628.69 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 36/3051 [00:16<23:35,  2.13it/s, est. speed input: 69633.23 toks/s, output: 2.13 toks/s]
Processed prompts:   1%|          | 37/3051 [00:17<23:34,  2.13it/s, est. speed input: 69637.01 toks/s, output: 2.13 toks/s]
Processed prompts:   1%|          | 38/3051 [00:17<23:34,  2.13it/s, est. speed input: 69640.25 toks/s, output: 2.13 toks/s]
Processed prompts:   1%|▏         | 39/3051 [00:18<23:34,  2.13it/s, est. speed input: 69642.82 toks/s, output: 2.13 toks/s]
Processed prompts:   1%|▏         | 40/3051 [00:18<23:34,  2.13it/s, est. speed input: 69645.95 toks/s, output: 2.13 toks/s]
Processed prompts:   1%|▏         | 41/3051 [00:19<23:33,  2.13it/s, est. speed input: 69650.37 toks/s, output: 2.13 toks/s]
Processed prompts:   1%|▏         | 42/3051 [00:19<23:33,  2.13it/s, est. speed input: 69651.64 toks/s, output: 2.13 toks/s]
Processed prompts:   1%|▏         | 43/3051 [00:20<23:32,  2.13it/s, est. speed input: 69655.35 toks/s, output: 2.13 toks/s]
Processed prompts:   1%|▏         | 44/3051 [00:20<23:32,  2.13it/s, est. speed input: 69657.51 toks/s, output: 2.13 toks/s]
Processed prompts:   1%|▏         | 45/3051 [00:21<23:31,  2.13it/s, est. speed input: 69660.37 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 46/3051 [00:21<23:31,  2.13it/s, est. speed input: 69661.35 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 47/3051 [00:22<23:31,  2.13it/s, est. speed input: 69663.73 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 48/3051 [00:22<23:30,  2.13it/s, est. speed input: 69665.52 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 49/3051 [00:23<23:30,  2.13it/s, est. speed input: 69666.69 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 50/3051 [00:23<23:30,  2.13it/s, est. speed input: 69666.79 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 51/3051 [00:23<23:29,  2.13it/s, est. speed input: 69667.64 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 52/3051 [00:24<23:29,  2.13it/s, est. speed input: 69669.59 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 53/3051 [00:24<23:28,  2.13it/s, est. speed input: 69670.58 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 54/3051 [00:25<23:28,  2.13it/s, est. speed input: 69672.04 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 55/3051 [00:25<23:27,  2.13it/s, est. speed input: 69673.49 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 56/3051 [00:26<23:27,  2.13it/s, est. speed input: 69674.54 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 57/3051 [00:26<23:26,  2.13it/s, est. speed input: 69676.95 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 58/3051 [00:27<23:26,  2.13it/s, est. speed input: 69677.52 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 59/3051 [00:27<23:25,  2.13it/s, est. speed input: 69678.67 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 60/3051 [00:28<23:25,  2.13it/s, est. speed input: 69679.60 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 61/3051 [00:28<23:25,  2.13it/s, est. speed input: 69678.23 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 62/3051 [00:29<23:24,  2.13it/s, est. speed input: 69679.34 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 63/3051 [00:29<23:24,  2.13it/s, est. speed input: 69680.82 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 64/3051 [00:30<23:23,  2.13it/s, est. speed input: 69681.36 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 65/3051 [00:30<23:23,  2.13it/s, est. speed input: 69681.37 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 66/3051 [00:31<23:22,  2.13it/s, est. speed input: 69682.71 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 67/3051 [00:31<23:22,  2.13it/s, est. speed input: 69683.62 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 68/3051 [00:31<23:21,  2.13it/s, est. speed input: 69683.95 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 69/3051 [00:32<23:21,  2.13it/s, est. speed input: 69685.25 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 70/3051 [00:32<23:20,  2.13it/s, est. speed input: 69685.55 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 71/3051 [00:33<23:20,  2.13it/s, est. speed input: 69685.84 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 72/3051 [00:33<23:20,  2.13it/s, est. speed input: 69686.28 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 73/3051 [00:34<23:19,  2.13it/s, est. speed input: 69686.97 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 74/3051 [00:34<23:19,  2.13it/s, est. speed input: 69685.77 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 75/3051 [00:35<23:19,  2.13it/s, est. speed input: 69686.07 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 76/3051 [00:35<23:18,  2.13it/s, est. speed input: 69686.60 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 77/3051 [00:36<23:17,  2.13it/s, est. speed input: 69687.38 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 78/3051 [00:36<23:17,  2.13it/s, est. speed input: 69688.22 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 79/3051 [00:37<23:17,  2.13it/s, est. speed input: 69687.12 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 80/3051 [00:37<23:16,  2.13it/s, est. speed input: 69687.14 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 81/3051 [00:38<23:17,  2.13it/s, est. speed input: 69686.00 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 82/3051 [00:38<23:16,  2.13it/s, est. speed input: 69686.05 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 83/3051 [00:39<23:16,  2.13it/s, est. speed input: 69685.36 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 84/3051 [00:39<23:15,  2.13it/s, est. speed input: 69685.33 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 85/3051 [00:39<23:15,  2.13it/s, est. speed input: 69684.90 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 86/3051 [00:40<23:14,  2.13it/s, est. speed input: 69684.34 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 87/3051 [00:40<23:14,  2.13it/s, est. speed input: 69684.23 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 88/3051 [00:41<23:13,  2.13it/s, est. speed input: 69683.82 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 89/3051 [00:41<23:13,  2.13it/s, est. speed input: 69683.18 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 90/3051 [00:42<23:13,  2.13it/s, est. speed input: 69682.76 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 91/3051 [00:42<23:12,  2.13it/s, est. speed input: 69683.33 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 92/3051 [00:43<23:11,  2.13it/s, est. speed input: 69683.22 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 93/3051 [00:43<23:12,  2.12it/s, est. speed input: 69681.63 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 94/3051 [00:44<23:12,  2.12it/s, est. speed input: 69680.11 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 95/3051 [00:44<23:10,  2.13it/s, est. speed input: 69680.52 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 96/3051 [00:45<23:10,  2.12it/s, est. speed input: 69679.28 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 97/3051 [00:45<23:10,  2.12it/s, est. speed input: 69678.74 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 98/3051 [00:46<23:10,  2.12it/s, est. speed input: 69677.91 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 99/3051 [00:46<23:10,  2.12it/s, est. speed input: 69676.49 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 100/3051 [00:47<23:09,  2.12it/s, est. speed input: 69675.80 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 101/3051 [00:47<23:09,  2.12it/s, est. speed input: 69674.92 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 102/3051 [00:47<23:07,  2.12it/s, est. speed input: 69675.17 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 103/3051 [00:48<23:08,  2.12it/s, est. speed input: 69673.72 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 104/3051 [00:48<23:07,  2.12it/s, est. speed input: 69672.79 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 105/3051 [00:49<23:07,  2.12it/s, est. speed input: 69671.95 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 106/3051 [00:49<23:07,  2.12it/s, est. speed input: 69670.34 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 107/3051 [00:50<23:06,  2.12it/s, est. speed input: 69669.09 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 108/3051 [00:50<23:05,  2.12it/s, est. speed input: 69669.33 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 109/3051 [00:51<23:04,  2.12it/s, est. speed input: 69669.20 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 110/3051 [00:51<23:04,  2.12it/s, est. speed input: 69668.98 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 111/3051 [00:52<23:03,  2.12it/s, est. speed input: 69668.73 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 112/3051 [00:52<23:02,  2.13it/s, est. speed input: 69668.63 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 113/3051 [00:53<23:02,  2.12it/s, est. speed input: 69667.86 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 114/3051 [00:53<23:02,  2.13it/s, est. speed input: 69667.91 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 115/3051 [00:54<23:01,  2.12it/s, est. speed input: 69667.16 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 116/3051 [00:54<23:01,  2.13it/s, est. speed input: 69667.15 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 117/3051 [00:55<23:00,  2.13it/s, est. speed input: 69667.25 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 118/3051 [00:55<23:01,  2.12it/s, est. speed input: 69664.53 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 119/3051 [00:55<23:02,  2.12it/s, est. speed input: 69662.07 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 120/3051 [00:56<23:02,  2.12it/s, est. speed input: 69659.82 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 121/3051 [00:56<23:02,  2.12it/s, est. speed input: 69657.89 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 122/3051 [00:57<23:02,  2.12it/s, est. speed input: 69655.03 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 123/3051 [00:57<23:02,  2.12it/s, est. speed input: 69652.95 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 124/3051 [00:58<23:01,  2.12it/s, est. speed input: 69650.97 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 125/3051 [00:58<23:01,  2.12it/s, est. speed input: 69648.84 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 126/3051 [00:59<23:01,  2.12it/s, est. speed input: 69646.74 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 127/3051 [00:59<23:00,  2.12it/s, est. speed input: 69644.38 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 128/3051 [01:00<23:00,  2.12it/s, est. speed input: 69642.25 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 129/3051 [01:00<23:00,  2.12it/s, est. speed input: 69639.91 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 130/3051 [01:01<22:59,  2.12it/s, est. speed input: 69637.87 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 131/3051 [01:01<22:58,  2.12it/s, est. speed input: 69636.24 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 132/3051 [01:02<22:58,  2.12it/s, est. speed input: 69634.02 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 133/3051 [01:02<22:58,  2.12it/s, est. speed input: 69632.22 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 134/3051 [01:03<22:58,  2.12it/s, est. speed input: 69628.80 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 135/3051 [01:03<22:58,  2.12it/s, est. speed input: 69626.35 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 136/3051 [01:04<22:58,  2.11it/s, est. speed input: 69623.67 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 137/3051 [01:04<22:57,  2.12it/s, est. speed input: 69621.97 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 138/3051 [01:04<22:56,  2.12it/s, est. speed input: 69619.94 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 139/3051 [01:05<22:55,  2.12it/s, est. speed input: 69618.52 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 140/3051 [01:05<22:55,  2.12it/s, est. speed input: 69616.51 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 141/3051 [01:06<22:55,  2.12it/s, est. speed input: 69614.47 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 142/3051 [01:06<22:54,  2.12it/s, est. speed input: 69612.50 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 143/3051 [01:07<22:53,  2.12it/s, est. speed input: 69610.85 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 144/3051 [01:07<22:54,  2.11it/s, est. speed input: 69607.38 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 145/3051 [01:08<22:55,  2.11it/s, est. speed input: 69604.37 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 146/3051 [01:08<22:53,  2.11it/s, est. speed input: 69602.69 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 147/3051 [01:09<22:53,  2.11it/s, est. speed input: 69600.20 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 148/3051 [01:09<22:52,  2.11it/s, est. speed input: 69598.57 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 149/3051 [01:10<22:52,  2.11it/s, est. speed input: 69596.60 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 150/3051 [01:10<22:51,  2.11it/s, est. speed input: 69594.46 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 151/3051 [01:11<22:51,  2.11it/s, est. speed input: 69592.57 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 152/3051 [01:11<22:50,  2.12it/s, est. speed input: 69591.14 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 153/3051 [01:12<22:49,  2.12it/s, est. speed input: 69589.77 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 154/3051 [01:12<22:50,  2.11it/s, est. speed input: 69586.99 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 155/3051 [01:12<22:49,  2.11it/s, est. speed input: 69584.92 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 156/3051 [01:13<22:48,  2.11it/s, est. speed input: 69583.42 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 157/3051 [01:13<22:51,  2.11it/s, est. speed input: 69578.30 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 158/3051 [01:14<22:49,  2.11it/s, est. speed input: 69577.15 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 159/3051 [01:14<22:48,  2.11it/s, est. speed input: 69575.39 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 160/3051 [01:15<22:47,  2.11it/s, est. speed input: 69573.69 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 161/3051 [01:15<22:47,  2.11it/s, est. speed input: 69571.91 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 162/3051 [01:16<22:46,  2.11it/s, est. speed input: 69570.38 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 163/3051 [01:16<22:45,  2.12it/s, est. speed input: 69569.31 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 164/3051 [01:17<22:44,  2.12it/s, est. speed input: 69567.77 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 165/3051 [01:17<22:43,  2.12it/s, est. speed input: 69566.54 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 166/3051 [01:18<22:43,  2.12it/s, est. speed input: 69565.22 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 167/3051 [01:18<22:44,  2.11it/s, est. speed input: 69561.92 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 168/3051 [01:19<22:44,  2.11it/s, est. speed input: 69559.74 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 169/3051 [01:19<22:43,  2.11it/s, est. speed input: 69558.72 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 170/3051 [01:20<22:42,  2.11it/s, est. speed input: 69557.42 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 171/3051 [01:20<22:42,  2.11it/s, est. speed input: 69554.93 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 172/3051 [01:21<22:41,  2.11it/s, est. speed input: 69553.77 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 173/3051 [01:21<22:40,  2.11it/s, est. speed input: 69552.48 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 174/3051 [01:21<22:40,  2.12it/s, est. speed input: 69551.31 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 175/3051 [01:22<22:40,  2.11it/s, est. speed input: 69549.59 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 176/3051 [01:22<22:39,  2.11it/s, est. speed input: 69548.04 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 177/3051 [01:23<22:41,  2.11it/s, est. speed input: 69544.48 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 178/3051 [01:23<22:40,  2.11it/s, est. speed input: 69542.89 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 179/3051 [01:24<22:39,  2.11it/s, est. speed input: 69541.32 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 180/3051 [01:24<22:40,  2.11it/s, est. speed input: 69538.00 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 181/3051 [01:25<22:39,  2.11it/s, est. speed input: 69536.25 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 182/3051 [01:25<23:41,  2.02it/s, est. speed input: 69475.35 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 183/3051 [01:26<23:20,  2.05it/s, est. speed input: 69475.71 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 184/3051 [01:26<23:06,  2.07it/s, est. speed input: 69475.24 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 185/3051 [01:27<22:56,  2.08it/s, est. speed input: 69474.59 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 186/3051 [01:27<22:49,  2.09it/s, est. speed input: 69473.52 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 187/3051 [01:28<22:44,  2.10it/s, est. speed input: 69472.67 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 188/3051 [01:28<22:40,  2.10it/s, est. speed input: 69472.51 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 189/3051 [01:29<22:37,  2.11it/s, est. speed input: 69471.75 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 190/3051 [01:29<22:35,  2.11it/s, est. speed input: 69471.12 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 191/3051 [01:30<22:34,  2.11it/s, est. speed input: 69470.30 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 192/3051 [01:30<22:32,  2.11it/s, est. speed input: 69469.99 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 193/3051 [01:31<22:31,  2.11it/s, est. speed input: 69469.25 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 194/3051 [01:31<22:30,  2.11it/s, est. speed input: 69468.55 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 195/3051 [01:31<22:30,  2.12it/s, est. speed input: 69467.98 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 196/3051 [01:32<22:29,  2.12it/s, est. speed input: 69467.12 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 197/3051 [01:32<22:28,  2.12it/s, est. speed input: 69467.10 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 198/3051 [01:33<22:28,  2.12it/s, est. speed input: 69466.01 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 199/3051 [01:33<22:28,  2.12it/s, est. speed input: 69464.95 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 200/3051 [01:34<22:27,  2.12it/s, est. speed input: 69464.21 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 201/3051 [01:34<22:27,  2.12it/s, est. speed input: 69463.46 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 202/3051 [01:35<22:27,  2.11it/s, est. speed input: 69462.45 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 203/3051 [01:35<22:26,  2.12it/s, est. speed input: 69461.98 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 204/3051 [01:36<22:26,  2.12it/s, est. speed input: 69461.08 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 205/3051 [01:36<22:25,  2.12it/s, est. speed input: 69460.53 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 206/3051 [01:37<22:24,  2.12it/s, est. speed input: 69459.84 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 207/3051 [01:37<22:24,  2.12it/s, est. speed input: 69459.13 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 208/3051 [01:38<22:24,  2.11it/s, est. speed input: 69457.97 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 209/3051 [01:38<22:23,  2.11it/s, est. speed input: 69457.30 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 210/3051 [01:39<22:23,  2.11it/s, est. speed input: 69456.41 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 211/3051 [01:39<22:24,  2.11it/s, est. speed input: 69454.24 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 212/3051 [01:40<22:23,  2.11it/s, est. speed input: 69453.50 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 213/3051 [01:40<22:22,  2.11it/s, est. speed input: 69452.85 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 214/3051 [01:40<22:21,  2.11it/s, est. speed input: 69452.47 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 215/3051 [01:41<22:20,  2.12it/s, est. speed input: 69452.07 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 216/3051 [01:41<22:20,  2.12it/s, est. speed input: 69451.55 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 217/3051 [01:42<22:20,  2.11it/s, est. speed input: 69450.41 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 218/3051 [01:42<22:19,  2.11it/s, est. speed input: 69449.63 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 219/3051 [01:43<22:19,  2.11it/s, est. speed input: 69448.85 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 220/3051 [01:43<22:18,  2.11it/s, est. speed input: 69448.15 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 221/3051 [01:44<22:22,  2.11it/s, est. speed input: 69444.24 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 222/3051 [01:44<22:20,  2.11it/s, est. speed input: 69443.51 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 223/3051 [01:45<22:18,  2.11it/s, est. speed input: 69443.37 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 224/3051 [01:45<22:17,  2.11it/s, est. speed input: 69443.20 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 225/3051 [01:46<22:16,  2.11it/s, est. speed input: 69442.53 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 226/3051 [01:46<22:16,  2.11it/s, est. speed input: 69441.82 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 227/3051 [01:47<22:15,  2.11it/s, est. speed input: 69440.94 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 228/3051 [01:47<22:15,  2.11it/s, est. speed input: 69440.45 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 229/3051 [01:48<22:14,  2.11it/s, est. speed input: 69439.89 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 230/3051 [01:48<22:13,  2.12it/s, est. speed input: 69439.74 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 231/3051 [01:49<22:13,  2.12it/s, est. speed input: 69439.00 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 232/3051 [01:49<22:12,  2.12it/s, est. speed input: 69438.42 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 233/3051 [01:49<22:12,  2.11it/s, est. speed input: 69437.70 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 234/3051 [01:50<22:12,  2.11it/s, est. speed input: 69437.09 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 235/3051 [01:50<22:12,  2.11it/s, est. speed input: 69436.16 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 236/3051 [01:51<22:11,  2.11it/s, est. speed input: 69435.74 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 237/3051 [01:51<22:10,  2.11it/s, est. speed input: 69435.18 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 238/3051 [01:52<22:09,  2.12it/s, est. speed input: 69434.94 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 239/3051 [01:52<22:10,  2.11it/s, est. speed input: 69433.59 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 240/3051 [01:53<22:11,  2.11it/s, est. speed input: 69431.67 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 241/3051 [01:53<22:10,  2.11it/s, est. speed input: 69430.76 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 242/3051 [01:54<22:09,  2.11it/s, est. speed input: 69430.19 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 243/3051 [01:54<22:09,  2.11it/s, est. speed input: 69429.06 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 244/3051 [01:55<22:08,  2.11it/s, est. speed input: 69428.35 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 245/3051 [01:55<22:08,  2.11it/s, est. speed input: 69427.63 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 246/3051 [01:56<22:07,  2.11it/s, est. speed input: 69427.07 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 247/3051 [01:56<22:06,  2.11it/s, est. speed input: 69426.51 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 248/3051 [01:57<22:06,  2.11it/s, est. speed input: 69425.86 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 249/3051 [01:57<22:06,  2.11it/s, est. speed input: 69424.95 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 250/3051 [01:57<22:05,  2.11it/s, est. speed input: 69424.42 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 251/3051 [01:58<22:05,  2.11it/s, est. speed input: 69423.15 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 252/3051 [01:58<22:05,  2.11it/s, est. speed input: 69422.03 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 253/3051 [01:59<22:05,  2.11it/s, est. speed input: 69420.87 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 254/3051 [01:59<22:04,  2.11it/s, est. speed input: 69420.43 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 255/3051 [02:00<22:03,  2.11it/s, est. speed input: 69419.64 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 256/3051 [02:00<22:03,  2.11it/s, est. speed input: 69418.75 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 257/3051 [02:01<22:03,  2.11it/s, est. speed input: 69417.29 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 258/3051 [02:01<22:02,  2.11it/s, est. speed input: 69416.75 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 259/3051 [02:02<22:02,  2.11it/s, est. speed input: 69415.67 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 260/3051 [02:02<22:03,  2.11it/s, est. speed input: 69413.95 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 261/3051 [02:03<22:02,  2.11it/s, est. speed input: 69413.08 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 262/3051 [02:03<22:00,  2.11it/s, est. speed input: 69412.65 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 263/3051 [02:04<22:00,  2.11it/s, est. speed input: 69412.03 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 264/3051 [02:04<21:59,  2.11it/s, est. speed input: 69411.64 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 265/3051 [02:05<21:57,  2.11it/s, est. speed input: 69411.58 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 266/3051 [02:05<22:00,  2.11it/s, est. speed input: 69408.85 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 267/3051 [02:06<21:59,  2.11it/s, est. speed input: 69408.13 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 268/3051 [02:06<21:58,  2.11it/s, est. speed input: 69407.69 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 269/3051 [02:07<21:59,  2.11it/s, est. speed input: 69405.48 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 270/3051 [02:07<21:57,  2.11it/s, est. speed input: 69405.40 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 271/3051 [02:07<21:56,  2.11it/s, est. speed input: 69404.74 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 272/3051 [02:08<21:56,  2.11it/s, est. speed input: 69403.76 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 273/3051 [02:08<21:56,  2.11it/s, est. speed input: 69402.48 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 274/3051 [02:09<21:55,  2.11it/s, est. speed input: 69401.93 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 275/3051 [02:09<21:54,  2.11it/s, est. speed input: 69401.29 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 276/3051 [02:10<21:53,  2.11it/s, est. speed input: 69400.87 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 277/3051 [02:10<21:52,  2.11it/s, est. speed input: 69400.52 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 278/3051 [02:11<21:52,  2.11it/s, est. speed input: 69399.87 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 279/3051 [02:11<21:52,  2.11it/s, est. speed input: 69399.12 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 280/3051 [02:12<21:51,  2.11it/s, est. speed input: 69398.53 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 281/3051 [02:12<21:51,  2.11it/s, est. speed input: 69397.97 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 282/3051 [02:13<21:52,  2.11it/s, est. speed input: 69396.42 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 283/3051 [02:13<21:51,  2.11it/s, est. speed input: 69395.45 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 284/3051 [02:14<21:50,  2.11it/s, est. speed input: 69395.18 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 285/3051 [02:14<21:49,  2.11it/s, est. speed input: 69394.87 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 286/3051 [02:15<21:48,  2.11it/s, est. speed input: 69394.42 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 287/3051 [02:15<21:48,  2.11it/s, est. speed input: 69393.81 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 288/3051 [02:15<21:49,  2.11it/s, est. speed input: 69392.15 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 289/3051 [02:16<21:48,  2.11it/s, est. speed input: 69391.41 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 290/3051 [02:16<21:47,  2.11it/s, est. speed input: 69391.18 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 291/3051 [02:17<21:48,  2.11it/s, est. speed input: 69389.26 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 292/3051 [02:17<21:47,  2.11it/s, est. speed input: 69389.06 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 293/3051 [02:18<21:46,  2.11it/s, est. speed input: 69388.56 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 294/3051 [02:18<21:47,  2.11it/s, est. speed input: 69386.91 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 295/3051 [02:19<21:46,  2.11it/s, est. speed input: 69386.22 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 296/3051 [02:19<21:44,  2.11it/s, est. speed input: 69386.14 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 297/3051 [02:20<21:44,  2.11it/s, est. speed input: 69385.45 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 298/3051 [02:20<21:43,  2.11it/s, est. speed input: 69384.99 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 299/3051 [02:21<21:42,  2.11it/s, est. speed input: 69384.52 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 300/3051 [02:21<21:42,  2.11it/s, est. speed input: 69384.11 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 301/3051 [02:22<21:40,  2.11it/s, est. speed input: 69383.97 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 302/3051 [02:22<21:40,  2.11it/s, est. speed input: 69383.45 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 303/3051 [02:23<21:40,  2.11it/s, est. speed input: 69383.06 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 304/3051 [02:23<21:39,  2.11it/s, est. speed input: 69382.73 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 305/3051 [02:24<21:39,  2.11it/s, est. speed input: 69382.37 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 306/3051 [02:24<21:38,  2.11it/s, est. speed input: 69382.08 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 307/3051 [02:24<21:38,  2.11it/s, est. speed input: 69381.38 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 308/3051 [02:25<21:37,  2.11it/s, est. speed input: 69381.02 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 309/3051 [02:25<21:37,  2.11it/s, est. speed input: 69380.63 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 310/3051 [02:26<21:36,  2.11it/s, est. speed input: 69380.18 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 311/3051 [02:26<21:36,  2.11it/s, est. speed input: 69379.61 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 312/3051 [02:27<21:36,  2.11it/s, est. speed input: 69378.97 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 313/3051 [02:27<21:35,  2.11it/s, est. speed input: 69378.64 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 314/3051 [02:28<21:34,  2.11it/s, est. speed input: 69378.49 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 315/3051 [02:28<21:34,  2.11it/s, est. speed input: 69378.07 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 316/3051 [02:29<21:34,  2.11it/s, est. speed input: 69377.59 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 317/3051 [02:29<21:35,  2.11it/s, est. speed input: 69376.38 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 318/3051 [02:30<21:34,  2.11it/s, est. speed input: 69375.57 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 319/3051 [02:30<21:33,  2.11it/s, est. speed input: 69375.41 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 320/3051 [02:31<21:34,  2.11it/s, est. speed input: 69373.91 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 321/3051 [02:31<21:34,  2.11it/s, est. speed input: 69373.19 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 322/3051 [02:32<21:32,  2.11it/s, est. speed input: 69372.98 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 323/3051 [02:32<21:33,  2.11it/s, est. speed input: 69371.80 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 324/3051 [02:33<21:32,  2.11it/s, est. speed input: 69371.34 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 325/3051 [02:33<21:30,  2.11it/s, est. speed input: 69371.15 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 326/3051 [02:33<21:32,  2.11it/s, est. speed input: 69369.71 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 327/3051 [02:34<21:30,  2.11it/s, est. speed input: 69369.32 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 328/3051 [02:34<21:29,  2.11it/s, est. speed input: 69369.08 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 329/3051 [02:35<21:28,  2.11it/s, est. speed input: 69368.74 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 330/3051 [02:35<21:28,  2.11it/s, est. speed input: 69368.33 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 331/3051 [02:36<21:28,  2.11it/s, est. speed input: 69367.43 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 332/3051 [02:36<21:27,  2.11it/s, est. speed input: 69366.86 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 333/3051 [02:37<21:26,  2.11it/s, est. speed input: 69366.73 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 334/3051 [02:37<21:26,  2.11it/s, est. speed input: 69366.39 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 335/3051 [02:38<21:25,  2.11it/s, est. speed input: 69366.11 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 336/3051 [02:38<21:25,  2.11it/s, est. speed input: 69365.46 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 337/3051 [02:39<21:25,  2.11it/s, est. speed input: 69364.42 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 338/3051 [02:39<21:24,  2.11it/s, est. speed input: 69364.27 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 339/3051 [02:40<21:23,  2.11it/s, est. speed input: 69364.23 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 340/3051 [02:40<21:27,  2.11it/s, est. speed input: 69361.64 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 341/3051 [02:41<21:25,  2.11it/s, est. speed input: 69361.28 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 342/3051 [02:41<21:23,  2.11it/s, est. speed input: 69361.20 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 343/3051 [02:42<21:21,  2.11it/s, est. speed input: 69361.20 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 344/3051 [02:42<21:20,  2.11it/s, est. speed input: 69361.10 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 345/3051 [02:42<21:22,  2.11it/s, est. speed input: 69359.81 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 346/3051 [02:43<21:21,  2.11it/s, est. speed input: 69359.11 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 347/3051 [02:43<21:21,  2.11it/s, est. speed input: 69358.62 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 348/3051 [02:44<21:20,  2.11it/s, est. speed input: 69358.40 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 349/3051 [02:44<21:18,  2.11it/s, est. speed input: 69358.39 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 350/3051 [02:45<21:19,  2.11it/s, est. speed input: 69357.16 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 351/3051 [02:45<21:19,  2.11it/s, est. speed input: 69356.36 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 352/3051 [02:46<21:18,  2.11it/s, est. speed input: 69356.30 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 353/3051 [02:46<21:17,  2.11it/s, est. speed input: 69355.97 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 354/3051 [02:47<21:16,  2.11it/s, est. speed input: 69355.91 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 355/3051 [02:47<21:15,  2.11it/s, est. speed input: 69355.65 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 356/3051 [02:48<21:15,  2.11it/s, est. speed input: 69355.10 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 357/3051 [02:48<21:15,  2.11it/s, est. speed input: 69354.40 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 358/3051 [02:49<21:15,  2.11it/s, est. speed input: 69353.85 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 359/3051 [02:49<21:14,  2.11it/s, est. speed input: 69353.68 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 360/3051 [02:50<21:13,  2.11it/s, est. speed input: 69353.60 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 361/3051 [02:50<21:12,  2.11it/s, est. speed input: 69353.54 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 362/3051 [02:51<21:12,  2.11it/s, est. speed input: 69353.28 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 363/3051 [02:51<21:11,  2.11it/s, est. speed input: 69353.00 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 364/3051 [02:51<21:11,  2.11it/s, est. speed input: 69352.50 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 365/3051 [02:52<21:10,  2.11it/s, est. speed input: 69352.50 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 366/3051 [02:52<21:10,  2.11it/s, est. speed input: 69352.26 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 367/3051 [02:53<21:09,  2.11it/s, est. speed input: 69351.98 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 368/3051 [02:53<21:09,  2.11it/s, est. speed input: 69351.58 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 369/3051 [02:54<21:08,  2.11it/s, est. speed input: 69351.52 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 370/3051 [02:54<21:08,  2.11it/s, est. speed input: 69351.22 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 371/3051 [02:55<21:07,  2.11it/s, est. speed input: 69350.99 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 372/3051 [02:55<21:06,  2.11it/s, est. speed input: 69350.99 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 373/3051 [02:56<21:06,  2.11it/s, est. speed input: 69350.89 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 374/3051 [02:56<21:07,  2.11it/s, est. speed input: 69349.95 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 375/3051 [02:57<21:07,  2.11it/s, est. speed input: 69349.46 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 376/3051 [02:57<21:06,  2.11it/s, est. speed input: 69349.04 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 377/3051 [02:58<21:05,  2.11it/s, est. speed input: 69348.88 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 378/3051 [02:58<21:05,  2.11it/s, est. speed input: 69348.51 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 379/3051 [02:59<21:05,  2.11it/s, est. speed input: 69348.03 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 380/3051 [02:59<21:04,  2.11it/s, est. speed input: 69347.64 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 381/3051 [03:00<21:03,  2.11it/s, est. speed input: 69347.52 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 382/3051 [03:00<21:03,  2.11it/s, est. speed input: 69347.28 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 383/3051 [03:00<21:02,  2.11it/s, est. speed input: 69347.03 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 384/3051 [03:01<21:04,  2.11it/s, est. speed input: 69345.73 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 385/3051 [03:01<21:04,  2.11it/s, est. speed input: 69344.62 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 386/3051 [03:02<21:03,  2.11it/s, est. speed input: 69344.43 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 387/3051 [03:02<21:02,  2.11it/s, est. speed input: 69344.21 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 388/3051 [03:03<21:00,  2.11it/s, est. speed input: 69344.15 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 389/3051 [03:03<21:00,  2.11it/s, est. speed input: 69343.92 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 390/3051 [03:04<21:01,  2.11it/s, est. speed input: 69342.59 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 391/3051 [03:04<21:01,  2.11it/s, est. speed input: 69341.93 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 392/3051 [03:05<20:59,  2.11it/s, est. speed input: 69341.89 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 393/3051 [03:05<20:58,  2.11it/s, est. speed input: 69341.66 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 394/3051 [03:06<20:57,  2.11it/s, est. speed input: 69341.60 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 395/3051 [03:06<20:56,  2.11it/s, est. speed input: 69341.45 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 396/3051 [03:07<20:56,  2.11it/s, est. speed input: 69341.31 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 397/3051 [03:07<20:56,  2.11it/s, est. speed input: 69340.96 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 398/3051 [03:08<20:55,  2.11it/s, est. speed input: 69340.74 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 399/3051 [03:08<20:55,  2.11it/s, est. speed input: 69340.34 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 400/3051 [03:09<20:55,  2.11it/s, est. speed input: 69339.93 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 401/3051 [03:09<20:55,  2.11it/s, est. speed input: 69339.19 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 402/3051 [03:09<20:54,  2.11it/s, est. speed input: 69339.03 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 403/3051 [03:10<20:53,  2.11it/s, est. speed input: 69338.82 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 404/3051 [03:10<20:52,  2.11it/s, est. speed input: 69338.84 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 405/3051 [03:11<20:52,  2.11it/s, est. speed input: 69338.54 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 406/3051 [03:11<20:51,  2.11it/s, est. speed input: 69338.30 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 407/3051 [03:12<20:52,  2.11it/s, est. speed input: 69337.60 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 408/3051 [03:12<20:50,  2.11it/s, est. speed input: 69337.76 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 409/3051 [03:13<21:31,  2.05it/s, est. speed input: 69318.71 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 410/3051 [03:13<21:17,  2.07it/s, est. speed input: 69318.99 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 411/3051 [03:14<21:08,  2.08it/s, est. speed input: 69318.88 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 412/3051 [03:14<21:01,  2.09it/s, est. speed input: 69318.92 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 413/3051 [03:15<20:58,  2.10it/s, est. speed input: 69318.51 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 414/3051 [03:15<20:54,  2.10it/s, est. speed input: 69318.52 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 415/3051 [03:16<20:51,  2.11it/s, est. speed input: 69318.46 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 416/3051 [03:16<20:49,  2.11it/s, est. speed input: 69318.60 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 417/3051 [03:17<20:47,  2.11it/s, est. speed input: 69318.48 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 418/3051 [03:17<20:46,  2.11it/s, est. speed input: 69318.42 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 419/3051 [03:18<20:45,  2.11it/s, est. speed input: 69318.51 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 420/3051 [03:18<20:45,  2.11it/s, est. speed input: 69318.28 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 421/3051 [03:19<20:44,  2.11it/s, est. speed input: 69318.00 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 422/3051 [03:19<20:43,  2.11it/s, est. speed input: 69318.05 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 423/3051 [03:19<20:42,  2.11it/s, est. speed input: 69318.12 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 424/3051 [03:20<20:42,  2.11it/s, est. speed input: 69317.99 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 425/3051 [03:20<20:41,  2.11it/s, est. speed input: 69317.97 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 426/3051 [03:21<20:41,  2.12it/s, est. speed input: 69318.07 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 427/3051 [03:21<20:40,  2.12it/s, est. speed input: 69318.16 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 428/3051 [03:22<20:40,  2.12it/s, est. speed input: 69318.06 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 429/3051 [03:22<20:39,  2.11it/s, est. speed input: 69317.94 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 430/3051 [03:23<20:39,  2.11it/s, est. speed input: 69317.86 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 431/3051 [03:23<20:38,  2.11it/s, est. speed input: 69317.81 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 432/3051 [03:24<20:38,  2.11it/s, est. speed input: 69317.61 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 433/3051 [03:24<20:37,  2.12it/s, est. speed input: 69317.76 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 434/3051 [03:25<20:36,  2.12it/s, est. speed input: 69317.87 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 435/3051 [03:25<20:36,  2.12it/s, est. speed input: 69317.94 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 436/3051 [03:26<20:36,  2.12it/s, est. speed input: 69317.86 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 437/3051 [03:26<20:36,  2.11it/s, est. speed input: 69317.73 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 438/3051 [03:27<20:35,  2.11it/s, est. speed input: 69317.62 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 439/3051 [03:27<20:35,  2.11it/s, est. speed input: 69317.55 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 440/3051 [03:27<20:35,  2.11it/s, est. speed input: 69317.39 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 441/3051 [03:28<20:36,  2.11it/s, est. speed input: 69316.30 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 442/3051 [03:28<20:37,  2.11it/s, est. speed input: 69315.59 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 443/3051 [03:29<20:35,  2.11it/s, est. speed input: 69315.45 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 444/3051 [03:29<20:34,  2.11it/s, est. speed input: 69315.35 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 445/3051 [03:30<20:33,  2.11it/s, est. speed input: 69315.27 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 446/3051 [03:30<20:32,  2.11it/s, est. speed input: 69315.27 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 447/3051 [03:31<20:42,  2.10it/s, est. speed input: 69310.80 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 448/3051 [03:31<20:38,  2.10it/s, est. speed input: 69310.69 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 449/3051 [03:32<20:35,  2.11it/s, est. speed input: 69310.68 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 450/3051 [03:32<20:33,  2.11it/s, est. speed input: 69310.79 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 451/3051 [03:33<20:32,  2.11it/s, est. speed input: 69310.53 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 452/3051 [03:33<20:31,  2.11it/s, est. speed input: 69310.38 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 453/3051 [03:34<20:30,  2.11it/s, est. speed input: 69310.34 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 454/3051 [03:34<20:28,  2.11it/s, est. speed input: 69310.51 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 455/3051 [03:35<20:28,  2.11it/s, est. speed input: 69310.48 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 456/3051 [03:35<20:27,  2.11it/s, est. speed input: 69310.44 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 457/3051 [03:36<20:26,  2.11it/s, est. speed input: 69310.45 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 458/3051 [03:36<20:26,  2.11it/s, est. speed input: 69310.44 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 459/3051 [03:37<20:26,  2.11it/s, est. speed input: 69310.20 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 460/3051 [03:37<20:25,  2.11it/s, est. speed input: 69310.08 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 461/3051 [03:37<20:24,  2.11it/s, est. speed input: 69310.13 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 462/3051 [03:38<20:25,  2.11it/s, est. speed input: 69309.77 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 463/3051 [03:38<20:26,  2.11it/s, est. speed input: 69309.00 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 464/3051 [03:39<20:25,  2.11it/s, est. speed input: 69308.85 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 465/3051 [03:39<20:24,  2.11it/s, est. speed input: 69308.63 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 466/3051 [03:40<20:23,  2.11it/s, est. speed input: 69308.57 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 467/3051 [03:40<20:23,  2.11it/s, est. speed input: 69308.46 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 468/3051 [03:41<20:21,  2.11it/s, est. speed input: 69308.58 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 469/3051 [03:41<20:21,  2.11it/s, est. speed input: 69308.36 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 470/3051 [03:42<20:20,  2.11it/s, est. speed input: 69308.42 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 471/3051 [03:42<20:20,  2.11it/s, est. speed input: 69308.25 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 472/3051 [03:43<20:20,  2.11it/s, est. speed input: 69308.14 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 473/3051 [03:43<20:19,  2.11it/s, est. speed input: 69308.25 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 474/3051 [03:44<20:18,  2.11it/s, est. speed input: 69308.12 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 475/3051 [03:44<20:18,  2.11it/s, est. speed input: 69308.15 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 476/3051 [03:45<20:17,  2.11it/s, est. speed input: 69308.18 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 477/3051 [03:45<20:17,  2.11it/s, est. speed input: 69308.16 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 478/3051 [03:45<20:16,  2.11it/s, est. speed input: 69308.14 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 479/3051 [03:46<20:16,  2.11it/s, est. speed input: 69308.05 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 480/3051 [03:46<20:15,  2.11it/s, est. speed input: 69307.96 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 481/3051 [03:47<20:15,  2.11it/s, est. speed input: 69307.91 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 482/3051 [03:47<20:14,  2.11it/s, est. speed input: 69307.95 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 483/3051 [03:48<20:13,  2.12it/s, est. speed input: 69308.17 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 484/3051 [03:48<20:14,  2.11it/s, est. speed input: 69307.77 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 485/3051 [03:49<20:14,  2.11it/s, est. speed input: 69307.61 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 486/3051 [03:49<20:13,  2.11it/s, est. speed input: 69307.58 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 487/3051 [03:50<20:13,  2.11it/s, est. speed input: 69307.46 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 488/3051 [03:50<20:13,  2.11it/s, est. speed input: 69307.15 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 489/3051 [03:51<20:12,  2.11it/s, est. speed input: 69307.03 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 490/3051 [03:51<20:12,  2.11it/s, est. speed input: 69306.76 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 491/3051 [03:52<20:11,  2.11it/s, est. speed input: 69306.88 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 492/3051 [03:52<20:28,  2.08it/s, est. speed input: 69299.90 toks/s, output: 2.11 toks/s]
Processed prompts:  16%|█▌        | 493/3051 [03:53<20:22,  2.09it/s, est. speed input: 69299.98 toks/s, output: 2.11 toks/s]
Processed prompts:  16%|█▌        | 494/3051 [03:53<20:17,  2.10it/s, est. speed input: 69300.03 toks/s, output: 2.11 toks/s]
Processed prompts:  16%|█▌        | 495/3051 [03:54<20:14,  2.10it/s, est. speed input: 69300.02 toks/s, output: 2.11 toks/s]
Processed prompts:  16%|█▋        | 496/3051 [03:54<20:12,  2.11it/s, est. speed input: 69300.03 toks/s, output: 2.11 toks/s]
Processed prompts:  16%|█▋        | 497/3051 [03:55<20:11,  2.11it/s, est. speed input: 69299.70 toks/s, output: 2.11 toks/s]
Processed prompts:  16%|█▋        | 498/3051 [03:55<20:09,  2.11it/s, est. speed input: 69299.78 toks/s, output: 2.11 toks/s]
Processed prompts:  16%|█▋        | 499/3051 [03:55<20:08,  2.11it/s, est. speed input: 69299.68 toks/s, output: 2.11 toks/s]
Processed prompts:  16%|█▋        | 500/3051 [03:56<20:09,  2.11it/s, est. speed input: 69299.07 toks/s, output: 2.11 toks/s]
Processed prompts:  16%|█▋        | 501/3051 [03:56<20:07,  2.11it/s, est. speed input: 69299.13 toks/s, output: 2.11 toks/s]
Processed prompts:  16%|█▋        | 502/3051 [03:57<20:06,  2.11it/s, est. speed input: 69299.21 toks/s, output: 2.11 toks/s]
Processed prompts:  16%|█▋        | 503/3051 [03:57<20:05,  2.11it/s, est. speed input: 69299.29 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 504/3051 [03:58<20:04,  2.11it/s, est. speed input: 69299.38 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 505/3051 [03:58<20:08,  2.11it/s, est. speed input: 69297.59 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 506/3051 [03:59<20:08,  2.11it/s, est. speed input: 69296.98 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 507/3051 [03:59<20:06,  2.11it/s, est. speed input: 69297.07 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 508/3051 [04:00<20:05,  2.11it/s, est. speed input: 69296.97 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 509/3051 [04:00<20:03,  2.11it/s, est. speed input: 69296.91 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 510/3051 [04:01<20:02,  2.11it/s, est. speed input: 69297.07 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 511/3051 [04:01<20:01,  2.11it/s, est. speed input: 69297.07 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 512/3051 [04:02<20:01,  2.11it/s, est. speed input: 69296.99 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 513/3051 [04:02<20:00,  2.11it/s, est. speed input: 69297.06 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 514/3051 [04:03<19:59,  2.11it/s, est. speed input: 69297.11 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 515/3051 [04:03<19:59,  2.11it/s, est. speed input: 69296.97 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 516/3051 [04:03<19:59,  2.11it/s, est. speed input: 69296.71 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 517/3051 [04:04<19:58,  2.11it/s, est. speed input: 69296.79 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 518/3051 [04:04<19:58,  2.11it/s, est. speed input: 69296.76 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 519/3051 [04:05<19:57,  2.11it/s, est. speed input: 69296.76 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 520/3051 [04:05<19:56,  2.12it/s, est. speed input: 69296.89 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 521/3051 [04:06<19:59,  2.11it/s, est. speed input: 69295.85 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 522/3051 [04:06<19:58,  2.11it/s, est. speed input: 69295.48 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 523/3051 [04:07<19:57,  2.11it/s, est. speed input: 69295.63 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 524/3051 [04:07<19:56,  2.11it/s, est. speed input: 69295.53 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 525/3051 [04:08<19:55,  2.11it/s, est. speed input: 69295.59 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 526/3051 [04:08<19:55,  2.11it/s, est. speed input: 69295.15 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 527/3051 [04:09<19:55,  2.11it/s, est. speed input: 69294.99 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 528/3051 [04:09<19:54,  2.11it/s, est. speed input: 69294.90 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 529/3051 [04:10<19:54,  2.11it/s, est. speed input: 69294.45 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 530/3051 [04:10<19:53,  2.11it/s, est. speed input: 69294.38 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 531/3051 [04:11<19:53,  2.11it/s, est. speed input: 69294.19 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 532/3051 [04:11<19:52,  2.11it/s, est. speed input: 69294.07 toks/s, output: 2.11 toks/s]
Processed prompts:  17%|█▋        | 533/3051 [04:12<19:51,  2.11it/s, est. speed input: 69294.22 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 534/3051 [04:12<19:52,  2.11it/s, est. speed input: 69293.62 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 535/3051 [04:12<19:51,  2.11it/s, est. speed input: 69293.59 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 536/3051 [04:13<19:50,  2.11it/s, est. speed input: 69293.60 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 537/3051 [04:13<19:51,  2.11it/s, est. speed input: 69292.73 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 538/3051 [04:14<19:52,  2.11it/s, est. speed input: 69292.02 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 539/3051 [04:14<19:50,  2.11it/s, est. speed input: 69291.99 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 540/3051 [04:15<19:49,  2.11it/s, est. speed input: 69292.07 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 541/3051 [04:15<19:49,  2.11it/s, est. speed input: 69291.67 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 542/3051 [04:16<19:48,  2.11it/s, est. speed input: 69291.73 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 543/3051 [04:16<19:46,  2.11it/s, est. speed input: 69291.82 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 544/3051 [04:17<19:45,  2.12it/s, est. speed input: 69292.15 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 545/3051 [04:17<19:44,  2.12it/s, est. speed input: 69292.25 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 546/3051 [04:18<19:44,  2.11it/s, est. speed input: 69292.16 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 547/3051 [04:18<19:44,  2.11it/s, est. speed input: 69291.89 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 548/3051 [04:19<19:44,  2.11it/s, est. speed input: 69291.73 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 549/3051 [04:19<19:43,  2.11it/s, est. speed input: 69291.80 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 550/3051 [04:20<19:43,  2.11it/s, est. speed input: 69291.56 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 551/3051 [04:20<19:44,  2.11it/s, est. speed input: 69291.13 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 552/3051 [04:21<19:42,  2.11it/s, est. speed input: 69291.28 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 553/3051 [04:21<19:42,  2.11it/s, est. speed input: 69291.23 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 554/3051 [04:21<19:41,  2.11it/s, est. speed input: 69291.33 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 555/3051 [04:22<19:40,  2.11it/s, est. speed input: 69291.40 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 556/3051 [04:22<19:39,  2.12it/s, est. speed input: 69291.51 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 557/3051 [04:23<19:39,  2.11it/s, est. speed input: 69291.40 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 558/3051 [04:23<19:38,  2.11it/s, est. speed input: 69291.49 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 559/3051 [04:24<19:37,  2.12it/s, est. speed input: 69291.66 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 560/3051 [04:24<19:39,  2.11it/s, est. speed input: 69291.11 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 561/3051 [04:25<19:38,  2.11it/s, est. speed input: 69291.16 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 562/3051 [04:25<19:37,  2.11it/s, est. speed input: 69291.15 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 563/3051 [04:26<19:36,  2.11it/s, est. speed input: 69291.27 toks/s, output: 2.11 toks/s]
Processed prompts:  18%|█▊        | 564/3051 [04:26<19:36,  2.11it/s, est. speed input: 69291.20 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▊        | 565/3051 [04:27<19:35,  2.12it/s, est. speed input: 69291.36 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▊        | 566/3051 [04:27<19:35,  2.11it/s, est. speed input: 69291.26 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▊        | 567/3051 [04:28<19:34,  2.12it/s, est. speed input: 69291.41 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▊        | 568/3051 [04:28<19:34,  2.11it/s, est. speed input: 69291.26 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▊        | 569/3051 [04:29<19:34,  2.11it/s, est. speed input: 69291.14 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▊        | 570/3051 [04:29<19:34,  2.11it/s, est. speed input: 69290.98 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▊        | 571/3051 [04:30<19:33,  2.11it/s, est. speed input: 69290.95 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▊        | 572/3051 [04:30<19:32,  2.11it/s, est. speed input: 69291.02 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 573/3051 [04:30<19:32,  2.11it/s, est. speed input: 69290.98 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 574/3051 [04:31<19:31,  2.11it/s, est. speed input: 69290.91 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 575/3051 [04:31<19:31,  2.11it/s, est. speed input: 69290.92 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 576/3051 [04:32<19:35,  2.11it/s, est. speed input: 69289.38 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 577/3051 [04:32<19:34,  2.11it/s, est. speed input: 69289.11 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 578/3051 [04:33<19:32,  2.11it/s, est. speed input: 69289.01 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 579/3051 [04:33<19:31,  2.11it/s, est. speed input: 69288.86 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 580/3051 [04:34<19:30,  2.11it/s, est. speed input: 69288.85 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 581/3051 [04:34<19:29,  2.11it/s, est. speed input: 69288.73 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 582/3051 [04:35<19:28,  2.11it/s, est. speed input: 69288.82 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 583/3051 [04:35<19:27,  2.11it/s, est. speed input: 69288.85 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 584/3051 [04:36<19:27,  2.11it/s, est. speed input: 69288.55 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 585/3051 [04:36<19:27,  2.11it/s, est. speed input: 69288.49 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 586/3051 [04:37<19:27,  2.11it/s, est. speed input: 69288.22 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 587/3051 [04:37<19:26,  2.11it/s, est. speed input: 69288.07 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 588/3051 [04:38<19:25,  2.11it/s, est. speed input: 69288.16 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 589/3051 [04:38<19:25,  2.11it/s, est. speed input: 69287.89 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 590/3051 [04:39<19:25,  2.11it/s, est. speed input: 69287.49 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 591/3051 [04:39<19:25,  2.11it/s, est. speed input: 69287.36 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 592/3051 [04:39<19:23,  2.11it/s, est. speed input: 69287.48 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 593/3051 [04:40<19:23,  2.11it/s, est. speed input: 69287.42 toks/s, output: 2.11 toks/s]
Processed prompts:  19%|█▉        | 594/3051 [04:40<19:23,  2.11it/s, est. speed input: 69287.27 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 595/3051 [04:41<19:22,  2.11it/s, est. speed input: 69287.35 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 596/3051 [04:41<19:21,  2.11it/s, est. speed input: 69287.26 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 597/3051 [04:42<19:20,  2.11it/s, est. speed input: 69287.37 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 598/3051 [04:42<19:21,  2.11it/s, est. speed input: 69286.77 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 599/3051 [04:43<19:21,  2.11it/s, est. speed input: 69286.65 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 600/3051 [04:43<19:20,  2.11it/s, est. speed input: 69286.68 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 601/3051 [04:44<19:19,  2.11it/s, est. speed input: 69286.73 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 602/3051 [04:44<19:18,  2.11it/s, est. speed input: 69286.65 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 603/3051 [04:45<19:17,  2.11it/s, est. speed input: 69286.77 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 604/3051 [04:45<19:17,  2.11it/s, est. speed input: 69286.72 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 605/3051 [04:46<19:17,  2.11it/s, est. speed input: 69286.58 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 606/3051 [04:46<19:17,  2.11it/s, est. speed input: 69286.36 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 607/3051 [04:47<19:16,  2.11it/s, est. speed input: 69286.44 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 608/3051 [04:47<19:17,  2.11it/s, est. speed input: 69285.82 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 609/3051 [04:48<19:16,  2.11it/s, est. speed input: 69285.72 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|█▉        | 610/3051 [04:48<19:16,  2.11it/s, est. speed input: 69285.60 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 611/3051 [04:48<19:16,  2.11it/s, est. speed input: 69285.28 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 612/3051 [04:49<19:15,  2.11it/s, est. speed input: 69285.23 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 613/3051 [04:49<19:14,  2.11it/s, est. speed input: 69285.15 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 614/3051 [04:50<19:14,  2.11it/s, est. speed input: 69284.83 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 615/3051 [04:50<19:14,  2.11it/s, est. speed input: 69284.61 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 616/3051 [04:51<19:12,  2.11it/s, est. speed input: 69284.72 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 617/3051 [04:51<19:11,  2.11it/s, est. speed input: 69284.88 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 618/3051 [04:52<19:11,  2.11it/s, est. speed input: 69284.83 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 619/3051 [04:52<19:10,  2.11it/s, est. speed input: 69284.97 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 620/3051 [04:53<19:09,  2.11it/s, est. speed input: 69284.90 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 621/3051 [04:53<19:08,  2.11it/s, est. speed input: 69285.02 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 622/3051 [04:54<19:08,  2.11it/s, est. speed input: 69284.96 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 623/3051 [04:54<19:07,  2.12it/s, est. speed input: 69285.22 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 624/3051 [04:55<19:07,  2.11it/s, est. speed input: 69285.09 toks/s, output: 2.11 toks/s]
Processed prompts:  20%|██        | 625/3051 [04:55<19:07,  2.11it/s, est. speed input: 69285.04 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 626/3051 [04:56<19:06,  2.11it/s, est. speed input: 69285.12 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 627/3051 [04:56<19:06,  2.11it/s, est. speed input: 69285.14 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 628/3051 [04:57<19:06,  2.11it/s, est. speed input: 69284.99 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 629/3051 [04:57<19:05,  2.11it/s, est. speed input: 69284.98 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 630/3051 [04:57<19:04,  2.11it/s, est. speed input: 69285.10 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 631/3051 [04:58<19:04,  2.11it/s, est. speed input: 69284.91 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 632/3051 [04:58<19:05,  2.11it/s, est. speed input: 69284.42 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 633/3051 [04:59<19:05,  2.11it/s, est. speed input: 69284.33 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 634/3051 [04:59<19:05,  2.11it/s, est. speed input: 69283.85 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 635/3051 [05:00<19:04,  2.11it/s, est. speed input: 69283.86 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 636/3051 [05:00<19:03,  2.11it/s, est. speed input: 69283.99 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 637/3051 [05:01<19:05,  2.11it/s, est. speed input: 69282.82 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 638/3051 [05:01<19:03,  2.11it/s, est. speed input: 69282.90 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 639/3051 [05:02<19:02,  2.11it/s, est. speed input: 69282.89 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 640/3051 [05:02<19:01,  2.11it/s, est. speed input: 69282.86 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 641/3051 [05:03<19:00,  2.11it/s, est. speed input: 69283.10 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 642/3051 [05:03<18:59,  2.11it/s, est. speed input: 69283.21 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 643/3051 [05:04<19:00,  2.11it/s, est. speed input: 69282.53 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 644/3051 [05:04<18:59,  2.11it/s, est. speed input: 69282.66 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 645/3051 [05:05<18:58,  2.11it/s, est. speed input: 69282.76 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 646/3051 [05:05<18:58,  2.11it/s, est. speed input: 69282.67 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 647/3051 [05:06<18:57,  2.11it/s, est. speed input: 69282.56 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██        | 648/3051 [05:06<18:58,  2.11it/s, est. speed input: 69282.07 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██▏       | 649/3051 [05:06<18:57,  2.11it/s, est. speed input: 69282.02 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██▏       | 650/3051 [05:07<18:57,  2.11it/s, est. speed input: 69281.85 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██▏       | 651/3051 [05:07<18:56,  2.11it/s, est. speed input: 69281.78 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██▏       | 652/3051 [05:08<18:55,  2.11it/s, est. speed input: 69281.79 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██▏       | 653/3051 [05:08<18:54,  2.11it/s, est. speed input: 69281.78 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██▏       | 654/3051 [05:09<18:54,  2.11it/s, est. speed input: 69281.69 toks/s, output: 2.11 toks/s]
Processed prompts:  21%|██▏       | 655/3051 [05:09<18:53,  2.11it/s, est. speed input: 69281.81 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 656/3051 [05:10<18:53,  2.11it/s, est. speed input: 69281.71 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 657/3051 [05:10<18:52,  2.11it/s, est. speed input: 69281.67 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 658/3051 [05:11<18:52,  2.11it/s, est. speed input: 69281.69 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 659/3051 [05:11<18:53,  2.11it/s, est. speed input: 69280.94 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 660/3051 [05:12<18:53,  2.11it/s, est. speed input: 69280.62 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 661/3051 [05:12<18:52,  2.11it/s, est. speed input: 69280.62 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 662/3051 [05:13<18:50,  2.11it/s, est. speed input: 69280.75 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 663/3051 [05:13<18:50,  2.11it/s, est. speed input: 69280.73 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 664/3051 [05:14<18:49,  2.11it/s, est. speed input: 69280.62 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 665/3051 [05:14<18:49,  2.11it/s, est. speed input: 69280.63 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 666/3051 [05:15<18:49,  2.11it/s, est. speed input: 69280.39 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 667/3051 [05:15<18:50,  2.11it/s, est. speed input: 69279.61 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 668/3051 [05:15<18:49,  2.11it/s, est. speed input: 69279.70 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 669/3051 [05:16<18:48,  2.11it/s, est. speed input: 69279.56 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 670/3051 [05:16<18:48,  2.11it/s, est. speed input: 69279.34 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 671/3051 [05:17<18:46,  2.11it/s, est. speed input: 69279.42 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 672/3051 [05:17<18:45,  2.11it/s, est. speed input: 69279.49 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 673/3051 [05:18<18:47,  2.11it/s, est. speed input: 69278.84 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 674/3051 [05:18<18:46,  2.11it/s, est. speed input: 69278.82 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 675/3051 [05:19<18:45,  2.11it/s, est. speed input: 69278.72 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 676/3051 [05:19<18:44,  2.11it/s, est. speed input: 69278.74 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 677/3051 [05:20<18:43,  2.11it/s, est. speed input: 69278.73 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 678/3051 [05:20<18:42,  2.11it/s, est. speed input: 69278.74 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 679/3051 [05:21<18:42,  2.11it/s, est. speed input: 69278.79 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 680/3051 [05:21<18:41,  2.11it/s, est. speed input: 69278.75 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 681/3051 [05:22<18:40,  2.11it/s, est. speed input: 69278.83 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 682/3051 [05:22<18:40,  2.12it/s, est. speed input: 69278.94 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 683/3051 [05:23<18:39,  2.12it/s, est. speed input: 69278.99 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 684/3051 [05:23<18:39,  2.11it/s, est. speed input: 69279.01 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 685/3051 [05:23<18:39,  2.11it/s, est. speed input: 69278.78 toks/s, output: 2.11 toks/s]
Processed prompts:  22%|██▏       | 686/3051 [05:24<18:38,  2.11it/s, est. speed input: 69278.78 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 687/3051 [05:24<18:38,  2.11it/s, est. speed input: 69278.71 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 688/3051 [05:25<18:37,  2.11it/s, est. speed input: 69278.79 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 689/3051 [05:25<18:36,  2.11it/s, est. speed input: 69278.92 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 690/3051 [05:26<18:35,  2.12it/s, est. speed input: 69279.09 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 691/3051 [05:26<18:35,  2.12it/s, est. speed input: 69279.10 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 692/3051 [05:27<18:35,  2.11it/s, est. speed input: 69279.06 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 693/3051 [05:27<18:35,  2.11it/s, est. speed input: 69279.05 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 694/3051 [05:28<18:34,  2.12it/s, est. speed input: 69279.17 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 695/3051 [05:28<18:34,  2.11it/s, est. speed input: 69279.04 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 696/3051 [05:29<18:34,  2.11it/s, est. speed input: 69278.96 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 697/3051 [05:29<18:33,  2.11it/s, est. speed input: 69279.06 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 698/3051 [05:30<18:34,  2.11it/s, est. speed input: 69278.71 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 699/3051 [05:30<18:33,  2.11it/s, est. speed input: 69278.71 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 700/3051 [05:31<18:32,  2.11it/s, est. speed input: 69278.70 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 701/3051 [05:31<18:33,  2.11it/s, est. speed input: 69278.33 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 702/3051 [05:32<18:32,  2.11it/s, est. speed input: 69278.29 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 703/3051 [05:32<18:30,  2.11it/s, est. speed input: 69278.44 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 704/3051 [05:32<18:30,  2.11it/s, est. speed input: 69278.50 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 705/3051 [05:33<18:29,  2.11it/s, est. speed input: 69278.49 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 706/3051 [05:33<18:29,  2.11it/s, est. speed input: 69278.54 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 707/3051 [05:34<18:28,  2.11it/s, est. speed input: 69278.55 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 708/3051 [05:34<18:28,  2.11it/s, est. speed input: 69278.38 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 709/3051 [05:35<18:28,  2.11it/s, est. speed input: 69278.27 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 710/3051 [05:35<18:28,  2.11it/s, est. speed input: 69278.16 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 711/3051 [05:36<18:26,  2.11it/s, est. speed input: 69278.33 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 712/3051 [05:36<18:26,  2.11it/s, est. speed input: 69278.31 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 713/3051 [05:37<18:25,  2.11it/s, est. speed input: 69278.34 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 714/3051 [05:37<18:26,  2.11it/s, est. speed input: 69278.01 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 715/3051 [05:38<18:25,  2.11it/s, est. speed input: 69278.01 toks/s, output: 2.11 toks/s]
Processed prompts:  23%|██▎       | 716/3051 [05:38<18:25,  2.11it/s, est. speed input: 69277.95 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▎       | 717/3051 [05:39<18:24,  2.11it/s, est. speed input: 69277.95 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▎       | 718/3051 [05:39<18:23,  2.11it/s, est. speed input: 69277.94 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▎       | 719/3051 [05:40<18:23,  2.11it/s, est. speed input: 69277.78 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▎       | 720/3051 [05:40<18:23,  2.11it/s, est. speed input: 69277.61 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▎       | 721/3051 [05:41<18:24,  2.11it/s, est. speed input: 69277.12 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▎       | 722/3051 [05:41<18:23,  2.11it/s, est. speed input: 69277.20 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▎       | 723/3051 [05:41<18:22,  2.11it/s, est. speed input: 69277.05 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▎       | 724/3051 [05:42<18:25,  2.11it/s, est. speed input: 69276.09 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 725/3051 [05:42<18:23,  2.11it/s, est. speed input: 69276.16 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 726/3051 [05:43<18:21,  2.11it/s, est. speed input: 69276.11 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 727/3051 [05:43<18:20,  2.11it/s, est. speed input: 69276.14 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 728/3051 [05:44<18:19,  2.11it/s, est. speed input: 69276.12 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 729/3051 [05:44<18:18,  2.11it/s, est. speed input: 69276.33 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 730/3051 [05:45<18:18,  2.11it/s, est. speed input: 69276.25 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 731/3051 [05:45<18:17,  2.11it/s, est. speed input: 69276.17 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 732/3051 [05:46<18:16,  2.11it/s, est. speed input: 69276.29 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 733/3051 [05:46<18:16,  2.11it/s, est. speed input: 69276.16 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 734/3051 [05:47<18:15,  2.11it/s, est. speed input: 69276.23 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 735/3051 [05:47<18:16,  2.11it/s, est. speed input: 69276.03 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 736/3051 [05:48<18:15,  2.11it/s, est. speed input: 69276.09 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 737/3051 [05:48<18:15,  2.11it/s, est. speed input: 69275.95 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 738/3051 [05:49<18:15,  2.11it/s, est. speed input: 69275.79 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 739/3051 [05:49<18:14,  2.11it/s, est. speed input: 69275.77 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 740/3051 [05:50<18:13,  2.11it/s, est. speed input: 69275.74 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 741/3051 [05:50<18:13,  2.11it/s, est. speed input: 69275.72 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 742/3051 [05:50<18:12,  2.11it/s, est. speed input: 69275.76 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 743/3051 [05:51<18:11,  2.11it/s, est. speed input: 69275.80 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 744/3051 [05:51<18:11,  2.11it/s, est. speed input: 69275.74 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 745/3051 [05:52<18:10,  2.11it/s, est. speed input: 69275.72 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 746/3051 [05:52<18:10,  2.11it/s, est. speed input: 69275.71 toks/s, output: 2.11 toks/s]
Processed prompts:  24%|██▍       | 747/3051 [05:53<18:09,  2.11it/s, est. speed input: 69275.80 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 748/3051 [05:53<18:09,  2.11it/s, est. speed input: 69275.78 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 749/3051 [05:54<18:08,  2.11it/s, est. speed input: 69275.74 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 750/3051 [05:54<18:08,  2.11it/s, est. speed input: 69275.68 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 751/3051 [05:55<18:08,  2.11it/s, est. speed input: 69275.69 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 752/3051 [05:55<18:07,  2.11it/s, est. speed input: 69275.64 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 753/3051 [05:56<18:10,  2.11it/s, est. speed input: 69274.64 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 754/3051 [05:56<18:08,  2.11it/s, est. speed input: 69274.71 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 755/3051 [05:57<18:07,  2.11it/s, est. speed input: 69274.76 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 756/3051 [05:57<18:08,  2.11it/s, est. speed input: 69274.23 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 757/3051 [05:58<18:07,  2.11it/s, est. speed input: 69274.23 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 758/3051 [05:58<18:06,  2.11it/s, est. speed input: 69274.08 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 759/3051 [05:59<18:07,  2.11it/s, est. speed input: 69273.61 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 760/3051 [05:59<18:05,  2.11it/s, est. speed input: 69273.60 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 761/3051 [05:59<18:04,  2.11it/s, est. speed input: 69273.55 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▍       | 762/3051 [06:00<18:03,  2.11it/s, est. speed input: 69273.57 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 763/3051 [06:00<18:02,  2.11it/s, est. speed input: 69273.63 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 764/3051 [06:01<18:01,  2.11it/s, est. speed input: 69273.77 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 765/3051 [06:01<18:02,  2.11it/s, est. speed input: 69273.29 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 766/3051 [06:02<18:02,  2.11it/s, est. speed input: 69273.17 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 767/3051 [06:02<18:01,  2.11it/s, est. speed input: 69273.14 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 768/3051 [06:03<18:00,  2.11it/s, est. speed input: 69273.18 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 769/3051 [06:03<17:59,  2.11it/s, est. speed input: 69273.22 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 770/3051 [06:04<17:59,  2.11it/s, est. speed input: 69273.20 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 771/3051 [06:04<17:58,  2.11it/s, est. speed input: 69273.17 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 772/3051 [06:05<18:00,  2.11it/s, est. speed input: 69272.42 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 773/3051 [06:05<17:59,  2.11it/s, est. speed input: 69272.33 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 774/3051 [06:06<17:58,  2.11it/s, est. speed input: 69272.34 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 775/3051 [06:06<17:57,  2.11it/s, est. speed input: 69272.43 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 776/3051 [06:07<17:56,  2.11it/s, est. speed input: 69272.51 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 777/3051 [06:07<17:55,  2.11it/s, est. speed input: 69272.57 toks/s, output: 2.11 toks/s]
Processed prompts:  25%|██▌       | 778/3051 [06:08<17:56,  2.11it/s, est. speed input: 69272.13 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 779/3051 [06:08<17:57,  2.11it/s, est. speed input: 69271.79 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 780/3051 [06:08<17:57,  2.11it/s, est. speed input: 69271.48 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 781/3051 [06:09<17:55,  2.11it/s, est. speed input: 69271.52 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 782/3051 [06:09<17:54,  2.11it/s, est. speed input: 69271.59 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 783/3051 [06:10<17:53,  2.11it/s, est. speed input: 69271.65 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 784/3051 [06:10<17:53,  2.11it/s, est. speed input: 69271.35 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 785/3051 [06:11<17:52,  2.11it/s, est. speed input: 69271.34 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 786/3051 [06:11<17:52,  2.11it/s, est. speed input: 69271.35 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 787/3051 [06:12<17:51,  2.11it/s, est. speed input: 69271.38 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 788/3051 [06:12<17:50,  2.11it/s, est. speed input: 69271.37 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 789/3051 [06:13<17:50,  2.11it/s, est. speed input: 69271.30 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 790/3051 [06:13<17:49,  2.11it/s, est. speed input: 69271.36 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 791/3051 [06:14<17:49,  2.11it/s, est. speed input: 69271.40 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 792/3051 [06:14<17:49,  2.11it/s, est. speed input: 69271.27 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 793/3051 [06:15<17:48,  2.11it/s, est. speed input: 69271.29 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 794/3051 [06:15<17:48,  2.11it/s, est. speed input: 69270.97 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 795/3051 [06:16<17:48,  2.11it/s, est. speed input: 69270.95 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 796/3051 [06:16<17:47,  2.11it/s, est. speed input: 69270.92 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 797/3051 [06:17<17:48,  2.11it/s, est. speed input: 69270.45 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 798/3051 [06:17<17:47,  2.11it/s, est. speed input: 69270.33 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 799/3051 [06:17<17:46,  2.11it/s, est. speed input: 69270.50 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▌       | 800/3051 [06:18<17:45,  2.11it/s, est. speed input: 69270.38 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▋       | 801/3051 [06:18<17:45,  2.11it/s, est. speed input: 69270.15 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▋       | 802/3051 [06:19<17:45,  2.11it/s, est. speed input: 69270.03 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▋       | 803/3051 [06:19<17:44,  2.11it/s, est. speed input: 69270.11 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▋       | 804/3051 [06:20<17:43,  2.11it/s, est. speed input: 69270.21 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▋       | 805/3051 [06:20<17:42,  2.11it/s, est. speed input: 69270.27 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▋       | 806/3051 [06:21<17:42,  2.11it/s, est. speed input: 69270.21 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▋       | 807/3051 [06:21<17:41,  2.11it/s, est. speed input: 69270.28 toks/s, output: 2.11 toks/s]
Processed prompts:  26%|██▋       | 808/3051 [06:22<17:41,  2.11it/s, est. speed input: 69270.16 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 809/3051 [06:22<17:40,  2.11it/s, est. speed input: 69270.16 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 810/3051 [06:23<17:41,  2.11it/s, est. speed input: 69269.90 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 811/3051 [06:23<17:40,  2.11it/s, est. speed input: 69269.83 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 812/3051 [06:24<17:39,  2.11it/s, est. speed input: 69269.83 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 813/3051 [06:24<17:39,  2.11it/s, est. speed input: 69269.85 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 814/3051 [06:25<17:38,  2.11it/s, est. speed input: 69269.77 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 815/3051 [06:25<17:38,  2.11it/s, est. speed input: 69269.77 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 816/3051 [06:26<17:56,  2.08it/s, est. speed input: 69264.66 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 817/3051 [06:26<17:49,  2.09it/s, est. speed input: 69264.89 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 818/3051 [06:26<17:45,  2.10it/s, est. speed input: 69264.88 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 819/3051 [06:27<17:42,  2.10it/s, est. speed input: 69264.87 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 820/3051 [06:27<17:39,  2.11it/s, est. speed input: 69264.94 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 821/3051 [06:28<17:37,  2.11it/s, est. speed input: 69265.06 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 822/3051 [06:28<17:36,  2.11it/s, est. speed input: 69264.86 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 823/3051 [06:29<17:35,  2.11it/s, est. speed input: 69264.83 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 824/3051 [06:29<17:35,  2.11it/s, est. speed input: 69264.67 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 825/3051 [06:30<17:34,  2.11it/s, est. speed input: 69264.69 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 826/3051 [06:30<17:33,  2.11it/s, est. speed input: 69264.76 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 827/3051 [06:31<17:33,  2.11it/s, est. speed input: 69264.41 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 828/3051 [06:31<17:33,  2.11it/s, est. speed input: 69264.36 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 829/3051 [06:32<17:31,  2.11it/s, est. speed input: 69264.42 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 830/3051 [06:32<17:32,  2.11it/s, est. speed input: 69264.14 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 831/3051 [06:33<17:31,  2.11it/s, est. speed input: 69264.21 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 832/3051 [06:33<17:30,  2.11it/s, est. speed input: 69264.23 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 833/3051 [06:34<17:29,  2.11it/s, est. speed input: 69264.23 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 834/3051 [06:34<17:28,  2.11it/s, est. speed input: 69264.31 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 835/3051 [06:35<17:27,  2.11it/s, est. speed input: 69264.44 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 836/3051 [06:35<17:27,  2.12it/s, est. speed input: 69264.51 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 837/3051 [06:35<17:27,  2.11it/s, est. speed input: 69264.40 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 838/3051 [06:36<17:26,  2.11it/s, est. speed input: 69264.46 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 839/3051 [06:36<17:26,  2.11it/s, est. speed input: 69264.38 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 840/3051 [06:37<17:26,  2.11it/s, est. speed input: 69264.35 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 841/3051 [06:37<17:26,  2.11it/s, est. speed input: 69264.26 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 842/3051 [06:38<17:25,  2.11it/s, est. speed input: 69264.23 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 843/3051 [06:38<17:26,  2.11it/s, est. speed input: 69263.86 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 844/3051 [06:39<17:25,  2.11it/s, est. speed input: 69263.88 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 845/3051 [06:39<17:24,  2.11it/s, est. speed input: 69263.84 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 846/3051 [06:40<17:23,  2.11it/s, est. speed input: 69263.84 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 847/3051 [06:40<17:23,  2.11it/s, est. speed input: 69263.77 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 848/3051 [06:41<17:22,  2.11it/s, est. speed input: 69263.84 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 849/3051 [06:41<17:21,  2.11it/s, est. speed input: 69263.93 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 850/3051 [06:42<17:20,  2.12it/s, est. speed input: 69264.13 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 851/3051 [06:42<17:20,  2.11it/s, est. speed input: 69264.09 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 852/3051 [06:43<17:19,  2.12it/s, est. speed input: 69264.24 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 853/3051 [06:43<17:19,  2.12it/s, est. speed input: 69264.24 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 854/3051 [06:44<17:18,  2.11it/s, est. speed input: 69264.24 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 855/3051 [06:44<17:18,  2.11it/s, est. speed input: 69264.30 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 856/3051 [06:44<17:17,  2.11it/s, est. speed input: 69264.31 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 857/3051 [06:45<17:17,  2.11it/s, est. speed input: 69264.28 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 858/3051 [06:45<17:17,  2.11it/s, est. speed input: 69264.23 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 859/3051 [06:46<17:16,  2.11it/s, est. speed input: 69264.25 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 860/3051 [06:46<17:16,  2.11it/s, est. speed input: 69264.28 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 861/3051 [06:47<17:16,  2.11it/s, est. speed input: 69264.26 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 862/3051 [06:47<17:15,  2.11it/s, est. speed input: 69264.27 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 863/3051 [06:48<17:14,  2.11it/s, est. speed input: 69264.31 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 864/3051 [06:48<17:14,  2.11it/s, est. speed input: 69264.24 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 865/3051 [06:49<17:14,  2.11it/s, est. speed input: 69264.09 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 866/3051 [06:49<17:14,  2.11it/s, est. speed input: 69263.93 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 867/3051 [06:50<17:14,  2.11it/s, est. speed input: 69263.91 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 868/3051 [06:50<17:14,  2.11it/s, est. speed input: 69263.69 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 869/3051 [06:51<17:13,  2.11it/s, est. speed input: 69263.65 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 870/3051 [06:51<17:13,  2.11it/s, est. speed input: 69263.55 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 871/3051 [06:52<17:15,  2.10it/s, est. speed input: 69262.62 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 872/3051 [06:52<17:13,  2.11it/s, est. speed input: 69262.76 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 873/3051 [06:53<17:12,  2.11it/s, est. speed input: 69262.77 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 874/3051 [06:53<17:11,  2.11it/s, est. speed input: 69262.73 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 875/3051 [06:53<17:11,  2.11it/s, est. speed input: 69262.43 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 876/3051 [06:54<17:10,  2.11it/s, est. speed input: 69262.49 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 877/3051 [06:54<17:10,  2.11it/s, est. speed input: 69262.15 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 878/3051 [06:55<17:09,  2.11it/s, est. speed input: 69262.11 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 879/3051 [06:55<17:08,  2.11it/s, est. speed input: 69262.07 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 880/3051 [06:56<17:07,  2.11it/s, est. speed input: 69262.12 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 881/3051 [06:56<17:06,  2.11it/s, est. speed input: 69262.22 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 882/3051 [06:57<17:06,  2.11it/s, est. speed input: 69262.26 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 883/3051 [06:57<17:04,  2.12it/s, est. speed input: 69262.45 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 884/3051 [06:58<17:04,  2.11it/s, est. speed input: 69262.45 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 885/3051 [06:58<17:04,  2.11it/s, est. speed input: 69262.42 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 886/3051 [06:59<17:04,  2.11it/s, est. speed input: 69262.33 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 887/3051 [06:59<17:03,  2.11it/s, est. speed input: 69262.34 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 888/3051 [07:00<17:03,  2.11it/s, est. speed input: 69262.35 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 889/3051 [07:00<17:02,  2.11it/s, est. speed input: 69262.37 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 890/3051 [07:01<17:02,  2.11it/s, est. speed input: 69262.21 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 891/3051 [07:01<17:02,  2.11it/s, est. speed input: 69262.20 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 892/3051 [07:02<17:02,  2.11it/s, est. speed input: 69262.08 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 893/3051 [07:02<17:01,  2.11it/s, est. speed input: 69262.17 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 894/3051 [07:02<17:00,  2.11it/s, est. speed input: 69262.11 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 895/3051 [07:03<17:00,  2.11it/s, est. speed input: 69262.17 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 896/3051 [07:03<17:31,  2.05it/s, est. speed input: 69254.04 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 897/3051 [07:04<17:21,  2.07it/s, est. speed input: 69254.05 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 898/3051 [07:04<17:13,  2.08it/s, est. speed input: 69254.17 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 899/3051 [07:05<17:08,  2.09it/s, est. speed input: 69254.20 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 900/3051 [07:05<17:05,  2.10it/s, est. speed input: 69254.14 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 901/3051 [07:06<17:02,  2.10it/s, est. speed input: 69254.20 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 902/3051 [07:06<17:01,  2.10it/s, est. speed input: 69253.85 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 903/3051 [07:07<17:00,  2.11it/s, est. speed input: 69253.79 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 904/3051 [07:07<16:57,  2.11it/s, est. speed input: 69253.93 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 905/3051 [07:08<16:56,  2.11it/s, est. speed input: 69253.99 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 906/3051 [07:08<16:56,  2.11it/s, est. speed input: 69253.92 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 907/3051 [07:09<16:55,  2.11it/s, est. speed input: 69253.94 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 908/3051 [07:09<16:54,  2.11it/s, est. speed input: 69254.03 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 909/3051 [07:10<16:53,  2.11it/s, est. speed input: 69254.09 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 910/3051 [07:10<16:52,  2.11it/s, est. speed input: 69254.08 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 911/3051 [07:11<16:52,  2.11it/s, est. speed input: 69254.11 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 912/3051 [07:11<16:51,  2.11it/s, est. speed input: 69254.11 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 913/3051 [07:11<16:51,  2.11it/s, est. speed input: 69254.12 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 914/3051 [07:12<16:50,  2.12it/s, est. speed input: 69254.30 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 915/3051 [07:12<16:50,  2.11it/s, est. speed input: 69254.31 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 916/3051 [07:13<16:49,  2.11it/s, est. speed input: 69254.33 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 917/3051 [07:13<16:49,  2.11it/s, est. speed input: 69254.26 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 918/3051 [07:14<16:49,  2.11it/s, est. speed input: 69254.30 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 919/3051 [07:14<16:48,  2.11it/s, est. speed input: 69254.27 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 920/3051 [07:15<16:48,  2.11it/s, est. speed input: 69254.24 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 921/3051 [07:15<16:48,  2.11it/s, est. speed input: 69254.18 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 922/3051 [07:16<16:46,  2.11it/s, est. speed input: 69254.37 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 923/3051 [07:16<16:46,  2.11it/s, est. speed input: 69254.36 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 924/3051 [07:17<16:45,  2.11it/s, est. speed input: 69254.40 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 925/3051 [07:17<16:46,  2.11it/s, est. speed input: 69254.21 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 926/3051 [07:18<16:45,  2.11it/s, est. speed input: 69254.24 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 927/3051 [07:18<16:45,  2.11it/s, est. speed input: 69254.21 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 928/3051 [07:19<16:44,  2.11it/s, est. speed input: 69254.20 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 929/3051 [07:19<16:43,  2.11it/s, est. speed input: 69254.27 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 930/3051 [07:20<16:43,  2.11it/s, est. speed input: 69254.35 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 931/3051 [07:20<16:42,  2.11it/s, est. speed input: 69254.43 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 932/3051 [07:20<16:42,  2.11it/s, est. speed input: 69254.33 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 933/3051 [07:21<16:42,  2.11it/s, est. speed input: 69254.31 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 934/3051 [07:21<16:41,  2.11it/s, est. speed input: 69254.28 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 935/3051 [07:22<16:40,  2.11it/s, est. speed input: 69254.38 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 936/3051 [07:22<16:41,  2.11it/s, est. speed input: 69254.02 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 937/3051 [07:23<16:41,  2.11it/s, est. speed input: 69253.97 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 938/3051 [07:23<16:43,  2.11it/s, est. speed input: 69253.17 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 939/3051 [07:24<16:41,  2.11it/s, est. speed input: 69253.26 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 940/3051 [07:24<16:40,  2.11it/s, est. speed input: 69253.27 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 941/3051 [07:25<16:40,  2.11it/s, est. speed input: 69252.96 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 942/3051 [07:25<16:39,  2.11it/s, est. speed input: 69252.95 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 943/3051 [07:26<16:38,  2.11it/s, est. speed input: 69252.93 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 944/3051 [07:26<16:37,  2.11it/s, est. speed input: 69252.95 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 945/3051 [07:27<16:37,  2.11it/s, est. speed input: 69252.83 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 946/3051 [07:27<16:36,  2.11it/s, est. speed input: 69252.86 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 947/3051 [07:28<16:36,  2.11it/s, est. speed input: 69252.77 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 948/3051 [07:28<16:35,  2.11it/s, est. speed input: 69252.73 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 949/3051 [07:29<16:35,  2.11it/s, est. speed input: 69252.54 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 950/3051 [07:29<16:34,  2.11it/s, est. speed input: 69252.62 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 951/3051 [07:29<16:33,  2.11it/s, est. speed input: 69252.65 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 952/3051 [07:30<16:33,  2.11it/s, est. speed input: 69252.73 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 953/3051 [07:30<16:32,  2.11it/s, est. speed input: 69252.81 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 954/3051 [07:31<16:32,  2.11it/s, est. speed input: 69252.77 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 955/3051 [07:31<16:31,  2.11it/s, est. speed input: 69252.75 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 956/3051 [07:32<16:30,  2.11it/s, est. speed input: 69252.83 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 957/3051 [07:32<16:30,  2.12it/s, est. speed input: 69252.96 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 958/3051 [07:33<16:29,  2.11it/s, est. speed input: 69252.94 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 959/3051 [07:33<16:29,  2.11it/s, est. speed input: 69252.98 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 960/3051 [07:34<16:30,  2.11it/s, est. speed input: 69252.71 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 961/3051 [07:34<16:29,  2.11it/s, est. speed input: 69252.82 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 962/3051 [07:35<16:28,  2.11it/s, est. speed input: 69252.85 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 963/3051 [07:35<16:27,  2.11it/s, est. speed input: 69252.91 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 964/3051 [07:36<16:27,  2.11it/s, est. speed input: 69252.83 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 965/3051 [07:36<16:26,  2.11it/s, est. speed input: 69252.92 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 966/3051 [07:37<16:26,  2.11it/s, est. speed input: 69252.92 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 967/3051 [07:37<16:25,  2.11it/s, est. speed input: 69252.98 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 968/3051 [07:38<16:25,  2.11it/s, est. speed input: 69252.97 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 969/3051 [07:38<16:24,  2.11it/s, est. speed input: 69252.99 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 970/3051 [07:38<16:25,  2.11it/s, est. speed input: 69252.69 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 971/3051 [07:39<16:24,  2.11it/s, est. speed input: 69252.68 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 972/3051 [07:39<16:24,  2.11it/s, est. speed input: 69252.68 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 973/3051 [07:40<16:23,  2.11it/s, est. speed input: 69252.75 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 974/3051 [07:40<16:22,  2.11it/s, est. speed input: 69252.78 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 975/3051 [07:41<16:21,  2.11it/s, est. speed input: 69252.87 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 976/3051 [07:41<16:22,  2.11it/s, est. speed input: 69252.58 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 977/3051 [07:42<16:21,  2.11it/s, est. speed input: 69252.64 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 978/3051 [07:42<16:21,  2.11it/s, est. speed input: 69252.66 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 979/3051 [07:43<16:25,  2.10it/s, est. speed input: 69251.54 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 980/3051 [07:43<16:22,  2.11it/s, est. speed input: 69251.62 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 981/3051 [07:44<16:21,  2.11it/s, est. speed input: 69251.61 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 982/3051 [07:44<16:20,  2.11it/s, est. speed input: 69251.61 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 983/3051 [07:45<16:19,  2.11it/s, est. speed input: 69251.68 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 984/3051 [07:45<16:18,  2.11it/s, est. speed input: 69251.79 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 985/3051 [07:46<16:17,  2.11it/s, est. speed input: 69251.75 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 986/3051 [07:46<16:17,  2.11it/s, est. speed input: 69251.67 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 987/3051 [07:47<16:17,  2.11it/s, est. speed input: 69251.54 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 988/3051 [07:47<16:16,  2.11it/s, est. speed input: 69251.60 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 989/3051 [07:47<16:15,  2.11it/s, est. speed input: 69251.64 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 990/3051 [07:48<16:15,  2.11it/s, est. speed input: 69251.51 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 991/3051 [07:48<16:16,  2.11it/s, est. speed input: 69251.20 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 992/3051 [07:49<16:15,  2.11it/s, est. speed input: 69251.09 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 993/3051 [07:49<16:16,  2.11it/s, est. speed input: 69250.72 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 994/3051 [07:50<16:15,  2.11it/s, est. speed input: 69250.64 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 995/3051 [07:50<16:13,  2.11it/s, est. speed input: 69250.77 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 996/3051 [07:51<16:13,  2.11it/s, est. speed input: 69250.71 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 997/3051 [07:51<16:12,  2.11it/s, est. speed input: 69250.72 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 998/3051 [07:52<16:11,  2.11it/s, est. speed input: 69250.75 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 999/3051 [07:52<16:11,  2.11it/s, est. speed input: 69250.75 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1000/3051 [07:53<16:10,  2.11it/s, est. speed input: 69250.81 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1001/3051 [07:53<16:09,  2.11it/s, est. speed input: 69250.88 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1002/3051 [07:54<16:09,  2.11it/s, est. speed input: 69250.82 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1003/3051 [07:54<16:10,  2.11it/s, est. speed input: 69250.40 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1004/3051 [07:55<16:09,  2.11it/s, est. speed input: 69250.47 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1005/3051 [07:55<16:08,  2.11it/s, est. speed input: 69250.43 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1006/3051 [07:56<16:11,  2.11it/s, est. speed input: 69249.70 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1007/3051 [07:56<16:09,  2.11it/s, est. speed input: 69249.70 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1008/3051 [07:56<16:08,  2.11it/s, est. speed input: 69249.70 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1009/3051 [07:57<16:07,  2.11it/s, est. speed input: 69249.64 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1010/3051 [07:57<16:06,  2.11it/s, est. speed input: 69249.65 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1011/3051 [07:58<16:06,  2.11it/s, est. speed input: 69249.62 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1012/3051 [07:58<16:05,  2.11it/s, est. speed input: 69249.54 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1013/3051 [07:59<16:05,  2.11it/s, est. speed input: 69249.46 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1014/3051 [07:59<16:04,  2.11it/s, est. speed input: 69249.50 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1015/3051 [08:00<16:03,  2.11it/s, est. speed input: 69249.57 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1016/3051 [08:00<16:02,  2.11it/s, est. speed input: 69249.58 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1017/3051 [08:01<16:02,  2.11it/s, est. speed input: 69249.62 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1018/3051 [08:01<16:01,  2.11it/s, est. speed input: 69249.63 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1019/3051 [08:02<16:01,  2.11it/s, est. speed input: 69249.63 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1020/3051 [08:02<16:01,  2.11it/s, est. speed input: 69249.58 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1021/3051 [08:03<16:00,  2.11it/s, est. speed input: 69249.63 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1022/3051 [08:03<16:00,  2.11it/s, est. speed input: 69249.50 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1023/3051 [08:04<15:59,  2.11it/s, est. speed input: 69249.57 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1024/3051 [08:04<15:59,  2.11it/s, est. speed input: 69249.57 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1025/3051 [08:05<15:59,  2.11it/s, est. speed input: 69249.29 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1026/3051 [08:05<15:59,  2.11it/s, est. speed input: 69249.28 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1027/3051 [08:05<15:57,  2.11it/s, est. speed input: 69249.36 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1028/3051 [08:06<15:57,  2.11it/s, est. speed input: 69249.32 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1029/3051 [08:06<15:56,  2.11it/s, est. speed input: 69249.39 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1030/3051 [08:07<16:22,  2.06it/s, est. speed input: 69243.20 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1031/3051 [08:07<16:13,  2.08it/s, est. speed input: 69243.51 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1032/3051 [08:08<16:07,  2.09it/s, est. speed input: 69243.53 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1033/3051 [08:08<16:04,  2.09it/s, est. speed input: 69243.15 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1034/3051 [08:09<16:01,  2.10it/s, est. speed input: 69243.07 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1035/3051 [08:09<15:58,  2.10it/s, est. speed input: 69243.11 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1036/3051 [08:10<15:57,  2.10it/s, est. speed input: 69242.91 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1037/3051 [08:10<15:56,  2.10it/s, est. speed input: 69242.73 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1038/3051 [08:11<15:55,  2.11it/s, est. speed input: 69242.77 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1039/3051 [08:11<15:53,  2.11it/s, est. speed input: 69242.76 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1040/3051 [08:12<15:52,  2.11it/s, est. speed input: 69242.83 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1041/3051 [08:12<15:51,  2.11it/s, est. speed input: 69242.88 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1042/3051 [08:13<15:50,  2.11it/s, est. speed input: 69242.94 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1043/3051 [08:13<15:49,  2.11it/s, est. speed input: 69243.02 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1044/3051 [08:14<15:49,  2.11it/s, est. speed input: 69243.15 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1045/3051 [08:14<15:48,  2.11it/s, est. speed input: 69243.17 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1046/3051 [08:14<15:48,  2.11it/s, est. speed input: 69243.12 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1047/3051 [08:15<15:50,  2.11it/s, est. speed input: 69242.70 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1048/3051 [08:15<15:48,  2.11it/s, est. speed input: 69242.77 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1049/3051 [08:16<15:48,  2.11it/s, est. speed input: 69242.76 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1050/3051 [08:16<15:46,  2.11it/s, est. speed input: 69242.86 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1051/3051 [08:17<15:46,  2.11it/s, est. speed input: 69242.90 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1052/3051 [08:17<15:45,  2.11it/s, est. speed input: 69242.94 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1053/3051 [08:18<15:45,  2.11it/s, est. speed input: 69242.92 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1054/3051 [08:18<15:44,  2.11it/s, est. speed input: 69242.92 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1055/3051 [08:19<15:44,  2.11it/s, est. speed input: 69242.85 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1056/3051 [08:19<15:44,  2.11it/s, est. speed input: 69242.91 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1057/3051 [08:20<15:43,  2.11it/s, est. speed input: 69242.84 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1058/3051 [08:20<15:42,  2.11it/s, est. speed input: 69242.97 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1059/3051 [08:21<15:42,  2.11it/s, est. speed input: 69242.93 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1060/3051 [08:21<15:41,  2.11it/s, est. speed input: 69242.96 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1061/3051 [08:22<15:41,  2.11it/s, est. speed input: 69243.04 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1062/3051 [08:22<15:40,  2.11it/s, est. speed input: 69243.04 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1063/3051 [08:23<15:41,  2.11it/s, est. speed input: 69242.89 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1064/3051 [08:23<15:40,  2.11it/s, est. speed input: 69242.87 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1065/3051 [08:23<15:41,  2.11it/s, est. speed input: 69242.66 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1066/3051 [08:24<15:40,  2.11it/s, est. speed input: 69242.64 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1067/3051 [08:24<15:39,  2.11it/s, est. speed input: 69242.68 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1068/3051 [08:25<15:38,  2.11it/s, est. speed input: 69242.73 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1069/3051 [08:25<15:37,  2.11it/s, est. speed input: 69242.83 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1070/3051 [08:26<15:37,  2.11it/s, est. speed input: 69242.75 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1071/3051 [08:26<15:36,  2.11it/s, est. speed input: 69242.84 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1072/3051 [08:27<15:36,  2.11it/s, est. speed input: 69242.83 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1073/3051 [08:27<15:36,  2.11it/s, est. speed input: 69242.70 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1074/3051 [08:28<15:35,  2.11it/s, est. speed input: 69242.76 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1075/3051 [08:28<15:35,  2.11it/s, est. speed input: 69242.61 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1076/3051 [08:29<15:35,  2.11it/s, est. speed input: 69242.56 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1077/3051 [08:29<15:34,  2.11it/s, est. speed input: 69242.62 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1078/3051 [08:30<15:35,  2.11it/s, est. speed input: 69242.29 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1079/3051 [08:30<15:35,  2.11it/s, est. speed input: 69242.12 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1080/3051 [08:31<15:33,  2.11it/s, est. speed input: 69242.14 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1081/3051 [08:31<15:33,  2.11it/s, est. speed input: 69242.14 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1082/3051 [08:32<15:33,  2.11it/s, est. speed input: 69241.94 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1083/3051 [08:32<15:32,  2.11it/s, est. speed input: 69241.94 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1084/3051 [08:32<15:30,  2.11it/s, est. speed input: 69242.09 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1085/3051 [08:33<15:30,  2.11it/s, est. speed input: 69242.11 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1086/3051 [08:33<15:29,  2.11it/s, est. speed input: 69242.13 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1087/3051 [08:34<15:29,  2.11it/s, est. speed input: 69242.17 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1088/3051 [08:34<15:28,  2.11it/s, est. speed input: 69242.22 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1089/3051 [08:35<15:27,  2.11it/s, est. speed input: 69242.30 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1090/3051 [08:35<15:27,  2.11it/s, est. speed input: 69242.26 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1091/3051 [08:36<15:27,  2.11it/s, est. speed input: 69242.19 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1092/3051 [08:36<15:28,  2.11it/s, est. speed input: 69241.90 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1093/3051 [08:37<15:27,  2.11it/s, est. speed input: 69241.92 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1094/3051 [08:37<15:26,  2.11it/s, est. speed input: 69242.00 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1095/3051 [08:38<15:25,  2.11it/s, est. speed input: 69241.97 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1096/3051 [08:38<15:25,  2.11it/s, est. speed input: 69241.94 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1097/3051 [08:39<15:25,  2.11it/s, est. speed input: 69241.91 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1098/3051 [08:39<15:24,  2.11it/s, est. speed input: 69241.95 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1099/3051 [08:40<15:24,  2.11it/s, est. speed input: 69241.90 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1100/3051 [08:40<15:24,  2.11it/s, est. speed input: 69241.75 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1101/3051 [08:41<15:23,  2.11it/s, est. speed input: 69241.76 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1102/3051 [08:41<15:22,  2.11it/s, est. speed input: 69241.72 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1103/3051 [08:41<15:22,  2.11it/s, est. speed input: 69241.77 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1104/3051 [08:42<15:21,  2.11it/s, est. speed input: 69241.80 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1105/3051 [08:42<15:21,  2.11it/s, est. speed input: 69241.59 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1106/3051 [08:43<15:21,  2.11it/s, est. speed input: 69241.61 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1107/3051 [08:43<15:20,  2.11it/s, est. speed input: 69241.61 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1108/3051 [08:44<15:21,  2.11it/s, est. speed input: 69241.29 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1109/3051 [08:44<15:20,  2.11it/s, est. speed input: 69241.32 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1110/3051 [08:45<15:19,  2.11it/s, est. speed input: 69241.37 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1111/3051 [08:45<15:19,  2.11it/s, est. speed input: 69241.19 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1112/3051 [08:46<15:17,  2.11it/s, est. speed input: 69241.32 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1113/3051 [08:46<15:17,  2.11it/s, est. speed input: 69241.38 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1114/3051 [08:47<15:16,  2.11it/s, est. speed input: 69241.32 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1115/3051 [08:47<15:16,  2.11it/s, est. speed input: 69241.36 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1116/3051 [08:48<15:16,  2.11it/s, est. speed input: 69241.27 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1117/3051 [08:48<15:16,  2.11it/s, est. speed input: 69241.13 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1118/3051 [08:49<15:15,  2.11it/s, est. speed input: 69241.07 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1119/3051 [08:49<15:14,  2.11it/s, est. speed input: 69241.22 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1120/3051 [08:50<15:13,  2.11it/s, est. speed input: 69241.29 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1121/3051 [08:50<15:12,  2.11it/s, est. speed input: 69241.33 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1122/3051 [08:50<15:12,  2.12it/s, est. speed input: 69241.45 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1123/3051 [08:51<15:11,  2.11it/s, est. speed input: 69241.47 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1124/3051 [08:51<15:11,  2.11it/s, est. speed input: 69241.39 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1125/3051 [08:52<15:10,  2.11it/s, est. speed input: 69241.48 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1126/3051 [08:52<15:11,  2.11it/s, est. speed input: 69241.35 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1127/3051 [08:53<15:11,  2.11it/s, est. speed input: 69241.26 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1128/3051 [08:53<15:10,  2.11it/s, est. speed input: 69241.22 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1129/3051 [08:54<15:09,  2.11it/s, est. speed input: 69241.28 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1130/3051 [08:54<15:09,  2.11it/s, est. speed input: 69241.26 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1131/3051 [08:55<15:08,  2.11it/s, est. speed input: 69241.31 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1132/3051 [08:55<15:48,  2.02it/s, est. speed input: 69232.17 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1133/3051 [08:56<15:34,  2.05it/s, est. speed input: 69232.51 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1134/3051 [08:56<15:25,  2.07it/s, est. speed input: 69232.60 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1135/3051 [08:57<15:19,  2.08it/s, est. speed input: 69232.64 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1136/3051 [08:57<15:15,  2.09it/s, est. speed input: 69232.48 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1137/3051 [08:58<15:12,  2.10it/s, est. speed input: 69232.49 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1138/3051 [08:58<15:11,  2.10it/s, est. speed input: 69232.11 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1139/3051 [08:59<15:10,  2.10it/s, est. speed input: 69231.96 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1140/3051 [08:59<15:07,  2.11it/s, est. speed input: 69232.03 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1141/3051 [09:00<15:06,  2.11it/s, est. speed input: 69232.06 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1142/3051 [09:00<15:05,  2.11it/s, est. speed input: 69232.04 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1143/3051 [09:00<15:03,  2.11it/s, est. speed input: 69232.09 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1144/3051 [09:01<15:02,  2.11it/s, est. speed input: 69232.17 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1145/3051 [09:01<15:02,  2.11it/s, est. speed input: 69232.16 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1146/3051 [09:02<15:01,  2.11it/s, est. speed input: 69232.20 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1147/3051 [09:02<15:20,  2.07it/s, est. speed input: 69227.83 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1148/3051 [09:03<15:13,  2.08it/s, est. speed input: 69227.96 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1149/3051 [09:03<15:09,  2.09it/s, est. speed input: 69228.04 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1150/3051 [09:04<15:05,  2.10it/s, est. speed input: 69228.11 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1151/3051 [09:04<15:04,  2.10it/s, est. speed input: 69227.80 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1152/3051 [09:05<15:02,  2.10it/s, est. speed input: 69227.82 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1153/3051 [09:05<15:00,  2.11it/s, est. speed input: 69227.91 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1154/3051 [09:06<15:01,  2.10it/s, est. speed input: 69227.38 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1155/3051 [09:06<15:00,  2.11it/s, est. speed input: 69227.36 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1156/3051 [09:07<14:58,  2.11it/s, est. speed input: 69227.40 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1157/3051 [09:07<14:58,  2.11it/s, est. speed input: 69227.19 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1158/3051 [09:08<14:57,  2.11it/s, est. speed input: 69227.12 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1159/3051 [09:08<14:57,  2.11it/s, est. speed input: 69227.09 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1160/3051 [09:09<14:58,  2.11it/s, est. speed input: 69226.66 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1161/3051 [09:09<14:57,  2.11it/s, est. speed input: 69226.49 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1162/3051 [09:10<14:55,  2.11it/s, est. speed input: 69226.53 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1163/3051 [09:10<14:54,  2.11it/s, est. speed input: 69226.56 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1164/3051 [09:10<14:55,  2.11it/s, est. speed input: 69226.16 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1165/3051 [09:11<14:54,  2.11it/s, est. speed input: 69226.08 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1166/3051 [09:11<14:53,  2.11it/s, est. speed input: 69226.13 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1167/3051 [09:12<14:53,  2.11it/s, est. speed input: 69225.87 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1168/3051 [09:12<14:52,  2.11it/s, est. speed input: 69225.90 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1169/3051 [09:13<14:51,  2.11it/s, est. speed input: 69225.87 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1170/3051 [09:13<14:52,  2.11it/s, est. speed input: 69225.61 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1171/3051 [09:14<14:50,  2.11it/s, est. speed input: 69225.67 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1172/3051 [09:14<14:53,  2.10it/s, est. speed input: 69225.03 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1173/3051 [09:15<14:52,  2.10it/s, est. speed input: 69224.84 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1174/3051 [09:15<14:50,  2.11it/s, est. speed input: 69224.89 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1175/3051 [09:16<14:49,  2.11it/s, est. speed input: 69224.96 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1176/3051 [09:16<14:48,  2.11it/s, est. speed input: 69225.01 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1177/3051 [09:17<14:47,  2.11it/s, est. speed input: 69225.04 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1178/3051 [09:17<14:47,  2.11it/s, est. speed input: 69224.92 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1179/3051 [09:18<14:47,  2.11it/s, est. speed input: 69224.72 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1180/3051 [09:18<14:46,  2.11it/s, est. speed input: 69224.68 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1181/3051 [09:19<14:46,  2.11it/s, est. speed input: 69224.58 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1182/3051 [09:19<14:45,  2.11it/s, est. speed input: 69224.57 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1183/3051 [09:19<14:44,  2.11it/s, est. speed input: 69224.70 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1184/3051 [09:20<14:43,  2.11it/s, est. speed input: 69224.78 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1185/3051 [09:20<14:42,  2.11it/s, est. speed input: 69224.80 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1186/3051 [09:21<14:42,  2.11it/s, est. speed input: 69224.84 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1187/3051 [09:21<14:42,  2.11it/s, est. speed input: 69224.72 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1188/3051 [09:22<14:41,  2.11it/s, est. speed input: 69224.75 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1189/3051 [09:22<14:41,  2.11it/s, est. speed input: 69224.68 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1190/3051 [09:23<14:42,  2.11it/s, est. speed input: 69224.35 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1191/3051 [09:23<14:41,  2.11it/s, est. speed input: 69224.36 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1192/3051 [09:24<14:41,  2.11it/s, est. speed input: 69224.14 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1193/3051 [09:24<14:41,  2.11it/s, est. speed input: 69224.03 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1194/3051 [09:25<14:39,  2.11it/s, est. speed input: 69224.12 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1195/3051 [09:25<14:39,  2.11it/s, est. speed input: 69223.99 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1196/3051 [09:26<14:39,  2.11it/s, est. speed input: 69223.78 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1197/3051 [09:26<14:38,  2.11it/s, est. speed input: 69223.80 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1198/3051 [09:27<14:39,  2.11it/s, est. speed input: 69223.51 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1199/3051 [09:27<14:39,  2.11it/s, est. speed input: 69223.29 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1200/3051 [09:28<14:37,  2.11it/s, est. speed input: 69223.34 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1201/3051 [09:28<14:37,  2.11it/s, est. speed input: 69223.29 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1202/3051 [09:28<14:36,  2.11it/s, est. speed input: 69223.14 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1203/3051 [09:29<14:36,  2.11it/s, est. speed input: 69223.08 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1204/3051 [09:29<14:35,  2.11it/s, est. speed input: 69223.10 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1205/3051 [09:30<14:35,  2.11it/s, est. speed input: 69222.90 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1206/3051 [09:30<14:34,  2.11it/s, est. speed input: 69222.99 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1207/3051 [09:31<14:59,  2.05it/s, est. speed input: 69217.31 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1208/3051 [09:31<14:50,  2.07it/s, est. speed input: 69217.38 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1209/3051 [09:32<14:44,  2.08it/s, est. speed input: 69217.43 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1210/3051 [09:32<14:40,  2.09it/s, est. speed input: 69217.46 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1211/3051 [09:33<14:37,  2.10it/s, est. speed input: 69217.40 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1212/3051 [09:33<14:34,  2.10it/s, est. speed input: 69217.49 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1213/3051 [09:34<14:32,  2.11it/s, est. speed input: 69217.50 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1214/3051 [09:34<14:32,  2.11it/s, est. speed input: 69217.31 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1215/3051 [09:35<14:30,  2.11it/s, est. speed input: 69217.37 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1216/3051 [09:35<14:29,  2.11it/s, est. speed input: 69217.46 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1217/3051 [09:36<14:28,  2.11it/s, est. speed input: 69217.53 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1218/3051 [09:36<14:27,  2.11it/s, est. speed input: 69217.59 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1219/3051 [09:37<14:26,  2.11it/s, est. speed input: 69217.63 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1220/3051 [09:37<14:26,  2.11it/s, est. speed input: 69217.59 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1221/3051 [09:38<14:25,  2.11it/s, est. speed input: 69217.67 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1222/3051 [09:38<14:25,  2.11it/s, est. speed input: 69217.70 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1223/3051 [09:38<14:25,  2.11it/s, est. speed input: 69217.54 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1224/3051 [09:39<14:25,  2.11it/s, est. speed input: 69217.50 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1225/3051 [09:39<14:24,  2.11it/s, est. speed input: 69217.53 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1226/3051 [09:40<14:24,  2.11it/s, est. speed input: 69217.54 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1227/3051 [09:40<14:24,  2.11it/s, est. speed input: 69217.40 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1228/3051 [09:41<14:23,  2.11it/s, est. speed input: 69217.46 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1229/3051 [09:41<14:24,  2.11it/s, est. speed input: 69217.02 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1230/3051 [09:42<14:24,  2.11it/s, est. speed input: 69216.86 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1231/3051 [09:42<14:22,  2.11it/s, est. speed input: 69216.93 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1232/3051 [09:43<14:21,  2.11it/s, est. speed input: 69216.98 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1233/3051 [09:43<14:20,  2.11it/s, est. speed input: 69217.01 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1234/3051 [09:44<14:20,  2.11it/s, est. speed input: 69217.05 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1235/3051 [09:44<14:19,  2.11it/s, est. speed input: 69217.04 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1236/3051 [09:45<14:20,  2.11it/s, est. speed input: 69216.68 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1237/3051 [09:45<14:19,  2.11it/s, est. speed input: 69216.75 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1238/3051 [09:46<14:18,  2.11it/s, est. speed input: 69216.82 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1239/3051 [09:46<14:18,  2.11it/s, est. speed input: 69216.82 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1240/3051 [09:47<14:17,  2.11it/s, est. speed input: 69216.71 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1241/3051 [09:47<14:17,  2.11it/s, est. speed input: 69216.75 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1242/3051 [09:47<14:16,  2.11it/s, est. speed input: 69216.82 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1243/3051 [09:48<14:15,  2.11it/s, est. speed input: 69216.81 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1244/3051 [09:48<14:16,  2.11it/s, est. speed input: 69216.59 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1245/3051 [09:49<14:16,  2.11it/s, est. speed input: 69216.51 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1246/3051 [09:49<14:15,  2.11it/s, est. speed input: 69216.55 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1247/3051 [09:50<14:14,  2.11it/s, est. speed input: 69216.56 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1248/3051 [09:50<14:13,  2.11it/s, est. speed input: 69216.64 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1249/3051 [09:51<14:12,  2.11it/s, est. speed input: 69216.67 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1250/3051 [09:51<14:12,  2.11it/s, est. speed input: 69216.70 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1251/3051 [09:52<14:12,  2.11it/s, est. speed input: 69216.66 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1252/3051 [09:52<14:11,  2.11it/s, est. speed input: 69216.61 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1253/3051 [09:53<14:11,  2.11it/s, est. speed input: 69216.61 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1254/3051 [09:53<14:10,  2.11it/s, est. speed input: 69216.68 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1255/3051 [09:54<14:10,  2.11it/s, est. speed input: 69216.63 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1256/3051 [09:54<14:09,  2.11it/s, est. speed input: 69216.66 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1257/3051 [09:55<14:08,  2.11it/s, est. speed input: 69216.72 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1258/3051 [09:55<14:11,  2.11it/s, est. speed input: 69216.10 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1259/3051 [09:56<14:09,  2.11it/s, est. speed input: 69216.15 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1260/3051 [09:56<14:08,  2.11it/s, est. speed input: 69216.25 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1261/3051 [09:56<14:08,  2.11it/s, est. speed input: 69216.21 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1262/3051 [09:57<14:08,  2.11it/s, est. speed input: 69216.07 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1263/3051 [09:57<14:07,  2.11it/s, est. speed input: 69216.01 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1264/3051 [09:58<14:06,  2.11it/s, est. speed input: 69216.08 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1265/3051 [09:58<14:05,  2.11it/s, est. speed input: 69216.07 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1266/3051 [09:59<14:04,  2.11it/s, est. speed input: 69216.13 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1267/3051 [09:59<14:04,  2.11it/s, est. speed input: 69216.16 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1268/3051 [10:00<14:04,  2.11it/s, est. speed input: 69216.02 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1269/3051 [10:00<14:04,  2.11it/s, est. speed input: 69215.98 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1270/3051 [10:01<14:02,  2.11it/s, est. speed input: 69216.10 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1271/3051 [10:01<14:03,  2.11it/s, est. speed input: 69215.89 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1272/3051 [10:02<14:02,  2.11it/s, est. speed input: 69215.93 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1273/3051 [10:02<14:01,  2.11it/s, est. speed input: 69216.02 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1274/3051 [10:03<14:00,  2.11it/s, est. speed input: 69216.09 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1275/3051 [10:03<14:00,  2.11it/s, est. speed input: 69216.13 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1276/3051 [10:04<13:59,  2.11it/s, est. speed input: 69216.26 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1277/3051 [10:04<13:59,  2.11it/s, est. speed input: 69216.22 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1278/3051 [10:05<13:58,  2.11it/s, est. speed input: 69216.32 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1279/3051 [10:05<13:58,  2.11it/s, est. speed input: 69216.39 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1280/3051 [10:05<13:57,  2.11it/s, est. speed input: 69216.43 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1281/3051 [10:06<13:57,  2.11it/s, est. speed input: 69216.51 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1282/3051 [10:06<13:56,  2.11it/s, est. speed input: 69216.56 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1283/3051 [10:07<13:56,  2.11it/s, est. speed input: 69216.63 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1284/3051 [10:07<13:55,  2.11it/s, est. speed input: 69216.68 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1285/3051 [10:08<13:55,  2.11it/s, est. speed input: 69216.67 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1286/3051 [10:08<13:55,  2.11it/s, est. speed input: 69216.64 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1287/3051 [10:09<13:57,  2.11it/s, est. speed input: 69216.08 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1288/3051 [10:09<13:56,  2.11it/s, est. speed input: 69216.06 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1289/3051 [10:10<13:55,  2.11it/s, est. speed input: 69216.13 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1290/3051 [10:10<13:54,  2.11it/s, est. speed input: 69216.03 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1291/3051 [10:11<13:54,  2.11it/s, est. speed input: 69215.97 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1292/3051 [10:11<13:53,  2.11it/s, est. speed input: 69216.04 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1293/3051 [10:12<13:53,  2.11it/s, est. speed input: 69215.75 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1294/3051 [10:12<13:52,  2.11it/s, est. speed input: 69215.79 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1295/3051 [10:13<13:51,  2.11it/s, est. speed input: 69215.86 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1296/3051 [10:13<13:50,  2.11it/s, est. speed input: 69215.96 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1297/3051 [10:14<13:49,  2.11it/s, est. speed input: 69216.03 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1298/3051 [10:14<13:49,  2.11it/s, est. speed input: 69216.04 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1299/3051 [10:14<13:49,  2.11it/s, est. speed input: 69215.97 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1300/3051 [10:15<13:49,  2.11it/s, est. speed input: 69215.94 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1301/3051 [10:15<13:49,  2.11it/s, est. speed input: 69215.69 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1302/3051 [10:16<13:48,  2.11it/s, est. speed input: 69215.73 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1303/3051 [10:16<13:47,  2.11it/s, est. speed input: 69215.77 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1304/3051 [10:17<13:47,  2.11it/s, est. speed input: 69215.82 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1305/3051 [10:17<13:46,  2.11it/s, est. speed input: 69215.91 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1306/3051 [10:18<13:47,  2.11it/s, est. speed input: 69215.46 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1307/3051 [10:18<13:47,  2.11it/s, est. speed input: 69215.36 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1308/3051 [10:19<13:46,  2.11it/s, est. speed input: 69215.34 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1309/3051 [10:19<13:46,  2.11it/s, est. speed input: 69215.23 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1310/3051 [10:20<13:44,  2.11it/s, est. speed input: 69215.29 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1311/3051 [10:20<13:43,  2.11it/s, est. speed input: 69215.36 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1312/3051 [10:21<13:46,  2.10it/s, est. speed input: 69214.62 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1313/3051 [10:21<13:44,  2.11it/s, est. speed input: 69214.70 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1314/3051 [10:22<13:43,  2.11it/s, est. speed input: 69214.82 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1315/3051 [10:22<13:42,  2.11it/s, est. speed input: 69214.87 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1316/3051 [10:23<13:41,  2.11it/s, est. speed input: 69214.80 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1317/3051 [10:23<13:41,  2.11it/s, est. speed input: 69214.80 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1318/3051 [10:23<13:40,  2.11it/s, est. speed input: 69214.91 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1319/3051 [10:24<13:39,  2.11it/s, est. speed input: 69214.95 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1320/3051 [10:24<13:38,  2.11it/s, est. speed input: 69215.08 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1321/3051 [10:25<13:38,  2.11it/s, est. speed input: 69215.12 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1322/3051 [10:25<13:37,  2.11it/s, est. speed input: 69215.14 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1323/3051 [10:26<13:38,  2.11it/s, est. speed input: 69214.97 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1324/3051 [10:26<13:37,  2.11it/s, est. speed input: 69215.10 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1325/3051 [10:27<13:36,  2.11it/s, est. speed input: 69215.12 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1326/3051 [10:27<13:35,  2.11it/s, est. speed input: 69215.22 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1327/3051 [10:28<13:35,  2.11it/s, est. speed input: 69215.25 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1328/3051 [10:28<13:34,  2.11it/s, est. speed input: 69215.31 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1329/3051 [10:29<13:34,  2.11it/s, est. speed input: 69215.37 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1330/3051 [10:29<13:33,  2.12it/s, est. speed input: 69215.50 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1331/3051 [10:30<13:33,  2.12it/s, est. speed input: 69215.58 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1332/3051 [10:30<13:32,  2.11it/s, est. speed input: 69215.61 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1333/3051 [10:31<13:32,  2.11it/s, est. speed input: 69215.65 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1334/3051 [10:31<13:31,  2.11it/s, est. speed input: 69215.74 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1335/3051 [10:32<13:32,  2.11it/s, est. speed input: 69215.59 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1336/3051 [10:32<13:31,  2.11it/s, est. speed input: 69215.59 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1337/3051 [10:32<13:30,  2.11it/s, est. speed input: 69215.74 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1338/3051 [10:33<13:31,  2.11it/s, est. speed input: 69215.51 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1339/3051 [10:33<13:31,  2.11it/s, est. speed input: 69215.44 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1340/3051 [10:34<13:30,  2.11it/s, est. speed input: 69215.53 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1341/3051 [10:34<13:29,  2.11it/s, est. speed input: 69215.56 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1342/3051 [10:35<13:31,  2.11it/s, est. speed input: 69215.04 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1343/3051 [10:35<13:30,  2.11it/s, est. speed input: 69215.03 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1344/3051 [10:36<13:28,  2.11it/s, est. speed input: 69215.16 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1345/3051 [10:36<13:27,  2.11it/s, est. speed input: 69215.22 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1346/3051 [10:37<13:26,  2.11it/s, est. speed input: 69215.36 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1347/3051 [10:37<13:26,  2.11it/s, est. speed input: 69215.29 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1348/3051 [10:38<13:26,  2.11it/s, est. speed input: 69215.27 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1349/3051 [10:38<13:26,  2.11it/s, est. speed input: 69215.17 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1350/3051 [10:39<13:25,  2.11it/s, est. speed input: 69215.25 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1351/3051 [10:39<13:25,  2.11it/s, est. speed input: 69215.07 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1352/3051 [10:40<13:24,  2.11it/s, est. speed input: 69215.07 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1353/3051 [10:40<13:24,  2.11it/s, est. speed input: 69215.06 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1354/3051 [10:41<13:23,  2.11it/s, est. speed input: 69215.06 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1355/3051 [10:41<13:25,  2.11it/s, est. speed input: 69214.56 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1356/3051 [10:41<13:23,  2.11it/s, est. speed input: 69214.62 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1357/3051 [10:42<13:22,  2.11it/s, est. speed input: 69214.67 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1358/3051 [10:42<13:21,  2.11it/s, est. speed input: 69214.72 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1359/3051 [10:43<13:20,  2.11it/s, est. speed input: 69214.81 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1360/3051 [10:43<13:20,  2.11it/s, est. speed input: 69214.89 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1361/3051 [10:44<13:19,  2.11it/s, est. speed input: 69214.94 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1362/3051 [10:44<13:18,  2.11it/s, est. speed input: 69215.02 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1363/3051 [10:45<13:18,  2.12it/s, est. speed input: 69215.13 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1364/3051 [10:45<13:17,  2.12it/s, est. speed input: 69215.19 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1365/3051 [10:46<13:17,  2.11it/s, est. speed input: 69215.13 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1366/3051 [10:46<13:17,  2.11it/s, est. speed input: 69215.17 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1367/3051 [10:47<13:16,  2.11it/s, est. speed input: 69215.19 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1368/3051 [10:47<13:15,  2.11it/s, est. speed input: 69215.32 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1369/3051 [10:48<13:15,  2.11it/s, est. speed input: 69215.36 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1370/3051 [10:48<13:15,  2.11it/s, est. speed input: 69215.34 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1371/3051 [10:49<13:15,  2.11it/s, est. speed input: 69215.22 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1372/3051 [10:49<13:14,  2.11it/s, est. speed input: 69215.31 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1373/3051 [10:50<13:14,  2.11it/s, est. speed input: 69215.32 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1374/3051 [10:50<13:14,  2.11it/s, est. speed input: 69215.10 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1375/3051 [10:50<13:13,  2.11it/s, est. speed input: 69215.14 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1376/3051 [10:51<13:13,  2.11it/s, est. speed input: 69215.16 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1377/3051 [10:51<13:12,  2.11it/s, est. speed input: 69215.21 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1378/3051 [10:52<13:12,  2.11it/s, est. speed input: 69215.07 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1379/3051 [10:52<13:12,  2.11it/s, est. speed input: 69214.98 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1380/3051 [10:53<13:11,  2.11it/s, est. speed input: 69215.00 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1381/3051 [10:53<13:10,  2.11it/s, est. speed input: 69215.02 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1382/3051 [10:54<13:10,  2.11it/s, est. speed input: 69215.05 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1383/3051 [10:54<13:10,  2.11it/s, est. speed input: 69214.83 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1384/3051 [10:55<13:09,  2.11it/s, est. speed input: 69214.86 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1385/3051 [10:55<13:09,  2.11it/s, est. speed input: 69214.89 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1386/3051 [10:56<13:08,  2.11it/s, est. speed input: 69214.93 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1387/3051 [10:56<13:08,  2.11it/s, est. speed input: 69214.85 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1388/3051 [10:57<13:07,  2.11it/s, est. speed input: 69214.82 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1389/3051 [10:57<13:06,  2.11it/s, est. speed input: 69214.87 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1390/3051 [10:58<13:06,  2.11it/s, est. speed input: 69214.86 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1391/3051 [10:58<13:06,  2.11it/s, est. speed input: 69214.78 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1392/3051 [10:59<13:05,  2.11it/s, est. speed input: 69214.76 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1393/3051 [10:59<13:04,  2.11it/s, est. speed input: 69214.82 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1394/3051 [10:59<13:04,  2.11it/s, est. speed input: 69214.89 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1395/3051 [11:00<13:03,  2.11it/s, est. speed input: 69214.88 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1396/3051 [11:00<13:04,  2.11it/s, est. speed input: 69214.73 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1397/3051 [11:01<13:03,  2.11it/s, est. speed input: 69214.77 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1398/3051 [11:01<13:02,  2.11it/s, est. speed input: 69214.81 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1399/3051 [11:02<13:01,  2.11it/s, est. speed input: 69214.92 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1400/3051 [11:02<13:01,  2.11it/s, est. speed input: 69214.96 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1401/3051 [11:03<13:00,  2.11it/s, est. speed input: 69214.98 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1402/3051 [11:03<12:59,  2.11it/s, est. speed input: 69215.08 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1403/3051 [11:04<12:59,  2.11it/s, est. speed input: 69215.17 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1404/3051 [11:04<12:58,  2.12it/s, est. speed input: 69215.30 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1405/3051 [11:05<12:58,  2.12it/s, est. speed input: 69215.34 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1406/3051 [11:05<12:59,  2.11it/s, est. speed input: 69214.97 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1407/3051 [11:06<12:58,  2.11it/s, est. speed input: 69214.97 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1408/3051 [11:06<12:57,  2.11it/s, est. speed input: 69215.06 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1409/3051 [11:07<12:57,  2.11it/s, est. speed input: 69215.08 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1410/3051 [11:07<12:56,  2.11it/s, est. speed input: 69215.12 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1411/3051 [11:07<12:56,  2.11it/s, est. speed input: 69215.09 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1412/3051 [11:08<12:56,  2.11it/s, est. speed input: 69214.92 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1413/3051 [11:08<12:57,  2.11it/s, est. speed input: 69214.62 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1414/3051 [11:09<12:56,  2.11it/s, est. speed input: 69214.56 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1415/3051 [11:09<12:58,  2.10it/s, est. speed input: 69213.98 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1416/3051 [11:10<12:57,  2.10it/s, est. speed input: 69213.91 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1417/3051 [11:10<12:55,  2.11it/s, est. speed input: 69213.96 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1418/3051 [11:11<12:54,  2.11it/s, est. speed input: 69213.88 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1419/3051 [11:11<12:53,  2.11it/s, est. speed input: 69213.90 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1420/3051 [11:12<12:53,  2.11it/s, est. speed input: 69213.89 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1421/3051 [11:12<12:54,  2.11it/s, est. speed input: 69213.52 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1422/3051 [11:13<12:53,  2.11it/s, est. speed input: 69213.45 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1423/3051 [11:13<12:51,  2.11it/s, est. speed input: 69213.53 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1424/3051 [11:14<12:57,  2.09it/s, est. speed input: 69212.21 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1425/3051 [11:14<12:54,  2.10it/s, est. speed input: 69212.31 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1426/3051 [11:15<12:51,  2.11it/s, est. speed input: 69212.42 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1427/3051 [11:15<12:50,  2.11it/s, est. speed input: 69212.50 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1428/3051 [11:16<12:49,  2.11it/s, est. speed input: 69212.57 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1429/3051 [11:16<12:48,  2.11it/s, est. speed input: 69212.61 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1430/3051 [11:17<12:47,  2.11it/s, est. speed input: 69212.69 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1431/3051 [11:17<12:46,  2.11it/s, est. speed input: 69212.67 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1432/3051 [11:17<12:46,  2.11it/s, est. speed input: 69212.71 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1433/3051 [11:18<12:45,  2.11it/s, est. speed input: 69212.72 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1434/3051 [11:18<12:46,  2.11it/s, est. speed input: 69212.59 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1435/3051 [11:19<12:45,  2.11it/s, est. speed input: 69212.53 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1436/3051 [11:19<12:44,  2.11it/s, est. speed input: 69212.60 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1437/3051 [11:20<12:46,  2.10it/s, est. speed input: 69212.06 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1438/3051 [11:20<12:46,  2.11it/s, est. speed input: 69211.93 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1439/3051 [11:21<12:44,  2.11it/s, est. speed input: 69211.99 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1440/3051 [11:21<12:43,  2.11it/s, est. speed input: 69211.99 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1441/3051 [11:22<12:42,  2.11it/s, est. speed input: 69211.99 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1442/3051 [11:22<12:41,  2.11it/s, est. speed input: 69212.05 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1443/3051 [11:23<12:41,  2.11it/s, est. speed input: 69212.09 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1444/3051 [11:23<12:40,  2.11it/s, est. speed input: 69212.15 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1445/3051 [11:24<12:40,  2.11it/s, est. speed input: 69212.16 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1446/3051 [11:24<12:39,  2.11it/s, est. speed input: 69212.20 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1447/3051 [11:25<12:39,  2.11it/s, est. speed input: 69212.18 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1448/3051 [11:25<12:38,  2.11it/s, est. speed input: 69212.23 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1449/3051 [11:26<12:37,  2.11it/s, est. speed input: 69212.28 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1450/3051 [11:26<12:40,  2.11it/s, est. speed input: 69211.76 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1451/3051 [11:26<12:38,  2.11it/s, est. speed input: 69211.86 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1452/3051 [11:27<12:37,  2.11it/s, est. speed input: 69211.98 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1453/3051 [11:27<12:37,  2.11it/s, est. speed input: 69211.89 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1454/3051 [11:28<12:36,  2.11it/s, est. speed input: 69211.84 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1455/3051 [11:28<12:36,  2.11it/s, est. speed input: 69211.81 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1456/3051 [11:29<12:35,  2.11it/s, est. speed input: 69211.84 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1457/3051 [11:29<12:34,  2.11it/s, est. speed input: 69211.85 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1458/3051 [11:30<12:33,  2.11it/s, est. speed input: 69211.93 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1459/3051 [11:30<12:47,  2.07it/s, est. speed input: 69209.00 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1460/3051 [11:31<12:42,  2.09it/s, est. speed input: 69209.17 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1461/3051 [11:31<12:38,  2.10it/s, est. speed input: 69209.25 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1462/3051 [11:32<12:36,  2.10it/s, est. speed input: 69209.28 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1463/3051 [11:32<12:35,  2.10it/s, est. speed input: 69209.18 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1464/3051 [11:33<12:33,  2.11it/s, est. speed input: 69209.30 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1465/3051 [11:33<12:32,  2.11it/s, est. speed input: 69209.24 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1466/3051 [11:34<12:31,  2.11it/s, est. speed input: 69209.32 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1467/3051 [11:34<12:30,  2.11it/s, est. speed input: 69209.27 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1468/3051 [11:35<12:29,  2.11it/s, est. speed input: 69209.33 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1469/3051 [11:35<12:29,  2.11it/s, est. speed input: 69209.28 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1470/3051 [11:35<12:28,  2.11it/s, est. speed input: 69209.36 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1471/3051 [11:36<12:27,  2.11it/s, est. speed input: 69209.39 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1472/3051 [11:36<12:27,  2.11it/s, est. speed input: 69209.42 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1473/3051 [11:37<12:26,  2.11it/s, est. speed input: 69209.46 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1474/3051 [11:37<12:26,  2.11it/s, est. speed input: 69209.46 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1475/3051 [11:38<12:26,  2.11it/s, est. speed input: 69209.41 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1476/3051 [11:38<12:25,  2.11it/s, est. speed input: 69209.38 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1477/3051 [11:39<12:25,  2.11it/s, est. speed input: 69209.33 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1478/3051 [11:39<12:25,  2.11it/s, est. speed input: 69209.33 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1479/3051 [11:40<12:27,  2.10it/s, est. speed input: 69208.78 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1480/3051 [11:40<12:26,  2.11it/s, est. speed input: 69208.73 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1481/3051 [11:41<12:24,  2.11it/s, est. speed input: 69208.79 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1482/3051 [11:41<12:23,  2.11it/s, est. speed input: 69208.81 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1483/3051 [11:42<12:22,  2.11it/s, est. speed input: 69208.81 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1484/3051 [11:42<12:22,  2.11it/s, est. speed input: 69208.74 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1485/3051 [11:43<12:22,  2.11it/s, est. speed input: 69208.60 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1486/3051 [11:43<12:21,  2.11it/s, est. speed input: 69208.66 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1487/3051 [11:44<12:20,  2.11it/s, est. speed input: 69208.69 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1488/3051 [11:44<12:20,  2.11it/s, est. speed input: 69208.72 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1489/3051 [11:44<12:19,  2.11it/s, est. speed input: 69208.80 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1490/3051 [11:45<12:18,  2.11it/s, est. speed input: 69208.88 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1491/3051 [11:45<12:18,  2.11it/s, est. speed input: 69208.78 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1492/3051 [11:46<12:18,  2.11it/s, est. speed input: 69208.72 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1493/3051 [11:46<12:17,  2.11it/s, est. speed input: 69208.72 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1494/3051 [11:47<12:17,  2.11it/s, est. speed input: 69208.75 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1495/3051 [11:47<12:18,  2.11it/s, est. speed input: 69208.34 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1496/3051 [11:48<12:17,  2.11it/s, est. speed input: 69208.34 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1497/3051 [11:48<12:16,  2.11it/s, est. speed input: 69208.35 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1498/3051 [11:49<12:16,  2.11it/s, est. speed input: 69208.30 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1499/3051 [11:49<12:15,  2.11it/s, est. speed input: 69208.36 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1500/3051 [11:50<12:14,  2.11it/s, est. speed input: 69208.33 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1501/3051 [11:50<12:14,  2.11it/s, est. speed input: 69208.20 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1502/3051 [11:51<12:14,  2.11it/s, est. speed input: 69208.10 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1503/3051 [11:51<12:13,  2.11it/s, est. speed input: 69208.16 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1504/3051 [11:52<12:12,  2.11it/s, est. speed input: 69208.27 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1505/3051 [11:52<12:11,  2.11it/s, est. speed input: 69208.29 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1506/3051 [11:53<12:11,  2.11it/s, est. speed input: 69208.35 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1507/3051 [11:53<12:10,  2.11it/s, est. speed input: 69208.34 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1508/3051 [11:53<12:11,  2.11it/s, est. speed input: 69208.21 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1509/3051 [11:54<12:10,  2.11it/s, est. speed input: 69208.24 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1510/3051 [11:54<12:10,  2.11it/s, est. speed input: 69208.08 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1511/3051 [11:55<12:09,  2.11it/s, est. speed input: 69208.06 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1512/3051 [11:55<12:09,  2.11it/s, est. speed input: 69208.06 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1513/3051 [11:56<12:07,  2.11it/s, est. speed input: 69208.18 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1514/3051 [11:56<12:07,  2.11it/s, est. speed input: 69208.22 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1515/3051 [11:57<12:06,  2.11it/s, est. speed input: 69208.24 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1516/3051 [11:57<12:06,  2.11it/s, est. speed input: 69208.32 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1517/3051 [11:58<12:06,  2.11it/s, est. speed input: 69208.28 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1518/3051 [11:58<12:05,  2.11it/s, est. speed input: 69208.22 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1519/3051 [11:59<12:05,  2.11it/s, est. speed input: 69208.18 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1520/3051 [11:59<12:05,  2.11it/s, est. speed input: 69208.07 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1521/3051 [12:00<12:04,  2.11it/s, est. speed input: 69208.14 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1522/3051 [12:00<12:03,  2.11it/s, est. speed input: 69208.16 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1523/3051 [12:01<12:03,  2.11it/s, est. speed input: 69208.22 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1524/3051 [12:01<12:02,  2.11it/s, est. speed input: 69208.24 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1525/3051 [12:02<12:01,  2.11it/s, est. speed input: 69208.33 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1526/3051 [12:02<12:01,  2.11it/s, est. speed input: 69208.35 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1527/3051 [12:02<12:01,  2.11it/s, est. speed input: 69208.33 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1528/3051 [12:03<12:00,  2.11it/s, est. speed input: 69208.40 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1529/3051 [12:03<12:00,  2.11it/s, est. speed input: 69208.45 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1530/3051 [12:04<11:59,  2.11it/s, est. speed input: 69208.49 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1531/3051 [12:04<11:58,  2.11it/s, est. speed input: 69208.57 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1532/3051 [12:05<11:58,  2.11it/s, est. speed input: 69208.60 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1533/3051 [12:05<11:57,  2.11it/s, est. speed input: 69208.66 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1534/3051 [12:06<11:57,  2.11it/s, est. speed input: 69208.72 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1535/3051 [12:06<11:57,  2.11it/s, est. speed input: 69208.74 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1536/3051 [12:07<11:57,  2.11it/s, est. speed input: 69208.55 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1537/3051 [12:07<11:57,  2.11it/s, est. speed input: 69208.41 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1538/3051 [12:08<11:56,  2.11it/s, est. speed input: 69208.45 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1539/3051 [12:08<11:56,  2.11it/s, est. speed input: 69208.41 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1540/3051 [12:09<11:56,  2.11it/s, est. speed input: 69208.35 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1541/3051 [12:09<11:56,  2.11it/s, est. speed input: 69208.09 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1542/3051 [12:10<11:56,  2.10it/s, est. speed input: 69207.83 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1543/3051 [12:10<11:55,  2.11it/s, est. speed input: 69207.92 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1544/3051 [12:11<11:54,  2.11it/s, est. speed input: 69207.92 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1545/3051 [12:11<11:53,  2.11it/s, est. speed input: 69208.01 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1546/3051 [12:11<11:52,  2.11it/s, est. speed input: 69208.03 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1547/3051 [12:12<11:52,  2.11it/s, est. speed input: 69208.02 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1548/3051 [12:12<11:52,  2.11it/s, est. speed input: 69207.89 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1549/3051 [12:13<11:51,  2.11it/s, est. speed input: 69207.85 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1550/3051 [12:13<11:51,  2.11it/s, est. speed input: 69207.84 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1551/3051 [12:14<11:50,  2.11it/s, est. speed input: 69207.88 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1552/3051 [12:14<11:49,  2.11it/s, est. speed input: 69207.96 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1553/3051 [12:15<11:49,  2.11it/s, est. speed input: 69207.84 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1554/3051 [12:15<11:49,  2.11it/s, est. speed input: 69207.82 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1555/3051 [12:16<11:48,  2.11it/s, est. speed input: 69207.92 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1556/3051 [12:16<11:47,  2.11it/s, est. speed input: 69207.90 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1557/3051 [12:17<11:46,  2.11it/s, est. speed input: 69207.98 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1558/3051 [12:17<11:47,  2.11it/s, est. speed input: 69207.89 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1559/3051 [12:18<11:46,  2.11it/s, est. speed input: 69207.89 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1560/3051 [12:18<11:46,  2.11it/s, est. speed input: 69207.85 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1561/3051 [12:19<11:49,  2.10it/s, est. speed input: 69207.08 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1562/3051 [12:19<11:47,  2.10it/s, est. speed input: 69207.09 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1563/3051 [12:20<11:46,  2.11it/s, est. speed input: 69207.15 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1564/3051 [12:20<11:44,  2.11it/s, est. speed input: 69207.21 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1565/3051 [12:20<11:43,  2.11it/s, est. speed input: 69207.28 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1566/3051 [12:21<11:42,  2.11it/s, est. speed input: 69207.35 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1567/3051 [12:21<11:42,  2.11it/s, est. speed input: 69207.42 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1568/3051 [12:22<11:42,  2.11it/s, est. speed input: 69207.32 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1569/3051 [12:22<11:41,  2.11it/s, est. speed input: 69207.42 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1570/3051 [12:23<11:40,  2.11it/s, est. speed input: 69207.48 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1571/3051 [12:23<11:39,  2.12it/s, est. speed input: 69207.61 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1572/3051 [12:24<11:39,  2.11it/s, est. speed input: 69207.52 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1573/3051 [12:24<11:39,  2.11it/s, est. speed input: 69207.59 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1574/3051 [12:25<11:38,  2.11it/s, est. speed input: 69207.60 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1575/3051 [12:25<11:38,  2.11it/s, est. speed input: 69207.70 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1576/3051 [12:26<11:37,  2.12it/s, est. speed input: 69207.80 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1577/3051 [12:26<11:36,  2.12it/s, est. speed input: 69207.86 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1578/3051 [12:27<11:36,  2.11it/s, est. speed input: 69207.89 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1579/3051 [12:27<11:36,  2.11it/s, est. speed input: 69207.90 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1580/3051 [12:28<11:35,  2.11it/s, est. speed input: 69207.98 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1581/3051 [12:28<11:35,  2.11it/s, est. speed input: 69207.93 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1582/3051 [12:29<11:35,  2.11it/s, est. speed input: 69207.95 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1583/3051 [12:29<11:34,  2.11it/s, est. speed input: 69208.03 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1584/3051 [12:29<11:34,  2.11it/s, est. speed input: 69208.02 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1585/3051 [12:30<11:34,  2.11it/s, est. speed input: 69207.82 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1586/3051 [12:30<11:34,  2.11it/s, est. speed input: 69207.83 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1587/3051 [12:31<11:33,  2.11it/s, est. speed input: 69207.88 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1588/3051 [12:31<11:33,  2.11it/s, est. speed input: 69207.80 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1589/3051 [12:32<11:32,  2.11it/s, est. speed input: 69207.87 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1590/3051 [12:32<11:31,  2.11it/s, est. speed input: 69207.85 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1591/3051 [12:33<11:31,  2.11it/s, est. speed input: 69207.91 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1592/3051 [12:33<11:30,  2.11it/s, est. speed input: 69207.92 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1593/3051 [12:34<11:35,  2.10it/s, est. speed input: 69206.87 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1594/3051 [12:34<11:33,  2.10it/s, est. speed input: 69206.91 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1595/3051 [12:37<29:08,  1.20s/it, est. speed input: 68985.94 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1596/3051 [12:38<23:47,  1.02it/s, est. speed input: 68986.45 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1597/3051 [12:38<20:04,  1.21it/s, est. speed input: 68986.81 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1598/3051 [12:39<17:27,  1.39it/s, est. speed input: 68987.20 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1599/3051 [12:39<15:37,  1.55it/s, est. speed input: 68987.60 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1600/3051 [12:39<14:21,  1.68it/s, est. speed input: 68987.93 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1601/3051 [12:40<13:27,  1.80it/s, est. speed input: 68988.35 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1602/3051 [12:40<12:49,  1.88it/s, est. speed input: 68988.68 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1603/3051 [12:41<12:23,  1.95it/s, est. speed input: 68989.00 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1604/3051 [12:41<12:04,  2.00it/s, est. speed input: 68989.36 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1605/3051 [12:42<11:51,  2.03it/s, est. speed input: 68989.68 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1606/3051 [12:42<11:41,  2.06it/s, est. speed input: 68990.08 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1607/3051 [12:43<11:34,  2.08it/s, est. speed input: 68990.50 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1608/3051 [12:43<11:29,  2.09it/s, est. speed input: 68990.92 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1609/3051 [12:44<11:25,  2.10it/s, est. speed input: 68991.31 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1610/3051 [12:44<11:23,  2.11it/s, est. speed input: 68991.72 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1611/3051 [12:45<11:21,  2.11it/s, est. speed input: 68992.09 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1612/3051 [12:45<11:19,  2.12it/s, est. speed input: 68992.47 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1613/3051 [12:46<11:18,  2.12it/s, est. speed input: 68992.87 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1614/3051 [12:46<11:17,  2.12it/s, est. speed input: 68993.19 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1615/3051 [12:47<11:17,  2.12it/s, est. speed input: 68993.50 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1616/3051 [12:47<11:16,  2.12it/s, est. speed input: 68993.91 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1617/3051 [12:47<11:15,  2.12it/s, est. speed input: 68994.31 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1618/3051 [12:48<11:15,  2.12it/s, est. speed input: 68994.63 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1619/3051 [12:48<11:15,  2.12it/s, est. speed input: 68994.81 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1620/3051 [12:49<11:14,  2.12it/s, est. speed input: 68995.13 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1621/3051 [12:49<11:13,  2.12it/s, est. speed input: 68995.52 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1622/3051 [12:50<11:13,  2.12it/s, est. speed input: 68995.91 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1623/3051 [12:50<11:12,  2.12it/s, est. speed input: 68996.29 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1624/3051 [12:51<11:11,  2.12it/s, est. speed input: 68996.66 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1625/3051 [12:51<11:12,  2.12it/s, est. speed input: 68996.90 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1626/3051 [12:52<11:11,  2.12it/s, est. speed input: 68997.17 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1627/3051 [12:52<11:11,  2.12it/s, est. speed input: 68997.50 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1628/3051 [12:53<11:10,  2.12it/s, est. speed input: 68997.80 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1629/3051 [12:53<11:10,  2.12it/s, est. speed input: 68998.10 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1630/3051 [12:54<11:09,  2.12it/s, est. speed input: 68998.46 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1631/3051 [12:54<11:08,  2.12it/s, est. speed input: 68998.84 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1632/3051 [12:55<11:08,  2.12it/s, est. speed input: 68999.20 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1633/3051 [12:55<11:08,  2.12it/s, est. speed input: 68999.43 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1634/3051 [12:55<11:07,  2.12it/s, est. speed input: 68999.80 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1635/3051 [12:56<11:07,  2.12it/s, est. speed input: 69000.05 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1636/3051 [12:56<11:07,  2.12it/s, est. speed input: 69000.36 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1637/3051 [12:57<11:08,  2.12it/s, est. speed input: 69000.35 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1638/3051 [12:57<11:07,  2.12it/s, est. speed input: 69000.62 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1639/3051 [12:58<11:06,  2.12it/s, est. speed input: 69000.90 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1640/3051 [12:58<11:06,  2.12it/s, est. speed input: 69001.16 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1641/3051 [12:59<11:05,  2.12it/s, est. speed input: 69001.42 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1642/3051 [12:59<11:04,  2.12it/s, est. speed input: 69001.73 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1643/3051 [13:00<11:04,  2.12it/s, est. speed input: 69001.96 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1644/3051 [13:00<11:04,  2.12it/s, est. speed input: 69002.17 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1645/3051 [13:01<11:04,  2.12it/s, est. speed input: 69002.36 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1646/3051 [13:01<11:03,  2.12it/s, est. speed input: 69002.60 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1647/3051 [13:02<11:02,  2.12it/s, est. speed input: 69002.97 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1648/3051 [13:02<11:01,  2.12it/s, est. speed input: 69003.25 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1649/3051 [13:03<11:01,  2.12it/s, est. speed input: 69003.47 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1650/3051 [13:03<11:01,  2.12it/s, est. speed input: 69003.71 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1651/3051 [13:04<11:00,  2.12it/s, est. speed input: 69003.99 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1652/3051 [13:04<11:00,  2.12it/s, est. speed input: 69004.28 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1653/3051 [13:04<10:59,  2.12it/s, est. speed input: 69004.54 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1654/3051 [13:05<10:59,  2.12it/s, est. speed input: 69004.79 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1655/3051 [13:05<10:59,  2.12it/s, est. speed input: 69004.97 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1656/3051 [13:06<10:58,  2.12it/s, est. speed input: 69005.24 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1657/3051 [13:06<10:59,  2.11it/s, est. speed input: 69005.17 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1658/3051 [13:07<10:58,  2.11it/s, est. speed input: 69005.40 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1659/3051 [13:07<10:57,  2.12it/s, est. speed input: 69005.72 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1660/3051 [13:08<10:57,  2.12it/s, est. speed input: 69005.89 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1661/3051 [13:08<10:56,  2.12it/s, est. speed input: 69006.11 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1662/3051 [13:09<10:56,  2.12it/s, est. speed input: 69006.34 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1663/3051 [13:09<10:55,  2.12it/s, est. speed input: 69006.63 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1664/3051 [13:10<10:54,  2.12it/s, est. speed input: 69006.95 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1665/3051 [13:10<10:54,  2.12it/s, est. speed input: 69007.06 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1666/3051 [13:11<10:54,  2.12it/s, est. speed input: 69007.23 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1667/3051 [13:11<10:54,  2.12it/s, est. speed input: 69007.40 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1668/3051 [13:12<10:53,  2.12it/s, est. speed input: 69007.67 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1669/3051 [13:12<10:53,  2.11it/s, est. speed input: 69007.65 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1670/3051 [13:12<10:54,  2.11it/s, est. speed input: 69007.52 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1671/3051 [13:13<10:53,  2.11it/s, est. speed input: 69007.84 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1672/3051 [13:13<10:52,  2.11it/s, est. speed input: 69008.10 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1673/3051 [13:14<10:51,  2.11it/s, est. speed input: 69008.26 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1674/3051 [13:14<10:50,  2.12it/s, est. speed input: 69008.48 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1675/3051 [13:15<10:51,  2.11it/s, est. speed input: 69008.53 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1676/3051 [13:15<10:50,  2.11it/s, est. speed input: 69008.74 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1677/3051 [13:16<10:51,  2.11it/s, est. speed input: 69008.54 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1678/3051 [13:16<10:50,  2.11it/s, est. speed input: 69008.76 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1679/3051 [13:17<10:49,  2.11it/s, est. speed input: 69009.03 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1680/3051 [13:17<10:48,  2.11it/s, est. speed input: 69009.20 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1681/3051 [13:18<10:47,  2.12it/s, est. speed input: 69009.46 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1682/3051 [13:18<10:47,  2.12it/s, est. speed input: 69009.66 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1683/3051 [13:19<10:46,  2.12it/s, est. speed input: 69009.82 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1684/3051 [13:19<10:46,  2.11it/s, est. speed input: 69009.97 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1685/3051 [13:20<10:45,  2.12it/s, est. speed input: 69010.18 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1686/3051 [13:20<10:44,  2.12it/s, est. speed input: 69010.46 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1687/3051 [13:21<10:44,  2.12it/s, est. speed input: 69010.65 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1688/3051 [13:21<10:43,  2.12it/s, est. speed input: 69010.97 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1689/3051 [13:21<10:43,  2.12it/s, est. speed input: 69011.08 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1690/3051 [13:22<10:44,  2.11it/s, est. speed input: 69011.06 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1691/3051 [13:22<10:43,  2.11it/s, est. speed input: 69011.20 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1692/3051 [13:23<10:42,  2.12it/s, est. speed input: 69011.54 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1693/3051 [13:23<10:41,  2.12it/s, est. speed input: 69011.77 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1694/3051 [13:24<10:41,  2.12it/s, est. speed input: 69011.89 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1695/3051 [13:24<10:40,  2.12it/s, est. speed input: 69012.09 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1696/3051 [13:25<10:40,  2.12it/s, est. speed input: 69012.32 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1697/3051 [13:25<10:40,  2.11it/s, est. speed input: 69012.40 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1698/3051 [13:26<10:39,  2.12it/s, est. speed input: 69012.66 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1699/3051 [13:26<10:40,  2.11it/s, est. speed input: 69012.51 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1700/3051 [13:27<10:39,  2.11it/s, est. speed input: 69012.69 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1701/3051 [13:27<10:38,  2.11it/s, est. speed input: 69012.98 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1702/3051 [13:28<10:38,  2.11it/s, est. speed input: 69012.96 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1703/3051 [13:28<10:37,  2.11it/s, est. speed input: 69013.17 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1704/3051 [13:29<10:37,  2.11it/s, est. speed input: 69013.37 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1705/3051 [13:29<10:36,  2.12it/s, est. speed input: 69013.65 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1706/3051 [13:30<10:35,  2.12it/s, est. speed input: 69013.91 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1707/3051 [13:30<10:36,  2.11it/s, est. speed input: 69013.71 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1708/3051 [13:30<10:35,  2.11it/s, est. speed input: 69014.00 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1709/3051 [13:31<10:34,  2.12it/s, est. speed input: 69014.24 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1710/3051 [13:31<10:34,  2.11it/s, est. speed input: 69014.29 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1711/3051 [13:32<10:33,  2.11it/s, est. speed input: 69014.46 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1712/3051 [13:32<10:32,  2.12it/s, est. speed input: 69014.77 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1713/3051 [13:33<10:32,  2.12it/s, est. speed input: 69014.99 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1714/3051 [13:33<10:31,  2.12it/s, est. speed input: 69015.14 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1715/3051 [13:34<10:32,  2.11it/s, est. speed input: 69015.17 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1716/3051 [13:34<10:31,  2.11it/s, est. speed input: 69015.25 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1717/3051 [13:35<10:30,  2.11it/s, est. speed input: 69015.47 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1718/3051 [13:35<10:30,  2.12it/s, est. speed input: 69015.72 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1719/3051 [13:36<10:29,  2.12it/s, est. speed input: 69015.97 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1720/3051 [13:36<10:28,  2.12it/s, est. speed input: 69016.16 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1721/3051 [13:37<10:27,  2.12it/s, est. speed input: 69016.44 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1722/3051 [13:37<10:29,  2.11it/s, est. speed input: 69016.17 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1723/3051 [13:38<10:28,  2.11it/s, est. speed input: 69016.34 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1724/3051 [13:38<10:27,  2.11it/s, est. speed input: 69016.62 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1725/3051 [13:39<10:27,  2.11it/s, est. speed input: 69016.77 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1726/3051 [13:39<10:26,  2.11it/s, est. speed input: 69016.87 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1727/3051 [13:39<10:25,  2.12it/s, est. speed input: 69017.11 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1728/3051 [13:40<10:24,  2.12it/s, est. speed input: 69017.41 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1729/3051 [13:40<10:25,  2.11it/s, est. speed input: 69017.47 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1730/3051 [13:41<10:24,  2.11it/s, est. speed input: 69017.57 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1731/3051 [13:41<10:24,  2.11it/s, est. speed input: 69017.74 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1732/3051 [13:42<10:23,  2.11it/s, est. speed input: 69017.88 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1733/3051 [13:42<10:22,  2.12it/s, est. speed input: 69018.12 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1734/3051 [13:43<10:22,  2.12it/s, est. speed input: 69018.33 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1735/3051 [13:43<10:22,  2.11it/s, est. speed input: 69018.36 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1736/3051 [13:44<10:21,  2.12it/s, est. speed input: 69018.61 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1737/3051 [13:44<10:21,  2.11it/s, est. speed input: 69018.61 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1738/3051 [13:45<10:21,  2.11it/s, est. speed input: 69018.79 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1739/3051 [13:45<10:20,  2.11it/s, est. speed input: 69018.90 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1740/3051 [13:46<10:19,  2.11it/s, est. speed input: 69019.13 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1741/3051 [13:46<10:19,  2.12it/s, est. speed input: 69019.37 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1742/3051 [13:47<10:18,  2.11it/s, est. speed input: 69019.48 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1743/3051 [13:47<10:18,  2.12it/s, est. speed input: 69019.68 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1744/3051 [13:47<10:19,  2.11it/s, est. speed input: 69019.56 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1745/3051 [13:48<10:17,  2.12it/s, est. speed input: 69019.90 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1746/3051 [13:48<10:16,  2.12it/s, est. speed input: 69020.12 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1747/3051 [13:49<10:17,  2.11it/s, est. speed input: 69020.09 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1748/3051 [13:49<10:17,  2.11it/s, est. speed input: 69020.07 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1749/3051 [13:50<10:16,  2.11it/s, est. speed input: 69020.16 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1750/3051 [13:50<10:15,  2.11it/s, est. speed input: 69020.37 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1751/3051 [13:51<10:15,  2.11it/s, est. speed input: 69020.54 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1752/3051 [13:51<10:15,  2.11it/s, est. speed input: 69020.57 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1753/3051 [13:52<10:14,  2.11it/s, est. speed input: 69020.69 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1754/3051 [13:52<10:13,  2.11it/s, est. speed input: 69020.84 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1755/3051 [13:53<10:14,  2.11it/s, est. speed input: 69020.74 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1756/3051 [13:53<10:13,  2.11it/s, est. speed input: 69020.85 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1757/3051 [13:54<10:13,  2.11it/s, est. speed input: 69020.95 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1758/3051 [13:54<10:13,  2.11it/s, est. speed input: 69020.92 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1759/3051 [13:55<10:12,  2.11it/s, est. speed input: 69021.04 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1760/3051 [13:55<10:11,  2.11it/s, est. speed input: 69021.09 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1761/3051 [13:56<10:13,  2.10it/s, est. speed input: 69020.73 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1762/3051 [13:56<10:12,  2.11it/s, est. speed input: 69020.85 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1763/3051 [13:56<10:10,  2.11it/s, est. speed input: 69021.02 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1764/3051 [13:57<10:10,  2.11it/s, est. speed input: 69021.04 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1765/3051 [13:57<10:10,  2.11it/s, est. speed input: 69021.00 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1766/3051 [13:58<10:09,  2.11it/s, est. speed input: 69021.05 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1767/3051 [13:58<10:08,  2.11it/s, est. speed input: 69021.25 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1768/3051 [13:59<10:07,  2.11it/s, est. speed input: 69021.37 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1769/3051 [13:59<10:07,  2.11it/s, est. speed input: 69021.48 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1770/3051 [14:00<10:07,  2.11it/s, est. speed input: 69021.49 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1771/3051 [14:00<10:06,  2.11it/s, est. speed input: 69021.60 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1772/3051 [14:01<10:05,  2.11it/s, est. speed input: 69021.74 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1773/3051 [14:01<10:07,  2.10it/s, est. speed input: 69021.38 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1774/3051 [14:02<10:06,  2.10it/s, est. speed input: 69021.38 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1775/3051 [14:02<10:05,  2.11it/s, est. speed input: 69021.48 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1776/3051 [14:03<10:04,  2.11it/s, est. speed input: 69021.59 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1777/3051 [14:03<10:04,  2.11it/s, est. speed input: 69021.60 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1778/3051 [14:04<10:03,  2.11it/s, est. speed input: 69021.69 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1779/3051 [14:04<10:03,  2.11it/s, est. speed input: 69021.62 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1780/3051 [14:05<10:02,  2.11it/s, est. speed input: 69021.70 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1781/3051 [14:05<10:02,  2.11it/s, est. speed input: 69021.77 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1782/3051 [14:06<10:01,  2.11it/s, est. speed input: 69021.89 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1783/3051 [14:06<10:01,  2.11it/s, est. speed input: 69021.94 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1784/3051 [14:06<10:00,  2.11it/s, est. speed input: 69022.06 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1785/3051 [14:07<09:59,  2.11it/s, est. speed input: 69022.20 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1786/3051 [14:07<10:01,  2.10it/s, est. speed input: 69021.86 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1787/3051 [14:08<10:00,  2.11it/s, est. speed input: 69021.89 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1788/3051 [14:08<09:59,  2.11it/s, est. speed input: 69022.00 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1789/3051 [14:09<09:59,  2.11it/s, est. speed input: 69021.93 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1790/3051 [14:09<09:58,  2.11it/s, est. speed input: 69022.08 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1791/3051 [14:10<09:56,  2.11it/s, est. speed input: 69022.25 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1792/3051 [14:10<09:57,  2.11it/s, est. speed input: 69022.19 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1793/3051 [14:11<09:56,  2.11it/s, est. speed input: 69022.32 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1794/3051 [14:11<09:55,  2.11it/s, est. speed input: 69022.39 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1795/3051 [14:12<09:57,  2.10it/s, est. speed input: 69022.07 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1796/3051 [14:12<09:56,  2.11it/s, est. speed input: 69022.10 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1797/3051 [14:13<09:54,  2.11it/s, est. speed input: 69022.24 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1798/3051 [14:13<09:54,  2.11it/s, est. speed input: 69022.16 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1799/3051 [14:14<09:54,  2.11it/s, est. speed input: 69022.24 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1800/3051 [14:14<09:52,  2.11it/s, est. speed input: 69022.42 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1801/3051 [14:15<09:52,  2.11it/s, est. speed input: 69022.40 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1802/3051 [14:15<09:52,  2.11it/s, est. speed input: 69022.38 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1803/3051 [14:15<09:51,  2.11it/s, est. speed input: 69022.48 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1804/3051 [14:16<09:51,  2.11it/s, est. speed input: 69022.58 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1805/3051 [14:16<09:50,  2.11it/s, est. speed input: 69022.62 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1806/3051 [14:17<09:49,  2.11it/s, est. speed input: 69022.78 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1807/3051 [14:17<09:49,  2.11it/s, est. speed input: 69022.80 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1808/3051 [14:18<09:49,  2.11it/s, est. speed input: 69022.81 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1809/3051 [14:18<09:48,  2.11it/s, est. speed input: 69022.92 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1810/3051 [14:19<09:47,  2.11it/s, est. speed input: 69023.08 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1811/3051 [14:19<09:47,  2.11it/s, est. speed input: 69023.22 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1812/3051 [14:20<09:46,  2.11it/s, est. speed input: 69023.33 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1813/3051 [14:20<09:45,  2.11it/s, est. speed input: 69023.51 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1814/3051 [14:21<09:45,  2.11it/s, est. speed input: 69023.62 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1815/3051 [14:21<09:45,  2.11it/s, est. speed input: 69023.67 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1816/3051 [14:22<09:44,  2.11it/s, est. speed input: 69023.80 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1817/3051 [14:22<09:44,  2.11it/s, est. speed input: 69023.85 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1818/3051 [14:23<09:45,  2.11it/s, est. speed input: 69023.61 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1819/3051 [14:23<09:44,  2.11it/s, est. speed input: 69023.75 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1820/3051 [14:24<09:43,  2.11it/s, est. speed input: 69023.91 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1821/3051 [14:24<09:43,  2.11it/s, est. speed input: 69023.93 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1822/3051 [14:24<09:42,  2.11it/s, est. speed input: 69024.04 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1823/3051 [14:25<09:41,  2.11it/s, est. speed input: 69024.18 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1824/3051 [14:25<09:42,  2.10it/s, est. speed input: 69023.87 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1825/3051 [14:26<09:42,  2.11it/s, est. speed input: 69023.93 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1826/3051 [14:26<09:41,  2.11it/s, est. speed input: 69024.04 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1827/3051 [14:27<09:41,  2.10it/s, est. speed input: 69023.84 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1828/3051 [14:27<09:41,  2.10it/s, est. speed input: 69023.76 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1829/3051 [14:28<09:40,  2.11it/s, est. speed input: 69023.88 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1830/3051 [14:28<09:39,  2.11it/s, est. speed input: 69023.95 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1831/3051 [14:29<09:38,  2.11it/s, est. speed input: 69023.96 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1832/3051 [14:29<09:37,  2.11it/s, est. speed input: 69024.07 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1833/3051 [14:30<09:38,  2.10it/s, est. speed input: 69023.83 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1834/3051 [14:30<09:37,  2.11it/s, est. speed input: 69023.99 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1835/3051 [14:31<09:36,  2.11it/s, est. speed input: 69024.13 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1836/3051 [14:31<09:36,  2.11it/s, est. speed input: 69024.10 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1837/3051 [14:32<09:35,  2.11it/s, est. speed input: 69024.12 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1838/3051 [14:32<09:34,  2.11it/s, est. speed input: 69024.27 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1839/3051 [14:33<09:34,  2.11it/s, est. speed input: 69024.39 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1840/3051 [14:33<09:35,  2.10it/s, est. speed input: 69024.06 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1841/3051 [14:33<09:34,  2.11it/s, est. speed input: 69024.13 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1842/3051 [14:34<09:33,  2.11it/s, est. speed input: 69024.29 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1843/3051 [14:34<09:33,  2.11it/s, est. speed input: 69024.26 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1844/3051 [14:35<09:32,  2.11it/s, est. speed input: 69024.34 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1845/3051 [14:35<09:31,  2.11it/s, est. speed input: 69024.37 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1846/3051 [14:36<09:31,  2.11it/s, est. speed input: 69024.35 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1847/3051 [14:36<09:31,  2.11it/s, est. speed input: 69024.32 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1848/3051 [14:37<09:30,  2.11it/s, est. speed input: 69024.45 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1849/3051 [14:37<09:30,  2.11it/s, est. speed input: 69024.47 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1850/3051 [14:38<09:29,  2.11it/s, est. speed input: 69024.54 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1851/3051 [14:38<09:28,  2.11it/s, est. speed input: 69024.73 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1852/3051 [14:39<09:28,  2.11it/s, est. speed input: 69024.71 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1853/3051 [14:39<09:27,  2.11it/s, est. speed input: 69024.82 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1854/3051 [14:40<09:27,  2.11it/s, est. speed input: 69024.83 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1855/3051 [14:40<09:27,  2.11it/s, est. speed input: 69024.80 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1856/3051 [14:41<09:26,  2.11it/s, est. speed input: 69024.86 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1857/3051 [14:41<09:25,  2.11it/s, est. speed input: 69025.01 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1858/3051 [14:42<09:39,  2.06it/s, est. speed input: 69022.02 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1859/3051 [14:42<09:33,  2.08it/s, est. speed input: 69022.29 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1860/3051 [14:43<09:30,  2.09it/s, est. speed input: 69022.46 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1861/3051 [14:43<09:27,  2.10it/s, est. speed input: 69022.55 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1862/3051 [14:43<09:25,  2.10it/s, est. speed input: 69022.68 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1863/3051 [14:44<09:24,  2.10it/s, est. speed input: 69022.80 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1864/3051 [14:44<09:23,  2.11it/s, est. speed input: 69022.91 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1865/3051 [14:45<09:22,  2.11it/s, est. speed input: 69022.93 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1866/3051 [14:45<09:21,  2.11it/s, est. speed input: 69023.04 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1867/3051 [14:46<09:21,  2.11it/s, est. speed input: 69023.17 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1868/3051 [14:46<09:21,  2.11it/s, est. speed input: 69023.13 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1869/3051 [14:47<09:21,  2.11it/s, est. speed input: 69023.07 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1870/3051 [14:47<09:20,  2.11it/s, est. speed input: 69023.17 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1871/3051 [14:48<09:19,  2.11it/s, est. speed input: 69023.29 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1872/3051 [14:48<09:18,  2.11it/s, est. speed input: 69023.38 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1873/3051 [14:49<09:17,  2.11it/s, est. speed input: 69023.55 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1874/3051 [14:49<09:17,  2.11it/s, est. speed input: 69023.56 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1875/3051 [14:50<09:16,  2.11it/s, est. speed input: 69023.68 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1876/3051 [14:50<09:16,  2.11it/s, est. speed input: 69023.85 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1877/3051 [14:51<09:16,  2.11it/s, est. speed input: 69023.69 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1878/3051 [14:51<09:16,  2.11it/s, est. speed input: 69023.72 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1879/3051 [14:52<09:15,  2.11it/s, est. speed input: 69023.89 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1880/3051 [14:52<09:15,  2.11it/s, est. speed input: 69023.90 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1881/3051 [14:52<09:14,  2.11it/s, est. speed input: 69024.01 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1882/3051 [14:53<09:13,  2.11it/s, est. speed input: 69024.13 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1883/3051 [14:53<09:15,  2.10it/s, est. speed input: 69023.76 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1884/3051 [14:54<09:14,  2.11it/s, est. speed input: 69023.86 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1885/3051 [14:54<09:13,  2.11it/s, est. speed input: 69023.98 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1886/3051 [14:55<09:12,  2.11it/s, est. speed input: 69024.12 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1887/3051 [14:55<09:11,  2.11it/s, est. speed input: 69024.12 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1888/3051 [14:56<09:10,  2.11it/s, est. speed input: 69024.27 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1889/3051 [14:56<09:10,  2.11it/s, est. speed input: 69024.32 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1890/3051 [14:57<09:10,  2.11it/s, est. speed input: 69024.41 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1891/3051 [14:57<09:09,  2.11it/s, est. speed input: 69024.56 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1892/3051 [14:58<09:08,  2.11it/s, est. speed input: 69024.62 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1893/3051 [14:58<09:08,  2.11it/s, est. speed input: 69024.73 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1894/3051 [14:59<09:09,  2.11it/s, est. speed input: 69024.43 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1895/3051 [14:59<09:08,  2.11it/s, est. speed input: 69024.56 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1896/3051 [15:00<09:07,  2.11it/s, est. speed input: 69024.73 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1897/3051 [15:00<09:06,  2.11it/s, est. speed input: 69024.90 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1898/3051 [15:01<09:06,  2.11it/s, est. speed input: 69024.91 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1899/3051 [15:01<09:05,  2.11it/s, est. speed input: 69024.98 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1900/3051 [15:01<09:06,  2.10it/s, est. speed input: 69024.70 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1901/3051 [15:02<09:05,  2.11it/s, est. speed input: 69024.80 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1902/3051 [15:02<09:04,  2.11it/s, est. speed input: 69024.97 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1903/3051 [15:03<09:04,  2.11it/s, est. speed input: 69025.02 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1904/3051 [15:03<09:04,  2.11it/s, est. speed input: 69025.01 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1905/3051 [15:04<09:03,  2.11it/s, est. speed input: 69025.08 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1906/3051 [15:04<09:02,  2.11it/s, est. speed input: 69025.19 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1907/3051 [15:05<09:03,  2.11it/s, est. speed input: 69025.01 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1908/3051 [15:05<09:02,  2.11it/s, est. speed input: 69025.09 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1909/3051 [15:06<09:01,  2.11it/s, est. speed input: 69025.18 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1910/3051 [15:06<09:00,  2.11it/s, est. speed input: 69025.29 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1911/3051 [15:07<09:00,  2.11it/s, est. speed input: 69025.41 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1912/3051 [15:07<08:59,  2.11it/s, est. speed input: 69025.54 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1913/3051 [15:08<08:58,  2.11it/s, est. speed input: 69025.70 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1914/3051 [15:08<08:57,  2.11it/s, est. speed input: 69025.83 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1915/3051 [15:09<08:58,  2.11it/s, est. speed input: 69025.74 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1916/3051 [15:09<08:57,  2.11it/s, est. speed input: 69025.84 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1917/3051 [15:10<08:57,  2.11it/s, est. speed input: 69025.78 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1918/3051 [15:10<08:58,  2.10it/s, est. speed input: 69025.52 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1919/3051 [15:10<08:58,  2.10it/s, est. speed input: 69025.50 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1920/3051 [15:11<08:56,  2.11it/s, est. speed input: 69025.68 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1921/3051 [15:11<08:55,  2.11it/s, est. speed input: 69025.80 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1922/3051 [15:12<08:55,  2.11it/s, est. speed input: 69025.84 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1923/3051 [15:12<08:54,  2.11it/s, est. speed input: 69025.91 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1924/3051 [15:13<08:53,  2.11it/s, est. speed input: 69026.04 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1925/3051 [15:13<08:53,  2.11it/s, est. speed input: 69026.19 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1926/3051 [15:14<08:52,  2.11it/s, est. speed input: 69026.29 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1927/3051 [15:14<08:51,  2.11it/s, est. speed input: 69026.43 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1928/3051 [15:15<08:51,  2.11it/s, est. speed input: 69026.55 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1929/3051 [15:15<08:51,  2.11it/s, est. speed input: 69026.55 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1930/3051 [15:16<08:50,  2.11it/s, est. speed input: 69026.65 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1931/3051 [15:16<08:50,  2.11it/s, est. speed input: 69026.79 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1932/3051 [15:17<08:49,  2.11it/s, est. speed input: 69026.82 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1933/3051 [15:17<08:49,  2.11it/s, est. speed input: 69026.94 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1934/3051 [15:18<08:48,  2.11it/s, est. speed input: 69027.05 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1935/3051 [15:18<08:49,  2.11it/s, est. speed input: 69026.88 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1936/3051 [15:19<08:49,  2.11it/s, est. speed input: 69026.89 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1937/3051 [15:19<08:48,  2.11it/s, est. speed input: 69026.96 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1938/3051 [15:19<08:49,  2.10it/s, est. speed input: 69026.73 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1939/3051 [15:20<08:48,  2.11it/s, est. speed input: 69026.81 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1940/3051 [15:20<08:47,  2.11it/s, est. speed input: 69026.92 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1941/3051 [15:21<08:46,  2.11it/s, est. speed input: 69026.97 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1942/3051 [15:21<08:46,  2.11it/s, est. speed input: 69026.97 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1943/3051 [15:22<08:45,  2.11it/s, est. speed input: 69027.08 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1944/3051 [15:22<08:44,  2.11it/s, est. speed input: 69027.19 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1945/3051 [15:23<08:44,  2.11it/s, est. speed input: 69027.16 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1946/3051 [15:23<08:43,  2.11it/s, est. speed input: 69027.28 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1947/3051 [15:24<08:43,  2.11it/s, est. speed input: 69027.23 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1948/3051 [15:24<08:42,  2.11it/s, est. speed input: 69027.34 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1949/3051 [15:25<08:42,  2.11it/s, est. speed input: 69027.42 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1950/3051 [15:25<08:42,  2.11it/s, est. speed input: 69027.32 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1951/3051 [15:26<08:42,  2.11it/s, est. speed input: 69027.27 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1952/3051 [15:26<08:41,  2.11it/s, est. speed input: 69027.42 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1953/3051 [15:27<08:44,  2.09it/s, est. speed input: 69026.65 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1954/3051 [15:27<08:42,  2.10it/s, est. speed input: 69026.77 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1955/3051 [15:28<08:40,  2.11it/s, est. speed input: 69026.93 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1956/3051 [15:28<08:39,  2.11it/s, est. speed input: 69027.03 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1957/3051 [15:29<08:39,  2.11it/s, est. speed input: 69026.98 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1958/3051 [15:29<08:39,  2.10it/s, est. speed input: 69026.90 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1959/3051 [15:29<08:38,  2.11it/s, est. speed input: 69027.02 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1960/3051 [15:30<08:37,  2.11it/s, est. speed input: 69027.14 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1961/3051 [15:30<08:37,  2.11it/s, est. speed input: 69027.09 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1962/3051 [15:31<08:36,  2.11it/s, est. speed input: 69027.13 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1963/3051 [15:31<08:36,  2.11it/s, est. speed input: 69027.17 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1964/3051 [15:32<08:35,  2.11it/s, est. speed input: 69027.32 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1965/3051 [15:32<08:34,  2.11it/s, est. speed input: 69027.43 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1966/3051 [15:33<08:35,  2.11it/s, est. speed input: 69027.16 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1967/3051 [15:33<08:34,  2.11it/s, est. speed input: 69027.32 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1968/3051 [15:34<08:33,  2.11it/s, est. speed input: 69027.47 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1969/3051 [15:34<08:32,  2.11it/s, est. speed input: 69027.48 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1970/3051 [15:35<08:32,  2.11it/s, est. speed input: 69027.61 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1971/3051 [15:35<08:31,  2.11it/s, est. speed input: 69027.75 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1972/3051 [15:36<08:32,  2.10it/s, est. speed input: 69027.41 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1973/3051 [15:36<08:32,  2.11it/s, est. speed input: 69027.42 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1974/3051 [15:37<08:30,  2.11it/s, est. speed input: 69027.55 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1975/3051 [15:37<08:31,  2.10it/s, est. speed input: 69027.31 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1976/3051 [15:38<08:30,  2.11it/s, est. speed input: 69027.46 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1977/3051 [15:38<08:29,  2.11it/s, est. speed input: 69027.56 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1978/3051 [15:38<08:29,  2.11it/s, est. speed input: 69027.51 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1979/3051 [15:39<08:28,  2.11it/s, est. speed input: 69027.56 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1980/3051 [15:39<08:27,  2.11it/s, est. speed input: 69027.65 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1981/3051 [15:40<08:27,  2.11it/s, est. speed input: 69027.74 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1982/3051 [15:40<08:26,  2.11it/s, est. speed input: 69027.79 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1983/3051 [15:41<08:26,  2.11it/s, est. speed input: 69027.85 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1984/3051 [15:41<08:25,  2.11it/s, est. speed input: 69027.97 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1985/3051 [15:42<08:25,  2.11it/s, est. speed input: 69027.89 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1986/3051 [15:42<08:25,  2.11it/s, est. speed input: 69027.91 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1987/3051 [15:43<08:24,  2.11it/s, est. speed input: 69028.07 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1988/3051 [15:43<08:24,  2.11it/s, est. speed input: 69028.00 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1989/3051 [15:44<08:23,  2.11it/s, est. speed input: 69028.02 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1990/3051 [15:44<08:22,  2.11it/s, est. speed input: 69028.14 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1991/3051 [15:45<08:22,  2.11it/s, est. speed input: 69028.08 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1992/3051 [15:45<08:22,  2.11it/s, est. speed input: 69028.15 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1993/3051 [15:46<08:21,  2.11it/s, est. speed input: 69028.28 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1994/3051 [15:46<08:20,  2.11it/s, est. speed input: 69028.41 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1995/3051 [15:47<08:19,  2.11it/s, est. speed input: 69028.53 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1996/3051 [15:47<08:19,  2.11it/s, est. speed input: 69028.67 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1997/3051 [15:47<08:18,  2.11it/s, est. speed input: 69028.79 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1998/3051 [15:48<08:18,  2.11it/s, est. speed input: 69028.86 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 1999/3051 [15:48<08:18,  2.11it/s, est. speed input: 69028.82 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2000/3051 [15:49<08:17,  2.11it/s, est. speed input: 69028.93 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2001/3051 [15:49<08:17,  2.11it/s, est. speed input: 69029.01 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2002/3051 [15:50<08:16,  2.11it/s, est. speed input: 69029.15 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2003/3051 [15:50<08:15,  2.11it/s, est. speed input: 69029.28 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2004/3051 [15:51<08:17,  2.10it/s, est. speed input: 69028.94 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2005/3051 [15:51<08:17,  2.10it/s, est. speed input: 69028.75 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2006/3051 [15:52<08:16,  2.11it/s, est. speed input: 69028.87 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2007/3051 [15:52<08:15,  2.11it/s, est. speed input: 69028.98 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2008/3051 [15:53<08:14,  2.11it/s, est. speed input: 69029.05 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2009/3051 [15:53<08:14,  2.11it/s, est. speed input: 69028.98 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2010/3051 [15:54<08:13,  2.11it/s, est. speed input: 69029.11 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2011/3051 [15:54<08:12,  2.11it/s, est. speed input: 69029.18 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2012/3051 [15:55<08:12,  2.11it/s, est. speed input: 69029.29 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2013/3051 [15:55<08:11,  2.11it/s, est. speed input: 69029.36 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2014/3051 [15:56<08:11,  2.11it/s, est. speed input: 69029.31 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2015/3051 [15:56<08:11,  2.11it/s, est. speed input: 69029.30 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2016/3051 [15:56<08:10,  2.11it/s, est. speed input: 69029.43 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2017/3051 [15:57<08:09,  2.11it/s, est. speed input: 69029.52 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2018/3051 [15:57<08:09,  2.11it/s, est. speed input: 69029.56 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2019/3051 [15:58<08:08,  2.11it/s, est. speed input: 69029.72 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2020/3051 [15:58<08:08,  2.11it/s, est. speed input: 69029.76 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2021/3051 [15:59<08:09,  2.10it/s, est. speed input: 69029.47 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2022/3051 [15:59<08:08,  2.11it/s, est. speed input: 69029.55 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2023/3051 [16:00<08:07,  2.11it/s, est. speed input: 69029.66 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2024/3051 [16:00<08:06,  2.11it/s, est. speed input: 69029.72 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2025/3051 [16:01<08:06,  2.11it/s, est. speed input: 69029.84 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2026/3051 [16:01<08:05,  2.11it/s, est. speed input: 69029.82 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2027/3051 [16:02<08:05,  2.11it/s, est. speed input: 69029.92 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2028/3051 [16:02<08:05,  2.11it/s, est. speed input: 69029.84 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2029/3051 [16:03<08:04,  2.11it/s, est. speed input: 69029.95 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2030/3051 [16:03<08:03,  2.11it/s, est. speed input: 69030.08 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2031/3051 [16:04<08:03,  2.11it/s, est. speed input: 69030.17 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2032/3051 [16:04<08:07,  2.09it/s, est. speed input: 69029.11 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2033/3051 [16:05<08:05,  2.10it/s, est. speed input: 69029.26 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2034/3051 [16:05<08:03,  2.10it/s, est. speed input: 69029.39 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2035/3051 [16:06<08:02,  2.11it/s, est. speed input: 69029.48 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2036/3051 [16:06<08:02,  2.11it/s, est. speed input: 69029.44 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2037/3051 [16:06<08:01,  2.11it/s, est. speed input: 69029.54 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2038/3051 [16:07<08:00,  2.11it/s, est. speed input: 69029.68 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2039/3051 [16:07<08:01,  2.10it/s, est. speed input: 69029.39 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2040/3051 [16:08<08:00,  2.11it/s, est. speed input: 69029.44 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2041/3051 [16:08<07:59,  2.11it/s, est. speed input: 69029.50 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2042/3051 [16:09<07:58,  2.11it/s, est. speed input: 69029.57 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2043/3051 [16:09<07:57,  2.11it/s, est. speed input: 69029.66 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2044/3051 [16:10<07:57,  2.11it/s, est. speed input: 69029.74 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2045/3051 [16:10<07:57,  2.11it/s, est. speed input: 69029.69 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2046/3051 [16:11<07:56,  2.11it/s, est. speed input: 69029.73 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2047/3051 [16:11<07:56,  2.11it/s, est. speed input: 69029.80 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2048/3051 [16:12<07:55,  2.11it/s, est. speed input: 69029.94 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2049/3051 [16:12<07:54,  2.11it/s, est. speed input: 69030.09 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2050/3051 [16:13<07:53,  2.11it/s, est. speed input: 69030.17 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2051/3051 [16:13<07:53,  2.11it/s, est. speed input: 69030.19 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2052/3051 [16:14<07:53,  2.11it/s, est. speed input: 69030.19 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2053/3051 [16:14<07:52,  2.11it/s, est. speed input: 69030.31 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2054/3051 [16:15<07:52,  2.11it/s, est. speed input: 69030.44 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2055/3051 [16:15<07:51,  2.11it/s, est. speed input: 69030.57 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2056/3051 [16:15<07:51,  2.11it/s, est. speed input: 69030.64 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2057/3051 [16:16<07:50,  2.11it/s, est. speed input: 69030.76 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2058/3051 [16:16<07:50,  2.11it/s, est. speed input: 69030.84 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2059/3051 [16:17<07:49,  2.11it/s, est. speed input: 69030.90 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2060/3051 [16:17<07:49,  2.11it/s, est. speed input: 69031.03 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2061/3051 [16:18<07:48,  2.11it/s, est. speed input: 69031.11 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2062/3051 [16:18<07:49,  2.10it/s, est. speed input: 69030.80 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2063/3051 [16:19<07:49,  2.11it/s, est. speed input: 69030.83 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2064/3051 [16:19<07:48,  2.11it/s, est. speed input: 69030.91 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2065/3051 [16:20<07:48,  2.10it/s, est. speed input: 69030.76 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2066/3051 [16:20<07:47,  2.11it/s, est. speed input: 69030.87 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2067/3051 [16:21<07:46,  2.11it/s, est. speed input: 69031.00 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2068/3051 [16:21<07:45,  2.11it/s, est. speed input: 69031.09 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2069/3051 [16:22<07:45,  2.11it/s, est. speed input: 69031.15 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2070/3051 [16:22<07:44,  2.11it/s, est. speed input: 69031.28 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2071/3051 [16:23<07:43,  2.11it/s, est. speed input: 69031.39 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2072/3051 [16:23<07:43,  2.11it/s, est. speed input: 69031.56 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2073/3051 [16:24<07:42,  2.11it/s, est. speed input: 69031.63 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2074/3051 [16:24<07:42,  2.11it/s, est. speed input: 69031.65 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2075/3051 [16:24<07:42,  2.11it/s, est. speed input: 69031.75 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2076/3051 [16:25<07:41,  2.11it/s, est. speed input: 69031.88 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2077/3051 [16:25<07:41,  2.11it/s, est. speed input: 69031.77 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2078/3051 [16:26<07:41,  2.11it/s, est. speed input: 69031.73 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2079/3051 [16:26<07:40,  2.11it/s, est. speed input: 69031.87 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2080/3051 [16:27<07:39,  2.11it/s, est. speed input: 69031.98 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2081/3051 [16:27<07:39,  2.11it/s, est. speed input: 69032.03 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2082/3051 [16:28<07:39,  2.11it/s, est. speed input: 69032.06 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2083/3051 [16:28<07:38,  2.11it/s, est. speed input: 69032.07 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2084/3051 [16:29<07:38,  2.11it/s, est. speed input: 69032.06 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2085/3051 [16:29<07:37,  2.11it/s, est. speed input: 69032.14 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2086/3051 [16:30<07:37,  2.11it/s, est. speed input: 69032.21 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2087/3051 [16:30<07:36,  2.11it/s, est. speed input: 69032.28 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2088/3051 [16:31<07:37,  2.10it/s, est. speed input: 69032.00 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2089/3051 [16:31<07:37,  2.10it/s, est. speed input: 69031.88 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2090/3051 [16:32<07:36,  2.11it/s, est. speed input: 69032.04 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2091/3051 [16:32<07:35,  2.11it/s, est. speed input: 69032.06 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2092/3051 [16:33<07:34,  2.11it/s, est. speed input: 69032.20 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2093/3051 [16:33<07:34,  2.11it/s, est. speed input: 69032.21 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2094/3051 [16:33<07:33,  2.11it/s, est. speed input: 69032.20 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2095/3051 [16:34<07:33,  2.11it/s, est. speed input: 69032.30 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2096/3051 [16:34<07:32,  2.11it/s, est. speed input: 69032.40 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2097/3051 [16:35<07:32,  2.11it/s, est. speed input: 69032.30 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2098/3051 [16:35<07:31,  2.11it/s, est. speed input: 69032.41 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2099/3051 [16:36<07:30,  2.11it/s, est. speed input: 69032.55 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2100/3051 [16:36<07:30,  2.11it/s, est. speed input: 69032.61 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2101/3051 [16:37<07:30,  2.11it/s, est. speed input: 69032.53 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2102/3051 [16:37<07:30,  2.11it/s, est. speed input: 69032.51 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2103/3051 [16:38<07:29,  2.11it/s, est. speed input: 69032.55 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2104/3051 [16:38<07:29,  2.11it/s, est. speed input: 69032.61 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2105/3051 [16:39<07:28,  2.11it/s, est. speed input: 69032.70 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2106/3051 [16:39<07:27,  2.11it/s, est. speed input: 69032.77 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2107/3051 [16:40<07:26,  2.11it/s, est. speed input: 69032.91 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2108/3051 [16:40<07:26,  2.11it/s, est. speed input: 69033.05 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2109/3051 [16:41<07:27,  2.11it/s, est. speed input: 69032.79 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2110/3051 [16:41<07:26,  2.11it/s, est. speed input: 69032.89 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2111/3051 [16:42<07:25,  2.11it/s, est. speed input: 69033.02 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2112/3051 [16:42<07:25,  2.11it/s, est. speed input: 69033.00 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2113/3051 [16:42<07:24,  2.11it/s, est. speed input: 69033.10 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2114/3051 [16:43<07:23,  2.11it/s, est. speed input: 69033.18 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2115/3051 [16:43<07:23,  2.11it/s, est. speed input: 69033.15 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2116/3051 [16:44<07:23,  2.11it/s, est. speed input: 69033.15 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2117/3051 [16:44<07:22,  2.11it/s, est. speed input: 69033.29 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2118/3051 [16:45<07:22,  2.11it/s, est. speed input: 69033.24 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2119/3051 [16:45<07:22,  2.11it/s, est. speed input: 69033.26 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2120/3051 [16:46<07:21,  2.11it/s, est. speed input: 69033.39 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2121/3051 [16:46<07:20,  2.11it/s, est. speed input: 69033.52 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2122/3051 [16:47<07:19,  2.11it/s, est. speed input: 69033.64 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2123/3051 [16:47<07:19,  2.11it/s, est. speed input: 69033.65 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2124/3051 [16:48<07:19,  2.11it/s, est. speed input: 69033.69 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2125/3051 [16:48<07:18,  2.11it/s, est. speed input: 69033.72 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2126/3051 [16:49<07:18,  2.11it/s, est. speed input: 69033.84 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2127/3051 [16:49<07:17,  2.11it/s, est. speed input: 69033.89 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2128/3051 [16:50<07:17,  2.11it/s, est. speed input: 69034.00 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2129/3051 [16:50<07:16,  2.11it/s, est. speed input: 69034.12 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2130/3051 [16:51<07:15,  2.11it/s, est. speed input: 69034.20 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2131/3051 [16:51<07:15,  2.11it/s, est. speed input: 69034.31 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2132/3051 [16:51<07:16,  2.11it/s, est. speed input: 69034.05 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2133/3051 [16:52<07:15,  2.11it/s, est. speed input: 69034.16 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2134/3051 [16:52<07:14,  2.11it/s, est. speed input: 69034.24 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2135/3051 [16:53<07:14,  2.11it/s, est. speed input: 69034.25 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2136/3051 [16:53<07:13,  2.11it/s, est. speed input: 69034.35 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2137/3051 [16:54<07:12,  2.11it/s, est. speed input: 69034.46 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2138/3051 [16:54<07:12,  2.11it/s, est. speed input: 69034.40 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2139/3051 [16:55<07:12,  2.11it/s, est. speed input: 69034.52 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2140/3051 [16:55<07:11,  2.11it/s, est. speed input: 69034.62 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2141/3051 [16:56<07:12,  2.10it/s, est. speed input: 69034.32 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2142/3051 [16:56<07:11,  2.11it/s, est. speed input: 69034.34 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2143/3051 [16:57<07:10,  2.11it/s, est. speed input: 69034.45 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2144/3051 [16:57<07:09,  2.11it/s, est. speed input: 69034.55 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2145/3051 [16:58<07:09,  2.11it/s, est. speed input: 69034.65 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2146/3051 [16:58<07:08,  2.11it/s, est. speed input: 69034.71 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2147/3051 [16:59<07:09,  2.10it/s, est. speed input: 69034.38 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2148/3051 [16:59<07:08,  2.11it/s, est. speed input: 69034.45 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2149/3051 [17:00<07:07,  2.11it/s, est. speed input: 69034.59 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2150/3051 [17:00<07:07,  2.11it/s, est. speed input: 69034.61 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2151/3051 [17:00<07:06,  2.11it/s, est. speed input: 69034.72 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2152/3051 [17:01<07:05,  2.11it/s, est. speed input: 69034.84 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2153/3051 [17:01<07:06,  2.10it/s, est. speed input: 69034.57 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2154/3051 [17:02<07:06,  2.11it/s, est. speed input: 69034.57 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2155/3051 [17:02<07:05,  2.11it/s, est. speed input: 69034.66 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2156/3051 [17:03<07:04,  2.11it/s, est. speed input: 69034.78 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2157/3051 [17:03<07:03,  2.11it/s, est. speed input: 69034.91 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2158/3051 [17:04<07:02,  2.11it/s, est. speed input: 69034.98 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2159/3051 [17:04<07:26,  2.00it/s, est. speed input: 69028.94 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2160/3051 [17:05<07:18,  2.03it/s, est. speed input: 69029.09 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2161/3051 [17:05<07:12,  2.06it/s, est. speed input: 69029.23 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2162/3051 [17:06<07:08,  2.07it/s, est. speed input: 69029.37 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2163/3051 [17:06<07:05,  2.09it/s, est. speed input: 69029.46 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2164/3051 [17:07<07:03,  2.09it/s, est. speed input: 69029.61 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2165/3051 [17:07<07:01,  2.10it/s, est. speed input: 69029.74 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2166/3051 [17:08<07:00,  2.10it/s, est. speed input: 69029.84 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2167/3051 [17:08<06:59,  2.11it/s, est. speed input: 69029.95 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2168/3051 [17:09<06:58,  2.11it/s, est. speed input: 69030.03 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2169/3051 [17:09<06:58,  2.11it/s, est. speed input: 69030.11 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2170/3051 [17:10<06:57,  2.11it/s, est. speed input: 69030.20 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2171/3051 [17:10<06:56,  2.11it/s, est. speed input: 69030.30 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2172/3051 [17:11<06:56,  2.11it/s, est. speed input: 69030.44 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2173/3051 [17:11<06:55,  2.11it/s, est. speed input: 69030.56 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2174/3051 [17:11<06:54,  2.11it/s, est. speed input: 69030.66 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2175/3051 [17:12<06:54,  2.11it/s, est. speed input: 69030.75 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2176/3051 [17:12<06:53,  2.11it/s, est. speed input: 69030.90 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2177/3051 [17:13<06:53,  2.11it/s, est. speed input: 69031.03 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2178/3051 [17:13<06:52,  2.12it/s, est. speed input: 69031.18 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2179/3051 [17:14<06:52,  2.12it/s, est. speed input: 69031.32 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2180/3051 [17:14<06:51,  2.12it/s, est. speed input: 69031.45 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2181/3051 [17:15<06:51,  2.12it/s, est. speed input: 69031.58 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2182/3051 [17:15<06:50,  2.12it/s, est. speed input: 69031.75 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2183/3051 [17:16<06:50,  2.12it/s, est. speed input: 69031.88 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2184/3051 [17:16<06:49,  2.12it/s, est. speed input: 69032.02 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2185/3051 [17:17<06:49,  2.12it/s, est. speed input: 69032.15 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2186/3051 [17:17<06:49,  2.11it/s, est. speed input: 69032.22 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2187/3051 [17:18<06:48,  2.11it/s, est. speed input: 69032.33 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2188/3051 [17:18<06:48,  2.11it/s, est. speed input: 69032.42 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2189/3051 [17:19<06:48,  2.11it/s, est. speed input: 69032.44 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2190/3051 [17:19<06:47,  2.11it/s, est. speed input: 69032.57 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2191/3051 [17:20<06:46,  2.11it/s, est. speed input: 69032.69 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2192/3051 [17:20<06:46,  2.11it/s, est. speed input: 69032.78 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2193/3051 [17:20<06:46,  2.11it/s, est. speed input: 69032.86 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2194/3051 [17:21<06:45,  2.11it/s, est. speed input: 69033.00 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2195/3051 [17:21<06:45,  2.11it/s, est. speed input: 69033.07 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2196/3051 [17:22<06:44,  2.11it/s, est. speed input: 69033.20 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2197/3051 [17:22<06:44,  2.11it/s, est. speed input: 69033.30 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2198/3051 [17:23<06:43,  2.11it/s, est. speed input: 69033.43 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2199/3051 [17:23<06:42,  2.12it/s, est. speed input: 69033.58 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2200/3051 [17:24<06:42,  2.12it/s, est. speed input: 69033.72 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2201/3051 [17:24<06:41,  2.12it/s, est. speed input: 69033.88 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2202/3051 [17:25<06:41,  2.12it/s, est. speed input: 69034.02 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2203/3051 [17:25<06:40,  2.12it/s, est. speed input: 69034.11 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2204/3051 [17:26<06:41,  2.11it/s, est. speed input: 69034.05 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2205/3051 [17:26<06:40,  2.11it/s, est. speed input: 69034.17 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2206/3051 [17:27<06:43,  2.09it/s, est. speed input: 69033.37 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2207/3051 [17:27<06:41,  2.10it/s, est. speed input: 69033.47 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2208/3051 [17:28<06:40,  2.11it/s, est. speed input: 69033.63 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2209/3051 [17:28<06:39,  2.11it/s, est. speed input: 69033.71 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2210/3051 [17:29<06:39,  2.11it/s, est. speed input: 69033.71 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2211/3051 [17:29<06:38,  2.11it/s, est. speed input: 69033.80 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2212/3051 [17:29<06:37,  2.11it/s, est. speed input: 69033.96 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2213/3051 [17:30<06:36,  2.11it/s, est. speed input: 69034.07 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2214/3051 [17:30<06:36,  2.11it/s, est. speed input: 69034.21 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2215/3051 [17:31<06:35,  2.11it/s, est. speed input: 69034.33 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2216/3051 [17:31<06:34,  2.11it/s, est. speed input: 69034.46 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2217/3051 [17:32<06:34,  2.11it/s, est. speed input: 69034.57 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2218/3051 [17:32<06:33,  2.11it/s, est. speed input: 69034.69 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2219/3051 [17:33<06:33,  2.11it/s, est. speed input: 69034.79 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2220/3051 [17:33<06:32,  2.11it/s, est. speed input: 69034.92 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2221/3051 [17:34<06:32,  2.11it/s, est. speed input: 69035.01 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2222/3051 [17:34<06:32,  2.11it/s, est. speed input: 69035.15 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2223/3051 [17:35<06:31,  2.12it/s, est. speed input: 69035.28 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2224/3051 [17:35<06:30,  2.12it/s, est. speed input: 69035.42 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2225/3051 [17:36<06:30,  2.11it/s, est. speed input: 69035.49 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2226/3051 [17:36<06:30,  2.11it/s, est. speed input: 69035.54 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2227/3051 [17:37<06:29,  2.11it/s, est. speed input: 69035.62 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2228/3051 [17:37<06:29,  2.11it/s, est. speed input: 69035.75 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2229/3051 [17:37<06:28,  2.11it/s, est. speed input: 69035.87 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2230/3051 [17:38<06:28,  2.11it/s, est. speed input: 69036.02 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2231/3051 [17:38<06:27,  2.11it/s, est. speed input: 69036.11 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2232/3051 [17:39<06:27,  2.11it/s, est. speed input: 69036.24 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2233/3051 [17:39<06:26,  2.11it/s, est. speed input: 69036.35 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2234/3051 [17:40<06:26,  2.11it/s, est. speed input: 69036.44 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2235/3051 [17:40<06:25,  2.11it/s, est. speed input: 69036.55 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2236/3051 [17:41<06:25,  2.11it/s, est. speed input: 69036.64 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2237/3051 [17:41<06:24,  2.11it/s, est. speed input: 69036.77 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2238/3051 [17:42<06:24,  2.11it/s, est. speed input: 69036.85 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2239/3051 [17:42<06:24,  2.11it/s, est. speed input: 69036.99 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2240/3051 [17:43<06:23,  2.11it/s, est. speed input: 69037.11 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2241/3051 [17:43<06:22,  2.12it/s, est. speed input: 69037.26 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2242/3051 [17:44<06:22,  2.11it/s, est. speed input: 69037.36 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2243/3051 [17:44<06:22,  2.11it/s, est. speed input: 69037.36 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2244/3051 [17:45<06:22,  2.11it/s, est. speed input: 69037.44 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2245/3051 [17:45<06:21,  2.11it/s, est. speed input: 69037.57 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2246/3051 [17:46<06:20,  2.11it/s, est. speed input: 69037.69 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2247/3051 [17:46<06:20,  2.11it/s, est. speed input: 69037.76 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2248/3051 [17:46<06:19,  2.11it/s, est. speed input: 69037.90 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2249/3051 [17:47<06:19,  2.11it/s, est. speed input: 69038.01 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2250/3051 [17:47<06:18,  2.12it/s, est. speed input: 69038.16 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2251/3051 [17:48<06:18,  2.12it/s, est. speed input: 69038.30 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2252/3051 [17:48<06:17,  2.11it/s, est. speed input: 69038.36 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2253/3051 [17:49<06:17,  2.11it/s, est. speed input: 69038.43 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2254/3051 [17:49<06:16,  2.11it/s, est. speed input: 69038.57 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2255/3051 [17:50<06:16,  2.11it/s, est. speed input: 69038.66 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2256/3051 [17:50<06:16,  2.11it/s, est. speed input: 69038.77 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2257/3051 [17:51<06:15,  2.11it/s, est. speed input: 69038.87 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2258/3051 [17:51<06:15,  2.11it/s, est. speed input: 69038.98 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2259/3051 [17:52<06:14,  2.11it/s, est. speed input: 69039.09 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2260/3051 [17:52<06:14,  2.11it/s, est. speed input: 69039.19 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2261/3051 [17:53<06:13,  2.11it/s, est. speed input: 69039.22 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2262/3051 [17:53<06:13,  2.11it/s, est. speed input: 69039.33 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2263/3051 [17:54<06:13,  2.11it/s, est. speed input: 69039.37 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2264/3051 [17:54<06:14,  2.10it/s, est. speed input: 69038.91 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2265/3051 [17:55<06:13,  2.11it/s, est. speed input: 69039.06 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2266/3051 [17:55<06:12,  2.11it/s, est. speed input: 69039.19 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2267/3051 [17:55<06:11,  2.11it/s, est. speed input: 69039.29 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2268/3051 [17:56<06:11,  2.11it/s, est. speed input: 69039.32 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2269/3051 [17:56<06:10,  2.11it/s, est. speed input: 69039.38 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2270/3051 [17:57<06:09,  2.11it/s, est. speed input: 69039.51 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2271/3051 [17:57<06:09,  2.11it/s, est. speed input: 69039.66 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2272/3051 [17:58<06:08,  2.11it/s, est. speed input: 69039.78 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2273/3051 [17:58<06:08,  2.11it/s, est. speed input: 69039.80 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2274/3051 [17:59<06:08,  2.11it/s, est. speed input: 69039.81 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2275/3051 [17:59<06:07,  2.11it/s, est. speed input: 69039.91 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2276/3051 [18:00<06:06,  2.11it/s, est. speed input: 69040.01 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2277/3051 [18:00<06:06,  2.11it/s, est. speed input: 69040.14 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2278/3051 [18:01<06:05,  2.11it/s, est. speed input: 69040.17 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2279/3051 [18:01<06:05,  2.11it/s, est. speed input: 69040.26 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2280/3051 [18:02<06:04,  2.11it/s, est. speed input: 69040.43 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2281/3051 [18:02<06:04,  2.11it/s, est. speed input: 69040.53 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2282/3051 [18:03<06:03,  2.11it/s, est. speed input: 69040.65 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2283/3051 [18:03<06:03,  2.11it/s, est. speed input: 69040.75 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2284/3051 [18:04<06:02,  2.12it/s, est. speed input: 69040.89 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2285/3051 [18:04<06:02,  2.11it/s, est. speed input: 69041.00 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2286/3051 [18:04<06:01,  2.11it/s, est. speed input: 69041.08 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2287/3051 [18:05<06:01,  2.12it/s, est. speed input: 69041.22 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2288/3051 [18:05<06:00,  2.12it/s, est. speed input: 69041.36 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2289/3051 [18:06<06:00,  2.12it/s, est. speed input: 69041.49 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2290/3051 [18:06<06:00,  2.11it/s, est. speed input: 69041.52 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2291/3051 [18:07<05:59,  2.11it/s, est. speed input: 69041.61 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2292/3051 [18:07<05:58,  2.11it/s, est. speed input: 69041.74 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2293/3051 [18:08<05:58,  2.11it/s, est. speed input: 69041.86 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2294/3051 [18:08<05:58,  2.11it/s, est. speed input: 69041.92 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2295/3051 [18:09<05:57,  2.11it/s, est. speed input: 69041.99 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2296/3051 [18:09<06:13,  2.02it/s, est. speed input: 69037.54 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2297/3051 [18:10<06:07,  2.05it/s, est. speed input: 69037.70 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2298/3051 [18:10<06:04,  2.07it/s, est. speed input: 69037.79 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2299/3051 [18:11<06:01,  2.08it/s, est. speed input: 69037.92 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2300/3051 [18:11<05:58,  2.09it/s, est. speed input: 69038.05 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2301/3051 [18:12<05:57,  2.10it/s, est. speed input: 69038.18 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2302/3051 [18:12<05:56,  2.10it/s, est. speed input: 69038.27 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2303/3051 [18:13<05:55,  2.10it/s, est. speed input: 69038.28 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2304/3051 [18:13<05:54,  2.11it/s, est. speed input: 69038.40 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2305/3051 [18:14<05:53,  2.11it/s, est. speed input: 69038.53 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2306/3051 [18:14<05:52,  2.11it/s, est. speed input: 69038.64 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2307/3051 [18:14<05:52,  2.11it/s, est. speed input: 69038.73 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2308/3051 [18:15<05:51,  2.11it/s, est. speed input: 69038.89 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2309/3051 [18:15<05:51,  2.11it/s, est. speed input: 69038.97 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2310/3051 [18:16<05:50,  2.11it/s, est. speed input: 69038.99 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2311/3051 [18:16<05:50,  2.11it/s, est. speed input: 69039.12 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2312/3051 [18:17<05:49,  2.11it/s, est. speed input: 69039.21 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2313/3051 [18:17<05:49,  2.11it/s, est. speed input: 69039.30 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2314/3051 [18:18<05:48,  2.11it/s, est. speed input: 69039.40 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2315/3051 [18:18<05:48,  2.11it/s, est. speed input: 69039.32 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2316/3051 [18:19<05:48,  2.11it/s, est. speed input: 69039.37 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2317/3051 [18:19<05:47,  2.11it/s, est. speed input: 69039.47 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2318/3051 [18:20<05:47,  2.11it/s, est. speed input: 69039.59 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2319/3051 [18:20<05:46,  2.11it/s, est. speed input: 69039.71 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2320/3051 [18:21<05:45,  2.11it/s, est. speed input: 69039.80 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2321/3051 [18:21<05:45,  2.11it/s, est. speed input: 69039.84 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2322/3051 [18:22<05:45,  2.11it/s, est. speed input: 69039.94 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2323/3051 [18:22<05:44,  2.11it/s, est. speed input: 69040.05 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2324/3051 [18:23<05:43,  2.11it/s, est. speed input: 69040.16 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2325/3051 [18:23<05:43,  2.11it/s, est. speed input: 69040.27 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2326/3051 [18:23<05:42,  2.11it/s, est. speed input: 69040.38 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2327/3051 [18:24<05:42,  2.11it/s, est. speed input: 69040.47 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2328/3051 [18:24<05:41,  2.12it/s, est. speed input: 69040.61 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2329/3051 [18:25<05:41,  2.12it/s, est. speed input: 69040.75 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2330/3051 [18:25<05:40,  2.12it/s, est. speed input: 69040.87 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2331/3051 [18:26<05:40,  2.11it/s, est. speed input: 69040.89 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2332/3051 [18:26<05:40,  2.11it/s, est. speed input: 69040.96 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2333/3051 [18:27<05:39,  2.11it/s, est. speed input: 69041.08 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2334/3051 [18:27<05:39,  2.11it/s, est. speed input: 69041.17 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2335/3051 [18:28<05:39,  2.11it/s, est. speed input: 69041.17 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2336/3051 [18:28<05:38,  2.11it/s, est. speed input: 69041.16 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2337/3051 [18:29<05:38,  2.11it/s, est. speed input: 69041.27 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2338/3051 [18:29<05:37,  2.11it/s, est. speed input: 69041.33 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2339/3051 [18:30<05:37,  2.11it/s, est. speed input: 69041.41 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2340/3051 [18:30<05:36,  2.11it/s, est. speed input: 69041.50 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2341/3051 [18:31<05:35,  2.11it/s, est. speed input: 69041.62 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2342/3051 [18:31<05:35,  2.11it/s, est. speed input: 69041.72 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2343/3051 [18:32<05:34,  2.11it/s, est. speed input: 69041.85 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2344/3051 [18:32<05:34,  2.11it/s, est. speed input: 69041.88 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2345/3051 [18:32<05:34,  2.11it/s, est. speed input: 69041.99 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2346/3051 [18:33<05:33,  2.11it/s, est. speed input: 69042.13 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2347/3051 [18:33<05:33,  2.11it/s, est. speed input: 69042.09 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2348/3051 [18:34<05:32,  2.11it/s, est. speed input: 69042.18 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2349/3051 [18:34<05:32,  2.11it/s, est. speed input: 69042.29 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2350/3051 [18:35<05:31,  2.11it/s, est. speed input: 69042.45 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2351/3051 [18:35<05:31,  2.11it/s, est. speed input: 69042.53 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2352/3051 [18:36<05:30,  2.11it/s, est. speed input: 69042.62 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2353/3051 [18:36<05:30,  2.11it/s, est. speed input: 69042.72 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2354/3051 [18:37<05:29,  2.11it/s, est. speed input: 69042.85 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2355/3051 [18:37<05:29,  2.11it/s, est. speed input: 69042.95 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2356/3051 [18:38<05:28,  2.11it/s, est. speed input: 69043.06 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2357/3051 [18:38<05:29,  2.11it/s, est. speed input: 69042.92 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2358/3051 [18:39<05:28,  2.11it/s, est. speed input: 69042.99 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2359/3051 [18:39<05:27,  2.11it/s, est. speed input: 69043.11 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2360/3051 [18:40<05:26,  2.11it/s, est. speed input: 69043.25 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2361/3051 [18:40<05:26,  2.11it/s, est. speed input: 69043.37 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2362/3051 [18:41<05:25,  2.12it/s, est. speed input: 69043.51 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2363/3051 [18:41<05:25,  2.12it/s, est. speed input: 69043.63 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2364/3051 [18:41<05:25,  2.11it/s, est. speed input: 69043.63 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2365/3051 [18:42<05:24,  2.11it/s, est. speed input: 69043.70 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2366/3051 [18:42<05:24,  2.11it/s, est. speed input: 69043.85 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2367/3051 [18:43<05:23,  2.11it/s, est. speed input: 69043.89 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2368/3051 [18:43<05:23,  2.11it/s, est. speed input: 69043.99 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2369/3051 [18:44<05:22,  2.11it/s, est. speed input: 69044.10 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2370/3051 [18:44<05:22,  2.11it/s, est. speed input: 69044.08 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2371/3051 [18:45<05:21,  2.11it/s, est. speed input: 69044.20 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2372/3051 [18:45<05:21,  2.11it/s, est. speed input: 69044.31 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2373/3051 [18:46<05:20,  2.11it/s, est. speed input: 69044.38 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2374/3051 [18:46<05:20,  2.11it/s, est. speed input: 69044.51 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2375/3051 [18:47<05:19,  2.11it/s, est. speed input: 69044.63 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2376/3051 [18:47<05:19,  2.11it/s, est. speed input: 69044.73 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2377/3051 [18:48<05:18,  2.12it/s, est. speed input: 69044.86 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2378/3051 [18:48<05:18,  2.11it/s, est. speed input: 69044.93 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2379/3051 [18:49<05:18,  2.11it/s, est. speed input: 69044.92 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2380/3051 [18:49<05:17,  2.11it/s, est. speed input: 69045.00 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2381/3051 [18:49<05:16,  2.11it/s, est. speed input: 69045.14 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2382/3051 [18:50<05:16,  2.11it/s, est. speed input: 69045.27 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2383/3051 [18:50<05:16,  2.11it/s, est. speed input: 69045.22 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2384/3051 [18:51<05:15,  2.11it/s, est. speed input: 69045.31 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2385/3051 [18:51<05:15,  2.11it/s, est. speed input: 69045.43 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2386/3051 [18:52<05:14,  2.11it/s, est. speed input: 69045.52 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2387/3051 [18:52<05:14,  2.11it/s, est. speed input: 69045.64 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2388/3051 [18:53<05:13,  2.11it/s, est. speed input: 69045.73 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2389/3051 [18:53<05:12,  2.12it/s, est. speed input: 69045.88 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2390/3051 [18:54<05:12,  2.11it/s, est. speed input: 69045.96 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2391/3051 [18:54<05:12,  2.11it/s, est. speed input: 69046.07 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2392/3051 [18:55<05:11,  2.11it/s, est. speed input: 69046.16 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2393/3051 [18:55<05:11,  2.11it/s, est. speed input: 69046.28 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2394/3051 [18:56<05:10,  2.11it/s, est. speed input: 69046.39 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2395/3051 [18:56<05:10,  2.11it/s, est. speed input: 69046.46 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2396/3051 [18:57<05:09,  2.11it/s, est. speed input: 69046.55 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2397/3051 [18:57<05:09,  2.11it/s, est. speed input: 69046.67 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2398/3051 [18:58<05:08,  2.11it/s, est. speed input: 69046.79 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2399/3051 [18:58<05:08,  2.11it/s, est. speed input: 69046.87 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2400/3051 [18:58<05:08,  2.11it/s, est. speed input: 69046.85 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2401/3051 [18:59<05:07,  2.11it/s, est. speed input: 69046.93 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2402/3051 [18:59<05:07,  2.11it/s, est. speed input: 69047.05 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2403/3051 [19:00<05:06,  2.11it/s, est. speed input: 69047.12 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2404/3051 [19:00<05:06,  2.11it/s, est. speed input: 69047.21 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2405/3051 [19:01<05:05,  2.11it/s, est. speed input: 69047.33 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2406/3051 [19:01<05:05,  2.11it/s, est. speed input: 69047.36 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2407/3051 [19:02<05:04,  2.11it/s, est. speed input: 69047.41 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2408/3051 [19:02<05:04,  2.11it/s, est. speed input: 69047.51 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2409/3051 [19:03<05:03,  2.11it/s, est. speed input: 69047.64 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2410/3051 [19:03<05:03,  2.11it/s, est. speed input: 69047.75 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2411/3051 [19:04<05:09,  2.07it/s, est. speed input: 69045.58 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2412/3051 [19:04<05:07,  2.08it/s, est. speed input: 69045.74 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2413/3051 [19:05<05:04,  2.09it/s, est. speed input: 69045.89 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2414/3051 [19:05<05:03,  2.10it/s, est. speed input: 69046.02 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2415/3051 [19:06<05:02,  2.10it/s, est. speed input: 69046.11 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2416/3051 [19:06<05:01,  2.11it/s, est. speed input: 69046.22 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2417/3051 [19:07<05:00,  2.11it/s, est. speed input: 69046.32 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2418/3051 [19:07<05:01,  2.10it/s, est. speed input: 69046.05 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2419/3051 [19:08<05:00,  2.11it/s, est. speed input: 69046.15 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2420/3051 [19:08<04:59,  2.11it/s, est. speed input: 69046.22 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2421/3051 [19:08<04:58,  2.11it/s, est. speed input: 69046.23 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2422/3051 [19:09<04:58,  2.11it/s, est. speed input: 69046.21 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2423/3051 [19:09<04:57,  2.11it/s, est. speed input: 69046.29 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2424/3051 [19:10<04:57,  2.11it/s, est. speed input: 69046.37 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2425/3051 [19:10<04:56,  2.11it/s, est. speed input: 69046.43 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2426/3051 [19:11<04:56,  2.11it/s, est. speed input: 69046.50 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2427/3051 [19:11<04:55,  2.11it/s, est. speed input: 69046.60 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2428/3051 [19:12<04:54,  2.11it/s, est. speed input: 69046.70 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2429/3051 [19:12<04:54,  2.11it/s, est. speed input: 69046.82 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2430/3051 [19:13<04:54,  2.11it/s, est. speed input: 69046.81 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2431/3051 [19:13<04:53,  2.11it/s, est. speed input: 69046.85 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2432/3051 [19:14<04:52,  2.11it/s, est. speed input: 69047.00 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2433/3051 [19:14<04:52,  2.11it/s, est. speed input: 69047.07 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2434/3051 [19:15<04:51,  2.11it/s, est. speed input: 69047.18 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2435/3051 [19:15<04:51,  2.11it/s, est. speed input: 69047.27 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2436/3051 [19:16<04:50,  2.11it/s, est. speed input: 69047.39 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2437/3051 [19:16<04:50,  2.11it/s, est. speed input: 69047.50 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2438/3051 [19:17<04:50,  2.11it/s, est. speed input: 69047.54 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2439/3051 [19:17<04:49,  2.11it/s, est. speed input: 69047.65 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2440/3051 [19:17<04:49,  2.11it/s, est. speed input: 69047.58 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2441/3051 [19:18<04:48,  2.11it/s, est. speed input: 69047.70 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2442/3051 [19:18<04:48,  2.11it/s, est. speed input: 69047.75 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2443/3051 [19:19<04:47,  2.11it/s, est. speed input: 69047.80 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2444/3051 [19:19<04:47,  2.11it/s, est. speed input: 69047.92 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2445/3051 [19:20<04:46,  2.11it/s, est. speed input: 69048.00 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2446/3051 [19:20<04:46,  2.11it/s, est. speed input: 69047.97 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2447/3051 [19:21<04:46,  2.11it/s, est. speed input: 69048.04 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2448/3051 [19:21<04:45,  2.11it/s, est. speed input: 69048.15 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2449/3051 [19:22<04:45,  2.11it/s, est. speed input: 69048.05 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2450/3051 [19:22<04:45,  2.11it/s, est. speed input: 69048.06 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2451/3051 [19:23<04:44,  2.11it/s, est. speed input: 69048.14 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2452/3051 [19:23<04:43,  2.11it/s, est. speed input: 69048.27 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2453/3051 [19:24<04:43,  2.11it/s, est. speed input: 69048.35 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2454/3051 [19:24<04:42,  2.11it/s, est. speed input: 69048.46 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2455/3051 [19:25<04:41,  2.11it/s, est. speed input: 69048.57 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2456/3051 [19:25<04:41,  2.11it/s, est. speed input: 69048.65 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2457/3051 [19:26<04:41,  2.11it/s, est. speed input: 69048.70 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2458/3051 [19:26<04:40,  2.11it/s, est. speed input: 69048.79 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2459/3051 [19:26<04:39,  2.11it/s, est. speed input: 69048.93 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2460/3051 [19:27<04:39,  2.12it/s, est. speed input: 69049.06 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2461/3051 [19:27<04:38,  2.12it/s, est. speed input: 69049.16 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2462/3051 [19:28<04:38,  2.11it/s, est. speed input: 69049.15 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2463/3051 [19:28<04:38,  2.11it/s, est. speed input: 69049.22 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2464/3051 [19:29<04:38,  2.11it/s, est. speed input: 69049.22 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2465/3051 [19:29<04:37,  2.11it/s, est. speed input: 69049.33 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2466/3051 [19:30<04:36,  2.11it/s, est. speed input: 69049.43 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2467/3051 [19:30<04:36,  2.11it/s, est. speed input: 69049.54 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2468/3051 [19:31<04:35,  2.11it/s, est. speed input: 69049.62 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2469/3051 [19:31<04:35,  2.11it/s, est. speed input: 69049.65 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2470/3051 [19:32<04:35,  2.11it/s, est. speed input: 69049.67 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2471/3051 [19:32<04:34,  2.11it/s, est. speed input: 69049.79 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2472/3051 [19:33<04:34,  2.11it/s, est. speed input: 69049.78 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2473/3051 [19:33<04:33,  2.11it/s, est. speed input: 69049.88 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2474/3051 [19:34<04:33,  2.11it/s, est. speed input: 69049.97 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2475/3051 [19:34<04:32,  2.11it/s, est. speed input: 69050.11 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2476/3051 [19:34<04:31,  2.11it/s, est. speed input: 69050.21 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2477/3051 [19:35<04:31,  2.12it/s, est. speed input: 69050.34 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2478/3051 [19:35<04:30,  2.11it/s, est. speed input: 69050.42 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2479/3051 [19:36<04:30,  2.12it/s, est. speed input: 69050.54 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2480/3051 [19:36<04:29,  2.12it/s, est. speed input: 69050.66 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2481/3051 [19:37<04:29,  2.12it/s, est. speed input: 69050.75 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2482/3051 [19:37<04:29,  2.11it/s, est. speed input: 69050.74 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2483/3051 [19:38<04:28,  2.11it/s, est. speed input: 69050.84 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2484/3051 [19:38<04:28,  2.11it/s, est. speed input: 69050.87 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2485/3051 [19:39<04:27,  2.11it/s, est. speed input: 69050.96 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2486/3051 [19:39<04:27,  2.11it/s, est. speed input: 69051.07 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2487/3051 [19:40<04:26,  2.11it/s, est. speed input: 69051.17 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2488/3051 [19:40<04:26,  2.11it/s, est. speed input: 69051.28 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2489/3051 [19:41<04:25,  2.11it/s, est. speed input: 69051.35 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2490/3051 [19:41<04:25,  2.11it/s, est. speed input: 69051.44 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2491/3051 [19:42<04:24,  2.11it/s, est. speed input: 69051.53 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2492/3051 [19:42<04:24,  2.11it/s, est. speed input: 69051.63 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2493/3051 [19:43<04:23,  2.11it/s, est. speed input: 69051.74 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2494/3051 [19:43<04:23,  2.12it/s, est. speed input: 69051.85 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2495/3051 [19:43<04:23,  2.11it/s, est. speed input: 69051.85 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2496/3051 [19:44<04:22,  2.11it/s, est. speed input: 69051.96 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2497/3051 [19:44<04:22,  2.11it/s, est. speed input: 69052.05 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2498/3051 [19:45<04:22,  2.11it/s, est. speed input: 69051.84 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2499/3051 [19:45<04:21,  2.11it/s, est. speed input: 69051.93 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2500/3051 [19:46<04:21,  2.11it/s, est. speed input: 69052.01 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2501/3051 [19:46<04:20,  2.11it/s, est. speed input: 69052.12 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2502/3051 [19:47<04:19,  2.11it/s, est. speed input: 69052.23 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2503/3051 [19:47<04:19,  2.11it/s, est. speed input: 69052.29 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2504/3051 [19:48<04:18,  2.11it/s, est. speed input: 69052.39 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2505/3051 [19:48<04:18,  2.11it/s, est. speed input: 69052.44 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2506/3051 [19:49<04:17,  2.11it/s, est. speed input: 69052.52 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2507/3051 [19:49<04:17,  2.11it/s, est. speed input: 69052.58 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2508/3051 [19:50<04:17,  2.11it/s, est. speed input: 69052.55 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2509/3051 [19:50<04:16,  2.11it/s, est. speed input: 69052.61 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2510/3051 [19:51<04:16,  2.11it/s, est. speed input: 69052.74 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2511/3051 [19:51<04:15,  2.11it/s, est. speed input: 69052.75 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2512/3051 [19:52<04:15,  2.11it/s, est. speed input: 69052.82 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2513/3051 [19:52<04:14,  2.11it/s, est. speed input: 69052.93 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2514/3051 [19:52<04:14,  2.11it/s, est. speed input: 69052.72 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2515/3051 [19:53<04:14,  2.11it/s, est. speed input: 69052.73 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2516/3051 [19:53<04:13,  2.11it/s, est. speed input: 69052.87 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2517/3051 [19:54<04:13,  2.11it/s, est. speed input: 69052.93 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2518/3051 [19:54<04:12,  2.11it/s, est. speed input: 69053.05 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2519/3051 [19:55<04:11,  2.11it/s, est. speed input: 69053.15 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2520/3051 [19:55<04:11,  2.11it/s, est. speed input: 69053.24 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2521/3051 [19:56<04:10,  2.11it/s, est. speed input: 69053.34 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2522/3051 [19:56<04:10,  2.11it/s, est. speed input: 69053.45 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2523/3051 [19:57<04:09,  2.11it/s, est. speed input: 69053.56 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2524/3051 [19:57<04:09,  2.11it/s, est. speed input: 69053.65 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2525/3051 [19:58<04:08,  2.11it/s, est. speed input: 69053.72 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2526/3051 [19:58<04:08,  2.11it/s, est. speed input: 69053.78 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2527/3051 [19:59<04:07,  2.11it/s, est. speed input: 69053.85 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2528/3051 [19:59<04:07,  2.11it/s, est. speed input: 69053.91 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2529/3051 [20:00<04:06,  2.11it/s, est. speed input: 69054.02 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2530/3051 [20:00<04:07,  2.11it/s, est. speed input: 69053.90 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2531/3051 [20:01<04:06,  2.11it/s, est. speed input: 69053.94 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2532/3051 [20:01<04:05,  2.11it/s, est. speed input: 69054.04 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2533/3051 [20:01<04:05,  2.11it/s, est. speed input: 69054.12 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2534/3051 [20:02<04:04,  2.11it/s, est. speed input: 69054.20 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2535/3051 [20:02<04:04,  2.11it/s, est. speed input: 69054.29 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2536/3051 [20:03<04:03,  2.11it/s, est. speed input: 69054.38 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2537/3051 [20:03<04:03,  2.11it/s, est. speed input: 69054.48 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2538/3051 [20:04<04:02,  2.11it/s, est. speed input: 69054.58 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2539/3051 [20:04<04:02,  2.12it/s, est. speed input: 69054.71 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2540/3051 [20:05<04:01,  2.11it/s, est. speed input: 69054.77 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2541/3051 [20:05<04:01,  2.11it/s, est. speed input: 69054.89 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2542/3051 [20:06<04:00,  2.11it/s, est. speed input: 69054.95 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2543/3051 [20:06<04:00,  2.11it/s, est. speed input: 69055.03 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2544/3051 [20:07<03:59,  2.11it/s, est. speed input: 69055.13 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2545/3051 [20:07<03:59,  2.12it/s, est. speed input: 69055.26 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2546/3051 [20:08<03:58,  2.11it/s, est. speed input: 69055.34 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2547/3051 [20:08<03:58,  2.11it/s, est. speed input: 69055.34 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2548/3051 [20:09<03:58,  2.11it/s, est. speed input: 69055.31 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2549/3051 [20:09<03:57,  2.11it/s, est. speed input: 69055.40 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2550/3051 [20:10<03:57,  2.11it/s, est. speed input: 69055.51 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2551/3051 [20:10<03:56,  2.11it/s, est. speed input: 69055.53 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2552/3051 [20:10<03:56,  2.11it/s, est. speed input: 69055.63 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2553/3051 [20:11<03:55,  2.11it/s, est. speed input: 69055.66 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2554/3051 [20:11<03:55,  2.11it/s, est. speed input: 69055.65 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2555/3051 [20:12<03:54,  2.11it/s, est. speed input: 69055.72 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2556/3051 [20:12<03:54,  2.11it/s, est. speed input: 69055.86 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2557/3051 [20:13<03:53,  2.11it/s, est. speed input: 69055.91 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2558/3051 [20:13<03:53,  2.11it/s, est. speed input: 69056.01 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2559/3051 [20:14<03:52,  2.11it/s, est. speed input: 69056.03 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2560/3051 [20:14<03:52,  2.11it/s, est. speed input: 69056.13 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2561/3051 [20:15<03:51,  2.11it/s, est. speed input: 69056.21 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2562/3051 [20:15<03:52,  2.11it/s, est. speed input: 69056.04 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2563/3051 [20:16<03:51,  2.11it/s, est. speed input: 69056.11 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2564/3051 [20:16<03:50,  2.11it/s, est. speed input: 69056.21 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2565/3051 [20:17<03:50,  2.11it/s, est. speed input: 69056.30 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2566/3051 [20:17<03:49,  2.11it/s, est. speed input: 69056.37 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2567/3051 [20:18<03:49,  2.11it/s, est. speed input: 69056.48 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2568/3051 [20:18<03:48,  2.11it/s, est. speed input: 69056.54 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2569/3051 [20:19<03:48,  2.11it/s, est. speed input: 69056.53 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2570/3051 [20:19<03:47,  2.11it/s, est. speed input: 69056.57 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2571/3051 [20:19<03:47,  2.11it/s, est. speed input: 69056.64 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2572/3051 [20:20<03:46,  2.11it/s, est. speed input: 69056.74 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2573/3051 [20:20<03:46,  2.11it/s, est. speed input: 69056.83 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2574/3051 [20:21<03:45,  2.11it/s, est. speed input: 69056.92 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2575/3051 [20:21<03:45,  2.11it/s, est. speed input: 69057.00 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2576/3051 [20:22<03:44,  2.11it/s, est. speed input: 69057.13 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2577/3051 [20:22<03:44,  2.11it/s, est. speed input: 69057.20 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2578/3051 [20:23<03:43,  2.11it/s, est. speed input: 69057.29 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2579/3051 [20:23<03:43,  2.11it/s, est. speed input: 69057.39 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2580/3051 [20:24<03:42,  2.11it/s, est. speed input: 69057.46 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2581/3051 [20:24<03:42,  2.11it/s, est. speed input: 69057.57 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2582/3051 [20:25<03:41,  2.11it/s, est. speed input: 69057.63 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2583/3051 [20:25<03:41,  2.11it/s, est. speed input: 69057.70 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2584/3051 [20:26<03:40,  2.11it/s, est. speed input: 69057.78 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2585/3051 [20:26<03:40,  2.12it/s, est. speed input: 69057.92 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2586/3051 [20:27<03:39,  2.12it/s, est. speed input: 69058.02 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2587/3051 [20:27<03:39,  2.11it/s, est. speed input: 69058.10 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2588/3051 [20:28<03:39,  2.11it/s, est. speed input: 69058.13 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2589/3051 [20:28<03:38,  2.11it/s, est. speed input: 69058.23 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2590/3051 [20:28<03:38,  2.11it/s, est. speed input: 69058.29 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2591/3051 [20:29<03:37,  2.11it/s, est. speed input: 69058.36 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2592/3051 [20:29<03:37,  2.11it/s, est. speed input: 69058.48 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2593/3051 [20:30<03:36,  2.11it/s, est. speed input: 69058.53 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2594/3051 [20:30<03:36,  2.11it/s, est. speed input: 69058.62 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2595/3051 [20:31<03:35,  2.11it/s, est. speed input: 69058.72 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2596/3051 [20:31<03:35,  2.11it/s, est. speed input: 69058.79 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2597/3051 [20:32<03:34,  2.11it/s, est. speed input: 69058.86 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2598/3051 [20:32<03:34,  2.11it/s, est. speed input: 69058.84 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2599/3051 [20:33<03:34,  2.11it/s, est. speed input: 69058.90 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2600/3051 [20:33<03:33,  2.11it/s, est. speed input: 69058.99 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2601/3051 [20:34<03:33,  2.11it/s, est. speed input: 69059.05 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2602/3051 [20:34<03:32,  2.11it/s, est. speed input: 69059.13 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2603/3051 [20:35<03:32,  2.11it/s, est. speed input: 69059.19 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2604/3051 [20:35<03:31,  2.11it/s, est. speed input: 69059.32 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2605/3051 [20:36<03:30,  2.11it/s, est. speed input: 69059.40 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2606/3051 [20:36<03:30,  2.11it/s, est. speed input: 69059.50 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2607/3051 [20:36<03:29,  2.11it/s, est. speed input: 69059.59 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2608/3051 [20:37<03:30,  2.11it/s, est. speed input: 69059.44 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2609/3051 [20:37<03:29,  2.11it/s, est. speed input: 69059.48 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2610/3051 [20:38<03:28,  2.11it/s, est. speed input: 69059.55 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2611/3051 [20:38<03:28,  2.11it/s, est. speed input: 69059.56 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2612/3051 [20:39<03:27,  2.11it/s, est. speed input: 69059.63 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2613/3051 [20:39<03:27,  2.11it/s, est. speed input: 69059.71 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2614/3051 [20:40<03:26,  2.11it/s, est. speed input: 69059.79 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2615/3051 [20:40<03:26,  2.11it/s, est. speed input: 69059.85 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2616/3051 [20:41<03:25,  2.11it/s, est. speed input: 69059.94 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2617/3051 [20:41<03:25,  2.11it/s, est. speed input: 69059.94 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2618/3051 [20:42<03:24,  2.11it/s, est. speed input: 69060.05 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2619/3051 [20:42<03:24,  2.11it/s, est. speed input: 69060.11 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2620/3051 [20:43<03:24,  2.11it/s, est. speed input: 69060.12 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2621/3051 [20:43<03:23,  2.11it/s, est. speed input: 69060.19 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2622/3051 [20:44<03:22,  2.11it/s, est. speed input: 69060.33 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2623/3051 [20:44<03:22,  2.11it/s, est. speed input: 69060.44 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2624/3051 [20:45<03:21,  2.11it/s, est. speed input: 69060.50 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2625/3051 [20:45<03:21,  2.11it/s, est. speed input: 69060.60 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2626/3051 [20:45<03:20,  2.11it/s, est. speed input: 69060.69 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2627/3051 [20:46<03:20,  2.11it/s, est. speed input: 69060.78 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2628/3051 [20:46<03:19,  2.12it/s, est. speed input: 69060.91 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2629/3051 [20:47<03:19,  2.12it/s, est. speed input: 69061.02 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2630/3051 [20:47<03:19,  2.11it/s, est. speed input: 69061.07 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2631/3051 [20:48<03:18,  2.11it/s, est. speed input: 69061.15 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2632/3051 [20:48<03:18,  2.11it/s, est. speed input: 69061.24 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2633/3051 [20:49<03:17,  2.11it/s, est. speed input: 69061.32 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2634/3051 [20:49<03:17,  2.11it/s, est. speed input: 69061.42 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2635/3051 [20:50<03:16,  2.11it/s, est. speed input: 69061.51 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2636/3051 [20:50<03:16,  2.11it/s, est. speed input: 69061.57 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2637/3051 [20:51<03:15,  2.11it/s, est. speed input: 69061.58 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2638/3051 [20:51<03:15,  2.11it/s, est. speed input: 69061.67 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2639/3051 [20:52<03:14,  2.11it/s, est. speed input: 69061.75 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2640/3051 [20:52<03:14,  2.11it/s, est. speed input: 69061.82 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2641/3051 [20:53<03:13,  2.11it/s, est. speed input: 69061.89 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2642/3051 [20:53<03:13,  2.11it/s, est. speed input: 69061.96 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2643/3051 [20:54<03:13,  2.11it/s, est. speed input: 69062.05 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2644/3051 [20:54<03:12,  2.11it/s, est. speed input: 69062.15 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2645/3051 [20:54<03:14,  2.08it/s, est. speed input: 69060.99 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2646/3051 [20:55<03:13,  2.09it/s, est. speed input: 69061.10 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2647/3051 [20:55<03:12,  2.10it/s, est. speed input: 69061.20 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2648/3051 [20:56<03:12,  2.10it/s, est. speed input: 69061.03 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2649/3051 [20:56<03:11,  2.10it/s, est. speed input: 69061.02 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2650/3051 [20:57<03:10,  2.11it/s, est. speed input: 69061.11 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2651/3051 [20:57<03:09,  2.11it/s, est. speed input: 69061.19 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2652/3051 [20:58<03:09,  2.11it/s, est. speed input: 69061.18 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2653/3051 [20:58<03:08,  2.11it/s, est. speed input: 69061.19 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2654/3051 [20:59<03:08,  2.11it/s, est. speed input: 69061.25 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2655/3051 [20:59<03:07,  2.11it/s, est. speed input: 69061.36 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2656/3051 [21:00<03:07,  2.11it/s, est. speed input: 69061.44 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2657/3051 [21:00<03:06,  2.11it/s, est. speed input: 69061.50 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2658/3051 [21:01<03:06,  2.11it/s, est. speed input: 69061.59 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2659/3051 [21:01<03:05,  2.11it/s, est. speed input: 69061.62 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2660/3051 [21:02<03:05,  2.11it/s, est. speed input: 69061.72 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2661/3051 [21:02<03:04,  2.11it/s, est. speed input: 69061.81 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2662/3051 [21:03<03:04,  2.11it/s, est. speed input: 69061.64 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2663/3051 [21:03<03:03,  2.11it/s, est. speed input: 69061.72 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2664/3051 [21:03<03:03,  2.11it/s, est. speed input: 69061.83 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2665/3051 [21:04<03:02,  2.11it/s, est. speed input: 69061.84 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2666/3051 [21:04<03:02,  2.11it/s, est. speed input: 69061.97 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2667/3051 [21:05<03:01,  2.11it/s, est. speed input: 69062.04 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2668/3051 [21:05<03:01,  2.11it/s, est. speed input: 69062.13 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2669/3051 [21:06<03:00,  2.11it/s, est. speed input: 69062.23 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2670/3051 [21:06<03:00,  2.11it/s, est. speed input: 69062.32 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2671/3051 [21:07<02:59,  2.11it/s, est. speed input: 69062.42 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2672/3051 [21:07<02:59,  2.12it/s, est. speed input: 69062.52 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2673/3051 [21:08<02:58,  2.11it/s, est. speed input: 69062.60 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2674/3051 [21:08<02:58,  2.11it/s, est. speed input: 69062.56 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2675/3051 [21:09<02:58,  2.11it/s, est. speed input: 69062.51 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2676/3051 [21:09<02:57,  2.11it/s, est. speed input: 69062.63 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2677/3051 [21:10<02:57,  2.11it/s, est. speed input: 69062.71 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2678/3051 [21:10<02:56,  2.11it/s, est. speed input: 69062.78 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2679/3051 [21:11<02:56,  2.11it/s, est. speed input: 69062.87 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2680/3051 [21:11<02:55,  2.11it/s, est. speed input: 69062.96 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2681/3051 [21:12<02:55,  2.11it/s, est. speed input: 69063.05 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2682/3051 [21:12<02:54,  2.11it/s, est. speed input: 69063.14 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2683/3051 [21:12<02:54,  2.11it/s, est. speed input: 69063.23 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2684/3051 [21:13<02:53,  2.11it/s, est. speed input: 69063.31 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2685/3051 [21:13<02:53,  2.11it/s, est. speed input: 69063.41 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2686/3051 [21:14<02:52,  2.11it/s, est. speed input: 69063.49 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2687/3051 [21:14<02:52,  2.11it/s, est. speed input: 69063.59 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2688/3051 [21:15<02:51,  2.12it/s, est. speed input: 69063.68 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2689/3051 [21:15<02:51,  2.12it/s, est. speed input: 69063.77 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2690/3051 [21:16<02:50,  2.11it/s, est. speed input: 69063.72 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2691/3051 [21:16<02:50,  2.11it/s, est. speed input: 69063.77 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2692/3051 [21:17<02:49,  2.11it/s, est. speed input: 69063.84 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2693/3051 [21:17<02:49,  2.11it/s, est. speed input: 69063.93 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2694/3051 [21:18<02:48,  2.11it/s, est. speed input: 69064.03 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2695/3051 [21:18<02:48,  2.11it/s, est. speed input: 69064.08 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2696/3051 [21:19<02:48,  2.11it/s, est. speed input: 69064.13 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2697/3051 [21:19<02:47,  2.11it/s, est. speed input: 69064.15 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2698/3051 [21:20<02:47,  2.11it/s, est. speed input: 69064.18 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2699/3051 [21:20<02:46,  2.11it/s, est. speed input: 69064.25 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2700/3051 [21:21<02:46,  2.11it/s, est. speed input: 69064.31 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2701/3051 [21:21<02:45,  2.11it/s, est. speed input: 69064.22 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2702/3051 [21:21<02:45,  2.11it/s, est. speed input: 69064.31 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2703/3051 [21:22<02:44,  2.11it/s, est. speed input: 69064.39 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2704/3051 [21:22<02:44,  2.11it/s, est. speed input: 69064.38 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2705/3051 [21:23<02:43,  2.11it/s, est. speed input: 69064.46 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2706/3051 [21:23<02:43,  2.11it/s, est. speed input: 69064.54 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2707/3051 [21:24<02:43,  2.11it/s, est. speed input: 69064.36 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2708/3051 [21:24<02:42,  2.11it/s, est. speed input: 69064.44 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2709/3051 [21:25<02:42,  2.11it/s, est. speed input: 69064.54 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2710/3051 [21:25<02:41,  2.11it/s, est. speed input: 69064.63 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2711/3051 [21:26<02:40,  2.11it/s, est. speed input: 69064.71 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2712/3051 [21:26<02:40,  2.11it/s, est. speed input: 69064.81 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2713/3051 [21:27<02:39,  2.11it/s, est. speed input: 69064.89 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2714/3051 [21:27<02:39,  2.11it/s, est. speed input: 69064.99 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2715/3051 [21:28<02:38,  2.12it/s, est. speed input: 69065.10 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2716/3051 [21:28<02:38,  2.11it/s, est. speed input: 69065.13 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2717/3051 [21:29<02:38,  2.11it/s, est. speed input: 69065.06 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2718/3051 [21:29<02:37,  2.11it/s, est. speed input: 69065.13 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2719/3051 [21:30<02:37,  2.11it/s, est. speed input: 69065.22 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2720/3051 [21:30<02:36,  2.11it/s, est. speed input: 69065.30 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2721/3051 [21:30<02:36,  2.11it/s, est. speed input: 69065.38 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2722/3051 [21:31<02:35,  2.11it/s, est. speed input: 69065.49 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2723/3051 [21:31<02:35,  2.11it/s, est. speed input: 69065.56 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2724/3051 [21:32<02:34,  2.11it/s, est. speed input: 69065.65 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2725/3051 [21:32<02:34,  2.11it/s, est. speed input: 69065.75 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2726/3051 [21:33<02:33,  2.12it/s, est. speed input: 69065.84 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2727/3051 [21:33<02:33,  2.12it/s, est. speed input: 69065.94 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2728/3051 [21:34<02:32,  2.11it/s, est. speed input: 69065.99 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2729/3051 [21:34<02:32,  2.11it/s, est. speed input: 69066.08 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2730/3051 [21:35<02:31,  2.11it/s, est. speed input: 69066.16 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2731/3051 [21:35<02:31,  2.12it/s, est. speed input: 69066.27 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2732/3051 [21:36<02:30,  2.12it/s, est. speed input: 69066.36 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2733/3051 [21:36<02:30,  2.12it/s, est. speed input: 69066.45 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2734/3051 [21:37<02:29,  2.12it/s, est. speed input: 69066.54 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2735/3051 [21:37<02:30,  2.10it/s, est. speed input: 69065.91 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2736/3051 [21:38<02:29,  2.10it/s, est. speed input: 69066.00 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2737/3051 [21:38<02:29,  2.11it/s, est. speed input: 69066.05 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2738/3051 [21:39<02:28,  2.11it/s, est. speed input: 69066.01 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2739/3051 [21:39<02:28,  2.11it/s, est. speed input: 69066.04 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2740/3051 [21:39<02:27,  2.11it/s, est. speed input: 69066.13 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2741/3051 [21:40<02:26,  2.11it/s, est. speed input: 69066.26 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2742/3051 [21:40<02:26,  2.11it/s, est. speed input: 69066.33 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2743/3051 [21:41<02:25,  2.11it/s, est. speed input: 69066.42 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2744/3051 [21:41<02:25,  2.11it/s, est. speed input: 69066.51 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2745/3051 [21:42<02:24,  2.11it/s, est. speed input: 69066.61 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2746/3051 [21:42<02:24,  2.11it/s, est. speed input: 69066.63 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2747/3051 [21:43<02:23,  2.11it/s, est. speed input: 69066.67 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2748/3051 [21:43<02:23,  2.11it/s, est. speed input: 69066.76 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2749/3051 [21:44<02:22,  2.11it/s, est. speed input: 69066.85 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2750/3051 [21:44<02:22,  2.11it/s, est. speed input: 69066.95 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2751/3051 [21:45<02:21,  2.11it/s, est. speed input: 69067.02 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2752/3051 [21:45<02:21,  2.11it/s, est. speed input: 69067.06 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2753/3051 [21:46<02:20,  2.11it/s, est. speed input: 69067.17 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2754/3051 [21:46<02:20,  2.11it/s, est. speed input: 69067.24 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2755/3051 [21:47<02:20,  2.11it/s, est. speed input: 69067.31 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2756/3051 [21:47<02:19,  2.11it/s, est. speed input: 69067.38 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2757/3051 [21:48<02:19,  2.11it/s, est. speed input: 69067.46 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2758/3051 [21:48<02:18,  2.11it/s, est. speed input: 69067.50 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2759/3051 [21:48<02:18,  2.11it/s, est. speed input: 69067.57 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2760/3051 [21:49<02:17,  2.11it/s, est. speed input: 69067.59 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2761/3051 [21:49<02:17,  2.11it/s, est. speed input: 69067.66 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2762/3051 [21:50<02:16,  2.11it/s, est. speed input: 69067.75 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2763/3051 [21:50<02:16,  2.11it/s, est. speed input: 69067.79 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2764/3051 [21:51<02:15,  2.11it/s, est. speed input: 69067.82 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2765/3051 [21:51<02:15,  2.11it/s, est. speed input: 69067.94 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2766/3051 [21:52<02:14,  2.11it/s, est. speed input: 69067.97 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2767/3051 [21:52<02:14,  2.11it/s, est. speed input: 69068.07 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2768/3051 [21:53<02:13,  2.11it/s, est. speed input: 69068.09 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2769/3051 [21:53<02:13,  2.11it/s, est. speed input: 69068.18 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2770/3051 [21:54<02:12,  2.11it/s, est. speed input: 69068.23 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2771/3051 [21:54<02:12,  2.11it/s, est. speed input: 69068.33 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2772/3051 [21:55<02:12,  2.11it/s, est. speed input: 69068.37 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2773/3051 [21:55<02:11,  2.11it/s, est. speed input: 69068.47 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2774/3051 [21:56<02:11,  2.11it/s, est. speed input: 69068.55 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2775/3051 [21:56<02:10,  2.11it/s, est. speed input: 69068.55 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2776/3051 [21:57<02:10,  2.11it/s, est. speed input: 69068.62 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2777/3051 [21:57<02:09,  2.11it/s, est. speed input: 69068.73 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2778/3051 [21:57<02:09,  2.11it/s, est. speed input: 69068.80 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2779/3051 [21:58<02:08,  2.11it/s, est. speed input: 69068.88 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2780/3051 [21:58<02:08,  2.11it/s, est. speed input: 69068.84 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2781/3051 [21:59<02:07,  2.11it/s, est. speed input: 69068.90 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2782/3051 [21:59<02:07,  2.11it/s, est. speed input: 69068.97 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2783/3051 [22:00<02:07,  2.11it/s, est. speed input: 69068.74 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2784/3051 [22:00<02:06,  2.10it/s, est. speed input: 69068.70 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2785/3051 [22:01<02:06,  2.11it/s, est. speed input: 69068.81 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2786/3051 [22:01<02:05,  2.11it/s, est. speed input: 69068.88 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2787/3051 [22:02<02:05,  2.11it/s, est. speed input: 69068.91 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2788/3051 [22:02<02:04,  2.11it/s, est. speed input: 69068.97 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2789/3051 [22:03<02:04,  2.11it/s, est. speed input: 69069.05 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2790/3051 [22:03<02:03,  2.11it/s, est. speed input: 69069.15 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2791/3051 [22:04<02:02,  2.11it/s, est. speed input: 69069.25 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2792/3051 [22:04<02:02,  2.11it/s, est. speed input: 69069.33 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2793/3051 [22:05<02:02,  2.11it/s, est. speed input: 69069.41 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2794/3051 [22:05<02:01,  2.11it/s, est. speed input: 69069.48 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2795/3051 [22:06<02:01,  2.12it/s, est. speed input: 69069.60 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2796/3051 [22:06<02:00,  2.11it/s, est. speed input: 69069.58 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2797/3051 [22:06<02:00,  2.11it/s, est. speed input: 69069.64 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2798/3051 [22:07<01:59,  2.11it/s, est. speed input: 69069.68 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2799/3051 [22:07<01:59,  2.11it/s, est. speed input: 69069.77 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2800/3051 [22:08<01:58,  2.11it/s, est. speed input: 69069.86 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2801/3051 [22:08<01:58,  2.11it/s, est. speed input: 69069.90 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2802/3051 [22:09<01:57,  2.11it/s, est. speed input: 69069.93 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2803/3051 [22:09<01:57,  2.11it/s, est. speed input: 69070.01 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2804/3051 [22:10<01:56,  2.11it/s, est. speed input: 69070.10 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2805/3051 [22:10<01:56,  2.11it/s, est. speed input: 69070.18 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2806/3051 [22:11<01:55,  2.11it/s, est. speed input: 69070.25 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2807/3051 [22:11<01:55,  2.11it/s, est. speed input: 69070.10 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2808/3051 [22:12<01:55,  2.11it/s, est. speed input: 69070.21 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2809/3051 [22:12<01:54,  2.11it/s, est. speed input: 69070.25 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2810/3051 [22:13<01:54,  2.11it/s, est. speed input: 69070.30 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2811/3051 [22:13<01:53,  2.11it/s, est. speed input: 69070.34 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2812/3051 [22:14<01:53,  2.11it/s, est. speed input: 69070.43 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2813/3051 [22:14<01:52,  2.11it/s, est. speed input: 69070.50 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2814/3051 [22:14<01:52,  2.11it/s, est. speed input: 69070.58 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2815/3051 [22:15<01:53,  2.09it/s, est. speed input: 69069.56 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2816/3051 [22:15<01:52,  2.09it/s, est. speed input: 69069.64 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2817/3051 [22:16<01:51,  2.10it/s, est. speed input: 69069.71 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2818/3051 [22:16<01:50,  2.11it/s, est. speed input: 69069.82 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2819/3051 [22:17<01:50,  2.11it/s, est. speed input: 69069.91 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2820/3051 [22:17<01:49,  2.11it/s, est. speed input: 69070.01 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2821/3051 [22:18<01:48,  2.11it/s, est. speed input: 69070.09 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2822/3051 [22:18<01:48,  2.11it/s, est. speed input: 69070.11 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2823/3051 [22:19<01:48,  2.11it/s, est. speed input: 69070.10 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2824/3051 [22:19<01:47,  2.11it/s, est. speed input: 69070.20 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2825/3051 [22:20<01:46,  2.11it/s, est. speed input: 69070.28 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2826/3051 [22:20<01:46,  2.11it/s, est. speed input: 69070.25 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2827/3051 [22:21<01:46,  2.11it/s, est. speed input: 69070.36 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2828/3051 [22:21<01:45,  2.11it/s, est. speed input: 69070.15 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2829/3051 [22:22<01:45,  2.11it/s, est. speed input: 69070.17 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2830/3051 [22:22<01:44,  2.11it/s, est. speed input: 69070.22 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2831/3051 [22:23<01:44,  2.11it/s, est. speed input: 69070.29 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2832/3051 [22:23<01:43,  2.11it/s, est. speed input: 69070.39 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2833/3051 [22:24<01:43,  2.11it/s, est. speed input: 69070.42 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2834/3051 [22:24<01:42,  2.11it/s, est. speed input: 69070.51 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2835/3051 [22:24<01:42,  2.11it/s, est. speed input: 69070.62 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2836/3051 [22:25<01:41,  2.11it/s, est. speed input: 69070.69 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2837/3051 [22:25<01:43,  2.07it/s, est. speed input: 69068.99 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2838/3051 [22:26<01:42,  2.08it/s, est. speed input: 69069.11 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2839/3051 [22:26<01:41,  2.09it/s, est. speed input: 69069.21 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2840/3051 [22:27<01:40,  2.10it/s, est. speed input: 69069.31 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2841/3051 [22:27<01:39,  2.10it/s, est. speed input: 69069.37 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2842/3051 [22:28<01:39,  2.11it/s, est. speed input: 69069.47 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2843/3051 [22:28<01:38,  2.11it/s, est. speed input: 69069.49 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2844/3051 [22:29<01:38,  2.11it/s, est. speed input: 69069.42 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2845/3051 [22:29<01:37,  2.11it/s, est. speed input: 69069.48 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2846/3051 [22:30<01:37,  2.11it/s, est. speed input: 69069.56 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2847/3051 [22:30<01:36,  2.11it/s, est. speed input: 69069.63 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2848/3051 [22:31<01:36,  2.11it/s, est. speed input: 69069.72 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2849/3051 [22:31<01:35,  2.11it/s, est. speed input: 69069.80 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2850/3051 [22:32<01:35,  2.11it/s, est. speed input: 69069.89 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2851/3051 [22:32<01:34,  2.11it/s, est. speed input: 69069.96 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2852/3051 [22:33<01:34,  2.11it/s, est. speed input: 69070.05 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2853/3051 [22:33<01:33,  2.11it/s, est. speed input: 69070.09 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2854/3051 [22:33<01:33,  2.11it/s, est. speed input: 69070.17 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2855/3051 [22:34<01:32,  2.11it/s, est. speed input: 69070.26 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2856/3051 [22:34<01:32,  2.11it/s, est. speed input: 69070.35 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2857/3051 [22:35<01:31,  2.12it/s, est. speed input: 69070.44 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2858/3051 [22:35<01:31,  2.12it/s, est. speed input: 69070.55 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2859/3051 [22:36<01:30,  2.12it/s, est. speed input: 69070.66 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2860/3051 [22:36<01:30,  2.12it/s, est. speed input: 69070.73 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2861/3051 [22:37<01:29,  2.12it/s, est. speed input: 69070.83 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2862/3051 [22:37<01:29,  2.11it/s, est. speed input: 69070.88 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2863/3051 [22:38<01:28,  2.12it/s, est. speed input: 69070.97 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2864/3051 [22:38<01:28,  2.11it/s, est. speed input: 69070.85 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2865/3051 [22:39<01:28,  2.11it/s, est. speed input: 69070.89 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2866/3051 [22:39<01:27,  2.11it/s, est. speed input: 69070.99 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2867/3051 [22:40<01:27,  2.11it/s, est. speed input: 69071.11 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2868/3051 [22:40<01:26,  2.11it/s, est. speed input: 69071.17 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2869/3051 [22:41<01:26,  2.11it/s, est. speed input: 69071.26 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2870/3051 [22:41<01:25,  2.11it/s, est. speed input: 69071.34 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2871/3051 [22:42<01:25,  2.12it/s, est. speed input: 69071.48 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2872/3051 [22:42<01:24,  2.12it/s, est. speed input: 69071.56 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2873/3051 [22:42<01:24,  2.12it/s, est. speed input: 69071.64 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2874/3051 [22:43<01:23,  2.12it/s, est. speed input: 69071.73 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2875/3051 [22:43<01:23,  2.12it/s, est. speed input: 69071.82 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2876/3051 [22:44<01:22,  2.12it/s, est. speed input: 69071.89 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2877/3051 [22:44<01:22,  2.11it/s, est. speed input: 69071.96 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2878/3051 [22:45<01:21,  2.11it/s, est. speed input: 69072.04 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2879/3051 [22:45<01:21,  2.11it/s, est. speed input: 69072.05 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2880/3051 [22:46<01:20,  2.11it/s, est. speed input: 69072.12 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2881/3051 [22:46<01:20,  2.11it/s, est. speed input: 69072.20 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2882/3051 [22:47<01:19,  2.11it/s, est. speed input: 69072.28 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2883/3051 [22:47<01:19,  2.12it/s, est. speed input: 69072.39 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2884/3051 [22:48<01:18,  2.12it/s, est. speed input: 69072.48 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2885/3051 [22:48<01:18,  2.12it/s, est. speed input: 69072.55 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2886/3051 [22:49<01:18,  2.11it/s, est. speed input: 69072.57 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2887/3051 [22:49<01:17,  2.11it/s, est. speed input: 69072.63 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2888/3051 [22:50<01:17,  2.11it/s, est. speed input: 69072.72 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2889/3051 [22:50<01:16,  2.11it/s, est. speed input: 69072.81 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2890/3051 [22:51<01:16,  2.11it/s, est. speed input: 69072.87 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2891/3051 [22:51<01:15,  2.11it/s, est. speed input: 69072.96 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2892/3051 [22:51<01:15,  2.11it/s, est. speed input: 69073.03 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2893/3051 [22:52<01:14,  2.11it/s, est. speed input: 69073.11 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2894/3051 [22:52<01:14,  2.12it/s, est. speed input: 69073.21 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2895/3051 [22:53<01:13,  2.12it/s, est. speed input: 69073.33 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2896/3051 [22:53<01:13,  2.12it/s, est. speed input: 69073.42 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2897/3051 [22:54<01:12,  2.12it/s, est. speed input: 69073.49 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2898/3051 [22:54<01:12,  2.12it/s, est. speed input: 69073.56 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2899/3051 [22:55<01:11,  2.11it/s, est. speed input: 69073.62 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2900/3051 [22:55<01:11,  2.12it/s, est. speed input: 69073.72 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2901/3051 [22:56<01:10,  2.12it/s, est. speed input: 69073.79 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2902/3051 [22:56<01:10,  2.12it/s, est. speed input: 69073.87 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2903/3051 [22:57<01:09,  2.12it/s, est. speed input: 69073.96 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2904/3051 [22:57<01:09,  2.12it/s, est. speed input: 69074.04 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2905/3051 [22:58<01:09,  2.12it/s, est. speed input: 69074.12 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2906/3051 [22:58<01:08,  2.11it/s, est. speed input: 69074.16 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2907/3051 [22:59<01:08,  2.11it/s, est. speed input: 69074.14 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2908/3051 [22:59<01:07,  2.11it/s, est. speed input: 69074.21 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2909/3051 [22:59<01:07,  2.11it/s, est. speed input: 69074.25 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2910/3051 [23:00<01:06,  2.11it/s, est. speed input: 69074.35 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2911/3051 [23:00<01:06,  2.11it/s, est. speed input: 69074.43 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2912/3051 [23:01<01:05,  2.11it/s, est. speed input: 69074.53 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2913/3051 [23:01<01:05,  2.11it/s, est. speed input: 69074.59 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2914/3051 [23:02<01:04,  2.12it/s, est. speed input: 69074.73 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2915/3051 [23:02<01:04,  2.12it/s, est. speed input: 69074.79 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2916/3051 [23:03<01:03,  2.12it/s, est. speed input: 69074.88 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2917/3051 [23:03<01:03,  2.12it/s, est. speed input: 69074.95 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2918/3051 [23:04<01:02,  2.12it/s, est. speed input: 69075.03 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2919/3051 [23:04<01:02,  2.11it/s, est. speed input: 69075.10 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2920/3051 [23:05<01:01,  2.12it/s, est. speed input: 69075.18 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2921/3051 [23:05<01:01,  2.12it/s, est. speed input: 69075.27 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2922/3051 [23:06<01:00,  2.12it/s, est. speed input: 69075.36 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2923/3051 [23:06<01:00,  2.12it/s, est. speed input: 69075.43 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2924/3051 [23:07<01:00,  2.11it/s, est. speed input: 69075.49 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2925/3051 [23:07<00:59,  2.11it/s, est. speed input: 69075.57 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2926/3051 [23:08<00:59,  2.11it/s, est. speed input: 69075.64 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2927/3051 [23:08<00:58,  2.11it/s, est. speed input: 69075.70 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2928/3051 [23:08<00:58,  2.11it/s, est. speed input: 69075.74 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2929/3051 [23:09<00:57,  2.11it/s, est. speed input: 69075.80 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2930/3051 [23:09<00:57,  2.11it/s, est. speed input: 69075.89 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2931/3051 [23:10<00:56,  2.11it/s, est. speed input: 69075.96 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2932/3051 [23:10<00:56,  2.11it/s, est. speed input: 69076.04 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2933/3051 [23:11<00:55,  2.12it/s, est. speed input: 69076.13 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2934/3051 [23:11<00:55,  2.11it/s, est. speed input: 69076.11 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2935/3051 [23:12<00:54,  2.11it/s, est. speed input: 69076.16 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2936/3051 [23:12<00:54,  2.11it/s, est. speed input: 69076.23 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2937/3051 [23:13<00:53,  2.11it/s, est. speed input: 69076.30 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2938/3051 [23:13<00:53,  2.11it/s, est. speed input: 69076.39 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2939/3051 [23:14<00:52,  2.11it/s, est. speed input: 69076.47 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2940/3051 [23:14<00:52,  2.11it/s, est. speed input: 69076.55 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2941/3051 [23:15<00:52,  2.11it/s, est. speed input: 69076.59 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2942/3051 [23:15<00:51,  2.11it/s, est. speed input: 69076.69 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2943/3051 [23:16<00:51,  2.11it/s, est. speed input: 69076.73 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2944/3051 [23:16<00:50,  2.11it/s, est. speed input: 69076.81 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2945/3051 [23:17<00:50,  2.11it/s, est. speed input: 69076.86 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2946/3051 [23:17<00:49,  2.11it/s, est. speed input: 69076.96 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2947/3051 [23:17<00:49,  2.11it/s, est. speed input: 69077.02 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2948/3051 [23:18<00:48,  2.11it/s, est. speed input: 69077.08 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2949/3051 [23:18<00:48,  2.11it/s, est. speed input: 69077.15 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2950/3051 [23:19<00:47,  2.11it/s, est. speed input: 69077.21 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2951/3051 [23:19<00:47,  2.11it/s, est. speed input: 69077.26 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2952/3051 [23:20<00:46,  2.11it/s, est. speed input: 69077.33 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2953/3051 [23:20<00:46,  2.11it/s, est. speed input: 69077.40 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2954/3051 [23:21<00:45,  2.11it/s, est. speed input: 69077.50 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2955/3051 [23:21<00:45,  2.11it/s, est. speed input: 69077.57 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2956/3051 [23:22<00:44,  2.11it/s, est. speed input: 69077.63 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2957/3051 [23:22<00:44,  2.11it/s, est. speed input: 69077.64 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2958/3051 [23:23<00:44,  2.11it/s, est. speed input: 69077.72 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2959/3051 [23:23<00:43,  2.11it/s, est. speed input: 69077.80 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2960/3051 [23:24<00:43,  2.11it/s, est. speed input: 69077.85 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2961/3051 [23:24<00:42,  2.11it/s, est. speed input: 69077.93 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2962/3051 [23:25<00:42,  2.12it/s, est. speed input: 69078.04 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2963/3051 [23:25<00:41,  2.11it/s, est. speed input: 69078.08 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2964/3051 [23:26<00:41,  2.11it/s, est. speed input: 69078.14 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2965/3051 [23:26<00:40,  2.11it/s, est. speed input: 69078.23 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2966/3051 [23:26<00:40,  2.12it/s, est. speed input: 69078.34 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2967/3051 [23:27<00:39,  2.12it/s, est. speed input: 69078.41 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2968/3051 [23:27<00:39,  2.12it/s, est. speed input: 69078.49 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2969/3051 [23:28<00:38,  2.12it/s, est. speed input: 69078.58 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2970/3051 [23:28<00:38,  2.11it/s, est. speed input: 69078.64 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2971/3051 [23:29<00:37,  2.11it/s, est. speed input: 69078.70 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2972/3051 [23:29<00:37,  2.11it/s, est. speed input: 69078.77 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2973/3051 [23:30<00:36,  2.11it/s, est. speed input: 69078.73 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2974/3051 [23:30<00:36,  2.11it/s, est. speed input: 69078.81 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2975/3051 [23:31<00:35,  2.11it/s, est. speed input: 69078.87 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2976/3051 [23:31<00:35,  2.11it/s, est. speed input: 69078.95 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2977/3051 [23:32<00:34,  2.11it/s, est. speed input: 69079.05 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2978/3051 [23:32<00:34,  2.12it/s, est. speed input: 69079.17 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2979/3051 [23:33<00:34,  2.12it/s, est. speed input: 69079.27 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2980/3051 [23:33<00:33,  2.12it/s, est. speed input: 69079.34 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2981/3051 [23:34<00:33,  2.12it/s, est. speed input: 69079.43 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2982/3051 [23:34<00:32,  2.12it/s, est. speed input: 69079.50 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2983/3051 [23:34<00:32,  2.12it/s, est. speed input: 69079.59 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2984/3051 [23:35<00:31,  2.12it/s, est. speed input: 69079.65 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2985/3051 [23:35<00:31,  2.12it/s, est. speed input: 69079.74 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2986/3051 [23:36<00:30,  2.12it/s, est. speed input: 69079.82 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2987/3051 [23:36<00:30,  2.11it/s, est. speed input: 69079.86 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2988/3051 [23:37<00:29,  2.11it/s, est. speed input: 69079.93 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2989/3051 [23:37<00:29,  2.12it/s, est. speed input: 69080.03 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2990/3051 [23:38<00:28,  2.12it/s, est. speed input: 69080.12 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2991/3051 [23:38<00:28,  2.11it/s, est. speed input: 69080.18 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2992/3051 [23:39<00:27,  2.11it/s, est. speed input: 69080.22 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2993/3051 [23:39<00:27,  2.11it/s, est. speed input: 69080.29 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2994/3051 [23:40<00:26,  2.11it/s, est. speed input: 69080.35 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2995/3051 [23:40<00:26,  2.11it/s, est. speed input: 69080.43 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2996/3051 [23:41<00:26,  2.11it/s, est. speed input: 69080.50 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2997/3051 [23:41<00:25,  2.11it/s, est. speed input: 69080.58 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2998/3051 [23:42<00:25,  2.12it/s, est. speed input: 69080.68 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2999/3051 [23:42<00:24,  2.11it/s, est. speed input: 69080.68 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3000/3051 [23:43<00:24,  2.11it/s, est. speed input: 69080.75 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3001/3051 [23:43<00:23,  2.11it/s, est. speed input: 69080.80 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3002/3051 [23:43<00:23,  2.11it/s, est. speed input: 69080.89 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3003/3051 [23:44<00:22,  2.11it/s, est. speed input: 69080.96 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3004/3051 [23:44<00:22,  2.11it/s, est. speed input: 69081.03 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3005/3051 [23:45<00:21,  2.11it/s, est. speed input: 69081.11 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3006/3051 [23:45<00:21,  2.11it/s, est. speed input: 69081.18 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3007/3051 [23:46<00:20,  2.11it/s, est. speed input: 69081.24 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3008/3051 [23:46<00:20,  2.12it/s, est. speed input: 69081.35 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3009/3051 [23:47<00:19,  2.11it/s, est. speed input: 69081.31 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3010/3051 [23:47<00:19,  2.11it/s, est. speed input: 69081.37 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3011/3051 [23:48<00:18,  2.11it/s, est. speed input: 69081.43 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3012/3051 [23:48<00:18,  2.11it/s, est. speed input: 69081.49 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3013/3051 [23:49<00:17,  2.11it/s, est. speed input: 69081.54 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3014/3051 [23:49<00:17,  2.11it/s, est. speed input: 69081.60 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3015/3051 [23:50<00:17,  2.11it/s, est. speed input: 69081.47 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3016/3051 [23:50<00:16,  2.11it/s, est. speed input: 69081.50 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3017/3051 [23:51<00:16,  2.11it/s, est. speed input: 69081.60 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3018/3051 [23:51<00:15,  2.11it/s, est. speed input: 69081.69 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3019/3051 [23:52<00:15,  2.11it/s, est. speed input: 69081.71 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3020/3051 [23:52<00:14,  2.11it/s, est. speed input: 69081.78 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3021/3051 [23:52<00:14,  2.11it/s, est. speed input: 69081.85 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3022/3051 [23:53<00:13,  2.11it/s, est. speed input: 69081.92 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3023/3051 [23:53<00:13,  2.11it/s, est. speed input: 69081.98 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3024/3051 [23:54<00:12,  2.11it/s, est. speed input: 69082.06 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3025/3051 [23:54<00:12,  2.11it/s, est. speed input: 69081.97 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3026/3051 [23:55<00:11,  2.11it/s, est. speed input: 69082.06 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3027/3051 [23:55<00:11,  2.11it/s, est. speed input: 69082.11 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3028/3051 [23:56<00:10,  2.11it/s, est. speed input: 69082.11 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3029/3051 [23:56<00:10,  2.11it/s, est. speed input: 69082.18 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3030/3051 [23:57<00:09,  2.11it/s, est. speed input: 69082.23 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3031/3051 [23:57<00:09,  2.11it/s, est. speed input: 69082.19 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3032/3051 [23:58<00:08,  2.11it/s, est. speed input: 69082.30 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3033/3051 [23:58<00:08,  2.11it/s, est. speed input: 69082.34 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3034/3051 [23:59<00:08,  2.11it/s, est. speed input: 69082.37 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3035/3051 [23:59<00:07,  2.11it/s, est. speed input: 69082.43 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3036/3051 [24:00<00:07,  2.11it/s, est. speed input: 69082.51 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3037/3051 [24:00<00:06,  2.11it/s, est. speed input: 69082.54 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3038/3051 [24:01<00:06,  2.11it/s, est. speed input: 69082.64 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3039/3051 [24:01<00:05,  2.11it/s, est. speed input: 69082.72 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3040/3051 [24:01<00:05,  2.12it/s, est. speed input: 69082.83 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3041/3051 [24:02<00:04,  2.12it/s, est. speed input: 69082.92 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3042/3051 [24:02<00:04,  2.12it/s, est. speed input: 69082.97 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3043/3051 [24:03<00:03,  2.11it/s, est. speed input: 69083.01 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3044/3051 [24:03<00:03,  2.11it/s, est. speed input: 69082.99 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3045/3051 [24:04<00:02,  2.11it/s, est. speed input: 69083.07 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3046/3051 [24:04<00:02,  2.11it/s, est. speed input: 69083.15 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3047/3051 [24:05<00:01,  2.11it/s, est. speed input: 69083.24 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3048/3051 [24:05<00:01,  2.11it/s, est. speed input: 69083.29 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3049/3051 [24:06<00:00,  2.11it/s, est. speed input: 69083.38 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3050/3051 [24:06<00:00,  2.12it/s, est. speed input: 69083.49 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 3051/3051 [24:07<00:00,  2.11it/s, est. speed input: 69083.42 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 3051/3051 [24:07<00:00,  2.11it/s, est. speed input: 69083.42 toks/s, output: 2.11 toks/s]
Saved activation counts to new_activations/activation.en.train.q2.5-zh.pt
Processed 99975168 tokens total

Activation analysis complete for model: q2.5-zh!

=== Processing model: /home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_150M_en_750M_chinese_wikipedia_corpus_2e_240925/checkpoint-3494, type: qwen ===
INFO 09-26 13:47:14 [config.py:2700] Downcasting torch.float32 to torch.float16.
INFO 09-26 13:47:14 [config.py:600] This model supports multiple tasks: {'classify', 'embed', 'reward', 'score', 'generate'}. Defaulting to 'generate'.
WARNING 09-26 13:47:14 [cuda.py:96] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
INFO 09-26 13:47:14 [llm_engine.py:242] Initializing a V0 LLM engine (v0.8.3) with config: model='/home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_150M_en_750M_chinese_wikipedia_corpus_2e_240925/checkpoint-3494', speculative_config=None, tokenizer='/home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_150M_en_750M_chinese_wikipedia_corpus_2e_240925/checkpoint-3494', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=32768, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar', reasoning_backend=None), observability_config=ObservabilityConfig(show_hidden_metrics=False, otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=None, served_model_name=/home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_150M_en_750M_chinese_wikipedia_corpus_2e_240925/checkpoint-3494, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=None, chunked_prefill_enabled=False, use_async_output_proc=False, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"splitting_ops":[],"compile_sizes":[],"cudagraph_capture_sizes":[],"max_capture_size":0}, use_cached_outputs=False, 
INFO 09-26 13:47:16 [model_runner.py:1110] Starting to load model /home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_150M_en_750M_chinese_wikipedia_corpus_2e_240925/checkpoint-3494...

Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:04<00:00,  4.35s/it]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:04<00:00,  4.35s/it]

INFO 09-26 13:47:20 [loader.py:447] Loading weights took 4.42 seconds
INFO 09-26 13:47:22 [model_runner.py:1146] Model loading took 0.9229 GiB and 4.476209 seconds
INFO 09-26 13:47:24 [worker.py:267] Memory profiling takes 1.40 seconds
INFO 09-26 13:47:24 [worker.py:267] the current vLLM instance can use total_gpu_memory (79.25GiB) x gpu_memory_utilization (0.90) = 71.33GiB
INFO 09-26 13:47:24 [worker.py:267] model weights take 0.92GiB; non_torch_memory takes 0.00GiB; PyTorch activation peak memory takes 1.43GiB; the rest of the memory reserved for KV Cache is 68.97GiB.
INFO 09-26 13:47:25 [executor_base.py:112] # cuda blocks: 376690, # CPU blocks: 21845
INFO 09-26 13:47:25 [executor_base.py:117] Maximum concurrency for 32768 tokens per request: 183.93x
INFO 09-26 13:47:26 [llm_engine.py:448] init engine (profile, create kv cache, warmup model) took 3.86 seconds
Layers: 24, Intermediate size: 4864, Max length: 32768

Loading IDs from ./ids/q2.5/id.zh.train.qwen2.5-0.5 (lang: zh)...
ID shape: torch.Size([100000000])
Processing 3051 sequences of length 32768
Running inference...

Processed prompts:   0%|          | 0/3051 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts:   0%|          | 1/3051 [00:00<25:13,  2.02it/s, est. speed input: 66052.35 toks/s, output: 2.02 toks/s]
Processed prompts:   0%|          | 2/3051 [00:00<24:21,  2.09it/s, est. speed input: 67998.12 toks/s, output: 2.08 toks/s]
Processed prompts:   0%|          | 3/3051 [00:01<24:05,  2.11it/s, est. speed input: 68659.88 toks/s, output: 2.10 toks/s]
Processed prompts:   0%|          | 4/3051 [00:02<26:46,  1.90it/s, est. speed input: 64247.79 toks/s, output: 1.96 toks/s]
Processed prompts:   0%|          | 5/3051 [00:02<25:39,  1.98it/s, est. speed input: 65342.34 toks/s, output: 1.99 toks/s]
Processed prompts:   0%|          | 6/3051 [00:02<25:00,  2.03it/s, est. speed input: 66075.75 toks/s, output: 2.02 toks/s]
Processed prompts:   0%|          | 7/3051 [00:03<24:34,  2.06it/s, est. speed input: 66623.94 toks/s, output: 2.03 toks/s]
Processed prompts:   0%|          | 8/3051 [00:03<24:18,  2.09it/s, est. speed input: 67031.12 toks/s, output: 2.05 toks/s]
Processed prompts:   0%|          | 9/3051 [00:04<24:06,  2.10it/s, est. speed input: 67353.59 toks/s, output: 2.06 toks/s]
Processed prompts:   0%|          | 10/3051 [00:04<23:59,  2.11it/s, est. speed input: 67612.94 toks/s, output: 2.06 toks/s]
Processed prompts:   0%|          | 11/3051 [00:05<23:53,  2.12it/s, est. speed input: 67825.33 toks/s, output: 2.07 toks/s]
Processed prompts:   0%|          | 12/3051 [00:05<23:49,  2.13it/s, est. speed input: 68003.49 toks/s, output: 2.08 toks/s]
Processed prompts:   0%|          | 13/3051 [00:06<23:47,  2.13it/s, est. speed input: 68151.99 toks/s, output: 2.08 toks/s]
Processed prompts:   0%|          | 14/3051 [00:06<23:45,  2.13it/s, est. speed input: 68281.87 toks/s, output: 2.08 toks/s]
Processed prompts:   0%|          | 15/3051 [00:07<23:43,  2.13it/s, est. speed input: 68395.56 toks/s, output: 2.09 toks/s]
Processed prompts:   1%|          | 16/3051 [00:07<23:41,  2.13it/s, est. speed input: 68498.66 toks/s, output: 2.09 toks/s]
Processed prompts:   1%|          | 17/3051 [00:08<23:40,  2.14it/s, est. speed input: 68587.72 toks/s, output: 2.09 toks/s]
Processed prompts:   1%|          | 18/3051 [00:08<23:40,  2.14it/s, est. speed input: 68661.68 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 19/3051 [00:09<23:39,  2.14it/s, est. speed input: 68732.71 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 20/3051 [00:09<23:39,  2.14it/s, est. speed input: 68794.78 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 21/3051 [00:09<23:38,  2.14it/s, est. speed input: 68851.15 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 22/3051 [00:10<23:37,  2.14it/s, est. speed input: 68903.61 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 23/3051 [00:10<23:37,  2.14it/s, est. speed input: 68947.96 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 24/3051 [00:11<23:37,  2.14it/s, est. speed input: 68992.55 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 25/3051 [00:11<23:36,  2.14it/s, est. speed input: 69029.58 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 26/3051 [00:12<23:36,  2.14it/s, est. speed input: 69066.86 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 27/3051 [00:12<23:36,  2.14it/s, est. speed input: 69098.89 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 28/3051 [00:13<23:36,  2.13it/s, est. speed input: 69127.15 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 29/3051 [00:13<23:36,  2.13it/s, est. speed input: 69152.86 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 30/3051 [00:14<23:35,  2.13it/s, est. speed input: 69177.61 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 31/3051 [00:14<23:35,  2.13it/s, est. speed input: 69200.38 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 32/3051 [00:15<23:34,  2.13it/s, est. speed input: 69223.18 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 33/3051 [00:15<23:34,  2.13it/s, est. speed input: 69244.70 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 34/3051 [00:16<23:34,  2.13it/s, est. speed input: 69261.79 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 35/3051 [00:16<23:34,  2.13it/s, est. speed input: 69278.59 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 36/3051 [00:17<23:33,  2.13it/s, est. speed input: 69297.13 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 37/3051 [00:17<23:32,  2.13it/s, est. speed input: 69313.25 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 38/3051 [00:17<23:32,  2.13it/s, est. speed input: 69329.32 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 39/3051 [00:18<23:31,  2.13it/s, est. speed input: 69344.67 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 40/3051 [00:18<23:30,  2.13it/s, est. speed input: 69359.71 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 41/3051 [00:19<23:30,  2.13it/s, est. speed input: 69373.35 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 42/3051 [00:19<23:30,  2.13it/s, est. speed input: 69386.14 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 43/3051 [00:20<23:29,  2.13it/s, est. speed input: 69397.66 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 44/3051 [00:20<23:29,  2.13it/s, est. speed input: 69408.03 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 45/3051 [00:21<23:28,  2.13it/s, est. speed input: 69420.71 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 46/3051 [00:21<23:28,  2.13it/s, est. speed input: 69430.99 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 47/3051 [00:22<23:27,  2.13it/s, est. speed input: 69442.21 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 48/3051 [00:22<23:27,  2.13it/s, est. speed input: 69451.61 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 49/3051 [00:23<23:27,  2.13it/s, est. speed input: 69459.69 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 50/3051 [00:23<23:26,  2.13it/s, est. speed input: 69469.07 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 51/3051 [00:24<23:26,  2.13it/s, est. speed input: 69477.07 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 52/3051 [00:24<23:26,  2.13it/s, est. speed input: 69484.53 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 53/3051 [00:24<23:25,  2.13it/s, est. speed input: 69491.06 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 54/3051 [00:25<23:25,  2.13it/s, est. speed input: 69498.41 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 55/3051 [00:25<23:24,  2.13it/s, est. speed input: 69505.06 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 56/3051 [00:26<23:24,  2.13it/s, est. speed input: 69511.83 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 57/3051 [00:26<23:24,  2.13it/s, est. speed input: 69517.82 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 58/3051 [00:27<23:23,  2.13it/s, est. speed input: 69523.83 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 59/3051 [00:27<23:23,  2.13it/s, est. speed input: 69529.19 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 60/3051 [00:28<23:22,  2.13it/s, est. speed input: 69535.96 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 61/3051 [00:28<23:21,  2.13it/s, est. speed input: 69541.69 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 62/3051 [00:29<23:21,  2.13it/s, est. speed input: 69547.71 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 63/3051 [00:29<23:20,  2.13it/s, est. speed input: 69553.60 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 64/3051 [00:30<23:20,  2.13it/s, est. speed input: 69559.01 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 65/3051 [00:30<23:20,  2.13it/s, est. speed input: 69563.43 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 66/3051 [00:31<23:19,  2.13it/s, est. speed input: 69569.45 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 67/3051 [00:31<23:19,  2.13it/s, est. speed input: 69573.10 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 68/3051 [00:32<23:18,  2.13it/s, est. speed input: 69578.63 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 69/3051 [00:32<23:18,  2.13it/s, est. speed input: 69582.96 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 70/3051 [00:32<23:17,  2.13it/s, est. speed input: 69586.75 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 71/3051 [00:33<23:17,  2.13it/s, est. speed input: 69590.33 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 72/3051 [00:33<23:17,  2.13it/s, est. speed input: 69594.45 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 73/3051 [00:34<23:16,  2.13it/s, est. speed input: 69597.72 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 74/3051 [00:34<23:16,  2.13it/s, est. speed input: 69601.26 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 75/3051 [00:35<23:16,  2.13it/s, est. speed input: 69604.20 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 76/3051 [00:35<23:15,  2.13it/s, est. speed input: 69607.82 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 77/3051 [00:36<23:15,  2.13it/s, est. speed input: 69610.42 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 78/3051 [00:36<23:14,  2.13it/s, est. speed input: 69613.62 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 79/3051 [00:37<23:14,  2.13it/s, est. speed input: 69616.55 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 80/3051 [00:37<23:13,  2.13it/s, est. speed input: 69620.25 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 81/3051 [00:38<23:12,  2.13it/s, est. speed input: 69623.62 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 82/3051 [00:38<23:12,  2.13it/s, est. speed input: 69626.06 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 83/3051 [00:39<23:12,  2.13it/s, est. speed input: 69627.62 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 84/3051 [00:39<23:11,  2.13it/s, est. speed input: 69630.94 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 85/3051 [00:39<23:11,  2.13it/s, est. speed input: 69633.31 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 86/3051 [00:40<23:10,  2.13it/s, est. speed input: 69635.89 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 87/3051 [00:40<23:10,  2.13it/s, est. speed input: 69637.99 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 88/3051 [00:41<23:10,  2.13it/s, est. speed input: 69640.51 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 89/3051 [00:41<23:09,  2.13it/s, est. speed input: 69642.60 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 90/3051 [00:42<23:09,  2.13it/s, est. speed input: 69644.04 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 91/3051 [00:42<23:09,  2.13it/s, est. speed input: 69646.06 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 92/3051 [00:43<23:08,  2.13it/s, est. speed input: 69647.91 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 93/3051 [00:43<23:08,  2.13it/s, est. speed input: 69649.74 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 94/3051 [00:44<23:07,  2.13it/s, est. speed input: 69651.37 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 95/3051 [00:44<23:07,  2.13it/s, est. speed input: 69653.14 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 96/3051 [00:45<23:07,  2.13it/s, est. speed input: 69654.27 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 97/3051 [00:45<23:06,  2.13it/s, est. speed input: 69656.01 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 98/3051 [00:46<23:06,  2.13it/s, est. speed input: 69657.43 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 99/3051 [00:46<23:05,  2.13it/s, est. speed input: 69658.78 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 100/3051 [00:47<23:05,  2.13it/s, est. speed input: 69659.70 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 101/3051 [00:47<23:04,  2.13it/s, est. speed input: 69661.32 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 102/3051 [00:47<23:03,  2.13it/s, est. speed input: 69663.70 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 103/3051 [00:48<23:03,  2.13it/s, est. speed input: 69665.10 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 104/3051 [00:48<23:03,  2.13it/s, est. speed input: 69666.64 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 105/3051 [00:49<23:02,  2.13it/s, est. speed input: 69667.91 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 106/3051 [00:49<23:02,  2.13it/s, est. speed input: 69669.46 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 107/3051 [00:50<23:02,  2.13it/s, est. speed input: 69670.21 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 108/3051 [00:50<23:02,  2.13it/s, est. speed input: 69670.62 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 109/3051 [00:51<23:01,  2.13it/s, est. speed input: 69671.93 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 110/3051 [00:51<23:00,  2.13it/s, est. speed input: 69673.29 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 111/3051 [00:52<23:00,  2.13it/s, est. speed input: 69674.61 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 112/3051 [00:52<22:59,  2.13it/s, est. speed input: 69675.74 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 113/3051 [00:53<22:59,  2.13it/s, est. speed input: 69676.75 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 114/3051 [00:53<22:58,  2.13it/s, est. speed input: 69677.56 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 115/3051 [00:54<22:58,  2.13it/s, est. speed input: 69678.83 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 116/3051 [00:54<22:57,  2.13it/s, est. speed input: 69680.27 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 117/3051 [00:55<22:57,  2.13it/s, est. speed input: 69680.44 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 118/3051 [00:55<22:57,  2.13it/s, est. speed input: 69681.63 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 119/3051 [00:55<22:56,  2.13it/s, est. speed input: 69682.32 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 120/3051 [00:56<22:56,  2.13it/s, est. speed input: 69682.78 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 121/3051 [00:56<22:56,  2.13it/s, est. speed input: 69683.51 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 122/3051 [00:57<22:55,  2.13it/s, est. speed input: 69684.31 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 123/3051 [00:57<22:54,  2.13it/s, est. speed input: 69685.23 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 124/3051 [00:58<22:54,  2.13it/s, est. speed input: 69685.99 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 125/3051 [00:58<22:54,  2.13it/s, est. speed input: 69685.91 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 126/3051 [00:59<22:54,  2.13it/s, est. speed input: 69686.39 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 127/3051 [00:59<22:54,  2.13it/s, est. speed input: 69686.38 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 128/3051 [01:00<22:53,  2.13it/s, est. speed input: 69686.81 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 129/3051 [01:00<22:52,  2.13it/s, est. speed input: 69687.67 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 130/3051 [01:01<22:52,  2.13it/s, est. speed input: 69687.38 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 131/3051 [01:01<22:51,  2.13it/s, est. speed input: 69688.14 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 132/3051 [01:02<22:51,  2.13it/s, est. speed input: 69688.60 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 133/3051 [01:02<22:51,  2.13it/s, est. speed input: 69688.39 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 134/3051 [01:03<22:50,  2.13it/s, est. speed input: 69688.62 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 135/3051 [01:03<22:50,  2.13it/s, est. speed input: 69688.62 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 136/3051 [01:03<22:50,  2.13it/s, est. speed input: 69689.05 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 137/3051 [01:04<22:49,  2.13it/s, est. speed input: 69689.10 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 138/3051 [01:04<22:48,  2.13it/s, est. speed input: 69689.70 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 139/3051 [01:05<22:49,  2.13it/s, est. speed input: 69689.25 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 140/3051 [01:05<22:48,  2.13it/s, est. speed input: 69689.65 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 141/3051 [01:06<22:47,  2.13it/s, est. speed input: 69689.88 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 142/3051 [01:06<22:47,  2.13it/s, est. speed input: 69690.16 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 143/3051 [01:07<22:46,  2.13it/s, est. speed input: 69690.14 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 144/3051 [01:07<22:46,  2.13it/s, est. speed input: 69690.32 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 145/3051 [01:08<22:46,  2.13it/s, est. speed input: 69689.79 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 146/3051 [01:08<22:46,  2.13it/s, est. speed input: 69689.55 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 147/3051 [01:09<22:45,  2.13it/s, est. speed input: 69689.37 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 148/3051 [01:09<22:45,  2.13it/s, est. speed input: 69689.07 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 149/3051 [01:10<22:45,  2.13it/s, est. speed input: 69688.56 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 150/3051 [01:10<22:44,  2.13it/s, est. speed input: 69688.62 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 151/3051 [01:11<22:43,  2.13it/s, est. speed input: 69689.02 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 152/3051 [01:11<22:43,  2.13it/s, est. speed input: 69688.89 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 153/3051 [01:11<22:43,  2.13it/s, est. speed input: 69688.38 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 154/3051 [01:12<22:41,  2.13it/s, est. speed input: 69689.01 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 155/3051 [01:12<22:42,  2.13it/s, est. speed input: 69688.11 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 156/3051 [01:13<22:42,  2.13it/s, est. speed input: 69687.76 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 157/3051 [01:13<22:41,  2.13it/s, est. speed input: 69687.32 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 158/3051 [01:14<22:41,  2.12it/s, est. speed input: 69686.85 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 159/3051 [01:14<22:41,  2.12it/s, est. speed input: 69686.36 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 160/3051 [01:15<22:39,  2.13it/s, est. speed input: 69686.66 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 161/3051 [01:15<22:39,  2.13it/s, est. speed input: 69685.96 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 162/3051 [01:16<22:38,  2.13it/s, est. speed input: 69686.40 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 163/3051 [01:16<22:38,  2.13it/s, est. speed input: 69686.41 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 164/3051 [01:17<22:38,  2.13it/s, est. speed input: 69685.97 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 165/3051 [01:17<22:37,  2.13it/s, est. speed input: 69685.58 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 166/3051 [01:18<22:37,  2.12it/s, est. speed input: 69684.77 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 167/3051 [01:18<22:37,  2.12it/s, est. speed input: 69684.28 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 168/3051 [01:18<22:36,  2.12it/s, est. speed input: 69684.01 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 169/3051 [01:19<22:36,  2.13it/s, est. speed input: 69684.08 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 170/3051 [01:19<22:36,  2.12it/s, est. speed input: 69683.35 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 171/3051 [01:20<22:35,  2.12it/s, est. speed input: 69682.95 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 172/3051 [01:20<22:34,  2.13it/s, est. speed input: 69682.97 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 173/3051 [01:21<22:34,  2.12it/s, est. speed input: 69682.56 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 174/3051 [01:21<22:34,  2.12it/s, est. speed input: 69681.89 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 175/3051 [01:22<22:33,  2.12it/s, est. speed input: 69681.74 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 176/3051 [01:22<22:33,  2.12it/s, est. speed input: 69681.24 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 177/3051 [01:23<22:32,  2.13it/s, est. speed input: 69681.43 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 178/3051 [01:23<22:32,  2.12it/s, est. speed input: 69680.45 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 179/3051 [01:24<22:31,  2.12it/s, est. speed input: 69680.23 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 180/3051 [01:24<22:31,  2.12it/s, est. speed input: 69679.56 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 181/3051 [01:25<22:30,  2.12it/s, est. speed input: 69679.51 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 182/3051 [01:25<22:30,  2.13it/s, est. speed input: 69679.42 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 183/3051 [01:26<22:28,  2.13it/s, est. speed input: 69679.90 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 184/3051 [01:26<22:28,  2.13it/s, est. speed input: 69680.07 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 185/3051 [01:26<22:27,  2.13it/s, est. speed input: 69680.29 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 186/3051 [01:27<22:29,  2.12it/s, est. speed input: 69678.22 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 187/3051 [01:27<22:28,  2.12it/s, est. speed input: 69678.26 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 188/3051 [01:28<22:28,  2.12it/s, est. speed input: 69677.15 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 189/3051 [01:28<22:28,  2.12it/s, est. speed input: 69676.57 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 190/3051 [01:29<22:27,  2.12it/s, est. speed input: 69676.48 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 191/3051 [01:29<22:26,  2.12it/s, est. speed input: 69676.33 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 192/3051 [01:30<22:25,  2.12it/s, est. speed input: 69676.23 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 193/3051 [01:30<22:25,  2.12it/s, est. speed input: 69675.98 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 194/3051 [01:31<22:24,  2.13it/s, est. speed input: 69676.15 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 195/3051 [01:31<22:23,  2.13it/s, est. speed input: 69676.45 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 196/3051 [01:32<22:22,  2.13it/s, est. speed input: 69676.29 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 197/3051 [01:32<22:22,  2.13it/s, est. speed input: 69676.06 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 198/3051 [01:33<22:22,  2.13it/s, est. speed input: 69675.67 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 199/3051 [01:33<22:28,  2.12it/s, est. speed input: 69669.90 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 200/3051 [01:34<22:26,  2.12it/s, est. speed input: 69669.47 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 201/3051 [01:34<22:24,  2.12it/s, est. speed input: 69668.82 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 202/3051 [01:35<22:23,  2.12it/s, est. speed input: 69668.83 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 203/3051 [01:35<22:22,  2.12it/s, est. speed input: 69668.58 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 204/3051 [01:35<22:21,  2.12it/s, est. speed input: 69668.40 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 205/3051 [01:36<22:19,  2.12it/s, est. speed input: 69668.56 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 206/3051 [01:36<22:24,  2.12it/s, est. speed input: 69663.55 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 207/3051 [01:37<22:23,  2.12it/s, est. speed input: 69662.91 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 208/3051 [01:37<22:21,  2.12it/s, est. speed input: 69662.55 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 209/3051 [01:38<22:19,  2.12it/s, est. speed input: 69662.66 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 210/3051 [01:38<22:18,  2.12it/s, est. speed input: 69662.22 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 211/3051 [01:39<22:18,  2.12it/s, est. speed input: 69661.17 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 212/3051 [01:39<22:17,  2.12it/s, est. speed input: 69661.09 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 213/3051 [01:40<22:17,  2.12it/s, est. speed input: 69660.65 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 214/3051 [01:40<22:16,  2.12it/s, est. speed input: 69660.52 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 215/3051 [01:41<22:15,  2.12it/s, est. speed input: 69660.43 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 216/3051 [01:41<22:14,  2.12it/s, est. speed input: 69660.56 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 217/3051 [01:42<22:13,  2.12it/s, est. speed input: 69660.24 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 218/3051 [01:42<22:15,  2.12it/s, est. speed input: 69658.47 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 219/3051 [01:43<22:16,  2.12it/s, est. speed input: 69656.55 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 220/3051 [01:43<22:14,  2.12it/s, est. speed input: 69656.86 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 221/3051 [01:43<22:12,  2.12it/s, est. speed input: 69656.99 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 222/3051 [01:44<22:11,  2.12it/s, est. speed input: 69657.25 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 223/3051 [01:44<22:10,  2.13it/s, est. speed input: 69657.49 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 224/3051 [01:45<22:09,  2.13it/s, est. speed input: 69657.64 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 225/3051 [01:45<22:09,  2.13it/s, est. speed input: 69657.94 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 226/3051 [01:46<22:12,  2.12it/s, est. speed input: 69655.21 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 227/3051 [01:46<22:12,  2.12it/s, est. speed input: 69654.17 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 228/3051 [01:47<22:11,  2.12it/s, est. speed input: 69653.54 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 229/3051 [01:47<22:09,  2.12it/s, est. speed input: 69653.52 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 230/3051 [01:48<22:09,  2.12it/s, est. speed input: 69652.97 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 231/3051 [01:48<22:09,  2.12it/s, est. speed input: 69651.97 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 232/3051 [01:49<22:08,  2.12it/s, est. speed input: 69651.58 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 233/3051 [01:49<22:11,  2.12it/s, est. speed input: 69648.96 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 234/3051 [01:50<22:09,  2.12it/s, est. speed input: 69648.45 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 235/3051 [01:50<22:08,  2.12it/s, est. speed input: 69648.14 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 236/3051 [01:51<22:08,  2.12it/s, est. speed input: 69647.22 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 237/3051 [01:51<22:06,  2.12it/s, est. speed input: 69647.41 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 238/3051 [01:51<22:05,  2.12it/s, est. speed input: 69646.85 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 239/3051 [01:52<22:04,  2.12it/s, est. speed input: 69646.94 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 240/3051 [01:52<22:03,  2.12it/s, est. speed input: 69647.31 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 241/3051 [01:53<22:02,  2.12it/s, est. speed input: 69647.36 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 242/3051 [01:53<22:01,  2.13it/s, est. speed input: 69647.53 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 243/3051 [01:54<22:02,  2.12it/s, est. speed input: 69646.26 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 244/3051 [01:54<22:02,  2.12it/s, est. speed input: 69645.64 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 245/3051 [01:55<22:02,  2.12it/s, est. speed input: 69645.06 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 246/3051 [01:55<22:01,  2.12it/s, est. speed input: 69644.62 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 247/3051 [01:56<22:06,  2.11it/s, est. speed input: 69640.71 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 248/3051 [01:56<22:04,  2.12it/s, est. speed input: 69639.93 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 249/3051 [01:57<22:03,  2.12it/s, est. speed input: 69639.59 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 250/3051 [01:57<22:01,  2.12it/s, est. speed input: 69639.66 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 251/3051 [01:58<22:01,  2.12it/s, est. speed input: 69638.64 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 252/3051 [01:58<22:00,  2.12it/s, est. speed input: 69637.92 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 253/3051 [01:59<22:00,  2.12it/s, est. speed input: 69636.95 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 254/3051 [01:59<21:58,  2.12it/s, est. speed input: 69637.14 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 255/3051 [01:59<21:57,  2.12it/s, est. speed input: 69636.98 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 256/3051 [02:00<22:00,  2.12it/s, est. speed input: 69634.42 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 257/3051 [02:00<21:58,  2.12it/s, est. speed input: 69634.23 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 258/3051 [02:01<21:56,  2.12it/s, est. speed input: 69634.47 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 259/3051 [02:01<21:55,  2.12it/s, est. speed input: 69634.36 toks/s, output: 2.13 toks/s]
Processed prompts:   9%|▊         | 260/3051 [02:02<21:55,  2.12it/s, est. speed input: 69633.86 toks/s, output: 2.13 toks/s]
Processed prompts:   9%|▊         | 261/3051 [02:02<21:54,  2.12it/s, est. speed input: 69633.89 toks/s, output: 2.13 toks/s]
Processed prompts:   9%|▊         | 262/3051 [02:03<21:57,  2.12it/s, est. speed input: 69631.21 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 263/3051 [02:03<21:57,  2.12it/s, est. speed input: 69630.03 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 264/3051 [02:04<21:55,  2.12it/s, est. speed input: 69629.62 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 265/3051 [02:04<21:54,  2.12it/s, est. speed input: 69629.62 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 266/3051 [02:05<21:53,  2.12it/s, est. speed input: 69629.40 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 267/3051 [02:05<21:52,  2.12it/s, est. speed input: 69628.99 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 268/3051 [02:06<21:51,  2.12it/s, est. speed input: 69629.18 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 269/3051 [02:06<21:53,  2.12it/s, est. speed input: 69626.80 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 270/3051 [02:07<21:53,  2.12it/s, est. speed input: 69625.90 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 271/3051 [02:07<21:52,  2.12it/s, est. speed input: 69625.32 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 272/3051 [02:08<21:50,  2.12it/s, est. speed input: 69625.46 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 273/3051 [02:08<21:49,  2.12it/s, est. speed input: 69625.34 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 274/3051 [02:08<21:48,  2.12it/s, est. speed input: 69625.19 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 275/3051 [02:09<21:47,  2.12it/s, est. speed input: 69625.02 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 276/3051 [02:09<21:50,  2.12it/s, est. speed input: 69622.75 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 277/3051 [02:10<21:49,  2.12it/s, est. speed input: 69622.33 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 278/3051 [02:10<21:47,  2.12it/s, est. speed input: 69622.38 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 279/3051 [02:11<21:46,  2.12it/s, est. speed input: 69622.28 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 280/3051 [02:11<21:45,  2.12it/s, est. speed input: 69622.01 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 281/3051 [02:12<21:45,  2.12it/s, est. speed input: 69621.59 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 282/3051 [02:12<21:45,  2.12it/s, est. speed input: 69621.30 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 283/3051 [02:13<21:44,  2.12it/s, est. speed input: 69621.12 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 284/3051 [02:13<21:43,  2.12it/s, est. speed input: 69620.80 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 285/3051 [02:14<21:42,  2.12it/s, est. speed input: 69620.79 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 286/3051 [02:14<21:43,  2.12it/s, est. speed input: 69619.90 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 287/3051 [02:15<21:43,  2.12it/s, est. speed input: 69619.21 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 288/3051 [02:15<21:42,  2.12it/s, est. speed input: 69618.93 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 289/3051 [02:16<21:41,  2.12it/s, est. speed input: 69618.80 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 290/3051 [02:16<21:42,  2.12it/s, est. speed input: 69618.04 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 291/3051 [02:16<21:41,  2.12it/s, est. speed input: 69617.78 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 292/3051 [02:17<21:42,  2.12it/s, est. speed input: 69616.27 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 293/3051 [02:17<21:42,  2.12it/s, est. speed input: 69615.51 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 294/3051 [02:18<21:40,  2.12it/s, est. speed input: 69615.54 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 295/3051 [02:18<21:45,  2.11it/s, est. speed input: 69612.00 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 296/3051 [02:19<21:42,  2.12it/s, est. speed input: 69612.05 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 297/3051 [02:19<21:39,  2.12it/s, est. speed input: 69612.25 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 298/3051 [02:20<21:37,  2.12it/s, est. speed input: 69612.56 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 299/3051 [02:20<21:39,  2.12it/s, est. speed input: 69610.70 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 300/3051 [02:21<21:39,  2.12it/s, est. speed input: 69610.07 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 301/3051 [02:21<21:36,  2.12it/s, est. speed input: 69610.47 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 302/3051 [02:22<21:36,  2.12it/s, est. speed input: 69610.15 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 303/3051 [02:22<21:35,  2.12it/s, est. speed input: 69610.04 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 304/3051 [02:23<21:34,  2.12it/s, est. speed input: 69610.10 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 305/3051 [02:23<21:34,  2.12it/s, est. speed input: 69609.52 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 306/3051 [02:24<21:34,  2.12it/s, est. speed input: 69608.74 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 307/3051 [02:24<21:33,  2.12it/s, est. speed input: 69609.00 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 308/3051 [02:24<21:34,  2.12it/s, est. speed input: 69607.92 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 309/3051 [02:25<21:35,  2.12it/s, est. speed input: 69606.52 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 310/3051 [02:25<21:34,  2.12it/s, est. speed input: 69605.66 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 311/3051 [02:26<21:33,  2.12it/s, est. speed input: 69605.60 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 312/3051 [02:26<21:32,  2.12it/s, est. speed input: 69605.13 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 313/3051 [02:27<21:32,  2.12it/s, est. speed input: 69604.32 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 314/3051 [02:27<21:31,  2.12it/s, est. speed input: 69604.07 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 315/3051 [02:28<21:29,  2.12it/s, est. speed input: 69604.39 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 316/3051 [02:28<21:29,  2.12it/s, est. speed input: 69603.85 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 317/3051 [02:29<21:29,  2.12it/s, est. speed input: 69603.04 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 318/3051 [02:29<21:28,  2.12it/s, est. speed input: 69603.17 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 319/3051 [02:30<21:27,  2.12it/s, est. speed input: 69602.86 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 320/3051 [02:30<21:27,  2.12it/s, est. speed input: 69602.53 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 321/3051 [02:31<21:26,  2.12it/s, est. speed input: 69602.32 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 322/3051 [02:31<21:26,  2.12it/s, est. speed input: 69602.27 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 323/3051 [02:32<21:26,  2.12it/s, est. speed input: 69601.54 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 324/3051 [02:32<21:25,  2.12it/s, est. speed input: 69601.24 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 325/3051 [02:33<21:29,  2.11it/s, est. speed input: 69598.56 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 326/3051 [02:33<21:28,  2.11it/s, est. speed input: 69597.80 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 327/3051 [02:33<21:26,  2.12it/s, est. speed input: 69597.85 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 328/3051 [02:34<21:24,  2.12it/s, est. speed input: 69597.95 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 329/3051 [02:34<21:23,  2.12it/s, est. speed input: 69597.62 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 330/3051 [02:35<21:23,  2.12it/s, est. speed input: 69597.19 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 331/3051 [02:35<21:22,  2.12it/s, est. speed input: 69596.99 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 332/3051 [02:36<21:21,  2.12it/s, est. speed input: 69596.89 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 333/3051 [02:36<21:20,  2.12it/s, est. speed input: 69597.03 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 334/3051 [02:37<21:18,  2.12it/s, est. speed input: 69597.50 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 335/3051 [02:37<21:20,  2.12it/s, est. speed input: 69596.24 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 336/3051 [02:38<21:21,  2.12it/s, est. speed input: 69595.26 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 337/3051 [02:38<21:19,  2.12it/s, est. speed input: 69595.35 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 338/3051 [02:39<21:20,  2.12it/s, est. speed input: 69594.59 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 339/3051 [02:39<21:19,  2.12it/s, est. speed input: 69594.14 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 340/3051 [02:40<21:18,  2.12it/s, est. speed input: 69594.00 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 341/3051 [02:40<21:17,  2.12it/s, est. speed input: 69593.89 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 342/3051 [02:41<21:19,  2.12it/s, est. speed input: 69592.70 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 343/3051 [02:41<21:18,  2.12it/s, est. speed input: 69592.35 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 344/3051 [02:41<21:16,  2.12it/s, est. speed input: 69592.49 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 345/3051 [02:42<21:17,  2.12it/s, est. speed input: 69591.21 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 346/3051 [02:42<21:16,  2.12it/s, est. speed input: 69590.93 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 347/3051 [02:43<21:15,  2.12it/s, est. speed input: 69590.90 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 348/3051 [02:43<21:13,  2.12it/s, est. speed input: 69591.19 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 349/3051 [02:44<21:14,  2.12it/s, est. speed input: 69590.61 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 350/3051 [02:44<21:13,  2.12it/s, est. speed input: 69590.45 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 351/3051 [02:45<21:12,  2.12it/s, est. speed input: 69590.43 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 352/3051 [02:45<21:11,  2.12it/s, est. speed input: 69590.39 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 353/3051 [02:46<21:10,  2.12it/s, est. speed input: 69590.43 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 354/3051 [02:46<21:10,  2.12it/s, est. speed input: 69590.21 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 355/3051 [02:47<21:12,  2.12it/s, est. speed input: 69589.10 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 356/3051 [02:47<21:12,  2.12it/s, est. speed input: 69588.15 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 357/3051 [02:48<21:11,  2.12it/s, est. speed input: 69587.78 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 358/3051 [02:48<21:10,  2.12it/s, est. speed input: 69587.71 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 359/3051 [02:49<21:10,  2.12it/s, est. speed input: 69587.27 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 360/3051 [02:49<21:09,  2.12it/s, est. speed input: 69586.80 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 361/3051 [02:49<21:08,  2.12it/s, est. speed input: 69586.82 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 362/3051 [02:50<21:11,  2.12it/s, est. speed input: 69584.89 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 363/3051 [02:50<21:09,  2.12it/s, est. speed input: 69584.67 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 364/3051 [02:51<21:27,  2.09it/s, est. speed input: 69574.85 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 365/3051 [02:51<21:20,  2.10it/s, est. speed input: 69574.64 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 366/3051 [02:52<21:15,  2.11it/s, est. speed input: 69574.82 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 367/3051 [02:52<21:18,  2.10it/s, est. speed input: 69571.26 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 368/3051 [02:53<21:13,  2.11it/s, est. speed input: 69571.46 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 369/3051 [02:53<21:11,  2.11it/s, est. speed input: 69570.97 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 370/3051 [02:54<21:08,  2.11it/s, est. speed input: 69570.76 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 371/3051 [02:54<21:07,  2.11it/s, est. speed input: 69570.10 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 372/3051 [02:55<21:07,  2.11it/s, est. speed input: 69569.07 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 373/3051 [02:55<21:05,  2.12it/s, est. speed input: 69569.04 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 374/3051 [02:56<21:03,  2.12it/s, est. speed input: 69569.30 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 375/3051 [02:56<21:02,  2.12it/s, est. speed input: 69569.31 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 376/3051 [02:57<21:01,  2.12it/s, est. speed input: 69569.24 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 377/3051 [02:57<21:00,  2.12it/s, est. speed input: 69569.31 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 378/3051 [02:58<21:03,  2.12it/s, est. speed input: 69567.30 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 379/3051 [02:58<21:02,  2.12it/s, est. speed input: 69566.84 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 380/3051 [02:58<21:01,  2.12it/s, est. speed input: 69566.79 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 381/3051 [02:59<21:00,  2.12it/s, est. speed input: 69566.65 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 382/3051 [02:59<20:58,  2.12it/s, est. speed input: 69566.85 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 383/3051 [03:00<20:57,  2.12it/s, est. speed input: 69566.74 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 384/3051 [03:00<20:58,  2.12it/s, est. speed input: 69565.93 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 385/3051 [03:01<20:57,  2.12it/s, est. speed input: 69565.90 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 386/3051 [03:01<20:57,  2.12it/s, est. speed input: 69565.55 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 387/3051 [03:02<20:59,  2.11it/s, est. speed input: 69563.79 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 388/3051 [03:02<21:00,  2.11it/s, est. speed input: 69562.33 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 389/3051 [03:03<20:58,  2.12it/s, est. speed input: 69562.54 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 390/3051 [03:03<20:57,  2.12it/s, est. speed input: 69561.96 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 391/3051 [03:04<20:57,  2.12it/s, est. speed input: 69561.44 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 392/3051 [03:04<20:55,  2.12it/s, est. speed input: 69561.44 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 393/3051 [03:05<20:53,  2.12it/s, est. speed input: 69561.54 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 394/3051 [03:05<20:52,  2.12it/s, est. speed input: 69561.52 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 395/3051 [03:06<20:51,  2.12it/s, est. speed input: 69561.67 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 396/3051 [03:06<20:50,  2.12it/s, est. speed input: 69561.83 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 397/3051 [03:07<20:50,  2.12it/s, est. speed input: 69561.64 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 398/3051 [03:07<20:54,  2.12it/s, est. speed input: 69559.68 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 399/3051 [03:07<20:52,  2.12it/s, est. speed input: 69559.74 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 400/3051 [03:08<20:51,  2.12it/s, est. speed input: 69559.67 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 401/3051 [03:08<20:49,  2.12it/s, est. speed input: 69559.78 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 402/3051 [03:09<20:48,  2.12it/s, est. speed input: 69559.81 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 403/3051 [03:09<20:47,  2.12it/s, est. speed input: 69559.86 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 404/3051 [03:10<20:47,  2.12it/s, est. speed input: 69559.63 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 405/3051 [03:10<20:51,  2.11it/s, est. speed input: 69557.50 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 406/3051 [03:11<20:50,  2.12it/s, est. speed input: 69557.33 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 407/3051 [03:11<20:48,  2.12it/s, est. speed input: 69557.31 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 408/3051 [03:12<20:47,  2.12it/s, est. speed input: 69556.92 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 409/3051 [03:12<20:46,  2.12it/s, est. speed input: 69557.20 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 410/3051 [03:13<20:49,  2.11it/s, est. speed input: 69555.30 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 411/3051 [03:13<20:48,  2.11it/s, est. speed input: 69554.80 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 412/3051 [03:14<20:46,  2.12it/s, est. speed input: 69554.93 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 413/3051 [03:14<20:45,  2.12it/s, est. speed input: 69554.61 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 414/3051 [03:15<20:44,  2.12it/s, est. speed input: 69554.64 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 415/3051 [03:15<20:43,  2.12it/s, est. speed input: 69554.61 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 416/3051 [03:15<20:42,  2.12it/s, est. speed input: 69554.74 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 417/3051 [03:16<20:44,  2.12it/s, est. speed input: 69553.36 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 418/3051 [03:16<20:43,  2.12it/s, est. speed input: 69552.93 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 419/3051 [03:17<20:42,  2.12it/s, est. speed input: 69553.06 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 420/3051 [03:17<20:40,  2.12it/s, est. speed input: 69553.28 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 421/3051 [03:18<20:42,  2.12it/s, est. speed input: 69552.22 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 422/3051 [03:18<20:40,  2.12it/s, est. speed input: 69552.27 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 423/3051 [03:19<20:38,  2.12it/s, est. speed input: 69552.53 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 424/3051 [03:19<20:39,  2.12it/s, est. speed input: 69552.13 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 425/3051 [03:20<20:38,  2.12it/s, est. speed input: 69552.14 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 426/3051 [03:20<20:37,  2.12it/s, est. speed input: 69552.05 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 427/3051 [03:21<20:38,  2.12it/s, est. speed input: 69551.45 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 428/3051 [03:21<20:37,  2.12it/s, est. speed input: 69551.18 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 429/3051 [03:22<20:37,  2.12it/s, est. speed input: 69550.95 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 430/3051 [03:22<20:40,  2.11it/s, est. speed input: 69548.98 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 431/3051 [03:23<20:38,  2.11it/s, est. speed input: 69548.84 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 432/3051 [03:23<20:36,  2.12it/s, est. speed input: 69548.91 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 433/3051 [03:24<20:35,  2.12it/s, est. speed input: 69549.06 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 434/3051 [03:24<20:33,  2.12it/s, est. speed input: 69549.17 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 435/3051 [03:24<20:32,  2.12it/s, est. speed input: 69549.54 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 436/3051 [03:25<20:32,  2.12it/s, est. speed input: 69549.19 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 437/3051 [03:25<20:34,  2.12it/s, est. speed input: 69548.23 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 438/3051 [03:26<20:32,  2.12it/s, est. speed input: 69548.53 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 439/3051 [03:26<20:31,  2.12it/s, est. speed input: 69548.57 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 440/3051 [03:27<20:30,  2.12it/s, est. speed input: 69548.45 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 441/3051 [03:27<20:33,  2.12it/s, est. speed input: 69546.98 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 442/3051 [03:28<20:32,  2.12it/s, est. speed input: 69546.97 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 443/3051 [03:28<20:30,  2.12it/s, est. speed input: 69546.99 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 444/3051 [03:29<20:32,  2.12it/s, est. speed input: 69545.93 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 445/3051 [03:29<20:30,  2.12it/s, est. speed input: 69545.84 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 446/3051 [03:30<20:31,  2.12it/s, est. speed input: 69545.01 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 447/3051 [03:30<20:30,  2.12it/s, est. speed input: 69544.66 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 448/3051 [03:31<20:30,  2.12it/s, est. speed input: 69544.19 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 449/3051 [03:31<20:27,  2.12it/s, est. speed input: 69544.45 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 450/3051 [03:32<20:28,  2.12it/s, est. speed input: 69543.86 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 451/3051 [03:32<20:27,  2.12it/s, est. speed input: 69543.49 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 452/3051 [03:32<20:26,  2.12it/s, est. speed input: 69543.45 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 453/3051 [03:33<20:25,  2.12it/s, est. speed input: 69543.67 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 454/3051 [03:33<20:24,  2.12it/s, est. speed input: 69543.64 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 455/3051 [03:34<20:23,  2.12it/s, est. speed input: 69543.64 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 456/3051 [03:34<20:23,  2.12it/s, est. speed input: 69543.67 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 457/3051 [03:35<20:22,  2.12it/s, est. speed input: 69543.45 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 458/3051 [03:35<20:22,  2.12it/s, est. speed input: 69543.43 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 459/3051 [03:36<20:21,  2.12it/s, est. speed input: 69543.57 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 460/3051 [03:36<20:23,  2.12it/s, est. speed input: 69542.53 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 461/3051 [03:37<20:22,  2.12it/s, est. speed input: 69542.26 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 462/3051 [03:37<20:22,  2.12it/s, est. speed input: 69542.07 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 463/3051 [03:38<20:21,  2.12it/s, est. speed input: 69541.85 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 464/3051 [03:38<20:20,  2.12it/s, est. speed input: 69541.78 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 465/3051 [03:39<20:19,  2.12it/s, est. speed input: 69541.97 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 466/3051 [03:39<20:19,  2.12it/s, est. speed input: 69541.56 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 467/3051 [03:40<20:20,  2.12it/s, est. speed input: 69540.85 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 468/3051 [03:40<20:19,  2.12it/s, est. speed input: 69540.86 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 469/3051 [03:40<20:17,  2.12it/s, est. speed input: 69541.08 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 470/3051 [03:41<20:17,  2.12it/s, est. speed input: 69540.90 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 471/3051 [03:41<20:17,  2.12it/s, est. speed input: 69540.53 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 472/3051 [03:42<20:17,  2.12it/s, est. speed input: 69540.24 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 473/3051 [03:42<20:16,  2.12it/s, est. speed input: 69539.90 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 474/3051 [03:43<20:15,  2.12it/s, est. speed input: 69540.15 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 475/3051 [03:43<20:13,  2.12it/s, est. speed input: 69540.39 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 476/3051 [03:44<20:15,  2.12it/s, est. speed input: 69539.61 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 477/3051 [03:44<20:14,  2.12it/s, est. speed input: 69539.66 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 478/3051 [03:45<20:13,  2.12it/s, est. speed input: 69539.74 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 479/3051 [03:45<20:14,  2.12it/s, est. speed input: 69538.76 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 480/3051 [03:46<20:14,  2.12it/s, est. speed input: 69538.40 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 481/3051 [03:46<20:12,  2.12it/s, est. speed input: 69538.49 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 482/3051 [03:47<20:11,  2.12it/s, est. speed input: 69538.50 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 483/3051 [03:47<20:12,  2.12it/s, est. speed input: 69537.97 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 484/3051 [03:48<20:11,  2.12it/s, est. speed input: 69537.94 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 485/3051 [03:48<20:11,  2.12it/s, est. speed input: 69537.29 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 486/3051 [03:49<20:11,  2.12it/s, est. speed input: 69536.93 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 487/3051 [03:49<20:11,  2.12it/s, est. speed input: 69536.53 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 488/3051 [03:49<20:09,  2.12it/s, est. speed input: 69536.83 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 489/3051 [03:50<20:07,  2.12it/s, est. speed input: 69536.94 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 490/3051 [03:50<20:06,  2.12it/s, est. speed input: 69537.11 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 491/3051 [03:51<20:06,  2.12it/s, est. speed input: 69537.17 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 492/3051 [03:51<20:06,  2.12it/s, est. speed input: 69536.92 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 493/3051 [03:52<20:05,  2.12it/s, est. speed input: 69536.98 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 494/3051 [03:52<20:05,  2.12it/s, est. speed input: 69536.95 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 495/3051 [03:53<20:04,  2.12it/s, est. speed input: 69537.11 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 496/3051 [03:53<20:05,  2.12it/s, est. speed input: 69536.54 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 497/3051 [03:54<20:05,  2.12it/s, est. speed input: 69536.19 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 498/3051 [03:54<20:04,  2.12it/s, est. speed input: 69536.31 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 499/3051 [03:55<20:04,  2.12it/s, est. speed input: 69535.79 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 500/3051 [03:55<20:03,  2.12it/s, est. speed input: 69535.71 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 501/3051 [03:56<20:02,  2.12it/s, est. speed input: 69535.75 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 502/3051 [03:56<20:04,  2.12it/s, est. speed input: 69534.74 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 503/3051 [03:57<20:03,  2.12it/s, est. speed input: 69534.42 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 504/3051 [03:57<20:02,  2.12it/s, est. speed input: 69534.32 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 505/3051 [03:57<20:01,  2.12it/s, est. speed input: 69534.31 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 506/3051 [03:58<20:00,  2.12it/s, est. speed input: 69534.23 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 507/3051 [03:58<19:59,  2.12it/s, est. speed input: 69534.45 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 508/3051 [03:59<19:58,  2.12it/s, est. speed input: 69534.37 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 509/3051 [03:59<20:02,  2.11it/s, est. speed input: 69532.88 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 510/3051 [04:00<20:01,  2.11it/s, est. speed input: 69532.44 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 511/3051 [04:00<19:58,  2.12it/s, est. speed input: 69532.78 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 512/3051 [04:01<19:58,  2.12it/s, est. speed input: 69532.51 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 513/3051 [04:01<19:58,  2.12it/s, est. speed input: 69532.13 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 514/3051 [04:02<19:56,  2.12it/s, est. speed input: 69532.26 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 515/3051 [04:02<19:55,  2.12it/s, est. speed input: 69532.36 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 516/3051 [04:03<19:57,  2.12it/s, est. speed input: 69531.56 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 517/3051 [04:03<19:56,  2.12it/s, est. speed input: 69531.35 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 518/3051 [04:04<19:55,  2.12it/s, est. speed input: 69531.40 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 519/3051 [04:04<19:54,  2.12it/s, est. speed input: 69531.28 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 520/3051 [04:05<19:55,  2.12it/s, est. speed input: 69530.65 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 521/3051 [04:05<19:53,  2.12it/s, est. speed input: 69530.75 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 522/3051 [04:06<19:54,  2.12it/s, est. speed input: 69530.05 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 523/3051 [04:06<19:54,  2.12it/s, est. speed input: 69529.74 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 524/3051 [04:06<19:53,  2.12it/s, est. speed input: 69529.61 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 525/3051 [04:07<19:51,  2.12it/s, est. speed input: 69529.74 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 526/3051 [04:07<19:51,  2.12it/s, est. speed input: 69529.55 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 527/3051 [04:08<19:49,  2.12it/s, est. speed input: 69529.79 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 528/3051 [04:08<19:49,  2.12it/s, est. speed input: 69529.81 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 529/3051 [04:09<19:48,  2.12it/s, est. speed input: 69530.00 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 530/3051 [04:09<19:47,  2.12it/s, est. speed input: 69530.12 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 531/3051 [04:10<19:47,  2.12it/s, est. speed input: 69530.03 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 532/3051 [04:10<19:47,  2.12it/s, est. speed input: 69529.71 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 533/3051 [04:11<19:47,  2.12it/s, est. speed input: 69529.55 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 534/3051 [04:11<19:46,  2.12it/s, est. speed input: 69529.54 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 535/3051 [04:12<19:47,  2.12it/s, est. speed input: 69529.17 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 536/3051 [04:12<19:49,  2.11it/s, est. speed input: 69528.09 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 537/3051 [04:13<19:47,  2.12it/s, est. speed input: 69528.29 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 538/3051 [04:13<19:47,  2.12it/s, est. speed input: 69527.75 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 539/3051 [04:14<19:47,  2.11it/s, est. speed input: 69527.07 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 540/3051 [04:14<19:46,  2.12it/s, est. speed input: 69527.02 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 541/3051 [04:14<19:44,  2.12it/s, est. speed input: 69527.19 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 542/3051 [04:15<19:44,  2.12it/s, est. speed input: 69526.73 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 543/3051 [04:15<19:45,  2.12it/s, est. speed input: 69526.13 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 544/3051 [04:16<19:43,  2.12it/s, est. speed input: 69526.29 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 545/3051 [04:16<19:44,  2.12it/s, est. speed input: 69525.56 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 546/3051 [04:17<19:43,  2.12it/s, est. speed input: 69525.29 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 547/3051 [04:17<19:42,  2.12it/s, est. speed input: 69525.32 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 548/3051 [04:18<19:40,  2.12it/s, est. speed input: 69525.54 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 549/3051 [04:18<19:40,  2.12it/s, est. speed input: 69525.20 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 550/3051 [04:19<19:40,  2.12it/s, est. speed input: 69525.12 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 551/3051 [04:19<19:39,  2.12it/s, est. speed input: 69525.10 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 552/3051 [04:20<19:40,  2.12it/s, est. speed input: 69524.33 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 553/3051 [04:20<19:38,  2.12it/s, est. speed input: 69524.50 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 554/3051 [04:21<19:37,  2.12it/s, est. speed input: 69524.75 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 555/3051 [04:21<19:36,  2.12it/s, est. speed input: 69524.77 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 556/3051 [04:22<19:35,  2.12it/s, est. speed input: 69524.84 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 557/3051 [04:22<19:35,  2.12it/s, est. speed input: 69524.89 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 558/3051 [04:22<19:38,  2.12it/s, est. speed input: 69523.69 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 559/3051 [04:23<19:37,  2.12it/s, est. speed input: 69523.49 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 560/3051 [04:23<19:35,  2.12it/s, est. speed input: 69523.52 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 561/3051 [04:24<19:38,  2.11it/s, est. speed input: 69522.26 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 562/3051 [04:24<19:35,  2.12it/s, est. speed input: 69522.48 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 563/3051 [04:25<19:35,  2.12it/s, est. speed input: 69522.15 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 564/3051 [04:25<19:34,  2.12it/s, est. speed input: 69522.03 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 565/3051 [04:26<19:33,  2.12it/s, est. speed input: 69522.10 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 566/3051 [04:26<19:32,  2.12it/s, est. speed input: 69522.10 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 567/3051 [04:27<19:31,  2.12it/s, est. speed input: 69521.99 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 568/3051 [04:27<19:33,  2.12it/s, est. speed input: 69521.27 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 569/3051 [04:28<19:31,  2.12it/s, est. speed input: 69521.29 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 570/3051 [04:28<19:30,  2.12it/s, est. speed input: 69521.37 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 571/3051 [04:29<19:29,  2.12it/s, est. speed input: 69521.40 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 572/3051 [04:29<19:28,  2.12it/s, est. speed input: 69521.72 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 573/3051 [04:30<19:27,  2.12it/s, est. speed input: 69521.78 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 574/3051 [04:30<19:27,  2.12it/s, est. speed input: 69521.71 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 575/3051 [04:31<19:27,  2.12it/s, est. speed input: 69521.50 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 576/3051 [04:31<19:29,  2.12it/s, est. speed input: 69520.46 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 577/3051 [04:31<19:28,  2.12it/s, est. speed input: 69520.48 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 578/3051 [04:32<19:27,  2.12it/s, est. speed input: 69520.50 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 579/3051 [04:32<19:26,  2.12it/s, est. speed input: 69520.36 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 580/3051 [04:33<19:25,  2.12it/s, est. speed input: 69520.55 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 581/3051 [04:33<19:24,  2.12it/s, est. speed input: 69520.68 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 582/3051 [04:34<19:25,  2.12it/s, est. speed input: 69520.14 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 583/3051 [04:34<19:23,  2.12it/s, est. speed input: 69520.41 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 584/3051 [04:35<19:23,  2.12it/s, est. speed input: 69520.37 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 585/3051 [04:35<19:23,  2.12it/s, est. speed input: 69520.09 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 586/3051 [04:36<19:21,  2.12it/s, est. speed input: 69520.31 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 587/3051 [04:36<20:16,  2.03it/s, est. speed input: 69501.73 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 588/3051 [04:37<19:59,  2.05it/s, est. speed input: 69501.67 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 589/3051 [04:37<19:47,  2.07it/s, est. speed input: 69501.75 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 590/3051 [04:38<19:38,  2.09it/s, est. speed input: 69501.89 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 591/3051 [04:38<19:32,  2.10it/s, est. speed input: 69501.74 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 592/3051 [04:39<19:27,  2.11it/s, est. speed input: 69502.04 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 593/3051 [04:39<19:23,  2.11it/s, est. speed input: 69502.30 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 594/3051 [04:40<19:21,  2.11it/s, est. speed input: 69502.35 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 595/3051 [04:40<19:19,  2.12it/s, est. speed input: 69502.47 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 596/3051 [04:40<19:19,  2.12it/s, est. speed input: 69502.37 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 597/3051 [04:41<19:21,  2.11it/s, est. speed input: 69501.41 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 598/3051 [04:41<19:18,  2.12it/s, est. speed input: 69501.56 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 599/3051 [04:42<19:17,  2.12it/s, est. speed input: 69501.65 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 600/3051 [04:42<19:16,  2.12it/s, est. speed input: 69501.55 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 601/3051 [04:43<19:18,  2.12it/s, est. speed input: 69500.83 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 602/3051 [04:43<19:16,  2.12it/s, est. speed input: 69500.81 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 603/3051 [04:44<19:15,  2.12it/s, est. speed input: 69500.85 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 604/3051 [04:44<19:16,  2.11it/s, est. speed input: 69500.09 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 605/3051 [04:45<19:15,  2.12it/s, est. speed input: 69500.05 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 606/3051 [04:45<19:13,  2.12it/s, est. speed input: 69500.34 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 607/3051 [04:46<19:12,  2.12it/s, est. speed input: 69500.33 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 608/3051 [04:46<19:12,  2.12it/s, est. speed input: 69500.11 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 609/3051 [04:47<19:11,  2.12it/s, est. speed input: 69500.19 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 610/3051 [04:47<19:12,  2.12it/s, est. speed input: 69499.73 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 611/3051 [04:48<19:12,  2.12it/s, est. speed input: 69499.55 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 612/3051 [04:48<19:10,  2.12it/s, est. speed input: 69499.83 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 613/3051 [04:49<19:12,  2.11it/s, est. speed input: 69498.85 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 614/3051 [04:49<19:11,  2.12it/s, est. speed input: 69498.88 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 615/3051 [04:49<19:09,  2.12it/s, est. speed input: 69499.15 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 616/3051 [04:50<19:12,  2.11it/s, est. speed input: 69497.97 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 617/3051 [04:50<19:10,  2.11it/s, est. speed input: 69497.81 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 618/3051 [04:51<19:09,  2.12it/s, est. speed input: 69497.87 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 619/3051 [04:51<19:07,  2.12it/s, est. speed input: 69497.97 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 620/3051 [04:52<19:06,  2.12it/s, est. speed input: 69498.13 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 621/3051 [04:52<19:06,  2.12it/s, est. speed input: 69497.93 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 622/3051 [04:53<19:06,  2.12it/s, est. speed input: 69497.58 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 623/3051 [04:53<19:06,  2.12it/s, est. speed input: 69497.45 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 624/3051 [04:54<19:05,  2.12it/s, est. speed input: 69497.38 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 625/3051 [04:54<19:04,  2.12it/s, est. speed input: 69497.51 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 626/3051 [04:55<19:05,  2.12it/s, est. speed input: 69497.03 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 627/3051 [04:55<19:05,  2.12it/s, est. speed input: 69496.66 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 628/3051 [04:56<19:03,  2.12it/s, est. speed input: 69496.76 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 629/3051 [04:56<19:04,  2.12it/s, est. speed input: 69496.15 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 630/3051 [04:57<19:03,  2.12it/s, est. speed input: 69496.14 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 631/3051 [04:57<19:02,  2.12it/s, est. speed input: 69496.01 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 632/3051 [04:57<19:02,  2.12it/s, est. speed input: 69495.93 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 633/3051 [04:58<19:01,  2.12it/s, est. speed input: 69495.76 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 634/3051 [04:58<19:00,  2.12it/s, est. speed input: 69495.81 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 635/3051 [04:59<18:59,  2.12it/s, est. speed input: 69495.79 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 636/3051 [04:59<19:02,  2.11it/s, est. speed input: 69494.78 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 637/3051 [05:00<19:00,  2.12it/s, est. speed input: 69494.87 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 638/3051 [05:00<18:59,  2.12it/s, est. speed input: 69494.89 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 639/3051 [05:01<18:58,  2.12it/s, est. speed input: 69494.82 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 640/3051 [05:01<18:58,  2.12it/s, est. speed input: 69494.56 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 641/3051 [05:02<18:57,  2.12it/s, est. speed input: 69494.65 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 642/3051 [05:02<18:57,  2.12it/s, est. speed input: 69494.23 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 643/3051 [05:03<18:58,  2.12it/s, est. speed input: 69493.81 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 644/3051 [05:03<18:56,  2.12it/s, est. speed input: 69493.92 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 645/3051 [05:04<18:55,  2.12it/s, est. speed input: 69493.92 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 646/3051 [05:04<18:55,  2.12it/s, est. speed input: 69493.54 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 647/3051 [05:05<18:56,  2.12it/s, est. speed input: 69493.10 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 648/3051 [05:05<18:54,  2.12it/s, est. speed input: 69493.17 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 649/3051 [05:06<18:54,  2.12it/s, est. speed input: 69493.09 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 650/3051 [05:06<18:53,  2.12it/s, est. speed input: 69492.89 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 651/3051 [05:06<18:52,  2.12it/s, est. speed input: 69492.95 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 652/3051 [05:07<18:51,  2.12it/s, est. speed input: 69493.14 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 653/3051 [05:07<18:50,  2.12it/s, est. speed input: 69493.07 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 654/3051 [05:08<18:50,  2.12it/s, est. speed input: 69493.02 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 655/3051 [05:08<18:49,  2.12it/s, est. speed input: 69493.12 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 656/3051 [05:09<18:50,  2.12it/s, est. speed input: 69492.83 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 657/3051 [05:09<18:48,  2.12it/s, est. speed input: 69492.99 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 658/3051 [05:10<18:47,  2.12it/s, est. speed input: 69493.20 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 659/3051 [05:10<18:50,  2.12it/s, est. speed input: 69492.29 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 660/3051 [05:11<18:49,  2.12it/s, est. speed input: 69492.28 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 661/3051 [05:11<18:47,  2.12it/s, est. speed input: 69492.43 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 662/3051 [05:12<18:48,  2.12it/s, est. speed input: 69492.03 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 663/3051 [05:12<18:48,  2.12it/s, est. speed input: 69491.70 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 664/3051 [05:13<18:46,  2.12it/s, est. speed input: 69491.80 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 665/3051 [05:13<18:46,  2.12it/s, est. speed input: 69491.64 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 666/3051 [05:14<18:46,  2.12it/s, est. speed input: 69491.52 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 667/3051 [05:14<18:45,  2.12it/s, est. speed input: 69491.52 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 668/3051 [05:14<18:43,  2.12it/s, est. speed input: 69491.72 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 669/3051 [05:15<18:43,  2.12it/s, est. speed input: 69491.50 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 670/3051 [05:15<18:44,  2.12it/s, est. speed input: 69491.21 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 671/3051 [05:16<18:42,  2.12it/s, est. speed input: 69491.41 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 672/3051 [05:16<18:42,  2.12it/s, est. speed input: 69491.35 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 673/3051 [05:17<18:41,  2.12it/s, est. speed input: 69491.28 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 674/3051 [05:17<18:41,  2.12it/s, est. speed input: 69491.29 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 675/3051 [05:18<18:40,  2.12it/s, est. speed input: 69491.48 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 676/3051 [05:18<18:40,  2.12it/s, est. speed input: 69491.28 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 677/3051 [05:19<18:39,  2.12it/s, est. speed input: 69491.25 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 678/3051 [05:19<18:45,  2.11it/s, est. speed input: 69489.32 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 679/3051 [05:20<18:42,  2.11it/s, est. speed input: 69489.47 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 680/3051 [05:20<18:40,  2.12it/s, est. speed input: 69489.65 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 681/3051 [05:21<18:39,  2.12it/s, est. speed input: 69489.65 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 682/3051 [05:21<18:39,  2.12it/s, est. speed input: 69489.37 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 683/3051 [05:22<18:38,  2.12it/s, est. speed input: 69489.11 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 684/3051 [05:22<18:37,  2.12it/s, est. speed input: 69489.29 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 685/3051 [05:23<18:37,  2.12it/s, est. speed input: 69488.86 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 686/3051 [05:23<18:38,  2.11it/s, est. speed input: 69488.39 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 687/3051 [05:23<18:36,  2.12it/s, est. speed input: 69488.46 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 688/3051 [05:24<18:36,  2.12it/s, est. speed input: 69488.21 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 689/3051 [05:24<18:36,  2.12it/s, est. speed input: 69487.99 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 690/3051 [05:25<18:34,  2.12it/s, est. speed input: 69488.13 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 691/3051 [05:25<18:35,  2.12it/s, est. speed input: 69487.55 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 692/3051 [05:26<18:34,  2.12it/s, est. speed input: 69487.41 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 693/3051 [05:26<18:33,  2.12it/s, est. speed input: 69487.53 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 694/3051 [05:27<18:32,  2.12it/s, est. speed input: 69487.48 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 695/3051 [05:27<18:32,  2.12it/s, est. speed input: 69487.18 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 696/3051 [05:28<18:31,  2.12it/s, est. speed input: 69487.33 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 697/3051 [05:28<18:29,  2.12it/s, est. speed input: 69487.55 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 698/3051 [05:29<18:29,  2.12it/s, est. speed input: 69487.64 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 699/3051 [05:29<18:29,  2.12it/s, est. speed input: 69487.54 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 700/3051 [05:30<18:28,  2.12it/s, est. speed input: 69487.67 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 701/3051 [05:30<18:32,  2.11it/s, est. speed input: 69486.41 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 702/3051 [05:31<18:31,  2.11it/s, est. speed input: 69486.24 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 703/3051 [05:31<18:29,  2.12it/s, est. speed input: 69486.34 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 704/3051 [05:31<18:27,  2.12it/s, est. speed input: 69486.59 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 705/3051 [05:32<18:26,  2.12it/s, est. speed input: 69486.73 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 706/3051 [05:32<18:25,  2.12it/s, est. speed input: 69486.68 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 707/3051 [05:33<18:25,  2.12it/s, est. speed input: 69486.69 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 708/3051 [05:33<18:27,  2.11it/s, est. speed input: 69485.82 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 709/3051 [05:34<18:26,  2.12it/s, est. speed input: 69485.77 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 710/3051 [05:34<18:25,  2.12it/s, est. speed input: 69485.85 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 711/3051 [05:35<18:27,  2.11it/s, est. speed input: 69484.90 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 712/3051 [05:35<18:26,  2.11it/s, est. speed input: 69484.69 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 713/3051 [05:36<18:25,  2.12it/s, est. speed input: 69484.62 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 714/3051 [05:36<18:23,  2.12it/s, est. speed input: 69484.76 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 715/3051 [05:37<18:22,  2.12it/s, est. speed input: 69484.78 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 716/3051 [05:37<18:21,  2.12it/s, est. speed input: 69484.80 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 717/3051 [05:38<19:10,  2.03it/s, est. speed input: 69470.44 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 718/3051 [05:38<18:53,  2.06it/s, est. speed input: 69470.77 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 719/3051 [05:39<18:42,  2.08it/s, est. speed input: 69470.92 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 720/3051 [05:39<18:35,  2.09it/s, est. speed input: 69471.03 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 721/3051 [05:40<18:29,  2.10it/s, est. speed input: 69471.08 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 722/3051 [05:40<18:25,  2.11it/s, est. speed input: 69471.23 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 723/3051 [05:41<18:26,  2.10it/s, est. speed input: 69470.22 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 724/3051 [05:41<18:24,  2.11it/s, est. speed input: 69470.00 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 725/3051 [05:41<18:20,  2.11it/s, est. speed input: 69470.34 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 726/3051 [05:42<18:18,  2.12it/s, est. speed input: 69470.48 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 727/3051 [05:42<18:17,  2.12it/s, est. speed input: 69470.57 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 728/3051 [05:43<18:16,  2.12it/s, est. speed input: 69470.64 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 729/3051 [05:43<18:40,  2.07it/s, est. speed input: 69463.31 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 730/3051 [05:44<18:31,  2.09it/s, est. speed input: 69463.57 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 731/3051 [05:44<18:25,  2.10it/s, est. speed input: 69463.73 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 732/3051 [05:45<18:21,  2.11it/s, est. speed input: 69463.86 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 733/3051 [05:45<18:17,  2.11it/s, est. speed input: 69464.08 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 734/3051 [05:46<18:15,  2.11it/s, est. speed input: 69464.19 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 735/3051 [05:46<18:13,  2.12it/s, est. speed input: 69464.32 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 736/3051 [05:47<18:13,  2.12it/s, est. speed input: 69464.15 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 737/3051 [05:47<18:13,  2.12it/s, est. speed input: 69464.02 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 738/3051 [05:48<18:11,  2.12it/s, est. speed input: 69464.21 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 739/3051 [05:48<18:12,  2.12it/s, est. speed input: 69463.67 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 740/3051 [05:49<18:11,  2.12it/s, est. speed input: 69463.71 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 741/3051 [05:49<18:09,  2.12it/s, est. speed input: 69464.03 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 742/3051 [05:50<18:08,  2.12it/s, est. speed input: 69464.09 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 743/3051 [05:50<18:08,  2.12it/s, est. speed input: 69463.98 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 744/3051 [05:50<18:07,  2.12it/s, est. speed input: 69464.10 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 745/3051 [05:51<18:06,  2.12it/s, est. speed input: 69464.27 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 746/3051 [05:51<18:07,  2.12it/s, est. speed input: 69464.12 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 747/3051 [05:52<18:06,  2.12it/s, est. speed input: 69464.10 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 748/3051 [05:52<18:05,  2.12it/s, est. speed input: 69464.31 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 749/3051 [05:53<18:05,  2.12it/s, est. speed input: 69464.39 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 750/3051 [05:53<18:04,  2.12it/s, est. speed input: 69464.43 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 751/3051 [05:54<18:03,  2.12it/s, est. speed input: 69464.58 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 752/3051 [05:54<18:04,  2.12it/s, est. speed input: 69464.33 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 753/3051 [05:55<18:04,  2.12it/s, est. speed input: 69464.26 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 754/3051 [05:55<18:04,  2.12it/s, est. speed input: 69464.18 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 755/3051 [05:56<18:03,  2.12it/s, est. speed input: 69464.08 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 756/3051 [05:56<18:03,  2.12it/s, est. speed input: 69463.91 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 757/3051 [05:57<18:02,  2.12it/s, est. speed input: 69464.01 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 758/3051 [05:57<18:01,  2.12it/s, est. speed input: 69464.11 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 759/3051 [05:58<18:01,  2.12it/s, est. speed input: 69464.13 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 760/3051 [05:58<17:59,  2.12it/s, est. speed input: 69464.35 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 761/3051 [05:58<17:58,  2.12it/s, est. speed input: 69464.58 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 762/3051 [05:59<17:58,  2.12it/s, est. speed input: 69464.75 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 763/3051 [05:59<17:58,  2.12it/s, est. speed input: 69464.58 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 764/3051 [06:00<17:58,  2.12it/s, est. speed input: 69464.66 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 765/3051 [06:00<18:02,  2.11it/s, est. speed input: 69463.38 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 766/3051 [06:01<18:00,  2.12it/s, est. speed input: 69463.52 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 767/3051 [06:01<17:58,  2.12it/s, est. speed input: 69463.68 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 768/3051 [06:02<17:56,  2.12it/s, est. speed input: 69463.90 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 769/3051 [06:02<17:59,  2.11it/s, est. speed input: 69462.91 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 770/3051 [06:03<17:59,  2.11it/s, est. speed input: 69462.69 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 771/3051 [06:03<17:57,  2.12it/s, est. speed input: 69462.87 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 772/3051 [06:04<17:56,  2.12it/s, est. speed input: 69462.79 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 773/3051 [06:04<17:59,  2.11it/s, est. speed input: 69461.81 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 774/3051 [06:05<17:56,  2.12it/s, est. speed input: 69462.01 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 775/3051 [06:05<17:57,  2.11it/s, est. speed input: 69461.49 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 776/3051 [06:06<17:57,  2.11it/s, est. speed input: 69460.93 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 777/3051 [06:06<17:55,  2.11it/s, est. speed input: 69461.12 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 778/3051 [06:07<17:53,  2.12it/s, est. speed input: 69461.22 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 779/3051 [06:07<17:52,  2.12it/s, est. speed input: 69461.17 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 780/3051 [06:07<17:52,  2.12it/s, est. speed input: 69461.11 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 781/3051 [06:08<17:51,  2.12it/s, est. speed input: 69461.24 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 782/3051 [06:08<17:50,  2.12it/s, est. speed input: 69461.14 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 783/3051 [06:09<17:51,  2.12it/s, est. speed input: 69460.93 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 784/3051 [06:09<17:49,  2.12it/s, est. speed input: 69461.19 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 785/3051 [06:10<17:47,  2.12it/s, est. speed input: 69461.51 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 786/3051 [06:10<17:47,  2.12it/s, est. speed input: 69461.63 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 787/3051 [06:11<17:46,  2.12it/s, est. speed input: 69461.78 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 788/3051 [06:11<17:46,  2.12it/s, est. speed input: 69461.67 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 789/3051 [06:12<17:46,  2.12it/s, est. speed input: 69461.64 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 790/3051 [06:12<17:46,  2.12it/s, est. speed input: 69461.69 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 791/3051 [06:13<17:45,  2.12it/s, est. speed input: 69461.72 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 792/3051 [06:13<17:45,  2.12it/s, est. speed input: 69461.64 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 793/3051 [06:14<17:45,  2.12it/s, est. speed input: 69461.54 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 794/3051 [06:14<17:44,  2.12it/s, est. speed input: 69461.56 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 795/3051 [06:15<17:44,  2.12it/s, est. speed input: 69461.50 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 796/3051 [06:15<17:44,  2.12it/s, est. speed input: 69461.47 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 797/3051 [06:15<17:43,  2.12it/s, est. speed input: 69461.57 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 798/3051 [06:16<17:43,  2.12it/s, est. speed input: 69461.28 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 799/3051 [06:16<17:43,  2.12it/s, est. speed input: 69461.10 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 800/3051 [06:17<17:41,  2.12it/s, est. speed input: 69461.33 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 801/3051 [06:17<17:40,  2.12it/s, est. speed input: 69461.50 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 802/3051 [06:18<17:40,  2.12it/s, est. speed input: 69461.47 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 803/3051 [06:18<17:39,  2.12it/s, est. speed input: 69461.58 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 804/3051 [06:19<18:41,  2.00it/s, est. speed input: 69444.78 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 805/3051 [06:19<18:21,  2.04it/s, est. speed input: 69445.04 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 806/3051 [06:20<18:08,  2.06it/s, est. speed input: 69445.08 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 807/3051 [06:20<17:58,  2.08it/s, est. speed input: 69445.40 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 808/3051 [06:21<17:51,  2.09it/s, est. speed input: 69445.62 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 809/3051 [06:21<17:45,  2.10it/s, est. speed input: 69445.83 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 810/3051 [06:22<17:44,  2.11it/s, est. speed input: 69445.43 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 811/3051 [06:22<17:41,  2.11it/s, est. speed input: 69445.52 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 812/3051 [06:23<17:38,  2.11it/s, est. speed input: 69445.79 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 813/3051 [06:23<17:37,  2.12it/s, est. speed input: 69445.94 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 814/3051 [06:24<17:36,  2.12it/s, est. speed input: 69445.85 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 815/3051 [06:24<17:34,  2.12it/s, est. speed input: 69446.09 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 816/3051 [06:25<17:34,  2.12it/s, est. speed input: 69446.17 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 817/3051 [06:25<17:33,  2.12it/s, est. speed input: 69446.32 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 818/3051 [06:25<17:33,  2.12it/s, est. speed input: 69446.08 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 819/3051 [06:26<17:33,  2.12it/s, est. speed input: 69446.10 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 820/3051 [06:26<17:35,  2.11it/s, est. speed input: 69445.30 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 821/3051 [06:27<17:34,  2.12it/s, est. speed input: 69445.32 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 822/3051 [06:27<17:32,  2.12it/s, est. speed input: 69445.63 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 823/3051 [06:28<17:32,  2.12it/s, est. speed input: 69445.45 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 824/3051 [06:28<17:32,  2.11it/s, est. speed input: 69445.03 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 825/3051 [06:29<17:31,  2.12it/s, est. speed input: 69445.22 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 826/3051 [06:29<17:30,  2.12it/s, est. speed input: 69445.14 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 827/3051 [06:30<17:33,  2.11it/s, est. speed input: 69444.34 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 828/3051 [06:30<17:31,  2.11it/s, est. speed input: 69444.32 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 829/3051 [06:31<17:29,  2.12it/s, est. speed input: 69444.45 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 830/3051 [06:31<17:28,  2.12it/s, est. speed input: 69444.43 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 831/3051 [06:32<17:28,  2.12it/s, est. speed input: 69444.27 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 832/3051 [06:32<17:27,  2.12it/s, est. speed input: 69444.48 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 833/3051 [06:33<17:27,  2.12it/s, est. speed input: 69444.22 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 834/3051 [06:33<17:26,  2.12it/s, est. speed input: 69444.30 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 835/3051 [06:34<17:25,  2.12it/s, est. speed input: 69444.47 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 836/3051 [06:34<17:24,  2.12it/s, est. speed input: 69444.57 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 837/3051 [06:34<17:23,  2.12it/s, est. speed input: 69444.71 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 838/3051 [06:35<17:23,  2.12it/s, est. speed input: 69444.74 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 839/3051 [06:35<17:23,  2.12it/s, est. speed input: 69444.61 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 840/3051 [06:36<17:23,  2.12it/s, est. speed input: 69444.66 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 841/3051 [06:36<17:22,  2.12it/s, est. speed input: 69444.81 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 842/3051 [06:37<17:21,  2.12it/s, est. speed input: 69445.01 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 843/3051 [06:37<17:21,  2.12it/s, est. speed input: 69444.77 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 844/3051 [06:38<17:21,  2.12it/s, est. speed input: 69444.77 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 845/3051 [06:38<17:20,  2.12it/s, est. speed input: 69444.88 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 846/3051 [06:39<17:22,  2.11it/s, est. speed input: 69444.25 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 847/3051 [06:39<17:21,  2.12it/s, est. speed input: 69444.36 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 848/3051 [06:40<17:19,  2.12it/s, est. speed input: 69444.66 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 849/3051 [06:40<17:20,  2.12it/s, est. speed input: 69444.15 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 850/3051 [06:41<17:21,  2.11it/s, est. speed input: 69443.71 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 851/3051 [06:41<17:19,  2.12it/s, est. speed input: 69443.86 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 852/3051 [06:42<17:19,  2.12it/s, est. speed input: 69443.59 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 853/3051 [06:42<17:19,  2.12it/s, est. speed input: 69443.47 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 854/3051 [06:42<17:17,  2.12it/s, est. speed input: 69443.61 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 855/3051 [06:43<17:16,  2.12it/s, est. speed input: 69443.75 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 856/3051 [06:43<17:15,  2.12it/s, est. speed input: 69443.90 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 857/3051 [06:44<17:14,  2.12it/s, est. speed input: 69444.11 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 858/3051 [06:44<17:13,  2.12it/s, est. speed input: 69444.20 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 859/3051 [06:45<17:15,  2.12it/s, est. speed input: 69443.56 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 860/3051 [06:45<17:15,  2.12it/s, est. speed input: 69443.54 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 861/3051 [06:46<17:13,  2.12it/s, est. speed input: 69443.77 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 862/3051 [06:46<17:12,  2.12it/s, est. speed input: 69443.91 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 863/3051 [06:47<17:13,  2.12it/s, est. speed input: 69443.40 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 864/3051 [06:47<17:12,  2.12it/s, est. speed input: 69443.50 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 865/3051 [06:48<17:11,  2.12it/s, est. speed input: 69443.62 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 866/3051 [06:48<17:14,  2.11it/s, est. speed input: 69442.83 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 867/3051 [06:49<17:12,  2.12it/s, est. speed input: 69442.97 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 868/3051 [06:49<17:10,  2.12it/s, est. speed input: 69443.09 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 869/3051 [06:50<17:09,  2.12it/s, est. speed input: 69443.18 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 870/3051 [06:50<17:12,  2.11it/s, est. speed input: 69442.31 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 871/3051 [06:51<17:10,  2.12it/s, est. speed input: 69442.48 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 872/3051 [06:51<17:08,  2.12it/s, est. speed input: 69442.63 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 873/3051 [06:51<17:08,  2.12it/s, est. speed input: 69442.63 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 874/3051 [06:52<17:07,  2.12it/s, est. speed input: 69442.76 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 875/3051 [06:52<17:10,  2.11it/s, est. speed input: 69441.84 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 876/3051 [06:53<17:08,  2.11it/s, est. speed input: 69441.77 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 877/3051 [06:53<17:07,  2.12it/s, est. speed input: 69441.91 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 878/3051 [06:54<17:05,  2.12it/s, est. speed input: 69442.07 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 879/3051 [06:54<17:05,  2.12it/s, est. speed input: 69441.96 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 880/3051 [06:55<17:05,  2.12it/s, est. speed input: 69441.78 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 881/3051 [06:55<17:04,  2.12it/s, est. speed input: 69441.95 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 882/3051 [06:56<17:04,  2.12it/s, est. speed input: 69441.74 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 883/3051 [06:56<17:04,  2.12it/s, est. speed input: 69441.58 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 884/3051 [06:57<17:03,  2.12it/s, est. speed input: 69441.60 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 885/3051 [06:57<17:04,  2.11it/s, est. speed input: 69441.18 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 886/3051 [06:58<17:03,  2.12it/s, est. speed input: 69441.14 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 887/3051 [06:58<17:01,  2.12it/s, est. speed input: 69441.24 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 888/3051 [06:59<17:01,  2.12it/s, est. speed input: 69441.29 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 889/3051 [06:59<17:01,  2.12it/s, est. speed input: 69440.93 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 890/3051 [06:59<16:59,  2.12it/s, est. speed input: 69441.17 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 891/3051 [07:00<16:58,  2.12it/s, est. speed input: 69441.49 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 892/3051 [07:00<16:57,  2.12it/s, est. speed input: 69441.62 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 893/3051 [07:01<16:57,  2.12it/s, est. speed input: 69441.50 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 894/3051 [07:01<16:57,  2.12it/s, est. speed input: 69441.51 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 895/3051 [07:02<16:56,  2.12it/s, est. speed input: 69441.61 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 896/3051 [07:02<16:57,  2.12it/s, est. speed input: 69441.41 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 897/3051 [07:03<16:56,  2.12it/s, est. speed input: 69441.56 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 898/3051 [07:03<16:56,  2.12it/s, est. speed input: 69441.48 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 899/3051 [07:04<16:56,  2.12it/s, est. speed input: 69441.33 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 900/3051 [07:04<16:55,  2.12it/s, est. speed input: 69441.37 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 901/3051 [07:05<16:54,  2.12it/s, est. speed input: 69441.43 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 902/3051 [07:05<16:55,  2.12it/s, est. speed input: 69440.95 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 903/3051 [07:06<16:56,  2.11it/s, est. speed input: 69440.62 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 904/3051 [07:06<16:54,  2.12it/s, est. speed input: 69440.83 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 905/3051 [07:07<16:53,  2.12it/s, est. speed input: 69440.70 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 906/3051 [07:07<16:53,  2.12it/s, est. speed input: 69440.68 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 907/3051 [07:07<16:51,  2.12it/s, est. speed input: 69440.79 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 908/3051 [07:08<16:53,  2.11it/s, est. speed input: 69440.23 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 909/3051 [07:08<16:53,  2.11it/s, est. speed input: 69440.03 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 910/3051 [07:09<16:51,  2.12it/s, est. speed input: 69440.26 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 911/3051 [07:09<16:49,  2.12it/s, est. speed input: 69440.52 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 912/3051 [07:10<16:50,  2.12it/s, est. speed input: 69440.15 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 913/3051 [07:10<16:49,  2.12it/s, est. speed input: 69440.14 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 914/3051 [07:11<17:11,  2.07it/s, est. speed input: 69434.51 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 915/3051 [07:11<17:04,  2.09it/s, est. speed input: 69434.51 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 916/3051 [07:12<16:59,  2.10it/s, est. speed input: 69434.45 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 917/3051 [07:12<16:55,  2.10it/s, est. speed input: 69434.27 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 918/3051 [07:13<16:53,  2.10it/s, est. speed input: 69434.08 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 919/3051 [07:13<16:51,  2.11it/s, est. speed input: 69434.01 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 920/3051 [07:14<16:49,  2.11it/s, est. speed input: 69434.04 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 921/3051 [07:14<16:48,  2.11it/s, est. speed input: 69433.73 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 922/3051 [07:15<16:48,  2.11it/s, est. speed input: 69433.55 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 923/3051 [07:15<16:46,  2.11it/s, est. speed input: 69433.45 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 924/3051 [07:16<16:44,  2.12it/s, est. speed input: 69433.63 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 925/3051 [07:16<16:45,  2.11it/s, est. speed input: 69433.23 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 926/3051 [07:17<16:44,  2.11it/s, est. speed input: 69433.16 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 927/3051 [07:17<16:43,  2.12it/s, est. speed input: 69433.17 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 928/3051 [07:17<16:43,  2.12it/s, est. speed input: 69433.06 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 929/3051 [07:18<16:42,  2.12it/s, est. speed input: 69432.92 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 930/3051 [07:18<16:42,  2.12it/s, est. speed input: 69432.73 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 931/3051 [07:19<16:44,  2.11it/s, est. speed input: 69432.09 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 932/3051 [07:19<16:42,  2.11it/s, est. speed input: 69432.21 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 933/3051 [07:20<16:42,  2.11it/s, est. speed input: 69431.92 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 934/3051 [07:20<16:43,  2.11it/s, est. speed input: 69431.41 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 935/3051 [07:21<16:42,  2.11it/s, est. speed input: 69431.21 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 936/3051 [07:21<16:40,  2.11it/s, est. speed input: 69431.31 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 937/3051 [07:22<16:39,  2.12it/s, est. speed input: 69431.29 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 938/3051 [07:22<16:38,  2.12it/s, est. speed input: 69431.18 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 939/3051 [07:23<16:40,  2.11it/s, est. speed input: 69430.42 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 940/3051 [07:23<16:39,  2.11it/s, est. speed input: 69430.44 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 941/3051 [07:24<16:40,  2.11it/s, est. speed input: 69429.83 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 942/3051 [07:24<16:38,  2.11it/s, est. speed input: 69429.73 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 943/3051 [07:25<16:37,  2.11it/s, est. speed input: 69429.73 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 944/3051 [07:25<16:36,  2.11it/s, est. speed input: 69429.63 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 945/3051 [07:26<16:35,  2.12it/s, est. speed input: 69429.60 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 946/3051 [07:26<16:34,  2.12it/s, est. speed input: 69429.62 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 947/3051 [07:26<16:33,  2.12it/s, est. speed input: 69429.56 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 948/3051 [07:27<16:33,  2.12it/s, est. speed input: 69429.55 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 949/3051 [07:27<16:32,  2.12it/s, est. speed input: 69429.47 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 950/3051 [07:28<16:32,  2.12it/s, est. speed input: 69429.37 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 951/3051 [07:28<16:32,  2.12it/s, est. speed input: 69429.21 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 952/3051 [07:29<16:30,  2.12it/s, est. speed input: 69429.36 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 953/3051 [07:29<16:32,  2.11it/s, est. speed input: 69428.95 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 954/3051 [07:30<16:31,  2.12it/s, est. speed input: 69428.93 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 955/3051 [07:30<16:29,  2.12it/s, est. speed input: 69428.99 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 956/3051 [07:31<16:30,  2.12it/s, est. speed input: 69428.79 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 957/3051 [07:31<16:29,  2.12it/s, est. speed input: 69428.74 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 958/3051 [07:32<16:28,  2.12it/s, est. speed input: 69428.74 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 959/3051 [07:32<16:28,  2.12it/s, est. speed input: 69428.57 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 960/3051 [07:33<16:28,  2.11it/s, est. speed input: 69428.35 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 961/3051 [07:33<16:27,  2.12it/s, est. speed input: 69428.34 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 962/3051 [07:34<16:26,  2.12it/s, est. speed input: 69428.43 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 963/3051 [07:34<16:29,  2.11it/s, est. speed input: 69427.52 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 964/3051 [07:34<16:28,  2.11it/s, est. speed input: 69427.32 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 965/3051 [07:35<16:26,  2.11it/s, est. speed input: 69427.35 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 966/3051 [07:35<16:26,  2.11it/s, est. speed input: 69427.06 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 967/3051 [07:36<16:25,  2.11it/s, est. speed input: 69427.04 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 968/3051 [07:36<16:24,  2.12it/s, est. speed input: 69427.11 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 969/3051 [07:37<16:22,  2.12it/s, est. speed input: 69427.34 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 970/3051 [07:37<16:24,  2.11it/s, est. speed input: 69426.83 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 971/3051 [07:38<16:23,  2.12it/s, est. speed input: 69426.76 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 972/3051 [07:38<16:22,  2.12it/s, est. speed input: 69426.80 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 973/3051 [07:39<16:22,  2.12it/s, est. speed input: 69426.59 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 974/3051 [07:39<16:21,  2.12it/s, est. speed input: 69426.55 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 975/3051 [07:40<16:20,  2.12it/s, est. speed input: 69426.67 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 976/3051 [07:40<16:20,  2.12it/s, est. speed input: 69426.53 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 977/3051 [07:41<16:20,  2.12it/s, est. speed input: 69426.37 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 978/3051 [07:41<16:20,  2.12it/s, est. speed input: 69426.19 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 979/3051 [07:42<16:21,  2.11it/s, est. speed input: 69425.66 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 980/3051 [07:42<16:20,  2.11it/s, est. speed input: 69425.46 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 981/3051 [07:43<16:19,  2.11it/s, est. speed input: 69425.44 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 982/3051 [07:43<16:17,  2.12it/s, est. speed input: 69425.51 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 983/3051 [07:43<16:20,  2.11it/s, est. speed input: 69424.58 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 984/3051 [07:44<16:18,  2.11it/s, est. speed input: 69424.59 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 985/3051 [07:44<16:17,  2.11it/s, est. speed input: 69424.62 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 986/3051 [07:45<16:16,  2.11it/s, est. speed input: 69424.50 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 987/3051 [07:45<16:15,  2.12it/s, est. speed input: 69424.52 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 988/3051 [07:46<16:14,  2.12it/s, est. speed input: 69424.58 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 989/3051 [07:46<16:13,  2.12it/s, est. speed input: 69424.58 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 990/3051 [07:47<16:16,  2.11it/s, est. speed input: 69423.82 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 991/3051 [07:47<16:14,  2.11it/s, est. speed input: 69423.80 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 992/3051 [07:48<16:13,  2.11it/s, est. speed input: 69423.77 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 993/3051 [07:48<16:13,  2.12it/s, est. speed input: 69423.69 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 994/3051 [07:49<16:12,  2.12it/s, est. speed input: 69423.64 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 995/3051 [07:49<16:11,  2.12it/s, est. speed input: 69423.68 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 996/3051 [07:50<16:10,  2.12it/s, est. speed input: 69423.66 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 997/3051 [07:50<16:10,  2.12it/s, est. speed input: 69423.55 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 998/3051 [07:51<16:10,  2.12it/s, est. speed input: 69423.37 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 999/3051 [07:51<16:10,  2.11it/s, est. speed input: 69423.06 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1000/3051 [07:52<16:10,  2.11it/s, est. speed input: 69422.96 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1001/3051 [07:52<16:08,  2.12it/s, est. speed input: 69423.04 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1002/3051 [07:52<16:10,  2.11it/s, est. speed input: 69422.49 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1003/3051 [07:53<16:08,  2.11it/s, est. speed input: 69422.52 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1004/3051 [07:53<16:07,  2.12it/s, est. speed input: 69422.52 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1005/3051 [07:54<16:10,  2.11it/s, est. speed input: 69421.56 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1006/3051 [07:54<16:08,  2.11it/s, est. speed input: 69421.65 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1007/3051 [07:55<16:06,  2.12it/s, est. speed input: 69421.78 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1008/3051 [07:55<16:05,  2.12it/s, est. speed input: 69421.77 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1009/3051 [07:56<16:04,  2.12it/s, est. speed input: 69421.82 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1010/3051 [07:56<16:04,  2.12it/s, est. speed input: 69421.76 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1011/3051 [07:57<16:04,  2.11it/s, est. speed input: 69421.45 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1012/3051 [07:57<16:03,  2.12it/s, est. speed input: 69421.41 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1013/3051 [07:58<16:03,  2.12it/s, est. speed input: 69421.32 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1014/3051 [07:58<16:02,  2.12it/s, est. speed input: 69421.25 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1015/3051 [07:59<16:01,  2.12it/s, est. speed input: 69421.23 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1016/3051 [07:59<16:00,  2.12it/s, est. speed input: 69421.32 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1017/3051 [08:00<16:00,  2.12it/s, est. speed input: 69421.25 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1018/3051 [08:00<16:02,  2.11it/s, est. speed input: 69420.73 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1019/3051 [08:00<16:00,  2.12it/s, est. speed input: 69420.87 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1020/3051 [08:01<15:59,  2.12it/s, est. speed input: 69420.89 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1021/3051 [08:01<16:01,  2.11it/s, est. speed input: 69420.31 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1022/3051 [08:02<16:00,  2.11it/s, est. speed input: 69420.18 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1023/3051 [08:02<15:58,  2.11it/s, est. speed input: 69420.22 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1024/3051 [08:03<15:57,  2.12it/s, est. speed input: 69420.38 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1025/3051 [08:03<15:56,  2.12it/s, est. speed input: 69420.36 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1026/3051 [08:04<15:56,  2.12it/s, est. speed input: 69420.35 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1027/3051 [08:04<16:00,  2.11it/s, est. speed input: 69419.25 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1028/3051 [08:05<15:57,  2.11it/s, est. speed input: 69419.35 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1029/3051 [08:05<15:55,  2.12it/s, est. speed input: 69419.58 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1030/3051 [08:06<15:54,  2.12it/s, est. speed input: 69419.69 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1031/3051 [08:06<15:56,  2.11it/s, est. speed input: 69418.98 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1032/3051 [08:07<15:54,  2.12it/s, est. speed input: 69419.19 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1033/3051 [08:07<15:53,  2.12it/s, est. speed input: 69419.31 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1034/3051 [08:08<15:52,  2.12it/s, est. speed input: 69419.19 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1035/3051 [08:08<15:53,  2.11it/s, est. speed input: 69418.86 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1036/3051 [08:09<15:52,  2.12it/s, est. speed input: 69418.93 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1037/3051 [08:09<15:53,  2.11it/s, est. speed input: 69418.50 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1038/3051 [08:09<15:53,  2.11it/s, est. speed input: 69418.24 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1039/3051 [08:10<15:50,  2.12it/s, est. speed input: 69418.46 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1040/3051 [08:10<15:49,  2.12it/s, est. speed input: 69418.57 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1041/3051 [08:11<15:49,  2.12it/s, est. speed input: 69418.44 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1042/3051 [08:11<15:49,  2.12it/s, est. speed input: 69418.30 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1043/3051 [08:12<15:48,  2.12it/s, est. speed input: 69418.36 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1044/3051 [08:12<15:49,  2.11it/s, est. speed input: 69418.06 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1045/3051 [08:13<15:48,  2.11it/s, est. speed input: 69417.96 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1046/3051 [08:13<15:47,  2.12it/s, est. speed input: 69418.08 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1047/3051 [08:14<15:45,  2.12it/s, est. speed input: 69418.31 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1048/3051 [08:14<15:45,  2.12it/s, est. speed input: 69418.26 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1049/3051 [08:15<15:44,  2.12it/s, est. speed input: 69418.24 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1050/3051 [08:15<15:44,  2.12it/s, est. speed input: 69418.27 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1051/3051 [08:16<15:42,  2.12it/s, est. speed input: 69418.51 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1052/3051 [08:16<15:42,  2.12it/s, est. speed input: 69418.60 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1053/3051 [08:17<15:41,  2.12it/s, est. speed input: 69418.74 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1054/3051 [08:17<15:41,  2.12it/s, est. speed input: 69418.81 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1055/3051 [08:17<15:43,  2.12it/s, est. speed input: 69418.31 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1056/3051 [08:18<15:42,  2.12it/s, est. speed input: 69418.30 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1057/3051 [08:18<15:41,  2.12it/s, est. speed input: 69418.34 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1058/3051 [08:19<15:41,  2.12it/s, est. speed input: 69418.30 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1059/3051 [08:19<15:40,  2.12it/s, est. speed input: 69418.35 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1060/3051 [08:20<15:41,  2.11it/s, est. speed input: 69418.01 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1061/3051 [08:20<15:40,  2.12it/s, est. speed input: 69417.98 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1062/3051 [08:21<15:39,  2.12it/s, est. speed input: 69418.01 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1063/3051 [08:21<15:40,  2.11it/s, est. speed input: 69417.69 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1064/3051 [08:22<15:39,  2.12it/s, est. speed input: 69417.69 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1065/3051 [08:22<15:38,  2.12it/s, est. speed input: 69417.73 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1066/3051 [08:23<15:37,  2.12it/s, est. speed input: 69417.84 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1067/3051 [08:23<15:36,  2.12it/s, est. speed input: 69417.78 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1068/3051 [08:24<15:36,  2.12it/s, est. speed input: 69417.78 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1069/3051 [08:24<15:35,  2.12it/s, est. speed input: 69417.85 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1070/3051 [08:25<15:36,  2.11it/s, est. speed input: 69417.41 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1071/3051 [08:25<15:35,  2.12it/s, est. speed input: 69417.43 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1072/3051 [08:26<15:34,  2.12it/s, est. speed input: 69417.59 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1073/3051 [08:26<15:33,  2.12it/s, est. speed input: 69417.59 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1074/3051 [08:26<15:35,  2.11it/s, est. speed input: 69417.12 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1075/3051 [08:27<15:34,  2.11it/s, est. speed input: 69417.10 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1076/3051 [08:27<15:33,  2.12it/s, est. speed input: 69417.16 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1077/3051 [08:28<15:35,  2.11it/s, est. speed input: 69416.55 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1078/3051 [08:28<15:33,  2.11it/s, est. speed input: 69416.58 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1079/3051 [08:29<15:45,  2.09it/s, est. speed input: 69413.66 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1080/3051 [08:29<15:39,  2.10it/s, est. speed input: 69413.89 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1081/3051 [08:30<15:35,  2.11it/s, est. speed input: 69414.00 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1082/3051 [08:30<15:33,  2.11it/s, est. speed input: 69414.01 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1083/3051 [08:31<15:31,  2.11it/s, est. speed input: 69414.06 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1084/3051 [08:31<15:30,  2.11it/s, est. speed input: 69413.91 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1085/3051 [08:32<15:29,  2.12it/s, est. speed input: 69414.04 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1086/3051 [08:32<15:29,  2.11it/s, est. speed input: 69413.85 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1087/3051 [08:33<15:28,  2.11it/s, est. speed input: 69413.76 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1088/3051 [08:33<15:27,  2.12it/s, est. speed input: 69413.85 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1089/3051 [08:34<15:29,  2.11it/s, est. speed input: 69413.25 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1090/3051 [08:34<15:28,  2.11it/s, est. speed input: 69413.13 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1091/3051 [08:35<15:27,  2.11it/s, est. speed input: 69413.13 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1092/3051 [08:35<15:26,  2.12it/s, est. speed input: 69413.14 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1093/3051 [08:35<15:24,  2.12it/s, est. speed input: 69413.23 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1094/3051 [08:36<15:23,  2.12it/s, est. speed input: 69413.32 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1095/3051 [08:36<15:23,  2.12it/s, est. speed input: 69413.39 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1096/3051 [08:37<15:23,  2.12it/s, est. speed input: 69413.23 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1097/3051 [08:37<15:22,  2.12it/s, est. speed input: 69413.28 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1098/3051 [08:38<15:21,  2.12it/s, est. speed input: 69413.39 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1099/3051 [08:38<15:20,  2.12it/s, est. speed input: 69413.47 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1100/3051 [08:39<15:22,  2.11it/s, est. speed input: 69413.03 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1101/3051 [08:39<15:21,  2.12it/s, est. speed input: 69413.07 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1102/3051 [08:40<15:20,  2.12it/s, est. speed input: 69413.15 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1103/3051 [08:40<15:19,  2.12it/s, est. speed input: 69413.11 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1104/3051 [08:41<15:18,  2.12it/s, est. speed input: 69413.28 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1105/3051 [08:41<15:22,  2.11it/s, est. speed input: 69412.40 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1106/3051 [08:42<15:21,  2.11it/s, est. speed input: 69412.28 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1107/3051 [08:42<15:19,  2.11it/s, est. speed input: 69412.40 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1108/3051 [08:43<15:17,  2.12it/s, est. speed input: 69412.52 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1109/3051 [08:43<15:17,  2.12it/s, est. speed input: 69412.51 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1110/3051 [08:44<15:18,  2.11it/s, est. speed input: 69412.00 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1111/3051 [08:44<15:17,  2.12it/s, est. speed input: 69412.10 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1112/3051 [08:44<15:15,  2.12it/s, est. speed input: 69412.25 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1113/3051 [08:45<15:15,  2.12it/s, est. speed input: 69412.19 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1114/3051 [08:45<15:14,  2.12it/s, est. speed input: 69412.22 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1115/3051 [08:46<15:16,  2.11it/s, est. speed input: 69411.68 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1116/3051 [08:46<15:15,  2.11it/s, est. speed input: 69411.66 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1117/3051 [08:47<15:13,  2.12it/s, est. speed input: 69411.76 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1118/3051 [08:47<15:12,  2.12it/s, est. speed input: 69411.89 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1119/3051 [08:48<15:14,  2.11it/s, est. speed input: 69411.35 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1120/3051 [08:48<15:13,  2.11it/s, est. speed input: 69411.39 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1121/3051 [08:49<15:12,  2.12it/s, est. speed input: 69411.36 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1122/3051 [08:49<15:11,  2.12it/s, est. speed input: 69411.43 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1123/3051 [08:50<15:10,  2.12it/s, est. speed input: 69411.41 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1124/3051 [08:50<15:09,  2.12it/s, est. speed input: 69411.50 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1125/3051 [08:51<15:08,  2.12it/s, est. speed input: 69411.60 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1126/3051 [08:51<15:08,  2.12it/s, est. speed input: 69411.52 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1127/3051 [08:52<15:08,  2.12it/s, est. speed input: 69411.54 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1128/3051 [08:52<15:07,  2.12it/s, est. speed input: 69411.67 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1129/3051 [08:52<15:06,  2.12it/s, est. speed input: 69411.68 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1130/3051 [08:53<15:05,  2.12it/s, est. speed input: 69411.81 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1131/3051 [08:53<15:05,  2.12it/s, est. speed input: 69411.96 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1132/3051 [08:54<15:04,  2.12it/s, est. speed input: 69412.03 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1133/3051 [08:54<15:06,  2.12it/s, est. speed input: 69411.69 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1134/3051 [08:55<15:05,  2.12it/s, est. speed input: 69411.57 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1135/3051 [08:55<15:05,  2.12it/s, est. speed input: 69411.59 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1136/3051 [08:56<15:04,  2.12it/s, est. speed input: 69411.66 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1137/3051 [08:56<15:03,  2.12it/s, est. speed input: 69411.76 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1138/3051 [08:57<15:04,  2.12it/s, est. speed input: 69411.40 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1139/3051 [08:57<15:03,  2.12it/s, est. speed input: 69411.42 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1140/3051 [08:58<15:02,  2.12it/s, est. speed input: 69411.52 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1141/3051 [08:58<15:02,  2.12it/s, est. speed input: 69411.32 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1142/3051 [08:59<15:01,  2.12it/s, est. speed input: 69411.31 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1143/3051 [08:59<15:01,  2.12it/s, est. speed input: 69411.35 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1144/3051 [09:00<15:00,  2.12it/s, est. speed input: 69411.31 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1145/3051 [09:00<15:00,  2.12it/s, est. speed input: 69411.31 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1146/3051 [09:01<14:59,  2.12it/s, est. speed input: 69411.36 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1147/3051 [09:01<14:58,  2.12it/s, est. speed input: 69411.39 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1148/3051 [09:01<14:59,  2.12it/s, est. speed input: 69411.19 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1149/3051 [09:02<14:59,  2.12it/s, est. speed input: 69411.03 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1150/3051 [09:02<14:58,  2.12it/s, est. speed input: 69411.09 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1151/3051 [09:03<14:57,  2.12it/s, est. speed input: 69411.15 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1152/3051 [09:03<14:56,  2.12it/s, est. speed input: 69411.12 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1153/3051 [09:04<14:55,  2.12it/s, est. speed input: 69411.19 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1154/3051 [09:04<14:54,  2.12it/s, est. speed input: 69411.35 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1155/3051 [09:05<14:54,  2.12it/s, est. speed input: 69411.45 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1156/3051 [09:05<14:54,  2.12it/s, est. speed input: 69411.34 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1157/3051 [09:06<14:54,  2.12it/s, est. speed input: 69411.26 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1158/3051 [09:06<14:54,  2.12it/s, est. speed input: 69411.20 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1159/3051 [09:07<14:53,  2.12it/s, est. speed input: 69411.26 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1160/3051 [09:07<14:52,  2.12it/s, est. speed input: 69411.35 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1161/3051 [09:08<14:52,  2.12it/s, est. speed input: 69411.13 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1162/3051 [09:08<14:52,  2.12it/s, est. speed input: 69411.13 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1163/3051 [09:09<14:51,  2.12it/s, est. speed input: 69411.26 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1164/3051 [09:09<14:51,  2.12it/s, est. speed input: 69411.15 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1165/3051 [09:09<14:50,  2.12it/s, est. speed input: 69411.07 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1166/3051 [09:10<14:49,  2.12it/s, est. speed input: 69411.14 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1167/3051 [09:10<14:57,  2.10it/s, est. speed input: 69409.30 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1168/3051 [09:11<14:54,  2.11it/s, est. speed input: 69409.39 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1169/3051 [09:11<14:51,  2.11it/s, est. speed input: 69409.48 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1170/3051 [09:12<14:49,  2.11it/s, est. speed input: 69409.58 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1171/3051 [09:12<14:48,  2.12it/s, est. speed input: 69409.59 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1172/3051 [09:13<14:47,  2.12it/s, est. speed input: 69409.63 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1173/3051 [09:13<14:46,  2.12it/s, est. speed input: 69409.73 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1174/3051 [09:14<14:45,  2.12it/s, est. speed input: 69409.92 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1175/3051 [09:14<14:45,  2.12it/s, est. speed input: 69409.92 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1176/3051 [09:15<14:44,  2.12it/s, est. speed input: 69409.98 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1177/3051 [09:15<14:44,  2.12it/s, est. speed input: 69410.02 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1178/3051 [09:16<14:43,  2.12it/s, est. speed input: 69410.03 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1179/3051 [09:16<14:42,  2.12it/s, est. speed input: 69410.21 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1180/3051 [09:17<14:47,  2.11it/s, est. speed input: 69409.19 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1181/3051 [09:17<14:44,  2.11it/s, est. speed input: 69409.42 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1182/3051 [09:18<14:42,  2.12it/s, est. speed input: 69409.57 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1183/3051 [09:18<14:42,  2.12it/s, est. speed input: 69409.57 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1184/3051 [09:18<14:42,  2.12it/s, est. speed input: 69409.47 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1185/3051 [09:19<14:43,  2.11it/s, est. speed input: 69409.03 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1186/3051 [09:19<14:42,  2.11it/s, est. speed input: 69409.03 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1187/3051 [09:20<14:40,  2.12it/s, est. speed input: 69409.14 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1188/3051 [09:20<14:39,  2.12it/s, est. speed input: 69409.15 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1189/3051 [09:21<14:39,  2.12it/s, est. speed input: 69409.17 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1190/3051 [09:21<14:40,  2.11it/s, est. speed input: 69408.79 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1191/3051 [09:22<14:39,  2.11it/s, est. speed input: 69408.72 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1192/3051 [09:22<14:38,  2.12it/s, est. speed input: 69408.81 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1193/3051 [09:23<14:37,  2.12it/s, est. speed input: 69408.81 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1194/3051 [09:23<14:37,  2.12it/s, est. speed input: 69408.68 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1195/3051 [09:24<14:37,  2.12it/s, est. speed input: 69408.61 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1196/3051 [09:24<14:35,  2.12it/s, est. speed input: 69408.72 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1197/3051 [09:25<14:36,  2.12it/s, est. speed input: 69408.51 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1198/3051 [09:25<14:36,  2.11it/s, est. speed input: 69408.39 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1199/3051 [09:26<14:34,  2.12it/s, est. speed input: 69408.63 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1200/3051 [09:26<14:33,  2.12it/s, est. speed input: 69408.71 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1201/3051 [09:26<14:32,  2.12it/s, est. speed input: 69408.78 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1202/3051 [09:27<14:31,  2.12it/s, est. speed input: 69408.91 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1203/3051 [09:27<14:32,  2.12it/s, est. speed input: 69408.70 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1204/3051 [09:28<14:32,  2.12it/s, est. speed input: 69408.69 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1205/3051 [09:28<14:31,  2.12it/s, est. speed input: 69408.69 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1206/3051 [09:29<14:30,  2.12it/s, est. speed input: 69408.82 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1207/3051 [09:29<14:30,  2.12it/s, est. speed input: 69408.86 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1208/3051 [09:30<14:30,  2.12it/s, est. speed input: 69408.76 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1209/3051 [09:30<14:29,  2.12it/s, est. speed input: 69408.74 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1210/3051 [09:31<14:29,  2.12it/s, est. speed input: 69408.71 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1211/3051 [09:31<14:29,  2.12it/s, est. speed input: 69408.57 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1212/3051 [09:32<14:28,  2.12it/s, est. speed input: 69408.73 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1213/3051 [09:32<14:28,  2.12it/s, est. speed input: 69408.61 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1214/3051 [09:33<14:27,  2.12it/s, est. speed input: 69408.52 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1215/3051 [09:33<14:26,  2.12it/s, est. speed input: 69408.66 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1216/3051 [09:34<14:26,  2.12it/s, est. speed input: 69408.61 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1217/3051 [09:34<14:25,  2.12it/s, est. speed input: 69408.68 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1218/3051 [09:35<14:25,  2.12it/s, est. speed input: 69408.72 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1219/3051 [09:35<14:27,  2.11it/s, est. speed input: 69407.99 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1220/3051 [09:35<14:25,  2.12it/s, est. speed input: 69408.22 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1221/3051 [09:36<14:24,  2.12it/s, est. speed input: 69408.25 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1222/3051 [09:36<14:24,  2.12it/s, est. speed input: 69408.07 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1223/3051 [09:37<14:24,  2.11it/s, est. speed input: 69407.86 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1224/3051 [09:37<14:23,  2.12it/s, est. speed input: 69407.91 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1225/3051 [09:38<14:22,  2.12it/s, est. speed input: 69408.03 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1226/3051 [09:38<14:22,  2.12it/s, est. speed input: 69407.91 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1227/3051 [09:39<14:21,  2.12it/s, est. speed input: 69407.84 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1228/3051 [09:39<14:21,  2.12it/s, est. speed input: 69407.87 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1229/3051 [09:40<14:21,  2.11it/s, est. speed input: 69407.64 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1230/3051 [09:40<14:21,  2.11it/s, est. speed input: 69407.55 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1231/3051 [09:41<14:19,  2.12it/s, est. speed input: 69407.69 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1232/3051 [09:41<14:21,  2.11it/s, est. speed input: 69407.15 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1233/3051 [09:42<14:21,  2.11it/s, est. speed input: 69406.90 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1234/3051 [09:42<14:19,  2.11it/s, est. speed input: 69407.03 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1235/3051 [09:43<14:17,  2.12it/s, est. speed input: 69407.23 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1236/3051 [09:43<14:17,  2.12it/s, est. speed input: 69407.16 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1237/3051 [09:44<14:18,  2.11it/s, est. speed input: 69406.80 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1238/3051 [09:44<14:16,  2.12it/s, est. speed input: 69406.95 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1239/3051 [09:44<14:16,  2.12it/s, est. speed input: 69406.85 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1240/3051 [09:45<14:15,  2.12it/s, est. speed input: 69406.81 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1241/3051 [09:45<14:14,  2.12it/s, est. speed input: 69406.86 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1242/3051 [09:46<14:14,  2.12it/s, est. speed input: 69406.86 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1243/3051 [09:46<14:14,  2.12it/s, est. speed input: 69406.75 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1244/3051 [09:47<14:13,  2.12it/s, est. speed input: 69406.76 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1245/3051 [09:47<14:12,  2.12it/s, est. speed input: 69406.89 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1246/3051 [09:48<14:11,  2.12it/s, est. speed input: 69407.00 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1247/3051 [09:48<14:10,  2.12it/s, est. speed input: 69407.18 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1248/3051 [09:49<14:09,  2.12it/s, est. speed input: 69407.28 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1249/3051 [09:49<14:09,  2.12it/s, est. speed input: 69407.26 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1250/3051 [09:50<14:09,  2.12it/s, est. speed input: 69407.24 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1251/3051 [09:50<14:09,  2.12it/s, est. speed input: 69407.29 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1252/3051 [09:51<14:11,  2.11it/s, est. speed input: 69406.80 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1253/3051 [09:51<14:10,  2.11it/s, est. speed input: 69406.66 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1254/3051 [09:52<14:09,  2.11it/s, est. speed input: 69406.65 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1255/3051 [09:52<14:14,  2.10it/s, est. speed input: 69405.36 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1256/3051 [09:52<14:12,  2.11it/s, est. speed input: 69405.41 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1257/3051 [09:53<14:09,  2.11it/s, est. speed input: 69405.55 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1258/3051 [09:53<14:07,  2.12it/s, est. speed input: 69405.69 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1259/3051 [09:54<14:06,  2.12it/s, est. speed input: 69405.67 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1260/3051 [09:54<14:05,  2.12it/s, est. speed input: 69405.84 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1261/3051 [09:55<14:04,  2.12it/s, est. speed input: 69406.04 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1262/3051 [09:55<14:04,  2.12it/s, est. speed input: 69405.94 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1263/3051 [09:56<14:04,  2.12it/s, est. speed input: 69405.93 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1264/3051 [09:56<14:03,  2.12it/s, est. speed input: 69405.98 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1265/3051 [09:57<14:05,  2.11it/s, est. speed input: 69405.43 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1266/3051 [09:57<14:04,  2.11it/s, est. speed input: 69405.45 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1267/3051 [09:58<14:02,  2.12it/s, est. speed input: 69405.63 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1268/3051 [09:58<14:02,  2.12it/s, est. speed input: 69405.55 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1269/3051 [09:59<14:01,  2.12it/s, est. speed input: 69405.54 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1270/3051 [09:59<14:01,  2.12it/s, est. speed input: 69405.47 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1271/3051 [10:00<14:00,  2.12it/s, est. speed input: 69405.53 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1272/3051 [10:00<13:59,  2.12it/s, est. speed input: 69405.62 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1273/3051 [10:01<13:59,  2.12it/s, est. speed input: 69405.60 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1274/3051 [10:01<13:58,  2.12it/s, est. speed input: 69405.58 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1275/3051 [10:01<13:58,  2.12it/s, est. speed input: 69405.58 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1276/3051 [10:02<13:58,  2.12it/s, est. speed input: 69405.58 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1277/3051 [10:02<13:57,  2.12it/s, est. speed input: 69405.65 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1278/3051 [10:03<13:56,  2.12it/s, est. speed input: 69405.72 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1279/3051 [10:03<13:56,  2.12it/s, est. speed input: 69405.71 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1280/3051 [10:04<13:55,  2.12it/s, est. speed input: 69405.70 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1281/3051 [10:04<13:55,  2.12it/s, est. speed input: 69405.73 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1282/3051 [10:05<13:54,  2.12it/s, est. speed input: 69405.74 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1283/3051 [10:05<13:54,  2.12it/s, est. speed input: 69405.84 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1284/3051 [10:06<13:53,  2.12it/s, est. speed input: 69405.83 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1285/3051 [10:06<13:53,  2.12it/s, est. speed input: 69405.77 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1286/3051 [10:07<13:52,  2.12it/s, est. speed input: 69405.85 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1287/3051 [10:07<13:51,  2.12it/s, est. speed input: 69405.99 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1288/3051 [10:08<13:52,  2.12it/s, est. speed input: 69405.75 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1289/3051 [10:08<13:51,  2.12it/s, est. speed input: 69405.87 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1290/3051 [10:09<13:51,  2.12it/s, est. speed input: 69405.81 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1291/3051 [10:09<13:52,  2.11it/s, est. speed input: 69405.51 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1292/3051 [10:09<13:50,  2.12it/s, est. speed input: 69405.66 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1293/3051 [10:10<13:50,  2.12it/s, est. speed input: 69405.67 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1294/3051 [10:10<13:49,  2.12it/s, est. speed input: 69405.62 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1295/3051 [10:11<13:49,  2.12it/s, est. speed input: 69405.51 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1296/3051 [10:11<13:48,  2.12it/s, est. speed input: 69405.56 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1297/3051 [10:12<13:48,  2.12it/s, est. speed input: 69405.46 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1298/3051 [10:12<13:49,  2.11it/s, est. speed input: 69405.20 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1299/3051 [10:13<13:48,  2.11it/s, est. speed input: 69405.16 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1300/3051 [10:13<13:47,  2.12it/s, est. speed input: 69405.22 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1301/3051 [10:14<13:46,  2.12it/s, est. speed input: 69405.27 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1302/3051 [10:14<13:49,  2.11it/s, est. speed input: 69404.59 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1303/3051 [10:15<13:47,  2.11it/s, est. speed input: 69404.61 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1304/3051 [10:15<13:46,  2.11it/s, est. speed input: 69404.68 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1305/3051 [10:16<13:45,  2.12it/s, est. speed input: 69404.67 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1306/3051 [10:16<13:45,  2.11it/s, est. speed input: 69404.53 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1307/3051 [10:17<13:44,  2.12it/s, est. speed input: 69404.50 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1308/3051 [10:17<13:43,  2.12it/s, est. speed input: 69404.46 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1309/3051 [10:18<13:42,  2.12it/s, est. speed input: 69404.54 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1310/3051 [10:18<13:41,  2.12it/s, est. speed input: 69404.64 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1311/3051 [10:18<13:41,  2.12it/s, est. speed input: 69404.64 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1312/3051 [10:19<13:41,  2.12it/s, est. speed input: 69404.53 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1313/3051 [10:19<13:40,  2.12it/s, est. speed input: 69404.68 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1314/3051 [10:20<13:39,  2.12it/s, est. speed input: 69404.65 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1315/3051 [10:20<13:39,  2.12it/s, est. speed input: 69404.62 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1316/3051 [10:21<13:38,  2.12it/s, est. speed input: 69404.75 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1317/3051 [10:21<13:38,  2.12it/s, est. speed input: 69404.71 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1318/3051 [10:22<13:37,  2.12it/s, est. speed input: 69404.76 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1319/3051 [10:22<13:37,  2.12it/s, est. speed input: 69404.75 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1320/3051 [10:23<13:36,  2.12it/s, est. speed input: 69404.93 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1321/3051 [10:23<13:35,  2.12it/s, est. speed input: 69405.03 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1322/3051 [10:24<13:35,  2.12it/s, est. speed input: 69405.08 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1323/3051 [10:24<13:35,  2.12it/s, est. speed input: 69405.09 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1324/3051 [10:25<13:34,  2.12it/s, est. speed input: 69405.22 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1325/3051 [10:25<13:34,  2.12it/s, est. speed input: 69405.06 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1326/3051 [10:26<13:34,  2.12it/s, est. speed input: 69405.09 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1327/3051 [10:26<13:35,  2.11it/s, est. speed input: 69404.70 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1328/3051 [10:26<13:34,  2.11it/s, est. speed input: 69404.63 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1329/3051 [10:27<13:33,  2.12it/s, est. speed input: 69404.79 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1330/3051 [10:27<13:33,  2.12it/s, est. speed input: 69404.73 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1331/3051 [10:28<13:32,  2.12it/s, est. speed input: 69404.69 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1332/3051 [10:28<13:32,  2.12it/s, est. speed input: 69404.67 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1333/3051 [10:29<13:31,  2.12it/s, est. speed input: 69404.72 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1334/3051 [10:29<13:30,  2.12it/s, est. speed input: 69404.69 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1335/3051 [10:30<13:30,  2.12it/s, est. speed input: 69404.61 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1336/3051 [10:30<13:29,  2.12it/s, est. speed input: 69404.72 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1337/3051 [10:31<13:29,  2.12it/s, est. speed input: 69404.72 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1338/3051 [10:31<13:29,  2.12it/s, est. speed input: 69404.50 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1339/3051 [10:32<13:28,  2.12it/s, est. speed input: 69404.52 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1340/3051 [10:32<13:28,  2.12it/s, est. speed input: 69404.46 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1341/3051 [10:33<13:28,  2.12it/s, est. speed input: 69404.42 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1342/3051 [10:33<13:27,  2.12it/s, est. speed input: 69404.43 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1343/3051 [10:34<13:26,  2.12it/s, est. speed input: 69404.51 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1344/3051 [10:34<13:25,  2.12it/s, est. speed input: 69404.53 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1345/3051 [10:35<13:24,  2.12it/s, est. speed input: 69404.68 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1346/3051 [10:35<13:23,  2.12it/s, est. speed input: 69404.80 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1347/3051 [10:35<13:23,  2.12it/s, est. speed input: 69404.92 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1348/3051 [10:36<13:24,  2.12it/s, est. speed input: 69404.52 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1349/3051 [10:36<13:23,  2.12it/s, est. speed input: 69404.59 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1350/3051 [10:37<13:22,  2.12it/s, est. speed input: 69404.68 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1351/3051 [10:37<13:23,  2.12it/s, est. speed input: 69404.53 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1352/3051 [10:38<13:22,  2.12it/s, est. speed input: 69404.58 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1353/3051 [10:38<13:23,  2.11it/s, est. speed input: 69404.15 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1354/3051 [10:39<13:22,  2.11it/s, est. speed input: 69404.22 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1355/3051 [10:39<13:22,  2.11it/s, est. speed input: 69404.05 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1356/3051 [10:40<13:21,  2.12it/s, est. speed input: 69404.06 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1357/3051 [10:40<13:20,  2.12it/s, est. speed input: 69403.99 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1358/3051 [10:41<13:20,  2.11it/s, est. speed input: 69403.86 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1359/3051 [10:41<13:20,  2.11it/s, est. speed input: 69403.70 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1360/3051 [10:42<13:20,  2.11it/s, est. speed input: 69403.59 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1361/3051 [10:42<13:19,  2.12it/s, est. speed input: 69403.60 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1362/3051 [10:43<13:17,  2.12it/s, est. speed input: 69403.67 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1363/3051 [10:43<13:18,  2.11it/s, est. speed input: 69403.47 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1364/3051 [10:43<13:16,  2.12it/s, est. speed input: 69403.59 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1365/3051 [10:44<13:16,  2.12it/s, est. speed input: 69403.53 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1366/3051 [10:44<13:16,  2.12it/s, est. speed input: 69403.41 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1367/3051 [10:45<13:15,  2.12it/s, est. speed input: 69403.40 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1368/3051 [10:45<13:15,  2.12it/s, est. speed input: 69403.39 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1369/3051 [10:46<13:14,  2.12it/s, est. speed input: 69403.31 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1370/3051 [10:46<13:16,  2.11it/s, est. speed input: 69402.74 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1371/3051 [10:47<13:14,  2.11it/s, est. speed input: 69402.87 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1372/3051 [10:47<13:13,  2.12it/s, est. speed input: 69403.01 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1373/3051 [10:48<13:12,  2.12it/s, est. speed input: 69403.01 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1374/3051 [10:48<13:11,  2.12it/s, est. speed input: 69403.03 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1375/3051 [10:49<13:11,  2.12it/s, est. speed input: 69403.05 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1376/3051 [10:49<13:12,  2.11it/s, est. speed input: 69402.70 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1377/3051 [10:50<13:11,  2.11it/s, est. speed input: 69402.65 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1378/3051 [10:50<13:10,  2.12it/s, est. speed input: 69402.69 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1379/3051 [10:51<13:09,  2.12it/s, est. speed input: 69402.75 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1380/3051 [10:51<13:10,  2.11it/s, est. speed input: 69402.47 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1381/3051 [10:52<13:09,  2.12it/s, est. speed input: 69402.54 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1382/3051 [10:52<13:08,  2.12it/s, est. speed input: 69402.56 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1383/3051 [10:52<13:08,  2.11it/s, est. speed input: 69402.36 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1384/3051 [10:53<13:07,  2.12it/s, est. speed input: 69402.53 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1385/3051 [10:53<13:07,  2.12it/s, est. speed input: 69402.42 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1386/3051 [10:54<13:07,  2.12it/s, est. speed input: 69402.27 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1387/3051 [10:54<13:07,  2.11it/s, est. speed input: 69402.10 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1388/3051 [10:55<13:05,  2.12it/s, est. speed input: 69402.18 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1389/3051 [10:55<13:05,  2.12it/s, est. speed input: 69402.13 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1390/3051 [10:56<13:04,  2.12it/s, est. speed input: 69402.11 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1391/3051 [10:56<13:04,  2.12it/s, est. speed input: 69402.14 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1392/3051 [10:57<13:02,  2.12it/s, est. speed input: 69402.28 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1393/3051 [10:57<13:02,  2.12it/s, est. speed input: 69402.26 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1394/3051 [10:58<13:02,  2.12it/s, est. speed input: 69402.20 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1395/3051 [10:58<13:01,  2.12it/s, est. speed input: 69402.30 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1396/3051 [10:59<13:00,  2.12it/s, est. speed input: 69402.44 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1397/3051 [10:59<13:00,  2.12it/s, est. speed input: 69402.49 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1398/3051 [11:00<12:59,  2.12it/s, est. speed input: 69402.54 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1399/3051 [11:00<12:59,  2.12it/s, est. speed input: 69402.52 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1400/3051 [11:01<12:59,  2.12it/s, est. speed input: 69402.52 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1401/3051 [11:01<12:58,  2.12it/s, est. speed input: 69402.50 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1402/3051 [11:01<13:00,  2.11it/s, est. speed input: 69402.10 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1403/3051 [11:02<12:59,  2.11it/s, est. speed input: 69402.05 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1404/3051 [11:02<12:58,  2.12it/s, est. speed input: 69402.13 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1405/3051 [11:03<12:58,  2.12it/s, est. speed input: 69402.02 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1406/3051 [11:03<12:57,  2.12it/s, est. speed input: 69402.03 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1407/3051 [11:04<12:56,  2.12it/s, est. speed input: 69402.08 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1408/3051 [11:04<12:55,  2.12it/s, est. speed input: 69402.15 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1409/3051 [11:05<12:55,  2.12it/s, est. speed input: 69402.14 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1410/3051 [11:05<12:55,  2.12it/s, est. speed input: 69402.03 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1411/3051 [11:06<12:54,  2.12it/s, est. speed input: 69402.12 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1412/3051 [11:06<12:53,  2.12it/s, est. speed input: 69402.11 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1413/3051 [11:07<12:53,  2.12it/s, est. speed input: 69402.04 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1414/3051 [11:07<12:53,  2.12it/s, est. speed input: 69402.06 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1415/3051 [11:08<12:52,  2.12it/s, est. speed input: 69402.11 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1416/3051 [11:08<12:51,  2.12it/s, est. speed input: 69402.24 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1417/3051 [11:09<12:50,  2.12it/s, est. speed input: 69402.27 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1418/3051 [11:09<12:50,  2.12it/s, est. speed input: 69402.36 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1419/3051 [11:09<12:52,  2.11it/s, est. speed input: 69401.91 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1420/3051 [11:10<12:50,  2.12it/s, est. speed input: 69401.96 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1421/3051 [11:10<12:51,  2.11it/s, est. speed input: 69401.74 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1422/3051 [11:11<12:51,  2.11it/s, est. speed input: 69401.44 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1423/3051 [11:11<12:50,  2.11it/s, est. speed input: 69401.37 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1424/3051 [11:12<12:48,  2.12it/s, est. speed input: 69401.51 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1425/3051 [11:12<12:48,  2.12it/s, est. speed input: 69401.50 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1426/3051 [11:13<12:47,  2.12it/s, est. speed input: 69401.51 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1427/3051 [11:13<12:46,  2.12it/s, est. speed input: 69401.57 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1428/3051 [11:14<12:46,  2.12it/s, est. speed input: 69401.62 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1429/3051 [11:14<12:48,  2.11it/s, est. speed input: 69401.03 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1430/3051 [11:15<12:46,  2.11it/s, est. speed input: 69401.12 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1431/3051 [11:15<12:45,  2.12it/s, est. speed input: 69401.30 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1432/3051 [11:16<12:44,  2.12it/s, est. speed input: 69401.35 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1433/3051 [11:16<12:44,  2.12it/s, est. speed input: 69401.28 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1434/3051 [11:17<12:44,  2.12it/s, est. speed input: 69401.16 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1435/3051 [11:17<12:43,  2.12it/s, est. speed input: 69401.09 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1436/3051 [11:18<12:42,  2.12it/s, est. speed input: 69401.15 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1437/3051 [11:18<12:42,  2.12it/s, est. speed input: 69401.19 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1438/3051 [11:18<12:43,  2.11it/s, est. speed input: 69400.75 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1439/3051 [11:19<12:42,  2.12it/s, est. speed input: 69400.84 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1440/3051 [11:19<12:40,  2.12it/s, est. speed input: 69400.98 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1441/3051 [11:20<12:40,  2.12it/s, est. speed input: 69401.01 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1442/3051 [11:20<12:39,  2.12it/s, est. speed input: 69401.05 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1443/3051 [11:21<12:39,  2.12it/s, est. speed input: 69400.99 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1444/3051 [11:21<12:39,  2.12it/s, est. speed input: 69400.94 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1445/3051 [11:22<12:38,  2.12it/s, est. speed input: 69400.87 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1446/3051 [11:22<12:38,  2.12it/s, est. speed input: 69400.89 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1447/3051 [11:23<12:37,  2.12it/s, est. speed input: 69400.95 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1448/3051 [11:23<12:37,  2.12it/s, est. speed input: 69400.80 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1449/3051 [11:24<12:36,  2.12it/s, est. speed input: 69400.78 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1450/3051 [11:24<12:35,  2.12it/s, est. speed input: 69400.89 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1451/3051 [11:25<12:35,  2.12it/s, est. speed input: 69400.86 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1452/3051 [11:25<12:35,  2.12it/s, est. speed input: 69400.85 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1453/3051 [11:26<12:34,  2.12it/s, est. speed input: 69400.82 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1454/3051 [11:26<12:35,  2.11it/s, est. speed input: 69400.55 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1455/3051 [11:26<12:36,  2.11it/s, est. speed input: 69400.24 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1456/3051 [11:27<12:35,  2.11it/s, est. speed input: 69400.16 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1457/3051 [11:27<12:34,  2.11it/s, est. speed input: 69400.11 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1458/3051 [11:28<12:33,  2.11it/s, est. speed input: 69400.08 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1459/3051 [11:28<12:32,  2.12it/s, est. speed input: 69400.16 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1460/3051 [11:29<12:31,  2.12it/s, est. speed input: 69400.28 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1461/3051 [11:29<12:31,  2.12it/s, est. speed input: 69400.17 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1462/3051 [11:30<12:31,  2.11it/s, est. speed input: 69399.99 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1463/3051 [11:30<12:30,  2.12it/s, est. speed input: 69400.05 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1464/3051 [11:31<12:30,  2.11it/s, est. speed input: 69399.88 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1465/3051 [11:31<12:30,  2.11it/s, est. speed input: 69399.66 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1466/3051 [11:32<12:29,  2.11it/s, est. speed input: 69399.62 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1467/3051 [11:32<12:31,  2.11it/s, est. speed input: 69399.15 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1468/3051 [11:33<12:30,  2.11it/s, est. speed input: 69398.96 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1469/3051 [11:33<12:28,  2.11it/s, est. speed input: 69399.11 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1470/3051 [11:34<12:26,  2.12it/s, est. speed input: 69399.24 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1471/3051 [11:34<12:26,  2.12it/s, est. speed input: 69399.28 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1472/3051 [11:35<12:25,  2.12it/s, est. speed input: 69399.31 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1473/3051 [11:35<12:24,  2.12it/s, est. speed input: 69399.41 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1474/3051 [11:35<12:24,  2.12it/s, est. speed input: 69399.26 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1475/3051 [11:36<12:24,  2.12it/s, est. speed input: 69399.17 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1476/3051 [11:36<12:23,  2.12it/s, est. speed input: 69399.29 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1477/3051 [11:37<12:22,  2.12it/s, est. speed input: 69399.35 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1478/3051 [11:37<12:23,  2.12it/s, est. speed input: 69399.22 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1479/3051 [11:38<12:22,  2.12it/s, est. speed input: 69399.24 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1480/3051 [11:38<12:23,  2.11it/s, est. speed input: 69399.00 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1481/3051 [11:39<12:22,  2.12it/s, est. speed input: 69398.98 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1482/3051 [11:39<12:21,  2.12it/s, est. speed input: 69399.05 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1483/3051 [11:40<12:20,  2.12it/s, est. speed input: 69399.13 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1484/3051 [11:40<12:20,  2.12it/s, est. speed input: 69399.07 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1485/3051 [11:41<12:19,  2.12it/s, est. speed input: 69399.18 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1486/3051 [11:41<12:18,  2.12it/s, est. speed input: 69399.23 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1487/3051 [11:42<12:18,  2.12it/s, est. speed input: 69399.24 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1488/3051 [11:42<12:17,  2.12it/s, est. speed input: 69399.24 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1489/3051 [11:43<12:17,  2.12it/s, est. speed input: 69399.27 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1490/3051 [11:43<12:16,  2.12it/s, est. speed input: 69399.28 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1491/3051 [11:43<12:16,  2.12it/s, est. speed input: 69399.31 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1492/3051 [11:44<12:15,  2.12it/s, est. speed input: 69399.42 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1493/3051 [11:44<12:15,  2.12it/s, est. speed input: 69399.41 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1494/3051 [11:45<12:14,  2.12it/s, est. speed input: 69399.38 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1495/3051 [11:45<12:14,  2.12it/s, est. speed input: 69399.37 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1496/3051 [11:46<12:13,  2.12it/s, est. speed input: 69399.49 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1497/3051 [11:46<12:14,  2.12it/s, est. speed input: 69399.29 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1498/3051 [11:47<12:14,  2.11it/s, est. speed input: 69399.15 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1499/3051 [11:47<12:12,  2.12it/s, est. speed input: 69399.31 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1500/3051 [11:48<12:12,  2.12it/s, est. speed input: 69399.26 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1501/3051 [11:48<12:12,  2.12it/s, est. speed input: 69399.24 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1502/3051 [11:49<12:11,  2.12it/s, est. speed input: 69399.31 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1503/3051 [11:49<12:11,  2.12it/s, est. speed input: 69399.12 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1504/3051 [11:50<12:11,  2.12it/s, est. speed input: 69399.05 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1505/3051 [11:50<12:10,  2.12it/s, est. speed input: 69399.12 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1506/3051 [11:51<12:09,  2.12it/s, est. speed input: 69399.27 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1507/3051 [11:51<12:09,  2.12it/s, est. speed input: 69399.16 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1508/3051 [11:52<12:08,  2.12it/s, est. speed input: 69399.23 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1509/3051 [11:52<12:38,  2.03it/s, est. speed input: 69392.83 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1510/3051 [11:53<12:28,  2.06it/s, est. speed input: 69392.96 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1511/3051 [11:53<12:20,  2.08it/s, est. speed input: 69393.08 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1512/3051 [11:53<12:16,  2.09it/s, est. speed input: 69393.03 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1513/3051 [11:54<12:13,  2.10it/s, est. speed input: 69393.05 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1514/3051 [11:54<12:10,  2.10it/s, est. speed input: 69393.09 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1515/3051 [11:55<12:08,  2.11it/s, est. speed input: 69393.17 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1516/3051 [11:55<12:06,  2.11it/s, est. speed input: 69393.14 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1517/3051 [11:56<12:05,  2.11it/s, est. speed input: 69393.11 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1518/3051 [11:56<12:04,  2.11it/s, est. speed input: 69393.12 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1519/3051 [11:57<12:05,  2.11it/s, est. speed input: 69392.89 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1520/3051 [11:57<12:03,  2.12it/s, est. speed input: 69393.02 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1521/3051 [11:58<12:24,  2.05it/s, est. speed input: 69388.41 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1522/3051 [11:58<12:16,  2.08it/s, est. speed input: 69388.65 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1523/3051 [11:59<12:10,  2.09it/s, est. speed input: 69388.80 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1524/3051 [11:59<12:06,  2.10it/s, est. speed input: 69388.98 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1525/3051 [12:00<12:04,  2.11it/s, est. speed input: 69389.04 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1526/3051 [12:00<12:03,  2.11it/s, est. speed input: 69388.97 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1527/3051 [12:01<12:01,  2.11it/s, est. speed input: 69389.10 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1528/3051 [12:01<12:00,  2.11it/s, est. speed input: 69389.08 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1529/3051 [12:02<11:59,  2.12it/s, est. speed input: 69389.14 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1530/3051 [12:02<11:58,  2.12it/s, est. speed input: 69389.16 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1531/3051 [12:02<11:58,  2.11it/s, est. speed input: 69388.97 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1532/3051 [12:03<11:57,  2.12it/s, est. speed input: 69389.15 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1533/3051 [12:03<11:56,  2.12it/s, est. speed input: 69389.22 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1534/3051 [12:04<11:57,  2.11it/s, est. speed input: 69388.94 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1535/3051 [12:04<11:56,  2.11it/s, est. speed input: 69388.90 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1536/3051 [12:05<11:55,  2.12it/s, est. speed input: 69389.02 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1537/3051 [12:05<11:55,  2.12it/s, est. speed input: 69388.98 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1538/3051 [12:06<11:55,  2.11it/s, est. speed input: 69388.80 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1539/3051 [12:06<11:54,  2.12it/s, est. speed input: 69388.95 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1540/3051 [12:07<11:53,  2.12it/s, est. speed input: 69389.02 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1541/3051 [12:07<11:52,  2.12it/s, est. speed input: 69389.07 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1542/3051 [12:08<11:52,  2.12it/s, est. speed input: 69389.11 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1543/3051 [12:08<11:51,  2.12it/s, est. speed input: 69389.22 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1544/3051 [12:09<11:51,  2.12it/s, est. speed input: 69389.09 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1545/3051 [12:09<11:51,  2.12it/s, est. speed input: 69389.11 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1546/3051 [12:10<11:50,  2.12it/s, est. speed input: 69389.21 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1547/3051 [12:10<11:49,  2.12it/s, est. speed input: 69389.32 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1548/3051 [12:11<11:48,  2.12it/s, est. speed input: 69389.42 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1549/3051 [12:11<11:48,  2.12it/s, est. speed input: 69389.37 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1550/3051 [12:11<11:48,  2.12it/s, est. speed input: 69389.38 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1551/3051 [12:12<11:48,  2.12it/s, est. speed input: 69389.25 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1552/3051 [12:12<11:47,  2.12it/s, est. speed input: 69389.31 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1553/3051 [12:13<11:46,  2.12it/s, est. speed input: 69389.45 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1554/3051 [12:13<11:45,  2.12it/s, est. speed input: 69389.56 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1555/3051 [12:14<11:46,  2.12it/s, est. speed input: 69389.39 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1556/3051 [12:14<11:45,  2.12it/s, est. speed input: 69389.43 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1557/3051 [12:15<11:45,  2.12it/s, est. speed input: 69389.44 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1558/3051 [12:15<11:45,  2.12it/s, est. speed input: 69389.38 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1559/3051 [12:16<11:44,  2.12it/s, est. speed input: 69389.49 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1560/3051 [12:16<11:45,  2.11it/s, est. speed input: 69389.09 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1561/3051 [12:17<11:44,  2.12it/s, est. speed input: 69389.21 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1562/3051 [12:17<11:44,  2.11it/s, est. speed input: 69389.03 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1563/3051 [12:18<11:43,  2.12it/s, est. speed input: 69389.08 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1564/3051 [12:18<11:42,  2.12it/s, est. speed input: 69389.14 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1565/3051 [12:19<11:41,  2.12it/s, est. speed input: 69389.14 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1566/3051 [12:19<11:40,  2.12it/s, est. speed input: 69389.24 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1567/3051 [12:19<11:42,  2.11it/s, est. speed input: 69388.88 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1568/3051 [12:20<11:42,  2.11it/s, est. speed input: 69388.72 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1569/3051 [12:20<11:40,  2.11it/s, est. speed input: 69388.79 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1570/3051 [12:21<11:41,  2.11it/s, est. speed input: 69388.58 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1571/3051 [12:21<11:40,  2.11it/s, est. speed input: 69388.53 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1572/3051 [12:22<11:39,  2.12it/s, est. speed input: 69388.62 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1573/3051 [12:22<11:38,  2.12it/s, est. speed input: 69388.62 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1574/3051 [12:23<11:37,  2.12it/s, est. speed input: 69388.59 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1575/3051 [12:23<11:36,  2.12it/s, est. speed input: 69388.66 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1576/3051 [12:24<11:47,  2.08it/s, est. speed input: 69386.26 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1577/3051 [12:24<11:43,  2.10it/s, est. speed input: 69386.37 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1578/3051 [12:25<11:40,  2.10it/s, est. speed input: 69386.47 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1579/3051 [12:25<11:40,  2.10it/s, est. speed input: 69386.12 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1580/3051 [12:26<11:38,  2.11it/s, est. speed input: 69386.07 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1581/3051 [12:26<11:36,  2.11it/s, est. speed input: 69386.15 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1582/3051 [12:27<11:34,  2.11it/s, est. speed input: 69386.28 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1583/3051 [12:27<11:35,  2.11it/s, est. speed input: 69385.92 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1584/3051 [12:28<11:34,  2.11it/s, est. speed input: 69385.87 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1585/3051 [12:28<11:33,  2.12it/s, est. speed input: 69386.00 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1586/3051 [12:28<11:31,  2.12it/s, est. speed input: 69386.08 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1587/3051 [12:29<11:33,  2.11it/s, est. speed input: 69385.58 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1588/3051 [12:29<11:32,  2.11it/s, est. speed input: 69385.63 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1589/3051 [12:30<11:30,  2.12it/s, est. speed input: 69385.73 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1590/3051 [12:30<11:30,  2.12it/s, est. speed input: 69385.72 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1591/3051 [12:31<11:30,  2.12it/s, est. speed input: 69385.67 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1592/3051 [12:31<11:31,  2.11it/s, est. speed input: 69385.14 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1593/3051 [12:32<11:30,  2.11it/s, est. speed input: 69385.18 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1594/3051 [12:32<11:29,  2.11it/s, est. speed input: 69385.22 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1595/3051 [12:33<11:27,  2.12it/s, est. speed input: 69385.36 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1596/3051 [12:33<11:27,  2.12it/s, est. speed input: 69385.37 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1597/3051 [12:34<11:27,  2.12it/s, est. speed input: 69385.30 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1598/3051 [12:34<11:26,  2.12it/s, est. speed input: 69385.37 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1599/3051 [12:35<11:26,  2.11it/s, est. speed input: 69385.18 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1600/3051 [12:35<11:26,  2.11it/s, est. speed input: 69385.10 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1601/3051 [12:36<11:25,  2.12it/s, est. speed input: 69385.16 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1602/3051 [12:36<11:24,  2.12it/s, est. speed input: 69385.12 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1603/3051 [12:37<11:23,  2.12it/s, est. speed input: 69385.19 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1604/3051 [12:37<11:23,  2.12it/s, est. speed input: 69385.26 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1605/3051 [12:37<11:23,  2.11it/s, est. speed input: 69385.00 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1606/3051 [12:38<11:23,  2.12it/s, est. speed input: 69385.01 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1607/3051 [12:38<11:22,  2.12it/s, est. speed input: 69385.01 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1608/3051 [12:39<11:20,  2.12it/s, est. speed input: 69385.18 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1609/3051 [12:39<11:20,  2.12it/s, est. speed input: 69385.18 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1610/3051 [12:40<11:19,  2.12it/s, est. speed input: 69385.24 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1611/3051 [12:40<11:19,  2.12it/s, est. speed input: 69385.34 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1612/3051 [12:41<11:18,  2.12it/s, est. speed input: 69385.36 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1613/3051 [12:41<11:19,  2.12it/s, est. speed input: 69385.27 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1614/3051 [12:42<11:18,  2.12it/s, est. speed input: 69385.38 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1615/3051 [12:42<11:18,  2.12it/s, est. speed input: 69385.21 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1616/3051 [12:43<11:18,  2.11it/s, est. speed input: 69385.04 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1617/3051 [12:43<11:17,  2.12it/s, est. speed input: 69385.16 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1618/3051 [12:44<11:16,  2.12it/s, est. speed input: 69385.32 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1619/3051 [12:44<11:15,  2.12it/s, est. speed input: 69385.36 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1620/3051 [12:45<11:15,  2.12it/s, est. speed input: 69385.37 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1621/3051 [12:45<11:14,  2.12it/s, est. speed input: 69385.37 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1622/3051 [12:46<11:14,  2.12it/s, est. speed input: 69385.40 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1623/3051 [12:46<11:15,  2.11it/s, est. speed input: 69385.08 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1624/3051 [12:46<11:14,  2.12it/s, est. speed input: 69385.19 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1625/3051 [12:47<11:12,  2.12it/s, est. speed input: 69385.36 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1626/3051 [12:47<11:12,  2.12it/s, est. speed input: 69385.42 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1627/3051 [12:48<11:12,  2.12it/s, est. speed input: 69385.36 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1628/3051 [12:48<11:12,  2.12it/s, est. speed input: 69385.26 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1629/3051 [12:49<11:11,  2.12it/s, est. speed input: 69385.27 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1630/3051 [12:49<11:13,  2.11it/s, est. speed input: 69384.88 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1631/3051 [12:50<11:11,  2.12it/s, est. speed input: 69385.04 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1632/3051 [12:50<11:10,  2.12it/s, est. speed input: 69385.11 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1633/3051 [12:51<11:09,  2.12it/s, est. speed input: 69385.09 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1634/3051 [12:51<11:09,  2.12it/s, est. speed input: 69385.02 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1635/3051 [12:52<11:09,  2.12it/s, est. speed input: 69384.96 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1636/3051 [12:52<11:09,  2.11it/s, est. speed input: 69384.85 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1637/3051 [12:53<11:07,  2.12it/s, est. speed input: 69384.94 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1638/3051 [12:53<11:07,  2.12it/s, est. speed input: 69384.92 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1639/3051 [12:54<11:06,  2.12it/s, est. speed input: 69384.96 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1640/3051 [12:54<11:05,  2.12it/s, est. speed input: 69385.05 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1641/3051 [12:54<11:06,  2.11it/s, est. speed input: 69384.77 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1642/3051 [12:55<11:06,  2.11it/s, est. speed input: 69384.65 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1643/3051 [12:55<11:05,  2.12it/s, est. speed input: 69384.76 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1644/3051 [12:56<11:25,  2.05it/s, est. speed input: 69380.35 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1645/3051 [12:56<11:18,  2.07it/s, est. speed input: 69380.47 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1646/3051 [12:57<11:12,  2.09it/s, est. speed input: 69380.60 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1647/3051 [12:57<11:09,  2.10it/s, est. speed input: 69380.68 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1648/3051 [12:58<11:06,  2.10it/s, est. speed input: 69380.72 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1649/3051 [12:58<11:04,  2.11it/s, est. speed input: 69380.87 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1650/3051 [12:59<11:04,  2.11it/s, est. speed input: 69380.65 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1651/3051 [12:59<11:03,  2.11it/s, est. speed input: 69380.53 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1652/3051 [13:00<11:01,  2.11it/s, est. speed input: 69380.61 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1653/3051 [13:00<11:00,  2.12it/s, est. speed input: 69380.76 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1654/3051 [13:01<11:01,  2.11it/s, est. speed input: 69380.40 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1655/3051 [13:01<11:01,  2.11it/s, est. speed input: 69380.30 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1656/3051 [13:02<10:59,  2.11it/s, est. speed input: 69380.39 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1657/3051 [13:02<10:58,  2.12it/s, est. speed input: 69380.41 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1658/3051 [13:03<10:58,  2.12it/s, est. speed input: 69380.44 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1659/3051 [13:03<10:57,  2.12it/s, est. speed input: 69380.51 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1660/3051 [13:04<10:56,  2.12it/s, est. speed input: 69380.47 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1661/3051 [13:04<10:56,  2.12it/s, est. speed input: 69380.42 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1662/3051 [13:04<10:55,  2.12it/s, est. speed input: 69380.56 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1663/3051 [13:05<10:54,  2.12it/s, est. speed input: 69380.70 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1664/3051 [13:05<10:54,  2.12it/s, est. speed input: 69380.71 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1665/3051 [13:06<10:53,  2.12it/s, est. speed input: 69380.83 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1666/3051 [13:06<10:52,  2.12it/s, est. speed input: 69380.96 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1667/3051 [13:07<10:52,  2.12it/s, est. speed input: 69381.10 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1668/3051 [13:07<10:51,  2.12it/s, est. speed input: 69381.12 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1669/3051 [13:08<10:51,  2.12it/s, est. speed input: 69381.23 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1670/3051 [13:08<10:52,  2.12it/s, est. speed input: 69381.04 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1671/3051 [13:09<10:51,  2.12it/s, est. speed input: 69381.05 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1672/3051 [13:09<10:51,  2.12it/s, est. speed input: 69381.03 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1673/3051 [13:10<10:51,  2.11it/s, est. speed input: 69380.83 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1674/3051 [13:10<10:51,  2.11it/s, est. speed input: 69380.76 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1675/3051 [13:11<10:49,  2.12it/s, est. speed input: 69380.89 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1676/3051 [13:11<11:02,  2.07it/s, est. speed input: 69378.02 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1677/3051 [13:12<10:58,  2.09it/s, est. speed input: 69378.10 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1678/3051 [13:12<10:54,  2.10it/s, est. speed input: 69378.20 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1679/3051 [13:13<10:51,  2.11it/s, est. speed input: 69378.35 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1680/3051 [13:13<10:49,  2.11it/s, est. speed input: 69378.44 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1681/3051 [13:13<10:47,  2.11it/s, est. speed input: 69378.57 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1682/3051 [13:14<10:46,  2.12it/s, est. speed input: 69378.61 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1683/3051 [13:14<10:47,  2.11it/s, est. speed input: 69378.44 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1684/3051 [13:15<10:46,  2.12it/s, est. speed input: 69378.46 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1685/3051 [13:15<10:46,  2.11it/s, est. speed input: 69378.34 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1686/3051 [13:16<10:45,  2.11it/s, est. speed input: 69378.31 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1687/3051 [13:16<10:44,  2.12it/s, est. speed input: 69378.42 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1688/3051 [13:17<10:43,  2.12it/s, est. speed input: 69378.55 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1689/3051 [13:17<10:43,  2.12it/s, est. speed input: 69378.46 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1690/3051 [13:18<10:42,  2.12it/s, est. speed input: 69378.47 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1691/3051 [13:18<10:42,  2.12it/s, est. speed input: 69378.46 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1692/3051 [13:19<10:45,  2.10it/s, est. speed input: 69377.66 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1693/3051 [13:19<10:43,  2.11it/s, est. speed input: 69377.80 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1694/3051 [13:20<10:41,  2.11it/s, est. speed input: 69377.90 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1695/3051 [13:20<10:40,  2.12it/s, est. speed input: 69377.99 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1696/3051 [13:21<10:40,  2.12it/s, est. speed input: 69378.00 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1697/3051 [13:21<10:40,  2.11it/s, est. speed input: 69377.84 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1698/3051 [13:21<10:39,  2.12it/s, est. speed input: 69377.96 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1699/3051 [13:22<10:38,  2.12it/s, est. speed input: 69377.89 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1700/3051 [13:22<10:39,  2.11it/s, est. speed input: 69377.58 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1701/3051 [13:23<10:38,  2.12it/s, est. speed input: 69377.71 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1702/3051 [13:23<10:37,  2.12it/s, est. speed input: 69377.79 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1703/3051 [13:24<10:37,  2.11it/s, est. speed input: 69377.54 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1704/3051 [13:24<10:36,  2.12it/s, est. speed input: 69377.68 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1705/3051 [13:25<10:36,  2.12it/s, est. speed input: 69377.57 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1706/3051 [13:25<10:35,  2.12it/s, est. speed input: 69377.54 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1707/3051 [13:26<10:35,  2.12it/s, est. speed input: 69377.52 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1708/3051 [13:26<10:34,  2.12it/s, est. speed input: 69377.57 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1709/3051 [13:27<10:34,  2.12it/s, est. speed input: 69377.52 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1710/3051 [13:27<10:33,  2.12it/s, est. speed input: 69377.45 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1711/3051 [13:28<10:33,  2.12it/s, est. speed input: 69377.47 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1712/3051 [13:28<10:33,  2.11it/s, est. speed input: 69377.29 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1713/3051 [13:29<10:32,  2.12it/s, est. speed input: 69377.31 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1714/3051 [13:29<10:31,  2.12it/s, est. speed input: 69377.38 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1715/3051 [13:30<10:32,  2.11it/s, est. speed input: 69377.08 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1716/3051 [13:30<10:31,  2.12it/s, est. speed input: 69377.20 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1717/3051 [13:30<10:29,  2.12it/s, est. speed input: 69377.31 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1718/3051 [13:31<10:29,  2.12it/s, est. speed input: 69377.32 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1719/3051 [13:31<10:30,  2.11it/s, est. speed input: 69377.03 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1720/3051 [13:32<10:29,  2.12it/s, est. speed input: 69377.08 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1721/3051 [13:32<10:38,  2.08it/s, est. speed input: 69374.90 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1722/3051 [13:33<10:34,  2.09it/s, est. speed input: 69375.06 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1723/3051 [13:33<10:31,  2.10it/s, est. speed input: 69375.20 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1724/3051 [13:34<10:31,  2.10it/s, est. speed input: 69374.88 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1725/3051 [13:34<10:29,  2.11it/s, est. speed input: 69374.78 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1726/3051 [13:35<10:27,  2.11it/s, est. speed input: 69374.90 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1727/3051 [13:35<10:27,  2.11it/s, est. speed input: 69374.83 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1728/3051 [13:36<10:26,  2.11it/s, est. speed input: 69374.73 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1729/3051 [13:36<10:25,  2.11it/s, est. speed input: 69374.75 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1730/3051 [13:37<10:24,  2.12it/s, est. speed input: 69374.87 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1731/3051 [13:37<10:23,  2.12it/s, est. speed input: 69374.93 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1732/3051 [13:38<10:24,  2.11it/s, est. speed input: 69374.59 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1733/3051 [13:38<10:22,  2.12it/s, est. speed input: 69374.74 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1734/3051 [13:39<10:21,  2.12it/s, est. speed input: 69374.84 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1735/3051 [13:39<10:20,  2.12it/s, est. speed input: 69375.01 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1736/3051 [13:39<10:19,  2.12it/s, est. speed input: 69375.15 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1737/3051 [13:40<10:19,  2.12it/s, est. speed input: 69375.23 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1738/3051 [13:40<10:19,  2.12it/s, est. speed input: 69375.23 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1739/3051 [13:41<10:19,  2.12it/s, est. speed input: 69375.23 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1740/3051 [13:41<10:18,  2.12it/s, est. speed input: 69375.25 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1741/3051 [13:42<10:19,  2.11it/s, est. speed input: 69375.03 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1742/3051 [13:42<10:18,  2.11it/s, est. speed input: 69374.98 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1743/3051 [13:43<10:17,  2.12it/s, est. speed input: 69375.12 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1744/3051 [13:43<10:16,  2.12it/s, est. speed input: 69375.20 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1745/3051 [13:44<10:18,  2.11it/s, est. speed input: 69374.87 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1746/3051 [13:44<10:16,  2.12it/s, est. speed input: 69374.97 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1747/3051 [13:45<10:15,  2.12it/s, est. speed input: 69375.01 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1748/3051 [13:45<10:15,  2.12it/s, est. speed input: 69374.98 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1749/3051 [13:46<10:14,  2.12it/s, est. speed input: 69375.12 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1750/3051 [13:46<10:13,  2.12it/s, est. speed input: 69375.24 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1751/3051 [13:47<10:12,  2.12it/s, est. speed input: 69375.40 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1752/3051 [13:47<10:12,  2.12it/s, est. speed input: 69375.37 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1753/3051 [13:47<10:12,  2.12it/s, est. speed input: 69375.42 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1754/3051 [13:48<10:13,  2.12it/s, est. speed input: 69375.18 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1755/3051 [13:48<10:13,  2.11it/s, est. speed input: 69375.07 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1756/3051 [13:49<10:11,  2.12it/s, est. speed input: 69375.18 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1757/3051 [13:49<10:11,  2.12it/s, est. speed input: 69375.09 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1758/3051 [13:50<10:11,  2.12it/s, est. speed input: 69375.10 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1759/3051 [13:50<10:10,  2.12it/s, est. speed input: 69375.20 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1760/3051 [13:51<10:09,  2.12it/s, est. speed input: 69375.18 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1761/3051 [13:51<10:11,  2.11it/s, est. speed input: 69374.71 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1762/3051 [13:52<10:10,  2.11it/s, est. speed input: 69374.75 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1763/3051 [13:52<10:08,  2.12it/s, est. speed input: 69374.87 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1764/3051 [13:53<10:08,  2.11it/s, est. speed input: 69374.70 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1765/3051 [13:53<10:08,  2.11it/s, est. speed input: 69374.62 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1766/3051 [13:54<10:07,  2.12it/s, est. speed input: 69374.67 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1767/3051 [13:54<10:08,  2.11it/s, est. speed input: 69374.23 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1768/3051 [13:55<10:07,  2.11it/s, est. speed input: 69374.25 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1769/3051 [13:55<10:06,  2.11it/s, est. speed input: 69374.35 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1770/3051 [13:56<10:05,  2.11it/s, est. speed input: 69374.27 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1771/3051 [13:56<10:05,  2.12it/s, est. speed input: 69374.29 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1772/3051 [13:56<10:04,  2.12it/s, est. speed input: 69374.36 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1773/3051 [13:57<10:03,  2.12it/s, est. speed input: 69374.43 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1774/3051 [13:57<10:02,  2.12it/s, est. speed input: 69374.43 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1775/3051 [13:58<10:02,  2.12it/s, est. speed input: 69374.42 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1776/3051 [13:58<10:02,  2.12it/s, est. speed input: 69374.43 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1777/3051 [13:59<10:01,  2.12it/s, est. speed input: 69374.54 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1778/3051 [13:59<10:00,  2.12it/s, est. speed input: 69374.59 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1779/3051 [14:00<09:59,  2.12it/s, est. speed input: 69374.74 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1780/3051 [14:00<09:59,  2.12it/s, est. speed input: 69374.83 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1781/3051 [14:01<09:59,  2.12it/s, est. speed input: 69374.82 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1782/3051 [14:01<09:59,  2.12it/s, est. speed input: 69374.70 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1783/3051 [14:02<09:59,  2.11it/s, est. speed input: 69374.54 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1784/3051 [14:02<09:59,  2.12it/s, est. speed input: 69374.53 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1785/3051 [14:03<09:58,  2.12it/s, est. speed input: 69374.56 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1786/3051 [14:03<09:57,  2.12it/s, est. speed input: 69374.67 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1787/3051 [14:04<09:56,  2.12it/s, est. speed input: 69374.66 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1788/3051 [14:04<09:56,  2.12it/s, est. speed input: 69374.59 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1789/3051 [14:05<09:55,  2.12it/s, est. speed input: 69374.70 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1790/3051 [14:05<09:55,  2.12it/s, est. speed input: 69374.69 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1791/3051 [14:05<09:54,  2.12it/s, est. speed input: 69374.82 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1792/3051 [14:06<09:54,  2.12it/s, est. speed input: 69374.84 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1793/3051 [14:06<09:54,  2.12it/s, est. speed input: 69374.79 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1794/3051 [14:07<09:53,  2.12it/s, est. speed input: 69374.79 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1795/3051 [14:07<09:52,  2.12it/s, est. speed input: 69374.88 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1796/3051 [14:08<09:53,  2.11it/s, est. speed input: 69374.63 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1797/3051 [14:08<09:53,  2.11it/s, est. speed input: 69374.58 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1798/3051 [14:09<09:51,  2.12it/s, est. speed input: 69374.68 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1799/3051 [14:09<09:51,  2.12it/s, est. speed input: 69374.65 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1800/3051 [14:10<09:51,  2.12it/s, est. speed input: 69374.60 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1801/3051 [14:10<09:50,  2.12it/s, est. speed input: 69374.70 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1802/3051 [14:11<09:49,  2.12it/s, est. speed input: 69374.83 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1803/3051 [14:11<09:49,  2.12it/s, est. speed input: 69374.73 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1804/3051 [14:12<09:50,  2.11it/s, est. speed input: 69374.31 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1805/3051 [14:12<09:49,  2.11it/s, est. speed input: 69374.39 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1806/3051 [14:13<09:48,  2.12it/s, est. speed input: 69374.43 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1807/3051 [14:13<09:47,  2.12it/s, est. speed input: 69374.45 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1808/3051 [14:13<09:46,  2.12it/s, est. speed input: 69374.56 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1809/3051 [14:14<09:46,  2.12it/s, est. speed input: 69374.44 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1810/3051 [14:14<09:46,  2.11it/s, est. speed input: 69374.32 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1811/3051 [14:15<09:45,  2.12it/s, est. speed input: 69374.39 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1812/3051 [14:15<09:55,  2.08it/s, est. speed input: 69372.22 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1813/3051 [14:16<09:51,  2.09it/s, est. speed input: 69372.36 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1814/3051 [14:16<09:49,  2.10it/s, est. speed input: 69372.28 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1815/3051 [14:17<09:47,  2.10it/s, est. speed input: 69372.14 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1816/3051 [14:17<09:46,  2.11it/s, est. speed input: 69372.12 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1817/3051 [14:18<09:44,  2.11it/s, est. speed input: 69372.17 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1818/3051 [14:18<09:43,  2.11it/s, est. speed input: 69372.31 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1819/3051 [14:19<09:43,  2.11it/s, est. speed input: 69372.16 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1820/3051 [14:19<09:42,  2.11it/s, est. speed input: 69372.06 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1821/3051 [14:20<09:41,  2.12it/s, est. speed input: 69372.18 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1822/3051 [14:20<09:40,  2.12it/s, est. speed input: 69372.20 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1823/3051 [14:21<09:40,  2.12it/s, est. speed input: 69372.18 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1824/3051 [14:21<09:39,  2.12it/s, est. speed input: 69372.18 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1825/3051 [14:22<09:40,  2.11it/s, est. speed input: 69371.91 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1826/3051 [14:22<09:39,  2.11it/s, est. speed input: 69371.95 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1827/3051 [14:22<09:38,  2.12it/s, est. speed input: 69372.02 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1828/3051 [14:23<09:37,  2.12it/s, est. speed input: 69372.06 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1829/3051 [14:23<09:38,  2.11it/s, est. speed input: 69371.83 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1830/3051 [14:24<09:37,  2.12it/s, est. speed input: 69371.87 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1831/3051 [14:24<09:36,  2.12it/s, est. speed input: 69371.95 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1832/3051 [14:25<09:36,  2.11it/s, est. speed input: 69371.74 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1833/3051 [14:25<09:35,  2.11it/s, est. speed input: 69371.75 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1834/3051 [14:26<09:34,  2.12it/s, est. speed input: 69371.84 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1835/3051 [14:26<09:34,  2.12it/s, est. speed input: 69371.91 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1836/3051 [14:27<09:35,  2.11it/s, est. speed input: 69371.59 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1837/3051 [14:27<09:34,  2.11it/s, est. speed input: 69371.63 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1838/3051 [14:28<09:33,  2.12it/s, est. speed input: 69371.70 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1839/3051 [14:28<09:33,  2.11it/s, est. speed input: 69371.56 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1840/3051 [14:29<09:32,  2.12it/s, est. speed input: 69371.67 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1841/3051 [14:29<09:32,  2.11it/s, est. speed input: 69371.45 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1842/3051 [14:30<09:31,  2.11it/s, est. speed input: 69371.41 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1843/3051 [14:30<09:31,  2.12it/s, est. speed input: 69371.46 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1844/3051 [14:31<09:29,  2.12it/s, est. speed input: 69371.56 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1845/3051 [14:31<09:29,  2.12it/s, est. speed input: 69371.58 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1846/3051 [14:31<09:30,  2.11it/s, est. speed input: 69371.17 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1847/3051 [14:32<09:29,  2.11it/s, est. speed input: 69371.22 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1848/3051 [14:32<09:28,  2.12it/s, est. speed input: 69371.30 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1849/3051 [14:33<09:28,  2.12it/s, est. speed input: 69371.27 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1850/3051 [14:33<09:27,  2.12it/s, est. speed input: 69371.35 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1851/3051 [14:34<09:27,  2.11it/s, est. speed input: 69371.19 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1852/3051 [14:34<09:27,  2.11it/s, est. speed input: 69371.14 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1853/3051 [14:35<09:26,  2.12it/s, est. speed input: 69371.20 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1854/3051 [14:35<09:24,  2.12it/s, est. speed input: 69371.33 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1855/3051 [14:36<09:24,  2.12it/s, est. speed input: 69371.43 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1856/3051 [14:36<09:23,  2.12it/s, est. speed input: 69371.54 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1857/3051 [14:37<09:22,  2.12it/s, est. speed input: 69371.62 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1858/3051 [14:37<09:22,  2.12it/s, est. speed input: 69371.60 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1859/3051 [14:38<09:22,  2.12it/s, est. speed input: 69371.56 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1860/3051 [14:38<09:22,  2.12it/s, est. speed input: 69371.62 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1861/3051 [14:39<09:21,  2.12it/s, est. speed input: 69371.71 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1862/3051 [14:39<09:21,  2.12it/s, est. speed input: 69371.71 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1863/3051 [14:39<09:20,  2.12it/s, est. speed input: 69371.73 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1864/3051 [14:40<09:21,  2.11it/s, est. speed input: 69371.40 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1865/3051 [14:40<09:21,  2.11it/s, est. speed input: 69371.37 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1866/3051 [14:41<09:20,  2.11it/s, est. speed input: 69371.39 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1867/3051 [14:41<09:20,  2.11it/s, est. speed input: 69371.29 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1868/3051 [14:42<09:19,  2.11it/s, est. speed input: 69371.29 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1869/3051 [14:42<09:18,  2.12it/s, est. speed input: 69371.30 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1870/3051 [14:43<09:17,  2.12it/s, est. speed input: 69371.42 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1871/3051 [14:43<09:17,  2.12it/s, est. speed input: 69371.39 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1872/3051 [14:44<09:16,  2.12it/s, est. speed input: 69371.38 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1873/3051 [14:44<09:15,  2.12it/s, est. speed input: 69371.50 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1874/3051 [14:45<09:15,  2.12it/s, est. speed input: 69371.50 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1875/3051 [14:45<09:14,  2.12it/s, est. speed input: 69371.57 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1876/3051 [14:46<09:14,  2.12it/s, est. speed input: 69371.65 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1877/3051 [14:46<09:13,  2.12it/s, est. speed input: 69371.79 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1878/3051 [14:47<09:12,  2.12it/s, est. speed input: 69371.88 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1879/3051 [14:47<09:12,  2.12it/s, est. speed input: 69371.97 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1880/3051 [14:48<09:11,  2.12it/s, est. speed input: 69372.04 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1881/3051 [14:48<09:12,  2.12it/s, est. speed input: 69371.98 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1882/3051 [14:48<09:11,  2.12it/s, est. speed input: 69371.98 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1883/3051 [14:49<09:10,  2.12it/s, est. speed input: 69372.09 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1884/3051 [14:49<09:10,  2.12it/s, est. speed input: 69372.19 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1885/3051 [14:50<09:09,  2.12it/s, est. speed input: 69372.21 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1886/3051 [14:50<09:09,  2.12it/s, est. speed input: 69372.29 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1887/3051 [14:51<09:11,  2.11it/s, est. speed input: 69371.84 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1888/3051 [14:51<09:10,  2.11it/s, est. speed input: 69371.83 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1889/3051 [14:52<09:09,  2.12it/s, est. speed input: 69371.94 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1890/3051 [14:52<09:08,  2.12it/s, est. speed input: 69371.94 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1891/3051 [14:53<09:08,  2.12it/s, est. speed input: 69371.94 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1892/3051 [14:53<09:07,  2.12it/s, est. speed input: 69371.93 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1893/3051 [14:54<09:06,  2.12it/s, est. speed input: 69372.00 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1894/3051 [14:54<09:06,  2.12it/s, est. speed input: 69371.99 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1895/3051 [14:55<09:06,  2.12it/s, est. speed input: 69371.97 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1896/3051 [14:55<09:05,  2.12it/s, est. speed input: 69372.08 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1897/3051 [14:56<09:05,  2.12it/s, est. speed input: 69371.98 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1898/3051 [14:56<09:04,  2.12it/s, est. speed input: 69371.99 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1899/3051 [14:56<09:03,  2.12it/s, est. speed input: 69372.03 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1900/3051 [14:57<09:03,  2.12it/s, est. speed input: 69372.07 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1901/3051 [14:57<09:03,  2.12it/s, est. speed input: 69371.93 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1902/3051 [14:58<09:02,  2.12it/s, est. speed input: 69372.07 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1903/3051 [14:58<09:01,  2.12it/s, est. speed input: 69372.18 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1904/3051 [14:59<09:02,  2.11it/s, est. speed input: 69371.85 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1905/3051 [14:59<09:01,  2.12it/s, est. speed input: 69371.90 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1906/3051 [15:00<09:00,  2.12it/s, est. speed input: 69372.00 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1907/3051 [15:00<09:00,  2.12it/s, est. speed input: 69371.93 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1908/3051 [15:01<09:02,  2.11it/s, est. speed input: 69371.46 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1909/3051 [15:01<09:01,  2.11it/s, est. speed input: 69371.44 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1910/3051 [15:02<09:00,  2.11it/s, est. speed input: 69371.39 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1911/3051 [15:02<09:00,  2.11it/s, est. speed input: 69371.12 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1912/3051 [15:03<08:59,  2.11it/s, est. speed input: 69371.15 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1913/3051 [15:03<08:58,  2.11it/s, est. speed input: 69371.04 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1914/3051 [15:04<08:58,  2.11it/s, est. speed input: 69370.96 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1915/3051 [15:04<08:57,  2.11it/s, est. speed input: 69370.97 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1916/3051 [15:05<08:57,  2.11it/s, est. speed input: 69370.74 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1917/3051 [15:05<08:57,  2.11it/s, est. speed input: 69370.54 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1918/3051 [15:05<08:56,  2.11it/s, est. speed input: 69370.64 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1919/3051 [15:06<08:55,  2.11it/s, est. speed input: 69370.54 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1920/3051 [15:06<08:55,  2.11it/s, est. speed input: 69370.40 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1921/3051 [15:07<08:54,  2.11it/s, est. speed input: 69370.41 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1922/3051 [15:07<08:53,  2.11it/s, est. speed input: 69370.44 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1923/3051 [15:08<08:53,  2.12it/s, est. speed input: 69370.48 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1924/3051 [15:08<08:52,  2.12it/s, est. speed input: 69370.58 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1925/3051 [15:09<08:51,  2.12it/s, est. speed input: 69370.65 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1926/3051 [15:09<08:51,  2.12it/s, est. speed input: 69370.51 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1927/3051 [15:10<08:51,  2.12it/s, est. speed input: 69370.44 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1928/3051 [15:10<08:50,  2.12it/s, est. speed input: 69370.44 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1929/3051 [15:11<08:51,  2.11it/s, est. speed input: 69370.24 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1930/3051 [15:11<08:50,  2.11it/s, est. speed input: 69370.12 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1931/3051 [15:12<08:50,  2.11it/s, est. speed input: 69370.03 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1932/3051 [15:12<08:50,  2.11it/s, est. speed input: 69369.73 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1933/3051 [15:13<08:49,  2.11it/s, est. speed input: 69369.73 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1934/3051 [15:13<08:48,  2.11it/s, est. speed input: 69369.76 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1935/3051 [15:14<08:47,  2.12it/s, est. speed input: 69369.82 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1936/3051 [15:14<08:47,  2.12it/s, est. speed input: 69369.81 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1937/3051 [15:14<08:47,  2.11it/s, est. speed input: 69369.46 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1938/3051 [15:15<08:48,  2.11it/s, est. speed input: 69369.20 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1939/3051 [15:15<08:47,  2.11it/s, est. speed input: 69369.12 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1940/3051 [15:16<08:45,  2.11it/s, est. speed input: 69369.20 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1941/3051 [15:16<08:44,  2.11it/s, est. speed input: 69369.24 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1942/3051 [15:17<08:44,  2.11it/s, est. speed input: 69369.19 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1943/3051 [15:17<08:46,  2.11it/s, est. speed input: 69368.69 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1944/3051 [15:18<08:44,  2.11it/s, est. speed input: 69368.69 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1945/3051 [15:18<08:43,  2.11it/s, est. speed input: 69368.72 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1946/3051 [15:19<08:43,  2.11it/s, est. speed input: 69368.65 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1947/3051 [15:19<08:41,  2.12it/s, est. speed input: 69368.79 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1948/3051 [15:20<08:40,  2.12it/s, est. speed input: 69368.86 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1949/3051 [15:20<08:40,  2.12it/s, est. speed input: 69368.91 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1950/3051 [15:21<08:39,  2.12it/s, est. speed input: 69368.94 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1951/3051 [15:21<08:38,  2.12it/s, est. speed input: 69369.05 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1952/3051 [15:22<08:39,  2.11it/s, est. speed input: 69368.74 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1953/3051 [15:22<08:39,  2.11it/s, est. speed input: 69368.74 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1954/3051 [15:23<08:38,  2.12it/s, est. speed input: 69368.82 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1955/3051 [15:23<08:37,  2.12it/s, est. speed input: 69368.90 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1956/3051 [15:23<08:36,  2.12it/s, est. speed input: 69368.94 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1957/3051 [15:24<08:36,  2.12it/s, est. speed input: 69368.91 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1958/3051 [15:24<08:36,  2.12it/s, est. speed input: 69368.80 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1959/3051 [15:25<08:35,  2.12it/s, est. speed input: 69368.89 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1960/3051 [15:25<08:34,  2.12it/s, est. speed input: 69368.96 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1961/3051 [15:26<08:34,  2.12it/s, est. speed input: 69368.98 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1962/3051 [15:26<08:34,  2.12it/s, est. speed input: 69368.92 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1963/3051 [15:27<08:33,  2.12it/s, est. speed input: 69368.96 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1964/3051 [15:27<08:33,  2.12it/s, est. speed input: 69368.97 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1965/3051 [15:28<08:33,  2.12it/s, est. speed input: 69368.86 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1966/3051 [15:28<08:32,  2.12it/s, est. speed input: 69368.90 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1967/3051 [15:29<08:32,  2.11it/s, est. speed input: 69368.79 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1968/3051 [15:29<08:32,  2.11it/s, est. speed input: 69368.64 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1969/3051 [15:30<08:32,  2.11it/s, est. speed input: 69368.52 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1970/3051 [15:30<08:31,  2.11it/s, est. speed input: 69368.56 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1971/3051 [15:31<08:30,  2.12it/s, est. speed input: 69368.66 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1972/3051 [15:31<08:30,  2.12it/s, est. speed input: 69368.58 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1973/3051 [15:31<08:29,  2.12it/s, est. speed input: 69368.57 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1974/3051 [15:32<08:30,  2.11it/s, est. speed input: 69368.11 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1975/3051 [15:32<08:29,  2.11it/s, est. speed input: 69368.24 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1976/3051 [15:33<08:28,  2.12it/s, est. speed input: 69368.34 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1977/3051 [15:33<08:27,  2.12it/s, est. speed input: 69368.41 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1978/3051 [15:34<08:27,  2.11it/s, est. speed input: 69368.17 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1979/3051 [15:34<08:26,  2.12it/s, est. speed input: 69368.29 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1980/3051 [15:35<08:25,  2.12it/s, est. speed input: 69368.35 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1981/3051 [15:35<08:25,  2.12it/s, est. speed input: 69368.33 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1982/3051 [15:36<08:24,  2.12it/s, est. speed input: 69368.35 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1983/3051 [15:36<08:24,  2.12it/s, est. speed input: 69368.44 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1984/3051 [15:37<08:24,  2.11it/s, est. speed input: 69368.15 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1985/3051 [15:37<08:24,  2.11it/s, est. speed input: 69367.99 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1986/3051 [15:38<08:23,  2.11it/s, est. speed input: 69368.07 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1987/3051 [15:38<08:23,  2.11it/s, est. speed input: 69367.98 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1988/3051 [15:39<08:22,  2.11it/s, est. speed input: 69367.91 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1989/3051 [15:39<08:22,  2.11it/s, est. speed input: 69367.93 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1990/3051 [15:40<08:21,  2.12it/s, est. speed input: 69368.00 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1991/3051 [15:40<08:20,  2.12it/s, est. speed input: 69368.01 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1992/3051 [15:40<08:20,  2.12it/s, est. speed input: 69368.01 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1993/3051 [15:41<08:19,  2.12it/s, est. speed input: 69368.10 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1994/3051 [15:41<08:18,  2.12it/s, est. speed input: 69368.18 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1995/3051 [15:42<08:18,  2.12it/s, est. speed input: 69368.26 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1996/3051 [15:42<08:17,  2.12it/s, est. speed input: 69368.29 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1997/3051 [15:43<08:17,  2.12it/s, est. speed input: 69368.38 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1998/3051 [15:43<08:16,  2.12it/s, est. speed input: 69368.38 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1999/3051 [15:44<08:16,  2.12it/s, est. speed input: 69368.46 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2000/3051 [15:44<08:15,  2.12it/s, est. speed input: 69368.50 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2001/3051 [15:45<08:15,  2.12it/s, est. speed input: 69368.47 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2002/3051 [15:45<08:15,  2.12it/s, est. speed input: 69368.51 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2003/3051 [15:46<08:14,  2.12it/s, est. speed input: 69368.62 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2004/3051 [15:46<08:15,  2.11it/s, est. speed input: 69368.31 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2005/3051 [15:47<08:14,  2.11it/s, est. speed input: 69368.29 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2006/3051 [15:47<08:13,  2.12it/s, est. speed input: 69368.32 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2007/3051 [15:48<08:13,  2.12it/s, est. speed input: 69368.29 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2008/3051 [15:48<08:13,  2.11it/s, est. speed input: 69368.17 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2009/3051 [15:49<08:12,  2.12it/s, est. speed input: 69368.24 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2010/3051 [15:49<08:12,  2.11it/s, est. speed input: 69368.09 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2011/3051 [15:49<08:11,  2.12it/s, est. speed input: 69368.14 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2012/3051 [15:50<08:11,  2.12it/s, est. speed input: 69368.10 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2013/3051 [15:50<08:10,  2.12it/s, est. speed input: 69368.08 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2014/3051 [15:51<08:10,  2.11it/s, est. speed input: 69367.95 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2015/3051 [15:51<08:10,  2.11it/s, est. speed input: 69367.78 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2016/3051 [15:52<08:09,  2.11it/s, est. speed input: 69367.84 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2017/3051 [15:52<08:09,  2.11it/s, est. speed input: 69367.79 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2018/3051 [15:53<08:08,  2.11it/s, est. speed input: 69367.74 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2019/3051 [15:53<08:07,  2.12it/s, est. speed input: 69367.84 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2020/3051 [15:54<08:07,  2.12it/s, est. speed input: 69367.76 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 2021/3051 [15:54<08:06,  2.12it/s, est. speed input: 69367.75 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 2022/3051 [15:55<08:06,  2.12it/s, est. speed input: 69367.70 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 2023/3051 [15:55<08:06,  2.11it/s, est. speed input: 69367.64 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 2024/3051 [15:56<08:05,  2.12it/s, est. speed input: 69367.63 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 2025/3051 [15:56<08:04,  2.12it/s, est. speed input: 69367.74 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 2026/3051 [15:57<08:04,  2.12it/s, est. speed input: 69367.76 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 2027/3051 [15:57<08:03,  2.12it/s, est. speed input: 69367.70 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 2028/3051 [15:57<08:03,  2.12it/s, est. speed input: 69367.75 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2029/3051 [15:58<08:03,  2.12it/s, est. speed input: 69367.67 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2030/3051 [15:58<08:03,  2.11it/s, est. speed input: 69367.44 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2031/3051 [15:59<08:02,  2.12it/s, est. speed input: 69367.56 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2032/3051 [15:59<08:01,  2.12it/s, est. speed input: 69367.66 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2033/3051 [16:00<08:01,  2.12it/s, est. speed input: 69367.59 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2034/3051 [16:00<08:00,  2.12it/s, est. speed input: 69367.60 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2035/3051 [16:01<07:59,  2.12it/s, est. speed input: 69367.71 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2036/3051 [16:01<07:59,  2.12it/s, est. speed input: 69367.59 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2037/3051 [16:02<07:59,  2.12it/s, est. speed input: 69367.55 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2038/3051 [16:02<07:58,  2.12it/s, est. speed input: 69367.55 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2039/3051 [16:03<07:58,  2.11it/s, est. speed input: 69367.43 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2040/3051 [16:03<07:58,  2.11it/s, est. speed input: 69367.34 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2041/3051 [16:04<07:57,  2.12it/s, est. speed input: 69367.44 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2042/3051 [16:04<07:56,  2.12it/s, est. speed input: 69367.50 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2043/3051 [16:05<07:55,  2.12it/s, est. speed input: 69367.53 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2044/3051 [16:05<07:56,  2.12it/s, est. speed input: 69367.39 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2045/3051 [16:06<07:55,  2.12it/s, est. speed input: 69367.49 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2046/3051 [16:06<07:54,  2.12it/s, est. speed input: 69367.47 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2047/3051 [16:06<07:55,  2.11it/s, est. speed input: 69367.24 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2048/3051 [16:07<07:54,  2.11it/s, est. speed input: 69367.30 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2049/3051 [16:07<07:53,  2.12it/s, est. speed input: 69367.42 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2050/3051 [16:08<07:52,  2.12it/s, est. speed input: 69367.46 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2051/3051 [16:08<07:52,  2.12it/s, est. speed input: 69367.46 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2052/3051 [16:09<07:51,  2.12it/s, est. speed input: 69367.56 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2053/3051 [16:09<07:50,  2.12it/s, est. speed input: 69367.61 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2054/3051 [16:10<07:50,  2.12it/s, est. speed input: 69367.62 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2055/3051 [16:10<07:49,  2.12it/s, est. speed input: 69367.66 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2056/3051 [16:11<07:49,  2.12it/s, est. speed input: 69367.75 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2057/3051 [16:11<07:49,  2.12it/s, est. speed input: 69367.75 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2058/3051 [16:12<07:48,  2.12it/s, est. speed input: 69367.77 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 2059/3051 [16:12<07:48,  2.12it/s, est. speed input: 69367.78 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2060/3051 [16:13<07:48,  2.11it/s, est. speed input: 69367.56 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2061/3051 [16:13<07:47,  2.12it/s, est. speed input: 69367.67 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2062/3051 [16:14<07:47,  2.12it/s, est. speed input: 69367.65 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2063/3051 [16:14<07:46,  2.12it/s, est. speed input: 69367.67 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2064/3051 [16:14<07:46,  2.12it/s, est. speed input: 69367.72 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2065/3051 [16:15<07:45,  2.12it/s, est. speed input: 69367.80 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2066/3051 [16:15<07:45,  2.12it/s, est. speed input: 69367.78 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2067/3051 [16:16<07:44,  2.12it/s, est. speed input: 69367.79 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2068/3051 [16:16<07:43,  2.12it/s, est. speed input: 69367.89 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2069/3051 [16:17<07:43,  2.12it/s, est. speed input: 69367.88 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2070/3051 [16:17<07:43,  2.12it/s, est. speed input: 69367.90 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2071/3051 [16:18<07:42,  2.12it/s, est. speed input: 69367.95 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2072/3051 [16:18<07:41,  2.12it/s, est. speed input: 69368.00 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2073/3051 [16:19<07:41,  2.12it/s, est. speed input: 69368.06 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2074/3051 [16:19<07:40,  2.12it/s, est. speed input: 69368.16 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2075/3051 [16:20<07:40,  2.12it/s, est. speed input: 69368.21 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2076/3051 [16:20<07:40,  2.12it/s, est. speed input: 69368.14 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2077/3051 [16:21<07:39,  2.12it/s, est. speed input: 69368.14 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2078/3051 [16:21<07:39,  2.12it/s, est. speed input: 69368.20 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2079/3051 [16:22<07:38,  2.12it/s, est. speed input: 69368.22 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2080/3051 [16:22<07:38,  2.12it/s, est. speed input: 69368.22 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2081/3051 [16:23<07:37,  2.12it/s, est. speed input: 69368.26 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2082/3051 [16:23<07:38,  2.11it/s, est. speed input: 69367.94 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2083/3051 [16:23<07:38,  2.11it/s, est. speed input: 69367.90 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2084/3051 [16:24<07:37,  2.12it/s, est. speed input: 69367.96 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2085/3051 [16:24<07:36,  2.11it/s, est. speed input: 69367.91 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2086/3051 [16:25<07:36,  2.12it/s, est. speed input: 69367.90 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2087/3051 [16:25<07:35,  2.12it/s, est. speed input: 69367.89 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2088/3051 [16:26<07:34,  2.12it/s, est. speed input: 69367.94 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 2089/3051 [16:26<07:34,  2.12it/s, est. speed input: 69368.00 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▊   | 2090/3051 [16:27<07:34,  2.11it/s, est. speed input: 69367.84 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▊   | 2091/3051 [16:27<07:33,  2.12it/s, est. speed input: 69367.89 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▊   | 2092/3051 [16:28<07:34,  2.11it/s, est. speed input: 69367.66 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▊   | 2093/3051 [16:31<19:03,  1.19s/it, est. speed input: 69199.67 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2094/3051 [16:31<15:34,  1.02it/s, est. speed input: 69199.96 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2095/3051 [16:32<13:07,  1.21it/s, est. speed input: 69200.22 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2096/3051 [16:32<11:25,  1.39it/s, est. speed input: 69200.46 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2097/3051 [16:32<10:13,  1.55it/s, est. speed input: 69200.70 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2098/3051 [16:33<09:23,  1.69it/s, est. speed input: 69200.96 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2099/3051 [16:33<08:48,  1.80it/s, est. speed input: 69201.21 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2100/3051 [16:34<08:23,  1.89it/s, est. speed input: 69201.41 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2101/3051 [16:34<08:06,  1.95it/s, est. speed input: 69201.63 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2102/3051 [16:35<07:53,  2.00it/s, est. speed input: 69201.90 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2103/3051 [16:35<07:45,  2.04it/s, est. speed input: 69202.13 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2104/3051 [16:36<07:38,  2.06it/s, est. speed input: 69202.37 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2105/3051 [16:36<07:34,  2.08it/s, est. speed input: 69202.63 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2106/3051 [16:37<07:30,  2.10it/s, est. speed input: 69202.84 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2107/3051 [16:37<07:28,  2.10it/s, est. speed input: 69203.03 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2108/3051 [16:38<07:26,  2.11it/s, est. speed input: 69203.24 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2109/3051 [16:38<07:25,  2.11it/s, est. speed input: 69203.44 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2110/3051 [16:39<07:24,  2.12it/s, est. speed input: 69203.68 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2111/3051 [16:39<07:23,  2.12it/s, est. speed input: 69203.87 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2112/3051 [16:40<07:22,  2.12it/s, est. speed input: 69204.06 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2113/3051 [16:40<07:21,  2.12it/s, est. speed input: 69204.26 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2114/3051 [16:40<07:21,  2.12it/s, est. speed input: 69204.47 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2115/3051 [16:41<07:20,  2.12it/s, est. speed input: 69204.70 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2116/3051 [16:41<07:20,  2.12it/s, est. speed input: 69204.86 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2117/3051 [16:42<07:19,  2.12it/s, est. speed input: 69205.07 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2118/3051 [16:42<07:19,  2.12it/s, est. speed input: 69205.24 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2119/3051 [16:43<07:18,  2.12it/s, est. speed input: 69205.48 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2120/3051 [16:43<07:18,  2.12it/s, est. speed input: 69205.67 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2121/3051 [16:44<07:17,  2.13it/s, est. speed input: 69205.93 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2122/3051 [16:44<07:16,  2.13it/s, est. speed input: 69206.16 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2123/3051 [16:45<07:16,  2.13it/s, est. speed input: 69206.33 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2124/3051 [16:45<07:16,  2.13it/s, est. speed input: 69206.52 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2125/3051 [16:46<07:15,  2.13it/s, est. speed input: 69206.73 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2126/3051 [16:46<07:16,  2.12it/s, est. speed input: 69206.58 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2127/3051 [16:47<07:16,  2.12it/s, est. speed input: 69206.57 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2128/3051 [16:47<07:16,  2.12it/s, est. speed input: 69206.67 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2129/3051 [16:48<07:15,  2.12it/s, est. speed input: 69206.79 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2130/3051 [16:48<07:15,  2.12it/s, est. speed input: 69206.85 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2131/3051 [16:48<07:14,  2.12it/s, est. speed input: 69206.94 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2132/3051 [16:49<07:13,  2.12it/s, est. speed input: 69207.06 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2133/3051 [16:49<07:13,  2.12it/s, est. speed input: 69207.16 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2134/3051 [16:50<07:12,  2.12it/s, est. speed input: 69207.23 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2135/3051 [16:50<07:12,  2.12it/s, est. speed input: 69207.28 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2136/3051 [16:51<07:12,  2.12it/s, est. speed input: 69207.40 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2137/3051 [16:51<07:11,  2.12it/s, est. speed input: 69207.47 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2138/3051 [16:52<07:11,  2.12it/s, est. speed input: 69207.57 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2139/3051 [16:52<07:10,  2.12it/s, est. speed input: 69207.66 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2140/3051 [16:53<07:10,  2.12it/s, est. speed input: 69207.75 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2141/3051 [16:53<07:09,  2.12it/s, est. speed input: 69207.89 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2142/3051 [16:54<07:09,  2.12it/s, est. speed input: 69207.95 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2143/3051 [16:54<07:09,  2.11it/s, est. speed input: 69207.87 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2144/3051 [16:55<07:08,  2.12it/s, est. speed input: 69207.94 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2145/3051 [16:55<07:08,  2.12it/s, est. speed input: 69208.03 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2146/3051 [16:56<07:07,  2.12it/s, est. speed input: 69208.13 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2147/3051 [16:56<07:06,  2.12it/s, est. speed input: 69208.22 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2148/3051 [16:57<07:06,  2.12it/s, est. speed input: 69208.32 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2149/3051 [16:57<07:06,  2.12it/s, est. speed input: 69208.33 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2150/3051 [16:57<07:06,  2.11it/s, est. speed input: 69208.26 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2151/3051 [16:58<07:05,  2.11it/s, est. speed input: 69208.33 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2152/3051 [16:58<07:05,  2.11it/s, est. speed input: 69208.38 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2153/3051 [16:59<07:05,  2.11it/s, est. speed input: 69208.31 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2154/3051 [16:59<07:04,  2.11it/s, est. speed input: 69208.39 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2155/3051 [17:00<07:03,  2.11it/s, est. speed input: 69208.45 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2156/3051 [17:00<07:03,  2.12it/s, est. speed input: 69208.52 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2157/3051 [17:01<07:02,  2.12it/s, est. speed input: 69208.56 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2158/3051 [17:01<07:02,  2.12it/s, est. speed input: 69208.63 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2159/3051 [17:02<07:01,  2.12it/s, est. speed input: 69208.71 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2160/3051 [17:02<07:01,  2.11it/s, est. speed input: 69208.70 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2161/3051 [17:03<07:00,  2.11it/s, est. speed input: 69208.72 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2162/3051 [17:03<07:00,  2.11it/s, est. speed input: 69208.73 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2163/3051 [17:04<07:00,  2.11it/s, est. speed input: 69208.75 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2164/3051 [17:04<06:59,  2.12it/s, est. speed input: 69208.87 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2165/3051 [17:05<06:58,  2.12it/s, est. speed input: 69208.89 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2166/3051 [17:05<06:58,  2.12it/s, est. speed input: 69208.99 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2167/3051 [17:05<06:57,  2.12it/s, est. speed input: 69209.09 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2168/3051 [17:06<06:57,  2.12it/s, est. speed input: 69209.13 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2169/3051 [17:06<06:56,  2.12it/s, est. speed input: 69209.18 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2170/3051 [17:07<06:56,  2.12it/s, est. speed input: 69209.25 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2171/3051 [17:07<06:55,  2.12it/s, est. speed input: 69209.29 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2172/3051 [17:08<06:56,  2.11it/s, est. speed input: 69208.99 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2173/3051 [17:08<06:56,  2.11it/s, est. speed input: 69209.03 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2174/3051 [17:09<06:55,  2.11it/s, est. speed input: 69209.11 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2175/3051 [17:09<06:54,  2.11it/s, est. speed input: 69209.19 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2176/3051 [17:10<06:53,  2.12it/s, est. speed input: 69209.32 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2177/3051 [17:10<06:53,  2.12it/s, est. speed input: 69209.37 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2178/3051 [17:11<06:53,  2.11it/s, est. speed input: 69209.25 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2179/3051 [17:11<06:52,  2.11it/s, est. speed input: 69209.32 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2180/3051 [17:12<06:52,  2.11it/s, est. speed input: 69209.36 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2181/3051 [17:12<06:51,  2.11it/s, est. speed input: 69209.41 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2182/3051 [17:13<06:51,  2.11it/s, est. speed input: 69209.42 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2183/3051 [17:13<06:50,  2.11it/s, est. speed input: 69209.48 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2184/3051 [17:14<06:49,  2.12it/s, est. speed input: 69209.56 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2185/3051 [17:14<06:50,  2.11it/s, est. speed input: 69209.44 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2186/3051 [17:14<06:49,  2.11it/s, est. speed input: 69209.48 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2187/3051 [17:15<06:48,  2.12it/s, est. speed input: 69209.60 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2188/3051 [17:15<06:48,  2.11it/s, est. speed input: 69209.64 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2189/3051 [17:16<06:47,  2.12it/s, est. speed input: 69209.74 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2190/3051 [17:16<06:46,  2.12it/s, est. speed input: 69209.81 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2191/3051 [17:17<06:46,  2.12it/s, est. speed input: 69209.89 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2192/3051 [17:17<06:46,  2.12it/s, est. speed input: 69209.89 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2193/3051 [17:18<06:45,  2.12it/s, est. speed input: 69209.93 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2194/3051 [17:18<06:45,  2.11it/s, est. speed input: 69209.95 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2195/3051 [17:19<06:46,  2.11it/s, est. speed input: 69209.67 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2196/3051 [17:19<06:44,  2.11it/s, est. speed input: 69209.81 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2197/3051 [17:20<06:44,  2.11it/s, est. speed input: 69209.87 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2198/3051 [17:20<06:43,  2.11it/s, est. speed input: 69209.93 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2199/3051 [17:21<06:42,  2.11it/s, est. speed input: 69209.97 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2200/3051 [17:21<06:42,  2.12it/s, est. speed input: 69210.07 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2201/3051 [17:22<06:41,  2.12it/s, est. speed input: 69210.09 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2202/3051 [17:22<06:41,  2.12it/s, est. speed input: 69210.20 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2203/3051 [17:23<06:40,  2.12it/s, est. speed input: 69210.27 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2204/3051 [17:23<06:41,  2.11it/s, est. speed input: 69210.05 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2205/3051 [17:23<06:41,  2.11it/s, est. speed input: 69209.91 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2206/3051 [17:24<06:40,  2.11it/s, est. speed input: 69210.00 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2207/3051 [17:24<06:39,  2.11it/s, est. speed input: 69210.06 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2208/3051 [17:25<06:39,  2.11it/s, est. speed input: 69210.07 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2209/3051 [17:25<06:38,  2.11it/s, est. speed input: 69210.08 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2210/3051 [17:26<06:37,  2.11it/s, est. speed input: 69210.15 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2211/3051 [17:26<06:37,  2.11it/s, est. speed input: 69210.10 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2212/3051 [17:27<06:36,  2.11it/s, est. speed input: 69210.18 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2213/3051 [17:27<06:36,  2.11it/s, est. speed input: 69210.25 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2214/3051 [17:28<06:35,  2.12it/s, est. speed input: 69210.34 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2215/3051 [17:28<06:35,  2.11it/s, est. speed input: 69210.35 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2216/3051 [17:29<06:34,  2.11it/s, est. speed input: 69210.38 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2217/3051 [17:29<06:34,  2.12it/s, est. speed input: 69210.49 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2218/3051 [17:30<06:33,  2.12it/s, est. speed input: 69210.50 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2219/3051 [17:30<06:33,  2.12it/s, est. speed input: 69210.59 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2220/3051 [17:31<06:32,  2.12it/s, est. speed input: 69210.61 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2221/3051 [17:31<06:33,  2.11it/s, est. speed input: 69210.35 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2222/3051 [17:32<06:32,  2.11it/s, est. speed input: 69210.43 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2223/3051 [17:32<06:33,  2.11it/s, est. speed input: 69210.18 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2224/3051 [17:32<06:32,  2.10it/s, est. speed input: 69210.01 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2225/3051 [17:33<06:31,  2.11it/s, est. speed input: 69210.02 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2226/3051 [17:33<06:31,  2.11it/s, est. speed input: 69210.07 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2227/3051 [17:34<06:30,  2.11it/s, est. speed input: 69209.95 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2228/3051 [17:34<06:30,  2.11it/s, est. speed input: 69209.98 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2229/3051 [17:35<06:29,  2.11it/s, est. speed input: 69210.05 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2230/3051 [17:35<06:28,  2.11it/s, est. speed input: 69210.04 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2231/3051 [17:36<06:28,  2.11it/s, est. speed input: 69210.09 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2232/3051 [17:36<06:27,  2.11it/s, est. speed input: 69210.18 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2233/3051 [17:37<06:27,  2.11it/s, est. speed input: 69210.01 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2234/3051 [17:37<06:26,  2.11it/s, est. speed input: 69210.04 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2235/3051 [17:38<06:26,  2.11it/s, est. speed input: 69210.09 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2236/3051 [17:38<06:25,  2.11it/s, est. speed input: 69210.12 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2237/3051 [17:39<06:26,  2.11it/s, est. speed input: 69209.84 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2238/3051 [17:39<06:25,  2.11it/s, est. speed input: 69209.89 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2239/3051 [17:40<06:24,  2.11it/s, est. speed input: 69210.01 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2240/3051 [17:40<06:23,  2.11it/s, est. speed input: 69210.07 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2241/3051 [17:41<06:23,  2.11it/s, est. speed input: 69210.05 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2242/3051 [17:41<06:22,  2.11it/s, est. speed input: 69210.12 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2243/3051 [17:41<06:22,  2.11it/s, est. speed input: 69210.17 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2244/3051 [17:42<06:21,  2.11it/s, est. speed input: 69210.19 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2245/3051 [17:42<06:21,  2.11it/s, est. speed input: 69210.22 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2246/3051 [17:43<06:21,  2.11it/s, est. speed input: 69210.07 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2247/3051 [17:43<06:20,  2.11it/s, est. speed input: 69210.12 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2248/3051 [17:44<06:19,  2.11it/s, est. speed input: 69210.20 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2249/3051 [17:44<06:20,  2.11it/s, est. speed input: 69210.06 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2250/3051 [17:45<06:19,  2.11it/s, est. speed input: 69210.11 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2251/3051 [17:45<06:18,  2.11it/s, est. speed input: 69210.19 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2252/3051 [17:46<06:17,  2.11it/s, est. speed input: 69210.23 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2253/3051 [17:46<06:17,  2.11it/s, est. speed input: 69210.25 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2254/3051 [17:47<06:16,  2.11it/s, est. speed input: 69210.32 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2255/3051 [17:47<06:17,  2.11it/s, est. speed input: 69210.14 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2256/3051 [17:48<06:17,  2.11it/s, est. speed input: 69209.86 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2257/3051 [17:48<06:16,  2.11it/s, est. speed input: 69209.93 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2258/3051 [17:49<06:15,  2.11it/s, est. speed input: 69209.98 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2259/3051 [17:49<06:15,  2.11it/s, est. speed input: 69209.91 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2260/3051 [17:50<06:15,  2.11it/s, est. speed input: 69209.79 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2261/3051 [17:50<06:14,  2.11it/s, est. speed input: 69209.83 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2262/3051 [17:50<06:13,  2.11it/s, est. speed input: 69209.85 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2263/3051 [17:51<06:12,  2.11it/s, est. speed input: 69209.94 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2264/3051 [17:51<06:12,  2.11it/s, est. speed input: 69209.99 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2265/3051 [17:52<06:13,  2.11it/s, est. speed input: 69209.67 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2266/3051 [17:52<06:12,  2.11it/s, est. speed input: 69209.67 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2267/3051 [17:53<06:11,  2.11it/s, est. speed input: 69209.75 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2268/3051 [17:53<06:10,  2.11it/s, est. speed input: 69209.81 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2269/3051 [17:54<06:10,  2.11it/s, est. speed input: 69209.77 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2270/3051 [17:54<06:09,  2.11it/s, est. speed input: 69209.83 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2271/3051 [17:55<06:09,  2.11it/s, est. speed input: 69209.83 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2272/3051 [17:55<06:08,  2.11it/s, est. speed input: 69209.91 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2273/3051 [17:56<06:07,  2.11it/s, est. speed input: 69209.95 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2274/3051 [17:56<06:07,  2.11it/s, est. speed input: 69209.90 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2275/3051 [17:57<06:07,  2.11it/s, est. speed input: 69209.95 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2276/3051 [17:57<06:06,  2.11it/s, est. speed input: 69210.01 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2277/3051 [17:58<06:05,  2.12it/s, est. speed input: 69210.10 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2278/3051 [17:58<06:05,  2.11it/s, est. speed input: 69210.06 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2279/3051 [17:59<06:05,  2.11it/s, est. speed input: 69210.07 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2280/3051 [17:59<06:04,  2.11it/s, est. speed input: 69210.10 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2281/3051 [17:59<06:05,  2.11it/s, est. speed input: 69209.77 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2282/3051 [18:00<06:04,  2.11it/s, est. speed input: 69209.82 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2283/3051 [18:00<06:03,  2.11it/s, est. speed input: 69209.93 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2284/3051 [18:01<06:03,  2.11it/s, est. speed input: 69209.85 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2285/3051 [18:01<06:02,  2.11it/s, est. speed input: 69209.85 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2286/3051 [18:02<06:02,  2.11it/s, est. speed input: 69209.88 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2287/3051 [18:02<06:02,  2.11it/s, est. speed input: 69209.69 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2288/3051 [18:03<06:01,  2.11it/s, est. speed input: 69209.67 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2289/3051 [18:03<06:00,  2.11it/s, est. speed input: 69209.76 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2290/3051 [18:04<06:00,  2.11it/s, est. speed input: 69209.81 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2291/3051 [18:04<05:59,  2.11it/s, est. speed input: 69209.80 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2292/3051 [18:05<05:59,  2.11it/s, est. speed input: 69209.70 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2293/3051 [18:05<05:59,  2.11it/s, est. speed input: 69209.64 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2294/3051 [18:06<05:58,  2.11it/s, est. speed input: 69209.68 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2295/3051 [18:06<05:57,  2.11it/s, est. speed input: 69209.72 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2296/3051 [18:07<05:57,  2.11it/s, est. speed input: 69209.73 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2297/3051 [18:07<05:56,  2.11it/s, est. speed input: 69209.81 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2298/3051 [18:08<05:56,  2.11it/s, est. speed input: 69209.78 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2299/3051 [18:08<05:55,  2.11it/s, est. speed input: 69209.87 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2300/3051 [18:08<05:55,  2.11it/s, est. speed input: 69209.83 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2301/3051 [18:09<05:55,  2.11it/s, est. speed input: 69209.83 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2302/3051 [18:09<05:54,  2.11it/s, est. speed input: 69209.93 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2303/3051 [18:10<05:58,  2.09it/s, est. speed input: 69208.68 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2304/3051 [18:10<05:56,  2.10it/s, est. speed input: 69208.73 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2305/3051 [18:11<05:54,  2.10it/s, est. speed input: 69208.84 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2306/3051 [18:11<05:53,  2.11it/s, est. speed input: 69208.93 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2307/3051 [18:12<05:52,  2.11it/s, est. speed input: 69209.02 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2308/3051 [18:12<05:51,  2.11it/s, est. speed input: 69209.10 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2309/3051 [18:13<05:51,  2.11it/s, est. speed input: 69209.13 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2310/3051 [18:13<05:51,  2.11it/s, est. speed input: 69208.87 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2311/3051 [18:14<05:50,  2.11it/s, est. speed input: 69208.89 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2312/3051 [18:14<05:49,  2.11it/s, est. speed input: 69208.97 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2313/3051 [18:15<05:49,  2.11it/s, est. speed input: 69209.03 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2314/3051 [18:15<05:48,  2.11it/s, est. speed input: 69209.03 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2315/3051 [18:16<05:48,  2.11it/s, est. speed input: 69209.06 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2316/3051 [18:16<05:47,  2.11it/s, est. speed input: 69209.09 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2317/3051 [18:17<05:47,  2.11it/s, est. speed input: 69209.15 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2318/3051 [18:17<05:46,  2.11it/s, est. speed input: 69209.18 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2319/3051 [18:17<05:46,  2.12it/s, est. speed input: 69209.25 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2320/3051 [18:18<05:46,  2.11it/s, est. speed input: 69209.12 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2321/3051 [18:18<05:45,  2.11it/s, est. speed input: 69209.18 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2322/3051 [18:19<05:44,  2.12it/s, est. speed input: 69209.30 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2323/3051 [18:19<05:44,  2.11it/s, est. speed input: 69209.23 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2324/3051 [18:20<05:43,  2.11it/s, est. speed input: 69209.28 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2325/3051 [18:20<05:43,  2.11it/s, est. speed input: 69209.33 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2326/3051 [18:21<05:43,  2.11it/s, est. speed input: 69209.25 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2327/3051 [18:21<05:42,  2.11it/s, est. speed input: 69209.33 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2328/3051 [18:22<05:41,  2.11it/s, est. speed input: 69209.41 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2329/3051 [18:22<05:41,  2.12it/s, est. speed input: 69209.48 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2330/3051 [18:23<05:40,  2.12it/s, est. speed input: 69209.53 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2331/3051 [18:23<05:40,  2.12it/s, est. speed input: 69209.60 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2332/3051 [18:24<05:40,  2.11it/s, est. speed input: 69209.56 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2333/3051 [18:24<05:39,  2.11it/s, est. speed input: 69209.59 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2334/3051 [18:25<05:39,  2.11it/s, est. speed input: 69209.65 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2335/3051 [18:25<05:38,  2.12it/s, est. speed input: 69209.73 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2336/3051 [18:26<05:37,  2.12it/s, est. speed input: 69209.79 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2337/3051 [18:26<05:37,  2.11it/s, est. speed input: 69209.80 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2338/3051 [18:26<05:37,  2.11it/s, est. speed input: 69209.73 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2339/3051 [18:27<05:36,  2.11it/s, est. speed input: 69209.75 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2340/3051 [18:27<05:36,  2.11it/s, est. speed input: 69209.86 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2341/3051 [18:28<05:35,  2.12it/s, est. speed input: 69209.96 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2342/3051 [18:28<05:35,  2.11it/s, est. speed input: 69209.88 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2343/3051 [18:29<05:35,  2.11it/s, est. speed input: 69209.80 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2344/3051 [18:29<05:34,  2.11it/s, est. speed input: 69209.87 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2345/3051 [18:30<05:34,  2.11it/s, est. speed input: 69209.92 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2346/3051 [18:30<05:33,  2.11it/s, est. speed input: 69209.99 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2347/3051 [18:31<05:32,  2.11it/s, est. speed input: 69210.02 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2348/3051 [18:31<05:33,  2.11it/s, est. speed input: 69209.73 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2349/3051 [18:32<05:32,  2.11it/s, est. speed input: 69209.79 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2350/3051 [18:32<05:32,  2.11it/s, est. speed input: 69209.80 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2351/3051 [18:33<05:36,  2.08it/s, est. speed input: 69208.24 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2352/3051 [18:33<05:33,  2.09it/s, est. speed input: 69208.47 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2353/3051 [18:34<05:32,  2.10it/s, est. speed input: 69208.54 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2354/3051 [18:34<05:31,  2.10it/s, est. speed input: 69208.59 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2355/3051 [18:35<05:30,  2.11it/s, est. speed input: 69208.61 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2356/3051 [18:35<05:29,  2.11it/s, est. speed input: 69208.53 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2357/3051 [18:35<05:29,  2.11it/s, est. speed input: 69208.56 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2358/3051 [18:36<05:28,  2.11it/s, est. speed input: 69208.57 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2359/3051 [18:36<05:27,  2.11it/s, est. speed input: 69208.54 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2360/3051 [18:37<05:27,  2.11it/s, est. speed input: 69208.60 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2361/3051 [18:37<05:26,  2.11it/s, est. speed input: 69208.67 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2362/3051 [18:38<05:25,  2.11it/s, est. speed input: 69208.71 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2363/3051 [18:38<05:25,  2.12it/s, est. speed input: 69208.80 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2364/3051 [18:39<05:24,  2.11it/s, est. speed input: 69208.78 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2365/3051 [18:39<05:24,  2.11it/s, est. speed input: 69208.69 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2366/3051 [18:40<05:24,  2.11it/s, est. speed input: 69208.72 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2367/3051 [18:40<05:24,  2.11it/s, est. speed input: 69208.44 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2368/3051 [18:41<05:23,  2.11it/s, est. speed input: 69208.48 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2369/3051 [18:41<05:22,  2.11it/s, est. speed input: 69208.56 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2370/3051 [18:42<05:22,  2.11it/s, est. speed input: 69208.46 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2371/3051 [18:42<05:22,  2.11it/s, est. speed input: 69208.37 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2372/3051 [18:43<05:21,  2.11it/s, est. speed input: 69208.41 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2373/3051 [18:43<05:31,  2.04it/s, est. speed input: 69205.15 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2374/3051 [18:44<05:27,  2.07it/s, est. speed input: 69205.34 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2375/3051 [18:44<05:24,  2.08it/s, est. speed input: 69205.55 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2376/3051 [18:45<05:22,  2.10it/s, est. speed input: 69205.71 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2377/3051 [18:45<05:20,  2.10it/s, est. speed input: 69205.82 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2378/3051 [18:45<05:19,  2.11it/s, est. speed input: 69205.99 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2379/3051 [18:46<05:18,  2.11it/s, est. speed input: 69205.87 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2380/3051 [18:46<05:18,  2.11it/s, est. speed input: 69205.91 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2381/3051 [18:47<05:16,  2.11it/s, est. speed input: 69206.08 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2382/3051 [18:47<05:16,  2.12it/s, est. speed input: 69206.22 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2383/3051 [18:48<05:15,  2.12it/s, est. speed input: 69206.21 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2384/3051 [18:48<05:15,  2.12it/s, est. speed input: 69206.34 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2385/3051 [18:49<05:14,  2.12it/s, est. speed input: 69206.51 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2386/3051 [18:49<05:13,  2.12it/s, est. speed input: 69206.66 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2387/3051 [18:50<05:13,  2.12it/s, est. speed input: 69206.78 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2388/3051 [18:50<05:12,  2.12it/s, est. speed input: 69206.93 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2389/3051 [18:51<05:12,  2.12it/s, est. speed input: 69207.06 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2390/3051 [18:51<05:12,  2.12it/s, est. speed input: 69207.03 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2391/3051 [18:52<05:11,  2.12it/s, est. speed input: 69207.20 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2392/3051 [18:52<05:11,  2.12it/s, est. speed input: 69207.12 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2393/3051 [18:53<05:10,  2.12it/s, est. speed input: 69207.17 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2394/3051 [18:53<05:10,  2.12it/s, est. speed input: 69207.28 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2395/3051 [18:53<05:09,  2.12it/s, est. speed input: 69207.44 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2396/3051 [18:54<05:09,  2.12it/s, est. speed input: 69207.50 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2397/3051 [18:54<05:09,  2.11it/s, est. speed input: 69207.40 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2398/3051 [18:55<05:08,  2.12it/s, est. speed input: 69207.54 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2399/3051 [18:55<05:08,  2.11it/s, est. speed input: 69207.39 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2400/3051 [18:56<05:07,  2.11it/s, est. speed input: 69207.50 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2401/3051 [18:56<05:06,  2.12it/s, est. speed input: 69207.69 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2402/3051 [18:57<05:06,  2.12it/s, est. speed input: 69207.66 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2403/3051 [18:57<05:06,  2.12it/s, est. speed input: 69207.74 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2404/3051 [18:58<05:05,  2.12it/s, est. speed input: 69207.78 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2405/3051 [18:58<05:05,  2.12it/s, est. speed input: 69207.86 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2406/3051 [18:59<05:04,  2.12it/s, est. speed input: 69207.98 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2407/3051 [18:59<05:03,  2.12it/s, est. speed input: 69208.11 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2408/3051 [19:00<05:03,  2.12it/s, est. speed input: 69208.20 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2409/3051 [19:00<05:03,  2.12it/s, est. speed input: 69208.22 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2410/3051 [19:01<05:03,  2.12it/s, est. speed input: 69208.19 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2411/3051 [19:01<05:03,  2.11it/s, est. speed input: 69208.03 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2412/3051 [19:02<05:03,  2.10it/s, est. speed input: 69207.75 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2413/3051 [19:02<05:02,  2.11it/s, est. speed input: 69207.84 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2414/3051 [19:02<05:01,  2.11it/s, est. speed input: 69207.94 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2415/3051 [19:03<05:00,  2.11it/s, est. speed input: 69208.03 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2416/3051 [19:03<05:00,  2.11it/s, est. speed input: 69208.09 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2417/3051 [19:04<04:59,  2.12it/s, est. speed input: 69208.21 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2418/3051 [19:04<04:58,  2.12it/s, est. speed input: 69208.33 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2419/3051 [19:05<04:59,  2.11it/s, est. speed input: 69208.19 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2420/3051 [19:05<04:58,  2.11it/s, est. speed input: 69208.26 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2421/3051 [19:06<04:57,  2.12it/s, est. speed input: 69208.34 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2422/3051 [19:06<04:57,  2.11it/s, est. speed input: 69208.34 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2423/3051 [19:07<04:56,  2.12it/s, est. speed input: 69208.40 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2424/3051 [19:07<04:56,  2.12it/s, est. speed input: 69208.50 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2425/3051 [19:08<04:55,  2.12it/s, est. speed input: 69208.53 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2426/3051 [19:08<04:55,  2.12it/s, est. speed input: 69208.58 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2427/3051 [19:09<04:54,  2.12it/s, est. speed input: 69208.68 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2428/3051 [19:09<04:55,  2.11it/s, est. speed input: 69208.48 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2429/3051 [19:10<04:54,  2.11it/s, est. speed input: 69208.53 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2430/3051 [19:10<04:53,  2.12it/s, est. speed input: 69208.66 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2431/3051 [19:10<04:53,  2.12it/s, est. speed input: 69208.73 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2432/3051 [19:11<04:53,  2.11it/s, est. speed input: 69208.44 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2433/3051 [19:11<04:52,  2.11it/s, est. speed input: 69208.60 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2434/3051 [19:12<04:51,  2.12it/s, est. speed input: 69208.76 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2435/3051 [19:12<04:50,  2.12it/s, est. speed input: 69208.85 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2436/3051 [19:13<04:50,  2.12it/s, est. speed input: 69208.92 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2437/3051 [19:13<04:49,  2.12it/s, est. speed input: 69209.04 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2438/3051 [19:14<04:49,  2.11it/s, est. speed input: 69208.95 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2439/3051 [19:14<04:49,  2.11it/s, est. speed input: 69208.96 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2440/3051 [19:15<04:48,  2.12it/s, est. speed input: 69209.03 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2441/3051 [19:15<04:48,  2.12it/s, est. speed input: 69209.10 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2442/3051 [19:16<04:48,  2.11it/s, est. speed input: 69208.96 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2443/3051 [19:16<04:47,  2.11it/s, est. speed input: 69208.98 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2444/3051 [19:17<04:47,  2.11it/s, est. speed input: 69208.88 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2445/3051 [19:17<04:46,  2.11it/s, est. speed input: 69208.95 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2446/3051 [19:18<04:46,  2.11it/s, est. speed input: 69209.01 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2447/3051 [19:18<04:45,  2.12it/s, est. speed input: 69209.17 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2448/3051 [19:19<04:44,  2.12it/s, est. speed input: 69209.24 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2449/3051 [19:19<04:44,  2.12it/s, est. speed input: 69209.25 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2450/3051 [19:19<04:44,  2.12it/s, est. speed input: 69209.30 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2451/3051 [19:20<04:43,  2.12it/s, est. speed input: 69209.33 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2452/3051 [19:20<04:43,  2.11it/s, est. speed input: 69209.34 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2453/3051 [19:21<04:42,  2.12it/s, est. speed input: 69209.42 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2454/3051 [19:21<04:42,  2.11it/s, est. speed input: 69209.37 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2455/3051 [19:22<04:41,  2.11it/s, est. speed input: 69209.43 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2456/3051 [19:22<04:41,  2.12it/s, est. speed input: 69209.57 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2457/3051 [19:23<04:40,  2.11it/s, est. speed input: 69209.54 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2458/3051 [19:23<04:40,  2.12it/s, est. speed input: 69209.61 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2459/3051 [19:24<04:40,  2.11it/s, est. speed input: 69209.57 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2460/3051 [19:24<04:39,  2.11it/s, est. speed input: 69209.55 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2461/3051 [19:25<04:39,  2.11it/s, est. speed input: 69209.55 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2462/3051 [19:25<04:38,  2.12it/s, est. speed input: 69209.67 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2463/3051 [19:26<04:37,  2.12it/s, est. speed input: 69209.76 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2464/3051 [19:26<04:38,  2.11it/s, est. speed input: 69209.51 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2465/3051 [19:27<04:37,  2.11it/s, est. speed input: 69209.58 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2466/3051 [19:27<04:36,  2.12it/s, est. speed input: 69209.74 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2467/3051 [19:28<04:36,  2.11it/s, est. speed input: 69209.76 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2468/3051 [19:28<04:35,  2.12it/s, est. speed input: 69209.81 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2469/3051 [19:28<04:35,  2.12it/s, est. speed input: 69209.88 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2470/3051 [19:29<04:34,  2.12it/s, est. speed input: 69210.00 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2471/3051 [19:29<04:34,  2.12it/s, est. speed input: 69210.00 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2472/3051 [19:30<04:33,  2.12it/s, est. speed input: 69210.02 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2473/3051 [19:30<04:33,  2.12it/s, est. speed input: 69210.11 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2474/3051 [19:31<04:33,  2.11it/s, est. speed input: 69209.99 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2475/3051 [19:31<04:32,  2.11it/s, est. speed input: 69210.07 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2476/3051 [19:32<04:31,  2.12it/s, est. speed input: 69210.19 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2477/3051 [19:32<04:31,  2.11it/s, est. speed input: 69210.04 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2478/3051 [19:33<04:31,  2.11it/s, est. speed input: 69210.13 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2479/3051 [19:33<04:30,  2.12it/s, est. speed input: 69210.22 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2480/3051 [19:34<04:29,  2.12it/s, est. speed input: 69210.28 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2481/3051 [19:34<04:29,  2.11it/s, est. speed input: 69210.27 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2482/3051 [19:35<04:29,  2.12it/s, est. speed input: 69210.33 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2483/3051 [19:35<04:28,  2.12it/s, est. speed input: 69210.41 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2484/3051 [19:36<04:28,  2.12it/s, est. speed input: 69210.44 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2485/3051 [19:36<04:27,  2.12it/s, est. speed input: 69210.53 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2486/3051 [19:37<04:27,  2.11it/s, est. speed input: 69210.48 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2487/3051 [19:37<04:27,  2.11it/s, est. speed input: 69210.43 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2488/3051 [19:37<04:26,  2.12it/s, est. speed input: 69210.59 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2489/3051 [19:38<04:26,  2.11it/s, est. speed input: 69210.49 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2490/3051 [19:38<04:25,  2.11it/s, est. speed input: 69210.53 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2491/3051 [19:39<04:24,  2.11it/s, est. speed input: 69210.56 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2492/3051 [19:39<04:24,  2.12it/s, est. speed input: 69210.70 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2493/3051 [19:40<04:23,  2.12it/s, est. speed input: 69210.80 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2494/3051 [19:40<04:23,  2.12it/s, est. speed input: 69210.83 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2495/3051 [19:41<04:22,  2.12it/s, est. speed input: 69210.94 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2496/3051 [19:41<04:22,  2.11it/s, est. speed input: 69210.75 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2497/3051 [19:42<04:22,  2.11it/s, est. speed input: 69210.61 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2498/3051 [19:42<04:21,  2.11it/s, est. speed input: 69210.71 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2499/3051 [19:43<04:20,  2.12it/s, est. speed input: 69210.84 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2500/3051 [19:43<04:21,  2.11it/s, est. speed input: 69210.68 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2501/3051 [19:44<04:20,  2.12it/s, est. speed input: 69210.86 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2502/3051 [19:44<04:19,  2.12it/s, est. speed input: 69211.00 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2503/3051 [19:45<04:18,  2.12it/s, est. speed input: 69211.04 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2504/3051 [19:45<04:19,  2.11it/s, est. speed input: 69210.85 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2505/3051 [19:45<04:18,  2.11it/s, est. speed input: 69210.96 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2506/3051 [19:46<04:17,  2.11it/s, est. speed input: 69211.01 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2507/3051 [19:46<04:17,  2.11it/s, est. speed input: 69211.04 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2508/3051 [19:47<04:16,  2.12it/s, est. speed input: 69211.22 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2509/3051 [19:47<04:16,  2.12it/s, est. speed input: 69211.18 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2510/3051 [19:48<04:15,  2.12it/s, est. speed input: 69211.25 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2511/3051 [19:48<04:15,  2.12it/s, est. speed input: 69211.30 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2512/3051 [19:49<04:14,  2.11it/s, est. speed input: 69211.31 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2513/3051 [19:49<04:14,  2.12it/s, est. speed input: 69211.38 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2514/3051 [19:50<04:13,  2.12it/s, est. speed input: 69211.45 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2515/3051 [19:50<04:13,  2.12it/s, est. speed input: 69211.50 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2516/3051 [19:51<04:12,  2.12it/s, est. speed input: 69211.54 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2517/3051 [19:51<04:12,  2.11it/s, est. speed input: 69211.46 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2518/3051 [19:52<04:12,  2.11it/s, est. speed input: 69211.53 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2519/3051 [19:52<04:11,  2.12it/s, est. speed input: 69211.68 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2520/3051 [19:53<04:10,  2.12it/s, est. speed input: 69211.70 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2521/3051 [19:53<04:10,  2.11it/s, est. speed input: 69211.68 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2522/3051 [19:54<04:10,  2.11it/s, est. speed input: 69211.61 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2523/3051 [19:54<04:09,  2.11it/s, est. speed input: 69211.60 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2524/3051 [19:54<04:09,  2.11it/s, est. speed input: 69211.70 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2525/3051 [19:55<04:08,  2.12it/s, est. speed input: 69211.79 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2526/3051 [19:55<04:08,  2.12it/s, est. speed input: 69211.83 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2527/3051 [19:56<04:07,  2.12it/s, est. speed input: 69211.90 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2528/3051 [19:56<04:07,  2.12it/s, est. speed input: 69211.96 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2529/3051 [19:57<04:06,  2.12it/s, est. speed input: 69211.96 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2530/3051 [19:57<04:06,  2.12it/s, est. speed input: 69212.09 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2531/3051 [19:58<04:05,  2.12it/s, est. speed input: 69212.20 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2532/3051 [19:58<04:05,  2.11it/s, est. speed input: 69212.06 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2533/3051 [19:59<04:04,  2.12it/s, est. speed input: 69212.17 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2534/3051 [19:59<04:04,  2.12it/s, est. speed input: 69212.28 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2535/3051 [20:00<04:03,  2.12it/s, est. speed input: 69212.40 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2536/3051 [20:00<04:03,  2.12it/s, est. speed input: 69212.36 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2537/3051 [20:01<04:02,  2.12it/s, est. speed input: 69212.50 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2538/3051 [20:01<04:02,  2.12it/s, est. speed input: 69212.58 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2539/3051 [20:02<04:02,  2.11it/s, est. speed input: 69212.40 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2540/3051 [20:02<04:01,  2.12it/s, est. speed input: 69212.54 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2541/3051 [20:03<04:02,  2.11it/s, est. speed input: 69212.22 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2542/3051 [20:03<04:00,  2.11it/s, est. speed input: 69212.37 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2543/3051 [20:03<04:00,  2.11it/s, est. speed input: 69212.34 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2544/3051 [20:04<03:59,  2.11it/s, est. speed input: 69212.48 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2545/3051 [20:04<03:59,  2.12it/s, est. speed input: 69212.61 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2546/3051 [20:05<03:58,  2.12it/s, est. speed input: 69212.68 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2547/3051 [20:05<03:57,  2.12it/s, est. speed input: 69212.83 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2548/3051 [20:06<03:58,  2.11it/s, est. speed input: 69212.61 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2549/3051 [20:06<03:57,  2.11it/s, est. speed input: 69212.54 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2550/3051 [20:07<03:57,  2.11it/s, est. speed input: 69212.64 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2551/3051 [20:07<03:56,  2.12it/s, est. speed input: 69212.74 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2552/3051 [20:08<03:55,  2.12it/s, est. speed input: 69212.93 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2553/3051 [20:08<03:55,  2.12it/s, est. speed input: 69212.96 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2554/3051 [20:09<03:55,  2.11it/s, est. speed input: 69212.91 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2555/3051 [20:09<03:54,  2.11it/s, est. speed input: 69212.93 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2556/3051 [20:10<03:54,  2.11it/s, est. speed input: 69212.97 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2557/3051 [20:10<03:53,  2.12it/s, est. speed input: 69213.08 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2558/3051 [20:11<03:53,  2.11it/s, est. speed input: 69212.89 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2559/3051 [20:11<03:52,  2.11it/s, est. speed input: 69212.96 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2560/3051 [20:11<03:52,  2.12it/s, est. speed input: 69213.09 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2561/3051 [20:12<03:51,  2.12it/s, est. speed input: 69213.11 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2562/3051 [20:12<03:51,  2.11it/s, est. speed input: 69213.01 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2563/3051 [20:13<03:50,  2.11it/s, est. speed input: 69213.13 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2564/3051 [20:13<03:50,  2.12it/s, est. speed input: 69213.21 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2565/3051 [20:14<03:49,  2.11it/s, est. speed input: 69213.19 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2566/3051 [20:14<03:49,  2.12it/s, est. speed input: 69213.29 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2567/3051 [20:15<03:48,  2.12it/s, est. speed input: 69213.38 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2568/3051 [20:15<03:48,  2.11it/s, est. speed input: 69213.30 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2569/3051 [20:16<03:47,  2.12it/s, est. speed input: 69213.41 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2570/3051 [20:16<03:52,  2.07it/s, est. speed input: 69211.42 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2571/3051 [20:17<03:50,  2.09it/s, est. speed input: 69211.62 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2572/3051 [20:17<03:48,  2.10it/s, est. speed input: 69211.80 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2573/3051 [20:18<03:47,  2.11it/s, est. speed input: 69211.93 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2574/3051 [20:18<03:46,  2.11it/s, est. speed input: 69212.06 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2575/3051 [20:19<03:45,  2.11it/s, est. speed input: 69212.21 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2576/3051 [20:19<03:44,  2.12it/s, est. speed input: 69212.31 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2577/3051 [20:20<03:44,  2.11it/s, est. speed input: 69212.20 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2578/3051 [20:20<03:43,  2.11it/s, est. speed input: 69212.29 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2579/3051 [20:21<03:42,  2.12it/s, est. speed input: 69212.44 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2580/3051 [20:21<03:42,  2.12it/s, est. speed input: 69212.53 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2581/3051 [20:21<03:42,  2.11it/s, est. speed input: 69212.46 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2582/3051 [20:22<03:41,  2.12it/s, est. speed input: 69212.55 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2583/3051 [20:22<03:40,  2.12it/s, est. speed input: 69212.70 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2584/3051 [20:23<03:41,  2.11it/s, est. speed input: 69212.51 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2585/3051 [20:23<03:40,  2.11it/s, est. speed input: 69212.62 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2586/3051 [20:24<03:40,  2.11it/s, est. speed input: 69212.43 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2587/3051 [20:24<03:39,  2.11it/s, est. speed input: 69212.57 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2588/3051 [20:25<03:38,  2.12it/s, est. speed input: 69212.65 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2589/3051 [20:25<03:38,  2.12it/s, est. speed input: 69212.77 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2590/3051 [20:26<03:38,  2.11it/s, est. speed input: 69212.69 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2591/3051 [20:26<03:37,  2.11it/s, est. speed input: 69212.70 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2592/3051 [20:27<03:37,  2.12it/s, est. speed input: 69212.79 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2593/3051 [20:27<03:36,  2.12it/s, est. speed input: 69212.86 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2594/3051 [20:28<03:36,  2.11it/s, est. speed input: 69212.75 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2595/3051 [20:28<03:35,  2.12it/s, est. speed input: 69212.90 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2596/3051 [20:29<03:35,  2.11it/s, est. speed input: 69212.90 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2597/3051 [20:29<03:34,  2.12it/s, est. speed input: 69213.00 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2598/3051 [20:29<03:33,  2.12it/s, est. speed input: 69213.14 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2599/3051 [20:30<03:33,  2.11it/s, est. speed input: 69213.06 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2600/3051 [20:30<03:33,  2.11it/s, est. speed input: 69213.03 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2601/3051 [20:31<03:32,  2.12it/s, est. speed input: 69213.18 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2602/3051 [20:31<03:32,  2.12it/s, est. speed input: 69213.29 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2603/3051 [20:32<03:31,  2.11it/s, est. speed input: 69213.18 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2604/3051 [20:32<03:31,  2.12it/s, est. speed input: 69213.29 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2605/3051 [20:33<03:30,  2.12it/s, est. speed input: 69213.43 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2606/3051 [20:33<03:30,  2.12it/s, est. speed input: 69213.45 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2607/3051 [20:34<03:29,  2.11it/s, est. speed input: 69213.42 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2608/3051 [20:34<03:29,  2.12it/s, est. speed input: 69213.56 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2609/3051 [20:35<03:28,  2.12it/s, est. speed input: 69213.70 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2610/3051 [20:35<03:28,  2.11it/s, est. speed input: 69213.57 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2611/3051 [20:36<03:27,  2.12it/s, est. speed input: 69213.67 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2612/3051 [20:36<03:27,  2.12it/s, est. speed input: 69213.81 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2613/3051 [20:37<03:26,  2.12it/s, est. speed input: 69213.98 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2614/3051 [20:37<03:26,  2.12it/s, est. speed input: 69214.04 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2615/3051 [20:38<03:25,  2.12it/s, est. speed input: 69214.06 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2616/3051 [20:38<03:25,  2.12it/s, est. speed input: 69214.09 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2617/3051 [20:38<03:24,  2.12it/s, est. speed input: 69214.18 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2618/3051 [20:39<03:24,  2.12it/s, est. speed input: 69214.32 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2619/3051 [20:39<03:24,  2.11it/s, est. speed input: 69214.16 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2620/3051 [20:40<03:23,  2.11it/s, est. speed input: 69214.18 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2621/3051 [20:40<03:23,  2.12it/s, est. speed input: 69214.31 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2622/3051 [20:41<03:22,  2.12it/s, est. speed input: 69214.39 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2623/3051 [20:41<03:22,  2.12it/s, est. speed input: 69214.50 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2624/3051 [20:42<03:21,  2.12it/s, est. speed input: 69214.62 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2625/3051 [20:42<03:20,  2.12it/s, est. speed input: 69214.72 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2626/3051 [20:43<03:20,  2.12it/s, est. speed input: 69214.75 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2627/3051 [20:43<03:20,  2.12it/s, est. speed input: 69214.79 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2628/3051 [20:44<03:19,  2.12it/s, est. speed input: 69214.89 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2629/3051 [20:44<03:19,  2.12it/s, est. speed input: 69214.84 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2630/3051 [20:45<03:18,  2.12it/s, est. speed input: 69214.93 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2631/3051 [20:45<03:18,  2.12it/s, est. speed input: 69215.02 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2632/3051 [20:46<03:17,  2.12it/s, est. speed input: 69215.05 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2633/3051 [20:46<03:17,  2.12it/s, est. speed input: 69215.09 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2634/3051 [20:46<03:17,  2.12it/s, est. speed input: 69215.14 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2635/3051 [20:47<03:16,  2.12it/s, est. speed input: 69215.20 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2636/3051 [20:47<03:16,  2.12it/s, est. speed input: 69215.22 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2637/3051 [20:48<03:15,  2.12it/s, est. speed input: 69215.30 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2638/3051 [20:48<03:17,  2.09it/s, est. speed input: 69214.36 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2639/3051 [20:49<03:15,  2.10it/s, est. speed input: 69214.54 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2640/3051 [20:49<03:14,  2.11it/s, est. speed input: 69214.66 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2641/3051 [20:50<03:13,  2.11it/s, est. speed input: 69214.82 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2642/3051 [20:50<03:13,  2.11it/s, est. speed input: 69214.81 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2643/3051 [20:51<03:12,  2.11it/s, est. speed input: 69214.86 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2644/3051 [20:51<03:12,  2.12it/s, est. speed input: 69215.00 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2645/3051 [20:52<03:11,  2.12it/s, est. speed input: 69215.11 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2646/3051 [20:52<03:11,  2.11it/s, est. speed input: 69215.00 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2647/3051 [20:53<03:10,  2.12it/s, est. speed input: 69215.13 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2648/3051 [20:53<03:10,  2.12it/s, est. speed input: 69215.23 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2649/3051 [20:54<03:09,  2.12it/s, est. speed input: 69215.29 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2650/3051 [20:54<03:09,  2.12it/s, est. speed input: 69215.39 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2651/3051 [20:55<03:09,  2.11it/s, est. speed input: 69215.17 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2652/3051 [20:55<03:08,  2.11it/s, est. speed input: 69215.19 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2653/3051 [20:55<03:08,  2.12it/s, est. speed input: 69215.31 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2654/3051 [20:56<03:07,  2.11it/s, est. speed input: 69215.29 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2655/3051 [20:56<03:07,  2.11it/s, est. speed input: 69215.33 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2656/3051 [20:57<03:06,  2.12it/s, est. speed input: 69215.45 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2657/3051 [20:57<03:05,  2.12it/s, est. speed input: 69215.58 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2658/3051 [20:58<03:05,  2.12it/s, est. speed input: 69215.74 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2659/3051 [20:58<03:05,  2.12it/s, est. speed input: 69215.63 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2660/3051 [20:59<03:04,  2.12it/s, est. speed input: 69215.73 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2661/3051 [20:59<03:04,  2.11it/s, est. speed input: 69215.66 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2662/3051 [21:00<03:03,  2.12it/s, est. speed input: 69215.75 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2663/3051 [21:00<03:03,  2.11it/s, est. speed input: 69215.76 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2664/3051 [21:01<03:03,  2.11it/s, est. speed input: 69215.63 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2665/3051 [21:01<03:02,  2.11it/s, est. speed input: 69215.74 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2666/3051 [21:02<03:02,  2.11it/s, est. speed input: 69215.76 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2667/3051 [21:02<03:01,  2.11it/s, est. speed input: 69215.62 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2668/3051 [21:03<03:01,  2.11it/s, est. speed input: 69215.62 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2669/3051 [21:03<03:00,  2.11it/s, est. speed input: 69215.61 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2670/3051 [21:04<03:05,  2.06it/s, est. speed input: 69213.29 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2671/3051 [21:04<03:03,  2.07it/s, est. speed input: 69213.37 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2672/3051 [21:05<03:01,  2.09it/s, est. speed input: 69213.49 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2673/3051 [21:05<03:00,  2.10it/s, est. speed input: 69213.54 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2674/3051 [21:05<02:59,  2.10it/s, est. speed input: 69213.61 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2675/3051 [21:06<02:58,  2.11it/s, est. speed input: 69213.70 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2676/3051 [21:06<02:57,  2.11it/s, est. speed input: 69213.80 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2677/3051 [21:07<02:57,  2.11it/s, est. speed input: 69213.83 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2678/3051 [21:07<02:56,  2.12it/s, est. speed input: 69213.95 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2679/3051 [21:08<02:55,  2.12it/s, est. speed input: 69214.01 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2680/3051 [21:08<02:55,  2.12it/s, est. speed input: 69214.11 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2681/3051 [21:09<02:54,  2.12it/s, est. speed input: 69214.15 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2682/3051 [21:09<02:54,  2.12it/s, est. speed input: 69214.20 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2683/3051 [21:10<02:53,  2.12it/s, est. speed input: 69214.24 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2684/3051 [21:10<02:53,  2.11it/s, est. speed input: 69214.18 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2685/3051 [21:11<02:53,  2.11it/s, est. speed input: 69214.08 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2686/3051 [21:11<02:52,  2.11it/s, est. speed input: 69214.08 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2687/3051 [21:12<02:52,  2.11it/s, est. speed input: 69214.18 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2688/3051 [21:12<02:51,  2.12it/s, est. speed input: 69214.29 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2689/3051 [21:13<02:51,  2.11it/s, est. speed input: 69214.27 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2690/3051 [21:13<02:50,  2.11it/s, est. speed input: 69214.31 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2691/3051 [21:13<02:50,  2.11it/s, est. speed input: 69214.32 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2692/3051 [21:14<02:49,  2.11it/s, est. speed input: 69214.35 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2693/3051 [21:14<02:49,  2.11it/s, est. speed input: 69214.37 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2694/3051 [21:15<02:49,  2.11it/s, est. speed input: 69214.16 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2695/3051 [21:15<02:48,  2.11it/s, est. speed input: 69214.22 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2696/3051 [21:16<02:47,  2.11it/s, est. speed input: 69214.33 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2697/3051 [21:16<02:47,  2.11it/s, est. speed input: 69214.37 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2698/3051 [21:17<02:46,  2.12it/s, est. speed input: 69214.45 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2699/3051 [21:17<02:46,  2.11it/s, est. speed input: 69214.37 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2700/3051 [21:18<02:46,  2.11it/s, est. speed input: 69214.31 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2701/3051 [21:18<02:45,  2.11it/s, est. speed input: 69214.34 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2702/3051 [21:19<02:45,  2.11it/s, est. speed input: 69214.41 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2703/3051 [21:19<02:44,  2.12it/s, est. speed input: 69214.50 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2704/3051 [21:20<02:44,  2.11it/s, est. speed input: 69214.50 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2705/3051 [21:20<02:43,  2.11it/s, est. speed input: 69214.53 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2706/3051 [21:21<02:43,  2.11it/s, est. speed input: 69214.54 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2707/3051 [21:21<02:42,  2.11it/s, est. speed input: 69214.59 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2708/3051 [21:22<02:42,  2.12it/s, est. speed input: 69214.67 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2709/3051 [21:22<02:41,  2.12it/s, est. speed input: 69214.76 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2710/3051 [21:22<02:40,  2.12it/s, est. speed input: 69214.87 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2711/3051 [21:23<02:40,  2.12it/s, est. speed input: 69214.81 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2712/3051 [21:23<02:40,  2.11it/s, est. speed input: 69214.76 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2713/3051 [21:24<02:39,  2.11it/s, est. speed input: 69214.77 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2714/3051 [21:24<02:39,  2.12it/s, est. speed input: 69214.87 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2715/3051 [21:25<02:38,  2.12it/s, est. speed input: 69214.93 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2716/3051 [21:25<02:38,  2.12it/s, est. speed input: 69214.97 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2717/3051 [21:26<02:37,  2.12it/s, est. speed input: 69215.04 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2718/3051 [21:26<02:37,  2.12it/s, est. speed input: 69215.15 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2719/3051 [21:27<02:36,  2.12it/s, est. speed input: 69215.11 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2720/3051 [21:27<02:36,  2.12it/s, est. speed input: 69215.21 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2721/3051 [21:28<02:35,  2.12it/s, est. speed input: 69215.31 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2722/3051 [21:28<02:35,  2.11it/s, est. speed input: 69215.24 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2723/3051 [21:29<02:35,  2.11it/s, est. speed input: 69215.26 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2724/3051 [21:29<02:34,  2.12it/s, est. speed input: 69215.35 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2725/3051 [21:30<02:34,  2.11it/s, est. speed input: 69215.16 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2726/3051 [21:30<02:33,  2.11it/s, est. speed input: 69215.15 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2727/3051 [21:31<02:33,  2.11it/s, est. speed input: 69215.26 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2728/3051 [21:31<02:32,  2.12it/s, est. speed input: 69215.34 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2729/3051 [21:31<02:31,  2.12it/s, est. speed input: 69215.50 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2730/3051 [21:32<02:31,  2.12it/s, est. speed input: 69215.62 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2731/3051 [21:32<02:31,  2.12it/s, est. speed input: 69215.62 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2732/3051 [21:33<02:30,  2.11it/s, est. speed input: 69215.53 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2733/3051 [21:33<02:30,  2.11it/s, est. speed input: 69215.55 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2734/3051 [21:34<02:29,  2.12it/s, est. speed input: 69215.66 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2735/3051 [21:34<02:29,  2.12it/s, est. speed input: 69215.71 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2736/3051 [21:35<02:29,  2.11it/s, est. speed input: 69215.52 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2737/3051 [21:35<02:28,  2.11it/s, est. speed input: 69215.60 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2738/3051 [21:36<02:27,  2.12it/s, est. speed input: 69215.70 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2739/3051 [21:36<02:27,  2.12it/s, est. speed input: 69215.76 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2740/3051 [21:37<02:26,  2.12it/s, est. speed input: 69215.80 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2741/3051 [21:37<02:26,  2.11it/s, est. speed input: 69215.77 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2742/3051 [21:38<02:26,  2.11it/s, est. speed input: 69215.79 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2743/3051 [21:38<02:25,  2.12it/s, est. speed input: 69215.89 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2744/3051 [21:39<02:25,  2.11it/s, est. speed input: 69215.81 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2745/3051 [21:39<02:24,  2.11it/s, est. speed input: 69215.86 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2746/3051 [21:40<02:24,  2.11it/s, est. speed input: 69215.92 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2747/3051 [21:40<02:23,  2.12it/s, est. speed input: 69216.04 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2748/3051 [21:40<02:23,  2.12it/s, est. speed input: 69216.05 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2749/3051 [21:41<02:22,  2.12it/s, est. speed input: 69216.13 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2750/3051 [21:41<02:22,  2.12it/s, est. speed input: 69216.24 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2751/3051 [21:42<02:21,  2.11it/s, est. speed input: 69216.18 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2752/3051 [21:42<02:21,  2.12it/s, est. speed input: 69216.27 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2753/3051 [21:43<02:20,  2.12it/s, est. speed input: 69216.32 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2754/3051 [21:43<02:20,  2.11it/s, est. speed input: 69216.23 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2755/3051 [21:44<02:20,  2.11it/s, est. speed input: 69216.25 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2756/3051 [21:44<02:19,  2.12it/s, est. speed input: 69216.38 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2757/3051 [21:45<02:18,  2.12it/s, est. speed input: 69216.48 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2758/3051 [21:45<02:18,  2.11it/s, est. speed input: 69216.26 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2759/3051 [21:46<02:18,  2.11it/s, est. speed input: 69216.35 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2760/3051 [21:46<02:17,  2.12it/s, est. speed input: 69216.48 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2761/3051 [21:47<02:17,  2.12it/s, est. speed input: 69216.53 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2762/3051 [21:47<02:16,  2.12it/s, est. speed input: 69216.60 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2763/3051 [21:48<02:15,  2.12it/s, est. speed input: 69216.75 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2764/3051 [21:48<02:15,  2.12it/s, est. speed input: 69216.78 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2765/3051 [21:48<02:15,  2.12it/s, est. speed input: 69216.86 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2766/3051 [21:49<02:14,  2.12it/s, est. speed input: 69216.96 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2767/3051 [21:49<02:13,  2.12it/s, est. speed input: 69217.10 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2768/3051 [21:50<02:13,  2.12it/s, est. speed input: 69217.20 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2769/3051 [21:50<02:13,  2.12it/s, est. speed input: 69217.27 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2770/3051 [21:51<02:20,  2.01it/s, est. speed input: 69212.68 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2771/3051 [21:51<02:17,  2.04it/s, est. speed input: 69212.83 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2772/3051 [21:52<02:15,  2.07it/s, est. speed input: 69212.98 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2773/3051 [21:52<02:13,  2.08it/s, est. speed input: 69213.08 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2774/3051 [21:53<02:12,  2.09it/s, est. speed input: 69213.17 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2775/3051 [21:53<02:11,  2.10it/s, est. speed input: 69213.32 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2776/3051 [21:54<02:10,  2.11it/s, est. speed input: 69213.46 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2777/3051 [21:54<02:09,  2.11it/s, est. speed input: 69213.59 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2778/3051 [21:55<02:08,  2.12it/s, est. speed input: 69213.74 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2779/3051 [21:55<02:08,  2.11it/s, est. speed input: 69213.70 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2780/3051 [21:56<02:08,  2.12it/s, est. speed input: 69213.80 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2781/3051 [21:56<02:07,  2.12it/s, est. speed input: 69213.83 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2782/3051 [21:57<02:07,  2.11it/s, est. speed input: 69213.81 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2783/3051 [21:57<02:06,  2.11it/s, est. speed input: 69213.83 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2784/3051 [21:58<02:06,  2.12it/s, est. speed input: 69213.93 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2785/3051 [21:58<02:05,  2.12it/s, est. speed input: 69214.04 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2786/3051 [21:58<02:05,  2.11it/s, est. speed input: 69213.89 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2787/3051 [21:59<02:04,  2.11it/s, est. speed input: 69213.92 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2788/3051 [21:59<02:04,  2.12it/s, est. speed input: 69214.09 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2789/3051 [22:00<02:03,  2.12it/s, est. speed input: 69214.14 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2790/3051 [22:00<02:03,  2.12it/s, est. speed input: 69214.21 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2791/3051 [22:01<02:02,  2.12it/s, est. speed input: 69214.31 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2792/3051 [22:01<02:02,  2.12it/s, est. speed input: 69214.41 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2793/3051 [22:02<02:01,  2.12it/s, est. speed input: 69214.47 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2794/3051 [22:02<02:01,  2.12it/s, est. speed input: 69214.50 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2795/3051 [22:03<02:00,  2.12it/s, est. speed input: 69214.60 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2796/3051 [22:03<02:00,  2.12it/s, est. speed input: 69214.66 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2797/3051 [22:04<02:00,  2.12it/s, est. speed input: 69214.63 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2798/3051 [22:04<01:59,  2.12it/s, est. speed input: 69214.69 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2799/3051 [22:05<01:58,  2.12it/s, est. speed input: 69214.81 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2800/3051 [22:05<01:58,  2.12it/s, est. speed input: 69214.83 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2801/3051 [22:06<01:57,  2.12it/s, est. speed input: 69214.97 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2802/3051 [22:06<01:57,  2.11it/s, est. speed input: 69214.73 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2803/3051 [22:07<01:57,  2.11it/s, est. speed input: 69214.79 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2804/3051 [22:07<01:56,  2.11it/s, est. speed input: 69214.84 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2805/3051 [22:07<01:56,  2.11it/s, est. speed input: 69214.81 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2806/3051 [22:08<01:55,  2.11it/s, est. speed input: 69214.80 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2807/3051 [22:08<01:55,  2.11it/s, est. speed input: 69214.83 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2808/3051 [22:09<01:55,  2.11it/s, est. speed input: 69214.56 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2809/3051 [22:09<01:54,  2.11it/s, est. speed input: 69214.62 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2810/3051 [22:10<01:54,  2.11it/s, est. speed input: 69214.71 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2811/3051 [22:10<01:53,  2.11it/s, est. speed input: 69214.75 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2812/3051 [22:11<01:53,  2.11it/s, est. speed input: 69214.81 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2813/3051 [22:11<01:52,  2.11it/s, est. speed input: 69214.80 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2814/3051 [22:12<01:52,  2.11it/s, est. speed input: 69214.84 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2815/3051 [22:12<01:51,  2.11it/s, est. speed input: 69214.83 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2816/3051 [22:13<01:51,  2.11it/s, est. speed input: 69214.79 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2817/3051 [22:13<01:50,  2.11it/s, est. speed input: 69214.82 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2818/3051 [22:14<01:50,  2.11it/s, est. speed input: 69214.84 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2819/3051 [22:14<01:50,  2.11it/s, est. speed input: 69214.69 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2820/3051 [22:15<01:49,  2.11it/s, est. speed input: 69214.74 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2821/3051 [22:15<01:48,  2.11it/s, est. speed input: 69214.80 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2822/3051 [22:16<01:48,  2.11it/s, est. speed input: 69214.88 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2823/3051 [22:16<01:47,  2.11it/s, est. speed input: 69214.91 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2824/3051 [22:16<01:47,  2.12it/s, est. speed input: 69214.96 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2825/3051 [22:17<01:46,  2.12it/s, est. speed input: 69215.00 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2826/3051 [22:17<01:46,  2.11it/s, est. speed input: 69214.89 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2827/3051 [22:18<01:46,  2.11it/s, est. speed input: 69214.88 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2828/3051 [22:18<01:45,  2.11it/s, est. speed input: 69214.87 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2829/3051 [22:19<01:45,  2.11it/s, est. speed input: 69214.91 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2830/3051 [22:19<01:44,  2.11it/s, est. speed input: 69214.95 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2831/3051 [22:20<01:44,  2.11it/s, est. speed input: 69215.00 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2832/3051 [22:20<01:43,  2.11it/s, est. speed input: 69214.94 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2833/3051 [22:21<01:43,  2.11it/s, est. speed input: 69214.99 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2834/3051 [22:21<01:42,  2.11it/s, est. speed input: 69214.92 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2835/3051 [22:22<01:42,  2.11it/s, est. speed input: 69214.86 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2836/3051 [22:22<01:41,  2.11it/s, est. speed input: 69214.92 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2837/3051 [22:23<01:41,  2.11it/s, est. speed input: 69214.92 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2838/3051 [22:23<01:40,  2.11it/s, est. speed input: 69214.95 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2839/3051 [22:24<01:40,  2.11it/s, est. speed input: 69214.94 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2840/3051 [22:24<01:39,  2.11it/s, est. speed input: 69214.93 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2841/3051 [22:24<01:39,  2.11it/s, est. speed input: 69215.01 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2842/3051 [22:25<01:38,  2.11it/s, est. speed input: 69215.02 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2843/3051 [22:25<01:38,  2.11it/s, est. speed input: 69215.07 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2844/3051 [22:26<01:37,  2.12it/s, est. speed input: 69215.13 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2845/3051 [22:26<01:37,  2.11it/s, est. speed input: 69215.08 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2846/3051 [22:27<01:36,  2.11it/s, est. speed input: 69215.11 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2847/3051 [22:27<01:36,  2.11it/s, est. speed input: 69214.87 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2848/3051 [22:28<01:36,  2.11it/s, est. speed input: 69214.82 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2849/3051 [22:28<01:35,  2.11it/s, est. speed input: 69214.86 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2850/3051 [22:29<01:35,  2.11it/s, est. speed input: 69214.81 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2851/3051 [22:29<01:34,  2.11it/s, est. speed input: 69214.83 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2852/3051 [22:30<01:34,  2.11it/s, est. speed input: 69214.89 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2853/3051 [22:30<01:34,  2.10it/s, est. speed input: 69214.59 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2854/3051 [22:31<01:33,  2.11it/s, est. speed input: 69214.63 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2855/3051 [22:31<01:32,  2.11it/s, est. speed input: 69214.67 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2856/3051 [22:32<01:32,  2.11it/s, est. speed input: 69214.74 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2857/3051 [22:32<01:31,  2.11it/s, est. speed input: 69214.79 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2858/3051 [22:33<01:31,  2.11it/s, est. speed input: 69214.81 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2859/3051 [22:33<01:30,  2.12it/s, est. speed input: 69214.90 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2860/3051 [22:33<01:30,  2.11it/s, est. speed input: 69214.88 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2861/3051 [22:34<01:30,  2.11it/s, est. speed input: 69214.69 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2862/3051 [22:34<01:29,  2.11it/s, est. speed input: 69214.72 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2863/3051 [22:35<01:28,  2.11it/s, est. speed input: 69214.80 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2864/3051 [22:35<01:28,  2.11it/s, est. speed input: 69214.82 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2865/3051 [22:36<01:27,  2.11it/s, est. speed input: 69214.88 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2866/3051 [22:36<01:27,  2.12it/s, est. speed input: 69214.93 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2867/3051 [22:37<01:26,  2.12it/s, est. speed input: 69214.96 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2868/3051 [22:37<01:26,  2.12it/s, est. speed input: 69215.00 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2869/3051 [22:38<01:26,  2.12it/s, est. speed input: 69215.04 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2870/3051 [22:38<01:25,  2.12it/s, est. speed input: 69215.11 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2871/3051 [22:39<01:25,  2.12it/s, est. speed input: 69215.13 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2872/3051 [22:39<01:24,  2.12it/s, est. speed input: 69215.17 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2873/3051 [22:40<01:24,  2.11it/s, est. speed input: 69215.18 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2874/3051 [22:40<01:23,  2.11it/s, est. speed input: 69215.12 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2875/3051 [22:41<01:23,  2.11it/s, est. speed input: 69215.16 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2876/3051 [22:41<01:22,  2.11it/s, est. speed input: 69215.19 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2877/3051 [22:42<01:22,  2.11it/s, est. speed input: 69215.18 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2878/3051 [22:42<01:21,  2.11it/s, est. speed input: 69215.20 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2879/3051 [22:42<01:21,  2.11it/s, est. speed input: 69215.26 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2880/3051 [22:43<01:20,  2.11it/s, est. speed input: 69215.27 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2881/3051 [22:43<01:20,  2.12it/s, est. speed input: 69215.34 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2882/3051 [22:44<01:19,  2.12it/s, est. speed input: 69215.39 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2883/3051 [22:44<01:19,  2.11it/s, est. speed input: 69215.38 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2884/3051 [22:45<01:19,  2.11it/s, est. speed input: 69215.30 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2885/3051 [22:45<01:18,  2.11it/s, est. speed input: 69215.26 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2886/3051 [22:46<01:18,  2.11it/s, est. speed input: 69215.25 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2887/3051 [22:46<01:17,  2.11it/s, est. speed input: 69215.29 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2888/3051 [22:47<01:17,  2.11it/s, est. speed input: 69215.34 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2889/3051 [22:47<01:16,  2.11it/s, est. speed input: 69215.08 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2890/3051 [22:48<01:16,  2.11it/s, est. speed input: 69215.10 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2891/3051 [22:48<01:15,  2.11it/s, est. speed input: 69215.15 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2892/3051 [22:49<01:15,  2.11it/s, est. speed input: 69215.13 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2893/3051 [22:49<01:14,  2.11it/s, est. speed input: 69215.14 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2894/3051 [22:50<01:14,  2.11it/s, est. speed input: 69215.17 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2895/3051 [22:50<01:13,  2.11it/s, est. speed input: 69215.08 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2896/3051 [22:51<01:13,  2.11it/s, est. speed input: 69215.05 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2897/3051 [22:51<01:12,  2.11it/s, est. speed input: 69215.07 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2898/3051 [22:51<01:12,  2.11it/s, est. speed input: 69215.03 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2899/3051 [22:52<01:12,  2.11it/s, est. speed input: 69215.01 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2900/3051 [22:52<01:11,  2.11it/s, est. speed input: 69215.04 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2901/3051 [22:53<01:10,  2.11it/s, est. speed input: 69215.08 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2902/3051 [22:53<01:10,  2.11it/s, est. speed input: 69215.13 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2903/3051 [22:54<01:10,  2.11it/s, est. speed input: 69215.11 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2904/3051 [22:54<01:09,  2.11it/s, est. speed input: 69215.14 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2905/3051 [22:55<01:09,  2.11it/s, est. speed input: 69215.18 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2906/3051 [22:55<01:08,  2.11it/s, est. speed input: 69215.22 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2907/3051 [22:56<01:08,  2.11it/s, est. speed input: 69215.25 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2908/3051 [22:56<01:07,  2.11it/s, est. speed input: 69215.14 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2909/3051 [22:57<01:07,  2.11it/s, est. speed input: 69215.09 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2910/3051 [22:57<01:06,  2.11it/s, est. speed input: 69215.15 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2911/3051 [22:58<01:06,  2.11it/s, est. speed input: 69215.08 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2912/3051 [22:58<01:05,  2.11it/s, est. speed input: 69215.05 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2913/3051 [22:59<01:05,  2.11it/s, est. speed input: 69215.09 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2914/3051 [22:59<01:04,  2.11it/s, est. speed input: 69215.08 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2915/3051 [23:00<01:04,  2.10it/s, est. speed input: 69214.80 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2916/3051 [23:00<01:04,  2.11it/s, est. speed input: 69214.80 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2917/3051 [23:00<01:03,  2.11it/s, est. speed input: 69214.88 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2918/3051 [23:01<01:02,  2.11it/s, est. speed input: 69214.93 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2919/3051 [23:01<01:02,  2.11it/s, est. speed input: 69214.90 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2920/3051 [23:02<01:01,  2.11it/s, est. speed input: 69214.94 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2921/3051 [23:02<01:01,  2.11it/s, est. speed input: 69214.97 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2922/3051 [23:03<01:01,  2.11it/s, est. speed input: 69214.98 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2923/3051 [23:03<01:00,  2.11it/s, est. speed input: 69214.99 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2924/3051 [23:04<01:00,  2.11it/s, est. speed input: 69215.00 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2925/3051 [23:04<00:59,  2.11it/s, est. speed input: 69214.97 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2926/3051 [23:05<00:59,  2.11it/s, est. speed input: 69215.04 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2927/3051 [23:05<00:58,  2.11it/s, est. speed input: 69215.04 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2928/3051 [23:06<00:58,  2.11it/s, est. speed input: 69215.00 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2929/3051 [23:06<00:57,  2.11it/s, est. speed input: 69215.03 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2930/3051 [23:07<00:57,  2.11it/s, est. speed input: 69214.78 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2931/3051 [23:07<00:56,  2.11it/s, est. speed input: 69214.83 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2932/3051 [23:08<00:56,  2.11it/s, est. speed input: 69214.89 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2933/3051 [23:08<00:55,  2.11it/s, est. speed input: 69214.91 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2934/3051 [23:09<00:55,  2.11it/s, est. speed input: 69214.85 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2935/3051 [23:09<00:54,  2.11it/s, est. speed input: 69214.89 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2936/3051 [23:09<00:54,  2.11it/s, est. speed input: 69214.94 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2937/3051 [23:10<00:53,  2.11it/s, est. speed input: 69214.96 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2938/3051 [23:10<00:53,  2.11it/s, est. speed input: 69214.93 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2939/3051 [23:11<00:53,  2.11it/s, est. speed input: 69214.95 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2940/3051 [23:11<00:52,  2.11it/s, est. speed input: 69214.97 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2941/3051 [23:12<00:52,  2.11it/s, est. speed input: 69214.84 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2942/3051 [23:12<00:51,  2.11it/s, est. speed input: 69214.91 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2943/3051 [23:13<00:51,  2.11it/s, est. speed input: 69214.95 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2944/3051 [23:13<00:50,  2.11it/s, est. speed input: 69215.02 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2945/3051 [23:14<00:50,  2.12it/s, est. speed input: 69215.06 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2946/3051 [23:14<00:49,  2.12it/s, est. speed input: 69215.13 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2947/3051 [23:15<00:49,  2.12it/s, est. speed input: 69215.16 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2948/3051 [23:15<00:48,  2.11it/s, est. speed input: 69215.11 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2949/3051 [23:16<00:48,  2.11it/s, est. speed input: 69215.16 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2950/3051 [23:16<00:47,  2.11it/s, est. speed input: 69215.20 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2951/3051 [23:17<00:47,  2.12it/s, est. speed input: 69215.25 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2952/3051 [23:17<00:46,  2.12it/s, est. speed input: 69215.31 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2953/3051 [23:18<00:46,  2.11it/s, est. speed input: 69215.19 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2954/3051 [23:18<00:45,  2.11it/s, est. speed input: 69215.14 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2955/3051 [23:18<00:45,  2.11it/s, est. speed input: 69215.19 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2956/3051 [23:19<00:44,  2.11it/s, est. speed input: 69215.22 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2957/3051 [23:19<00:44,  2.11it/s, est. speed input: 69215.27 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2958/3051 [23:20<00:43,  2.11it/s, est. speed input: 69215.31 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2959/3051 [23:20<00:43,  2.12it/s, est. speed input: 69215.36 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2960/3051 [23:21<00:43,  2.12it/s, est. speed input: 69215.42 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2961/3051 [23:21<00:42,  2.12it/s, est. speed input: 69215.51 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2962/3051 [23:22<00:42,  2.12it/s, est. speed input: 69215.55 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2963/3051 [23:22<00:41,  2.11it/s, est. speed input: 69215.38 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2964/3051 [23:23<00:41,  2.11it/s, est. speed input: 69215.42 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2965/3051 [23:23<00:40,  2.11it/s, est. speed input: 69215.49 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2966/3051 [23:24<00:40,  2.11it/s, est. speed input: 69215.49 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2967/3051 [23:24<00:39,  2.11it/s, est. speed input: 69215.53 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2968/3051 [23:25<00:39,  2.12it/s, est. speed input: 69215.58 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2969/3051 [23:25<00:38,  2.11it/s, est. speed input: 69215.42 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2970/3051 [23:26<00:38,  2.11it/s, est. speed input: 69215.38 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2971/3051 [23:26<00:37,  2.11it/s, est. speed input: 69215.44 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2972/3051 [23:27<00:37,  2.11it/s, est. speed input: 69215.42 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2973/3051 [23:27<00:36,  2.11it/s, est. speed input: 69215.36 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2974/3051 [23:27<00:36,  2.11it/s, est. speed input: 69215.41 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2975/3051 [23:28<00:36,  2.07it/s, est. speed input: 69213.86 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2976/3051 [23:28<00:35,  2.09it/s, est. speed input: 69214.00 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2977/3051 [23:29<00:35,  2.10it/s, est. speed input: 69214.08 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2978/3051 [23:29<00:34,  2.10it/s, est. speed input: 69214.12 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2979/3051 [23:30<00:34,  2.11it/s, est. speed input: 69214.15 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2980/3051 [23:30<00:33,  2.11it/s, est. speed input: 69214.19 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2981/3051 [23:31<00:33,  2.11it/s, est. speed input: 69214.26 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2982/3051 [23:31<00:32,  2.11it/s, est. speed input: 69214.31 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2983/3051 [23:32<00:32,  2.11it/s, est. speed input: 69214.22 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2984/3051 [23:32<00:31,  2.11it/s, est. speed input: 69214.11 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2985/3051 [23:33<00:31,  2.11it/s, est. speed input: 69214.18 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2986/3051 [23:33<00:30,  2.11it/s, est. speed input: 69214.21 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2987/3051 [23:34<00:30,  2.11it/s, est. speed input: 69214.26 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2988/3051 [23:34<00:29,  2.11it/s, est. speed input: 69214.29 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2989/3051 [23:35<00:29,  2.11it/s, est. speed input: 69214.04 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2990/3051 [23:35<00:28,  2.11it/s, est. speed input: 69214.05 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2991/3051 [23:36<00:28,  2.11it/s, est. speed input: 69214.10 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2992/3051 [23:36<00:27,  2.11it/s, est. speed input: 69214.03 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2993/3051 [23:36<00:27,  2.11it/s, est. speed input: 69214.10 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2994/3051 [23:37<00:26,  2.11it/s, est. speed input: 69214.06 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2995/3051 [23:37<00:26,  2.11it/s, est. speed input: 69214.05 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2996/3051 [23:38<00:26,  2.11it/s, est. speed input: 69214.07 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2997/3051 [23:38<00:25,  2.11it/s, est. speed input: 69214.09 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2998/3051 [23:39<00:25,  2.11it/s, est. speed input: 69214.12 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2999/3051 [23:39<00:24,  2.11it/s, est. speed input: 69214.17 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3000/3051 [23:40<00:24,  2.12it/s, est. speed input: 69214.23 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3001/3051 [23:40<00:23,  2.12it/s, est. speed input: 69214.27 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3002/3051 [23:41<00:23,  2.12it/s, est. speed input: 69214.32 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3003/3051 [23:41<00:22,  2.12it/s, est. speed input: 69214.35 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3004/3051 [23:42<00:22,  2.12it/s, est. speed input: 69214.39 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3005/3051 [23:42<00:21,  2.12it/s, est. speed input: 69214.44 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3006/3051 [23:43<00:21,  2.11it/s, est. speed input: 69214.41 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3007/3051 [23:43<00:20,  2.11it/s, est. speed input: 69214.18 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3008/3051 [23:44<00:20,  2.11it/s, est. speed input: 69214.12 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3009/3051 [23:44<00:19,  2.11it/s, est. speed input: 69214.15 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3010/3051 [23:45<00:19,  2.11it/s, est. speed input: 69214.15 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3011/3051 [23:45<00:18,  2.11it/s, est. speed input: 69214.10 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3012/3051 [23:45<00:18,  2.11it/s, est. speed input: 69214.18 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3013/3051 [23:46<00:17,  2.11it/s, est. speed input: 69214.20 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3014/3051 [23:46<00:17,  2.11it/s, est. speed input: 69214.25 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3015/3051 [23:47<00:17,  2.11it/s, est. speed input: 69214.22 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3016/3051 [23:47<00:16,  2.11it/s, est. speed input: 69214.27 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3017/3051 [23:48<00:16,  2.11it/s, est. speed input: 69214.26 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3018/3051 [23:48<00:15,  2.11it/s, est. speed input: 69214.28 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3019/3051 [23:49<00:15,  2.11it/s, est. speed input: 69214.30 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3020/3051 [23:49<00:14,  2.11it/s, est. speed input: 69214.35 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3021/3051 [23:50<00:14,  2.11it/s, est. speed input: 69214.20 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3022/3051 [23:50<00:13,  2.11it/s, est. speed input: 69214.24 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3023/3051 [23:51<00:13,  2.11it/s, est. speed input: 69214.28 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3024/3051 [23:51<00:12,  2.11it/s, est. speed input: 69214.26 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3025/3051 [23:52<00:12,  2.11it/s, est. speed input: 69214.29 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3026/3051 [23:52<00:11,  2.11it/s, est. speed input: 69214.31 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3027/3051 [23:53<00:11,  2.11it/s, est. speed input: 69214.32 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3028/3051 [23:53<00:10,  2.11it/s, est. speed input: 69214.35 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3029/3051 [23:54<00:10,  2.11it/s, est. speed input: 69214.23 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3030/3051 [23:54<00:09,  2.11it/s, est. speed input: 69214.20 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3031/3051 [23:54<00:09,  2.11it/s, est. speed input: 69214.22 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3032/3051 [23:55<00:08,  2.11it/s, est. speed input: 69214.29 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3033/3051 [23:55<00:08,  2.11it/s, est. speed input: 69214.33 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3034/3051 [23:56<00:08,  2.11it/s, est. speed input: 69214.38 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3035/3051 [23:56<00:07,  2.12it/s, est. speed input: 69214.44 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3036/3051 [23:57<00:07,  2.11it/s, est. speed input: 69214.21 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3037/3051 [23:57<00:06,  2.11it/s, est. speed input: 69214.25 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3038/3051 [23:58<00:06,  2.11it/s, est. speed input: 69214.30 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3039/3051 [23:58<00:05,  2.11it/s, est. speed input: 69214.22 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3040/3051 [23:59<00:05,  2.11it/s, est. speed input: 69214.20 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3041/3051 [23:59<00:04,  2.11it/s, est. speed input: 69214.24 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3042/3051 [24:00<00:04,  2.11it/s, est. speed input: 69214.16 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3043/3051 [24:00<00:03,  2.11it/s, est. speed input: 69214.11 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3044/3051 [24:01<00:03,  2.11it/s, est. speed input: 69214.14 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3045/3051 [24:01<00:02,  2.11it/s, est. speed input: 69214.16 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3046/3051 [24:02<00:02,  2.11it/s, est. speed input: 69214.11 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3047/3051 [24:02<00:01,  2.11it/s, est. speed input: 69214.13 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3048/3051 [24:03<00:01,  2.11it/s, est. speed input: 69214.20 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3049/3051 [24:03<00:00,  2.11it/s, est. speed input: 69214.20 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3050/3051 [24:03<00:00,  2.11it/s, est. speed input: 69214.20 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 3051/3051 [24:04<00:00,  2.11it/s, est. speed input: 69214.25 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 3051/3051 [24:04<00:00,  2.11it/s, est. speed input: 69214.25 toks/s, output: 2.11 toks/s]
Saved activation counts to new_activations/activation.zh.train.q2.5-en+zh.pt
Processed 99975168 tokens total

Loading IDs from ./ids/q2.5/id.en.train.qwen2.5-0.5 (lang: en)...
ID shape: torch.Size([100000000])
Processing 3051 sequences of length 32768
Running inference...

Processed prompts:   0%|          | 0/3051 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts:   0%|          | 1/3051 [00:00<25:34,  1.99it/s, est. speed input: 65117.66 toks/s, output: 1.99 toks/s]
Processed prompts:   0%|          | 2/3051 [00:00<24:31,  2.07it/s, est. speed input: 67482.82 toks/s, output: 2.06 toks/s]
Processed prompts:   0%|          | 3/3051 [00:01<24:10,  2.10it/s, est. speed input: 68288.32 toks/s, output: 2.08 toks/s]
Processed prompts:   0%|          | 4/3051 [00:01<24:01,  2.11it/s, est. speed input: 68689.79 toks/s, output: 2.10 toks/s]
Processed prompts:   0%|          | 5/3051 [00:02<23:55,  2.12it/s, est. speed input: 68950.08 toks/s, output: 2.10 toks/s]
Processed prompts:   0%|          | 6/3051 [00:02<23:52,  2.13it/s, est. speed input: 69105.57 toks/s, output: 2.11 toks/s]
Processed prompts:   0%|          | 7/3051 [00:03<23:50,  2.13it/s, est. speed input: 69223.68 toks/s, output: 2.11 toks/s]
Processed prompts:   0%|          | 8/3051 [00:03<23:48,  2.13it/s, est. speed input: 69302.80 toks/s, output: 2.11 toks/s]
Processed prompts:   0%|          | 9/3051 [00:04<23:47,  2.13it/s, est. speed input: 69367.18 toks/s, output: 2.12 toks/s]
Processed prompts:   0%|          | 10/3051 [00:04<24:26,  2.07it/s, est. speed input: 68804.88 toks/s, output: 2.10 toks/s]
Processed prompts:   0%|          | 11/3051 [00:05<24:13,  2.09it/s, est. speed input: 68903.17 toks/s, output: 2.10 toks/s]
Processed prompts:   0%|          | 12/3051 [00:05<24:03,  2.10it/s, est. speed input: 68990.17 toks/s, output: 2.11 toks/s]
Processed prompts:   0%|          | 13/3051 [00:06<23:57,  2.11it/s, est. speed input: 69060.29 toks/s, output: 2.11 toks/s]
Processed prompts:   0%|          | 14/3051 [00:06<23:52,  2.12it/s, est. speed input: 69120.15 toks/s, output: 2.11 toks/s]
Processed prompts:   0%|          | 15/3051 [00:07<23:49,  2.12it/s, est. speed input: 69170.35 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 16/3051 [00:07<23:47,  2.13it/s, est. speed input: 69215.96 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 17/3051 [00:08<23:45,  2.13it/s, est. speed input: 69255.85 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 18/3051 [00:08<23:44,  2.13it/s, est. speed input: 69286.71 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 19/3051 [00:08<23:43,  2.13it/s, est. speed input: 69319.28 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 20/3051 [00:09<23:42,  2.13it/s, est. speed input: 69342.97 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 21/3051 [00:09<23:42,  2.13it/s, est. speed input: 69363.63 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 22/3051 [00:10<23:41,  2.13it/s, est. speed input: 69386.65 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 23/3051 [00:10<23:40,  2.13it/s, est. speed input: 69407.98 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 24/3051 [00:11<23:40,  2.13it/s, est. speed input: 69426.66 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 25/3051 [00:11<23:39,  2.13it/s, est. speed input: 69445.31 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 26/3051 [00:12<23:38,  2.13it/s, est. speed input: 69461.62 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 27/3051 [00:12<23:37,  2.13it/s, est. speed input: 69478.61 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 28/3051 [00:13<23:37,  2.13it/s, est. speed input: 69490.08 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 29/3051 [00:13<23:37,  2.13it/s, est. speed input: 69502.61 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 30/3051 [00:14<23:36,  2.13it/s, est. speed input: 69515.15 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 31/3051 [00:14<23:36,  2.13it/s, est. speed input: 69527.75 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 32/3051 [00:15<23:36,  2.13it/s, est. speed input: 69536.52 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 33/3051 [00:15<23:35,  2.13it/s, est. speed input: 69545.14 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 34/3051 [00:16<23:35,  2.13it/s, est. speed input: 69553.43 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 35/3051 [00:16<23:34,  2.13it/s, est. speed input: 69562.07 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 36/3051 [00:16<23:34,  2.13it/s, est. speed input: 69570.32 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 37/3051 [00:17<23:33,  2.13it/s, est. speed input: 69578.47 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 38/3051 [00:17<23:33,  2.13it/s, est. speed input: 69584.55 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 39/3051 [00:18<23:33,  2.13it/s, est. speed input: 69591.49 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 40/3051 [00:18<23:32,  2.13it/s, est. speed input: 69598.57 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 41/3051 [00:19<23:32,  2.13it/s, est. speed input: 69603.44 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 42/3051 [00:19<23:31,  2.13it/s, est. speed input: 69611.38 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 43/3051 [00:20<23:30,  2.13it/s, est. speed input: 69617.24 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 44/3051 [00:20<23:30,  2.13it/s, est. speed input: 69620.65 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 45/3051 [00:21<23:30,  2.13it/s, est. speed input: 69625.05 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 46/3051 [00:21<23:30,  2.13it/s, est. speed input: 69628.08 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 47/3051 [00:22<23:30,  2.13it/s, est. speed input: 69631.63 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 48/3051 [00:22<23:29,  2.13it/s, est. speed input: 69636.65 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 49/3051 [00:23<23:28,  2.13it/s, est. speed input: 69640.31 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 50/3051 [00:23<23:28,  2.13it/s, est. speed input: 69644.02 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 51/3051 [00:23<23:27,  2.13it/s, est. speed input: 69647.95 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 52/3051 [00:24<23:27,  2.13it/s, est. speed input: 69650.61 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 53/3051 [00:24<23:27,  2.13it/s, est. speed input: 69653.17 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 54/3051 [00:25<23:26,  2.13it/s, est. speed input: 69656.17 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 55/3051 [00:25<23:26,  2.13it/s, est. speed input: 69659.35 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 56/3051 [00:26<23:25,  2.13it/s, est. speed input: 69661.58 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 57/3051 [00:26<23:25,  2.13it/s, est. speed input: 69663.23 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 58/3051 [00:27<23:25,  2.13it/s, est. speed input: 69664.87 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 59/3051 [00:27<23:25,  2.13it/s, est. speed input: 69666.16 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 60/3051 [00:28<23:25,  2.13it/s, est. speed input: 69667.10 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 61/3051 [00:28<23:24,  2.13it/s, est. speed input: 69669.05 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 62/3051 [00:29<23:24,  2.13it/s, est. speed input: 69669.41 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 63/3051 [00:29<23:23,  2.13it/s, est. speed input: 69671.32 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 64/3051 [00:30<23:22,  2.13it/s, est. speed input: 69673.86 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 65/3051 [00:30<23:22,  2.13it/s, est. speed input: 69675.47 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 66/3051 [00:31<23:21,  2.13it/s, est. speed input: 69677.25 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 67/3051 [00:31<23:21,  2.13it/s, est. speed input: 69677.30 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 68/3051 [00:31<23:21,  2.13it/s, est. speed input: 69679.14 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 69/3051 [00:32<23:20,  2.13it/s, est. speed input: 69680.36 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 70/3051 [00:32<23:19,  2.13it/s, est. speed input: 69681.97 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 71/3051 [00:33<23:19,  2.13it/s, est. speed input: 69682.22 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 72/3051 [00:33<23:20,  2.13it/s, est. speed input: 69681.47 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 73/3051 [00:34<23:19,  2.13it/s, est. speed input: 69682.27 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 74/3051 [00:34<23:18,  2.13it/s, est. speed input: 69683.97 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 75/3051 [00:35<23:17,  2.13it/s, est. speed input: 69685.29 toks/s, output: 2.13 toks/s]
Processed prompts:   2%|▏         | 76/3051 [00:35<23:17,  2.13it/s, est. speed input: 69686.89 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 77/3051 [00:36<23:17,  2.13it/s, est. speed input: 69687.17 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 78/3051 [00:36<23:16,  2.13it/s, est. speed input: 69688.30 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 79/3051 [00:37<23:16,  2.13it/s, est. speed input: 69688.80 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 80/3051 [00:37<23:15,  2.13it/s, est. speed input: 69689.73 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 81/3051 [00:38<23:16,  2.13it/s, est. speed input: 69688.00 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 82/3051 [00:38<23:16,  2.13it/s, est. speed input: 69686.26 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 83/3051 [00:39<23:15,  2.13it/s, est. speed input: 69686.64 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 84/3051 [00:39<23:14,  2.13it/s, est. speed input: 69688.09 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 85/3051 [00:39<23:14,  2.13it/s, est. speed input: 69687.68 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 86/3051 [00:40<23:14,  2.13it/s, est. speed input: 69687.42 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 87/3051 [00:40<23:13,  2.13it/s, est. speed input: 69688.33 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 88/3051 [00:41<23:12,  2.13it/s, est. speed input: 69689.34 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 89/3051 [00:41<23:12,  2.13it/s, est. speed input: 69689.55 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 90/3051 [00:42<23:12,  2.13it/s, est. speed input: 69688.96 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 91/3051 [00:42<23:11,  2.13it/s, est. speed input: 69689.39 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 92/3051 [00:43<23:10,  2.13it/s, est. speed input: 69689.57 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 93/3051 [00:43<23:10,  2.13it/s, est. speed input: 69689.42 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 94/3051 [00:44<23:10,  2.13it/s, est. speed input: 69688.72 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 95/3051 [00:44<23:09,  2.13it/s, est. speed input: 69689.59 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 96/3051 [00:45<23:09,  2.13it/s, est. speed input: 69688.83 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 97/3051 [00:45<23:09,  2.13it/s, est. speed input: 69688.76 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 98/3051 [00:46<23:09,  2.13it/s, est. speed input: 69687.72 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 99/3051 [00:46<23:08,  2.13it/s, est. speed input: 69687.68 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 100/3051 [00:47<23:08,  2.13it/s, est. speed input: 69686.90 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 101/3051 [00:47<23:08,  2.13it/s, est. speed input: 69686.25 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 102/3051 [00:47<23:06,  2.13it/s, est. speed input: 69687.00 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 103/3051 [00:48<23:06,  2.13it/s, est. speed input: 69686.75 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 104/3051 [00:48<23:05,  2.13it/s, est. speed input: 69687.17 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 105/3051 [00:49<23:05,  2.13it/s, est. speed input: 69686.58 toks/s, output: 2.13 toks/s]
Processed prompts:   3%|▎         | 106/3051 [00:49<23:04,  2.13it/s, est. speed input: 69686.96 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 107/3051 [00:50<23:04,  2.13it/s, est. speed input: 69686.34 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 108/3051 [00:50<23:04,  2.13it/s, est. speed input: 69686.14 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 109/3051 [00:51<23:03,  2.13it/s, est. speed input: 69685.63 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 110/3051 [00:51<23:03,  2.13it/s, est. speed input: 69685.37 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 111/3051 [00:52<23:02,  2.13it/s, est. speed input: 69685.23 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 112/3051 [00:52<23:02,  2.13it/s, est. speed input: 69685.24 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 113/3051 [00:53<23:02,  2.13it/s, est. speed input: 69684.79 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 114/3051 [00:53<23:01,  2.13it/s, est. speed input: 69685.07 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 115/3051 [00:54<23:01,  2.13it/s, est. speed input: 69684.22 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 116/3051 [00:54<23:00,  2.13it/s, est. speed input: 69684.23 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 117/3051 [00:55<23:00,  2.13it/s, est. speed input: 69683.48 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 118/3051 [00:55<23:00,  2.13it/s, est. speed input: 69682.98 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 119/3051 [00:55<22:59,  2.13it/s, est. speed input: 69682.80 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 120/3051 [00:56<22:59,  2.12it/s, est. speed input: 69681.60 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 121/3051 [00:56<22:59,  2.12it/s, est. speed input: 69679.98 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 122/3051 [00:57<22:59,  2.12it/s, est. speed input: 69679.25 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 123/3051 [00:57<22:58,  2.12it/s, est. speed input: 69678.86 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 124/3051 [00:58<22:57,  2.12it/s, est. speed input: 69678.81 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 125/3051 [00:58<22:57,  2.12it/s, est. speed input: 69678.33 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 126/3051 [00:59<22:56,  2.12it/s, est. speed input: 69677.69 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 127/3051 [00:59<22:56,  2.12it/s, est. speed input: 69677.68 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 128/3051 [01:00<22:55,  2.13it/s, est. speed input: 69677.79 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 129/3051 [01:00<22:54,  2.13it/s, est. speed input: 69677.45 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 130/3051 [01:01<22:55,  2.12it/s, est. speed input: 69675.24 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 131/3051 [01:01<22:55,  2.12it/s, est. speed input: 69673.86 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 132/3051 [01:02<22:55,  2.12it/s, est. speed input: 69673.13 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 133/3051 [01:02<22:54,  2.12it/s, est. speed input: 69672.49 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 134/3051 [01:03<22:53,  2.12it/s, est. speed input: 69671.71 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 135/3051 [01:03<22:53,  2.12it/s, est. speed input: 69671.10 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 136/3051 [01:03<22:52,  2.12it/s, est. speed input: 69670.73 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 137/3051 [01:04<22:52,  2.12it/s, est. speed input: 69669.52 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 138/3051 [01:04<22:52,  2.12it/s, est. speed input: 69668.82 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 139/3051 [01:05<22:51,  2.12it/s, est. speed input: 69668.40 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 140/3051 [01:05<22:51,  2.12it/s, est. speed input: 69666.54 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 141/3051 [01:06<22:51,  2.12it/s, est. speed input: 69665.77 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 142/3051 [01:06<22:51,  2.12it/s, est. speed input: 69664.50 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 143/3051 [01:07<22:50,  2.12it/s, est. speed input: 69663.85 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 144/3051 [01:07<22:49,  2.12it/s, est. speed input: 69663.63 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 145/3051 [01:08<22:48,  2.12it/s, est. speed input: 69663.37 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 146/3051 [01:08<22:48,  2.12it/s, est. speed input: 69662.69 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 147/3051 [01:09<22:48,  2.12it/s, est. speed input: 69661.45 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 148/3051 [01:09<22:47,  2.12it/s, est. speed input: 69661.22 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 149/3051 [01:10<22:49,  2.12it/s, est. speed input: 69657.98 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 150/3051 [01:10<22:48,  2.12it/s, est. speed input: 69657.12 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 151/3051 [01:11<22:46,  2.12it/s, est. speed input: 69657.27 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 152/3051 [01:11<22:50,  2.11it/s, est. speed input: 69651.00 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 153/3051 [01:11<22:50,  2.11it/s, est. speed input: 69648.65 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 154/3051 [01:12<22:48,  2.12it/s, est. speed input: 69648.23 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 155/3051 [01:12<22:46,  2.12it/s, est. speed input: 69648.07 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 156/3051 [01:13<22:49,  2.11it/s, est. speed input: 69643.23 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 157/3051 [01:13<22:48,  2.12it/s, est. speed input: 69641.48 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 158/3051 [01:14<22:46,  2.12it/s, est. speed input: 69640.82 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 159/3051 [01:14<22:44,  2.12it/s, est. speed input: 69640.31 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 160/3051 [01:15<22:44,  2.12it/s, est. speed input: 69639.30 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 161/3051 [01:15<22:42,  2.12it/s, est. speed input: 69638.83 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 162/3051 [01:16<22:43,  2.12it/s, est. speed input: 69636.65 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 163/3051 [01:16<22:43,  2.12it/s, est. speed input: 69635.17 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 164/3051 [01:17<22:42,  2.12it/s, est. speed input: 69634.65 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 165/3051 [01:17<22:40,  2.12it/s, est. speed input: 69634.41 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 166/3051 [01:18<22:39,  2.12it/s, est. speed input: 69634.02 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 167/3051 [01:18<22:39,  2.12it/s, est. speed input: 69633.63 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 168/3051 [01:19<22:38,  2.12it/s, est. speed input: 69633.09 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 169/3051 [01:19<22:38,  2.12it/s, est. speed input: 69631.69 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 170/3051 [01:20<22:38,  2.12it/s, est. speed input: 69630.41 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 171/3051 [01:20<22:38,  2.12it/s, est. speed input: 69629.90 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 172/3051 [01:20<22:36,  2.12it/s, est. speed input: 69629.70 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 173/3051 [01:21<22:37,  2.12it/s, est. speed input: 69627.60 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 174/3051 [01:21<22:36,  2.12it/s, est. speed input: 69627.41 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 175/3051 [01:22<22:35,  2.12it/s, est. speed input: 69627.19 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 176/3051 [01:22<22:35,  2.12it/s, est. speed input: 69626.33 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 177/3051 [01:23<22:34,  2.12it/s, est. speed input: 69625.75 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 178/3051 [01:23<22:34,  2.12it/s, est. speed input: 69624.91 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 179/3051 [01:24<22:33,  2.12it/s, est. speed input: 69624.66 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 180/3051 [01:24<22:33,  2.12it/s, est. speed input: 69623.99 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 181/3051 [01:25<22:35,  2.12it/s, est. speed input: 69621.16 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 182/3051 [01:25<22:35,  2.12it/s, est. speed input: 69619.27 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 183/3051 [01:26<22:33,  2.12it/s, est. speed input: 69619.12 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 184/3051 [01:26<22:32,  2.12it/s, est. speed input: 69618.32 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 185/3051 [01:27<22:31,  2.12it/s, est. speed input: 69618.20 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 186/3051 [01:27<22:33,  2.12it/s, est. speed input: 69614.95 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 187/3051 [01:28<22:32,  2.12it/s, est. speed input: 69613.90 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 188/3051 [01:28<22:31,  2.12it/s, est. speed input: 69613.22 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 189/3051 [01:28<22:30,  2.12it/s, est. speed input: 69613.03 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 190/3051 [01:29<22:28,  2.12it/s, est. speed input: 69612.94 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 191/3051 [01:29<22:28,  2.12it/s, est. speed input: 69612.28 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 192/3051 [01:30<22:29,  2.12it/s, est. speed input: 69610.47 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 193/3051 [01:30<22:30,  2.12it/s, est. speed input: 69608.32 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 194/3051 [01:31<22:28,  2.12it/s, est. speed input: 69607.81 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 195/3051 [01:31<22:27,  2.12it/s, est. speed input: 69607.25 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 196/3051 [01:32<22:27,  2.12it/s, est. speed input: 69606.17 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 197/3051 [01:32<22:26,  2.12it/s, est. speed input: 69605.72 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 198/3051 [01:33<22:25,  2.12it/s, est. speed input: 69605.89 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 199/3051 [01:33<22:25,  2.12it/s, est. speed input: 69604.84 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 200/3051 [01:34<22:24,  2.12it/s, est. speed input: 69604.03 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 201/3051 [01:34<22:23,  2.12it/s, est. speed input: 69604.06 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 202/3051 [01:35<22:22,  2.12it/s, est. speed input: 69604.13 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 203/3051 [01:35<22:24,  2.12it/s, est. speed input: 69601.80 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 204/3051 [01:36<22:23,  2.12it/s, est. speed input: 69601.48 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 205/3051 [01:36<22:22,  2.12it/s, est. speed input: 69600.69 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 206/3051 [01:36<22:22,  2.12it/s, est. speed input: 69599.61 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 207/3051 [01:37<22:22,  2.12it/s, est. speed input: 69598.81 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 208/3051 [01:37<22:20,  2.12it/s, est. speed input: 69598.64 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 209/3051 [01:38<22:20,  2.12it/s, est. speed input: 69598.33 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 210/3051 [01:38<22:20,  2.12it/s, est. speed input: 69597.23 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 211/3051 [01:39<22:19,  2.12it/s, est. speed input: 69596.92 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 212/3051 [01:39<22:18,  2.12it/s, est. speed input: 69596.75 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 213/3051 [01:40<22:17,  2.12it/s, est. speed input: 69596.35 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 214/3051 [01:40<22:17,  2.12it/s, est. speed input: 69595.95 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 215/3051 [01:41<22:16,  2.12it/s, est. speed input: 69595.58 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 216/3051 [01:41<22:16,  2.12it/s, est. speed input: 69594.97 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 217/3051 [01:42<22:16,  2.12it/s, est. speed input: 69594.59 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 218/3051 [01:42<22:20,  2.11it/s, est. speed input: 69589.94 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 219/3051 [01:43<22:20,  2.11it/s, est. speed input: 69588.67 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 220/3051 [01:43<22:17,  2.12it/s, est. speed input: 69588.60 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 221/3051 [01:44<22:18,  2.11it/s, est. speed input: 69586.63 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 222/3051 [01:44<22:17,  2.12it/s, est. speed input: 69585.81 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 223/3051 [01:45<22:16,  2.12it/s, est. speed input: 69584.68 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 224/3051 [01:45<22:14,  2.12it/s, est. speed input: 69584.91 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 225/3051 [01:45<22:12,  2.12it/s, est. speed input: 69585.02 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 226/3051 [01:46<22:12,  2.12it/s, est. speed input: 69584.29 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 227/3051 [01:46<22:12,  2.12it/s, est. speed input: 69583.65 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 228/3051 [01:47<22:14,  2.11it/s, est. speed input: 69580.85 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 229/3051 [01:47<22:14,  2.11it/s, est. speed input: 69579.56 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 230/3051 [01:48<22:12,  2.12it/s, est. speed input: 69579.56 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 231/3051 [01:48<22:10,  2.12it/s, est. speed input: 69579.76 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 232/3051 [01:49<22:09,  2.12it/s, est. speed input: 69579.74 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 233/3051 [01:49<22:09,  2.12it/s, est. speed input: 69578.91 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 234/3051 [01:50<22:08,  2.12it/s, est. speed input: 69578.58 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 235/3051 [01:50<22:08,  2.12it/s, est. speed input: 69578.09 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 236/3051 [01:51<22:07,  2.12it/s, est. speed input: 69578.08 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 237/3051 [01:51<22:06,  2.12it/s, est. speed input: 69578.01 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 238/3051 [01:52<22:06,  2.12it/s, est. speed input: 69577.11 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 239/3051 [01:52<22:09,  2.12it/s, est. speed input: 69574.55 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 240/3051 [01:53<22:07,  2.12it/s, est. speed input: 69574.47 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 241/3051 [01:53<22:05,  2.12it/s, est. speed input: 69574.74 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 242/3051 [01:53<22:08,  2.12it/s, est. speed input: 69572.10 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 243/3051 [01:54<22:06,  2.12it/s, est. speed input: 69571.95 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 244/3051 [01:54<22:08,  2.11it/s, est. speed input: 69569.15 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 245/3051 [01:55<22:06,  2.12it/s, est. speed input: 69568.80 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 246/3051 [01:55<22:06,  2.12it/s, est. speed input: 69567.83 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 247/3051 [01:56<22:06,  2.11it/s, est. speed input: 69566.38 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 248/3051 [01:56<22:05,  2.11it/s, est. speed input: 69565.19 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 249/3051 [01:57<22:04,  2.12it/s, est. speed input: 69564.65 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 250/3051 [01:57<22:03,  2.12it/s, est. speed input: 69564.15 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 251/3051 [01:58<22:02,  2.12it/s, est. speed input: 69563.90 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 252/3051 [01:58<22:00,  2.12it/s, est. speed input: 69564.15 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 253/3051 [01:59<21:59,  2.12it/s, est. speed input: 69563.97 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 254/3051 [01:59<22:01,  2.12it/s, est. speed input: 69561.97 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 255/3051 [02:00<22:01,  2.12it/s, est. speed input: 69561.04 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 256/3051 [02:00<21:58,  2.12it/s, est. speed input: 69561.54 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 257/3051 [02:01<21:57,  2.12it/s, est. speed input: 69561.32 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 258/3051 [02:01<21:58,  2.12it/s, est. speed input: 69560.60 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 259/3051 [02:02<21:57,  2.12it/s, est. speed input: 69560.06 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 260/3051 [02:02<21:57,  2.12it/s, est. speed input: 69559.25 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 261/3051 [02:02<21:57,  2.12it/s, est. speed input: 69558.68 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 262/3051 [02:03<21:56,  2.12it/s, est. speed input: 69558.25 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 263/3051 [02:03<21:54,  2.12it/s, est. speed input: 69558.48 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 264/3051 [02:04<21:55,  2.12it/s, est. speed input: 69557.49 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 265/3051 [02:04<21:54,  2.12it/s, est. speed input: 69557.35 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 266/3051 [02:05<21:53,  2.12it/s, est. speed input: 69556.94 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 267/3051 [02:05<21:52,  2.12it/s, est. speed input: 69556.99 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 268/3051 [02:06<21:54,  2.12it/s, est. speed input: 69555.52 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 269/3051 [02:06<21:53,  2.12it/s, est. speed input: 69554.75 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 270/3051 [02:07<21:52,  2.12it/s, est. speed input: 69554.45 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 271/3051 [02:07<21:54,  2.11it/s, est. speed input: 69552.29 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 272/3051 [02:08<21:53,  2.12it/s, est. speed input: 69551.93 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 273/3051 [02:08<21:51,  2.12it/s, est. speed input: 69551.82 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 274/3051 [02:09<21:56,  2.11it/s, est. speed input: 69548.07 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 275/3051 [02:09<21:53,  2.11it/s, est. speed input: 69547.88 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 276/3051 [02:10<21:51,  2.12it/s, est. speed input: 69547.88 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 277/3051 [02:10<21:51,  2.12it/s, est. speed input: 69546.94 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 278/3051 [02:10<21:50,  2.12it/s, est. speed input: 69546.29 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 279/3051 [02:11<21:49,  2.12it/s, est. speed input: 69546.05 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 280/3051 [02:11<21:47,  2.12it/s, est. speed input: 69546.13 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 281/3051 [02:12<21:47,  2.12it/s, est. speed input: 69545.63 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 282/3051 [02:12<21:46,  2.12it/s, est. speed input: 69545.37 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 283/3051 [02:13<21:50,  2.11it/s, est. speed input: 69542.56 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 284/3051 [02:13<21:49,  2.11it/s, est. speed input: 69541.29 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 285/3051 [02:14<21:47,  2.12it/s, est. speed input: 69541.52 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 286/3051 [02:14<21:45,  2.12it/s, est. speed input: 69541.73 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 287/3051 [02:15<21:44,  2.12it/s, est. speed input: 69541.46 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 288/3051 [02:15<21:46,  2.12it/s, est. speed input: 69539.73 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 289/3051 [02:16<21:44,  2.12it/s, est. speed input: 69539.80 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 290/3051 [02:16<21:43,  2.12it/s, est. speed input: 69539.31 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 291/3051 [02:17<21:43,  2.12it/s, est. speed input: 69538.80 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 292/3051 [02:17<21:42,  2.12it/s, est. speed input: 69538.78 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 293/3051 [02:18<21:42,  2.12it/s, est. speed input: 69537.81 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 294/3051 [02:18<21:41,  2.12it/s, est. speed input: 69537.56 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 295/3051 [02:19<21:41,  2.12it/s, est. speed input: 69537.12 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 296/3051 [02:19<21:39,  2.12it/s, est. speed input: 69537.19 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 297/3051 [02:19<21:39,  2.12it/s, est. speed input: 69536.68 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 298/3051 [02:20<21:38,  2.12it/s, est. speed input: 69537.07 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 299/3051 [02:20<21:36,  2.12it/s, est. speed input: 69537.32 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 300/3051 [02:21<21:35,  2.12it/s, est. speed input: 69537.82 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 301/3051 [02:21<21:36,  2.12it/s, est. speed input: 69537.42 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 302/3051 [02:22<21:35,  2.12it/s, est. speed input: 69537.45 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 303/3051 [02:22<21:38,  2.12it/s, est. speed input: 69535.50 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 304/3051 [02:23<21:37,  2.12it/s, est. speed input: 69534.84 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 305/3051 [02:23<21:37,  2.12it/s, est. speed input: 69534.00 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 306/3051 [02:24<21:42,  2.11it/s, est. speed input: 69530.44 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 307/3051 [02:24<21:42,  2.11it/s, est. speed input: 69528.64 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 308/3051 [02:25<21:40,  2.11it/s, est. speed input: 69527.57 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 309/3051 [02:25<21:38,  2.11it/s, est. speed input: 69527.21 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 310/3051 [02:26<21:37,  2.11it/s, est. speed input: 69526.70 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 311/3051 [02:26<21:36,  2.11it/s, est. speed input: 69525.66 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 312/3051 [02:27<21:35,  2.11it/s, est. speed input: 69525.25 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 313/3051 [02:27<21:34,  2.12it/s, est. speed input: 69524.79 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 314/3051 [02:27<21:33,  2.12it/s, est. speed input: 69524.44 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 315/3051 [02:28<21:31,  2.12it/s, est. speed input: 69524.38 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 316/3051 [02:28<21:34,  2.11it/s, est. speed input: 69522.11 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 317/3051 [02:29<21:34,  2.11it/s, est. speed input: 69520.89 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 318/3051 [02:29<21:32,  2.11it/s, est. speed input: 69520.61 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 319/3051 [02:30<21:31,  2.11it/s, est. speed input: 69520.11 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 320/3051 [02:30<21:32,  2.11it/s, est. speed input: 69519.02 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 321/3051 [02:31<21:30,  2.12it/s, est. speed input: 69518.74 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 322/3051 [02:31<21:29,  2.12it/s, est. speed input: 69518.48 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 323/3051 [02:32<21:29,  2.12it/s, est. speed input: 69517.63 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 324/3051 [02:32<21:31,  2.11it/s, est. speed input: 69515.70 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 325/3051 [02:33<21:30,  2.11it/s, est. speed input: 69514.79 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 326/3051 [02:33<21:29,  2.11it/s, est. speed input: 69514.51 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 327/3051 [02:34<21:27,  2.11it/s, est. speed input: 69514.13 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 328/3051 [02:34<21:26,  2.12it/s, est. speed input: 69513.78 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 329/3051 [02:35<21:25,  2.12it/s, est. speed input: 69513.77 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 330/3051 [02:35<21:24,  2.12it/s, est. speed input: 69513.60 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 331/3051 [02:36<21:24,  2.12it/s, est. speed input: 69513.12 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 332/3051 [02:36<21:24,  2.12it/s, est. speed input: 69512.51 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 333/3051 [02:36<21:24,  2.12it/s, est. speed input: 69511.88 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 334/3051 [02:37<21:24,  2.12it/s, est. speed input: 69511.30 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 335/3051 [02:37<21:23,  2.12it/s, est. speed input: 69511.04 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 336/3051 [02:38<21:23,  2.12it/s, est. speed input: 69510.25 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 337/3051 [02:38<21:22,  2.12it/s, est. speed input: 69509.65 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 338/3051 [02:39<21:22,  2.12it/s, est. speed input: 69509.23 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 339/3051 [02:39<21:25,  2.11it/s, est. speed input: 69506.47 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 340/3051 [02:40<21:23,  2.11it/s, est. speed input: 69506.55 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 341/3051 [02:40<21:21,  2.12it/s, est. speed input: 69506.58 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 342/3051 [02:41<21:19,  2.12it/s, est. speed input: 69506.53 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 343/3051 [02:41<21:21,  2.11it/s, est. speed input: 69504.74 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 344/3051 [02:42<21:19,  2.11it/s, est. speed input: 69504.67 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 345/3051 [02:42<21:18,  2.12it/s, est. speed input: 69504.78 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 346/3051 [02:43<21:17,  2.12it/s, est. speed input: 69504.30 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 347/3051 [02:43<21:16,  2.12it/s, est. speed input: 69504.42 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 348/3051 [02:44<21:21,  2.11it/s, est. speed input: 69501.35 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 349/3051 [02:44<21:18,  2.11it/s, est. speed input: 69501.74 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 350/3051 [02:45<21:15,  2.12it/s, est. speed input: 69501.93 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 351/3051 [02:45<21:15,  2.12it/s, est. speed input: 69501.76 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 352/3051 [02:45<21:14,  2.12it/s, est. speed input: 69501.67 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 353/3051 [02:46<21:14,  2.12it/s, est. speed input: 69501.15 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 354/3051 [02:46<21:12,  2.12it/s, est. speed input: 69501.33 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 355/3051 [02:47<21:14,  2.12it/s, est. speed input: 69500.11 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 356/3051 [02:47<21:14,  2.11it/s, est. speed input: 69499.37 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 357/3051 [02:48<21:12,  2.12it/s, est. speed input: 69499.25 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 358/3051 [02:48<21:13,  2.12it/s, est. speed input: 69498.56 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 359/3051 [02:49<21:12,  2.12it/s, est. speed input: 69498.18 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 360/3051 [02:49<21:12,  2.12it/s, est. speed input: 69497.53 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 361/3051 [02:50<21:11,  2.12it/s, est. speed input: 69497.03 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 362/3051 [02:50<21:11,  2.11it/s, est. speed input: 69496.38 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 363/3051 [02:51<21:10,  2.12it/s, est. speed input: 69496.23 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 364/3051 [02:51<21:09,  2.12it/s, est. speed input: 69495.92 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 365/3051 [02:52<21:10,  2.11it/s, est. speed input: 69494.68 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 366/3051 [02:52<21:08,  2.12it/s, est. speed input: 69494.92 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 367/3051 [02:53<21:09,  2.11it/s, est. speed input: 69493.94 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 368/3051 [02:53<21:12,  2.11it/s, est. speed input: 69491.72 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 369/3051 [02:54<21:14,  2.10it/s, est. speed input: 69489.32 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 370/3051 [02:54<21:11,  2.11it/s, est. speed input: 69488.94 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 371/3051 [02:54<21:10,  2.11it/s, est. speed input: 69488.34 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 372/3051 [02:55<21:09,  2.11it/s, est. speed input: 69487.37 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 373/3051 [02:55<21:07,  2.11it/s, est. speed input: 69487.31 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 374/3051 [02:56<21:08,  2.11it/s, est. speed input: 69486.04 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 375/3051 [02:56<21:07,  2.11it/s, est. speed input: 69485.17 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 376/3051 [02:57<21:05,  2.11it/s, est. speed input: 69485.04 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 377/3051 [02:57<21:04,  2.11it/s, est. speed input: 69484.85 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 378/3051 [02:58<21:03,  2.12it/s, est. speed input: 69484.49 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 379/3051 [02:58<21:04,  2.11it/s, est. speed input: 69483.63 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 380/3051 [02:59<21:03,  2.11it/s, est. speed input: 69483.15 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 381/3051 [02:59<21:02,  2.11it/s, est. speed input: 69482.69 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 382/3051 [03:00<21:01,  2.12it/s, est. speed input: 69482.48 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 383/3051 [03:00<21:01,  2.12it/s, est. speed input: 69482.04 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 384/3051 [03:01<21:02,  2.11it/s, est. speed input: 69480.84 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 385/3051 [03:01<21:01,  2.11it/s, est. speed input: 69480.54 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 386/3051 [03:02<21:00,  2.11it/s, est. speed input: 69480.06 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 387/3051 [03:02<21:00,  2.11it/s, est. speed input: 69479.28 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 388/3051 [03:02<20:59,  2.11it/s, est. speed input: 69479.00 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 389/3051 [03:03<20:58,  2.11it/s, est. speed input: 69478.60 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 390/3051 [03:03<20:57,  2.12it/s, est. speed input: 69478.50 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 391/3051 [03:04<21:00,  2.11it/s, est. speed input: 69476.59 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 392/3051 [03:04<20:58,  2.11it/s, est. speed input: 69476.32 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 393/3051 [03:05<20:57,  2.11it/s, est. speed input: 69475.98 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 394/3051 [03:05<20:56,  2.11it/s, est. speed input: 69475.77 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 395/3051 [03:06<20:54,  2.12it/s, est. speed input: 69475.77 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 396/3051 [03:06<20:53,  2.12it/s, est. speed input: 69475.85 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 397/3051 [03:07<20:53,  2.12it/s, est. speed input: 69475.31 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 398/3051 [03:07<20:53,  2.12it/s, est. speed input: 69474.94 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 399/3051 [03:08<20:52,  2.12it/s, est. speed input: 69475.10 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 400/3051 [03:08<20:52,  2.12it/s, est. speed input: 69474.57 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 401/3051 [03:09<20:51,  2.12it/s, est. speed input: 69474.45 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 402/3051 [03:09<20:50,  2.12it/s, est. speed input: 69474.41 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 403/3051 [03:10<20:58,  2.10it/s, est. speed input: 69470.37 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 404/3051 [03:10<20:55,  2.11it/s, est. speed input: 69470.16 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 405/3051 [03:11<20:53,  2.11it/s, est. speed input: 69470.05 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 406/3051 [03:11<20:51,  2.11it/s, est. speed input: 69470.16 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 407/3051 [03:11<20:51,  2.11it/s, est. speed input: 69469.36 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 408/3051 [03:12<20:50,  2.11it/s, est. speed input: 69468.96 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 409/3051 [03:12<20:48,  2.12it/s, est. speed input: 69469.20 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 410/3051 [03:13<20:47,  2.12it/s, est. speed input: 69469.29 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 411/3051 [03:13<20:50,  2.11it/s, est. speed input: 69467.31 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 412/3051 [03:14<20:48,  2.11it/s, est. speed input: 69467.40 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 413/3051 [03:14<20:46,  2.12it/s, est. speed input: 69467.55 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 414/3051 [03:15<20:46,  2.12it/s, est. speed input: 69467.14 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 415/3051 [03:15<20:45,  2.12it/s, est. speed input: 69467.17 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 416/3051 [03:16<20:44,  2.12it/s, est. speed input: 69466.99 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 417/3051 [03:16<20:45,  2.11it/s, est. speed input: 69466.01 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 418/3051 [03:17<20:44,  2.12it/s, est. speed input: 69465.97 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 419/3051 [03:17<20:42,  2.12it/s, est. speed input: 69466.11 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 420/3051 [03:18<20:41,  2.12it/s, est. speed input: 69466.18 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 421/3051 [03:18<20:41,  2.12it/s, est. speed input: 69466.11 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 422/3051 [03:19<20:40,  2.12it/s, est. speed input: 69466.13 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 423/3051 [03:19<20:43,  2.11it/s, est. speed input: 69464.44 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 424/3051 [03:20<20:44,  2.11it/s, est. speed input: 69463.50 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 425/3051 [03:20<20:41,  2.11it/s, est. speed input: 69463.66 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 426/3051 [03:20<20:42,  2.11it/s, est. speed input: 69462.84 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 427/3051 [03:21<20:40,  2.11it/s, est. speed input: 69462.70 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 428/3051 [03:21<20:38,  2.12it/s, est. speed input: 69463.02 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 429/3051 [03:22<20:39,  2.12it/s, est. speed input: 69462.35 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 430/3051 [03:22<20:38,  2.12it/s, est. speed input: 69462.22 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 431/3051 [03:23<20:37,  2.12it/s, est. speed input: 69462.03 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 432/3051 [03:23<20:37,  2.12it/s, est. speed input: 69461.89 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 433/3051 [03:24<20:36,  2.12it/s, est. speed input: 69461.91 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 434/3051 [03:24<20:36,  2.12it/s, est. speed input: 69461.53 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 435/3051 [03:25<20:35,  2.12it/s, est. speed input: 69461.59 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 436/3051 [03:25<20:33,  2.12it/s, est. speed input: 69461.75 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 437/3051 [03:26<20:32,  2.12it/s, est. speed input: 69461.92 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 438/3051 [03:26<20:32,  2.12it/s, est. speed input: 69461.96 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 439/3051 [03:27<20:32,  2.12it/s, est. speed input: 69461.92 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 440/3051 [03:27<20:33,  2.12it/s, est. speed input: 69461.10 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 441/3051 [03:28<20:32,  2.12it/s, est. speed input: 69461.16 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 442/3051 [03:28<20:31,  2.12it/s, est. speed input: 69461.21 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 443/3051 [03:28<20:30,  2.12it/s, est. speed input: 69461.25 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 444/3051 [03:29<20:29,  2.12it/s, est. speed input: 69461.38 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 445/3051 [03:29<20:29,  2.12it/s, est. speed input: 69461.21 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 446/3051 [03:30<20:35,  2.11it/s, est. speed input: 69458.39 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 447/3051 [03:30<20:33,  2.11it/s, est. speed input: 69458.35 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 448/3051 [03:31<20:31,  2.11it/s, est. speed input: 69458.31 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 449/3051 [03:31<20:29,  2.12it/s, est. speed input: 69458.56 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 450/3051 [03:32<20:28,  2.12it/s, est. speed input: 69458.39 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 451/3051 [03:32<20:28,  2.12it/s, est. speed input: 69458.28 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 452/3051 [03:33<20:31,  2.11it/s, est. speed input: 69456.50 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 453/3051 [03:33<20:30,  2.11it/s, est. speed input: 69455.82 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 454/3051 [03:34<20:28,  2.11it/s, est. speed input: 69455.88 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 455/3051 [03:34<20:29,  2.11it/s, est. speed input: 69455.13 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 456/3051 [03:35<20:28,  2.11it/s, est. speed input: 69454.70 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 457/3051 [03:35<20:26,  2.11it/s, est. speed input: 69454.55 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 458/3051 [03:36<20:43,  2.08it/s, est. speed input: 69447.06 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 459/3051 [03:36<20:36,  2.10it/s, est. speed input: 69447.49 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 460/3051 [03:37<20:31,  2.10it/s, est. speed input: 69447.70 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 461/3051 [03:37<20:27,  2.11it/s, est. speed input: 69448.09 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 462/3051 [03:37<20:25,  2.11it/s, est. speed input: 69447.87 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 463/3051 [03:38<20:23,  2.11it/s, est. speed input: 69447.99 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 464/3051 [03:38<20:24,  2.11it/s, est. speed input: 69447.18 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 465/3051 [03:39<20:23,  2.11it/s, est. speed input: 69447.02 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 466/3051 [03:39<20:21,  2.12it/s, est. speed input: 69447.12 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 467/3051 [03:40<20:20,  2.12it/s, est. speed input: 69447.18 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 468/3051 [03:40<20:19,  2.12it/s, est. speed input: 69447.06 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 469/3051 [03:41<20:22,  2.11it/s, est. speed input: 69445.65 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 470/3051 [03:41<20:20,  2.11it/s, est. speed input: 69445.70 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 471/3051 [03:42<20:18,  2.12it/s, est. speed input: 69445.96 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 472/3051 [03:42<20:17,  2.12it/s, est. speed input: 69446.21 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 473/3051 [03:43<20:16,  2.12it/s, est. speed input: 69446.23 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 474/3051 [03:43<20:17,  2.12it/s, est. speed input: 69445.56 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 475/3051 [03:44<20:17,  2.12it/s, est. speed input: 69445.22 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 476/3051 [03:44<20:18,  2.11it/s, est. speed input: 69444.58 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 477/3051 [03:45<20:19,  2.11it/s, est. speed input: 69443.56 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 478/3051 [03:45<20:18,  2.11it/s, est. speed input: 69442.94 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 479/3051 [03:46<20:17,  2.11it/s, est. speed input: 69442.87 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 480/3051 [03:46<20:15,  2.11it/s, est. speed input: 69442.79 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 481/3051 [03:46<20:14,  2.12it/s, est. speed input: 69442.99 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 482/3051 [03:47<20:12,  2.12it/s, est. speed input: 69443.14 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 483/3051 [03:47<20:11,  2.12it/s, est. speed input: 69443.26 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 484/3051 [03:48<20:12,  2.12it/s, est. speed input: 69442.91 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 485/3051 [03:48<20:11,  2.12it/s, est. speed input: 69442.73 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 486/3051 [03:49<20:10,  2.12it/s, est. speed input: 69442.80 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 487/3051 [03:49<20:10,  2.12it/s, est. speed input: 69442.67 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 488/3051 [03:50<20:11,  2.11it/s, est. speed input: 69441.90 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 489/3051 [03:50<20:11,  2.11it/s, est. speed input: 69441.61 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 490/3051 [03:51<20:10,  2.12it/s, est. speed input: 69441.61 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 491/3051 [03:51<20:12,  2.11it/s, est. speed input: 69440.52 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 492/3051 [03:52<20:10,  2.11it/s, est. speed input: 69440.31 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 493/3051 [03:52<20:10,  2.11it/s, est. speed input: 69439.86 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 494/3051 [03:53<20:11,  2.11it/s, est. speed input: 69438.93 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 495/3051 [03:53<20:09,  2.11it/s, est. speed input: 69439.04 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 496/3051 [03:54<20:07,  2.12it/s, est. speed input: 69439.17 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 497/3051 [03:54<20:05,  2.12it/s, est. speed input: 69439.45 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 498/3051 [03:55<20:05,  2.12it/s, est. speed input: 69439.25 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 499/3051 [03:55<20:05,  2.12it/s, est. speed input: 69439.20 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 500/3051 [03:55<20:10,  2.11it/s, est. speed input: 69436.88 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 501/3051 [03:56<20:07,  2.11it/s, est. speed input: 69436.96 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 502/3051 [03:56<20:05,  2.11it/s, est. speed input: 69437.11 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 503/3051 [03:57<20:05,  2.11it/s, est. speed input: 69436.57 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 504/3051 [03:57<20:06,  2.11it/s, est. speed input: 69435.87 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 505/3051 [03:58<20:03,  2.11it/s, est. speed input: 69436.01 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 506/3051 [03:58<20:02,  2.12it/s, est. speed input: 69436.26 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 507/3051 [03:59<20:00,  2.12it/s, est. speed input: 69436.70 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 508/3051 [03:59<20:01,  2.12it/s, est. speed input: 69436.23 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 509/3051 [04:00<19:59,  2.12it/s, est. speed input: 69436.48 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 510/3051 [04:00<20:01,  2.12it/s, est. speed input: 69435.74 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 511/3051 [04:01<20:01,  2.11it/s, est. speed input: 69435.33 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 512/3051 [04:01<19:59,  2.12it/s, est. speed input: 69435.52 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 513/3051 [04:02<20:00,  2.11it/s, est. speed input: 69434.96 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 514/3051 [04:02<20:00,  2.11it/s, est. speed input: 69434.37 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 515/3051 [04:03<19:58,  2.12it/s, est. speed input: 69434.44 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 516/3051 [04:03<19:57,  2.12it/s, est. speed input: 69434.50 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 517/3051 [04:03<19:59,  2.11it/s, est. speed input: 69433.55 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 518/3051 [04:04<19:59,  2.11it/s, est. speed input: 69432.99 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 519/3051 [04:04<19:57,  2.11it/s, est. speed input: 69433.06 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 520/3051 [04:05<19:56,  2.12it/s, est. speed input: 69433.06 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 521/3051 [04:05<19:56,  2.11it/s, est. speed input: 69432.63 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 522/3051 [04:06<19:54,  2.12it/s, est. speed input: 69432.75 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 523/3051 [04:06<19:57,  2.11it/s, est. speed input: 69431.43 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 524/3051 [04:07<19:56,  2.11it/s, est. speed input: 69431.23 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 525/3051 [04:07<19:53,  2.12it/s, est. speed input: 69431.63 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 526/3051 [04:08<19:52,  2.12it/s, est. speed input: 69431.65 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 527/3051 [04:08<19:56,  2.11it/s, est. speed input: 69430.13 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 528/3051 [04:09<19:53,  2.11it/s, est. speed input: 69430.52 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 529/3051 [04:09<19:51,  2.12it/s, est. speed input: 69430.75 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 530/3051 [04:10<19:50,  2.12it/s, est. speed input: 69430.73 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 531/3051 [04:10<19:49,  2.12it/s, est. speed input: 69430.68 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 532/3051 [04:11<19:48,  2.12it/s, est. speed input: 69430.92 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 533/3051 [04:11<19:49,  2.12it/s, est. speed input: 69430.30 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 534/3051 [04:12<19:48,  2.12it/s, est. speed input: 69430.50 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 535/3051 [04:12<19:47,  2.12it/s, est. speed input: 69430.66 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 536/3051 [04:12<19:46,  2.12it/s, est. speed input: 69430.88 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 537/3051 [04:13<19:46,  2.12it/s, est. speed input: 69430.88 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 538/3051 [04:13<19:46,  2.12it/s, est. speed input: 69430.41 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 539/3051 [04:14<19:48,  2.11it/s, est. speed input: 69429.42 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 540/3051 [04:14<19:47,  2.11it/s, est. speed input: 69429.47 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 541/3051 [04:15<19:46,  2.12it/s, est. speed input: 69429.34 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 542/3051 [04:15<19:45,  2.12it/s, est. speed input: 69429.28 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 543/3051 [04:16<19:45,  2.12it/s, est. speed input: 69429.18 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 544/3051 [04:16<19:44,  2.12it/s, est. speed input: 69428.88 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 545/3051 [04:17<19:44,  2.12it/s, est. speed input: 69428.73 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 546/3051 [04:17<19:43,  2.12it/s, est. speed input: 69428.78 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 547/3051 [04:18<19:43,  2.12it/s, est. speed input: 69428.38 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 548/3051 [04:18<19:42,  2.12it/s, est. speed input: 69428.34 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 549/3051 [04:19<19:43,  2.11it/s, est. speed input: 69427.75 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 550/3051 [04:19<19:43,  2.11it/s, est. speed input: 69427.29 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 551/3051 [04:20<19:42,  2.11it/s, est. speed input: 69427.20 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 552/3051 [04:20<19:42,  2.11it/s, est. speed input: 69426.74 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 553/3051 [04:21<19:41,  2.11it/s, est. speed input: 69426.51 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 554/3051 [04:21<19:40,  2.12it/s, est. speed input: 69426.50 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 555/3051 [04:21<19:39,  2.12it/s, est. speed input: 69426.55 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 556/3051 [04:22<19:38,  2.12it/s, est. speed input: 69426.66 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 557/3051 [04:22<19:38,  2.12it/s, est. speed input: 69426.24 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 558/3051 [04:23<19:37,  2.12it/s, est. speed input: 69426.26 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 559/3051 [04:23<19:35,  2.12it/s, est. speed input: 69426.59 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 560/3051 [04:24<19:35,  2.12it/s, est. speed input: 69426.74 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 561/3051 [04:24<19:35,  2.12it/s, est. speed input: 69426.49 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 562/3051 [04:25<19:34,  2.12it/s, est. speed input: 69426.73 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 563/3051 [04:25<19:36,  2.12it/s, est. speed input: 69425.99 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 564/3051 [04:26<19:35,  2.12it/s, est. speed input: 69425.79 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 565/3051 [04:26<19:35,  2.11it/s, est. speed input: 69425.41 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 566/3051 [04:27<19:34,  2.12it/s, est. speed input: 69425.52 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 567/3051 [04:27<19:33,  2.12it/s, est. speed input: 69425.49 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 568/3051 [04:28<19:32,  2.12it/s, est. speed input: 69425.61 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 569/3051 [04:28<19:32,  2.12it/s, est. speed input: 69425.36 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 570/3051 [04:29<19:32,  2.12it/s, est. speed input: 69425.24 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 571/3051 [04:29<19:30,  2.12it/s, est. speed input: 69425.34 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 572/3051 [04:29<19:31,  2.12it/s, est. speed input: 69424.78 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 573/3051 [04:30<19:32,  2.11it/s, est. speed input: 69424.36 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 574/3051 [04:30<19:31,  2.11it/s, est. speed input: 69424.26 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 575/3051 [04:31<19:29,  2.12it/s, est. speed input: 69424.50 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 576/3051 [04:31<19:28,  2.12it/s, est. speed input: 69424.73 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 577/3051 [04:32<19:27,  2.12it/s, est. speed input: 69424.89 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 578/3051 [04:32<19:27,  2.12it/s, est. speed input: 69424.80 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 579/3051 [04:33<19:27,  2.12it/s, est. speed input: 69424.62 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 580/3051 [04:33<19:26,  2.12it/s, est. speed input: 69424.51 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 581/3051 [04:34<19:25,  2.12it/s, est. speed input: 69424.80 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 582/3051 [04:34<19:27,  2.11it/s, est. speed input: 69423.93 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 583/3051 [04:35<19:28,  2.11it/s, est. speed input: 69423.27 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 584/3051 [04:35<19:26,  2.11it/s, est. speed input: 69423.38 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 585/3051 [04:36<19:24,  2.12it/s, est. speed input: 69423.71 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 586/3051 [04:36<19:24,  2.12it/s, est. speed input: 69423.55 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 587/3051 [04:37<19:23,  2.12it/s, est. speed input: 69423.47 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 588/3051 [04:37<19:22,  2.12it/s, est. speed input: 69423.72 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 589/3051 [04:38<19:22,  2.12it/s, est. speed input: 69423.38 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 590/3051 [04:38<19:24,  2.11it/s, est. speed input: 69422.53 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 591/3051 [04:38<19:23,  2.11it/s, est. speed input: 69422.37 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 592/3051 [04:39<19:28,  2.11it/s, est. speed input: 69420.52 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 593/3051 [04:39<19:25,  2.11it/s, est. speed input: 69420.51 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 594/3051 [04:40<19:23,  2.11it/s, est. speed input: 69420.54 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 595/3051 [04:40<19:23,  2.11it/s, est. speed input: 69420.08 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 596/3051 [04:41<19:22,  2.11it/s, est. speed input: 69419.80 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 597/3051 [04:41<19:21,  2.11it/s, est. speed input: 69419.72 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 598/3051 [04:42<19:20,  2.11it/s, est. speed input: 69419.53 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 599/3051 [04:42<19:19,  2.11it/s, est. speed input: 69419.31 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 600/3051 [04:43<19:20,  2.11it/s, est. speed input: 69418.79 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 601/3051 [04:43<19:20,  2.11it/s, est. speed input: 69418.32 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 602/3051 [04:44<19:22,  2.11it/s, est. speed input: 69417.09 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 603/3051 [04:44<19:19,  2.11it/s, est. speed input: 69417.23 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 604/3051 [04:45<19:17,  2.11it/s, est. speed input: 69417.29 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 605/3051 [04:45<19:17,  2.11it/s, est. speed input: 69417.01 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 606/3051 [04:46<19:16,  2.11it/s, est. speed input: 69416.87 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 607/3051 [04:46<19:15,  2.11it/s, est. speed input: 69416.73 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 608/3051 [04:47<19:15,  2.11it/s, est. speed input: 69416.52 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 609/3051 [04:47<19:14,  2.11it/s, est. speed input: 69416.26 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 610/3051 [04:47<19:13,  2.12it/s, est. speed input: 69416.49 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 611/3051 [04:48<19:12,  2.12it/s, est. speed input: 69416.41 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 612/3051 [04:48<19:14,  2.11it/s, est. speed input: 69415.43 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 613/3051 [04:49<19:13,  2.11it/s, est. speed input: 69415.35 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 614/3051 [04:49<19:11,  2.12it/s, est. speed input: 69415.46 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 615/3051 [04:50<19:12,  2.11it/s, est. speed input: 69414.94 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 616/3051 [04:50<19:12,  2.11it/s, est. speed input: 69414.67 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 617/3051 [04:51<19:11,  2.11it/s, est. speed input: 69414.59 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 618/3051 [04:51<19:11,  2.11it/s, est. speed input: 69414.06 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 619/3051 [04:52<19:11,  2.11it/s, est. speed input: 69413.79 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 620/3051 [04:52<19:09,  2.11it/s, est. speed input: 69413.72 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 621/3051 [04:53<19:09,  2.11it/s, est. speed input: 69413.47 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 622/3051 [04:53<19:09,  2.11it/s, est. speed input: 69412.96 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 623/3051 [04:54<19:08,  2.11it/s, est. speed input: 69412.86 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 624/3051 [04:54<19:07,  2.11it/s, est. speed input: 69412.79 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 625/3051 [04:55<19:08,  2.11it/s, est. speed input: 69412.32 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 626/3051 [04:55<19:06,  2.12it/s, est. speed input: 69412.40 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 627/3051 [04:55<19:05,  2.12it/s, est. speed input: 69412.31 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 628/3051 [04:56<19:05,  2.12it/s, est. speed input: 69412.17 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 629/3051 [04:56<19:04,  2.12it/s, est. speed input: 69412.16 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 630/3051 [04:57<19:04,  2.12it/s, est. speed input: 69412.01 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 631/3051 [04:57<19:05,  2.11it/s, est. speed input: 69411.20 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 632/3051 [04:58<19:04,  2.11it/s, est. speed input: 69411.19 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 633/3051 [04:58<19:02,  2.12it/s, est. speed input: 69411.44 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 634/3051 [04:59<19:01,  2.12it/s, est. speed input: 69411.47 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 635/3051 [04:59<19:02,  2.12it/s, est. speed input: 69411.05 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 636/3051 [05:00<19:00,  2.12it/s, est. speed input: 69411.18 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 637/3051 [05:00<19:00,  2.12it/s, est. speed input: 69411.10 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 638/3051 [05:01<19:02,  2.11it/s, est. speed input: 69410.08 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 639/3051 [05:01<19:01,  2.11it/s, est. speed input: 69410.09 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 640/3051 [05:02<18:59,  2.12it/s, est. speed input: 69410.25 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 641/3051 [05:02<18:58,  2.12it/s, est. speed input: 69410.24 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 642/3051 [05:03<18:57,  2.12it/s, est. speed input: 69410.37 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 643/3051 [05:03<18:56,  2.12it/s, est. speed input: 69410.49 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 644/3051 [05:04<18:56,  2.12it/s, est. speed input: 69410.40 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 645/3051 [05:04<18:56,  2.12it/s, est. speed input: 69410.38 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 646/3051 [05:04<18:55,  2.12it/s, est. speed input: 69410.30 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 647/3051 [05:05<18:59,  2.11it/s, est. speed input: 69408.94 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 648/3051 [05:05<18:57,  2.11it/s, est. speed input: 69409.12 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 649/3051 [05:06<18:55,  2.12it/s, est. speed input: 69409.33 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 650/3051 [05:06<18:55,  2.12it/s, est. speed input: 69409.10 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 651/3051 [05:07<18:54,  2.12it/s, est. speed input: 69409.04 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 652/3051 [05:07<18:52,  2.12it/s, est. speed input: 69409.25 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 653/3051 [05:08<18:55,  2.11it/s, est. speed input: 69408.38 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 654/3051 [05:08<18:53,  2.12it/s, est. speed input: 69408.51 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 655/3051 [05:09<18:51,  2.12it/s, est. speed input: 69408.61 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 656/3051 [05:09<18:51,  2.12it/s, est. speed input: 69408.64 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 657/3051 [05:10<18:53,  2.11it/s, est. speed input: 69407.82 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 658/3051 [05:10<18:51,  2.12it/s, est. speed input: 69408.01 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 659/3051 [05:11<18:49,  2.12it/s, est. speed input: 69408.19 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 660/3051 [05:11<18:48,  2.12it/s, est. speed input: 69408.38 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 661/3051 [05:12<18:47,  2.12it/s, est. speed input: 69408.56 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 662/3051 [05:12<18:55,  2.10it/s, est. speed input: 69405.89 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 663/3051 [05:13<18:51,  2.11it/s, est. speed input: 69406.26 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 664/3051 [05:13<18:50,  2.11it/s, est. speed input: 69406.12 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 665/3051 [05:13<18:51,  2.11it/s, est. speed input: 69405.46 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 666/3051 [05:14<18:51,  2.11it/s, est. speed input: 69404.79 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 667/3051 [05:14<18:48,  2.11it/s, est. speed input: 69405.11 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 668/3051 [05:15<18:46,  2.12it/s, est. speed input: 69405.31 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 669/3051 [05:15<18:47,  2.11it/s, est. speed input: 69404.56 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 670/3051 [05:16<18:46,  2.11it/s, est. speed input: 69404.68 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 671/3051 [05:16<18:43,  2.12it/s, est. speed input: 69405.04 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 672/3051 [05:17<18:45,  2.11it/s, est. speed input: 69404.52 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 673/3051 [05:17<18:44,  2.11it/s, est. speed input: 69404.32 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 674/3051 [05:18<18:42,  2.12it/s, est. speed input: 69404.53 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 675/3051 [05:18<18:42,  2.12it/s, est. speed input: 69404.47 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 676/3051 [05:19<18:41,  2.12it/s, est. speed input: 69404.54 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 677/3051 [05:19<18:40,  2.12it/s, est. speed input: 69404.61 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 678/3051 [05:20<18:39,  2.12it/s, est. speed input: 69404.84 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 679/3051 [05:20<18:39,  2.12it/s, est. speed input: 69404.79 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 680/3051 [05:21<18:40,  2.12it/s, est. speed input: 69404.49 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 681/3051 [05:21<18:38,  2.12it/s, est. speed input: 69404.66 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 682/3051 [05:21<18:38,  2.12it/s, est. speed input: 69404.75 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 683/3051 [05:22<18:36,  2.12it/s, est. speed input: 69405.00 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 684/3051 [05:22<18:36,  2.12it/s, est. speed input: 69405.11 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 685/3051 [05:23<18:36,  2.12it/s, est. speed input: 69405.02 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 686/3051 [05:23<18:36,  2.12it/s, est. speed input: 69404.92 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 687/3051 [05:24<18:35,  2.12it/s, est. speed input: 69405.12 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 688/3051 [05:24<18:34,  2.12it/s, est. speed input: 69405.35 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 689/3051 [05:25<18:34,  2.12it/s, est. speed input: 69405.16 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 690/3051 [05:25<18:34,  2.12it/s, est. speed input: 69405.11 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 691/3051 [05:26<18:34,  2.12it/s, est. speed input: 69405.10 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 692/3051 [05:26<18:33,  2.12it/s, est. speed input: 69405.04 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 693/3051 [05:27<18:33,  2.12it/s, est. speed input: 69404.97 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 694/3051 [05:27<18:32,  2.12it/s, est. speed input: 69405.11 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 695/3051 [05:28<18:32,  2.12it/s, est. speed input: 69405.02 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 696/3051 [05:28<18:32,  2.12it/s, est. speed input: 69404.96 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 697/3051 [05:29<18:31,  2.12it/s, est. speed input: 69405.07 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 698/3051 [05:29<18:31,  2.12it/s, est. speed input: 69404.75 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 699/3051 [05:30<18:30,  2.12it/s, est. speed input: 69404.90 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 700/3051 [05:30<18:30,  2.12it/s, est. speed input: 69404.85 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 701/3051 [05:30<18:28,  2.12it/s, est. speed input: 69405.07 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 702/3051 [05:31<18:28,  2.12it/s, est. speed input: 69405.14 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 703/3051 [05:31<18:27,  2.12it/s, est. speed input: 69405.28 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 704/3051 [05:32<18:26,  2.12it/s, est. speed input: 69405.42 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 705/3051 [05:32<18:26,  2.12it/s, est. speed input: 69405.56 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 706/3051 [05:33<18:26,  2.12it/s, est. speed input: 69405.54 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 707/3051 [05:33<18:25,  2.12it/s, est. speed input: 69405.68 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 708/3051 [05:34<18:28,  2.11it/s, est. speed input: 69404.73 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 709/3051 [05:34<18:27,  2.11it/s, est. speed input: 69404.63 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 710/3051 [05:35<18:25,  2.12it/s, est. speed input: 69404.85 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 711/3051 [05:35<18:24,  2.12it/s, est. speed input: 69405.17 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 712/3051 [05:36<18:23,  2.12it/s, est. speed input: 69405.34 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 713/3051 [05:36<18:23,  2.12it/s, est. speed input: 69405.17 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 714/3051 [05:37<18:23,  2.12it/s, est. speed input: 69405.24 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 715/3051 [05:37<18:22,  2.12it/s, est. speed input: 69405.27 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 716/3051 [05:38<18:24,  2.11it/s, est. speed input: 69404.73 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 717/3051 [05:38<18:22,  2.12it/s, est. speed input: 69404.95 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 718/3051 [05:38<18:22,  2.12it/s, est. speed input: 69404.87 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 719/3051 [05:39<18:21,  2.12it/s, est. speed input: 69404.84 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 720/3051 [05:39<18:20,  2.12it/s, est. speed input: 69404.87 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 721/3051 [05:40<18:20,  2.12it/s, est. speed input: 69404.81 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 722/3051 [05:40<18:20,  2.12it/s, est. speed input: 69404.75 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 723/3051 [05:41<18:20,  2.12it/s, est. speed input: 69404.39 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 724/3051 [05:41<18:19,  2.12it/s, est. speed input: 69404.60 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 725/3051 [05:42<18:19,  2.12it/s, est. speed input: 69404.24 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 726/3051 [05:42<18:19,  2.12it/s, est. speed input: 69404.13 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 727/3051 [05:43<18:18,  2.12it/s, est. speed input: 69404.07 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 728/3051 [05:43<18:19,  2.11it/s, est. speed input: 69403.64 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 729/3051 [05:44<18:20,  2.11it/s, est. speed input: 69402.87 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 730/3051 [05:44<18:18,  2.11it/s, est. speed input: 69403.05 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 731/3051 [05:45<18:16,  2.12it/s, est. speed input: 69403.18 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 732/3051 [05:45<18:15,  2.12it/s, est. speed input: 69403.18 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 733/3051 [05:46<18:14,  2.12it/s, est. speed input: 69403.28 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 734/3051 [05:46<18:13,  2.12it/s, est. speed input: 69403.45 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 735/3051 [05:47<18:13,  2.12it/s, est. speed input: 69403.42 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 736/3051 [05:47<18:12,  2.12it/s, est. speed input: 69403.60 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 737/3051 [05:48<19:15,  2.00it/s, est. speed input: 69385.37 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 738/3051 [05:48<18:55,  2.04it/s, est. speed input: 69385.64 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 739/3051 [05:48<18:41,  2.06it/s, est. speed input: 69385.87 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 740/3051 [05:49<18:31,  2.08it/s, est. speed input: 69386.01 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 741/3051 [05:49<18:24,  2.09it/s, est. speed input: 69386.02 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 742/3051 [05:50<18:19,  2.10it/s, est. speed input: 69386.10 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 743/3051 [05:50<18:18,  2.10it/s, est. speed input: 69385.57 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 744/3051 [05:51<18:15,  2.11it/s, est. speed input: 69385.53 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 745/3051 [05:51<18:12,  2.11it/s, est. speed input: 69385.81 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 746/3051 [05:52<18:12,  2.11it/s, est. speed input: 69385.20 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 747/3051 [05:52<18:10,  2.11it/s, est. speed input: 69385.38 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 748/3051 [05:53<18:08,  2.11it/s, est. speed input: 69385.48 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 749/3051 [05:53<18:11,  2.11it/s, est. speed input: 69384.55 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 750/3051 [05:54<18:10,  2.11it/s, est. speed input: 69384.38 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 751/3051 [05:54<18:07,  2.11it/s, est. speed input: 69384.66 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 752/3051 [05:55<18:06,  2.12it/s, est. speed input: 69384.61 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 753/3051 [05:55<18:09,  2.11it/s, est. speed input: 69383.67 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 754/3051 [05:56<18:06,  2.11it/s, est. speed input: 69383.92 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 755/3051 [05:56<18:05,  2.12it/s, est. speed input: 69384.01 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 756/3051 [05:57<18:06,  2.11it/s, est. speed input: 69383.50 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 757/3051 [05:57<18:04,  2.11it/s, est. speed input: 69383.58 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 758/3051 [05:57<18:03,  2.12it/s, est. speed input: 69383.77 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 759/3051 [05:58<18:02,  2.12it/s, est. speed input: 69383.75 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 760/3051 [05:58<18:02,  2.12it/s, est. speed input: 69383.74 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 761/3051 [05:59<18:00,  2.12it/s, est. speed input: 69384.08 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 762/3051 [05:59<18:01,  2.12it/s, est. speed input: 69383.79 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 763/3051 [06:00<18:01,  2.12it/s, est. speed input: 69383.55 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 764/3051 [06:00<18:00,  2.12it/s, est. speed input: 69383.74 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 765/3051 [06:01<18:12,  2.09it/s, est. speed input: 69380.18 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 766/3051 [06:01<18:07,  2.10it/s, est. speed input: 69380.37 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 767/3051 [06:02<18:03,  2.11it/s, est. speed input: 69380.56 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 768/3051 [06:02<18:00,  2.11it/s, est. speed input: 69380.91 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 769/3051 [06:03<17:59,  2.11it/s, est. speed input: 69380.80 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 770/3051 [06:03<18:00,  2.11it/s, est. speed input: 69380.43 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 771/3051 [06:04<17:58,  2.11it/s, est. speed input: 69380.60 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 772/3051 [06:04<17:56,  2.12it/s, est. speed input: 69380.89 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 773/3051 [06:05<17:55,  2.12it/s, est. speed input: 69381.07 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 774/3051 [06:05<17:54,  2.12it/s, est. speed input: 69381.13 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 775/3051 [06:06<17:53,  2.12it/s, est. speed input: 69381.34 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 776/3051 [06:06<17:53,  2.12it/s, est. speed input: 69381.41 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 777/3051 [06:06<17:53,  2.12it/s, est. speed input: 69381.46 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 778/3051 [06:07<17:55,  2.11it/s, est. speed input: 69380.78 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 779/3051 [06:07<17:54,  2.11it/s, est. speed input: 69380.65 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 780/3051 [06:08<17:52,  2.12it/s, est. speed input: 69380.91 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 781/3051 [06:08<17:51,  2.12it/s, est. speed input: 69381.09 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 782/3051 [06:09<17:52,  2.12it/s, est. speed input: 69380.87 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 783/3051 [06:09<17:51,  2.12it/s, est. speed input: 69380.89 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 784/3051 [06:10<17:50,  2.12it/s, est. speed input: 69381.00 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 785/3051 [06:10<17:50,  2.12it/s, est. speed input: 69381.01 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 786/3051 [06:11<17:49,  2.12it/s, est. speed input: 69380.97 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 787/3051 [06:11<17:48,  2.12it/s, est. speed input: 69381.11 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 788/3051 [06:12<17:49,  2.12it/s, est. speed input: 69380.95 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 789/3051 [06:12<17:47,  2.12it/s, est. speed input: 69381.15 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 790/3051 [06:13<17:46,  2.12it/s, est. speed input: 69381.38 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 791/3051 [06:13<17:47,  2.12it/s, est. speed input: 69381.23 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 792/3051 [06:14<17:48,  2.11it/s, est. speed input: 69380.78 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 793/3051 [06:14<17:46,  2.12it/s, est. speed input: 69381.02 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 794/3051 [06:15<18:13,  2.06it/s, est. speed input: 69373.62 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 795/3051 [06:15<18:03,  2.08it/s, est. speed input: 69373.88 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 796/3051 [06:15<17:57,  2.09it/s, est. speed input: 69374.06 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 797/3051 [06:16<17:52,  2.10it/s, est. speed input: 69374.26 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 798/3051 [06:16<17:49,  2.11it/s, est. speed input: 69374.36 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 799/3051 [06:17<17:46,  2.11it/s, est. speed input: 69374.52 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 800/3051 [06:17<17:44,  2.11it/s, est. speed input: 69374.66 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 801/3051 [06:18<17:45,  2.11it/s, est. speed input: 69374.32 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 802/3051 [06:18<17:43,  2.11it/s, est. speed input: 69374.43 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 803/3051 [06:19<17:41,  2.12it/s, est. speed input: 69374.67 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 804/3051 [06:19<17:40,  2.12it/s, est. speed input: 69374.97 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 805/3051 [06:20<17:41,  2.12it/s, est. speed input: 69374.60 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 806/3051 [06:20<17:40,  2.12it/s, est. speed input: 69374.69 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 807/3051 [06:21<17:39,  2.12it/s, est. speed input: 69374.91 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 808/3051 [06:21<17:38,  2.12it/s, est. speed input: 69374.90 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 809/3051 [06:22<17:38,  2.12it/s, est. speed input: 69375.02 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 810/3051 [06:22<17:40,  2.11it/s, est. speed input: 69374.20 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 811/3051 [06:23<17:40,  2.11it/s, est. speed input: 69374.02 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 812/3051 [06:23<17:39,  2.11it/s, est. speed input: 69373.93 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 813/3051 [06:24<17:37,  2.12it/s, est. speed input: 69374.15 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 814/3051 [06:24<17:37,  2.12it/s, est. speed input: 69374.15 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 815/3051 [06:24<17:38,  2.11it/s, est. speed input: 69373.57 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 816/3051 [06:25<17:36,  2.12it/s, est. speed input: 69373.79 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 817/3051 [06:25<17:35,  2.12it/s, est. speed input: 69373.97 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 818/3051 [06:26<17:34,  2.12it/s, est. speed input: 69373.97 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 819/3051 [06:26<17:33,  2.12it/s, est. speed input: 69374.09 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 820/3051 [06:27<17:35,  2.11it/s, est. speed input: 69373.47 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 821/3051 [06:27<17:35,  2.11it/s, est. speed input: 69373.36 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 822/3051 [06:28<17:33,  2.12it/s, est. speed input: 69373.62 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 823/3051 [06:28<17:32,  2.12it/s, est. speed input: 69373.63 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 824/3051 [06:29<17:32,  2.12it/s, est. speed input: 69373.46 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 825/3051 [06:29<17:31,  2.12it/s, est. speed input: 69373.70 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 826/3051 [06:30<17:29,  2.12it/s, est. speed input: 69374.02 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 827/3051 [06:30<17:30,  2.12it/s, est. speed input: 69373.81 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 828/3051 [06:31<17:29,  2.12it/s, est. speed input: 69373.78 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 829/3051 [06:31<17:28,  2.12it/s, est. speed input: 69374.05 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 830/3051 [06:32<17:27,  2.12it/s, est. speed input: 69374.34 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 831/3051 [06:32<17:29,  2.12it/s, est. speed input: 69373.83 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 832/3051 [06:32<17:28,  2.12it/s, est. speed input: 69373.94 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 833/3051 [06:33<17:27,  2.12it/s, est. speed input: 69373.97 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 834/3051 [06:33<17:27,  2.12it/s, est. speed input: 69373.90 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 835/3051 [06:34<17:26,  2.12it/s, est. speed input: 69373.96 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 836/3051 [06:34<17:25,  2.12it/s, est. speed input: 69374.13 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 837/3051 [06:35<17:24,  2.12it/s, est. speed input: 69374.28 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 838/3051 [06:35<17:24,  2.12it/s, est. speed input: 69374.29 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 839/3051 [06:36<17:24,  2.12it/s, est. speed input: 69374.35 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 840/3051 [06:36<17:24,  2.12it/s, est. speed input: 69374.25 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 841/3051 [06:37<17:23,  2.12it/s, est. speed input: 69374.26 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 842/3051 [06:37<17:22,  2.12it/s, est. speed input: 69374.38 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 843/3051 [06:38<17:23,  2.12it/s, est. speed input: 69374.19 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 844/3051 [06:38<17:22,  2.12it/s, est. speed input: 69374.27 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 845/3051 [06:39<17:21,  2.12it/s, est. speed input: 69374.49 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 846/3051 [06:39<17:20,  2.12it/s, est. speed input: 69374.58 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 847/3051 [06:40<17:22,  2.12it/s, est. speed input: 69374.13 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 848/3051 [06:40<17:20,  2.12it/s, est. speed input: 69374.26 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 849/3051 [06:41<17:19,  2.12it/s, est. speed input: 69374.46 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 850/3051 [06:41<17:19,  2.12it/s, est. speed input: 69374.50 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 851/3051 [06:41<17:18,  2.12it/s, est. speed input: 69374.52 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 852/3051 [06:42<17:17,  2.12it/s, est. speed input: 69374.70 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 853/3051 [06:42<17:16,  2.12it/s, est. speed input: 69374.95 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 854/3051 [06:43<17:16,  2.12it/s, est. speed input: 69374.90 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 855/3051 [06:43<17:16,  2.12it/s, est. speed input: 69374.84 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 856/3051 [06:44<17:15,  2.12it/s, est. speed input: 69375.01 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 857/3051 [06:44<17:14,  2.12it/s, est. speed input: 69375.19 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 858/3051 [06:45<17:14,  2.12it/s, est. speed input: 69375.24 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 859/3051 [06:45<17:13,  2.12it/s, est. speed input: 69375.48 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 860/3051 [06:46<17:12,  2.12it/s, est. speed input: 69375.72 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 861/3051 [06:46<17:13,  2.12it/s, est. speed input: 69375.52 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 862/3051 [06:47<17:13,  2.12it/s, est. speed input: 69375.59 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 863/3051 [06:47<17:16,  2.11it/s, est. speed input: 69374.69 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 864/3051 [06:48<17:13,  2.12it/s, est. speed input: 69374.98 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 865/3051 [06:48<17:12,  2.12it/s, est. speed input: 69375.20 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 866/3051 [06:49<17:11,  2.12it/s, est. speed input: 69375.21 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 867/3051 [06:49<17:11,  2.12it/s, est. speed input: 69375.09 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 868/3051 [06:49<17:10,  2.12it/s, est. speed input: 69375.41 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 869/3051 [06:50<17:09,  2.12it/s, est. speed input: 69375.47 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 870/3051 [06:50<17:10,  2.12it/s, est. speed input: 69375.19 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 871/3051 [06:51<17:09,  2.12it/s, est. speed input: 69375.19 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 872/3051 [06:51<17:12,  2.11it/s, est. speed input: 69374.49 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 873/3051 [06:52<17:09,  2.12it/s, est. speed input: 69374.75 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 874/3051 [06:52<17:08,  2.12it/s, est. speed input: 69374.85 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 875/3051 [06:53<17:07,  2.12it/s, est. speed input: 69374.90 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 876/3051 [06:53<17:07,  2.12it/s, est. speed input: 69374.86 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 877/3051 [06:54<17:07,  2.12it/s, est. speed input: 69374.72 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 878/3051 [06:54<17:05,  2.12it/s, est. speed input: 69375.02 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 879/3051 [06:55<17:05,  2.12it/s, est. speed input: 69375.04 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 880/3051 [06:55<17:05,  2.12it/s, est. speed input: 69374.93 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 881/3051 [06:56<17:04,  2.12it/s, est. speed input: 69375.15 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 882/3051 [06:56<17:02,  2.12it/s, est. speed input: 69375.47 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 883/3051 [06:57<17:02,  2.12it/s, est. speed input: 69375.65 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 884/3051 [06:57<17:02,  2.12it/s, est. speed input: 69375.62 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 885/3051 [06:58<17:02,  2.12it/s, est. speed input: 69375.65 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 886/3051 [06:58<17:04,  2.11it/s, est. speed input: 69374.98 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 887/3051 [06:58<17:03,  2.11it/s, est. speed input: 69375.04 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 888/3051 [06:59<17:01,  2.12it/s, est. speed input: 69375.17 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 889/3051 [06:59<17:02,  2.11it/s, est. speed input: 69374.93 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 890/3051 [07:00<17:01,  2.12it/s, est. speed input: 69375.00 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 891/3051 [07:00<16:59,  2.12it/s, est. speed input: 69375.29 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 892/3051 [07:01<16:58,  2.12it/s, est. speed input: 69375.38 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 893/3051 [07:01<16:59,  2.12it/s, est. speed input: 69375.10 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 894/3051 [07:02<16:58,  2.12it/s, est. speed input: 69375.28 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 895/3051 [07:02<16:58,  2.12it/s, est. speed input: 69375.29 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 896/3051 [07:03<17:00,  2.11it/s, est. speed input: 69374.51 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 897/3051 [07:03<16:59,  2.11it/s, est. speed input: 69374.43 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 898/3051 [07:04<16:58,  2.11it/s, est. speed input: 69374.52 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 899/3051 [07:04<16:56,  2.12it/s, est. speed input: 69374.70 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 900/3051 [07:05<16:55,  2.12it/s, est. speed input: 69374.79 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 901/3051 [07:05<16:54,  2.12it/s, est. speed input: 69374.99 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 902/3051 [07:06<16:54,  2.12it/s, est. speed input: 69374.97 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 903/3051 [07:06<16:54,  2.12it/s, est. speed input: 69374.77 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 904/3051 [07:06<16:53,  2.12it/s, est. speed input: 69375.01 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 905/3051 [07:07<16:54,  2.11it/s, est. speed input: 69374.60 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 906/3051 [07:07<16:55,  2.11it/s, est. speed input: 69374.33 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 907/3051 [07:08<16:53,  2.12it/s, est. speed input: 69374.49 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 908/3051 [07:08<16:53,  2.11it/s, est. speed input: 69374.22 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 909/3051 [07:09<16:53,  2.11it/s, est. speed input: 69374.07 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 910/3051 [07:09<16:51,  2.12it/s, est. speed input: 69374.21 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 911/3051 [07:10<16:50,  2.12it/s, est. speed input: 69374.39 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 912/3051 [07:10<16:50,  2.12it/s, est. speed input: 69374.20 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 913/3051 [07:11<16:51,  2.11it/s, est. speed input: 69373.88 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 914/3051 [07:11<16:49,  2.12it/s, est. speed input: 69374.02 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 915/3051 [07:12<16:50,  2.11it/s, est. speed input: 69373.78 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 916/3051 [07:12<16:49,  2.12it/s, est. speed input: 69373.83 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 917/3051 [07:13<16:47,  2.12it/s, est. speed input: 69373.99 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 918/3051 [07:13<16:46,  2.12it/s, est. speed input: 69374.20 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 919/3051 [07:14<16:46,  2.12it/s, est. speed input: 69374.19 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 920/3051 [07:14<16:45,  2.12it/s, est. speed input: 69374.28 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 921/3051 [07:15<16:44,  2.12it/s, est. speed input: 69374.42 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 922/3051 [07:15<16:46,  2.12it/s, est. speed input: 69374.00 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 923/3051 [07:15<16:44,  2.12it/s, est. speed input: 69374.20 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 924/3051 [07:16<16:59,  2.09it/s, est. speed input: 69370.40 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 925/3051 [07:16<16:53,  2.10it/s, est. speed input: 69370.71 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 926/3051 [07:17<16:49,  2.10it/s, est. speed input: 69370.90 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 927/3051 [07:17<16:55,  2.09it/s, est. speed input: 69368.86 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 928/3051 [07:18<16:50,  2.10it/s, est. speed input: 69369.11 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 929/3051 [07:18<16:46,  2.11it/s, est. speed input: 69369.31 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 930/3051 [07:19<16:44,  2.11it/s, est. speed input: 69369.51 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 931/3051 [07:19<16:42,  2.11it/s, est. speed input: 69369.63 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 932/3051 [07:20<16:41,  2.12it/s, est. speed input: 69369.71 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 933/3051 [07:20<16:39,  2.12it/s, est. speed input: 69369.92 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 934/3051 [07:21<16:39,  2.12it/s, est. speed input: 69370.01 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 935/3051 [07:21<16:39,  2.12it/s, est. speed input: 69370.00 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 936/3051 [07:22<16:38,  2.12it/s, est. speed input: 69369.94 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 937/3051 [07:22<16:39,  2.11it/s, est. speed input: 69369.59 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 938/3051 [07:23<16:39,  2.12it/s, est. speed input: 69369.60 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 939/3051 [07:23<16:37,  2.12it/s, est. speed input: 69369.85 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 940/3051 [07:24<16:36,  2.12it/s, est. speed input: 69370.02 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 941/3051 [07:24<16:37,  2.11it/s, est. speed input: 69369.60 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 942/3051 [07:24<16:36,  2.12it/s, est. speed input: 69369.73 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 943/3051 [07:25<16:34,  2.12it/s, est. speed input: 69369.99 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 944/3051 [07:25<16:38,  2.11it/s, est. speed input: 69369.15 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 945/3051 [07:26<16:35,  2.11it/s, est. speed input: 69369.34 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 946/3051 [07:26<16:34,  2.12it/s, est. speed input: 69369.42 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 947/3051 [07:27<16:34,  2.12it/s, est. speed input: 69369.32 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 948/3051 [07:27<16:33,  2.12it/s, est. speed input: 69369.35 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 949/3051 [07:28<16:32,  2.12it/s, est. speed input: 69369.50 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 950/3051 [07:28<16:31,  2.12it/s, est. speed input: 69369.56 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 951/3051 [07:29<16:31,  2.12it/s, est. speed input: 69369.58 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 952/3051 [07:29<16:30,  2.12it/s, est. speed input: 69369.86 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 953/3051 [07:30<16:30,  2.12it/s, est. speed input: 69369.80 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 954/3051 [07:30<16:30,  2.12it/s, est. speed input: 69369.56 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 955/3051 [07:31<16:29,  2.12it/s, est. speed input: 69369.74 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 956/3051 [07:31<16:28,  2.12it/s, est. speed input: 69369.94 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 957/3051 [07:32<16:30,  2.11it/s, est. speed input: 69369.34 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 958/3051 [07:32<16:29,  2.11it/s, est. speed input: 69369.35 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 959/3051 [07:33<16:28,  2.12it/s, est. speed input: 69369.56 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 960/3051 [07:33<16:27,  2.12it/s, est. speed input: 69369.52 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 961/3051 [07:33<16:29,  2.11it/s, est. speed input: 69368.88 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 962/3051 [07:34<16:27,  2.11it/s, est. speed input: 69369.09 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 963/3051 [07:34<16:29,  2.11it/s, est. speed input: 69368.45 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 964/3051 [07:35<16:28,  2.11it/s, est. speed input: 69368.46 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 965/3051 [07:35<16:25,  2.12it/s, est. speed input: 69368.75 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 966/3051 [07:36<16:23,  2.12it/s, est. speed input: 69369.05 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 967/3051 [07:36<16:26,  2.11it/s, est. speed input: 69368.50 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 968/3051 [07:37<16:25,  2.11it/s, est. speed input: 69368.50 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 969/3051 [07:37<16:23,  2.12it/s, est. speed input: 69368.63 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 970/3051 [07:38<16:22,  2.12it/s, est. speed input: 69368.81 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 971/3051 [07:38<16:24,  2.11it/s, est. speed input: 69368.36 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 972/3051 [07:39<16:22,  2.12it/s, est. speed input: 69368.58 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 973/3051 [07:39<16:23,  2.11it/s, est. speed input: 69368.20 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 974/3051 [07:40<16:21,  2.12it/s, est. speed input: 69368.37 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 975/3051 [07:40<16:21,  2.12it/s, est. speed input: 69368.32 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 976/3051 [07:41<16:21,  2.11it/s, est. speed input: 69367.97 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 977/3051 [07:41<16:19,  2.12it/s, est. speed input: 69368.26 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 978/3051 [07:41<16:18,  2.12it/s, est. speed input: 69368.55 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 979/3051 [07:42<16:35,  2.08it/s, est. speed input: 69364.33 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 980/3051 [07:42<16:28,  2.09it/s, est. speed input: 69364.59 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 981/3051 [07:43<16:24,  2.10it/s, est. speed input: 69364.79 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 982/3051 [07:43<16:21,  2.11it/s, est. speed input: 69364.83 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 983/3051 [07:44<16:20,  2.11it/s, est. speed input: 69364.68 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 984/3051 [07:44<16:18,  2.11it/s, est. speed input: 69364.91 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 985/3051 [07:45<16:16,  2.12it/s, est. speed input: 69365.02 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 986/3051 [07:45<16:15,  2.12it/s, est. speed input: 69365.20 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 987/3051 [07:46<16:14,  2.12it/s, est. speed input: 69365.32 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 988/3051 [07:46<16:13,  2.12it/s, est. speed input: 69365.54 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 989/3051 [07:47<16:14,  2.12it/s, est. speed input: 69365.19 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 990/3051 [07:47<16:14,  2.12it/s, est. speed input: 69365.15 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 991/3051 [07:48<16:12,  2.12it/s, est. speed input: 69365.35 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 992/3051 [07:48<16:12,  2.12it/s, est. speed input: 69365.32 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 993/3051 [07:49<16:11,  2.12it/s, est. speed input: 69365.46 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 994/3051 [07:49<16:10,  2.12it/s, est. speed input: 69365.71 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 995/3051 [07:50<16:09,  2.12it/s, est. speed input: 69365.93 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 996/3051 [07:50<16:09,  2.12it/s, est. speed input: 69365.95 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 997/3051 [07:50<16:08,  2.12it/s, est. speed input: 69366.06 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 998/3051 [07:51<16:08,  2.12it/s, est. speed input: 69366.21 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 999/3051 [07:51<16:07,  2.12it/s, est. speed input: 69366.39 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1000/3051 [07:52<16:07,  2.12it/s, est. speed input: 69366.35 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1001/3051 [07:52<16:07,  2.12it/s, est. speed input: 69366.46 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1002/3051 [07:53<16:06,  2.12it/s, est. speed input: 69366.53 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1003/3051 [07:53<16:06,  2.12it/s, est. speed input: 69366.52 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1004/3051 [07:54<16:05,  2.12it/s, est. speed input: 69366.71 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1005/3051 [07:54<16:08,  2.11it/s, est. speed input: 69366.01 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1006/3051 [07:55<16:07,  2.11it/s, est. speed input: 69365.91 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1007/3051 [07:55<16:06,  2.12it/s, est. speed input: 69366.10 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1008/3051 [07:56<16:04,  2.12it/s, est. speed input: 69366.42 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1009/3051 [07:56<16:04,  2.12it/s, est. speed input: 69366.29 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1010/3051 [07:57<16:04,  2.12it/s, est. speed input: 69366.26 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1011/3051 [07:57<16:02,  2.12it/s, est. speed input: 69366.62 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1012/3051 [07:58<16:02,  2.12it/s, est. speed input: 69366.60 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1013/3051 [07:58<16:01,  2.12it/s, est. speed input: 69366.67 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1014/3051 [07:59<16:00,  2.12it/s, est. speed input: 69366.81 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1015/3051 [07:59<15:59,  2.12it/s, est. speed input: 69367.12 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1016/3051 [07:59<15:59,  2.12it/s, est. speed input: 69367.12 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1017/3051 [08:00<15:59,  2.12it/s, est. speed input: 69367.21 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1018/3051 [08:00<15:58,  2.12it/s, est. speed input: 69367.44 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1019/3051 [08:01<15:57,  2.12it/s, est. speed input: 69367.57 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1020/3051 [08:01<16:00,  2.11it/s, est. speed input: 69366.92 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1021/3051 [08:02<15:59,  2.12it/s, est. speed input: 69367.02 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 1022/3051 [08:02<15:58,  2.12it/s, est. speed input: 69367.22 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1023/3051 [08:03<15:58,  2.12it/s, est. speed input: 69367.03 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1024/3051 [08:03<15:59,  2.11it/s, est. speed input: 69366.73 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1025/3051 [08:04<15:58,  2.11it/s, est. speed input: 69366.76 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1026/3051 [08:04<15:58,  2.11it/s, est. speed input: 69366.53 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1027/3051 [08:05<15:56,  2.12it/s, est. speed input: 69366.79 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1028/3051 [08:05<15:55,  2.12it/s, est. speed input: 69366.83 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 1029/3051 [08:06<15:55,  2.12it/s, est. speed input: 69366.81 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1030/3051 [08:06<15:54,  2.12it/s, est. speed input: 69366.93 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1031/3051 [08:07<15:53,  2.12it/s, est. speed input: 69367.11 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1032/3051 [08:07<15:55,  2.11it/s, est. speed input: 69366.63 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1033/3051 [08:07<15:53,  2.12it/s, est. speed input: 69366.73 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1034/3051 [08:08<15:51,  2.12it/s, est. speed input: 69367.05 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1035/3051 [08:08<15:54,  2.11it/s, est. speed input: 69366.46 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1036/3051 [08:09<15:53,  2.11it/s, est. speed input: 69366.35 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1037/3051 [08:09<15:51,  2.12it/s, est. speed input: 69366.50 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1038/3051 [08:10<15:52,  2.11it/s, est. speed input: 69366.31 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1039/3051 [08:10<15:51,  2.11it/s, est. speed input: 69366.24 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1040/3051 [08:11<15:49,  2.12it/s, est. speed input: 69366.43 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1041/3051 [08:11<15:49,  2.12it/s, est. speed input: 69366.48 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1042/3051 [08:12<15:48,  2.12it/s, est. speed input: 69366.56 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1043/3051 [08:12<15:47,  2.12it/s, est. speed input: 69366.73 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1044/3051 [08:13<16:06,  2.08it/s, est. speed input: 69362.29 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1045/3051 [08:13<16:00,  2.09it/s, est. speed input: 69362.18 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1046/3051 [08:14<15:55,  2.10it/s, est. speed input: 69362.36 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1047/3051 [08:14<15:52,  2.10it/s, est. speed input: 69362.49 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1048/3051 [08:15<15:51,  2.10it/s, est. speed input: 69362.07 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1049/3051 [08:15<15:49,  2.11it/s, est. speed input: 69362.20 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1050/3051 [08:16<15:46,  2.11it/s, est. speed input: 69362.41 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1051/3051 [08:16<15:45,  2.12it/s, est. speed input: 69362.49 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 1052/3051 [08:16<15:44,  2.12it/s, est. speed input: 69362.60 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1053/3051 [08:17<15:43,  2.12it/s, est. speed input: 69362.78 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1054/3051 [08:17<15:42,  2.12it/s, est. speed input: 69362.79 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1055/3051 [08:18<15:42,  2.12it/s, est. speed input: 69362.73 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1056/3051 [08:18<15:41,  2.12it/s, est. speed input: 69362.85 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1057/3051 [08:19<15:41,  2.12it/s, est. speed input: 69362.78 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1058/3051 [08:19<15:41,  2.12it/s, est. speed input: 69362.76 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1059/3051 [08:20<15:40,  2.12it/s, est. speed input: 69362.93 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1060/3051 [08:20<15:46,  2.10it/s, est. speed input: 69361.32 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1061/3051 [08:21<15:43,  2.11it/s, est. speed input: 69361.51 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1062/3051 [08:21<15:41,  2.11it/s, est. speed input: 69361.79 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1063/3051 [08:22<15:41,  2.11it/s, est. speed input: 69361.54 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1064/3051 [08:22<15:39,  2.11it/s, est. speed input: 69361.59 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1065/3051 [08:23<15:39,  2.11it/s, est. speed input: 69361.57 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1066/3051 [08:23<15:39,  2.11it/s, est. speed input: 69361.41 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 1067/3051 [08:24<15:37,  2.12it/s, est. speed input: 69361.66 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1068/3051 [08:24<15:37,  2.11it/s, est. speed input: 69361.44 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1069/3051 [08:25<15:36,  2.12it/s, est. speed input: 69361.48 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1070/3051 [08:25<15:35,  2.12it/s, est. speed input: 69361.64 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1071/3051 [08:25<15:34,  2.12it/s, est. speed input: 69361.80 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1072/3051 [08:26<15:33,  2.12it/s, est. speed input: 69362.07 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1073/3051 [08:26<15:35,  2.11it/s, est. speed input: 69361.51 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1074/3051 [08:27<15:35,  2.11it/s, est. speed input: 69361.42 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1075/3051 [08:27<15:33,  2.12it/s, est. speed input: 69361.57 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1076/3051 [08:28<15:32,  2.12it/s, est. speed input: 69361.63 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1077/3051 [08:28<15:35,  2.11it/s, est. speed input: 69360.99 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1078/3051 [08:29<15:33,  2.11it/s, est. speed input: 69361.02 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1079/3051 [08:29<15:31,  2.12it/s, est. speed input: 69361.26 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1080/3051 [08:30<15:31,  2.12it/s, est. speed input: 69361.12 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1081/3051 [08:30<15:30,  2.12it/s, est. speed input: 69361.20 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1082/3051 [08:31<15:29,  2.12it/s, est. speed input: 69361.32 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 1083/3051 [08:31<15:28,  2.12it/s, est. speed input: 69361.47 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1084/3051 [08:32<15:28,  2.12it/s, est. speed input: 69361.52 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1085/3051 [08:32<15:27,  2.12it/s, est. speed input: 69361.72 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1086/3051 [08:33<15:26,  2.12it/s, est. speed input: 69361.88 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1087/3051 [08:33<15:27,  2.12it/s, est. speed input: 69361.79 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1088/3051 [08:33<15:26,  2.12it/s, est. speed input: 69361.91 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1089/3051 [08:34<15:25,  2.12it/s, est. speed input: 69361.95 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1090/3051 [08:34<15:25,  2.12it/s, est. speed input: 69362.08 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1091/3051 [08:35<15:24,  2.12it/s, est. speed input: 69362.22 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1092/3051 [08:35<15:23,  2.12it/s, est. speed input: 69362.35 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1093/3051 [08:36<15:27,  2.11it/s, est. speed input: 69361.56 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1094/3051 [08:36<15:25,  2.11it/s, est. speed input: 69361.67 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1095/3051 [08:37<15:23,  2.12it/s, est. speed input: 69361.88 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1096/3051 [08:37<15:23,  2.12it/s, est. speed input: 69361.94 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1097/3051 [08:38<15:22,  2.12it/s, est. speed input: 69362.09 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1098/3051 [08:38<15:21,  2.12it/s, est. speed input: 69362.22 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1099/3051 [08:39<15:20,  2.12it/s, est. speed input: 69362.41 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1100/3051 [08:39<15:20,  2.12it/s, est. speed input: 69362.46 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1101/3051 [08:40<15:20,  2.12it/s, est. speed input: 69362.43 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1102/3051 [08:40<15:19,  2.12it/s, est. speed input: 69362.53 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1103/3051 [08:41<15:19,  2.12it/s, est. speed input: 69362.55 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1104/3051 [08:41<15:19,  2.12it/s, est. speed input: 69362.52 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 1105/3051 [08:42<15:18,  2.12it/s, est. speed input: 69362.64 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1106/3051 [08:42<15:17,  2.12it/s, est. speed input: 69362.85 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1107/3051 [08:42<15:16,  2.12it/s, est. speed input: 69363.00 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1108/3051 [08:43<15:16,  2.12it/s, est. speed input: 69363.04 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1109/3051 [08:43<15:16,  2.12it/s, est. speed input: 69363.04 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1110/3051 [08:44<15:16,  2.12it/s, est. speed input: 69362.98 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1111/3051 [08:44<15:15,  2.12it/s, est. speed input: 69363.13 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1112/3051 [08:45<15:14,  2.12it/s, est. speed input: 69363.28 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 1113/3051 [08:45<15:13,  2.12it/s, est. speed input: 69363.54 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1114/3051 [08:46<15:12,  2.12it/s, est. speed input: 69363.72 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1115/3051 [08:46<15:13,  2.12it/s, est. speed input: 69363.67 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1116/3051 [08:47<15:13,  2.12it/s, est. speed input: 69363.58 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1117/3051 [08:47<15:13,  2.12it/s, est. speed input: 69363.53 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1118/3051 [08:48<15:12,  2.12it/s, est. speed input: 69363.71 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1119/3051 [08:48<15:11,  2.12it/s, est. speed input: 69363.93 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1120/3051 [08:49<15:13,  2.11it/s, est. speed input: 69363.31 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1121/3051 [08:49<15:12,  2.11it/s, est. speed input: 69363.31 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1122/3051 [08:50<15:11,  2.12it/s, est. speed input: 69363.49 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1123/3051 [08:50<15:10,  2.12it/s, est. speed input: 69363.48 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1124/3051 [08:50<15:10,  2.12it/s, est. speed input: 69363.46 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1125/3051 [08:51<15:09,  2.12it/s, est. speed input: 69363.61 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1126/3051 [08:51<15:07,  2.12it/s, est. speed input: 69363.84 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1127/3051 [08:52<15:07,  2.12it/s, est. speed input: 69363.86 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1128/3051 [08:52<15:07,  2.12it/s, est. speed input: 69363.94 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1129/3051 [08:53<15:06,  2.12it/s, est. speed input: 69364.18 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1130/3051 [08:53<15:04,  2.12it/s, est. speed input: 69364.45 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1131/3051 [08:54<15:06,  2.12it/s, est. speed input: 69364.27 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1132/3051 [08:54<15:05,  2.12it/s, est. speed input: 69364.36 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1133/3051 [08:55<15:05,  2.12it/s, est. speed input: 69364.36 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1134/3051 [08:55<15:04,  2.12it/s, est. speed input: 69364.49 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1135/3051 [08:56<15:04,  2.12it/s, est. speed input: 69364.46 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1136/3051 [08:56<15:04,  2.12it/s, est. speed input: 69364.33 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1137/3051 [08:57<15:03,  2.12it/s, est. speed input: 69364.45 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1138/3051 [08:57<15:02,  2.12it/s, est. speed input: 69364.61 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1139/3051 [08:58<15:02,  2.12it/s, est. speed input: 69364.66 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1140/3051 [08:58<15:03,  2.11it/s, est. speed input: 69364.31 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1141/3051 [08:59<15:02,  2.12it/s, est. speed input: 69364.38 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1142/3051 [08:59<15:01,  2.12it/s, est. speed input: 69364.51 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1143/3051 [08:59<15:01,  2.12it/s, est. speed input: 69364.39 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 1144/3051 [09:00<15:00,  2.12it/s, est. speed input: 69364.50 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1145/3051 [09:00<15:05,  2.10it/s, est. speed input: 69363.28 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1146/3051 [09:01<15:02,  2.11it/s, est. speed input: 69363.49 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1147/3051 [09:01<15:00,  2.11it/s, est. speed input: 69363.65 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1148/3051 [09:02<14:59,  2.11it/s, est. speed input: 69363.62 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1149/3051 [09:02<15:00,  2.11it/s, est. speed input: 69363.37 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1150/3051 [09:03<14:58,  2.12it/s, est. speed input: 69363.52 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1151/3051 [09:03<14:57,  2.12it/s, est. speed input: 69363.67 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1152/3051 [09:04<14:56,  2.12it/s, est. speed input: 69363.69 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1153/3051 [09:04<14:56,  2.12it/s, est. speed input: 69363.73 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1154/3051 [09:05<14:55,  2.12it/s, est. speed input: 69363.87 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1155/3051 [09:05<14:55,  2.12it/s, est. speed input: 69363.71 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1156/3051 [09:06<14:55,  2.12it/s, est. speed input: 69363.71 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1157/3051 [09:06<14:54,  2.12it/s, est. speed input: 69363.86 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1158/3051 [09:07<14:53,  2.12it/s, est. speed input: 69363.99 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1159/3051 [09:07<14:53,  2.12it/s, est. speed input: 69364.00 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1160/3051 [09:07<14:52,  2.12it/s, est. speed input: 69364.04 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1161/3051 [09:08<14:51,  2.12it/s, est. speed input: 69364.22 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1162/3051 [09:08<14:52,  2.12it/s, est. speed input: 69364.08 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1163/3051 [09:09<14:51,  2.12it/s, est. speed input: 69364.16 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1164/3051 [09:09<14:50,  2.12it/s, est. speed input: 69364.33 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1165/3051 [09:10<14:49,  2.12it/s, est. speed input: 69364.53 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1166/3051 [09:10<14:48,  2.12it/s, est. speed input: 69364.60 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1167/3051 [09:11<14:48,  2.12it/s, est. speed input: 69364.80 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1168/3051 [09:11<14:49,  2.12it/s, est. speed input: 69364.61 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1169/3051 [09:12<14:48,  2.12it/s, est. speed input: 69364.65 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1170/3051 [09:12<14:47,  2.12it/s, est. speed input: 69364.76 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1171/3051 [09:13<14:46,  2.12it/s, est. speed input: 69365.01 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1172/3051 [09:13<14:46,  2.12it/s, est. speed input: 69364.92 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1173/3051 [09:14<14:47,  2.12it/s, est. speed input: 69364.77 toks/s, output: 2.12 toks/s]
Processed prompts:  38%|███▊      | 1174/3051 [09:14<14:45,  2.12it/s, est. speed input: 69364.99 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1175/3051 [09:15<14:45,  2.12it/s, est. speed input: 69365.09 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1176/3051 [09:15<14:44,  2.12it/s, est. speed input: 69365.10 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1177/3051 [09:16<14:43,  2.12it/s, est. speed input: 69365.27 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1178/3051 [09:16<14:42,  2.12it/s, est. speed input: 69365.48 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1179/3051 [09:16<14:45,  2.11it/s, est. speed input: 69364.94 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1180/3051 [09:17<14:43,  2.12it/s, est. speed input: 69365.19 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1181/3051 [09:17<14:44,  2.11it/s, est. speed input: 69364.91 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▊      | 1182/3051 [09:18<14:45,  2.11it/s, est. speed input: 69364.51 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1183/3051 [09:18<14:43,  2.11it/s, est. speed input: 69364.69 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1184/3051 [09:19<14:44,  2.11it/s, est. speed input: 69364.23 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1185/3051 [09:19<14:43,  2.11it/s, est. speed input: 69364.12 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1186/3051 [09:20<14:41,  2.12it/s, est. speed input: 69364.38 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1187/3051 [09:20<14:40,  2.12it/s, est. speed input: 69364.53 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1188/3051 [09:21<14:41,  2.11it/s, est. speed input: 69364.26 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1189/3051 [09:21<14:39,  2.12it/s, est. speed input: 69364.46 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1190/3051 [09:22<14:38,  2.12it/s, est. speed input: 69364.64 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1191/3051 [09:22<14:36,  2.12it/s, est. speed input: 69364.86 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1192/3051 [09:23<14:36,  2.12it/s, est. speed input: 69364.88 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1193/3051 [09:23<14:36,  2.12it/s, est. speed input: 69364.95 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1194/3051 [09:24<14:36,  2.12it/s, est. speed input: 69364.98 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1195/3051 [09:24<14:36,  2.12it/s, est. speed input: 69364.83 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1196/3051 [09:24<14:36,  2.12it/s, est. speed input: 69364.86 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1197/3051 [09:25<14:35,  2.12it/s, est. speed input: 69364.90 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1198/3051 [09:25<14:35,  2.12it/s, est. speed input: 69364.92 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1199/3051 [09:26<14:34,  2.12it/s, est. speed input: 69364.94 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1200/3051 [09:26<14:33,  2.12it/s, est. speed input: 69365.19 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1201/3051 [09:27<14:35,  2.11it/s, est. speed input: 69364.64 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1202/3051 [09:27<14:33,  2.12it/s, est. speed input: 69364.78 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1203/3051 [09:28<14:32,  2.12it/s, est. speed input: 69364.91 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1204/3051 [09:28<14:31,  2.12it/s, est. speed input: 69365.08 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 1205/3051 [09:29<14:33,  2.11it/s, est. speed input: 69364.55 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1206/3051 [09:29<14:32,  2.11it/s, est. speed input: 69364.59 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1207/3051 [09:30<14:31,  2.12it/s, est. speed input: 69364.70 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1208/3051 [09:30<14:31,  2.12it/s, est. speed input: 69364.64 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1209/3051 [09:31<14:29,  2.12it/s, est. speed input: 69364.77 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1210/3051 [09:31<14:29,  2.12it/s, est. speed input: 69364.83 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1211/3051 [09:32<14:28,  2.12it/s, est. speed input: 69364.85 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1212/3051 [09:32<14:29,  2.12it/s, est. speed input: 69364.66 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1213/3051 [09:33<14:28,  2.12it/s, est. speed input: 69364.73 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1214/3051 [09:33<14:27,  2.12it/s, est. speed input: 69364.71 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1215/3051 [09:33<14:28,  2.11it/s, est. speed input: 69364.37 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1216/3051 [09:34<14:27,  2.11it/s, est. speed input: 69364.46 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1217/3051 [09:34<14:25,  2.12it/s, est. speed input: 69364.68 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1218/3051 [09:35<14:25,  2.12it/s, est. speed input: 69364.79 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1219/3051 [09:35<14:23,  2.12it/s, est. speed input: 69365.01 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 1220/3051 [09:36<14:35,  2.09it/s, est. speed input: 69362.42 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1221/3051 [09:36<14:31,  2.10it/s, est. speed input: 69362.52 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1222/3051 [09:37<14:28,  2.11it/s, est. speed input: 69362.66 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1223/3051 [09:37<14:25,  2.11it/s, est. speed input: 69362.79 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1224/3051 [09:38<14:24,  2.11it/s, est. speed input: 69362.93 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1225/3051 [09:38<14:22,  2.12it/s, est. speed input: 69363.19 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1226/3051 [09:39<14:20,  2.12it/s, est. speed input: 69363.45 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1227/3051 [09:39<14:21,  2.12it/s, est. speed input: 69363.17 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1228/3051 [09:40<14:22,  2.11it/s, est. speed input: 69362.99 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1229/3051 [09:40<14:20,  2.12it/s, est. speed input: 69363.19 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1230/3051 [09:41<14:20,  2.12it/s, est. speed input: 69363.16 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1231/3051 [09:41<14:20,  2.12it/s, est. speed input: 69363.01 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1232/3051 [09:42<14:19,  2.12it/s, est. speed input: 69363.17 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1233/3051 [09:42<14:20,  2.11it/s, est. speed input: 69362.86 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1234/3051 [09:42<14:18,  2.12it/s, est. speed input: 69362.98 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 1235/3051 [09:43<14:18,  2.11it/s, est. speed input: 69362.85 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1236/3051 [09:43<14:17,  2.12it/s, est. speed input: 69362.87 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1237/3051 [09:44<14:16,  2.12it/s, est. speed input: 69363.04 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1238/3051 [09:44<14:15,  2.12it/s, est. speed input: 69363.12 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1239/3051 [09:45<14:15,  2.12it/s, est. speed input: 69363.20 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1240/3051 [09:45<14:17,  2.11it/s, est. speed input: 69362.62 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1241/3051 [09:46<14:16,  2.11it/s, est. speed input: 69362.65 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1242/3051 [09:46<14:14,  2.12it/s, est. speed input: 69362.88 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1243/3051 [09:47<14:13,  2.12it/s, est. speed input: 69362.93 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1244/3051 [09:47<14:13,  2.12it/s, est. speed input: 69362.93 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1245/3051 [09:48<14:12,  2.12it/s, est. speed input: 69363.05 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1246/3051 [09:48<14:11,  2.12it/s, est. speed input: 69363.27 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1247/3051 [09:49<14:10,  2.12it/s, est. speed input: 69363.33 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1248/3051 [09:49<14:10,  2.12it/s, est. speed input: 69363.38 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1249/3051 [09:50<14:09,  2.12it/s, est. speed input: 69363.57 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1250/3051 [09:50<14:08,  2.12it/s, est. speed input: 69363.77 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1251/3051 [09:50<14:08,  2.12it/s, est. speed input: 69363.94 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1252/3051 [09:51<14:07,  2.12it/s, est. speed input: 69364.05 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1253/3051 [09:51<14:07,  2.12it/s, est. speed input: 69364.07 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1254/3051 [09:52<14:07,  2.12it/s, est. speed input: 69364.12 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1255/3051 [09:52<14:07,  2.12it/s, est. speed input: 69364.23 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1256/3051 [09:53<14:06,  2.12it/s, est. speed input: 69364.34 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1257/3051 [09:53<14:07,  2.12it/s, est. speed input: 69364.11 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 1258/3051 [09:54<14:06,  2.12it/s, est. speed input: 69364.22 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1259/3051 [09:54<14:05,  2.12it/s, est. speed input: 69364.31 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1260/3051 [09:55<14:05,  2.12it/s, est. speed input: 69364.29 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1261/3051 [09:55<14:05,  2.12it/s, est. speed input: 69364.26 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1262/3051 [09:56<14:04,  2.12it/s, est. speed input: 69364.46 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1263/3051 [09:56<14:03,  2.12it/s, est. speed input: 69364.50 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1264/3051 [09:57<14:03,  2.12it/s, est. speed input: 69364.56 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1265/3051 [09:57<14:02,  2.12it/s, est. speed input: 69364.73 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 1266/3051 [09:58<14:02,  2.12it/s, est. speed input: 69364.67 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1267/3051 [09:58<14:02,  2.12it/s, est. speed input: 69364.69 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1268/3051 [09:59<14:01,  2.12it/s, est. speed input: 69364.83 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1269/3051 [09:59<14:02,  2.12it/s, est. speed input: 69364.57 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1270/3051 [09:59<14:01,  2.12it/s, est. speed input: 69364.69 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1271/3051 [10:00<13:59,  2.12it/s, est. speed input: 69364.91 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1272/3051 [10:00<13:59,  2.12it/s, est. speed input: 69365.02 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1273/3051 [10:01<13:59,  2.12it/s, est. speed input: 69364.93 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1274/3051 [10:01<13:58,  2.12it/s, est. speed input: 69365.09 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1275/3051 [10:02<13:57,  2.12it/s, est. speed input: 69365.21 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1276/3051 [10:02<13:57,  2.12it/s, est. speed input: 69365.17 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1277/3051 [10:03<13:57,  2.12it/s, est. speed input: 69365.17 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1278/3051 [10:03<13:57,  2.12it/s, est. speed input: 69365.19 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1279/3051 [10:04<13:56,  2.12it/s, est. speed input: 69365.17 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1280/3051 [10:04<13:58,  2.11it/s, est. speed input: 69364.76 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1281/3051 [10:05<13:57,  2.11it/s, est. speed input: 69364.83 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1282/3051 [10:05<13:55,  2.12it/s, est. speed input: 69365.00 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1283/3051 [10:06<13:54,  2.12it/s, est. speed input: 69365.05 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1284/3051 [10:06<13:54,  2.12it/s, est. speed input: 69365.00 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1285/3051 [10:07<13:53,  2.12it/s, est. speed input: 69365.11 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1286/3051 [10:07<13:52,  2.12it/s, est. speed input: 69365.24 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1287/3051 [10:07<13:54,  2.11it/s, est. speed input: 69364.82 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1288/3051 [10:08<13:53,  2.12it/s, est. speed input: 69364.88 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1289/3051 [10:08<13:51,  2.12it/s, est. speed input: 69365.08 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1290/3051 [10:09<13:50,  2.12it/s, est. speed input: 69365.28 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1291/3051 [10:09<13:50,  2.12it/s, est. speed input: 69365.25 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1292/3051 [10:10<13:50,  2.12it/s, est. speed input: 69365.18 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1293/3051 [10:10<13:50,  2.12it/s, est. speed input: 69365.12 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1294/3051 [10:11<13:49,  2.12it/s, est. speed input: 69365.30 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1295/3051 [10:11<13:48,  2.12it/s, est. speed input: 69365.43 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 1296/3051 [10:12<13:50,  2.11it/s, est. speed input: 69364.89 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1297/3051 [10:12<13:49,  2.12it/s, est. speed input: 69365.02 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1298/3051 [10:13<13:47,  2.12it/s, est. speed input: 69365.25 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1299/3051 [10:13<13:47,  2.12it/s, est. speed input: 69365.24 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1300/3051 [10:14<13:46,  2.12it/s, est. speed input: 69365.24 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1301/3051 [10:14<13:45,  2.12it/s, est. speed input: 69365.39 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1302/3051 [10:15<13:44,  2.12it/s, est. speed input: 69365.61 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1303/3051 [10:15<13:45,  2.12it/s, est. speed input: 69365.47 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1304/3051 [10:16<13:44,  2.12it/s, est. speed input: 69365.52 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1305/3051 [10:16<13:43,  2.12it/s, est. speed input: 69365.67 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1306/3051 [10:16<13:42,  2.12it/s, est. speed input: 69365.80 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1307/3051 [10:17<13:42,  2.12it/s, est. speed input: 69365.94 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1308/3051 [10:17<13:41,  2.12it/s, est. speed input: 69366.04 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1309/3051 [10:18<13:41,  2.12it/s, est. speed input: 69366.05 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1310/3051 [10:18<13:41,  2.12it/s, est. speed input: 69366.06 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1311/3051 [10:19<13:40,  2.12it/s, est. speed input: 69366.18 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1312/3051 [10:19<13:40,  2.12it/s, est. speed input: 69366.15 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1313/3051 [10:20<13:40,  2.12it/s, est. speed input: 69366.18 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1314/3051 [10:20<13:39,  2.12it/s, est. speed input: 69366.31 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1315/3051 [10:21<13:38,  2.12it/s, est. speed input: 69366.46 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1316/3051 [10:21<13:39,  2.12it/s, est. speed input: 69366.23 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1317/3051 [10:22<13:38,  2.12it/s, est. speed input: 69366.34 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1318/3051 [10:22<13:37,  2.12it/s, est. speed input: 69366.55 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1319/3051 [10:23<13:36,  2.12it/s, est. speed input: 69366.68 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1320/3051 [10:23<13:36,  2.12it/s, est. speed input: 69366.76 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1321/3051 [10:24<13:36,  2.12it/s, est. speed input: 69366.80 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1322/3051 [10:24<13:36,  2.12it/s, est. speed input: 69366.77 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1323/3051 [10:24<13:36,  2.12it/s, est. speed input: 69366.66 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1324/3051 [10:25<13:35,  2.12it/s, est. speed input: 69366.69 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1325/3051 [10:25<13:35,  2.12it/s, est. speed input: 69366.54 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1326/3051 [10:26<13:35,  2.11it/s, est. speed input: 69366.45 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 1327/3051 [10:26<13:34,  2.12it/s, est. speed input: 69366.51 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1328/3051 [10:27<13:33,  2.12it/s, est. speed input: 69366.59 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1329/3051 [10:27<13:33,  2.12it/s, est. speed input: 69366.63 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1330/3051 [10:28<13:32,  2.12it/s, est. speed input: 69366.79 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1331/3051 [10:28<13:30,  2.12it/s, est. speed input: 69366.99 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1332/3051 [10:29<13:31,  2.12it/s, est. speed input: 69366.83 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1333/3051 [10:29<13:31,  2.12it/s, est. speed input: 69366.74 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 1334/3051 [10:30<13:30,  2.12it/s, est. speed input: 69366.94 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1335/3051 [10:30<13:29,  2.12it/s, est. speed input: 69367.03 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1336/3051 [10:31<13:29,  2.12it/s, est. speed input: 69367.08 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1337/3051 [10:31<13:28,  2.12it/s, est. speed input: 69367.29 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1338/3051 [10:32<13:28,  2.12it/s, est. speed input: 69367.29 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1339/3051 [10:32<13:27,  2.12it/s, est. speed input: 69367.41 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1340/3051 [10:32<13:26,  2.12it/s, est. speed input: 69367.51 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1341/3051 [10:33<13:26,  2.12it/s, est. speed input: 69367.60 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1342/3051 [10:33<13:27,  2.12it/s, est. speed input: 69367.39 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1343/3051 [10:34<13:27,  2.12it/s, est. speed input: 69367.32 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1344/3051 [10:34<13:28,  2.11it/s, est. speed input: 69367.01 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1345/3051 [10:35<13:28,  2.11it/s, est. speed input: 69366.72 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1346/3051 [10:35<13:27,  2.11it/s, est. speed input: 69366.64 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1347/3051 [10:36<13:25,  2.11it/s, est. speed input: 69366.78 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1348/3051 [10:36<13:25,  2.12it/s, est. speed input: 69366.78 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1349/3051 [10:37<13:24,  2.12it/s, est. speed input: 69366.75 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1350/3051 [10:37<13:23,  2.12it/s, est. speed input: 69366.89 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1351/3051 [10:38<13:35,  2.09it/s, est. speed input: 69364.27 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1352/3051 [10:38<14:10,  2.00it/s, est. speed input: 69355.86 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1353/3051 [10:39<13:56,  2.03it/s, est. speed input: 69355.71 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1354/3051 [10:39<13:48,  2.05it/s, est. speed input: 69355.08 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1355/3051 [10:40<13:40,  2.07it/s, est. speed input: 69355.05 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1356/3051 [10:40<13:34,  2.08it/s, est. speed input: 69355.01 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 1357/3051 [10:41<13:31,  2.09it/s, est. speed input: 69354.54 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1358/3051 [10:41<13:28,  2.09it/s, est. speed input: 69354.46 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1359/3051 [10:42<13:25,  2.10it/s, est. speed input: 69354.39 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1360/3051 [10:42<13:24,  2.10it/s, est. speed input: 69354.16 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1361/3051 [10:43<13:22,  2.11it/s, est. speed input: 69354.17 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1362/3051 [10:43<13:20,  2.11it/s, est. speed input: 69354.13 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1363/3051 [10:43<13:19,  2.11it/s, est. speed input: 69354.07 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1364/3051 [10:44<13:18,  2.11it/s, est. speed input: 69354.01 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1365/3051 [10:44<13:17,  2.11it/s, est. speed input: 69354.05 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1366/3051 [10:45<13:16,  2.11it/s, est. speed input: 69354.07 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1367/3051 [10:45<13:16,  2.11it/s, est. speed input: 69354.05 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1368/3051 [10:46<13:15,  2.12it/s, est. speed input: 69354.09 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1369/3051 [10:46<13:15,  2.12it/s, est. speed input: 69354.05 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1370/3051 [10:47<13:14,  2.12it/s, est. speed input: 69354.04 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1371/3051 [10:47<13:14,  2.12it/s, est. speed input: 69354.02 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 1372/3051 [10:48<13:16,  2.11it/s, est. speed input: 69353.42 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1373/3051 [10:48<13:15,  2.11it/s, est. speed input: 69353.21 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1374/3051 [10:49<13:16,  2.11it/s, est. speed input: 69352.79 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1375/3051 [10:49<13:15,  2.11it/s, est. speed input: 69352.70 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1376/3051 [10:50<13:14,  2.11it/s, est. speed input: 69352.62 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1377/3051 [10:50<13:13,  2.11it/s, est. speed input: 69352.46 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1378/3051 [10:51<13:14,  2.10it/s, est. speed input: 69351.93 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1379/3051 [10:51<13:13,  2.11it/s, est. speed input: 69351.93 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1380/3051 [10:52<13:12,  2.11it/s, est. speed input: 69351.77 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1381/3051 [10:52<13:11,  2.11it/s, est. speed input: 69351.64 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1382/3051 [10:52<13:10,  2.11it/s, est. speed input: 69351.53 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1383/3051 [10:53<13:09,  2.11it/s, est. speed input: 69351.49 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1384/3051 [10:53<13:09,  2.11it/s, est. speed input: 69351.44 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1385/3051 [10:54<13:09,  2.11it/s, est. speed input: 69351.10 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1386/3051 [10:54<13:09,  2.11it/s, est. speed input: 69350.97 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1387/3051 [10:55<13:08,  2.11it/s, est. speed input: 69350.91 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 1388/3051 [10:55<13:09,  2.11it/s, est. speed input: 69350.49 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1389/3051 [10:56<13:08,  2.11it/s, est. speed input: 69350.33 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1390/3051 [10:56<13:07,  2.11it/s, est. speed input: 69350.31 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1391/3051 [10:57<13:07,  2.11it/s, est. speed input: 69350.07 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1392/3051 [10:57<13:05,  2.11it/s, est. speed input: 69350.07 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1393/3051 [10:58<13:05,  2.11it/s, est. speed input: 69350.01 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1394/3051 [10:58<13:04,  2.11it/s, est. speed input: 69349.99 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1395/3051 [10:59<13:03,  2.11it/s, est. speed input: 69349.93 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1396/3051 [10:59<13:02,  2.11it/s, est. speed input: 69349.94 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1397/3051 [11:00<13:02,  2.11it/s, est. speed input: 69349.91 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1398/3051 [11:00<13:01,  2.11it/s, est. speed input: 69349.88 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1399/3051 [11:01<13:01,  2.11it/s, est. speed input: 69349.81 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1400/3051 [11:01<13:00,  2.12it/s, est. speed input: 69349.83 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1401/3051 [11:01<13:00,  2.11it/s, est. speed input: 69349.63 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1402/3051 [11:02<13:00,  2.11it/s, est. speed input: 69349.59 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1403/3051 [11:02<12:59,  2.11it/s, est. speed input: 69349.56 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1404/3051 [11:03<12:59,  2.11it/s, est. speed input: 69349.48 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1405/3051 [11:03<12:58,  2.11it/s, est. speed input: 69349.42 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1406/3051 [11:04<12:58,  2.11it/s, est. speed input: 69349.36 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1407/3051 [11:04<12:57,  2.11it/s, est. speed input: 69349.33 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1408/3051 [11:05<12:57,  2.11it/s, est. speed input: 69349.26 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1409/3051 [11:05<12:56,  2.11it/s, est. speed input: 69349.25 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1410/3051 [11:06<12:55,  2.11it/s, est. speed input: 69349.24 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 1411/3051 [11:06<12:55,  2.12it/s, est. speed input: 69349.25 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1412/3051 [11:07<12:54,  2.12it/s, est. speed input: 69349.24 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1413/3051 [11:07<12:54,  2.12it/s, est. speed input: 69349.25 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1414/3051 [11:08<12:53,  2.12it/s, est. speed input: 69349.22 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1415/3051 [11:08<12:53,  2.11it/s, est. speed input: 69349.11 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1416/3051 [11:09<12:54,  2.11it/s, est. speed input: 69348.78 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1417/3051 [11:09<12:53,  2.11it/s, est. speed input: 69348.78 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 1418/3051 [11:10<12:52,  2.11it/s, est. speed input: 69348.78 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1419/3051 [11:10<12:51,  2.11it/s, est. speed input: 69348.81 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1420/3051 [11:10<12:51,  2.11it/s, est. speed input: 69348.77 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1421/3051 [11:11<12:50,  2.12it/s, est. speed input: 69348.79 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1422/3051 [11:11<12:49,  2.12it/s, est. speed input: 69348.78 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1423/3051 [11:12<12:49,  2.12it/s, est. speed input: 69348.78 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1424/3051 [11:12<12:48,  2.12it/s, est. speed input: 69348.81 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1425/3051 [11:13<12:48,  2.11it/s, est. speed input: 69348.66 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1426/3051 [11:13<12:49,  2.11it/s, est. speed input: 69348.42 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1427/3051 [11:14<12:48,  2.11it/s, est. speed input: 69348.36 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1428/3051 [11:14<12:48,  2.11it/s, est. speed input: 69348.29 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1429/3051 [11:15<12:47,  2.11it/s, est. speed input: 69348.24 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1430/3051 [11:15<12:46,  2.11it/s, est. speed input: 69348.29 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1431/3051 [11:16<12:45,  2.11it/s, est. speed input: 69348.26 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1432/3051 [11:16<12:45,  2.11it/s, est. speed input: 69348.17 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1433/3051 [11:17<12:45,  2.11it/s, est. speed input: 69348.16 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1434/3051 [11:17<12:44,  2.12it/s, est. speed input: 69348.16 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1435/3051 [11:18<12:44,  2.11it/s, est. speed input: 69348.07 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1436/3051 [11:18<12:43,  2.12it/s, est. speed input: 69348.09 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1437/3051 [11:19<12:42,  2.12it/s, est. speed input: 69348.11 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1438/3051 [11:19<12:42,  2.12it/s, est. speed input: 69348.10 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1439/3051 [11:19<12:41,  2.12it/s, est. speed input: 69348.13 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1440/3051 [11:20<12:41,  2.12it/s, est. speed input: 69348.13 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1441/3051 [11:20<12:41,  2.11it/s, est. speed input: 69347.98 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1442/3051 [11:21<12:40,  2.12it/s, est. speed input: 69348.03 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1443/3051 [11:21<12:40,  2.12it/s, est. speed input: 69348.01 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1444/3051 [11:22<12:39,  2.11it/s, est. speed input: 69347.94 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1445/3051 [11:22<12:39,  2.12it/s, est. speed input: 69347.94 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1446/3051 [11:23<12:38,  2.11it/s, est. speed input: 69347.88 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1447/3051 [11:23<12:38,  2.12it/s, est. speed input: 69347.87 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1448/3051 [11:24<12:37,  2.11it/s, est. speed input: 69347.81 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 1449/3051 [11:24<12:37,  2.12it/s, est. speed input: 69347.82 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1450/3051 [11:25<12:36,  2.12it/s, est. speed input: 69347.88 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1451/3051 [11:25<12:36,  2.12it/s, est. speed input: 69347.85 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1452/3051 [11:26<12:35,  2.12it/s, est. speed input: 69347.82 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1453/3051 [11:26<12:37,  2.11it/s, est. speed input: 69347.25 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1454/3051 [11:27<12:37,  2.11it/s, est. speed input: 69347.12 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1455/3051 [11:27<12:39,  2.10it/s, est. speed input: 69346.43 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1456/3051 [11:28<12:44,  2.09it/s, est. speed input: 69344.89 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1457/3051 [11:28<12:51,  2.07it/s, est. speed input: 69342.68 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1458/3051 [11:28<12:45,  2.08it/s, est. speed input: 69342.56 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1459/3051 [11:29<12:42,  2.09it/s, est. speed input: 69342.25 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1460/3051 [11:29<12:39,  2.09it/s, est. speed input: 69342.18 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1461/3051 [11:30<12:39,  2.09it/s, est. speed input: 69341.62 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1462/3051 [11:30<12:36,  2.10it/s, est. speed input: 69341.64 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1463/3051 [11:31<12:35,  2.10it/s, est. speed input: 69341.50 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1464/3051 [11:31<12:34,  2.10it/s, est. speed input: 69341.19 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1465/3051 [11:32<12:32,  2.11it/s, est. speed input: 69341.19 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1466/3051 [11:32<12:31,  2.11it/s, est. speed input: 69341.18 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1467/3051 [11:33<12:30,  2.11it/s, est. speed input: 69341.13 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1468/3051 [11:33<12:29,  2.11it/s, est. speed input: 69341.13 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1469/3051 [11:34<12:28,  2.11it/s, est. speed input: 69341.17 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1470/3051 [11:34<12:27,  2.12it/s, est. speed input: 69341.19 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1471/3051 [11:35<12:26,  2.12it/s, est. speed input: 69341.25 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1472/3051 [11:35<12:27,  2.11it/s, est. speed input: 69341.05 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1473/3051 [11:36<12:28,  2.11it/s, est. speed input: 69340.65 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1474/3051 [11:36<12:27,  2.11it/s, est. speed input: 69340.65 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1475/3051 [11:37<12:25,  2.11it/s, est. speed input: 69340.69 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1476/3051 [11:37<12:25,  2.11it/s, est. speed input: 69340.63 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1477/3051 [11:37<12:24,  2.11it/s, est. speed input: 69340.59 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1478/3051 [11:38<12:24,  2.11it/s, est. speed input: 69340.54 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 1479/3051 [11:38<12:23,  2.11it/s, est. speed input: 69340.53 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1480/3051 [11:39<12:22,  2.11it/s, est. speed input: 69340.52 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1481/3051 [11:39<12:23,  2.11it/s, est. speed input: 69340.27 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1482/3051 [11:40<12:23,  2.11it/s, est. speed input: 69340.16 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1483/3051 [11:40<12:22,  2.11it/s, est. speed input: 69340.14 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1484/3051 [11:41<12:21,  2.11it/s, est. speed input: 69340.13 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1485/3051 [11:41<12:20,  2.11it/s, est. speed input: 69340.09 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1486/3051 [11:42<12:20,  2.11it/s, est. speed input: 69340.06 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 1487/3051 [11:42<12:19,  2.12it/s, est. speed input: 69340.09 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1488/3051 [11:43<12:19,  2.11it/s, est. speed input: 69340.04 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1489/3051 [11:43<12:18,  2.12it/s, est. speed input: 69340.04 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1490/3051 [11:44<12:18,  2.12it/s, est. speed input: 69340.01 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1491/3051 [11:44<12:17,  2.12it/s, est. speed input: 69340.02 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1492/3051 [11:45<12:16,  2.12it/s, est. speed input: 69340.01 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1493/3051 [11:45<12:16,  2.12it/s, est. speed input: 69340.00 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1494/3051 [11:46<12:15,  2.12it/s, est. speed input: 69340.00 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1495/3051 [11:46<12:15,  2.12it/s, est. speed input: 69339.98 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1496/3051 [11:46<12:14,  2.12it/s, est. speed input: 69340.00 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1497/3051 [11:47<12:14,  2.12it/s, est. speed input: 69340.01 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1498/3051 [11:47<12:14,  2.12it/s, est. speed input: 69339.97 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1499/3051 [11:48<12:13,  2.12it/s, est. speed input: 69340.01 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1500/3051 [11:48<12:12,  2.12it/s, est. speed input: 69340.04 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1501/3051 [11:49<12:12,  2.12it/s, est. speed input: 69340.05 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1502/3051 [11:49<12:11,  2.12it/s, est. speed input: 69340.03 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1503/3051 [11:50<12:11,  2.11it/s, est. speed input: 69339.95 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1504/3051 [11:50<12:11,  2.12it/s, est. speed input: 69339.95 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1505/3051 [11:51<12:10,  2.12it/s, est. speed input: 69339.94 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1506/3051 [11:51<12:37,  2.04it/s, est. speed input: 69334.22 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1507/3051 [11:52<12:28,  2.06it/s, est. speed input: 69334.36 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1508/3051 [11:52<12:21,  2.08it/s, est. speed input: 69334.42 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1509/3051 [11:53<12:17,  2.09it/s, est. speed input: 69334.44 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 1510/3051 [11:53<12:14,  2.10it/s, est. speed input: 69334.43 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1511/3051 [11:54<12:12,  2.10it/s, est. speed input: 69334.44 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1512/3051 [11:54<12:10,  2.11it/s, est. speed input: 69334.48 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1513/3051 [11:55<12:09,  2.11it/s, est. speed input: 69334.41 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1514/3051 [11:55<12:07,  2.11it/s, est. speed input: 69334.42 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1515/3051 [11:56<12:06,  2.11it/s, est. speed input: 69334.43 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1516/3051 [11:56<12:06,  2.11it/s, est. speed input: 69334.45 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1517/3051 [11:56<12:05,  2.11it/s, est. speed input: 69334.41 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1518/3051 [11:57<12:04,  2.11it/s, est. speed input: 69334.42 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1519/3051 [11:57<12:04,  2.12it/s, est. speed input: 69334.43 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1520/3051 [11:58<12:04,  2.11it/s, est. speed input: 69334.20 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1521/3051 [11:58<12:03,  2.11it/s, est. speed input: 69334.19 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1522/3051 [11:59<12:03,  2.11it/s, est. speed input: 69334.16 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1523/3051 [11:59<12:03,  2.11it/s, est. speed input: 69334.03 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1524/3051 [12:00<12:02,  2.11it/s, est. speed input: 69334.10 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 1525/3051 [12:00<12:02,  2.11it/s, est. speed input: 69333.93 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1526/3051 [12:01<12:01,  2.11it/s, est. speed input: 69333.84 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1527/3051 [12:01<12:01,  2.11it/s, est. speed input: 69333.82 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1528/3051 [12:02<12:00,  2.11it/s, est. speed input: 69333.80 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1529/3051 [12:02<11:59,  2.11it/s, est. speed input: 69333.80 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1530/3051 [12:03<11:59,  2.11it/s, est. speed input: 69333.80 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1531/3051 [12:03<11:58,  2.12it/s, est. speed input: 69333.80 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1532/3051 [12:04<11:58,  2.11it/s, est. speed input: 69333.72 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1533/3051 [12:04<11:58,  2.11it/s, est. speed input: 69333.62 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1534/3051 [12:04<11:57,  2.11it/s, est. speed input: 69333.64 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1535/3051 [12:05<11:56,  2.12it/s, est. speed input: 69333.66 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1536/3051 [12:05<11:56,  2.12it/s, est. speed input: 69333.64 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1537/3051 [12:06<11:55,  2.12it/s, est. speed input: 69333.69 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1538/3051 [12:06<11:56,  2.11it/s, est. speed input: 69333.47 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1539/3051 [12:07<11:55,  2.11it/s, est. speed input: 69333.46 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1540/3051 [12:07<11:54,  2.11it/s, est. speed input: 69333.48 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1541/3051 [12:08<11:53,  2.12it/s, est. speed input: 69333.51 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1542/3051 [12:08<11:53,  2.11it/s, est. speed input: 69333.43 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1543/3051 [12:09<11:52,  2.12it/s, est. speed input: 69333.44 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1544/3051 [12:09<11:52,  2.12it/s, est. speed input: 69333.46 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1545/3051 [12:10<11:52,  2.11it/s, est. speed input: 69333.40 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1546/3051 [12:10<11:51,  2.11it/s, est. speed input: 69333.37 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1547/3051 [12:11<11:51,  2.11it/s, est. speed input: 69333.36 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1548/3051 [12:11<11:50,  2.11it/s, est. speed input: 69333.34 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1549/3051 [12:12<11:50,  2.11it/s, est. speed input: 69333.30 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1550/3051 [12:12<11:49,  2.12it/s, est. speed input: 69333.31 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1551/3051 [12:13<11:49,  2.12it/s, est. speed input: 69333.29 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1552/3051 [12:13<11:48,  2.12it/s, est. speed input: 69333.28 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1553/3051 [12:13<11:48,  2.12it/s, est. speed input: 69333.30 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1554/3051 [12:14<11:47,  2.12it/s, est. speed input: 69333.27 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1555/3051 [12:14<11:48,  2.11it/s, est. speed input: 69333.03 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1556/3051 [12:15<11:47,  2.11it/s, est. speed input: 69333.03 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1557/3051 [12:15<11:46,  2.11it/s, est. speed input: 69332.99 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1558/3051 [12:16<11:47,  2.11it/s, est. speed input: 69332.78 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1559/3051 [12:16<11:46,  2.11it/s, est. speed input: 69332.68 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1560/3051 [12:17<11:45,  2.11it/s, est. speed input: 69332.68 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1561/3051 [12:17<11:45,  2.11it/s, est. speed input: 69332.65 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1562/3051 [12:18<11:44,  2.11it/s, est. speed input: 69332.68 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1563/3051 [12:18<11:44,  2.11it/s, est. speed input: 69332.55 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1564/3051 [12:19<11:44,  2.11it/s, est. speed input: 69332.40 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1565/3051 [12:19<11:44,  2.11it/s, est. speed input: 69332.07 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1566/3051 [12:20<11:44,  2.11it/s, est. speed input: 69331.82 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1567/3051 [12:20<11:44,  2.11it/s, est. speed input: 69331.61 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1568/3051 [12:21<11:43,  2.11it/s, est. speed input: 69331.54 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1569/3051 [12:21<11:42,  2.11it/s, est. speed input: 69331.53 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1570/3051 [12:22<11:42,  2.11it/s, est. speed input: 69331.33 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1571/3051 [12:22<11:41,  2.11it/s, est. speed input: 69331.19 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1572/3051 [12:22<11:40,  2.11it/s, est. speed input: 69331.14 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1573/3051 [12:23<11:39,  2.11it/s, est. speed input: 69331.11 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1574/3051 [12:23<11:39,  2.11it/s, est. speed input: 69331.12 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1575/3051 [12:24<11:38,  2.11it/s, est. speed input: 69331.10 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1576/3051 [12:24<11:37,  2.11it/s, est. speed input: 69331.05 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1577/3051 [12:25<11:39,  2.11it/s, est. speed input: 69330.51 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1578/3051 [12:25<11:38,  2.11it/s, est. speed input: 69330.43 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1579/3051 [12:26<11:37,  2.11it/s, est. speed input: 69330.45 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1580/3051 [12:26<11:36,  2.11it/s, est. speed input: 69330.43 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1581/3051 [12:27<11:36,  2.11it/s, est. speed input: 69330.21 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1582/3051 [12:27<11:35,  2.11it/s, est. speed input: 69330.22 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1583/3051 [12:28<11:34,  2.11it/s, est. speed input: 69330.24 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1584/3051 [12:28<11:34,  2.11it/s, est. speed input: 69330.14 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1585/3051 [12:29<11:33,  2.11it/s, est. speed input: 69330.17 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1586/3051 [12:29<11:33,  2.11it/s, est. speed input: 69330.03 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1587/3051 [12:30<11:33,  2.11it/s, est. speed input: 69329.94 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1588/3051 [12:30<11:32,  2.11it/s, est. speed input: 69329.91 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1589/3051 [12:31<11:40,  2.09it/s, est. speed input: 69328.08 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1590/3051 [12:31<11:36,  2.10it/s, est. speed input: 69328.10 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1591/3051 [12:31<11:34,  2.10it/s, est. speed input: 69328.04 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1592/3051 [12:32<11:40,  2.08it/s, est. speed input: 69326.51 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1593/3051 [12:32<11:46,  2.06it/s, est. speed input: 69324.46 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1594/3051 [12:33<11:42,  2.07it/s, est. speed input: 69323.99 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1595/3051 [12:33<11:38,  2.08it/s, est. speed input: 69323.86 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1596/3051 [12:34<11:37,  2.09it/s, est. speed input: 69323.40 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1597/3051 [12:34<11:35,  2.09it/s, est. speed input: 69323.07 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1598/3051 [12:35<11:33,  2.09it/s, est. speed input: 69322.85 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1599/3051 [12:35<11:32,  2.10it/s, est. speed input: 69322.61 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1600/3051 [12:36<11:30,  2.10it/s, est. speed input: 69322.60 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1601/3051 [12:36<11:29,  2.10it/s, est. speed input: 69322.38 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1602/3051 [12:37<11:28,  2.11it/s, est. speed input: 69322.28 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1603/3051 [12:37<11:26,  2.11it/s, est. speed input: 69322.27 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1604/3051 [12:38<11:25,  2.11it/s, est. speed input: 69322.26 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1605/3051 [12:38<11:26,  2.11it/s, est. speed input: 69321.83 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1606/3051 [12:39<11:25,  2.11it/s, est. speed input: 69321.79 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1607/3051 [12:39<11:24,  2.11it/s, est. speed input: 69321.81 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1608/3051 [12:40<11:23,  2.11it/s, est. speed input: 69321.80 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1609/3051 [12:40<11:23,  2.11it/s, est. speed input: 69321.55 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1610/3051 [12:41<11:22,  2.11it/s, est. speed input: 69321.53 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1611/3051 [12:41<11:21,  2.11it/s, est. speed input: 69321.50 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1612/3051 [12:41<11:21,  2.11it/s, est. speed input: 69321.47 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1613/3051 [12:42<11:20,  2.11it/s, est. speed input: 69321.54 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1614/3051 [12:42<11:21,  2.11it/s, est. speed input: 69321.10 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1615/3051 [12:43<11:20,  2.11it/s, est. speed input: 69321.01 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1616/3051 [12:43<11:19,  2.11it/s, est. speed input: 69321.02 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1617/3051 [12:44<11:18,  2.11it/s, est. speed input: 69320.99 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1618/3051 [12:44<11:20,  2.11it/s, est. speed input: 69320.58 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1619/3051 [12:47<30:17,  1.27s/it, est. speed input: 69081.26 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1620/3051 [12:48<24:33,  1.03s/it, est. speed input: 69081.69 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1621/3051 [12:48<20:31,  1.16it/s, est. speed input: 69082.11 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1622/3051 [12:49<17:43,  1.34it/s, est. speed input: 69082.45 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1623/3051 [12:49<15:45,  1.51it/s, est. speed input: 69082.74 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1624/3051 [12:50<14:24,  1.65it/s, est. speed input: 69082.79 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1625/3051 [12:50<13:26,  1.77it/s, est. speed input: 69083.03 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1626/3051 [12:51<12:45,  1.86it/s, est. speed input: 69083.32 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1627/3051 [12:51<12:16,  1.93it/s, est. speed input: 69083.55 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1628/3051 [12:52<11:56,  1.99it/s, est. speed input: 69083.83 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1629/3051 [12:52<11:42,  2.02it/s, est. speed input: 69084.12 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1630/3051 [12:53<11:32,  2.05it/s, est. speed input: 69084.41 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1631/3051 [12:53<11:24,  2.07it/s, est. speed input: 69084.68 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1632/3051 [12:54<11:19,  2.09it/s, est. speed input: 69085.01 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1633/3051 [12:54<11:15,  2.10it/s, est. speed input: 69085.30 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1634/3051 [12:55<11:12,  2.11it/s, est. speed input: 69085.56 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1635/3051 [12:55<11:10,  2.11it/s, est. speed input: 69085.88 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1636/3051 [12:55<11:09,  2.11it/s, est. speed input: 69086.09 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1637/3051 [12:56<11:08,  2.12it/s, est. speed input: 69086.39 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1638/3051 [12:56<11:06,  2.12it/s, est. speed input: 69086.70 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1639/3051 [12:57<11:06,  2.12it/s, est. speed input: 69086.96 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1640/3051 [12:57<11:05,  2.12it/s, est. speed input: 69087.22 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1641/3051 [12:58<11:04,  2.12it/s, est. speed input: 69087.54 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1642/3051 [12:58<11:03,  2.12it/s, est. speed input: 69087.87 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1643/3051 [12:59<11:03,  2.12it/s, est. speed input: 69088.16 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1644/3051 [12:59<11:02,  2.12it/s, est. speed input: 69088.43 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1645/3051 [13:00<11:02,  2.12it/s, est. speed input: 69088.67 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1646/3051 [13:00<11:02,  2.12it/s, est. speed input: 69088.94 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1647/3051 [13:01<11:02,  2.12it/s, est. speed input: 69089.13 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1648/3051 [13:01<11:02,  2.12it/s, est. speed input: 69089.24 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1649/3051 [13:02<11:02,  2.12it/s, est. speed input: 69089.35 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1650/3051 [13:02<11:01,  2.12it/s, est. speed input: 69089.59 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1651/3051 [13:03<11:00,  2.12it/s, est. speed input: 69089.78 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1652/3051 [13:03<11:00,  2.12it/s, est. speed input: 69090.04 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1653/3051 [13:03<11:00,  2.12it/s, est. speed input: 69090.18 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1654/3051 [13:04<11:00,  2.12it/s, est. speed input: 69090.22 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1655/3051 [13:04<11:00,  2.11it/s, est. speed input: 69090.25 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1656/3051 [13:05<10:59,  2.12it/s, est. speed input: 69090.44 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1657/3051 [13:05<10:58,  2.12it/s, est. speed input: 69090.73 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1658/3051 [13:06<10:57,  2.12it/s, est. speed input: 69091.03 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1659/3051 [13:06<10:56,  2.12it/s, est. speed input: 69091.31 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1660/3051 [13:07<10:56,  2.12it/s, est. speed input: 69091.38 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1661/3051 [13:07<10:56,  2.12it/s, est. speed input: 69091.53 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1662/3051 [13:08<10:55,  2.12it/s, est. speed input: 69091.85 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1663/3051 [13:08<10:54,  2.12it/s, est. speed input: 69092.12 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1664/3051 [13:09<10:53,  2.12it/s, est. speed input: 69092.42 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1665/3051 [13:09<10:53,  2.12it/s, est. speed input: 69092.64 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1666/3051 [13:10<10:52,  2.12it/s, est. speed input: 69092.91 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1667/3051 [13:10<10:52,  2.12it/s, est. speed input: 69093.15 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1668/3051 [13:11<10:52,  2.12it/s, est. speed input: 69093.37 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1669/3051 [13:11<10:51,  2.12it/s, est. speed input: 69093.66 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1670/3051 [13:12<10:50,  2.12it/s, est. speed input: 69093.94 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1671/3051 [13:12<10:51,  2.12it/s, est. speed input: 69093.99 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1672/3051 [13:12<10:50,  2.12it/s, est. speed input: 69094.29 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1673/3051 [13:13<10:49,  2.12it/s, est. speed input: 69094.59 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1674/3051 [13:13<10:50,  2.12it/s, est. speed input: 69094.52 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1675/3051 [13:14<10:49,  2.12it/s, est. speed input: 69094.73 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1676/3051 [13:14<10:48,  2.12it/s, est. speed input: 69095.03 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1677/3051 [13:15<10:48,  2.12it/s, est. speed input: 69095.25 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1678/3051 [13:15<10:48,  2.12it/s, est. speed input: 69095.36 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1679/3051 [13:16<10:47,  2.12it/s, est. speed input: 69095.71 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1680/3051 [13:16<10:46,  2.12it/s, est. speed input: 69096.03 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1681/3051 [13:17<10:45,  2.12it/s, est. speed input: 69096.33 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1682/3051 [13:17<10:45,  2.12it/s, est. speed input: 69096.59 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1683/3051 [13:18<10:44,  2.12it/s, est. speed input: 69096.88 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1684/3051 [13:18<10:44,  2.12it/s, est. speed input: 69097.04 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1685/3051 [13:19<10:44,  2.12it/s, est. speed input: 69097.21 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1686/3051 [13:19<10:55,  2.08it/s, est. speed input: 69095.02 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1687/3051 [13:20<11:03,  2.06it/s, est. speed input: 69092.74 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1688/3051 [13:20<10:56,  2.08it/s, est. speed input: 69092.95 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1689/3051 [13:21<10:52,  2.09it/s, est. speed input: 69093.18 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1690/3051 [13:21<10:48,  2.10it/s, est. speed input: 69093.49 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1691/3051 [13:21<10:45,  2.11it/s, est. speed input: 69093.73 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1692/3051 [13:22<10:44,  2.11it/s, est. speed input: 69093.97 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1693/3051 [13:22<10:42,  2.11it/s, est. speed input: 69094.23 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1694/3051 [13:23<10:41,  2.12it/s, est. speed input: 69094.47 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1695/3051 [13:23<10:40,  2.12it/s, est. speed input: 69094.74 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1696/3051 [13:24<10:39,  2.12it/s, est. speed input: 69094.96 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1697/3051 [13:24<10:38,  2.12it/s, est. speed input: 69095.24 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1698/3051 [13:25<10:37,  2.12it/s, est. speed input: 69095.55 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1699/3051 [13:25<10:37,  2.12it/s, est. speed input: 69095.73 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1700/3051 [13:26<10:37,  2.12it/s, est. speed input: 69095.85 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1701/3051 [13:26<10:36,  2.12it/s, est. speed input: 69096.12 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1702/3051 [13:27<10:36,  2.12it/s, est. speed input: 69096.41 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1703/3051 [13:27<10:37,  2.11it/s, est. speed input: 69096.23 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1704/3051 [13:28<10:37,  2.11it/s, est. speed input: 69096.33 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1705/3051 [13:28<10:36,  2.11it/s, est. speed input: 69096.48 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1706/3051 [13:29<10:35,  2.12it/s, est. speed input: 69096.67 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1707/3051 [13:29<10:35,  2.12it/s, est. speed input: 69096.82 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1708/3051 [13:29<10:34,  2.12it/s, est. speed input: 69097.02 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1709/3051 [13:30<10:33,  2.12it/s, est. speed input: 69097.16 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1710/3051 [13:30<10:34,  2.11it/s, est. speed input: 69097.19 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1711/3051 [13:31<10:33,  2.12it/s, est. speed input: 69097.32 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1712/3051 [13:31<10:33,  2.11it/s, est. speed input: 69097.43 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1713/3051 [13:32<10:33,  2.11it/s, est. speed input: 69097.32 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1714/3051 [13:32<10:33,  2.11it/s, est. speed input: 69097.33 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1715/3051 [13:33<10:34,  2.11it/s, est. speed input: 69097.11 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1716/3051 [13:33<10:32,  2.11it/s, est. speed input: 69097.22 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1717/3051 [13:34<10:31,  2.11it/s, est. speed input: 69097.35 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1718/3051 [13:34<10:31,  2.11it/s, est. speed input: 69097.31 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1719/3051 [13:35<10:33,  2.10it/s, est. speed input: 69096.92 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1720/3051 [13:35<10:32,  2.11it/s, est. speed input: 69096.95 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1721/3051 [13:36<10:31,  2.11it/s, est. speed input: 69096.94 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1722/3051 [13:36<10:31,  2.11it/s, est. speed input: 69096.89 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1723/3051 [13:37<10:30,  2.11it/s, est. speed input: 69096.89 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1724/3051 [13:37<10:30,  2.11it/s, est. speed input: 69096.78 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1725/3051 [13:38<10:29,  2.11it/s, est. speed input: 69096.83 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1726/3051 [13:38<10:28,  2.11it/s, est. speed input: 69096.76 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1727/3051 [13:39<10:28,  2.11it/s, est. speed input: 69096.80 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1728/3051 [13:39<10:27,  2.11it/s, est. speed input: 69096.92 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1729/3051 [13:39<10:25,  2.11it/s, est. speed input: 69097.06 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1730/3051 [13:40<10:25,  2.11it/s, est. speed input: 69097.15 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1731/3051 [13:40<10:25,  2.11it/s, est. speed input: 69097.18 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1732/3051 [13:41<10:25,  2.11it/s, est. speed input: 69097.00 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1733/3051 [13:41<10:24,  2.11it/s, est. speed input: 69097.08 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1734/3051 [13:42<10:23,  2.11it/s, est. speed input: 69097.21 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1735/3051 [13:42<10:22,  2.11it/s, est. speed input: 69097.35 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1736/3051 [13:43<10:22,  2.11it/s, est. speed input: 69097.47 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1737/3051 [13:43<10:21,  2.11it/s, est. speed input: 69097.57 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1738/3051 [13:44<10:20,  2.11it/s, est. speed input: 69097.73 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1739/3051 [13:44<10:20,  2.11it/s, est. speed input: 69097.81 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1740/3051 [13:45<10:19,  2.11it/s, est. speed input: 69097.94 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1741/3051 [13:45<10:19,  2.11it/s, est. speed input: 69098.06 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1742/3051 [13:46<10:19,  2.11it/s, est. speed input: 69098.17 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1743/3051 [13:46<10:18,  2.12it/s, est. speed input: 69098.32 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1744/3051 [13:47<10:18,  2.11it/s, est. speed input: 69098.20 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1745/3051 [13:47<10:18,  2.11it/s, est. speed input: 69098.30 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1746/3051 [13:47<10:17,  2.11it/s, est. speed input: 69098.37 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1747/3051 [13:48<10:17,  2.11it/s, est. speed input: 69098.50 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1748/3051 [13:48<10:16,  2.11it/s, est. speed input: 69098.61 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1749/3051 [13:49<10:16,  2.11it/s, est. speed input: 69098.68 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1750/3051 [13:49<10:15,  2.11it/s, est. speed input: 69098.82 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1751/3051 [13:50<10:14,  2.11it/s, est. speed input: 69098.96 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1752/3051 [13:50<10:14,  2.11it/s, est. speed input: 69099.08 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1753/3051 [13:51<10:21,  2.09it/s, est. speed input: 69097.52 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1754/3051 [13:51<10:18,  2.10it/s, est. speed input: 69097.75 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1755/3051 [13:52<10:16,  2.10it/s, est. speed input: 69097.87 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1756/3051 [13:52<10:14,  2.11it/s, est. speed input: 69098.00 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1757/3051 [13:53<10:13,  2.11it/s, est. speed input: 69098.12 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1758/3051 [13:53<10:12,  2.11it/s, est. speed input: 69098.23 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1759/3051 [13:54<10:11,  2.11it/s, est. speed input: 69098.35 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1760/3051 [13:54<10:12,  2.11it/s, est. speed input: 69098.19 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1761/3051 [13:55<10:11,  2.11it/s, est. speed input: 69098.31 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1762/3051 [13:55<10:10,  2.11it/s, est. speed input: 69098.46 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1763/3051 [13:56<10:10,  2.11it/s, est. speed input: 69098.34 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1764/3051 [13:56<10:10,  2.11it/s, est. speed input: 69098.34 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1765/3051 [13:57<10:09,  2.11it/s, est. speed input: 69098.46 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1766/3051 [13:57<10:08,  2.11it/s, est. speed input: 69098.57 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1767/3051 [13:57<10:07,  2.11it/s, est. speed input: 69098.69 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1768/3051 [13:58<10:06,  2.11it/s, est. speed input: 69098.82 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1769/3051 [13:58<10:07,  2.11it/s, est. speed input: 69098.74 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1770/3051 [13:59<10:07,  2.11it/s, est. speed input: 69098.66 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1771/3051 [13:59<10:06,  2.11it/s, est. speed input: 69098.78 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1772/3051 [14:00<10:05,  2.11it/s, est. speed input: 69098.90 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1773/3051 [14:00<10:04,  2.11it/s, est. speed input: 69098.99 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1774/3051 [14:01<10:05,  2.11it/s, est. speed input: 69098.88 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1775/3051 [14:01<10:04,  2.11it/s, est. speed input: 69098.98 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1776/3051 [14:02<10:03,  2.11it/s, est. speed input: 69099.06 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1777/3051 [14:02<10:03,  2.11it/s, est. speed input: 69099.11 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1778/3051 [14:03<10:02,  2.11it/s, est. speed input: 69099.25 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1779/3051 [14:03<10:02,  2.11it/s, est. speed input: 69099.27 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1780/3051 [14:04<10:01,  2.11it/s, est. speed input: 69099.42 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1781/3051 [14:04<10:00,  2.11it/s, est. speed input: 69099.51 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1782/3051 [14:05<10:00,  2.11it/s, est. speed input: 69099.66 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1783/3051 [14:05<09:59,  2.12it/s, est. speed input: 69099.82 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1784/3051 [14:05<09:59,  2.11it/s, est. speed input: 69099.80 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1785/3051 [14:06<09:58,  2.11it/s, est. speed input: 69099.95 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1786/3051 [14:06<09:58,  2.11it/s, est. speed input: 69100.03 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1787/3051 [14:07<09:57,  2.11it/s, est. speed input: 69100.18 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1788/3051 [14:07<09:57,  2.12it/s, est. speed input: 69100.31 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1789/3051 [14:08<09:57,  2.11it/s, est. speed input: 69100.35 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1790/3051 [14:08<09:56,  2.11it/s, est. speed input: 69100.45 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1791/3051 [14:09<09:56,  2.11it/s, est. speed input: 69100.54 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1792/3051 [14:09<09:55,  2.11it/s, est. speed input: 69100.65 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1793/3051 [14:10<09:54,  2.12it/s, est. speed input: 69100.82 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1794/3051 [14:10<09:54,  2.12it/s, est. speed input: 69100.98 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1795/3051 [14:11<09:53,  2.12it/s, est. speed input: 69101.11 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1796/3051 [14:11<09:53,  2.11it/s, est. speed input: 69101.12 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1797/3051 [14:12<09:52,  2.11it/s, est. speed input: 69101.27 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1798/3051 [14:12<09:52,  2.11it/s, est. speed input: 69101.36 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1799/3051 [14:13<09:53,  2.11it/s, est. speed input: 69101.18 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1800/3051 [14:13<09:52,  2.11it/s, est. speed input: 69101.26 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1801/3051 [14:14<09:51,  2.11it/s, est. speed input: 69101.36 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1802/3051 [14:14<09:51,  2.11it/s, est. speed input: 69101.47 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1803/3051 [14:14<09:50,  2.11it/s, est. speed input: 69101.60 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1804/3051 [14:15<09:49,  2.11it/s, est. speed input: 69101.73 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1805/3051 [14:15<09:49,  2.11it/s, est. speed input: 69101.86 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1806/3051 [14:16<09:48,  2.11it/s, est. speed input: 69101.97 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1807/3051 [14:16<09:48,  2.12it/s, est. speed input: 69102.09 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1808/3051 [14:17<09:47,  2.12it/s, est. speed input: 69102.22 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1809/3051 [14:17<09:47,  2.12it/s, est. speed input: 69102.35 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1810/3051 [14:18<09:46,  2.12it/s, est. speed input: 69102.49 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1811/3051 [14:18<09:46,  2.11it/s, est. speed input: 69102.44 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1812/3051 [14:19<09:46,  2.11it/s, est. speed input: 69102.47 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1813/3051 [14:19<09:45,  2.11it/s, est. speed input: 69102.56 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1814/3051 [14:20<09:45,  2.11it/s, est. speed input: 69102.68 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1815/3051 [14:20<09:44,  2.11it/s, est. speed input: 69102.78 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1816/3051 [14:21<09:44,  2.11it/s, est. speed input: 69102.92 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1817/3051 [14:21<09:55,  2.07it/s, est. speed input: 69100.54 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1818/3051 [14:22<09:50,  2.09it/s, est. speed input: 69100.79 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1819/3051 [14:22<09:47,  2.10it/s, est. speed input: 69100.92 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1820/3051 [14:23<09:45,  2.10it/s, est. speed input: 69101.04 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1821/3051 [14:23<09:45,  2.10it/s, est. speed input: 69100.93 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1822/3051 [14:24<09:43,  2.11it/s, est. speed input: 69101.04 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1823/3051 [14:24<09:42,  2.11it/s, est. speed input: 69101.11 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1824/3051 [14:24<09:42,  2.11it/s, est. speed input: 69101.10 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1825/3051 [14:25<09:41,  2.11it/s, est. speed input: 69101.21 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1826/3051 [14:25<09:39,  2.11it/s, est. speed input: 69101.37 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1827/3051 [14:26<09:39,  2.11it/s, est. speed input: 69101.46 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1828/3051 [14:26<09:38,  2.11it/s, est. speed input: 69101.58 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1829/3051 [14:27<09:38,  2.11it/s, est. speed input: 69101.68 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1830/3051 [14:27<09:37,  2.11it/s, est. speed input: 69101.75 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1831/3051 [14:28<09:38,  2.11it/s, est. speed input: 69101.65 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1832/3051 [14:28<09:37,  2.11it/s, est. speed input: 69101.79 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1833/3051 [14:29<09:36,  2.11it/s, est. speed input: 69101.81 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1834/3051 [14:29<09:38,  2.11it/s, est. speed input: 69101.48 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1835/3051 [14:30<09:37,  2.11it/s, est. speed input: 69101.44 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1836/3051 [14:30<09:36,  2.11it/s, est. speed input: 69101.44 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1837/3051 [14:31<09:37,  2.10it/s, est. speed input: 69101.23 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1838/3051 [14:31<09:41,  2.08it/s, est. speed input: 69099.99 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1839/3051 [14:32<09:47,  2.06it/s, est. speed input: 69098.23 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1840/3051 [14:32<09:43,  2.07it/s, est. speed input: 69098.09 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1841/3051 [14:33<09:40,  2.09it/s, est. speed input: 69098.17 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1842/3051 [14:33<09:37,  2.09it/s, est. speed input: 69098.24 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1843/3051 [14:33<09:35,  2.10it/s, est. speed input: 69098.33 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1844/3051 [14:34<09:34,  2.10it/s, est. speed input: 69098.34 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1845/3051 [14:34<09:33,  2.10it/s, est. speed input: 69098.20 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1846/3051 [14:35<09:32,  2.10it/s, est. speed input: 69098.18 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1847/3051 [14:35<09:31,  2.11it/s, est. speed input: 69098.28 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1848/3051 [14:36<09:30,  2.11it/s, est. speed input: 69098.34 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1849/3051 [14:36<09:29,  2.11it/s, est. speed input: 69098.45 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1850/3051 [14:37<09:29,  2.11it/s, est. speed input: 69098.45 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1851/3051 [14:37<09:28,  2.11it/s, est. speed input: 69098.53 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1852/3051 [14:38<09:28,  2.11it/s, est. speed input: 69098.50 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1853/3051 [14:38<09:27,  2.11it/s, est. speed input: 69098.52 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1854/3051 [14:39<09:27,  2.11it/s, est. speed input: 69098.54 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1855/3051 [14:39<09:28,  2.10it/s, est. speed input: 69098.16 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1856/3051 [14:40<09:27,  2.11it/s, est. speed input: 69098.19 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1857/3051 [14:40<09:26,  2.11it/s, est. speed input: 69098.28 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1858/3051 [14:41<09:25,  2.11it/s, est. speed input: 69098.37 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1859/3051 [14:41<09:24,  2.11it/s, est. speed input: 69098.46 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1860/3051 [14:42<09:23,  2.11it/s, est. speed input: 69098.55 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1861/3051 [14:42<09:23,  2.11it/s, est. speed input: 69098.54 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1862/3051 [14:43<09:23,  2.11it/s, est. speed input: 69098.54 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1863/3051 [14:43<09:22,  2.11it/s, est. speed input: 69098.63 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1864/3051 [14:43<09:22,  2.11it/s, est. speed input: 69098.69 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1865/3051 [14:44<09:21,  2.11it/s, est. speed input: 69098.67 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1866/3051 [14:44<09:21,  2.11it/s, est. speed input: 69098.71 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1867/3051 [14:45<09:45,  2.02it/s, est. speed input: 69093.44 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1868/3051 [14:45<09:36,  2.05it/s, est. speed input: 69093.65 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1869/3051 [14:46<09:31,  2.07it/s, est. speed input: 69093.68 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1870/3051 [14:46<09:27,  2.08it/s, est. speed input: 69093.76 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1871/3051 [14:47<09:24,  2.09it/s, est. speed input: 69093.81 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1872/3051 [14:47<09:22,  2.10it/s, est. speed input: 69093.84 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1873/3051 [14:48<09:20,  2.10it/s, est. speed input: 69093.95 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1874/3051 [14:48<09:19,  2.10it/s, est. speed input: 69093.98 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1875/3051 [14:49<09:17,  2.11it/s, est. speed input: 69094.13 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1876/3051 [14:49<09:17,  2.11it/s, est. speed input: 69094.18 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1877/3051 [14:50<09:15,  2.11it/s, est. speed input: 69094.33 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1878/3051 [14:50<09:14,  2.11it/s, est. speed input: 69094.51 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1879/3051 [14:51<09:14,  2.12it/s, est. speed input: 69094.67 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1880/3051 [14:51<09:13,  2.11it/s, est. speed input: 69094.77 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1881/3051 [14:52<09:13,  2.11it/s, est. speed input: 69094.85 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1882/3051 [14:52<09:12,  2.12it/s, est. speed input: 69095.02 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1883/3051 [14:53<09:12,  2.11it/s, est. speed input: 69095.10 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1884/3051 [14:53<09:11,  2.12it/s, est. speed input: 69095.24 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1885/3051 [14:53<09:11,  2.12it/s, est. speed input: 69095.34 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1886/3051 [14:54<09:10,  2.12it/s, est. speed input: 69095.52 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1887/3051 [14:54<09:09,  2.12it/s, est. speed input: 69095.66 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1888/3051 [14:55<09:09,  2.12it/s, est. speed input: 69095.78 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1889/3051 [14:55<09:09,  2.12it/s, est. speed input: 69095.90 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1890/3051 [14:56<09:08,  2.12it/s, est. speed input: 69096.02 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1891/3051 [14:56<09:08,  2.12it/s, est. speed input: 69096.12 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1892/3051 [14:57<09:08,  2.11it/s, est. speed input: 69096.19 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1893/3051 [14:57<09:07,  2.11it/s, est. speed input: 69096.29 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1894/3051 [14:58<09:06,  2.12it/s, est. speed input: 69096.43 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1895/3051 [14:58<09:06,  2.12it/s, est. speed input: 69096.55 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1896/3051 [14:59<09:05,  2.12it/s, est. speed input: 69096.70 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1897/3051 [14:59<09:05,  2.11it/s, est. speed input: 69096.77 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1898/3051 [15:00<09:05,  2.11it/s, est. speed input: 69096.86 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1899/3051 [15:00<09:04,  2.11it/s, est. speed input: 69096.97 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1900/3051 [15:01<09:04,  2.11it/s, est. speed input: 69096.97 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1901/3051 [15:01<09:04,  2.11it/s, est. speed input: 69097.06 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1902/3051 [15:01<09:03,  2.11it/s, est. speed input: 69097.15 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1903/3051 [15:02<09:03,  2.11it/s, est. speed input: 69097.27 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1904/3051 [15:02<09:02,  2.11it/s, est. speed input: 69097.40 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1905/3051 [15:03<09:02,  2.11it/s, est. speed input: 69097.44 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1906/3051 [15:03<09:01,  2.11it/s, est. speed input: 69097.54 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1907/3051 [15:04<09:00,  2.11it/s, est. speed input: 69097.67 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1908/3051 [15:04<09:00,  2.11it/s, est. speed input: 69097.79 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1909/3051 [15:05<09:00,  2.11it/s, est. speed input: 69097.87 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1910/3051 [15:05<08:59,  2.11it/s, est. speed input: 69097.92 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1911/3051 [15:06<08:59,  2.11it/s, est. speed input: 69098.06 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1912/3051 [15:06<08:58,  2.11it/s, est. speed input: 69098.15 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1913/3051 [15:07<08:58,  2.11it/s, est. speed input: 69098.26 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1914/3051 [15:07<08:57,  2.11it/s, est. speed input: 69098.36 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1915/3051 [15:08<08:56,  2.12it/s, est. speed input: 69098.51 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1916/3051 [15:08<08:56,  2.12it/s, est. speed input: 69098.63 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1917/3051 [15:09<08:56,  2.12it/s, est. speed input: 69098.75 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1918/3051 [15:09<08:56,  2.11it/s, est. speed input: 69098.75 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1919/3051 [15:10<08:55,  2.11it/s, est. speed input: 69098.78 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1920/3051 [15:10<08:56,  2.11it/s, est. speed input: 69098.65 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1921/3051 [15:10<08:55,  2.11it/s, est. speed input: 69098.74 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1922/3051 [15:11<08:54,  2.11it/s, est. speed input: 69098.83 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1923/3051 [15:11<08:53,  2.11it/s, est. speed input: 69098.95 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1924/3051 [15:12<09:01,  2.08it/s, est. speed input: 69097.30 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1925/3051 [15:12<08:57,  2.09it/s, est. speed input: 69097.46 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1926/3051 [15:13<08:56,  2.10it/s, est. speed input: 69097.49 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1927/3051 [15:13<08:54,  2.10it/s, est. speed input: 69097.60 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1928/3051 [15:14<08:53,  2.11it/s, est. speed input: 69097.67 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1929/3051 [15:14<08:51,  2.11it/s, est. speed input: 69097.83 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1930/3051 [15:15<08:50,  2.11it/s, est. speed input: 69097.94 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1931/3051 [15:15<08:50,  2.11it/s, est. speed input: 69098.04 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1932/3051 [15:16<08:49,  2.11it/s, est. speed input: 69098.16 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1933/3051 [15:16<08:49,  2.11it/s, est. speed input: 69098.21 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1934/3051 [15:17<08:48,  2.11it/s, est. speed input: 69098.30 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1935/3051 [15:17<08:48,  2.11it/s, est. speed input: 69098.39 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1936/3051 [15:18<08:47,  2.11it/s, est. speed input: 69098.48 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1937/3051 [15:18<08:46,  2.11it/s, est. speed input: 69098.58 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1938/3051 [15:19<08:46,  2.12it/s, est. speed input: 69098.73 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1939/3051 [15:19<08:45,  2.12it/s, est. speed input: 69098.87 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1940/3051 [15:19<08:45,  2.12it/s, est. speed input: 69098.98 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1941/3051 [15:20<08:44,  2.11it/s, est. speed input: 69099.06 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1942/3051 [15:20<08:44,  2.12it/s, est. speed input: 69099.18 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1943/3051 [15:21<08:43,  2.12it/s, est. speed input: 69099.28 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1944/3051 [15:21<08:43,  2.12it/s, est. speed input: 69099.42 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1945/3051 [15:22<08:42,  2.12it/s, est. speed input: 69099.56 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1946/3051 [15:22<08:42,  2.12it/s, est. speed input: 69099.63 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1947/3051 [15:23<08:41,  2.12it/s, est. speed input: 69099.76 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1948/3051 [15:23<08:41,  2.11it/s, est. speed input: 69099.79 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1949/3051 [15:24<08:41,  2.11it/s, est. speed input: 69099.91 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1950/3051 [15:24<08:40,  2.12it/s, est. speed input: 69100.06 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1951/3051 [15:25<08:39,  2.12it/s, est. speed input: 69100.18 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1952/3051 [15:25<08:39,  2.11it/s, est. speed input: 69100.20 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1953/3051 [15:26<08:39,  2.11it/s, est. speed input: 69100.31 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1954/3051 [15:26<08:38,  2.11it/s, est. speed input: 69100.42 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1955/3051 [15:27<08:38,  2.12it/s, est. speed input: 69100.54 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1956/3051 [15:27<08:37,  2.11it/s, est. speed input: 69100.60 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1957/3051 [15:28<08:37,  2.12it/s, est. speed input: 69100.74 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1958/3051 [15:28<08:36,  2.12it/s, est. speed input: 69100.85 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1959/3051 [15:28<08:36,  2.12it/s, est. speed input: 69100.95 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1960/3051 [15:29<08:35,  2.11it/s, est. speed input: 69101.03 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1961/3051 [15:29<08:35,  2.11it/s, est. speed input: 69101.14 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1962/3051 [15:30<08:34,  2.11it/s, est. speed input: 69101.25 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1963/3051 [15:30<08:34,  2.11it/s, est. speed input: 69101.35 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1964/3051 [15:31<08:34,  2.11it/s, est. speed input: 69101.42 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1965/3051 [15:31<08:33,  2.11it/s, est. speed input: 69101.50 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1966/3051 [15:32<08:32,  2.12it/s, est. speed input: 69101.67 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1967/3051 [15:32<08:32,  2.11it/s, est. speed input: 69101.74 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1968/3051 [15:33<08:32,  2.11it/s, est. speed input: 69101.84 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1969/3051 [15:33<08:31,  2.12it/s, est. speed input: 69101.97 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1970/3051 [15:34<08:30,  2.12it/s, est. speed input: 69102.14 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1971/3051 [15:34<08:30,  2.12it/s, est. speed input: 69102.23 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1972/3051 [15:35<08:30,  2.12it/s, est. speed input: 69102.31 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1973/3051 [15:35<08:29,  2.12it/s, est. speed input: 69102.41 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1974/3051 [15:36<08:29,  2.12it/s, est. speed input: 69102.53 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1975/3051 [15:36<08:28,  2.12it/s, est. speed input: 69102.64 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1976/3051 [15:37<08:28,  2.11it/s, est. speed input: 69102.59 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1977/3051 [15:37<08:28,  2.11it/s, est. speed input: 69102.63 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1978/3051 [15:37<08:28,  2.11it/s, est. speed input: 69102.66 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1979/3051 [15:38<08:27,  2.11it/s, est. speed input: 69102.78 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1980/3051 [15:38<08:26,  2.11it/s, est. speed input: 69102.89 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1981/3051 [15:39<08:26,  2.11it/s, est. speed input: 69103.00 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1982/3051 [15:39<08:25,  2.11it/s, est. speed input: 69103.11 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1983/3051 [15:40<08:24,  2.11it/s, est. speed input: 69103.22 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1984/3051 [15:40<08:24,  2.11it/s, est. speed input: 69103.30 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1985/3051 [15:41<08:23,  2.12it/s, est. speed input: 69103.45 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1986/3051 [15:41<08:23,  2.12it/s, est. speed input: 69103.57 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1987/3051 [15:42<08:24,  2.11it/s, est. speed input: 69103.37 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1988/3051 [15:42<08:23,  2.11it/s, est. speed input: 69103.44 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1989/3051 [15:43<08:22,  2.11it/s, est. speed input: 69103.54 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1990/3051 [15:43<08:21,  2.11it/s, est. speed input: 69103.68 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1991/3051 [15:44<08:21,  2.11it/s, est. speed input: 69103.67 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1992/3051 [15:44<08:27,  2.09it/s, est. speed input: 69102.38 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1993/3051 [15:45<08:32,  2.06it/s, est. speed input: 69100.61 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1994/3051 [15:45<08:29,  2.08it/s, est. speed input: 69100.58 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1995/3051 [15:46<08:25,  2.09it/s, est. speed input: 69100.72 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1996/3051 [15:46<08:23,  2.10it/s, est. speed input: 69100.81 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1997/3051 [15:46<08:21,  2.10it/s, est. speed input: 69100.92 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1998/3051 [15:47<08:20,  2.11it/s, est. speed input: 69101.02 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 1999/3051 [15:47<08:18,  2.11it/s, est. speed input: 69101.16 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2000/3051 [15:48<08:17,  2.11it/s, est. speed input: 69101.30 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2001/3051 [15:48<08:16,  2.11it/s, est. speed input: 69101.43 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2002/3051 [15:49<08:16,  2.11it/s, est. speed input: 69101.47 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2003/3051 [15:49<08:15,  2.11it/s, est. speed input: 69101.56 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2004/3051 [15:50<08:15,  2.11it/s, est. speed input: 69101.68 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2005/3051 [15:50<08:14,  2.11it/s, est. speed input: 69101.75 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2006/3051 [15:51<08:14,  2.11it/s, est. speed input: 69101.76 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2007/3051 [15:51<08:14,  2.11it/s, est. speed input: 69101.85 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2008/3051 [15:52<08:14,  2.11it/s, est. speed input: 69101.81 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2009/3051 [15:52<08:13,  2.11it/s, est. speed input: 69101.93 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2010/3051 [15:53<08:12,  2.11it/s, est. speed input: 69102.03 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2011/3051 [15:53<08:12,  2.11it/s, est. speed input: 69102.02 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2012/3051 [15:54<08:11,  2.11it/s, est. speed input: 69102.12 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2013/3051 [15:54<08:11,  2.11it/s, est. speed input: 69102.20 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2014/3051 [15:55<08:10,  2.11it/s, est. speed input: 69102.27 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2015/3051 [15:55<08:10,  2.11it/s, est. speed input: 69102.36 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2016/3051 [15:55<08:09,  2.11it/s, est. speed input: 69102.46 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2017/3051 [15:56<08:08,  2.11it/s, est. speed input: 69102.58 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2018/3051 [15:56<08:08,  2.12it/s, est. speed input: 69102.71 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2019/3051 [15:57<08:07,  2.12it/s, est. speed input: 69102.81 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2020/3051 [15:57<08:07,  2.12it/s, est. speed input: 69102.91 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2021/3051 [15:58<08:07,  2.11it/s, est. speed input: 69102.99 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2022/3051 [15:58<08:06,  2.11it/s, est. speed input: 69103.05 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2023/3051 [15:59<08:06,  2.11it/s, est. speed input: 69103.12 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2024/3051 [15:59<08:05,  2.11it/s, est. speed input: 69103.24 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2025/3051 [16:00<08:05,  2.12it/s, est. speed input: 69103.37 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2026/3051 [16:00<08:04,  2.12it/s, est. speed input: 69103.47 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2027/3051 [16:01<08:04,  2.11it/s, est. speed input: 69103.55 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2028/3051 [16:01<08:03,  2.12it/s, est. speed input: 69103.69 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2029/3051 [16:02<08:03,  2.11it/s, est. speed input: 69103.76 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2030/3051 [16:02<08:02,  2.11it/s, est. speed input: 69103.85 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2031/3051 [16:03<08:02,  2.11it/s, est. speed input: 69103.88 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2032/3051 [16:03<08:02,  2.11it/s, est. speed input: 69103.95 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2033/3051 [16:04<08:01,  2.11it/s, est. speed input: 69104.08 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2034/3051 [16:04<08:01,  2.11it/s, est. speed input: 69104.09 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2035/3051 [16:04<08:00,  2.11it/s, est. speed input: 69104.14 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2036/3051 [16:05<08:00,  2.11it/s, est. speed input: 69104.21 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2037/3051 [16:05<07:59,  2.11it/s, est. speed input: 69104.27 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2038/3051 [16:06<07:59,  2.11it/s, est. speed input: 69104.39 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2039/3051 [16:06<07:58,  2.11it/s, est. speed input: 69104.49 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2040/3051 [16:07<07:58,  2.11it/s, est. speed input: 69104.56 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2041/3051 [16:07<07:57,  2.11it/s, est. speed input: 69104.67 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2042/3051 [16:08<07:57,  2.11it/s, est. speed input: 69104.78 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2043/3051 [16:08<07:56,  2.12it/s, est. speed input: 69104.89 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2044/3051 [16:09<07:55,  2.12it/s, est. speed input: 69105.04 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2045/3051 [16:09<07:55,  2.12it/s, est. speed input: 69105.13 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2046/3051 [16:10<07:55,  2.12it/s, est. speed input: 69105.23 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2047/3051 [16:10<07:54,  2.12it/s, est. speed input: 69105.35 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2048/3051 [16:11<07:54,  2.12it/s, est. speed input: 69105.45 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2049/3051 [16:11<07:53,  2.11it/s, est. speed input: 69105.52 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2050/3051 [16:12<07:53,  2.11it/s, est. speed input: 69105.55 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2051/3051 [16:12<07:53,  2.11it/s, est. speed input: 69105.60 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2052/3051 [16:13<07:52,  2.11it/s, est. speed input: 69105.70 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2053/3051 [16:13<07:52,  2.11it/s, est. speed input: 69105.80 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2054/3051 [16:13<07:51,  2.11it/s, est. speed input: 69105.89 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2055/3051 [16:14<07:51,  2.11it/s, est. speed input: 69105.97 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2056/3051 [16:14<07:51,  2.11it/s, est. speed input: 69105.91 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2057/3051 [16:15<07:50,  2.11it/s, est. speed input: 69105.96 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2058/3051 [16:15<07:50,  2.11it/s, est. speed input: 69106.04 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2059/3051 [16:16<07:49,  2.11it/s, est. speed input: 69106.16 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2060/3051 [16:16<07:48,  2.11it/s, est. speed input: 69106.29 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2061/3051 [16:17<07:48,  2.11it/s, est. speed input: 69106.36 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2062/3051 [16:17<07:47,  2.11it/s, est. speed input: 69106.42 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2063/3051 [16:18<07:47,  2.11it/s, est. speed input: 69106.51 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2064/3051 [16:18<07:46,  2.11it/s, est. speed input: 69106.58 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2065/3051 [16:19<07:46,  2.11it/s, est. speed input: 69106.69 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2066/3051 [16:19<07:45,  2.11it/s, est. speed input: 69106.76 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2067/3051 [16:20<07:45,  2.11it/s, est. speed input: 69106.85 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2068/3051 [16:20<07:44,  2.11it/s, est. speed input: 69106.92 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2069/3051 [16:21<07:44,  2.12it/s, est. speed input: 69107.05 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2070/3051 [16:21<07:43,  2.11it/s, est. speed input: 69107.12 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2071/3051 [16:21<07:43,  2.12it/s, est. speed input: 69107.24 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2072/3051 [16:22<07:42,  2.11it/s, est. speed input: 69107.33 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2073/3051 [16:22<07:42,  2.12it/s, est. speed input: 69107.42 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2074/3051 [16:23<07:41,  2.11it/s, est. speed input: 69107.51 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2075/3051 [16:23<07:41,  2.11it/s, est. speed input: 69107.57 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2076/3051 [16:24<07:40,  2.12it/s, est. speed input: 69107.70 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2077/3051 [16:24<07:40,  2.12it/s, est. speed input: 69107.80 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2078/3051 [16:25<07:40,  2.12it/s, est. speed input: 69107.89 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2079/3051 [16:25<07:39,  2.12it/s, est. speed input: 69107.98 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2080/3051 [16:26<07:38,  2.12it/s, est. speed input: 69108.11 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2081/3051 [16:26<07:38,  2.12it/s, est. speed input: 69108.23 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2082/3051 [16:27<07:38,  2.12it/s, est. speed input: 69108.31 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2083/3051 [16:27<07:37,  2.12it/s, est. speed input: 69108.41 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2084/3051 [16:28<07:36,  2.12it/s, est. speed input: 69108.54 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2085/3051 [16:28<07:36,  2.11it/s, est. speed input: 69108.59 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2086/3051 [16:29<07:36,  2.11it/s, est. speed input: 69108.68 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2087/3051 [16:29<07:35,  2.11it/s, est. speed input: 69108.78 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2088/3051 [16:30<07:35,  2.12it/s, est. speed input: 69108.90 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2089/3051 [16:30<07:34,  2.12it/s, est. speed input: 69109.01 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2090/3051 [16:30<07:34,  2.12it/s, est. speed input: 69109.10 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2091/3051 [16:31<07:33,  2.12it/s, est. speed input: 69109.20 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2092/3051 [16:31<07:33,  2.12it/s, est. speed input: 69109.29 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2093/3051 [16:32<07:33,  2.11it/s, est. speed input: 69109.36 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2094/3051 [16:32<07:32,  2.11it/s, est. speed input: 69109.44 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2095/3051 [16:33<07:31,  2.12it/s, est. speed input: 69109.55 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2096/3051 [16:33<07:31,  2.12it/s, est. speed input: 69109.68 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2097/3051 [16:34<07:30,  2.12it/s, est. speed input: 69109.78 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2098/3051 [16:34<07:30,  2.11it/s, est. speed input: 69109.83 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2099/3051 [16:35<07:30,  2.11it/s, est. speed input: 69109.87 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2100/3051 [16:35<07:29,  2.11it/s, est. speed input: 69109.95 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2101/3051 [16:36<07:29,  2.11it/s, est. speed input: 69110.08 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2102/3051 [16:36<07:28,  2.11it/s, est. speed input: 69110.16 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2103/3051 [16:37<07:28,  2.12it/s, est. speed input: 69110.30 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2104/3051 [16:37<07:27,  2.12it/s, est. speed input: 69110.41 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2105/3051 [16:38<07:27,  2.12it/s, est. speed input: 69110.48 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2106/3051 [16:38<07:26,  2.12it/s, est. speed input: 69110.58 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2107/3051 [16:39<07:26,  2.11it/s, est. speed input: 69110.65 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2108/3051 [16:39<07:26,  2.11it/s, est. speed input: 69110.63 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2109/3051 [16:39<07:25,  2.11it/s, est. speed input: 69110.70 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2110/3051 [16:40<07:25,  2.11it/s, est. speed input: 69110.82 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2111/3051 [16:40<07:25,  2.11it/s, est. speed input: 69110.74 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2112/3051 [16:41<07:24,  2.11it/s, est. speed input: 69110.86 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2113/3051 [16:41<07:23,  2.11it/s, est. speed input: 69110.98 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2114/3051 [16:42<07:23,  2.11it/s, est. speed input: 69111.10 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2115/3051 [16:42<07:22,  2.11it/s, est. speed input: 69111.18 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2116/3051 [16:43<07:22,  2.11it/s, est. speed input: 69111.28 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2117/3051 [16:43<07:21,  2.11it/s, est. speed input: 69111.37 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2118/3051 [16:44<07:21,  2.12it/s, est. speed input: 69111.48 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2119/3051 [16:44<07:20,  2.12it/s, est. speed input: 69111.58 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2120/3051 [16:45<07:20,  2.12it/s, est. speed input: 69111.68 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2121/3051 [16:45<07:19,  2.11it/s, est. speed input: 69111.71 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2122/3051 [16:46<07:19,  2.11it/s, est. speed input: 69111.80 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2123/3051 [16:46<07:18,  2.11it/s, est. speed input: 69111.88 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2124/3051 [16:47<07:18,  2.11it/s, est. speed input: 69111.93 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2125/3051 [16:47<07:17,  2.11it/s, est. speed input: 69112.05 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2126/3051 [16:47<07:17,  2.11it/s, est. speed input: 69112.14 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2127/3051 [16:48<07:16,  2.12it/s, est. speed input: 69112.24 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2128/3051 [16:48<07:16,  2.12it/s, est. speed input: 69112.33 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2129/3051 [16:49<07:15,  2.11it/s, est. speed input: 69112.41 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2130/3051 [16:49<07:15,  2.11it/s, est. speed input: 69112.49 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2131/3051 [16:50<07:15,  2.11it/s, est. speed input: 69112.56 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2132/3051 [16:50<07:14,  2.11it/s, est. speed input: 69112.64 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2133/3051 [16:51<07:14,  2.11it/s, est. speed input: 69112.74 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2134/3051 [16:51<07:13,  2.11it/s, est. speed input: 69112.80 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2135/3051 [16:52<07:13,  2.12it/s, est. speed input: 69112.93 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2136/3051 [16:52<07:12,  2.12it/s, est. speed input: 69113.02 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2137/3051 [16:53<07:12,  2.12it/s, est. speed input: 69113.12 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2138/3051 [16:53<07:11,  2.12it/s, est. speed input: 69113.25 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2139/3051 [16:54<07:10,  2.12it/s, est. speed input: 69113.37 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2140/3051 [16:54<07:11,  2.11it/s, est. speed input: 69113.31 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2141/3051 [16:55<07:14,  2.10it/s, est. speed input: 69112.48 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2142/3051 [16:55<07:20,  2.06it/s, est. speed input: 69110.53 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2143/3051 [16:56<07:17,  2.08it/s, est. speed input: 69110.47 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2144/3051 [16:56<07:14,  2.09it/s, est. speed input: 69110.60 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2145/3051 [16:57<07:12,  2.10it/s, est. speed input: 69110.70 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2146/3051 [16:57<07:11,  2.10it/s, est. speed input: 69110.65 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2147/3051 [16:57<07:09,  2.10it/s, est. speed input: 69110.70 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2148/3051 [16:58<07:08,  2.11it/s, est. speed input: 69110.81 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2149/3051 [16:58<07:07,  2.11it/s, est. speed input: 69110.91 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2150/3051 [16:59<07:06,  2.11it/s, est. speed input: 69111.01 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2151/3051 [16:59<07:06,  2.11it/s, est. speed input: 69111.10 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2152/3051 [17:00<07:05,  2.11it/s, est. speed input: 69111.20 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2153/3051 [17:00<07:04,  2.11it/s, est. speed input: 69111.27 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2154/3051 [17:01<07:04,  2.11it/s, est. speed input: 69111.35 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2155/3051 [17:01<07:03,  2.11it/s, est. speed input: 69111.43 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2156/3051 [17:02<07:03,  2.11it/s, est. speed input: 69111.54 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2157/3051 [17:02<07:02,  2.12it/s, est. speed input: 69111.65 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2158/3051 [17:03<07:02,  2.12it/s, est. speed input: 69111.76 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2159/3051 [17:03<07:01,  2.12it/s, est. speed input: 69111.83 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2160/3051 [17:04<07:01,  2.11it/s, est. speed input: 69111.89 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2161/3051 [17:04<07:00,  2.12it/s, est. speed input: 69112.01 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2162/3051 [17:05<07:00,  2.11it/s, est. speed input: 69112.09 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2163/3051 [17:05<06:59,  2.12it/s, est. speed input: 69112.20 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2164/3051 [17:06<06:59,  2.12it/s, est. speed input: 69112.31 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2165/3051 [17:06<06:58,  2.12it/s, est. speed input: 69112.39 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2166/3051 [17:06<06:58,  2.11it/s, est. speed input: 69112.42 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2167/3051 [17:07<06:58,  2.11it/s, est. speed input: 69112.51 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2168/3051 [17:07<06:57,  2.11it/s, est. speed input: 69112.58 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2169/3051 [17:08<06:57,  2.11it/s, est. speed input: 69112.69 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2170/3051 [17:08<06:56,  2.12it/s, est. speed input: 69112.79 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2171/3051 [17:09<06:56,  2.11it/s, est. speed input: 69112.85 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2172/3051 [17:09<06:55,  2.11it/s, est. speed input: 69112.92 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2173/3051 [17:10<06:55,  2.11it/s, est. speed input: 69112.99 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2174/3051 [17:10<06:54,  2.11it/s, est. speed input: 69113.10 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2175/3051 [17:11<06:54,  2.11it/s, est. speed input: 69113.18 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2176/3051 [17:11<06:53,  2.12it/s, est. speed input: 69113.33 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2177/3051 [17:12<06:53,  2.12it/s, est. speed input: 69113.39 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2178/3051 [17:12<06:52,  2.11it/s, est. speed input: 69113.47 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2179/3051 [17:13<06:52,  2.11it/s, est. speed input: 69113.55 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2180/3051 [17:13<06:51,  2.11it/s, est. speed input: 69113.64 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2181/3051 [17:14<06:51,  2.11it/s, est. speed input: 69113.71 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2182/3051 [17:14<06:50,  2.12it/s, est. speed input: 69113.84 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2183/3051 [17:14<06:50,  2.12it/s, est. speed input: 69113.93 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2184/3051 [17:15<06:49,  2.12it/s, est. speed input: 69114.05 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2185/3051 [17:15<06:49,  2.12it/s, est. speed input: 69114.13 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2186/3051 [17:16<06:48,  2.12it/s, est. speed input: 69114.23 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2187/3051 [17:16<06:48,  2.12it/s, est. speed input: 69114.33 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2188/3051 [17:17<06:47,  2.12it/s, est. speed input: 69114.46 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2189/3051 [17:17<06:47,  2.12it/s, est. speed input: 69114.54 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2190/3051 [17:18<06:47,  2.12it/s, est. speed input: 69114.60 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2191/3051 [17:18<06:46,  2.12it/s, est. speed input: 69114.70 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2192/3051 [17:19<06:46,  2.12it/s, est. speed input: 69114.79 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2193/3051 [17:19<06:45,  2.12it/s, est. speed input: 69114.89 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2194/3051 [17:20<06:45,  2.12it/s, est. speed input: 69115.00 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2195/3051 [17:20<06:44,  2.11it/s, est. speed input: 69115.05 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2196/3051 [17:21<06:44,  2.11it/s, est. speed input: 69114.97 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2197/3051 [17:21<06:44,  2.11it/s, est. speed input: 69115.06 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2198/3051 [17:22<06:43,  2.11it/s, est. speed input: 69115.12 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2199/3051 [17:22<06:43,  2.11it/s, est. speed input: 69115.23 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2200/3051 [17:23<06:42,  2.11it/s, est. speed input: 69115.28 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2201/3051 [17:23<06:42,  2.11it/s, est. speed input: 69115.38 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2202/3051 [17:23<06:41,  2.11it/s, est. speed input: 69115.42 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2203/3051 [17:24<06:41,  2.11it/s, est. speed input: 69115.54 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2204/3051 [17:24<06:40,  2.11it/s, est. speed input: 69115.59 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2205/3051 [17:25<06:40,  2.11it/s, est. speed input: 69115.69 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2206/3051 [17:25<06:39,  2.11it/s, est. speed input: 69115.77 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2207/3051 [17:26<06:38,  2.12it/s, est. speed input: 69115.93 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2208/3051 [17:26<06:38,  2.12it/s, est. speed input: 69116.01 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2209/3051 [17:27<06:37,  2.12it/s, est. speed input: 69116.10 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2210/3051 [17:27<06:37,  2.11it/s, est. speed input: 69116.16 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2211/3051 [17:28<06:37,  2.11it/s, est. speed input: 69116.13 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2212/3051 [17:28<06:36,  2.11it/s, est. speed input: 69116.22 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2213/3051 [17:29<06:36,  2.11it/s, est. speed input: 69116.32 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2214/3051 [17:29<06:35,  2.11it/s, est. speed input: 69116.42 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2215/3051 [17:30<06:35,  2.12it/s, est. speed input: 69116.54 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2216/3051 [17:30<06:34,  2.12it/s, est. speed input: 69116.62 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2217/3051 [17:31<06:34,  2.11it/s, est. speed input: 69116.68 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2218/3051 [17:31<06:33,  2.12it/s, est. speed input: 69116.79 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2219/3051 [17:32<06:33,  2.12it/s, est. speed input: 69116.89 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2220/3051 [17:32<06:32,  2.12it/s, est. speed input: 69116.97 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2221/3051 [17:32<06:32,  2.11it/s, est. speed input: 69117.00 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2222/3051 [17:33<06:32,  2.11it/s, est. speed input: 69117.09 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2223/3051 [17:33<06:31,  2.12it/s, est. speed input: 69117.23 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2224/3051 [17:34<06:30,  2.12it/s, est. speed input: 69117.32 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2225/3051 [17:34<06:30,  2.12it/s, est. speed input: 69117.43 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2226/3051 [17:35<06:29,  2.12it/s, est. speed input: 69117.56 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2227/3051 [17:35<06:29,  2.12it/s, est. speed input: 69117.63 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2228/3051 [17:36<06:29,  2.12it/s, est. speed input: 69117.69 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2229/3051 [17:36<06:28,  2.12it/s, est. speed input: 69117.80 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2230/3051 [17:37<06:28,  2.12it/s, est. speed input: 69117.86 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2231/3051 [17:37<06:27,  2.11it/s, est. speed input: 69117.92 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2232/3051 [17:38<06:27,  2.11it/s, est. speed input: 69118.00 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2233/3051 [17:38<06:26,  2.12it/s, est. speed input: 69118.11 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2234/3051 [17:39<06:26,  2.12it/s, est. speed input: 69118.24 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2235/3051 [17:39<06:25,  2.11it/s, est. speed input: 69118.26 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2236/3051 [17:40<06:25,  2.11it/s, est. speed input: 69118.35 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2237/3051 [17:40<06:25,  2.11it/s, est. speed input: 69118.36 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2238/3051 [17:41<06:24,  2.11it/s, est. speed input: 69118.49 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2239/3051 [17:41<06:24,  2.11it/s, est. speed input: 69118.55 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2240/3051 [17:41<06:23,  2.11it/s, est. speed input: 69118.64 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2241/3051 [17:42<06:23,  2.11it/s, est. speed input: 69118.70 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2242/3051 [17:42<06:22,  2.12it/s, est. speed input: 69118.81 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2243/3051 [17:43<06:27,  2.09it/s, est. speed input: 69117.50 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2244/3051 [17:43<06:29,  2.07it/s, est. speed input: 69116.37 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2245/3051 [17:44<06:32,  2.05it/s, est. speed input: 69114.91 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2246/3051 [17:44<06:28,  2.07it/s, est. speed input: 69114.91 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2247/3051 [17:45<06:26,  2.08it/s, est. speed input: 69114.98 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2248/3051 [17:45<06:23,  2.09it/s, est. speed input: 69115.07 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2249/3051 [17:46<06:22,  2.10it/s, est. speed input: 69115.15 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2250/3051 [17:46<06:20,  2.10it/s, est. speed input: 69115.26 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2251/3051 [17:47<06:19,  2.11it/s, est. speed input: 69115.34 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2252/3051 [17:47<06:19,  2.11it/s, est. speed input: 69115.28 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2253/3051 [17:48<06:18,  2.11it/s, est. speed input: 69115.38 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2254/3051 [17:48<06:17,  2.11it/s, est. speed input: 69115.46 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2255/3051 [17:49<06:16,  2.11it/s, est. speed input: 69115.54 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2256/3051 [17:49<06:16,  2.11it/s, est. speed input: 69115.67 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2257/3051 [17:50<06:15,  2.12it/s, est. speed input: 69115.80 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2258/3051 [17:50<06:14,  2.12it/s, est. speed input: 69115.88 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2259/3051 [17:50<06:14,  2.12it/s, est. speed input: 69115.99 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2260/3051 [17:51<06:13,  2.12it/s, est. speed input: 69116.08 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2261/3051 [17:51<06:13,  2.12it/s, est. speed input: 69116.20 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2262/3051 [17:52<06:12,  2.12it/s, est. speed input: 69116.32 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2263/3051 [17:52<06:12,  2.12it/s, est. speed input: 69116.43 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2264/3051 [17:53<06:11,  2.12it/s, est. speed input: 69116.51 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2265/3051 [17:53<06:11,  2.12it/s, est. speed input: 69116.62 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2266/3051 [17:54<06:10,  2.12it/s, est. speed input: 69116.74 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2267/3051 [17:54<06:10,  2.12it/s, est. speed input: 69116.81 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2268/3051 [17:55<06:10,  2.12it/s, est. speed input: 69116.88 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2269/3051 [17:55<06:09,  2.12it/s, est. speed input: 69116.99 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2270/3051 [17:56<06:09,  2.11it/s, est. speed input: 69117.04 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2271/3051 [17:56<06:09,  2.11it/s, est. speed input: 69116.98 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2272/3051 [17:57<06:08,  2.11it/s, est. speed input: 69117.06 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2273/3051 [17:57<06:07,  2.11it/s, est. speed input: 69117.18 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2274/3051 [17:58<06:07,  2.11it/s, est. speed input: 69117.25 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2275/3051 [17:58<06:07,  2.11it/s, est. speed input: 69117.31 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2276/3051 [17:59<06:06,  2.11it/s, est. speed input: 69117.39 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2277/3051 [17:59<06:06,  2.11it/s, est. speed input: 69117.47 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2278/3051 [17:59<06:05,  2.12it/s, est. speed input: 69117.58 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2279/3051 [18:00<06:04,  2.12it/s, est. speed input: 69117.67 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2280/3051 [18:00<06:04,  2.12it/s, est. speed input: 69117.79 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2281/3051 [18:01<06:03,  2.12it/s, est. speed input: 69117.87 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2282/3051 [18:01<06:03,  2.12it/s, est. speed input: 69117.95 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2283/3051 [18:02<06:03,  2.12it/s, est. speed input: 69118.03 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2284/3051 [18:02<06:02,  2.12it/s, est. speed input: 69118.11 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2285/3051 [18:03<06:02,  2.12it/s, est. speed input: 69118.22 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2286/3051 [18:03<06:01,  2.11it/s, est. speed input: 69118.28 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2287/3051 [18:04<06:01,  2.11it/s, est. speed input: 69118.33 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2288/3051 [18:04<06:00,  2.11it/s, est. speed input: 69118.41 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2289/3051 [18:05<06:00,  2.12it/s, est. speed input: 69118.51 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2290/3051 [18:05<05:59,  2.11it/s, est. speed input: 69118.55 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2291/3051 [18:06<05:59,  2.11it/s, est. speed input: 69118.62 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2292/3051 [18:06<05:58,  2.12it/s, est. speed input: 69118.74 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2293/3051 [18:07<05:58,  2.12it/s, est. speed input: 69118.81 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2294/3051 [18:07<05:57,  2.12it/s, est. speed input: 69118.90 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2295/3051 [18:08<05:57,  2.11it/s, est. speed input: 69118.96 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2296/3051 [18:08<05:56,  2.12it/s, est. speed input: 69119.06 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2297/3051 [18:08<05:56,  2.12it/s, est. speed input: 69119.15 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2298/3051 [18:09<05:56,  2.12it/s, est. speed input: 69119.23 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2299/3051 [18:09<05:55,  2.12it/s, est. speed input: 69119.31 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2300/3051 [18:10<05:54,  2.12it/s, est. speed input: 69119.42 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2301/3051 [18:10<05:54,  2.12it/s, est. speed input: 69119.48 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2302/3051 [18:11<05:54,  2.12it/s, est. speed input: 69119.59 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2303/3051 [18:11<05:53,  2.12it/s, est. speed input: 69119.70 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2304/3051 [18:12<05:53,  2.12it/s, est. speed input: 69119.77 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2305/3051 [18:12<05:52,  2.12it/s, est. speed input: 69119.86 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2306/3051 [18:13<05:52,  2.11it/s, est. speed input: 69119.91 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2307/3051 [18:13<05:51,  2.12it/s, est. speed input: 69120.00 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2308/3051 [18:14<05:51,  2.11it/s, est. speed input: 69119.99 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2309/3051 [18:14<05:51,  2.11it/s, est. speed input: 69120.07 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2310/3051 [18:15<05:50,  2.11it/s, est. speed input: 69120.17 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2311/3051 [18:15<05:49,  2.11it/s, est. speed input: 69120.23 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2312/3051 [18:16<05:49,  2.11it/s, est. speed input: 69120.33 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2313/3051 [18:16<05:48,  2.12it/s, est. speed input: 69120.42 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2314/3051 [18:16<05:48,  2.12it/s, est. speed input: 69120.51 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2315/3051 [18:17<05:47,  2.12it/s, est. speed input: 69120.59 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2316/3051 [18:17<05:47,  2.12it/s, est. speed input: 69120.68 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2317/3051 [18:18<05:46,  2.12it/s, est. speed input: 69120.77 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2318/3051 [18:18<05:46,  2.11it/s, est. speed input: 69120.81 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2319/3051 [18:19<05:46,  2.12it/s, est. speed input: 69120.92 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2320/3051 [18:19<05:45,  2.12it/s, est. speed input: 69121.00 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2321/3051 [18:20<05:45,  2.11it/s, est. speed input: 69121.06 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2322/3051 [18:20<05:44,  2.12it/s, est. speed input: 69121.15 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2323/3051 [18:21<05:44,  2.11it/s, est. speed input: 69121.06 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2324/3051 [18:21<05:44,  2.11it/s, est. speed input: 69121.13 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2325/3051 [18:22<05:43,  2.11it/s, est. speed input: 69121.20 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2326/3051 [18:22<05:42,  2.11it/s, est. speed input: 69121.30 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2327/3051 [18:23<05:42,  2.11it/s, est. speed input: 69121.34 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2328/3051 [18:23<05:41,  2.11it/s, est. speed input: 69121.46 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2329/3051 [18:24<05:41,  2.11it/s, est. speed input: 69121.53 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2330/3051 [18:24<05:41,  2.11it/s, est. speed input: 69121.58 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2331/3051 [18:25<05:40,  2.11it/s, est. speed input: 69121.66 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2332/3051 [18:25<05:39,  2.12it/s, est. speed input: 69121.78 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2333/3051 [18:25<05:39,  2.11it/s, est. speed input: 69121.81 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2334/3051 [18:26<05:39,  2.11it/s, est. speed input: 69121.84 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2335/3051 [18:26<05:38,  2.11it/s, est. speed input: 69121.95 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2336/3051 [18:27<05:38,  2.11it/s, est. speed input: 69121.98 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2337/3051 [18:27<05:37,  2.11it/s, est. speed input: 69122.06 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2338/3051 [18:28<05:37,  2.11it/s, est. speed input: 69122.12 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2339/3051 [18:28<05:36,  2.11it/s, est. speed input: 69122.21 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2340/3051 [18:29<05:36,  2.11it/s, est. speed input: 69122.20 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2341/3051 [18:29<05:36,  2.11it/s, est. speed input: 69122.23 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2342/3051 [18:30<05:35,  2.11it/s, est. speed input: 69122.33 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2343/3051 [18:30<05:34,  2.11it/s, est. speed input: 69122.42 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2344/3051 [18:31<05:34,  2.11it/s, est. speed input: 69122.43 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2345/3051 [18:31<05:37,  2.09it/s, est. speed input: 69121.39 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2346/3051 [18:32<05:42,  2.06it/s, est. speed input: 69119.65 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2347/3051 [18:32<05:39,  2.07it/s, est. speed input: 69119.68 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2348/3051 [18:33<05:36,  2.09it/s, est. speed input: 69119.75 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2349/3051 [18:33<05:35,  2.09it/s, est. speed input: 69119.82 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2350/3051 [18:34<05:33,  2.10it/s, est. speed input: 69119.92 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2351/3051 [18:34<05:32,  2.11it/s, est. speed input: 69120.00 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2352/3051 [18:35<05:31,  2.11it/s, est. speed input: 69120.06 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2353/3051 [18:35<05:30,  2.11it/s, est. speed input: 69120.12 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2354/3051 [18:35<05:30,  2.11it/s, est. speed input: 69120.23 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2355/3051 [18:36<05:29,  2.11it/s, est. speed input: 69120.34 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2356/3051 [18:36<05:29,  2.11it/s, est. speed input: 69120.33 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2357/3051 [18:37<05:28,  2.11it/s, est. speed input: 69120.39 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2358/3051 [18:37<05:27,  2.11it/s, est. speed input: 69120.49 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2359/3051 [18:38<05:27,  2.11it/s, est. speed input: 69120.57 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2360/3051 [18:38<05:26,  2.11it/s, est. speed input: 69120.62 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2361/3051 [18:39<05:26,  2.11it/s, est. speed input: 69120.69 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2362/3051 [18:39<05:25,  2.12it/s, est. speed input: 69120.87 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2363/3051 [18:40<05:25,  2.12it/s, est. speed input: 69120.91 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2364/3051 [18:40<05:24,  2.11it/s, est. speed input: 69120.96 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2365/3051 [18:41<05:24,  2.11it/s, est. speed input: 69120.99 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2366/3051 [18:41<05:23,  2.12it/s, est. speed input: 69121.12 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2367/3051 [18:42<05:23,  2.12it/s, est. speed input: 69121.22 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2368/3051 [18:42<05:23,  2.11it/s, est. speed input: 69121.13 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2369/3051 [18:43<05:22,  2.11it/s, est. speed input: 69121.19 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2370/3051 [18:43<05:22,  2.11it/s, est. speed input: 69121.28 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2371/3051 [18:44<05:21,  2.11it/s, est. speed input: 69121.40 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2372/3051 [18:44<05:21,  2.11it/s, est. speed input: 69121.44 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2373/3051 [18:44<05:20,  2.11it/s, est. speed input: 69121.48 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2374/3051 [18:45<05:20,  2.11it/s, est. speed input: 69121.54 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2375/3051 [18:45<05:20,  2.11it/s, est. speed input: 69121.52 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2376/3051 [18:46<05:19,  2.11it/s, est. speed input: 69121.63 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2377/3051 [18:46<05:19,  2.11it/s, est. speed input: 69121.59 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2378/3051 [18:47<05:18,  2.11it/s, est. speed input: 69121.63 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2379/3051 [18:47<05:18,  2.11it/s, est. speed input: 69121.67 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2380/3051 [18:48<05:17,  2.11it/s, est. speed input: 69121.74 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2381/3051 [18:48<05:16,  2.11it/s, est. speed input: 69121.84 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2382/3051 [18:49<05:16,  2.12it/s, est. speed input: 69121.97 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2383/3051 [18:49<05:15,  2.11it/s, est. speed input: 69122.00 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2384/3051 [18:50<05:15,  2.11it/s, est. speed input: 69122.09 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2385/3051 [18:50<05:14,  2.12it/s, est. speed input: 69122.21 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2386/3051 [18:51<05:14,  2.12it/s, est. speed input: 69122.29 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2387/3051 [18:51<05:14,  2.11it/s, est. speed input: 69122.33 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2388/3051 [18:52<05:13,  2.11it/s, est. speed input: 69122.33 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2389/3051 [18:52<05:13,  2.11it/s, est. speed input: 69122.41 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2390/3051 [18:52<05:12,  2.11it/s, est. speed input: 69122.48 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2391/3051 [18:53<05:12,  2.11it/s, est. speed input: 69122.54 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2392/3051 [18:53<05:11,  2.11it/s, est. speed input: 69122.61 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2393/3051 [18:54<05:11,  2.11it/s, est. speed input: 69122.71 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2394/3051 [18:54<05:10,  2.11it/s, est. speed input: 69122.75 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2395/3051 [18:55<05:10,  2.11it/s, est. speed input: 69122.74 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2396/3051 [18:55<05:09,  2.11it/s, est. speed input: 69122.81 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2397/3051 [18:56<05:09,  2.11it/s, est. speed input: 69122.87 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2398/3051 [18:56<05:08,  2.11it/s, est. speed input: 69122.97 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2399/3051 [18:57<05:08,  2.12it/s, est. speed input: 69123.07 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2400/3051 [18:57<05:08,  2.11it/s, est. speed input: 69122.88 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2401/3051 [18:58<05:07,  2.11it/s, est. speed input: 69122.95 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2402/3051 [18:58<05:07,  2.11it/s, est. speed input: 69123.06 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2403/3051 [18:59<05:06,  2.11it/s, est. speed input: 69123.16 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2404/3051 [18:59<05:06,  2.11it/s, est. speed input: 69123.22 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2405/3051 [19:00<05:05,  2.11it/s, est. speed input: 69123.31 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2406/3051 [19:00<05:05,  2.11it/s, est. speed input: 69123.37 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2407/3051 [19:01<05:04,  2.11it/s, est. speed input: 69123.45 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2408/3051 [19:01<05:03,  2.12it/s, est. speed input: 69123.55 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2409/3051 [19:01<05:03,  2.12it/s, est. speed input: 69123.65 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2410/3051 [19:02<05:03,  2.11it/s, est. speed input: 69123.63 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2411/3051 [19:02<05:02,  2.11it/s, est. speed input: 69123.71 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2412/3051 [19:03<05:02,  2.11it/s, est. speed input: 69123.80 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2413/3051 [19:03<05:01,  2.12it/s, est. speed input: 69123.89 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2414/3051 [19:04<05:01,  2.12it/s, est. speed input: 69123.99 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2415/3051 [19:04<05:00,  2.12it/s, est. speed input: 69124.06 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2416/3051 [19:05<05:00,  2.11it/s, est. speed input: 69124.12 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2417/3051 [19:05<04:59,  2.12it/s, est. speed input: 69124.21 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2418/3051 [19:06<04:59,  2.12it/s, est. speed input: 69124.28 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2419/3051 [19:06<04:58,  2.11it/s, est. speed input: 69124.33 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2420/3051 [19:07<04:58,  2.12it/s, est. speed input: 69124.43 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2421/3051 [19:07<04:57,  2.12it/s, est. speed input: 69124.53 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2422/3051 [19:08<04:57,  2.12it/s, est. speed input: 69124.60 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2423/3051 [19:08<04:56,  2.11it/s, est. speed input: 69124.64 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2424/3051 [19:09<04:56,  2.12it/s, est. speed input: 69124.76 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2425/3051 [19:09<04:55,  2.12it/s, est. speed input: 69124.82 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2426/3051 [19:10<04:55,  2.12it/s, est. speed input: 69124.91 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2427/3051 [19:10<04:55,  2.11it/s, est. speed input: 69124.96 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2428/3051 [19:10<04:54,  2.12it/s, est. speed input: 69125.07 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2429/3051 [19:11<04:54,  2.12it/s, est. speed input: 69125.15 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2430/3051 [19:11<04:53,  2.12it/s, est. speed input: 69125.21 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2431/3051 [19:12<04:53,  2.12it/s, est. speed input: 69125.31 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2432/3051 [19:12<04:52,  2.11it/s, est. speed input: 69125.27 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2433/3051 [19:13<04:52,  2.11it/s, est. speed input: 69125.33 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2434/3051 [19:13<04:51,  2.11it/s, est. speed input: 69125.42 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2435/3051 [19:14<04:51,  2.11it/s, est. speed input: 69125.52 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2436/3051 [19:14<04:50,  2.11it/s, est. speed input: 69125.58 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2437/3051 [19:15<04:50,  2.12it/s, est. speed input: 69125.69 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2438/3051 [19:15<04:49,  2.12it/s, est. speed input: 69125.75 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2439/3051 [19:16<04:49,  2.11it/s, est. speed input: 69125.80 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2440/3051 [19:16<04:49,  2.11it/s, est. speed input: 69125.80 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2441/3051 [19:17<04:48,  2.11it/s, est. speed input: 69125.86 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2442/3051 [19:17<04:48,  2.11it/s, est. speed input: 69125.93 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2443/3051 [19:18<04:48,  2.11it/s, est. speed input: 69125.77 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2444/3051 [19:18<04:47,  2.11it/s, est. speed input: 69125.84 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2445/3051 [19:19<04:46,  2.11it/s, est. speed input: 69125.94 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2446/3051 [19:19<04:46,  2.11it/s, est. speed input: 69125.99 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2447/3051 [19:19<04:45,  2.11it/s, est. speed input: 69126.07 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2448/3051 [19:20<04:45,  2.11it/s, est. speed input: 69126.18 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2449/3051 [19:20<04:44,  2.12it/s, est. speed input: 69126.26 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2450/3051 [19:21<04:44,  2.12it/s, est. speed input: 69126.33 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2451/3051 [19:21<04:43,  2.12it/s, est. speed input: 69126.41 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2452/3051 [19:22<04:43,  2.12it/s, est. speed input: 69126.52 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2453/3051 [19:22<04:42,  2.12it/s, est. speed input: 69126.60 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2454/3051 [19:23<04:42,  2.12it/s, est. speed input: 69126.66 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2455/3051 [19:23<04:41,  2.11it/s, est. speed input: 69126.71 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2456/3051 [19:24<04:41,  2.11it/s, est. speed input: 69126.77 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2457/3051 [19:24<04:40,  2.11it/s, est. speed input: 69126.83 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2458/3051 [19:25<04:40,  2.11it/s, est. speed input: 69126.75 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2459/3051 [19:25<04:40,  2.11it/s, est. speed input: 69126.82 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2460/3051 [19:26<04:39,  2.11it/s, est. speed input: 69126.89 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2461/3051 [19:26<04:39,  2.11it/s, est. speed input: 69126.95 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2462/3051 [19:27<04:38,  2.11it/s, est. speed input: 69126.99 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2463/3051 [19:27<04:38,  2.11it/s, est. speed input: 69127.09 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2464/3051 [19:27<04:37,  2.11it/s, est. speed input: 69127.16 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2465/3051 [19:28<04:37,  2.11it/s, est. speed input: 69127.23 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2466/3051 [19:28<04:36,  2.12it/s, est. speed input: 69127.36 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2467/3051 [19:29<04:35,  2.12it/s, est. speed input: 69127.46 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2468/3051 [19:29<04:35,  2.11it/s, est. speed input: 69127.42 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2469/3051 [19:30<04:35,  2.11it/s, est. speed input: 69127.47 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2470/3051 [19:30<04:34,  2.11it/s, est. speed input: 69127.52 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2471/3051 [19:31<04:34,  2.11it/s, est. speed input: 69127.44 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2472/3051 [19:31<04:34,  2.11it/s, est. speed input: 69127.44 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2473/3051 [19:32<04:33,  2.11it/s, est. speed input: 69127.50 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2474/3051 [19:32<04:33,  2.11it/s, est. speed input: 69127.59 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2475/3051 [19:33<04:32,  2.11it/s, est. speed input: 69127.64 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2476/3051 [19:33<04:31,  2.11it/s, est. speed input: 69127.74 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2477/3051 [19:34<04:31,  2.11it/s, est. speed input: 69127.79 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2478/3051 [19:34<04:31,  2.11it/s, est. speed input: 69127.76 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2479/3051 [19:35<04:30,  2.11it/s, est. speed input: 69127.83 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2480/3051 [19:35<04:30,  2.11it/s, est. speed input: 69127.94 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2481/3051 [19:36<04:29,  2.11it/s, est. speed input: 69127.99 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2482/3051 [19:36<04:29,  2.11it/s, est. speed input: 69128.05 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2483/3051 [19:36<04:28,  2.11it/s, est. speed input: 69128.14 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2484/3051 [19:37<04:28,  2.11it/s, est. speed input: 69128.18 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2485/3051 [19:37<04:27,  2.11it/s, est. speed input: 69128.27 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2486/3051 [19:38<04:27,  2.11it/s, est. speed input: 69128.32 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2487/3051 [19:38<04:27,  2.11it/s, est. speed input: 69128.24 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2488/3051 [19:39<04:26,  2.11it/s, est. speed input: 69128.23 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2489/3051 [19:39<04:26,  2.11it/s, est. speed input: 69128.33 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2490/3051 [19:40<04:25,  2.11it/s, est. speed input: 69128.39 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2491/3051 [19:40<04:24,  2.11it/s, est. speed input: 69128.45 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2492/3051 [19:41<04:24,  2.11it/s, est. speed input: 69128.34 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2493/3051 [19:41<04:24,  2.11it/s, est. speed input: 69128.37 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2494/3051 [19:42<04:23,  2.11it/s, est. speed input: 69128.50 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2495/3051 [19:42<04:23,  2.11it/s, est. speed input: 69128.57 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2496/3051 [19:43<04:22,  2.11it/s, est. speed input: 69128.66 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2497/3051 [19:43<04:22,  2.11it/s, est. speed input: 69128.64 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2498/3051 [19:44<04:21,  2.11it/s, est. speed input: 69128.74 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2499/3051 [19:44<04:21,  2.11it/s, est. speed input: 69128.83 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2500/3051 [19:45<04:20,  2.11it/s, est. speed input: 69128.78 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2501/3051 [19:45<04:20,  2.11it/s, est. speed input: 69128.87 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2502/3051 [19:45<04:19,  2.11it/s, est. speed input: 69128.92 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2503/3051 [19:46<04:19,  2.11it/s, est. speed input: 69128.97 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2504/3051 [19:46<04:18,  2.11it/s, est. speed input: 69129.06 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2505/3051 [19:47<04:18,  2.12it/s, est. speed input: 69129.16 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2506/3051 [19:47<04:17,  2.12it/s, est. speed input: 69129.22 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2507/3051 [19:48<04:18,  2.11it/s, est. speed input: 69128.92 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2508/3051 [19:48<04:17,  2.11it/s, est. speed input: 69128.98 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2509/3051 [19:49<04:16,  2.11it/s, est. speed input: 69129.09 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2510/3051 [19:49<04:16,  2.11it/s, est. speed input: 69129.15 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2511/3051 [19:50<04:15,  2.11it/s, est. speed input: 69129.23 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2512/3051 [19:50<04:14,  2.11it/s, est. speed input: 69129.30 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2513/3051 [19:51<04:14,  2.12it/s, est. speed input: 69129.45 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2514/3051 [19:51<04:13,  2.12it/s, est. speed input: 69129.54 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2515/3051 [19:52<04:13,  2.12it/s, est. speed input: 69129.58 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2516/3051 [19:52<04:12,  2.12it/s, est. speed input: 69129.65 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2517/3051 [19:53<04:12,  2.12it/s, est. speed input: 69129.72 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2518/3051 [19:53<04:12,  2.11it/s, est. speed input: 69129.77 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2519/3051 [19:54<04:11,  2.11it/s, est. speed input: 69129.84 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2520/3051 [19:54<04:10,  2.12it/s, est. speed input: 69129.95 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2521/3051 [19:54<04:10,  2.12it/s, est. speed input: 69130.04 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2522/3051 [19:55<04:09,  2.12it/s, est. speed input: 69130.14 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2523/3051 [19:55<04:09,  2.12it/s, est. speed input: 69130.22 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2524/3051 [19:56<04:09,  2.12it/s, est. speed input: 69130.28 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2525/3051 [19:56<04:08,  2.12it/s, est. speed input: 69130.34 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2526/3051 [19:57<04:08,  2.12it/s, est. speed input: 69130.42 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2527/3051 [19:57<04:07,  2.12it/s, est. speed input: 69130.49 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2528/3051 [19:58<04:07,  2.12it/s, est. speed input: 69130.55 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2529/3051 [19:58<04:06,  2.12it/s, est. speed input: 69130.64 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2530/3051 [19:59<04:06,  2.11it/s, est. speed input: 69130.69 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2531/3051 [19:59<04:05,  2.12it/s, est. speed input: 69130.80 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2532/3051 [20:00<04:05,  2.11it/s, est. speed input: 69130.75 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2533/3051 [20:00<04:04,  2.11it/s, est. speed input: 69130.85 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2534/3051 [20:01<04:04,  2.11it/s, est. speed input: 69130.90 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2535/3051 [20:01<04:04,  2.11it/s, est. speed input: 69130.98 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2536/3051 [20:02<04:03,  2.12it/s, est. speed input: 69131.09 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2537/3051 [20:02<04:02,  2.12it/s, est. speed input: 69131.15 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2538/3051 [20:03<04:02,  2.11it/s, est. speed input: 69131.20 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2539/3051 [20:03<04:02,  2.11it/s, est. speed input: 69131.24 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2540/3051 [20:03<04:01,  2.11it/s, est. speed input: 69131.25 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2541/3051 [20:04<04:01,  2.11it/s, est. speed input: 69131.36 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2542/3051 [20:04<04:00,  2.11it/s, est. speed input: 69131.35 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2543/3051 [20:05<04:00,  2.11it/s, est. speed input: 69131.44 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2544/3051 [20:05<03:59,  2.11it/s, est. speed input: 69131.53 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2545/3051 [20:06<03:59,  2.11it/s, est. speed input: 69131.52 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2546/3051 [20:06<03:58,  2.11it/s, est. speed input: 69131.62 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2547/3051 [20:07<03:58,  2.11it/s, est. speed input: 69131.69 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2548/3051 [20:07<03:57,  2.12it/s, est. speed input: 69131.79 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2549/3051 [20:08<03:57,  2.11it/s, est. speed input: 69131.81 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2550/3051 [20:08<03:56,  2.11it/s, est. speed input: 69131.86 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2551/3051 [20:09<03:56,  2.11it/s, est. speed input: 69131.91 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2552/3051 [20:09<03:56,  2.11it/s, est. speed input: 69131.97 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2553/3051 [20:10<03:55,  2.11it/s, est. speed input: 69132.02 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2554/3051 [20:10<03:54,  2.12it/s, est. speed input: 69132.12 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2555/3051 [20:11<03:54,  2.11it/s, est. speed input: 69132.11 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2556/3051 [20:11<03:54,  2.11it/s, est. speed input: 69132.19 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2557/3051 [20:11<03:53,  2.11it/s, est. speed input: 69132.20 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2558/3051 [20:12<03:53,  2.11it/s, est. speed input: 69132.28 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2559/3051 [20:12<03:52,  2.11it/s, est. speed input: 69132.36 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2560/3051 [20:13<03:52,  2.11it/s, est. speed input: 69132.44 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2561/3051 [20:13<03:51,  2.11it/s, est. speed input: 69132.50 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2562/3051 [20:14<03:51,  2.12it/s, est. speed input: 69132.58 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2563/3051 [20:14<03:50,  2.12it/s, est. speed input: 69132.66 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2564/3051 [20:15<03:50,  2.12it/s, est. speed input: 69132.75 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2565/3051 [20:15<03:49,  2.11it/s, est. speed input: 69132.75 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2566/3051 [20:16<03:49,  2.11it/s, est. speed input: 69132.82 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2567/3051 [20:16<03:48,  2.11it/s, est. speed input: 69132.89 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2568/3051 [20:17<03:48,  2.11it/s, est. speed input: 69132.94 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2569/3051 [20:17<03:47,  2.12it/s, est. speed input: 69133.05 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2570/3051 [20:18<03:47,  2.11it/s, est. speed input: 69133.09 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2571/3051 [20:18<03:46,  2.11it/s, est. speed input: 69133.15 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2572/3051 [20:19<03:46,  2.12it/s, est. speed input: 69133.24 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2573/3051 [20:19<03:45,  2.12it/s, est. speed input: 69133.31 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2574/3051 [20:20<03:45,  2.11it/s, est. speed input: 69133.33 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2575/3051 [20:20<03:45,  2.11it/s, est. speed input: 69133.26 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2576/3051 [20:20<03:44,  2.11it/s, est. speed input: 69133.34 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2577/3051 [20:21<03:44,  2.11it/s, est. speed input: 69133.42 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2578/3051 [20:21<03:43,  2.11it/s, est. speed input: 69133.49 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2579/3051 [20:22<03:43,  2.11it/s, est. speed input: 69133.57 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2580/3051 [20:22<03:42,  2.12it/s, est. speed input: 69133.66 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2581/3051 [20:23<03:42,  2.12it/s, est. speed input: 69133.72 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2582/3051 [20:23<03:41,  2.11it/s, est. speed input: 69133.77 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2583/3051 [20:24<03:41,  2.12it/s, est. speed input: 69133.88 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2584/3051 [20:24<03:40,  2.12it/s, est. speed input: 69133.94 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2585/3051 [20:25<03:40,  2.11it/s, est. speed input: 69133.91 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2586/3051 [20:25<03:39,  2.11it/s, est. speed input: 69134.00 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2587/3051 [20:26<03:39,  2.12it/s, est. speed input: 69134.09 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2588/3051 [20:26<03:38,  2.12it/s, est. speed input: 69134.21 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2589/3051 [20:27<03:38,  2.12it/s, est. speed input: 69134.26 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2590/3051 [20:27<03:37,  2.12it/s, est. speed input: 69134.32 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2591/3051 [20:28<03:37,  2.12it/s, est. speed input: 69134.42 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2592/3051 [20:28<03:36,  2.12it/s, est. speed input: 69134.47 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2593/3051 [20:29<03:36,  2.11it/s, est. speed input: 69134.51 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2594/3051 [20:29<03:36,  2.11it/s, est. speed input: 69134.55 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2595/3051 [20:29<03:35,  2.12it/s, est. speed input: 69134.66 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2596/3051 [20:30<03:35,  2.11it/s, est. speed input: 69134.71 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2597/3051 [20:30<03:35,  2.11it/s, est. speed input: 69134.59 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2598/3051 [20:31<03:34,  2.11it/s, est. speed input: 69134.69 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2599/3051 [20:31<03:33,  2.11it/s, est. speed input: 69134.79 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2600/3051 [20:32<03:33,  2.11it/s, est. speed input: 69134.74 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2601/3051 [20:32<03:32,  2.11it/s, est. speed input: 69134.81 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2602/3051 [20:33<03:32,  2.11it/s, est. speed input: 69134.88 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2603/3051 [20:33<03:31,  2.11it/s, est. speed input: 69134.95 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2604/3051 [20:34<03:31,  2.11it/s, est. speed input: 69135.03 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2605/3051 [20:34<03:30,  2.11it/s, est. speed input: 69135.09 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2606/3051 [20:35<03:30,  2.11it/s, est. speed input: 69135.14 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2607/3051 [20:35<03:29,  2.12it/s, est. speed input: 69135.25 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2608/3051 [20:36<03:29,  2.12it/s, est. speed input: 69135.35 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2609/3051 [20:36<03:28,  2.12it/s, est. speed input: 69135.42 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2610/3051 [20:37<03:28,  2.11it/s, est. speed input: 69135.43 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2611/3051 [20:37<03:28,  2.12it/s, est. speed input: 69135.51 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2612/3051 [20:38<03:27,  2.11it/s, est. speed input: 69135.57 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2613/3051 [20:38<03:27,  2.12it/s, est. speed input: 69135.66 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2614/3051 [20:38<03:26,  2.11it/s, est. speed input: 69135.70 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2615/3051 [20:39<03:26,  2.12it/s, est. speed input: 69135.79 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2616/3051 [20:39<03:25,  2.12it/s, est. speed input: 69135.86 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2617/3051 [20:40<03:25,  2.12it/s, est. speed input: 69135.94 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2618/3051 [20:40<03:24,  2.12it/s, est. speed input: 69136.00 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2619/3051 [20:41<03:24,  2.12it/s, est. speed input: 69136.08 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2620/3051 [20:41<03:23,  2.11it/s, est. speed input: 69136.06 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2621/3051 [20:42<03:23,  2.11it/s, est. speed input: 69136.10 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2622/3051 [20:42<03:22,  2.11it/s, est. speed input: 69136.20 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2623/3051 [20:43<03:22,  2.11it/s, est. speed input: 69136.26 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2624/3051 [20:43<03:22,  2.11it/s, est. speed input: 69136.00 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2625/3051 [20:44<03:23,  2.09it/s, est. speed input: 69135.43 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2626/3051 [20:44<03:25,  2.07it/s, est. speed input: 69134.30 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2627/3051 [20:45<03:23,  2.08it/s, est. speed input: 69134.13 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2628/3051 [20:45<03:22,  2.09it/s, est. speed input: 69134.21 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2629/3051 [20:46<03:21,  2.10it/s, est. speed input: 69134.29 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2630/3051 [20:46<03:20,  2.10it/s, est. speed input: 69134.39 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2631/3051 [20:47<03:19,  2.11it/s, est. speed input: 69134.47 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2632/3051 [20:47<03:18,  2.11it/s, est. speed input: 69134.56 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2633/3051 [20:47<03:18,  2.11it/s, est. speed input: 69134.59 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2634/3051 [20:48<03:17,  2.11it/s, est. speed input: 69134.67 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2635/3051 [20:48<03:16,  2.11it/s, est. speed input: 69134.75 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2636/3051 [20:49<03:16,  2.11it/s, est. speed input: 69134.82 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2637/3051 [20:49<03:15,  2.11it/s, est. speed input: 69134.89 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2638/3051 [20:50<03:15,  2.12it/s, est. speed input: 69134.98 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2639/3051 [20:50<03:14,  2.12it/s, est. speed input: 69135.06 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2640/3051 [20:51<03:14,  2.12it/s, est. speed input: 69135.10 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2641/3051 [20:51<03:13,  2.12it/s, est. speed input: 69135.19 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2642/3051 [20:52<03:13,  2.11it/s, est. speed input: 69135.19 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2643/3051 [20:52<03:13,  2.11it/s, est. speed input: 69135.24 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2644/3051 [20:53<03:12,  2.11it/s, est. speed input: 69135.31 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2645/3051 [20:53<03:12,  2.11it/s, est. speed input: 69135.27 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2646/3051 [20:54<03:11,  2.11it/s, est. speed input: 69135.37 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2647/3051 [20:54<03:10,  2.12it/s, est. speed input: 69135.49 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2648/3051 [20:55<03:10,  2.12it/s, est. speed input: 69135.58 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2649/3051 [20:55<03:09,  2.12it/s, est. speed input: 69135.68 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2650/3051 [20:56<03:09,  2.12it/s, est. speed input: 69135.76 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2651/3051 [20:56<03:09,  2.12it/s, est. speed input: 69135.78 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2652/3051 [20:56<03:09,  2.11it/s, est. speed input: 69135.58 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2653/3051 [20:57<03:08,  2.11it/s, est. speed input: 69135.70 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2654/3051 [20:57<03:07,  2.11it/s, est. speed input: 69135.77 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2655/3051 [20:58<03:07,  2.11it/s, est. speed input: 69135.76 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2656/3051 [20:58<03:06,  2.11it/s, est. speed input: 69135.82 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2657/3051 [20:59<03:06,  2.11it/s, est. speed input: 69135.92 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2658/3051 [20:59<03:05,  2.11it/s, est. speed input: 69136.00 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2659/3051 [21:00<03:05,  2.12it/s, est. speed input: 69136.08 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2660/3051 [21:00<03:04,  2.12it/s, est. speed input: 69136.13 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2661/3051 [21:01<03:04,  2.11it/s, est. speed input: 69136.18 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2662/3051 [21:01<03:03,  2.11it/s, est. speed input: 69136.23 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2663/3051 [21:02<03:03,  2.12it/s, est. speed input: 69136.32 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2664/3051 [21:02<03:02,  2.11it/s, est. speed input: 69136.37 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2665/3051 [21:03<03:02,  2.12it/s, est. speed input: 69136.47 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2666/3051 [21:03<03:02,  2.11it/s, est. speed input: 69136.50 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2667/3051 [21:04<03:01,  2.12it/s, est. speed input: 69136.58 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2668/3051 [21:04<03:01,  2.11it/s, est. speed input: 69136.62 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2669/3051 [21:04<03:00,  2.12it/s, est. speed input: 69136.73 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2670/3051 [21:05<03:00,  2.12it/s, est. speed input: 69136.81 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2671/3051 [21:05<02:59,  2.12it/s, est. speed input: 69136.86 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2672/3051 [21:06<02:59,  2.12it/s, est. speed input: 69136.92 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2673/3051 [21:06<02:58,  2.12it/s, est. speed input: 69136.98 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2674/3051 [21:07<02:58,  2.12it/s, est. speed input: 69137.04 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2675/3051 [21:07<02:57,  2.11it/s, est. speed input: 69137.10 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2676/3051 [21:08<02:57,  2.12it/s, est. speed input: 69137.18 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2677/3051 [21:08<02:56,  2.11it/s, est. speed input: 69137.20 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2678/3051 [21:09<02:56,  2.11it/s, est. speed input: 69137.23 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2679/3051 [21:09<02:56,  2.11it/s, est. speed input: 69137.26 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2680/3051 [21:10<02:55,  2.11it/s, est. speed input: 69137.32 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2681/3051 [21:10<02:54,  2.12it/s, est. speed input: 69137.43 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2682/3051 [21:11<02:54,  2.11it/s, est. speed input: 69137.39 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2683/3051 [21:11<02:54,  2.11it/s, est. speed input: 69137.45 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2684/3051 [21:12<02:53,  2.11it/s, est. speed input: 69137.52 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2685/3051 [21:12<02:53,  2.11it/s, est. speed input: 69137.59 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2686/3051 [21:13<02:52,  2.11it/s, est. speed input: 69137.65 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2687/3051 [21:13<02:52,  2.12it/s, est. speed input: 69137.74 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2688/3051 [21:13<02:51,  2.11it/s, est. speed input: 69137.76 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2689/3051 [21:14<02:51,  2.12it/s, est. speed input: 69137.85 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2690/3051 [21:14<02:50,  2.11it/s, est. speed input: 69137.85 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2691/3051 [21:15<02:50,  2.11it/s, est. speed input: 69137.91 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2692/3051 [21:15<02:49,  2.11it/s, est. speed input: 69137.95 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2693/3051 [21:16<02:49,  2.11it/s, est. speed input: 69138.01 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2694/3051 [21:16<02:48,  2.11it/s, est. speed input: 69138.05 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2695/3051 [21:17<02:48,  2.11it/s, est. speed input: 69138.12 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2696/3051 [21:17<02:47,  2.11it/s, est. speed input: 69138.19 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2697/3051 [21:18<02:47,  2.12it/s, est. speed input: 69138.26 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2698/3051 [21:18<02:46,  2.12it/s, est. speed input: 69138.35 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2699/3051 [21:19<02:46,  2.12it/s, est. speed input: 69138.40 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2700/3051 [21:19<02:45,  2.11it/s, est. speed input: 69138.45 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2701/3051 [21:20<02:45,  2.12it/s, est. speed input: 69138.52 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2702/3051 [21:20<02:44,  2.12it/s, est. speed input: 69138.61 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2703/3051 [21:21<02:44,  2.12it/s, est. speed input: 69138.69 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2704/3051 [21:21<02:43,  2.12it/s, est. speed input: 69138.77 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2705/3051 [21:22<02:43,  2.12it/s, est. speed input: 69138.82 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2706/3051 [21:22<02:43,  2.12it/s, est. speed input: 69138.89 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2707/3051 [21:22<02:42,  2.11it/s, est. speed input: 69138.79 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2708/3051 [21:23<02:42,  2.11it/s, est. speed input: 69138.84 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2709/3051 [21:23<02:41,  2.11it/s, est. speed input: 69138.93 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2710/3051 [21:24<02:41,  2.11it/s, est. speed input: 69138.95 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2711/3051 [21:24<02:40,  2.11it/s, est. speed input: 69139.01 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2712/3051 [21:25<02:40,  2.11it/s, est. speed input: 69139.05 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2713/3051 [21:25<02:39,  2.11it/s, est. speed input: 69139.14 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2714/3051 [21:26<02:39,  2.12it/s, est. speed input: 69139.21 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2715/3051 [21:26<02:38,  2.11it/s, est. speed input: 69139.25 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2716/3051 [21:27<02:38,  2.11it/s, est. speed input: 69139.20 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2717/3051 [21:27<02:38,  2.11it/s, est. speed input: 69139.28 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2718/3051 [21:28<02:37,  2.11it/s, est. speed input: 69139.35 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2719/3051 [21:28<02:36,  2.12it/s, est. speed input: 69139.45 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2720/3051 [21:29<02:36,  2.11it/s, est. speed input: 69139.48 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2721/3051 [21:29<02:36,  2.11it/s, est. speed input: 69139.55 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2722/3051 [21:30<02:35,  2.11it/s, est. speed input: 69139.60 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2723/3051 [21:30<02:35,  2.11it/s, est. speed input: 69139.64 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2724/3051 [21:31<02:34,  2.11it/s, est. speed input: 69139.71 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2725/3051 [21:31<02:35,  2.09it/s, est. speed input: 69138.85 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2726/3051 [21:31<02:34,  2.10it/s, est. speed input: 69138.92 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2727/3051 [21:32<02:33,  2.10it/s, est. speed input: 69138.99 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2728/3051 [21:32<02:33,  2.11it/s, est. speed input: 69139.06 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2729/3051 [21:33<02:32,  2.11it/s, est. speed input: 69139.10 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2730/3051 [21:33<02:32,  2.11it/s, est. speed input: 69139.12 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2731/3051 [21:34<02:31,  2.11it/s, est. speed input: 69139.19 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2732/3051 [21:34<02:30,  2.11it/s, est. speed input: 69139.26 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2733/3051 [21:35<02:30,  2.11it/s, est. speed input: 69139.29 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2734/3051 [21:35<02:29,  2.11it/s, est. speed input: 69139.36 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2735/3051 [21:36<02:29,  2.12it/s, est. speed input: 69139.47 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2736/3051 [21:36<02:28,  2.11it/s, est. speed input: 69139.51 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2737/3051 [21:37<02:28,  2.12it/s, est. speed input: 69139.59 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2738/3051 [21:37<02:27,  2.12it/s, est. speed input: 69139.67 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2739/3051 [21:38<02:27,  2.12it/s, est. speed input: 69139.74 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2740/3051 [21:38<02:27,  2.12it/s, est. speed input: 69139.78 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2741/3051 [21:39<02:26,  2.11it/s, est. speed input: 69139.83 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2742/3051 [21:39<02:26,  2.11it/s, est. speed input: 69139.82 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2743/3051 [21:40<02:25,  2.11it/s, est. speed input: 69139.88 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2744/3051 [21:40<02:25,  2.11it/s, est. speed input: 69139.95 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2745/3051 [21:40<02:24,  2.11it/s, est. speed input: 69139.94 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2746/3051 [21:41<02:24,  2.11it/s, est. speed input: 69140.01 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2747/3051 [21:41<02:23,  2.11it/s, est. speed input: 69140.06 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2748/3051 [21:42<02:23,  2.11it/s, est. speed input: 69140.15 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2749/3051 [21:42<02:22,  2.12it/s, est. speed input: 69140.22 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2750/3051 [21:43<02:22,  2.12it/s, est. speed input: 69140.31 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2751/3051 [21:43<02:21,  2.12it/s, est. speed input: 69140.38 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2752/3051 [21:44<02:21,  2.11it/s, est. speed input: 69140.32 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2753/3051 [21:44<02:20,  2.11it/s, est. speed input: 69140.40 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2754/3051 [21:45<02:20,  2.11it/s, est. speed input: 69140.47 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2755/3051 [21:45<02:19,  2.12it/s, est. speed input: 69140.58 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2756/3051 [21:46<02:19,  2.12it/s, est. speed input: 69140.62 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2757/3051 [21:46<02:18,  2.12it/s, est. speed input: 69140.72 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2758/3051 [21:47<02:18,  2.12it/s, est. speed input: 69140.77 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2759/3051 [21:47<02:18,  2.11it/s, est. speed input: 69140.78 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2760/3051 [21:48<02:17,  2.11it/s, est. speed input: 69140.82 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2761/3051 [21:48<02:17,  2.11it/s, est. speed input: 69140.85 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2762/3051 [21:48<02:16,  2.11it/s, est. speed input: 69140.91 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2763/3051 [21:49<02:16,  2.11it/s, est. speed input: 69140.96 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2764/3051 [21:49<02:15,  2.11it/s, est. speed input: 69140.96 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2765/3051 [21:50<02:15,  2.11it/s, est. speed input: 69140.91 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2766/3051 [21:50<02:14,  2.11it/s, est. speed input: 69141.01 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2767/3051 [21:51<02:14,  2.11it/s, est. speed input: 69141.09 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2768/3051 [21:51<02:13,  2.12it/s, est. speed input: 69141.17 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2769/3051 [21:52<02:13,  2.11it/s, est. speed input: 69141.19 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2770/3051 [21:52<02:12,  2.11it/s, est. speed input: 69141.23 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2771/3051 [21:53<02:12,  2.11it/s, est. speed input: 69141.27 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2772/3051 [21:53<02:11,  2.12it/s, est. speed input: 69141.37 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2773/3051 [21:54<02:11,  2.12it/s, est. speed input: 69141.47 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2774/3051 [21:54<02:10,  2.12it/s, est. speed input: 69141.54 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2775/3051 [21:55<02:10,  2.11it/s, est. speed input: 69141.50 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2776/3051 [21:55<02:10,  2.11it/s, est. speed input: 69141.56 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2777/3051 [21:56<02:09,  2.11it/s, est. speed input: 69141.64 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2778/3051 [21:56<02:09,  2.12it/s, est. speed input: 69141.72 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2779/3051 [21:57<02:08,  2.12it/s, est. speed input: 69141.81 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2780/3051 [21:57<02:08,  2.12it/s, est. speed input: 69141.86 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2781/3051 [21:57<02:07,  2.11it/s, est. speed input: 69141.83 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2782/3051 [21:58<02:07,  2.11it/s, est. speed input: 69141.91 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2783/3051 [21:58<02:06,  2.12it/s, est. speed input: 69141.99 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2784/3051 [21:59<02:06,  2.11it/s, est. speed input: 69142.04 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2785/3051 [21:59<02:05,  2.12it/s, est. speed input: 69142.14 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2786/3051 [22:00<02:05,  2.12it/s, est. speed input: 69142.19 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2787/3051 [22:00<02:04,  2.11it/s, est. speed input: 69142.13 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2788/3051 [22:01<02:04,  2.11it/s, est. speed input: 69142.17 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2789/3051 [22:01<02:03,  2.11it/s, est. speed input: 69142.23 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2790/3051 [22:02<02:03,  2.11it/s, est. speed input: 69142.32 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2791/3051 [22:02<02:02,  2.12it/s, est. speed input: 69142.42 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2792/3051 [22:03<02:02,  2.12it/s, est. speed input: 69142.50 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2793/3051 [22:03<02:01,  2.12it/s, est. speed input: 69142.55 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2794/3051 [22:04<02:01,  2.12it/s, est. speed input: 69142.61 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2795/3051 [22:04<02:01,  2.12it/s, est. speed input: 69142.67 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2796/3051 [22:05<02:00,  2.11it/s, est. speed input: 69142.72 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2797/3051 [22:05<02:00,  2.11it/s, est. speed input: 69142.47 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2798/3051 [22:06<01:59,  2.11it/s, est. speed input: 69142.54 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2799/3051 [22:06<01:59,  2.11it/s, est. speed input: 69142.61 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2800/3051 [22:06<01:58,  2.11it/s, est. speed input: 69142.65 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2801/3051 [22:07<01:58,  2.11it/s, est. speed input: 69142.69 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2802/3051 [22:07<01:57,  2.11it/s, est. speed input: 69142.75 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2803/3051 [22:08<01:57,  2.11it/s, est. speed input: 69142.83 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2804/3051 [22:08<01:56,  2.11it/s, est. speed input: 69142.87 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2805/3051 [22:09<01:56,  2.12it/s, est. speed input: 69142.96 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2806/3051 [22:09<01:55,  2.12it/s, est. speed input: 69143.06 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2807/3051 [22:10<01:55,  2.12it/s, est. speed input: 69143.13 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2808/3051 [22:10<01:54,  2.12it/s, est. speed input: 69143.17 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2809/3051 [22:11<01:54,  2.12it/s, est. speed input: 69143.24 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2810/3051 [22:11<01:53,  2.12it/s, est. speed input: 69143.32 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2811/3051 [22:12<01:53,  2.12it/s, est. speed input: 69143.39 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2812/3051 [22:12<01:52,  2.12it/s, est. speed input: 69143.46 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2813/3051 [22:13<01:52,  2.12it/s, est. speed input: 69143.58 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2814/3051 [22:13<01:52,  2.12it/s, est. speed input: 69143.58 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2815/3051 [22:14<01:51,  2.11it/s, est. speed input: 69143.62 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2816/3051 [22:14<01:51,  2.11it/s, est. speed input: 69143.68 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2817/3051 [22:15<01:50,  2.11it/s, est. speed input: 69143.62 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2818/3051 [22:15<01:50,  2.11it/s, est. speed input: 69143.66 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2819/3051 [22:15<01:49,  2.11it/s, est. speed input: 69143.66 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2820/3051 [22:16<01:49,  2.11it/s, est. speed input: 69143.57 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2821/3051 [22:16<01:48,  2.11it/s, est. speed input: 69143.62 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2822/3051 [22:17<01:48,  2.11it/s, est. speed input: 69143.70 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2823/3051 [22:17<01:47,  2.11it/s, est. speed input: 69143.76 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2824/3051 [22:18<01:47,  2.11it/s, est. speed input: 69143.83 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2825/3051 [22:18<01:50,  2.05it/s, est. speed input: 69141.38 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2826/3051 [22:19<01:48,  2.07it/s, est. speed input: 69141.52 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2827/3051 [22:19<01:47,  2.09it/s, est. speed input: 69141.61 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2828/3051 [22:20<01:46,  2.10it/s, est. speed input: 69141.70 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2829/3051 [22:20<01:45,  2.10it/s, est. speed input: 69141.73 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2830/3051 [22:21<01:44,  2.11it/s, est. speed input: 69141.82 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2831/3051 [22:21<01:44,  2.10it/s, est. speed input: 69141.71 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2832/3051 [22:22<01:44,  2.11it/s, est. speed input: 69141.70 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2833/3051 [22:22<01:43,  2.11it/s, est. speed input: 69141.75 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2834/3051 [22:23<01:42,  2.11it/s, est. speed input: 69141.82 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2835/3051 [22:23<01:42,  2.11it/s, est. speed input: 69141.89 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2836/3051 [22:24<01:41,  2.11it/s, est. speed input: 69142.00 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2837/3051 [22:24<01:41,  2.11it/s, est. speed input: 69142.06 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2838/3051 [22:24<01:40,  2.11it/s, est. speed input: 69142.04 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2839/3051 [22:25<01:40,  2.11it/s, est. speed input: 69142.00 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2840/3051 [22:25<01:39,  2.11it/s, est. speed input: 69142.12 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2841/3051 [22:26<01:39,  2.11it/s, est. speed input: 69141.89 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2842/3051 [22:26<01:39,  2.11it/s, est. speed input: 69141.93 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2843/3051 [22:27<01:38,  2.11it/s, est. speed input: 69142.02 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2844/3051 [22:27<01:38,  2.11it/s, est. speed input: 69142.05 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2845/3051 [22:28<01:37,  2.11it/s, est. speed input: 69142.08 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2846/3051 [22:28<01:37,  2.11it/s, est. speed input: 69142.11 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2847/3051 [22:29<01:36,  2.11it/s, est. speed input: 69142.16 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2848/3051 [22:29<01:36,  2.11it/s, est. speed input: 69142.23 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2849/3051 [22:30<01:35,  2.12it/s, est. speed input: 69142.33 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2850/3051 [22:30<01:37,  2.07it/s, est. speed input: 69140.57 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2851/3051 [22:31<01:35,  2.08it/s, est. speed input: 69140.70 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2852/3051 [22:31<01:35,  2.09it/s, est. speed input: 69140.74 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2853/3051 [22:32<01:34,  2.10it/s, est. speed input: 69140.82 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2854/3051 [22:32<01:33,  2.10it/s, est. speed input: 69140.85 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2855/3051 [22:33<01:33,  2.11it/s, est. speed input: 69140.90 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2856/3051 [22:33<01:32,  2.11it/s, est. speed input: 69140.94 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2857/3051 [22:34<01:32,  2.11it/s, est. speed input: 69140.91 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2858/3051 [22:34<01:31,  2.11it/s, est. speed input: 69140.97 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2859/3051 [22:34<01:30,  2.11it/s, est. speed input: 69141.03 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2860/3051 [22:35<01:30,  2.11it/s, est. speed input: 69141.06 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2861/3051 [22:35<01:29,  2.11it/s, est. speed input: 69141.13 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2862/3051 [22:36<01:29,  2.11it/s, est. speed input: 69141.19 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2863/3051 [22:36<01:28,  2.11it/s, est. speed input: 69141.24 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2864/3051 [22:37<01:28,  2.11it/s, est. speed input: 69141.19 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2865/3051 [22:37<01:28,  2.11it/s, est. speed input: 69141.24 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2866/3051 [22:38<01:27,  2.11it/s, est. speed input: 69141.30 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2867/3051 [22:38<01:27,  2.11it/s, est. speed input: 69141.13 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2868/3051 [22:39<01:26,  2.11it/s, est. speed input: 69141.22 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2869/3051 [22:39<01:26,  2.11it/s, est. speed input: 69141.25 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2870/3051 [22:40<01:25,  2.11it/s, est. speed input: 69141.17 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2871/3051 [22:40<01:25,  2.11it/s, est. speed input: 69141.22 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2872/3051 [22:41<01:24,  2.11it/s, est. speed input: 69141.29 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2873/3051 [22:41<01:24,  2.11it/s, est. speed input: 69141.27 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2874/3051 [22:42<01:23,  2.11it/s, est. speed input: 69141.32 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2875/3051 [22:42<01:23,  2.11it/s, est. speed input: 69141.39 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2876/3051 [22:43<01:22,  2.11it/s, est. speed input: 69141.47 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2877/3051 [22:43<01:22,  2.11it/s, est. speed input: 69141.44 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2878/3051 [22:43<01:21,  2.11it/s, est. speed input: 69141.49 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2879/3051 [22:44<01:21,  2.11it/s, est. speed input: 69141.55 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2880/3051 [22:44<01:20,  2.11it/s, est. speed input: 69141.61 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2881/3051 [22:45<01:20,  2.11it/s, est. speed input: 69141.59 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2882/3051 [22:45<01:19,  2.11it/s, est. speed input: 69141.63 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2883/3051 [22:46<01:19,  2.11it/s, est. speed input: 69141.63 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2884/3051 [22:46<01:19,  2.11it/s, est. speed input: 69141.61 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2885/3051 [22:47<01:18,  2.11it/s, est. speed input: 69141.56 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2886/3051 [22:47<01:18,  2.11it/s, est. speed input: 69141.62 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2887/3051 [22:48<01:17,  2.11it/s, est. speed input: 69141.67 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2888/3051 [22:48<01:17,  2.11it/s, est. speed input: 69141.74 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2889/3051 [22:49<01:16,  2.11it/s, est. speed input: 69141.67 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2890/3051 [22:49<01:16,  2.11it/s, est. speed input: 69141.73 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2891/3051 [22:50<01:15,  2.11it/s, est. speed input: 69141.79 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2892/3051 [22:50<01:15,  2.11it/s, est. speed input: 69141.76 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2893/3051 [22:51<01:14,  2.11it/s, est. speed input: 69141.80 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2894/3051 [22:51<01:14,  2.11it/s, est. speed input: 69141.90 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2895/3051 [22:52<01:13,  2.11it/s, est. speed input: 69141.98 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2896/3051 [22:52<01:13,  2.11it/s, est. speed input: 69141.80 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2897/3051 [22:52<01:12,  2.11it/s, est. speed input: 69141.86 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2898/3051 [22:53<01:12,  2.11it/s, est. speed input: 69141.94 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2899/3051 [22:53<01:11,  2.11it/s, est. speed input: 69141.98 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2900/3051 [22:54<01:11,  2.11it/s, est. speed input: 69142.06 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2901/3051 [22:54<01:10,  2.11it/s, est. speed input: 69142.10 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2902/3051 [22:55<01:10,  2.12it/s, est. speed input: 69142.20 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2903/3051 [22:55<01:09,  2.12it/s, est. speed input: 69142.27 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2904/3051 [22:56<01:09,  2.12it/s, est. speed input: 69142.34 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2905/3051 [22:56<01:09,  2.12it/s, est. speed input: 69142.40 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2906/3051 [22:57<01:08,  2.11it/s, est. speed input: 69142.43 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2907/3051 [22:57<01:08,  2.11it/s, est. speed input: 69142.48 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2908/3051 [22:58<01:07,  2.12it/s, est. speed input: 69142.56 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2909/3051 [22:58<01:07,  2.12it/s, est. speed input: 69142.62 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2910/3051 [22:59<01:06,  2.12it/s, est. speed input: 69142.68 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2911/3051 [22:59<01:06,  2.12it/s, est. speed input: 69142.73 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2912/3051 [23:00<01:05,  2.11it/s, est. speed input: 69142.64 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2913/3051 [23:00<01:05,  2.11it/s, est. speed input: 69142.70 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2914/3051 [23:00<01:04,  2.11it/s, est. speed input: 69142.76 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2915/3051 [23:01<01:04,  2.11it/s, est. speed input: 69142.73 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2916/3051 [23:01<01:03,  2.11it/s, est. speed input: 69142.79 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2917/3051 [23:02<01:06,  2.02it/s, est. speed input: 69139.12 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2918/3051 [23:02<01:04,  2.05it/s, est. speed input: 69139.24 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2919/3051 [23:03<01:03,  2.07it/s, est. speed input: 69139.32 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2920/3051 [23:03<01:02,  2.08it/s, est. speed input: 69139.39 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2921/3051 [23:04<01:02,  2.09it/s, est. speed input: 69139.46 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2922/3051 [23:04<01:01,  2.10it/s, est. speed input: 69139.54 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2923/3051 [23:05<01:00,  2.10it/s, est. speed input: 69139.62 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2924/3051 [23:05<01:00,  2.11it/s, est. speed input: 69139.70 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2925/3051 [23:06<00:59,  2.11it/s, est. speed input: 69139.77 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2926/3051 [23:06<00:59,  2.11it/s, est. speed input: 69139.86 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2927/3051 [23:07<00:58,  2.11it/s, est. speed input: 69139.95 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2928/3051 [23:07<00:58,  2.11it/s, est. speed input: 69140.01 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2929/3051 [23:08<00:57,  2.12it/s, est. speed input: 69140.10 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2930/3051 [23:08<00:57,  2.12it/s, est. speed input: 69140.18 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2931/3051 [23:09<00:56,  2.11it/s, est. speed input: 69140.18 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2932/3051 [23:09<00:56,  2.11it/s, est. speed input: 69140.21 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2933/3051 [23:10<00:55,  2.11it/s, est. speed input: 69140.25 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2934/3051 [23:10<00:55,  2.11it/s, est. speed input: 69140.30 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2935/3051 [23:10<00:54,  2.11it/s, est. speed input: 69140.35 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2936/3051 [23:11<00:54,  2.11it/s, est. speed input: 69140.41 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2937/3051 [23:11<00:53,  2.11it/s, est. speed input: 69140.48 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2938/3051 [23:12<00:53,  2.12it/s, est. speed input: 69140.56 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2939/3051 [23:12<00:52,  2.11it/s, est. speed input: 69140.55 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2940/3051 [23:13<00:52,  2.11it/s, est. speed input: 69140.61 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2941/3051 [23:13<00:52,  2.11it/s, est. speed input: 69140.67 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2942/3051 [23:14<00:51,  2.12it/s, est. speed input: 69140.74 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2943/3051 [23:14<00:51,  2.12it/s, est. speed input: 69140.80 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2944/3051 [23:15<00:50,  2.12it/s, est. speed input: 69140.89 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2945/3051 [23:15<00:50,  2.12it/s, est. speed input: 69140.95 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2946/3051 [23:16<00:49,  2.11it/s, est. speed input: 69140.90 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2947/3051 [23:16<00:49,  2.11it/s, est. speed input: 69140.97 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2948/3051 [23:17<00:48,  2.12it/s, est. speed input: 69141.06 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2949/3051 [23:17<00:48,  2.11it/s, est. speed input: 69141.10 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2950/3051 [23:18<00:47,  2.12it/s, est. speed input: 69141.17 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2951/3051 [23:18<00:47,  2.11it/s, est. speed input: 69141.20 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2952/3051 [23:19<00:46,  2.12it/s, est. speed input: 69141.28 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2953/3051 [23:19<00:46,  2.12it/s, est. speed input: 69141.33 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2954/3051 [23:19<00:45,  2.11it/s, est. speed input: 69141.38 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2955/3051 [23:20<00:45,  2.11it/s, est. speed input: 69141.44 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2956/3051 [23:20<00:44,  2.11it/s, est. speed input: 69141.39 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2957/3051 [23:21<00:44,  2.11it/s, est. speed input: 69141.47 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2958/3051 [23:21<00:43,  2.12it/s, est. speed input: 69141.56 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2959/3051 [23:22<00:43,  2.12it/s, est. speed input: 69141.67 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2960/3051 [23:22<00:42,  2.12it/s, est. speed input: 69141.73 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2961/3051 [23:23<00:42,  2.12it/s, est. speed input: 69141.82 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2962/3051 [23:23<00:42,  2.12it/s, est. speed input: 69141.90 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2963/3051 [23:24<00:41,  2.12it/s, est. speed input: 69141.99 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2964/3051 [23:24<00:41,  2.12it/s, est. speed input: 69142.04 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2965/3051 [23:25<00:40,  2.12it/s, est. speed input: 69142.11 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2966/3051 [23:25<00:40,  2.12it/s, est. speed input: 69142.16 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2967/3051 [23:26<00:39,  2.12it/s, est. speed input: 69142.23 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2968/3051 [23:26<00:39,  2.12it/s, est. speed input: 69142.29 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2969/3051 [23:27<00:38,  2.12it/s, est. speed input: 69142.33 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2970/3051 [23:27<00:38,  2.12it/s, est. speed input: 69142.38 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2971/3051 [23:28<00:37,  2.11it/s, est. speed input: 69142.42 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2972/3051 [23:28<00:37,  2.12it/s, est. speed input: 69142.50 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2973/3051 [23:28<00:36,  2.11it/s, est. speed input: 69142.45 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2974/3051 [23:29<00:36,  2.11it/s, est. speed input: 69142.53 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2975/3051 [23:29<00:35,  2.11it/s, est. speed input: 69142.60 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2976/3051 [23:30<00:35,  2.12it/s, est. speed input: 69142.68 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2977/3051 [23:30<00:34,  2.12it/s, est. speed input: 69142.77 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2978/3051 [23:31<00:34,  2.12it/s, est. speed input: 69142.86 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2979/3051 [23:31<00:34,  2.12it/s, est. speed input: 69142.89 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2980/3051 [23:32<00:33,  2.12it/s, est. speed input: 69142.96 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2981/3051 [23:32<00:33,  2.12it/s, est. speed input: 69142.99 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2982/3051 [23:33<00:32,  2.12it/s, est. speed input: 69143.07 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2983/3051 [23:33<00:32,  2.12it/s, est. speed input: 69143.16 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2984/3051 [23:34<00:31,  2.12it/s, est. speed input: 69143.23 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2985/3051 [23:34<00:31,  2.12it/s, est. speed input: 69143.27 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2986/3051 [23:35<00:30,  2.12it/s, est. speed input: 69143.32 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2987/3051 [23:35<00:30,  2.11it/s, est. speed input: 69143.36 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2988/3051 [23:36<00:29,  2.12it/s, est. speed input: 69143.45 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2989/3051 [23:36<00:29,  2.12it/s, est. speed input: 69143.51 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2990/3051 [23:36<00:28,  2.12it/s, est. speed input: 69143.59 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2991/3051 [23:37<00:28,  2.12it/s, est. speed input: 69143.65 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2992/3051 [23:37<00:27,  2.12it/s, est. speed input: 69143.72 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2993/3051 [23:38<00:27,  2.12it/s, est. speed input: 69143.81 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2994/3051 [23:38<00:26,  2.12it/s, est. speed input: 69143.90 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2995/3051 [23:39<00:26,  2.11it/s, est. speed input: 69143.83 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2996/3051 [23:39<00:26,  2.11it/s, est. speed input: 69143.88 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2997/3051 [23:40<00:25,  2.12it/s, est. speed input: 69143.97 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2998/3051 [23:40<00:25,  2.12it/s, est. speed input: 69144.04 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2999/3051 [23:41<00:24,  2.12it/s, est. speed input: 69144.15 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3000/3051 [23:41<00:24,  2.12it/s, est. speed input: 69144.22 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3001/3051 [23:42<00:23,  2.12it/s, est. speed input: 69144.31 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3002/3051 [23:42<00:23,  2.12it/s, est. speed input: 69144.40 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3003/3051 [23:43<00:22,  2.12it/s, est. speed input: 69144.50 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3004/3051 [23:43<00:22,  2.12it/s, est. speed input: 69144.51 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3005/3051 [23:44<00:21,  2.12it/s, est. speed input: 69144.55 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3006/3051 [23:44<00:21,  2.12it/s, est. speed input: 69144.61 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3007/3051 [23:45<00:20,  2.12it/s, est. speed input: 69144.72 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3008/3051 [23:45<00:20,  2.12it/s, est. speed input: 69144.78 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3009/3051 [23:45<00:19,  2.12it/s, est. speed input: 69144.91 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3010/3051 [23:46<00:19,  2.12it/s, est. speed input: 69144.99 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3011/3051 [23:46<00:18,  2.11it/s, est. speed input: 69144.93 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3012/3051 [23:47<00:18,  2.11it/s, est. speed input: 69145.00 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3013/3051 [23:47<00:17,  2.12it/s, est. speed input: 69145.10 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3014/3051 [23:48<00:17,  2.11it/s, est. speed input: 69145.08 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3015/3051 [23:48<00:17,  2.11it/s, est. speed input: 69145.12 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3016/3051 [23:49<00:16,  2.12it/s, est. speed input: 69145.22 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3017/3051 [23:49<00:16,  2.12it/s, est. speed input: 69145.33 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3018/3051 [23:50<00:15,  2.12it/s, est. speed input: 69145.41 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3019/3051 [23:50<00:15,  2.12it/s, est. speed input: 69145.46 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3020/3051 [23:51<00:14,  2.12it/s, est. speed input: 69145.52 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3021/3051 [23:51<00:14,  2.12it/s, est. speed input: 69145.58 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3022/3051 [23:52<00:13,  2.12it/s, est. speed input: 69145.67 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3023/3051 [23:52<00:13,  2.12it/s, est. speed input: 69145.77 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3024/3051 [23:53<00:12,  2.12it/s, est. speed input: 69145.79 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3025/3051 [23:53<00:12,  2.12it/s, est. speed input: 69145.84 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3026/3051 [23:54<00:11,  2.12it/s, est. speed input: 69145.93 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3027/3051 [23:54<00:11,  2.12it/s, est. speed input: 69145.99 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3028/3051 [23:54<00:10,  2.11it/s, est. speed input: 69145.98 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3029/3051 [23:55<00:10,  2.12it/s, est. speed input: 69146.08 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3030/3051 [23:55<00:09,  2.12it/s, est. speed input: 69146.15 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3031/3051 [23:56<00:09,  2.11it/s, est. speed input: 69146.16 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3032/3051 [23:56<00:08,  2.12it/s, est. speed input: 69146.24 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3033/3051 [23:57<00:08,  2.11it/s, est. speed input: 69146.21 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3034/3051 [23:57<00:08,  2.11it/s, est. speed input: 69146.24 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3035/3051 [23:58<00:07,  2.11it/s, est. speed input: 69146.32 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3036/3051 [23:58<00:07,  2.11it/s, est. speed input: 69146.38 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3037/3051 [23:59<00:06,  2.12it/s, est. speed input: 69146.47 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3038/3051 [23:59<00:06,  2.12it/s, est. speed input: 69146.52 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3039/3051 [24:00<00:05,  2.12it/s, est. speed input: 69146.58 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3040/3051 [24:00<00:05,  2.11it/s, est. speed input: 69146.54 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3041/3051 [24:01<00:04,  2.11it/s, est. speed input: 69146.61 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3042/3051 [24:01<00:04,  2.11it/s, est. speed input: 69146.64 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3043/3051 [24:02<00:03,  2.12it/s, est. speed input: 69146.74 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3044/3051 [24:02<00:03,  2.12it/s, est. speed input: 69146.80 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3045/3051 [24:02<00:02,  2.12it/s, est. speed input: 69146.88 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3046/3051 [24:03<00:02,  2.11it/s, est. speed input: 69146.86 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3047/3051 [24:03<00:01,  2.12it/s, est. speed input: 69146.95 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3048/3051 [24:04<00:01,  2.12it/s, est. speed input: 69147.00 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3049/3051 [24:04<00:00,  2.12it/s, est. speed input: 69147.06 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3050/3051 [24:05<00:00,  2.12it/s, est. speed input: 69147.13 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 3051/3051 [24:05<00:00,  2.12it/s, est. speed input: 69147.22 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 3051/3051 [24:05<00:00,  2.11it/s, est. speed input: 69147.22 toks/s, output: 2.11 toks/s]
Saved activation counts to new_activations/activation.en.train.q2.5-en+zh.pt
Processed 99975168 tokens total

Activation analysis complete for model: q2.5-en+zh!

=== Processing model: /home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_1.5B_irish_corpus_240925/checkpoint-2854, type: qwen ===
INFO 09-26 14:36:18 [config.py:2700] Downcasting torch.float32 to torch.float16.
INFO 09-26 14:36:18 [config.py:600] This model supports multiple tasks: {'classify', 'embed', 'reward', 'score', 'generate'}. Defaulting to 'generate'.
WARNING 09-26 14:36:18 [cuda.py:96] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
INFO 09-26 14:36:18 [llm_engine.py:242] Initializing a V0 LLM engine (v0.8.3) with config: model='/home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_1.5B_irish_corpus_240925/checkpoint-2854', speculative_config=None, tokenizer='/home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_1.5B_irish_corpus_240925/checkpoint-2854', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=32768, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar', reasoning_backend=None), observability_config=ObservabilityConfig(show_hidden_metrics=False, otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=None, served_model_name=/home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_1.5B_irish_corpus_240925/checkpoint-2854, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=None, chunked_prefill_enabled=False, use_async_output_proc=False, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"splitting_ops":[],"compile_sizes":[],"cudagraph_capture_sizes":[],"max_capture_size":0}, use_cached_outputs=False, 
INFO 09-26 14:36:20 [model_runner.py:1110] Starting to load model /home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_1.5B_irish_corpus_240925/checkpoint-2854...

Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:03<00:00,  3.98s/it]

Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:03<00:00,  3.98s/it]

INFO 09-26 14:36:24 [loader.py:447] Loading weights took 4.04 seconds
INFO 09-26 14:36:26 [model_runner.py:1146] Model loading took 0.9226 GiB and 4.095651 seconds
INFO 09-26 14:36:28 [worker.py:267] Memory profiling takes 1.41 seconds
INFO 09-26 14:36:28 [worker.py:267] the current vLLM instance can use total_gpu_memory (79.25GiB) x gpu_memory_utilization (0.90) = 71.33GiB
INFO 09-26 14:36:28 [worker.py:267] model weights take 0.92GiB; non_torch_memory takes 0.00GiB; PyTorch activation peak memory takes 1.43GiB; the rest of the memory reserved for KV Cache is 68.97GiB.
INFO 09-26 14:36:29 [executor_base.py:112] # cuda blocks: 376692, # CPU blocks: 21845
INFO 09-26 14:36:29 [executor_base.py:117] Maximum concurrency for 32768 tokens per request: 183.93x
INFO 09-26 14:36:30 [llm_engine.py:448] init engine (profile, create kv cache, warmup model) took 3.96 seconds
Layers: 24, Intermediate size: 4864, Max length: 32768

Loading IDs from ./ids/q2.5/id.en.train.qwen2.5-0.5 (lang: en)...
ID shape: torch.Size([100000000])
Processing 3051 sequences of length 32768
Running inference...

Processed prompts:   0%|          | 0/3051 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts:   0%|          | 1/3051 [00:00<25:14,  2.01it/s, est. speed input: 66001.85 toks/s, output: 2.01 toks/s]
Processed prompts:   0%|          | 2/3051 [00:00<24:23,  2.08it/s, est. speed input: 67931.86 toks/s, output: 2.07 toks/s]
Processed prompts:   0%|          | 3/3051 [00:01<24:06,  2.11it/s, est. speed input: 68592.32 toks/s, output: 2.09 toks/s]
Processed prompts:   0%|          | 4/3051 [00:02<26:58,  1.88it/s, est. speed input: 63901.12 toks/s, output: 1.95 toks/s]
Processed prompts:   0%|          | 5/3051 [00:02<25:48,  1.97it/s, est. speed input: 65033.91 toks/s, output: 1.98 toks/s]
Processed prompts:   0%|          | 6/3051 [00:02<25:06,  2.02it/s, est. speed input: 65811.41 toks/s, output: 2.01 toks/s]
Processed prompts:   0%|          | 7/3051 [00:03<24:39,  2.06it/s, est. speed input: 66388.22 toks/s, output: 2.03 toks/s]
Processed prompts:   0%|          | 8/3051 [00:03<24:21,  2.08it/s, est. speed input: 66819.84 toks/s, output: 2.04 toks/s]
Processed prompts:   0%|          | 9/3051 [00:04<24:09,  2.10it/s, est. speed input: 67157.46 toks/s, output: 2.05 toks/s]
Processed prompts:   0%|          | 10/3051 [00:04<24:01,  2.11it/s, est. speed input: 67430.95 toks/s, output: 2.06 toks/s]
Processed prompts:   0%|          | 11/3051 [00:05<23:55,  2.12it/s, est. speed input: 67658.54 toks/s, output: 2.06 toks/s]
Processed prompts:   0%|          | 12/3051 [00:05<23:51,  2.12it/s, est. speed input: 67846.53 toks/s, output: 2.07 toks/s]
Processed prompts:   0%|          | 13/3051 [00:06<23:47,  2.13it/s, est. speed input: 68009.28 toks/s, output: 2.08 toks/s]
Processed prompts:   0%|          | 14/3051 [00:06<23:45,  2.13it/s, est. speed input: 68147.47 toks/s, output: 2.08 toks/s]
Processed prompts:   0%|          | 15/3051 [00:07<23:44,  2.13it/s, est. speed input: 68263.99 toks/s, output: 2.08 toks/s]
Processed prompts:   1%|          | 16/3051 [00:07<23:43,  2.13it/s, est. speed input: 68364.10 toks/s, output: 2.09 toks/s]
Processed prompts:   1%|          | 17/3051 [00:08<23:42,  2.13it/s, est. speed input: 68456.65 toks/s, output: 2.09 toks/s]
Processed prompts:   1%|          | 18/3051 [00:08<23:41,  2.13it/s, est. speed input: 68536.10 toks/s, output: 2.09 toks/s]
Processed prompts:   1%|          | 19/3051 [00:09<23:40,  2.13it/s, est. speed input: 68610.48 toks/s, output: 2.09 toks/s]
Processed prompts:   1%|          | 20/3051 [00:09<23:40,  2.13it/s, est. speed input: 68676.47 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 21/3051 [00:10<23:39,  2.13it/s, est. speed input: 68737.66 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 22/3051 [00:10<23:38,  2.13it/s, est. speed input: 68792.51 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 23/3051 [00:10<23:37,  2.14it/s, est. speed input: 68844.99 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 24/3051 [00:11<23:37,  2.13it/s, est. speed input: 68888.20 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 25/3051 [00:11<23:37,  2.13it/s, est. speed input: 68928.32 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 26/3051 [00:12<23:37,  2.13it/s, est. speed input: 68966.66 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 27/3051 [00:12<23:36,  2.13it/s, est. speed input: 69003.14 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 28/3051 [00:13<23:36,  2.13it/s, est. speed input: 69033.88 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 29/3051 [00:13<23:36,  2.13it/s, est. speed input: 69063.90 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 30/3051 [00:14<23:35,  2.13it/s, est. speed input: 69091.04 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 31/3051 [00:14<23:35,  2.13it/s, est. speed input: 69119.44 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 32/3051 [00:15<23:34,  2.13it/s, est. speed input: 69142.57 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 33/3051 [00:15<23:34,  2.13it/s, est. speed input: 69165.17 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 34/3051 [00:16<23:34,  2.13it/s, est. speed input: 69184.48 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 35/3051 [00:16<23:34,  2.13it/s, est. speed input: 69203.29 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 36/3051 [00:17<23:33,  2.13it/s, est. speed input: 69223.28 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 37/3051 [00:17<23:33,  2.13it/s, est. speed input: 69240.96 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 38/3051 [00:17<23:32,  2.13it/s, est. speed input: 69257.76 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|▏         | 39/3051 [00:18<23:32,  2.13it/s, est. speed input: 69273.71 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|▏         | 40/3051 [00:18<23:31,  2.13it/s, est. speed input: 69288.56 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|▏         | 41/3051 [00:19<23:31,  2.13it/s, est. speed input: 69303.93 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|▏         | 42/3051 [00:19<23:30,  2.13it/s, est. speed input: 69320.42 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 43/3051 [00:20<23:29,  2.13it/s, est. speed input: 69333.55 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 44/3051 [00:20<23:29,  2.13it/s, est. speed input: 69347.44 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 45/3051 [00:21<23:29,  2.13it/s, est. speed input: 69357.48 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 46/3051 [00:21<23:29,  2.13it/s, est. speed input: 69368.19 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 47/3051 [00:22<23:28,  2.13it/s, est. speed input: 69379.23 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 48/3051 [00:22<23:28,  2.13it/s, est. speed input: 69388.75 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 49/3051 [00:23<23:27,  2.13it/s, est. speed input: 69398.39 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 50/3051 [00:23<23:27,  2.13it/s, est. speed input: 69407.72 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 51/3051 [00:24<23:26,  2.13it/s, est. speed input: 69417.97 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 52/3051 [00:24<23:26,  2.13it/s, est. speed input: 69426.42 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 53/3051 [00:25<23:25,  2.13it/s, est. speed input: 69435.64 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 54/3051 [00:25<23:25,  2.13it/s, est. speed input: 69442.45 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 55/3051 [00:25<23:25,  2.13it/s, est. speed input: 69450.45 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 56/3051 [00:26<23:24,  2.13it/s, est. speed input: 69457.18 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 57/3051 [00:26<23:24,  2.13it/s, est. speed input: 69464.93 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 58/3051 [00:27<23:23,  2.13it/s, est. speed input: 69471.46 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 59/3051 [00:27<23:23,  2.13it/s, est. speed input: 69477.89 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 60/3051 [00:28<23:23,  2.13it/s, est. speed input: 69483.39 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 61/3051 [00:28<23:22,  2.13it/s, est. speed input: 69490.11 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 62/3051 [00:29<23:21,  2.13it/s, est. speed input: 69497.22 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 63/3051 [00:29<23:21,  2.13it/s, est. speed input: 69502.76 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 64/3051 [00:30<23:20,  2.13it/s, est. speed input: 69509.65 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 65/3051 [00:30<23:20,  2.13it/s, est. speed input: 69514.41 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 66/3051 [00:31<23:20,  2.13it/s, est. speed input: 69518.30 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 67/3051 [00:31<23:19,  2.13it/s, est. speed input: 69523.87 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 68/3051 [00:32<23:19,  2.13it/s, est. speed input: 69528.91 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 69/3051 [00:32<23:19,  2.13it/s, est. speed input: 69532.25 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 70/3051 [00:32<23:18,  2.13it/s, est. speed input: 69536.43 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 71/3051 [00:33<23:18,  2.13it/s, est. speed input: 69541.36 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 72/3051 [00:33<23:17,  2.13it/s, est. speed input: 69545.04 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 73/3051 [00:34<23:17,  2.13it/s, est. speed input: 69548.94 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 74/3051 [00:34<23:17,  2.13it/s, est. speed input: 69552.21 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 75/3051 [00:35<23:16,  2.13it/s, est. speed input: 69557.11 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 76/3051 [00:35<23:15,  2.13it/s, est. speed input: 69560.75 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 77/3051 [00:36<23:15,  2.13it/s, est. speed input: 69564.85 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 78/3051 [00:36<23:14,  2.13it/s, est. speed input: 69568.38 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 79/3051 [00:37<23:14,  2.13it/s, est. speed input: 69571.88 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 80/3051 [00:37<23:13,  2.13it/s, est. speed input: 69575.38 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 81/3051 [00:38<23:13,  2.13it/s, est. speed input: 69578.17 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 82/3051 [00:38<23:13,  2.13it/s, est. speed input: 69580.34 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 83/3051 [00:39<23:13,  2.13it/s, est. speed input: 69583.06 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 84/3051 [00:39<23:12,  2.13it/s, est. speed input: 69586.44 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 85/3051 [00:40<23:11,  2.13it/s, est. speed input: 69589.38 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 86/3051 [00:40<23:11,  2.13it/s, est. speed input: 69592.70 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 87/3051 [00:40<23:10,  2.13it/s, est. speed input: 69594.97 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 88/3051 [00:41<23:10,  2.13it/s, est. speed input: 69596.82 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 89/3051 [00:41<23:10,  2.13it/s, est. speed input: 69599.65 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 90/3051 [00:42<23:09,  2.13it/s, est. speed input: 69602.09 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 91/3051 [00:42<23:09,  2.13it/s, est. speed input: 69604.53 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 92/3051 [00:43<23:08,  2.13it/s, est. speed input: 69606.49 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 93/3051 [00:43<23:08,  2.13it/s, est. speed input: 69608.33 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 94/3051 [00:44<23:08,  2.13it/s, est. speed input: 69610.62 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 95/3051 [00:44<23:07,  2.13it/s, est. speed input: 69612.22 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 96/3051 [00:45<23:07,  2.13it/s, est. speed input: 69614.49 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 97/3051 [00:45<23:06,  2.13it/s, est. speed input: 69616.53 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 98/3051 [00:46<23:06,  2.13it/s, est. speed input: 69618.70 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 99/3051 [00:46<23:05,  2.13it/s, est. speed input: 69620.92 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 100/3051 [00:47<23:05,  2.13it/s, est. speed input: 69622.55 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 101/3051 [00:47<23:04,  2.13it/s, est. speed input: 69624.02 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 102/3051 [00:48<23:04,  2.13it/s, est. speed input: 69625.70 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 103/3051 [00:48<23:04,  2.13it/s, est. speed input: 69627.31 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 104/3051 [00:48<23:03,  2.13it/s, est. speed input: 69628.75 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 105/3051 [00:49<23:02,  2.13it/s, est. speed input: 69630.79 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 106/3051 [00:49<23:02,  2.13it/s, est. speed input: 69632.25 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 107/3051 [00:50<23:01,  2.13it/s, est. speed input: 69634.01 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 108/3051 [00:50<23:02,  2.13it/s, est. speed input: 69634.56 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 109/3051 [00:51<23:01,  2.13it/s, est. speed input: 69635.85 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 110/3051 [00:51<23:01,  2.13it/s, est. speed input: 69636.51 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 111/3051 [00:52<23:01,  2.13it/s, est. speed input: 69637.69 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 112/3051 [00:52<23:00,  2.13it/s, est. speed input: 69638.80 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 113/3051 [00:53<22:59,  2.13it/s, est. speed input: 69640.44 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▎         | 114/3051 [00:53<22:59,  2.13it/s, est. speed input: 69641.77 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 115/3051 [00:54<22:59,  2.13it/s, est. speed input: 69642.28 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 116/3051 [00:54<22:58,  2.13it/s, est. speed input: 69643.22 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 117/3051 [00:55<22:58,  2.13it/s, est. speed input: 69644.41 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 118/3051 [00:55<22:57,  2.13it/s, est. speed input: 69645.12 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 119/3051 [00:55<22:57,  2.13it/s, est. speed input: 69645.72 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 120/3051 [00:56<22:57,  2.13it/s, est. speed input: 69646.03 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 121/3051 [00:56<22:57,  2.13it/s, est. speed input: 69646.40 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 122/3051 [00:57<22:56,  2.13it/s, est. speed input: 69647.11 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 123/3051 [00:57<22:55,  2.13it/s, est. speed input: 69648.10 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 124/3051 [00:58<22:55,  2.13it/s, est. speed input: 69649.24 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 125/3051 [00:58<22:54,  2.13it/s, est. speed input: 69650.23 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 126/3051 [00:59<22:53,  2.13it/s, est. speed input: 69651.65 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 127/3051 [00:59<22:53,  2.13it/s, est. speed input: 69652.30 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 128/3051 [01:00<22:52,  2.13it/s, est. speed input: 69653.30 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 129/3051 [01:00<22:52,  2.13it/s, est. speed input: 69654.06 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 130/3051 [01:01<22:52,  2.13it/s, est. speed input: 69654.57 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 131/3051 [01:01<22:52,  2.13it/s, est. speed input: 69654.80 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 132/3051 [01:02<22:51,  2.13it/s, est. speed input: 69655.64 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 133/3051 [01:02<22:50,  2.13it/s, est. speed input: 69656.57 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 134/3051 [01:03<22:50,  2.13it/s, est. speed input: 69656.83 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 135/3051 [01:03<22:50,  2.13it/s, est. speed input: 69657.54 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 136/3051 [01:03<22:49,  2.13it/s, est. speed input: 69657.73 toks/s, output: 2.13 toks/s]
Processed prompts:   4%|▍         | 137/3051 [01:04<22:49,  2.13it/s, est. speed input: 69658.26 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 138/3051 [01:04<22:49,  2.13it/s, est. speed input: 69658.33 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 139/3051 [01:05<22:48,  2.13it/s, est. speed input: 69658.72 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 140/3051 [01:05<22:48,  2.13it/s, est. speed input: 69659.14 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 141/3051 [01:06<22:47,  2.13it/s, est. speed input: 69659.75 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 142/3051 [01:06<22:47,  2.13it/s, est. speed input: 69659.79 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 143/3051 [01:07<22:46,  2.13it/s, est. speed input: 69660.36 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 144/3051 [01:07<22:46,  2.13it/s, est. speed input: 69660.69 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 145/3051 [01:08<22:45,  2.13it/s, est. speed input: 69661.05 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 146/3051 [01:08<22:45,  2.13it/s, est. speed input: 69661.23 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 147/3051 [01:09<22:45,  2.13it/s, est. speed input: 69661.39 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 148/3051 [01:09<22:44,  2.13it/s, est. speed input: 69661.93 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 149/3051 [01:10<22:43,  2.13it/s, est. speed input: 69662.36 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 150/3051 [01:10<22:43,  2.13it/s, est. speed input: 69662.78 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 151/3051 [01:11<22:43,  2.13it/s, est. speed input: 69662.90 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▍         | 152/3051 [01:11<22:42,  2.13it/s, est. speed input: 69663.26 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 153/3051 [01:11<22:43,  2.13it/s, est. speed input: 69662.60 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 154/3051 [01:12<22:42,  2.13it/s, est. speed input: 69662.94 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 155/3051 [01:12<22:41,  2.13it/s, est. speed input: 69663.24 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 156/3051 [01:13<22:41,  2.13it/s, est. speed input: 69662.68 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 157/3051 [01:13<22:41,  2.13it/s, est. speed input: 69662.72 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 158/3051 [01:14<22:40,  2.13it/s, est. speed input: 69662.76 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 159/3051 [01:14<22:40,  2.13it/s, est. speed input: 69663.09 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 160/3051 [01:15<22:39,  2.13it/s, est. speed input: 69663.21 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 161/3051 [01:15<22:39,  2.13it/s, est. speed input: 69662.68 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 162/3051 [01:16<22:38,  2.13it/s, est. speed input: 69663.32 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 163/3051 [01:16<22:37,  2.13it/s, est. speed input: 69663.92 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 164/3051 [01:17<22:37,  2.13it/s, est. speed input: 69663.91 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 165/3051 [01:17<22:36,  2.13it/s, est. speed input: 69664.19 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 166/3051 [01:18<22:36,  2.13it/s, est. speed input: 69664.33 toks/s, output: 2.13 toks/s]
Processed prompts:   5%|▌         | 167/3051 [01:18<22:36,  2.13it/s, est. speed input: 69663.90 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 168/3051 [01:19<22:35,  2.13it/s, est. speed input: 69664.24 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 169/3051 [01:19<22:35,  2.13it/s, est. speed input: 69664.17 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 170/3051 [01:19<22:35,  2.13it/s, est. speed input: 69664.17 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 171/3051 [01:20<22:34,  2.13it/s, est. speed input: 69664.39 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 172/3051 [01:20<22:33,  2.13it/s, est. speed input: 69664.72 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 173/3051 [01:21<22:33,  2.13it/s, est. speed input: 69664.91 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 174/3051 [01:21<22:32,  2.13it/s, est. speed input: 69665.54 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 175/3051 [01:22<22:31,  2.13it/s, est. speed input: 69665.76 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 176/3051 [01:22<22:31,  2.13it/s, est. speed input: 69665.63 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 177/3051 [01:23<22:31,  2.13it/s, est. speed input: 69665.63 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 178/3051 [01:23<22:31,  2.13it/s, est. speed input: 69665.12 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 179/3051 [01:24<22:32,  2.12it/s, est. speed input: 69663.87 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 180/3051 [01:24<22:31,  2.12it/s, est. speed input: 69663.51 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 181/3051 [01:25<22:31,  2.12it/s, est. speed input: 69663.38 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 182/3051 [01:25<22:30,  2.12it/s, est. speed input: 69663.07 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 183/3051 [01:26<22:30,  2.12it/s, est. speed input: 69662.84 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 184/3051 [01:26<22:28,  2.13it/s, est. speed input: 69663.13 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 185/3051 [01:27<22:28,  2.13it/s, est. speed input: 69663.10 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 186/3051 [01:27<22:28,  2.12it/s, est. speed input: 69662.22 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 187/3051 [01:27<22:28,  2.12it/s, est. speed input: 69662.20 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 188/3051 [01:28<22:26,  2.13it/s, est. speed input: 69662.54 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 189/3051 [01:28<22:26,  2.13it/s, est. speed input: 69662.61 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▌         | 190/3051 [01:29<22:25,  2.13it/s, est. speed input: 69663.01 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 191/3051 [01:29<22:25,  2.13it/s, est. speed input: 69662.99 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 192/3051 [01:30<22:24,  2.13it/s, est. speed input: 69662.92 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 193/3051 [01:30<22:24,  2.13it/s, est. speed input: 69662.47 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 194/3051 [01:31<22:24,  2.12it/s, est. speed input: 69661.88 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 195/3051 [01:31<22:24,  2.12it/s, est. speed input: 69661.29 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 196/3051 [01:32<22:23,  2.12it/s, est. speed input: 69661.48 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 197/3051 [01:32<22:22,  2.13it/s, est. speed input: 69661.56 toks/s, output: 2.13 toks/s]
Processed prompts:   6%|▋         | 198/3051 [01:33<22:22,  2.13it/s, est. speed input: 69661.43 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 199/3051 [01:33<22:22,  2.13it/s, est. speed input: 69661.26 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 200/3051 [01:34<22:21,  2.13it/s, est. speed input: 69661.27 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 201/3051 [01:34<22:21,  2.13it/s, est. speed input: 69661.00 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 202/3051 [01:35<22:20,  2.13it/s, est. speed input: 69661.21 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 203/3051 [01:35<22:21,  2.12it/s, est. speed input: 69660.08 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 204/3051 [01:35<22:20,  2.12it/s, est. speed input: 69659.87 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 205/3051 [01:36<22:19,  2.12it/s, est. speed input: 69659.66 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 206/3051 [01:36<22:19,  2.12it/s, est. speed input: 69659.48 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 207/3051 [01:37<22:18,  2.12it/s, est. speed input: 69659.53 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 208/3051 [01:37<22:18,  2.12it/s, est. speed input: 69659.12 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 209/3051 [01:38<22:19,  2.12it/s, est. speed input: 69657.41 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 210/3051 [01:38<22:18,  2.12it/s, est. speed input: 69657.43 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 211/3051 [01:39<22:17,  2.12it/s, est. speed input: 69657.18 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 212/3051 [01:39<22:17,  2.12it/s, est. speed input: 69656.75 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 213/3051 [01:40<22:17,  2.12it/s, est. speed input: 69655.92 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 214/3051 [01:40<22:16,  2.12it/s, est. speed input: 69655.70 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 215/3051 [01:41<22:15,  2.12it/s, est. speed input: 69655.69 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 216/3051 [01:41<22:14,  2.12it/s, est. speed input: 69655.86 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 217/3051 [01:42<22:13,  2.12it/s, est. speed input: 69655.72 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 218/3051 [01:42<22:13,  2.12it/s, est. speed input: 69655.46 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 219/3051 [01:43<22:13,  2.12it/s, est. speed input: 69654.71 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 220/3051 [01:43<22:13,  2.12it/s, est. speed input: 69654.10 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 221/3051 [01:43<22:12,  2.12it/s, est. speed input: 69654.01 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 222/3051 [01:44<22:11,  2.12it/s, est. speed input: 69654.40 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 223/3051 [01:44<22:10,  2.13it/s, est. speed input: 69654.68 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 224/3051 [01:45<22:10,  2.12it/s, est. speed input: 69653.83 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 225/3051 [01:45<22:10,  2.12it/s, est. speed input: 69653.73 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 226/3051 [01:46<22:11,  2.12it/s, est. speed input: 69652.64 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 227/3051 [01:46<22:10,  2.12it/s, est. speed input: 69651.96 toks/s, output: 2.13 toks/s]
Processed prompts:   7%|▋         | 228/3051 [01:47<22:09,  2.12it/s, est. speed input: 69651.97 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 229/3051 [01:47<22:09,  2.12it/s, est. speed input: 69651.65 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 230/3051 [01:48<22:08,  2.12it/s, est. speed input: 69651.06 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 231/3051 [01:48<22:08,  2.12it/s, est. speed input: 69650.57 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 232/3051 [01:49<22:07,  2.12it/s, est. speed input: 69650.55 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 233/3051 [01:49<22:07,  2.12it/s, est. speed input: 69650.33 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 234/3051 [01:50<22:06,  2.12it/s, est. speed input: 69650.40 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 235/3051 [01:50<22:05,  2.12it/s, est. speed input: 69650.14 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 236/3051 [01:51<22:04,  2.12it/s, est. speed input: 69650.23 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 237/3051 [01:51<22:04,  2.13it/s, est. speed input: 69650.30 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 238/3051 [01:51<22:03,  2.12it/s, est. speed input: 69650.07 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 239/3051 [01:52<22:04,  2.12it/s, est. speed input: 69649.33 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 240/3051 [01:52<22:07,  2.12it/s, est. speed input: 69646.22 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 241/3051 [01:53<22:06,  2.12it/s, est. speed input: 69645.95 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 242/3051 [01:53<22:04,  2.12it/s, est. speed input: 69645.60 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 243/3051 [01:54<22:04,  2.12it/s, est. speed input: 69644.66 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 244/3051 [01:54<22:03,  2.12it/s, est. speed input: 69644.66 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 245/3051 [01:55<22:01,  2.12it/s, est. speed input: 69644.69 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 246/3051 [01:55<22:01,  2.12it/s, est. speed input: 69644.21 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 247/3051 [01:56<22:00,  2.12it/s, est. speed input: 69644.48 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 248/3051 [01:56<21:59,  2.12it/s, est. speed input: 69644.51 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 249/3051 [01:57<21:59,  2.12it/s, est. speed input: 69644.19 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 250/3051 [01:57<21:59,  2.12it/s, est. speed input: 69643.35 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 251/3051 [01:58<21:59,  2.12it/s, est. speed input: 69643.01 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 252/3051 [01:58<21:58,  2.12it/s, est. speed input: 69642.73 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 253/3051 [01:59<21:58,  2.12it/s, est. speed input: 69641.98 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 254/3051 [01:59<21:58,  2.12it/s, est. speed input: 69641.44 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 255/3051 [01:59<22:00,  2.12it/s, est. speed input: 69638.84 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 256/3051 [02:00<22:01,  2.11it/s, est. speed input: 69636.70 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 257/3051 [02:00<21:59,  2.12it/s, est. speed input: 69636.45 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 258/3051 [02:01<21:58,  2.12it/s, est. speed input: 69636.26 toks/s, output: 2.13 toks/s]
Processed prompts:   8%|▊         | 259/3051 [02:01<21:56,  2.12it/s, est. speed input: 69636.41 toks/s, output: 2.13 toks/s]
Processed prompts:   9%|▊         | 260/3051 [02:02<21:55,  2.12it/s, est. speed input: 69635.97 toks/s, output: 2.13 toks/s]
Processed prompts:   9%|▊         | 261/3051 [02:02<21:54,  2.12it/s, est. speed input: 69635.99 toks/s, output: 2.13 toks/s]
Processed prompts:   9%|▊         | 262/3051 [02:03<21:58,  2.12it/s, est. speed input: 69632.61 toks/s, output: 2.13 toks/s]
Processed prompts:   9%|▊         | 263/3051 [02:03<21:57,  2.12it/s, est. speed input: 69631.72 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 264/3051 [02:04<21:55,  2.12it/s, est. speed input: 69631.71 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 265/3051 [02:04<21:53,  2.12it/s, est. speed input: 69631.82 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 266/3051 [02:05<21:52,  2.12it/s, est. speed input: 69631.90 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 267/3051 [02:05<21:51,  2.12it/s, est. speed input: 69631.72 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 268/3051 [02:06<21:51,  2.12it/s, est. speed input: 69631.38 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 269/3051 [02:06<21:51,  2.12it/s, est. speed input: 69630.71 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 270/3051 [02:07<21:50,  2.12it/s, est. speed input: 69630.47 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 271/3051 [02:07<21:49,  2.12it/s, est. speed input: 69630.70 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 272/3051 [02:08<21:53,  2.12it/s, est. speed input: 69627.70 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 273/3051 [02:08<21:51,  2.12it/s, est. speed input: 69627.17 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 274/3051 [02:08<21:49,  2.12it/s, est. speed input: 69627.62 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 275/3051 [02:09<21:48,  2.12it/s, est. speed input: 69627.39 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 276/3051 [02:09<21:48,  2.12it/s, est. speed input: 69626.76 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 277/3051 [02:10<21:47,  2.12it/s, est. speed input: 69626.49 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 278/3051 [02:10<21:46,  2.12it/s, est. speed input: 69626.68 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 279/3051 [02:11<21:46,  2.12it/s, est. speed input: 69626.08 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 280/3051 [02:11<21:45,  2.12it/s, est. speed input: 69625.98 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 281/3051 [02:12<21:44,  2.12it/s, est. speed input: 69626.04 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 282/3051 [02:12<21:45,  2.12it/s, est. speed input: 69624.95 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 283/3051 [02:13<21:46,  2.12it/s, est. speed input: 69623.81 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 284/3051 [02:13<21:44,  2.12it/s, est. speed input: 69624.18 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 285/3051 [02:14<21:43,  2.12it/s, est. speed input: 69623.71 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 286/3051 [02:14<21:43,  2.12it/s, est. speed input: 69623.30 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 287/3051 [02:15<21:41,  2.12it/s, est. speed input: 69623.50 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 288/3051 [02:15<21:41,  2.12it/s, est. speed input: 69623.16 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 289/3051 [02:16<21:42,  2.12it/s, est. speed input: 69622.27 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 290/3051 [02:16<21:42,  2.12it/s, est. speed input: 69621.61 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 291/3051 [02:16<21:41,  2.12it/s, est. speed input: 69621.52 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 292/3051 [02:17<21:40,  2.12it/s, est. speed input: 69621.28 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 293/3051 [02:17<21:40,  2.12it/s, est. speed input: 69620.58 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 294/3051 [02:18<21:39,  2.12it/s, est. speed input: 69620.49 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 295/3051 [02:18<21:40,  2.12it/s, est. speed input: 69619.49 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 296/3051 [02:19<21:39,  2.12it/s, est. speed input: 69618.78 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 297/3051 [02:19<21:38,  2.12it/s, est. speed input: 69618.52 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 298/3051 [02:20<21:38,  2.12it/s, est. speed input: 69618.23 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 299/3051 [02:20<21:36,  2.12it/s, est. speed input: 69618.24 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 300/3051 [02:21<21:35,  2.12it/s, est. speed input: 69618.28 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 301/3051 [02:21<21:35,  2.12it/s, est. speed input: 69618.05 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 302/3051 [02:22<21:35,  2.12it/s, est. speed input: 69617.58 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 303/3051 [02:22<21:34,  2.12it/s, est. speed input: 69617.48 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 304/3051 [02:23<21:34,  2.12it/s, est. speed input: 69617.22 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 305/3051 [02:23<21:35,  2.12it/s, est. speed input: 69616.12 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 306/3051 [02:24<21:35,  2.12it/s, est. speed input: 69615.12 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 307/3051 [02:24<21:34,  2.12it/s, est. speed input: 69615.02 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 308/3051 [02:24<21:33,  2.12it/s, est. speed input: 69614.60 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 309/3051 [02:25<21:33,  2.12it/s, est. speed input: 69613.94 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 310/3051 [02:25<21:32,  2.12it/s, est. speed input: 69613.99 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 311/3051 [02:26<21:31,  2.12it/s, est. speed input: 69613.83 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 312/3051 [02:26<21:35,  2.11it/s, est. speed input: 69610.92 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 313/3051 [02:27<21:33,  2.12it/s, est. speed input: 69610.79 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 314/3051 [02:27<21:30,  2.12it/s, est. speed input: 69611.22 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 315/3051 [02:28<21:31,  2.12it/s, est. speed input: 69610.04 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 316/3051 [02:28<21:30,  2.12it/s, est. speed input: 69609.93 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 317/3051 [02:29<21:29,  2.12it/s, est. speed input: 69609.90 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 318/3051 [02:29<21:27,  2.12it/s, est. speed input: 69609.89 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 319/3051 [02:30<21:29,  2.12it/s, est. speed input: 69608.48 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 320/3051 [02:30<21:28,  2.12it/s, est. speed input: 69608.31 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 321/3051 [02:31<21:29,  2.12it/s, est. speed input: 69606.94 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 322/3051 [02:31<21:28,  2.12it/s, est. speed input: 69606.67 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 323/3051 [02:32<21:26,  2.12it/s, est. speed input: 69606.61 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 324/3051 [02:32<21:24,  2.12it/s, est. speed input: 69606.94 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 325/3051 [02:32<21:25,  2.12it/s, est. speed input: 69606.01 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 326/3051 [02:33<21:25,  2.12it/s, est. speed input: 69605.29 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 327/3051 [02:33<21:24,  2.12it/s, est. speed input: 69605.40 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 328/3051 [02:34<21:23,  2.12it/s, est. speed input: 69605.54 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 329/3051 [02:34<21:21,  2.12it/s, est. speed input: 69605.65 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 330/3051 [02:35<21:21,  2.12it/s, est. speed input: 69605.29 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 331/3051 [02:35<21:20,  2.12it/s, est. speed input: 69605.57 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 332/3051 [02:36<21:20,  2.12it/s, est. speed input: 69605.21 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 333/3051 [02:36<21:20,  2.12it/s, est. speed input: 69605.25 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 334/3051 [02:37<21:19,  2.12it/s, est. speed input: 69605.20 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 335/3051 [02:37<21:20,  2.12it/s, est. speed input: 69604.07 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 336/3051 [02:38<21:21,  2.12it/s, est. speed input: 69603.25 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 337/3051 [02:38<21:19,  2.12it/s, est. speed input: 69603.22 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 338/3051 [02:39<21:20,  2.12it/s, est. speed input: 69602.51 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 339/3051 [02:39<21:19,  2.12it/s, est. speed input: 69602.05 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 340/3051 [02:40<21:18,  2.12it/s, est. speed input: 69601.89 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 341/3051 [02:40<21:17,  2.12it/s, est. speed input: 69601.92 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 342/3051 [02:41<21:17,  2.12it/s, est. speed input: 69601.42 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 343/3051 [02:41<21:16,  2.12it/s, est. speed input: 69601.29 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 344/3051 [02:41<21:15,  2.12it/s, est. speed input: 69601.49 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 345/3051 [02:42<21:14,  2.12it/s, est. speed input: 69601.29 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 346/3051 [02:42<21:18,  2.12it/s, est. speed input: 69599.03 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 347/3051 [02:43<21:18,  2.11it/s, est. speed input: 69597.84 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 348/3051 [02:43<21:17,  2.12it/s, est. speed input: 69597.11 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 349/3051 [02:44<21:16,  2.12it/s, est. speed input: 69596.94 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 350/3051 [02:44<21:14,  2.12it/s, est. speed input: 69597.10 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 351/3051 [02:45<21:16,  2.12it/s, est. speed input: 69595.38 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 352/3051 [02:45<21:15,  2.12it/s, est. speed input: 69594.65 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 353/3051 [02:46<21:13,  2.12it/s, est. speed input: 69594.63 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 354/3051 [02:46<21:12,  2.12it/s, est. speed input: 69594.27 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 355/3051 [02:47<21:12,  2.12it/s, est. speed input: 69593.59 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 356/3051 [02:47<21:12,  2.12it/s, est. speed input: 69593.23 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 357/3051 [02:48<21:11,  2.12it/s, est. speed input: 69592.97 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 358/3051 [02:48<21:15,  2.11it/s, est. speed input: 69590.18 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 359/3051 [02:49<21:13,  2.11it/s, est. speed input: 69589.75 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 360/3051 [02:49<21:11,  2.12it/s, est. speed input: 69589.65 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 361/3051 [02:49<21:10,  2.12it/s, est. speed input: 69589.07 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 362/3051 [02:50<21:10,  2.12it/s, est. speed input: 69588.59 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 363/3051 [02:50<21:08,  2.12it/s, est. speed input: 69588.76 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 364/3051 [02:51<21:06,  2.12it/s, est. speed input: 69588.93 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 365/3051 [02:51<21:06,  2.12it/s, est. speed input: 69588.54 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 366/3051 [02:52<21:05,  2.12it/s, est. speed input: 69588.71 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 367/3051 [02:52<21:07,  2.12it/s, est. speed input: 69587.23 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 368/3051 [02:53<21:05,  2.12it/s, est. speed input: 69587.44 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 369/3051 [02:53<21:03,  2.12it/s, est. speed input: 69587.69 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 370/3051 [02:54<21:03,  2.12it/s, est. speed input: 69587.44 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 371/3051 [02:54<21:02,  2.12it/s, est. speed input: 69587.62 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 372/3051 [02:55<21:01,  2.12it/s, est. speed input: 69587.55 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 373/3051 [02:55<21:03,  2.12it/s, est. speed input: 69586.66 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 374/3051 [02:56<21:04,  2.12it/s, est. speed input: 69585.68 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 375/3051 [02:56<21:03,  2.12it/s, est. speed input: 69585.22 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 376/3051 [02:57<21:02,  2.12it/s, est. speed input: 69585.18 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 377/3051 [02:57<21:04,  2.11it/s, est. speed input: 69583.27 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 378/3051 [02:58<21:03,  2.11it/s, est. speed input: 69582.68 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 379/3051 [02:58<21:02,  2.12it/s, est. speed input: 69582.51 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 380/3051 [02:58<21:01,  2.12it/s, est. speed input: 69582.05 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 381/3051 [02:59<21:00,  2.12it/s, est. speed input: 69581.95 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 382/3051 [02:59<21:00,  2.12it/s, est. speed input: 69581.31 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 383/3051 [03:00<20:59,  2.12it/s, est. speed input: 69580.95 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 384/3051 [03:00<20:59,  2.12it/s, est. speed input: 69580.38 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 385/3051 [03:01<21:02,  2.11it/s, est. speed input: 69578.01 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 386/3051 [03:01<20:59,  2.12it/s, est. speed input: 69578.14 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 387/3051 [03:02<20:57,  2.12it/s, est. speed input: 69578.19 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 388/3051 [03:02<20:56,  2.12it/s, est. speed input: 69578.20 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 389/3051 [03:03<20:54,  2.12it/s, est. speed input: 69578.48 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 390/3051 [03:03<20:55,  2.12it/s, est. speed input: 69577.63 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 391/3051 [03:04<20:55,  2.12it/s, est. speed input: 69577.32 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 392/3051 [03:04<20:54,  2.12it/s, est. speed input: 69577.00 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 393/3051 [03:05<20:53,  2.12it/s, est. speed input: 69577.15 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 394/3051 [03:05<20:54,  2.12it/s, est. speed input: 69576.24 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 395/3051 [03:06<20:52,  2.12it/s, est. speed input: 69576.52 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 396/3051 [03:06<20:51,  2.12it/s, est. speed input: 69576.63 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 397/3051 [03:06<20:50,  2.12it/s, est. speed input: 69576.58 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 398/3051 [03:07<20:50,  2.12it/s, est. speed input: 69576.26 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 399/3051 [03:07<20:50,  2.12it/s, est. speed input: 69576.03 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 400/3051 [03:08<20:53,  2.12it/s, est. speed input: 69574.30 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 401/3051 [03:08<20:52,  2.12it/s, est. speed input: 69573.64 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 402/3051 [03:09<20:50,  2.12it/s, est. speed input: 69573.73 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 403/3051 [03:09<20:49,  2.12it/s, est. speed input: 69573.62 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 404/3051 [03:10<20:48,  2.12it/s, est. speed input: 69573.37 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 405/3051 [03:10<20:48,  2.12it/s, est. speed input: 69572.88 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 406/3051 [03:11<20:47,  2.12it/s, est. speed input: 69573.08 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 407/3051 [03:11<20:49,  2.12it/s, est. speed input: 69571.78 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 408/3051 [03:12<20:48,  2.12it/s, est. speed input: 69571.33 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 409/3051 [03:12<20:46,  2.12it/s, est. speed input: 69571.59 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 410/3051 [03:13<20:44,  2.12it/s, est. speed input: 69571.76 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 411/3051 [03:13<20:47,  2.12it/s, est. speed input: 69570.00 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 412/3051 [03:14<20:46,  2.12it/s, est. speed input: 69569.93 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 413/3051 [03:14<20:44,  2.12it/s, est. speed input: 69570.09 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 414/3051 [03:15<20:50,  2.11it/s, est. speed input: 69567.05 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 415/3051 [03:15<20:49,  2.11it/s, est. speed input: 69565.94 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 416/3051 [03:15<20:48,  2.11it/s, est. speed input: 69565.42 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 417/3051 [03:16<20:49,  2.11it/s, est. speed input: 69563.51 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 418/3051 [03:16<20:47,  2.11it/s, est. speed input: 69563.00 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 419/3051 [03:17<20:44,  2.11it/s, est. speed input: 69563.10 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 420/3051 [03:17<20:43,  2.12it/s, est. speed input: 69562.61 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 421/3051 [03:18<20:42,  2.12it/s, est. speed input: 69562.26 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 422/3051 [03:18<20:42,  2.12it/s, est. speed input: 69561.75 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 423/3051 [03:19<20:43,  2.11it/s, est. speed input: 69560.56 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 424/3051 [03:19<20:44,  2.11it/s, est. speed input: 69559.04 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 425/3051 [03:20<20:43,  2.11it/s, est. speed input: 69558.25 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 426/3051 [03:20<20:42,  2.11it/s, est. speed input: 69557.89 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 427/3051 [03:21<20:41,  2.11it/s, est. speed input: 69557.45 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 428/3051 [03:21<20:40,  2.11it/s, est. speed input: 69556.63 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 429/3051 [03:22<20:39,  2.12it/s, est. speed input: 69556.46 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 430/3051 [03:22<20:38,  2.12it/s, est. speed input: 69555.86 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 431/3051 [03:23<20:38,  2.11it/s, est. speed input: 69555.14 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 432/3051 [03:23<20:42,  2.11it/s, est. speed input: 69552.87 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 433/3051 [03:23<20:39,  2.11it/s, est. speed input: 69552.88 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 434/3051 [03:24<20:38,  2.11it/s, est. speed input: 69552.34 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 435/3051 [03:24<20:37,  2.11it/s, est. speed input: 69551.88 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 436/3051 [03:25<20:36,  2.12it/s, est. speed input: 69551.51 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 437/3051 [03:25<20:35,  2.12it/s, est. speed input: 69550.97 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 438/3051 [03:26<20:34,  2.12it/s, est. speed input: 69550.70 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 439/3051 [03:26<20:34,  2.12it/s, est. speed input: 69550.22 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 440/3051 [03:27<20:36,  2.11it/s, est. speed input: 69548.53 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 441/3051 [03:27<20:34,  2.11it/s, est. speed input: 69548.22 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 442/3051 [03:28<20:33,  2.12it/s, est. speed input: 69548.00 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 443/3051 [03:28<20:31,  2.12it/s, est. speed input: 69547.96 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 444/3051 [03:29<20:30,  2.12it/s, est. speed input: 69547.84 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 445/3051 [03:29<20:53,  2.08it/s, est. speed input: 69537.60 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 446/3051 [03:30<20:45,  2.09it/s, est. speed input: 69537.52 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 447/3051 [03:30<20:39,  2.10it/s, est. speed input: 69537.70 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 448/3051 [03:31<20:40,  2.10it/s, est. speed input: 69535.50 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 449/3051 [03:31<20:36,  2.10it/s, est. speed input: 69535.29 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 450/3051 [03:32<20:33,  2.11it/s, est. speed input: 69534.99 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 451/3051 [03:32<20:31,  2.11it/s, est. speed input: 69534.62 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 452/3051 [03:33<20:29,  2.11it/s, est. speed input: 69534.35 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 453/3051 [03:33<20:29,  2.11it/s, est. speed input: 69533.57 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 454/3051 [03:33<20:28,  2.11it/s, est. speed input: 69533.24 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 455/3051 [03:34<20:26,  2.12it/s, est. speed input: 69533.25 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 456/3051 [03:34<20:25,  2.12it/s, est. speed input: 69532.94 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 457/3051 [03:35<20:24,  2.12it/s, est. speed input: 69532.90 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 458/3051 [03:35<20:28,  2.11it/s, est. speed input: 69530.88 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 459/3051 [03:36<20:27,  2.11it/s, est. speed input: 69530.25 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 460/3051 [03:36<20:25,  2.11it/s, est. speed input: 69530.18 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 461/3051 [03:37<20:23,  2.12it/s, est. speed input: 69530.19 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 462/3051 [03:37<20:23,  2.12it/s, est. speed input: 69529.64 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 463/3051 [03:38<20:22,  2.12it/s, est. speed input: 69529.56 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 464/3051 [03:38<20:21,  2.12it/s, est. speed input: 69529.54 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 465/3051 [03:39<20:24,  2.11it/s, est. speed input: 69527.85 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 466/3051 [03:39<20:22,  2.11it/s, est. speed input: 69527.51 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 467/3051 [03:40<20:21,  2.12it/s, est. speed input: 69527.42 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 468/3051 [03:40<20:21,  2.11it/s, est. speed input: 69526.51 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 469/3051 [03:41<20:21,  2.11it/s, est. speed input: 69525.99 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 470/3051 [03:41<20:20,  2.11it/s, est. speed input: 69525.53 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 471/3051 [03:41<20:19,  2.12it/s, est. speed input: 69525.42 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 472/3051 [03:42<20:19,  2.12it/s, est. speed input: 69524.92 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 473/3051 [03:42<20:18,  2.12it/s, est. speed input: 69524.57 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 474/3051 [03:43<20:16,  2.12it/s, est. speed input: 69524.58 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 475/3051 [03:43<20:17,  2.12it/s, est. speed input: 69523.82 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 476/3051 [03:44<20:16,  2.12it/s, est. speed input: 69523.55 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 477/3051 [03:44<20:50,  2.06it/s, est. speed input: 69509.55 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 478/3051 [03:45<20:38,  2.08it/s, est. speed input: 69509.60 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 479/3051 [03:45<20:30,  2.09it/s, est. speed input: 69509.70 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 480/3051 [03:46<20:26,  2.10it/s, est. speed input: 69508.82 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 481/3051 [03:46<20:23,  2.10it/s, est. speed input: 69508.28 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 482/3051 [03:47<20:19,  2.11it/s, est. speed input: 69508.29 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 483/3051 [03:47<20:20,  2.10it/s, est. speed input: 69506.54 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 484/3051 [03:48<20:17,  2.11it/s, est. speed input: 69506.57 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 485/3051 [03:48<20:13,  2.11it/s, est. speed input: 69506.87 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 486/3051 [03:49<20:11,  2.12it/s, est. speed input: 69507.01 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 487/3051 [03:49<20:11,  2.12it/s, est. speed input: 69506.72 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 488/3051 [03:50<20:11,  2.12it/s, est. speed input: 69506.32 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 489/3051 [03:50<20:09,  2.12it/s, est. speed input: 69506.29 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 490/3051 [03:51<20:09,  2.12it/s, est. speed input: 69506.15 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 491/3051 [03:51<20:09,  2.12it/s, est. speed input: 69505.59 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 492/3051 [03:51<20:08,  2.12it/s, est. speed input: 69505.41 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 493/3051 [03:52<20:10,  2.11it/s, est. speed input: 69504.34 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 494/3051 [03:52<20:10,  2.11it/s, est. speed input: 69503.56 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 495/3051 [03:53<20:09,  2.11it/s, est. speed input: 69503.33 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 496/3051 [03:53<20:07,  2.12it/s, est. speed input: 69503.21 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 497/3051 [03:54<20:10,  2.11it/s, est. speed input: 69501.71 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 498/3051 [03:54<20:08,  2.11it/s, est. speed input: 69501.54 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 499/3051 [03:55<20:06,  2.12it/s, est. speed input: 69501.63 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 500/3051 [03:55<20:05,  2.12it/s, est. speed input: 69501.35 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 501/3051 [03:56<20:04,  2.12it/s, est. speed input: 69501.18 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 502/3051 [03:56<20:03,  2.12it/s, est. speed input: 69501.11 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 503/3051 [03:57<20:02,  2.12it/s, est. speed input: 69501.12 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 504/3051 [03:57<20:02,  2.12it/s, est. speed input: 69500.75 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 505/3051 [03:58<20:01,  2.12it/s, est. speed input: 69500.87 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 506/3051 [03:58<19:59,  2.12it/s, est. speed input: 69501.22 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 507/3051 [03:59<19:59,  2.12it/s, est. speed input: 69501.29 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 508/3051 [03:59<19:59,  2.12it/s, est. speed input: 69501.08 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 509/3051 [03:59<19:58,  2.12it/s, est. speed input: 69501.03 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 510/3051 [04:00<19:58,  2.12it/s, est. speed input: 69500.96 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 511/3051 [04:00<19:58,  2.12it/s, est. speed input: 69500.57 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 512/3051 [04:01<19:59,  2.12it/s, est. speed input: 69499.99 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 513/3051 [04:01<19:58,  2.12it/s, est. speed input: 69499.75 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 514/3051 [04:02<19:58,  2.12it/s, est. speed input: 69499.64 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 515/3051 [04:02<19:57,  2.12it/s, est. speed input: 69499.65 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 516/3051 [04:03<19:57,  2.12it/s, est. speed input: 69499.23 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 517/3051 [04:03<19:56,  2.12it/s, est. speed input: 69498.94 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 518/3051 [04:04<19:56,  2.12it/s, est. speed input: 69498.59 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 519/3051 [04:04<19:55,  2.12it/s, est. speed input: 69498.75 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 520/3051 [04:05<19:55,  2.12it/s, est. speed input: 69498.36 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 521/3051 [04:05<19:55,  2.12it/s, est. speed input: 69497.81 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 522/3051 [04:06<19:54,  2.12it/s, est. speed input: 69497.64 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 523/3051 [04:06<19:54,  2.12it/s, est. speed input: 69497.50 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 524/3051 [04:07<19:53,  2.12it/s, est. speed input: 69497.27 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 525/3051 [04:07<19:52,  2.12it/s, est. speed input: 69497.30 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 526/3051 [04:08<19:52,  2.12it/s, est. speed input: 69497.02 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 527/3051 [04:08<19:53,  2.11it/s, est. speed input: 69496.03 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 528/3051 [04:08<19:52,  2.12it/s, est. speed input: 69496.04 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 529/3051 [04:09<19:51,  2.12it/s, est. speed input: 69495.82 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 530/3051 [04:09<19:54,  2.11it/s, est. speed input: 69494.45 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 531/3051 [04:10<19:51,  2.11it/s, est. speed input: 69494.61 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 532/3051 [04:10<19:51,  2.11it/s, est. speed input: 69494.09 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 533/3051 [04:11<19:50,  2.11it/s, est. speed input: 69493.78 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 534/3051 [04:11<19:50,  2.11it/s, est. speed input: 69493.18 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 535/3051 [04:12<19:51,  2.11it/s, est. speed input: 69492.42 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 536/3051 [04:12<19:50,  2.11it/s, est. speed input: 69491.98 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 537/3051 [04:13<19:48,  2.12it/s, est. speed input: 69492.00 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 538/3051 [04:13<19:47,  2.12it/s, est. speed input: 69491.97 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 539/3051 [04:14<19:45,  2.12it/s, est. speed input: 69492.08 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 540/3051 [04:14<19:44,  2.12it/s, est. speed input: 69492.19 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 541/3051 [04:15<19:44,  2.12it/s, est. speed input: 69492.14 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 542/3051 [04:15<19:43,  2.12it/s, est. speed input: 69492.17 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 543/3051 [04:16<19:42,  2.12it/s, est. speed input: 69492.16 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 544/3051 [04:16<19:42,  2.12it/s, est. speed input: 69492.08 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 545/3051 [04:16<19:43,  2.12it/s, est. speed input: 69491.52 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 546/3051 [04:17<19:43,  2.12it/s, est. speed input: 69491.31 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 547/3051 [04:17<19:42,  2.12it/s, est. speed input: 69491.26 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 548/3051 [04:18<19:40,  2.12it/s, est. speed input: 69491.42 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 549/3051 [04:18<19:39,  2.12it/s, est. speed input: 69491.63 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 550/3051 [04:19<19:41,  2.12it/s, est. speed input: 69490.85 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 551/3051 [04:19<19:39,  2.12it/s, est. speed input: 69490.98 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 552/3051 [04:20<19:41,  2.11it/s, est. speed input: 69489.96 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 553/3051 [04:20<19:40,  2.12it/s, est. speed input: 69489.93 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 554/3051 [04:21<19:38,  2.12it/s, est. speed input: 69490.22 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 555/3051 [04:21<19:41,  2.11it/s, est. speed input: 69489.01 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 556/3051 [04:22<19:39,  2.12it/s, est. speed input: 69488.98 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 557/3051 [04:22<19:37,  2.12it/s, est. speed input: 69489.21 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 558/3051 [04:23<19:37,  2.12it/s, est. speed input: 69489.03 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 559/3051 [04:23<19:36,  2.12it/s, est. speed input: 69488.80 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 560/3051 [04:24<19:36,  2.12it/s, est. speed input: 69488.64 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 561/3051 [04:24<19:35,  2.12it/s, est. speed input: 69488.73 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 562/3051 [04:25<19:34,  2.12it/s, est. speed input: 69488.56 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 563/3051 [04:25<19:34,  2.12it/s, est. speed input: 69488.41 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 564/3051 [04:25<19:33,  2.12it/s, est. speed input: 69488.56 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 565/3051 [04:26<19:32,  2.12it/s, est. speed input: 69488.74 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 566/3051 [04:26<19:35,  2.11it/s, est. speed input: 69487.52 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 567/3051 [04:27<19:33,  2.12it/s, est. speed input: 69487.50 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 568/3051 [04:27<19:31,  2.12it/s, est. speed input: 69487.74 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 569/3051 [04:28<19:30,  2.12it/s, est. speed input: 69487.95 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 570/3051 [04:28<19:29,  2.12it/s, est. speed input: 69488.19 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 571/3051 [04:29<19:28,  2.12it/s, est. speed input: 69488.28 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 572/3051 [04:29<19:28,  2.12it/s, est. speed input: 69488.19 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 573/3051 [04:30<19:31,  2.11it/s, est. speed input: 69486.95 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 574/3051 [04:30<19:30,  2.12it/s, est. speed input: 69486.81 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 575/3051 [04:31<19:29,  2.12it/s, est. speed input: 69486.76 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 576/3051 [04:31<19:29,  2.12it/s, est. speed input: 69486.60 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 577/3051 [04:32<19:27,  2.12it/s, est. speed input: 69486.83 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 578/3051 [04:32<19:27,  2.12it/s, est. speed input: 69486.52 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 579/3051 [04:33<19:28,  2.12it/s, est. speed input: 69486.01 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 580/3051 [04:33<19:26,  2.12it/s, est. speed input: 69486.06 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 581/3051 [04:33<19:25,  2.12it/s, est. speed input: 69486.18 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 582/3051 [04:34<19:28,  2.11it/s, est. speed input: 69484.94 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 583/3051 [04:34<19:26,  2.11it/s, est. speed input: 69484.87 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 584/3051 [04:35<19:25,  2.12it/s, est. speed input: 69484.87 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 585/3051 [04:35<19:24,  2.12it/s, est. speed input: 69484.79 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 586/3051 [04:36<19:24,  2.12it/s, est. speed input: 69484.57 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 587/3051 [04:36<19:22,  2.12it/s, est. speed input: 69484.88 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 588/3051 [04:37<19:21,  2.12it/s, est. speed input: 69484.98 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 589/3051 [04:37<19:24,  2.11it/s, est. speed input: 69483.73 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 590/3051 [04:38<19:22,  2.12it/s, est. speed input: 69483.91 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 591/3051 [04:38<19:20,  2.12it/s, est. speed input: 69484.19 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 592/3051 [04:39<19:20,  2.12it/s, est. speed input: 69484.10 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 593/3051 [04:39<19:20,  2.12it/s, est. speed input: 69483.96 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 594/3051 [04:40<19:18,  2.12it/s, est. speed input: 69484.20 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 595/3051 [04:40<19:18,  2.12it/s, est. speed input: 69484.09 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 596/3051 [04:41<19:21,  2.11it/s, est. speed input: 69482.97 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 597/3051 [04:41<19:18,  2.12it/s, est. speed input: 69483.27 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 598/3051 [04:42<19:18,  2.12it/s, est. speed input: 69483.22 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 599/3051 [04:42<19:18,  2.12it/s, est. speed input: 69482.86 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 600/3051 [04:42<19:16,  2.12it/s, est. speed input: 69483.17 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 601/3051 [04:43<19:15,  2.12it/s, est. speed input: 69483.11 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 602/3051 [04:43<19:16,  2.12it/s, est. speed input: 69482.79 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 603/3051 [04:44<19:15,  2.12it/s, est. speed input: 69482.80 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 604/3051 [04:44<19:15,  2.12it/s, est. speed input: 69482.60 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 605/3051 [04:45<19:14,  2.12it/s, est. speed input: 69482.61 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 606/3051 [04:45<19:13,  2.12it/s, est. speed input: 69482.59 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 607/3051 [04:46<19:13,  2.12it/s, est. speed input: 69482.39 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 608/3051 [04:46<19:12,  2.12it/s, est. speed input: 69482.37 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 609/3051 [04:47<19:12,  2.12it/s, est. speed input: 69482.43 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 610/3051 [04:47<19:10,  2.12it/s, est. speed input: 69482.57 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 611/3051 [04:48<19:10,  2.12it/s, est. speed input: 69482.73 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 612/3051 [04:48<19:10,  2.12it/s, est. speed input: 69482.63 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 613/3051 [04:49<19:09,  2.12it/s, est. speed input: 69482.70 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 614/3051 [04:49<19:10,  2.12it/s, est. speed input: 69482.24 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 615/3051 [04:50<19:10,  2.12it/s, est. speed input: 69482.13 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 616/3051 [04:50<19:09,  2.12it/s, est. speed input: 69482.02 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 617/3051 [04:50<19:08,  2.12it/s, est. speed input: 69482.16 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 618/3051 [04:51<19:08,  2.12it/s, est. speed input: 69481.82 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 619/3051 [04:51<19:11,  2.11it/s, est. speed input: 69480.64 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 620/3051 [04:52<19:09,  2.12it/s, est. speed input: 69480.76 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 621/3051 [04:52<19:11,  2.11it/s, est. speed input: 69479.53 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 622/3051 [04:53<19:09,  2.11it/s, est. speed input: 69479.40 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 623/3051 [04:53<19:07,  2.12it/s, est. speed input: 69479.53 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 624/3051 [04:54<19:06,  2.12it/s, est. speed input: 69479.46 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 625/3051 [04:54<19:05,  2.12it/s, est. speed input: 69479.39 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 626/3051 [04:55<19:04,  2.12it/s, est. speed input: 69479.64 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 627/3051 [04:55<19:02,  2.12it/s, est. speed input: 69479.81 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 628/3051 [04:56<19:03,  2.12it/s, est. speed input: 69479.65 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 629/3051 [04:56<19:02,  2.12it/s, est. speed input: 69479.63 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 630/3051 [04:57<19:01,  2.12it/s, est. speed input: 69479.72 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 631/3051 [04:57<19:01,  2.12it/s, est. speed input: 69479.71 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 632/3051 [04:58<19:01,  2.12it/s, est. speed input: 69479.54 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 633/3051 [04:58<19:00,  2.12it/s, est. speed input: 69479.55 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 634/3051 [04:59<19:00,  2.12it/s, est. speed input: 69479.57 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 635/3051 [04:59<19:00,  2.12it/s, est. speed input: 69479.35 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 636/3051 [04:59<18:59,  2.12it/s, est. speed input: 69479.36 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 637/3051 [05:00<18:58,  2.12it/s, est. speed input: 69479.52 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 638/3051 [05:00<18:58,  2.12it/s, est. speed input: 69479.23 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 639/3051 [05:01<18:58,  2.12it/s, est. speed input: 69479.04 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 640/3051 [05:01<18:57,  2.12it/s, est. speed input: 69479.26 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 641/3051 [05:02<18:57,  2.12it/s, est. speed input: 69479.11 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 642/3051 [05:02<18:55,  2.12it/s, est. speed input: 69479.46 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 643/3051 [05:03<18:55,  2.12it/s, est. speed input: 69479.37 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 644/3051 [05:03<18:58,  2.11it/s, est. speed input: 69478.36 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 645/3051 [05:04<18:56,  2.12it/s, est. speed input: 69478.45 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 646/3051 [05:04<18:55,  2.12it/s, est. speed input: 69478.45 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 647/3051 [05:05<18:54,  2.12it/s, est. speed input: 69478.50 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 648/3051 [05:05<18:55,  2.12it/s, est. speed input: 69477.93 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 649/3051 [05:06<18:53,  2.12it/s, est. speed input: 69478.16 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 650/3051 [05:06<18:53,  2.12it/s, est. speed input: 69478.13 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 651/3051 [05:07<18:52,  2.12it/s, est. speed input: 69478.12 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 652/3051 [05:07<18:51,  2.12it/s, est. speed input: 69478.24 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 653/3051 [05:07<18:50,  2.12it/s, est. speed input: 69478.43 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 654/3051 [05:08<18:50,  2.12it/s, est. speed input: 69478.40 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 655/3051 [05:08<18:49,  2.12it/s, est. speed input: 69478.48 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 656/3051 [05:09<18:50,  2.12it/s, est. speed input: 69477.99 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 657/3051 [05:09<18:50,  2.12it/s, est. speed input: 69477.90 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 658/3051 [05:10<18:49,  2.12it/s, est. speed input: 69477.72 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 659/3051 [05:10<18:49,  2.12it/s, est. speed input: 69477.73 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 660/3051 [05:11<18:47,  2.12it/s, est. speed input: 69477.97 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 661/3051 [05:11<18:47,  2.12it/s, est. speed input: 69477.91 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 662/3051 [05:12<18:46,  2.12it/s, est. speed input: 69478.13 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 663/3051 [05:12<18:52,  2.11it/s, est. speed input: 69475.92 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 664/3051 [05:13<18:50,  2.11it/s, est. speed input: 69475.99 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 665/3051 [05:13<18:47,  2.12it/s, est. speed input: 69476.19 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 666/3051 [05:14<18:47,  2.11it/s, est. speed input: 69475.84 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 667/3051 [05:14<18:46,  2.12it/s, est. speed input: 69475.81 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 668/3051 [05:15<18:45,  2.12it/s, est. speed input: 69475.83 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 669/3051 [05:15<18:44,  2.12it/s, est. speed input: 69475.89 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 670/3051 [05:16<18:44,  2.12it/s, est. speed input: 69475.67 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 671/3051 [05:16<18:45,  2.11it/s, est. speed input: 69475.02 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 672/3051 [05:16<18:43,  2.12it/s, est. speed input: 69475.29 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 673/3051 [05:17<18:43,  2.12it/s, est. speed input: 69474.84 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 674/3051 [05:17<18:43,  2.12it/s, est. speed input: 69474.53 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 675/3051 [05:18<18:42,  2.12it/s, est. speed input: 69474.40 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 676/3051 [05:18<18:41,  2.12it/s, est. speed input: 69474.60 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 677/3051 [05:19<18:40,  2.12it/s, est. speed input: 69474.51 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 678/3051 [05:19<18:41,  2.12it/s, est. speed input: 69474.11 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 679/3051 [05:20<18:40,  2.12it/s, est. speed input: 69474.13 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 680/3051 [05:20<18:42,  2.11it/s, est. speed input: 69473.19 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 681/3051 [05:21<18:40,  2.12it/s, est. speed input: 69473.29 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 682/3051 [05:21<18:38,  2.12it/s, est. speed input: 69473.50 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 683/3051 [05:22<18:37,  2.12it/s, est. speed input: 69473.41 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 684/3051 [05:22<18:37,  2.12it/s, est. speed input: 69473.29 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 685/3051 [05:23<18:36,  2.12it/s, est. speed input: 69473.46 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 686/3051 [05:23<18:35,  2.12it/s, est. speed input: 69473.47 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 687/3051 [05:24<18:38,  2.11it/s, est. speed input: 69472.35 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 688/3051 [05:24<18:36,  2.12it/s, est. speed input: 69472.42 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 689/3051 [05:24<18:35,  2.12it/s, est. speed input: 69472.65 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 690/3051 [05:25<18:34,  2.12it/s, est. speed input: 69472.67 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 691/3051 [05:25<18:34,  2.12it/s, est. speed input: 69472.30 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 692/3051 [05:26<18:34,  2.12it/s, est. speed input: 69472.00 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 693/3051 [05:26<18:34,  2.12it/s, est. speed input: 69471.90 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 694/3051 [05:27<18:32,  2.12it/s, est. speed input: 69472.12 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 695/3051 [05:27<18:31,  2.12it/s, est. speed input: 69472.26 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 696/3051 [05:28<18:32,  2.12it/s, est. speed input: 69471.65 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 697/3051 [05:28<18:31,  2.12it/s, est. speed input: 69471.64 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 698/3051 [05:29<18:31,  2.12it/s, est. speed input: 69471.33 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 699/3051 [05:29<18:31,  2.12it/s, est. speed input: 69471.13 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 700/3051 [05:30<18:29,  2.12it/s, est. speed input: 69471.28 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 701/3051 [05:30<18:29,  2.12it/s, est. speed input: 69471.25 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 702/3051 [05:31<18:29,  2.12it/s, est. speed input: 69471.10 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 703/3051 [05:31<18:28,  2.12it/s, est. speed input: 69471.03 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 704/3051 [05:32<18:27,  2.12it/s, est. speed input: 69471.13 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 705/3051 [05:32<18:25,  2.12it/s, est. speed input: 69471.39 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 706/3051 [05:33<18:26,  2.12it/s, est. speed input: 69471.11 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 707/3051 [05:33<18:26,  2.12it/s, est. speed input: 69470.89 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 708/3051 [05:33<18:25,  2.12it/s, est. speed input: 69470.91 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 709/3051 [05:34<18:24,  2.12it/s, est. speed input: 69471.11 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 710/3051 [05:34<18:26,  2.12it/s, est. speed input: 69470.53 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 711/3051 [05:35<18:24,  2.12it/s, est. speed input: 69470.55 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 712/3051 [05:35<18:26,  2.11it/s, est. speed input: 69469.91 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 713/3051 [05:36<18:25,  2.12it/s, est. speed input: 69469.78 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 714/3051 [05:36<18:23,  2.12it/s, est. speed input: 69469.88 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 715/3051 [05:37<18:22,  2.12it/s, est. speed input: 69469.97 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 716/3051 [05:37<18:20,  2.12it/s, est. speed input: 69470.30 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 717/3051 [05:38<18:20,  2.12it/s, est. speed input: 69470.39 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 718/3051 [05:38<18:19,  2.12it/s, est. speed input: 69470.39 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 719/3051 [05:39<18:21,  2.12it/s, est. speed input: 69469.90 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 720/3051 [05:39<18:19,  2.12it/s, est. speed input: 69470.11 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 721/3051 [05:40<18:20,  2.12it/s, est. speed input: 69469.71 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 722/3051 [05:40<18:20,  2.12it/s, est. speed input: 69469.56 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 723/3051 [05:41<18:19,  2.12it/s, est. speed input: 69469.41 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 724/3051 [05:41<18:18,  2.12it/s, est. speed input: 69469.58 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 725/3051 [05:41<18:17,  2.12it/s, est. speed input: 69469.51 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 726/3051 [05:42<18:17,  2.12it/s, est. speed input: 69469.34 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 727/3051 [05:42<18:16,  2.12it/s, est. speed input: 69469.32 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 728/3051 [05:43<18:15,  2.12it/s, est. speed input: 69469.43 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 729/3051 [05:43<18:14,  2.12it/s, est. speed input: 69469.60 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 730/3051 [05:44<18:13,  2.12it/s, est. speed input: 69469.78 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 731/3051 [05:44<18:13,  2.12it/s, est. speed input: 69469.79 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 732/3051 [05:45<18:18,  2.11it/s, est. speed input: 69468.43 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 733/3051 [05:45<18:15,  2.12it/s, est. speed input: 69468.63 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 734/3051 [05:46<18:14,  2.12it/s, est. speed input: 69468.72 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 735/3051 [05:46<18:16,  2.11it/s, est. speed input: 69467.94 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 736/3051 [05:47<18:14,  2.11it/s, est. speed input: 69467.90 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 737/3051 [05:47<18:12,  2.12it/s, est. speed input: 69468.02 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 738/3051 [05:48<18:11,  2.12it/s, est. speed input: 69468.16 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 739/3051 [05:48<18:09,  2.12it/s, est. speed input: 69468.50 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 740/3051 [05:49<18:11,  2.12it/s, est. speed input: 69468.02 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 741/3051 [05:49<18:10,  2.12it/s, est. speed input: 69467.95 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 742/3051 [05:50<18:09,  2.12it/s, est. speed input: 69467.96 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 743/3051 [05:50<18:10,  2.12it/s, est. speed input: 69467.66 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 744/3051 [05:50<18:09,  2.12it/s, est. speed input: 69467.71 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 745/3051 [05:51<18:08,  2.12it/s, est. speed input: 69467.79 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 746/3051 [05:51<18:07,  2.12it/s, est. speed input: 69467.75 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 747/3051 [05:52<18:08,  2.12it/s, est. speed input: 69467.40 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 748/3051 [05:52<18:08,  2.12it/s, est. speed input: 69467.15 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 749/3051 [05:53<18:08,  2.12it/s, est. speed input: 69466.84 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 750/3051 [05:53<18:06,  2.12it/s, est. speed input: 69467.02 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 751/3051 [05:54<18:05,  2.12it/s, est. speed input: 69467.02 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 752/3051 [05:54<18:09,  2.11it/s, est. speed input: 69465.85 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 753/3051 [05:55<18:07,  2.11it/s, est. speed input: 69465.68 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 754/3051 [05:55<18:05,  2.12it/s, est. speed input: 69465.90 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 755/3051 [05:56<18:05,  2.12it/s, est. speed input: 69465.69 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 756/3051 [05:56<18:04,  2.12it/s, est. speed input: 69465.59 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 757/3051 [05:57<18:03,  2.12it/s, est. speed input: 69465.68 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 758/3051 [05:57<18:04,  2.11it/s, est. speed input: 69465.12 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 759/3051 [05:58<18:02,  2.12it/s, est. speed input: 69465.21 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 760/3051 [05:58<18:01,  2.12it/s, est. speed input: 69465.35 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 761/3051 [05:58<18:06,  2.11it/s, est. speed input: 69463.78 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 762/3051 [05:59<18:02,  2.11it/s, est. speed input: 69464.05 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 763/3051 [05:59<18:00,  2.12it/s, est. speed input: 69464.32 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 764/3051 [06:00<17:59,  2.12it/s, est. speed input: 69464.48 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 765/3051 [06:00<18:01,  2.11it/s, est. speed input: 69463.79 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 766/3051 [06:01<17:59,  2.12it/s, est. speed input: 69463.82 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 767/3051 [06:01<18:00,  2.11it/s, est. speed input: 69463.35 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 768/3051 [06:02<18:01,  2.11it/s, est. speed input: 69462.75 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 769/3051 [06:02<17:59,  2.11it/s, est. speed input: 69462.75 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 770/3051 [06:03<17:57,  2.12it/s, est. speed input: 69462.87 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 771/3051 [06:03<17:57,  2.12it/s, est. speed input: 69462.71 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 772/3051 [06:04<17:56,  2.12it/s, est. speed input: 69462.73 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 773/3051 [06:04<17:54,  2.12it/s, est. speed input: 69462.92 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 774/3051 [06:05<17:56,  2.12it/s, est. speed input: 69462.32 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 775/3051 [06:05<17:57,  2.11it/s, est. speed input: 69461.77 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 776/3051 [06:06<17:55,  2.11it/s, est. speed input: 69461.77 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 777/3051 [06:06<17:53,  2.12it/s, est. speed input: 69461.93 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 778/3051 [06:07<17:55,  2.11it/s, est. speed input: 69461.34 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 779/3051 [06:07<17:53,  2.12it/s, est. speed input: 69461.45 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 780/3051 [06:07<17:51,  2.12it/s, est. speed input: 69461.76 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 781/3051 [06:08<17:50,  2.12it/s, est. speed input: 69461.95 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 782/3051 [06:08<17:50,  2.12it/s, est. speed input: 69461.86 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 783/3051 [06:09<17:49,  2.12it/s, est. speed input: 69462.05 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 784/3051 [06:09<17:49,  2.12it/s, est. speed input: 69461.91 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 785/3051 [06:10<17:49,  2.12it/s, est. speed input: 69461.70 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 786/3051 [06:10<17:49,  2.12it/s, est. speed input: 69461.63 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 787/3051 [06:11<17:48,  2.12it/s, est. speed input: 69461.63 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 788/3051 [06:11<17:48,  2.12it/s, est. speed input: 69461.44 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 789/3051 [06:12<17:49,  2.11it/s, est. speed input: 69460.95 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 790/3051 [06:12<17:47,  2.12it/s, est. speed input: 69461.14 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 791/3051 [06:13<17:46,  2.12it/s, est. speed input: 69461.25 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 792/3051 [06:13<17:45,  2.12it/s, est. speed input: 69461.24 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 793/3051 [06:14<17:45,  2.12it/s, est. speed input: 69461.33 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 794/3051 [06:14<17:44,  2.12it/s, est. speed input: 69461.50 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 795/3051 [06:15<17:44,  2.12it/s, est. speed input: 69461.37 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 796/3051 [06:15<17:43,  2.12it/s, est. speed input: 69461.53 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 797/3051 [06:15<17:44,  2.12it/s, est. speed input: 69461.07 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 798/3051 [06:16<17:44,  2.12it/s, est. speed input: 69460.93 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 799/3051 [06:16<17:43,  2.12it/s, est. speed input: 69460.92 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 800/3051 [06:17<17:42,  2.12it/s, est. speed input: 69461.03 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 801/3051 [06:17<17:40,  2.12it/s, est. speed input: 69461.26 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 802/3051 [06:18<17:40,  2.12it/s, est. speed input: 69461.22 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 803/3051 [06:18<17:39,  2.12it/s, est. speed input: 69461.34 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 804/3051 [06:19<17:41,  2.12it/s, est. speed input: 69460.72 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 805/3051 [06:19<17:40,  2.12it/s, est. speed input: 69460.76 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 806/3051 [06:20<17:39,  2.12it/s, est. speed input: 69460.77 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 807/3051 [06:20<17:39,  2.12it/s, est. speed input: 69460.77 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 808/3051 [06:21<17:40,  2.11it/s, est. speed input: 69460.11 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 809/3051 [06:21<17:39,  2.12it/s, est. speed input: 69460.28 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 810/3051 [06:22<17:37,  2.12it/s, est. speed input: 69460.41 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 811/3051 [06:22<17:37,  2.12it/s, est. speed input: 69460.23 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 812/3051 [06:23<17:36,  2.12it/s, est. speed input: 69460.23 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 813/3051 [06:23<17:36,  2.12it/s, est. speed input: 69460.24 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 814/3051 [06:24<17:36,  2.12it/s, est. speed input: 69460.12 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 815/3051 [06:24<17:35,  2.12it/s, est. speed input: 69460.02 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 816/3051 [06:24<17:34,  2.12it/s, est. speed input: 69460.12 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 817/3051 [06:25<17:35,  2.12it/s, est. speed input: 69459.77 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 818/3051 [06:25<17:35,  2.12it/s, est. speed input: 69459.48 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 819/3051 [06:26<17:33,  2.12it/s, est. speed input: 69459.76 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 820/3051 [06:26<17:35,  2.11it/s, est. speed input: 69458.95 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 821/3051 [06:27<17:35,  2.11it/s, est. speed input: 69458.63 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 822/3051 [06:27<17:33,  2.12it/s, est. speed input: 69458.86 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 823/3051 [06:28<17:32,  2.12it/s, est. speed input: 69458.78 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 824/3051 [06:28<17:31,  2.12it/s, est. speed input: 69458.70 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 825/3051 [06:29<17:31,  2.12it/s, est. speed input: 69458.68 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 826/3051 [06:29<17:29,  2.12it/s, est. speed input: 69458.86 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 827/3051 [06:30<17:33,  2.11it/s, est. speed input: 69457.81 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 828/3051 [06:30<17:58,  2.06it/s, est. speed input: 69450.69 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 829/3051 [06:31<17:48,  2.08it/s, est. speed input: 69450.87 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 830/3051 [06:33<39:54,  1.08s/it, est. speed input: 69097.82 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 831/3051 [06:34<33:08,  1.12it/s, est. speed input: 69098.59 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 832/3051 [06:34<28:24,  1.30it/s, est. speed input: 69099.25 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 833/3051 [06:35<25:05,  1.47it/s, est. speed input: 69099.99 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 834/3051 [06:35<22:45,  1.62it/s, est. speed input: 69100.73 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 835/3051 [06:35<21:08,  1.75it/s, est. speed input: 69101.35 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 836/3051 [06:36<19:59,  1.85it/s, est. speed input: 69102.09 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 837/3051 [06:36<19:11,  1.92it/s, est. speed input: 69102.84 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 838/3051 [06:37<18:38,  1.98it/s, est. speed input: 69103.50 toks/s, output: 2.11 toks/s]
Processed prompts:  27%|██▋       | 839/3051 [06:37<18:14,  2.02it/s, est. speed input: 69104.22 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 840/3051 [06:38<17:57,  2.05it/s, est. speed input: 69104.95 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 841/3051 [06:38<17:45,  2.07it/s, est. speed input: 69105.69 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 842/3051 [06:39<17:36,  2.09it/s, est. speed input: 69106.39 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 843/3051 [06:39<17:30,  2.10it/s, est. speed input: 69107.15 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 844/3051 [06:40<17:26,  2.11it/s, est. speed input: 69107.90 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 845/3051 [06:40<17:23,  2.11it/s, est. speed input: 69108.53 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 846/3051 [06:41<17:20,  2.12it/s, est. speed input: 69109.28 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 847/3051 [06:41<17:19,  2.12it/s, est. speed input: 69109.98 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 848/3051 [06:42<17:17,  2.12it/s, est. speed input: 69110.70 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 849/3051 [06:42<17:16,  2.12it/s, est. speed input: 69111.40 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 850/3051 [06:43<17:15,  2.12it/s, est. speed input: 69112.04 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 851/3051 [06:43<17:15,  2.13it/s, est. speed input: 69112.67 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 852/3051 [06:43<17:14,  2.13it/s, est. speed input: 69113.34 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 853/3051 [06:44<17:13,  2.13it/s, est. speed input: 69114.00 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 854/3051 [06:44<17:13,  2.13it/s, est. speed input: 69114.69 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 855/3051 [06:45<17:12,  2.13it/s, est. speed input: 69115.33 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 856/3051 [06:45<17:12,  2.13it/s, est. speed input: 69115.99 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 857/3051 [06:46<17:11,  2.13it/s, est. speed input: 69116.69 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 858/3051 [06:46<17:11,  2.13it/s, est. speed input: 69117.25 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 859/3051 [06:47<17:10,  2.13it/s, est. speed input: 69117.94 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 860/3051 [06:47<17:10,  2.13it/s, est. speed input: 69118.61 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 861/3051 [06:48<17:09,  2.13it/s, est. speed input: 69119.25 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 862/3051 [06:48<17:09,  2.13it/s, est. speed input: 69119.77 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 863/3051 [06:49<17:09,  2.13it/s, est. speed input: 69120.41 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 864/3051 [06:49<17:09,  2.12it/s, est. speed input: 69120.73 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 865/3051 [06:50<17:10,  2.12it/s, est. speed input: 69121.06 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 866/3051 [06:50<17:09,  2.12it/s, est. speed input: 69121.61 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 867/3051 [06:51<17:08,  2.12it/s, est. speed input: 69122.12 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 868/3051 [06:51<17:08,  2.12it/s, est. speed input: 69122.69 toks/s, output: 2.11 toks/s]
Processed prompts:  28%|██▊       | 869/3051 [06:51<17:07,  2.12it/s, est. speed input: 69123.26 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 870/3051 [06:52<17:07,  2.12it/s, est. speed input: 69123.72 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 871/3051 [06:52<17:06,  2.12it/s, est. speed input: 69124.37 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 872/3051 [06:53<17:06,  2.12it/s, est. speed input: 69124.79 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 873/3051 [06:53<17:05,  2.12it/s, est. speed input: 69125.31 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 874/3051 [06:54<17:04,  2.12it/s, est. speed input: 69125.93 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 875/3051 [06:54<17:04,  2.12it/s, est. speed input: 69126.52 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 876/3051 [06:55<17:03,  2.13it/s, est. speed input: 69127.15 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▊       | 877/3051 [06:55<17:03,  2.12it/s, est. speed input: 69127.58 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 878/3051 [06:56<17:12,  2.10it/s, est. speed input: 69125.68 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 879/3051 [06:56<17:24,  2.08it/s, est. speed input: 69122.30 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 880/3051 [06:57<17:17,  2.09it/s, est. speed input: 69122.73 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 881/3051 [06:57<17:13,  2.10it/s, est. speed input: 69123.02 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 882/3051 [06:58<17:10,  2.11it/s, est. speed input: 69123.45 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 883/3051 [06:58<17:07,  2.11it/s, est. speed input: 69123.77 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 884/3051 [06:59<17:05,  2.11it/s, est. speed input: 69124.17 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 885/3051 [06:59<17:03,  2.12it/s, est. speed input: 69124.77 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 886/3051 [07:00<17:17,  2.09it/s, est. speed input: 69121.27 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 887/3051 [07:00<17:11,  2.10it/s, est. speed input: 69121.86 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 888/3051 [07:00<17:07,  2.11it/s, est. speed input: 69122.38 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 889/3051 [07:01<17:03,  2.11it/s, est. speed input: 69123.02 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 890/3051 [07:01<17:02,  2.11it/s, est. speed input: 69123.34 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 891/3051 [07:02<17:00,  2.12it/s, est. speed input: 69123.93 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 892/3051 [07:02<16:58,  2.12it/s, est. speed input: 69124.60 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 893/3051 [07:03<16:58,  2.12it/s, est. speed input: 69124.91 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 894/3051 [07:03<16:57,  2.12it/s, est. speed input: 69125.24 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 895/3051 [07:04<16:56,  2.12it/s, est. speed input: 69125.76 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 896/3051 [07:04<16:56,  2.12it/s, est. speed input: 69126.00 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 897/3051 [07:05<16:56,  2.12it/s, est. speed input: 69126.32 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 898/3051 [07:05<16:55,  2.12it/s, est. speed input: 69126.80 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 899/3051 [07:06<17:01,  2.11it/s, est. speed input: 69125.61 toks/s, output: 2.11 toks/s]
Processed prompts:  29%|██▉       | 900/3051 [07:06<16:58,  2.11it/s, est. speed input: 69126.18 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 901/3051 [07:07<16:56,  2.11it/s, est. speed input: 69126.55 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 902/3051 [07:07<16:55,  2.12it/s, est. speed input: 69126.90 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 903/3051 [07:08<16:53,  2.12it/s, est. speed input: 69127.46 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 904/3051 [07:08<16:52,  2.12it/s, est. speed input: 69127.94 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 905/3051 [07:08<16:51,  2.12it/s, est. speed input: 69128.59 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 906/3051 [07:09<16:51,  2.12it/s, est. speed input: 69128.94 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 907/3051 [07:09<16:50,  2.12it/s, est. speed input: 69129.28 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 908/3051 [07:10<16:50,  2.12it/s, est. speed input: 69129.64 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 909/3051 [07:10<16:49,  2.12it/s, est. speed input: 69130.20 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 910/3051 [07:11<16:50,  2.12it/s, est. speed input: 69130.33 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 911/3051 [07:11<16:49,  2.12it/s, est. speed input: 69130.79 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 912/3051 [07:12<16:47,  2.12it/s, est. speed input: 69131.38 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 913/3051 [07:12<16:48,  2.12it/s, est. speed input: 69131.67 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 914/3051 [07:13<16:49,  2.12it/s, est. speed input: 69131.52 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|██▉       | 915/3051 [07:13<16:48,  2.12it/s, est. speed input: 69132.05 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 916/3051 [07:14<16:46,  2.12it/s, est. speed input: 69132.61 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 917/3051 [07:14<16:48,  2.12it/s, est. speed input: 69132.53 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 918/3051 [07:15<16:46,  2.12it/s, est. speed input: 69132.95 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 919/3051 [07:15<16:45,  2.12it/s, est. speed input: 69133.38 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 920/3051 [07:16<16:45,  2.12it/s, est. speed input: 69133.66 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 921/3051 [07:16<16:45,  2.12it/s, est. speed input: 69133.82 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 922/3051 [07:17<16:45,  2.12it/s, est. speed input: 69134.09 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 923/3051 [07:17<16:46,  2.11it/s, est. speed input: 69134.02 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 924/3051 [07:17<16:44,  2.12it/s, est. speed input: 69134.57 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 925/3051 [07:18<17:18,  2.05it/s, est. speed input: 69126.38 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 926/3051 [07:18<17:06,  2.07it/s, est. speed input: 69126.81 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 927/3051 [07:19<16:58,  2.09it/s, est. speed input: 69127.31 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 928/3051 [07:19<16:53,  2.10it/s, est. speed input: 69127.67 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 929/3051 [07:20<16:49,  2.10it/s, est. speed input: 69128.03 toks/s, output: 2.11 toks/s]
Processed prompts:  30%|███       | 930/3051 [07:20<16:46,  2.11it/s, est. speed input: 69128.40 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 931/3051 [07:21<16:43,  2.11it/s, est. speed input: 69128.83 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 932/3051 [07:21<16:41,  2.12it/s, est. speed input: 69129.31 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 933/3051 [07:22<16:39,  2.12it/s, est. speed input: 69129.92 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 934/3051 [07:22<16:38,  2.12it/s, est. speed input: 69130.39 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 935/3051 [07:23<16:39,  2.12it/s, est. speed input: 69130.30 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 936/3051 [07:23<16:38,  2.12it/s, est. speed input: 69130.86 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 937/3051 [07:24<16:36,  2.12it/s, est. speed input: 69131.37 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 938/3051 [07:24<16:39,  2.11it/s, est. speed input: 69131.05 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 939/3051 [07:25<16:38,  2.12it/s, est. speed input: 69131.37 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 940/3051 [07:25<16:36,  2.12it/s, est. speed input: 69131.85 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 941/3051 [07:26<16:35,  2.12it/s, est. speed input: 69132.31 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 942/3051 [07:26<16:34,  2.12it/s, est. speed input: 69132.80 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 943/3051 [07:26<16:33,  2.12it/s, est. speed input: 69133.20 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 944/3051 [07:27<16:34,  2.12it/s, est. speed input: 69133.36 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 945/3051 [07:27<16:33,  2.12it/s, est. speed input: 69133.73 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 946/3051 [07:28<16:32,  2.12it/s, est. speed input: 69134.16 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 947/3051 [07:28<16:32,  2.12it/s, est. speed input: 69134.55 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 948/3051 [07:29<16:31,  2.12it/s, est. speed input: 69134.89 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 949/3051 [07:29<16:32,  2.12it/s, est. speed input: 69135.06 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 950/3051 [07:30<16:30,  2.12it/s, est. speed input: 69135.55 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 951/3051 [07:30<16:30,  2.12it/s, est. speed input: 69135.95 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 952/3051 [07:31<16:30,  2.12it/s, est. speed input: 69136.14 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███       | 953/3051 [07:31<16:29,  2.12it/s, est. speed input: 69136.59 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 954/3051 [07:32<16:28,  2.12it/s, est. speed input: 69137.09 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 955/3051 [07:32<16:30,  2.12it/s, est. speed input: 69136.84 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 956/3051 [07:33<16:29,  2.12it/s, est. speed input: 69137.11 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 957/3051 [07:33<16:31,  2.11it/s, est. speed input: 69136.80 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 958/3051 [07:34<16:39,  2.09it/s, est. speed input: 69134.88 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 959/3051 [07:34<16:49,  2.07it/s, est. speed input: 69131.69 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 960/3051 [07:35<16:41,  2.09it/s, est. speed input: 69132.16 toks/s, output: 2.11 toks/s]
Processed prompts:  31%|███▏      | 961/3051 [07:35<16:36,  2.10it/s, est. speed input: 69132.56 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 962/3051 [07:35<16:32,  2.10it/s, est. speed input: 69132.92 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 963/3051 [07:36<16:29,  2.11it/s, est. speed input: 69133.41 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 964/3051 [07:36<16:30,  2.11it/s, est. speed input: 69133.05 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 965/3051 [07:37<16:28,  2.11it/s, est. speed input: 69133.36 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 966/3051 [07:37<16:26,  2.11it/s, est. speed input: 69133.71 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 967/3051 [07:38<16:24,  2.12it/s, est. speed input: 69134.17 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 968/3051 [07:38<16:23,  2.12it/s, est. speed input: 69134.57 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 969/3051 [07:39<16:22,  2.12it/s, est. speed input: 69135.02 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 970/3051 [07:39<16:21,  2.12it/s, est. speed input: 69135.51 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 971/3051 [07:40<16:22,  2.12it/s, est. speed input: 69135.45 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 972/3051 [07:40<16:21,  2.12it/s, est. speed input: 69135.77 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 973/3051 [07:41<16:20,  2.12it/s, est. speed input: 69136.16 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 974/3051 [07:41<16:20,  2.12it/s, est. speed input: 69136.45 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 975/3051 [07:42<16:20,  2.12it/s, est. speed input: 69136.59 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 976/3051 [07:42<16:19,  2.12it/s, est. speed input: 69136.97 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 977/3051 [07:43<16:18,  2.12it/s, est. speed input: 69137.38 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 978/3051 [07:43<16:18,  2.12it/s, est. speed input: 69137.63 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 979/3051 [07:43<16:17,  2.12it/s, est. speed input: 69138.00 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 980/3051 [07:44<16:16,  2.12it/s, est. speed input: 69138.40 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 981/3051 [07:44<16:15,  2.12it/s, est. speed input: 69138.93 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 982/3051 [07:45<16:16,  2.12it/s, est. speed input: 69139.06 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 983/3051 [07:45<16:15,  2.12it/s, est. speed input: 69139.38 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 984/3051 [07:46<16:15,  2.12it/s, est. speed input: 69139.73 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 985/3051 [07:46<16:14,  2.12it/s, est. speed input: 69140.02 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 986/3051 [07:47<16:14,  2.12it/s, est. speed input: 69140.21 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 987/3051 [07:47<16:15,  2.12it/s, est. speed input: 69140.32 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 988/3051 [07:48<16:14,  2.12it/s, est. speed input: 69140.61 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 989/3051 [07:48<16:12,  2.12it/s, est. speed input: 69141.13 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 990/3051 [07:49<16:12,  2.12it/s, est. speed input: 69141.33 toks/s, output: 2.11 toks/s]
Processed prompts:  32%|███▏      | 991/3051 [07:49<16:12,  2.12it/s, est. speed input: 69141.46 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 992/3051 [07:50<16:11,  2.12it/s, est. speed input: 69141.84 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 993/3051 [07:50<16:12,  2.12it/s, est. speed input: 69141.79 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 994/3051 [07:51<16:13,  2.11it/s, est. speed input: 69141.61 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 995/3051 [07:51<16:11,  2.12it/s, est. speed input: 69142.03 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 996/3051 [07:52<16:13,  2.11it/s, est. speed input: 69141.72 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 997/3051 [07:52<16:12,  2.11it/s, est. speed input: 69141.98 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 998/3051 [07:52<16:10,  2.12it/s, est. speed input: 69142.36 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 999/3051 [07:53<16:08,  2.12it/s, est. speed input: 69142.88 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1000/3051 [07:53<16:10,  2.11it/s, est. speed input: 69142.51 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1001/3051 [07:54<16:09,  2.11it/s, est. speed input: 69142.76 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1002/3051 [07:54<16:07,  2.12it/s, est. speed input: 69143.26 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1003/3051 [07:55<16:07,  2.12it/s, est. speed input: 69143.48 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1004/3051 [07:55<16:07,  2.12it/s, est. speed input: 69143.66 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1005/3051 [07:56<16:05,  2.12it/s, est. speed input: 69144.04 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1006/3051 [07:56<16:04,  2.12it/s, est. speed input: 69144.51 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1007/3051 [07:57<16:04,  2.12it/s, est. speed input: 69144.69 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1008/3051 [07:57<16:03,  2.12it/s, est. speed input: 69145.09 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1009/3051 [07:58<16:02,  2.12it/s, est. speed input: 69145.59 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1010/3051 [07:58<16:03,  2.12it/s, est. speed input: 69145.68 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1011/3051 [07:59<16:02,  2.12it/s, est. speed input: 69145.99 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1012/3051 [07:59<16:01,  2.12it/s, est. speed input: 69146.35 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1013/3051 [08:00<16:01,  2.12it/s, est. speed input: 69146.70 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1014/3051 [08:00<16:03,  2.11it/s, est. speed input: 69146.40 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1015/3051 [08:00<16:02,  2.12it/s, est. speed input: 69146.73 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1016/3051 [08:01<16:00,  2.12it/s, est. speed input: 69147.19 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1017/3051 [08:01<16:02,  2.11it/s, est. speed input: 69146.86 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1018/3051 [08:02<16:00,  2.12it/s, est. speed input: 69147.28 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1019/3051 [08:02<15:59,  2.12it/s, est. speed input: 69147.63 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1020/3051 [08:03<16:00,  2.11it/s, est. speed input: 69147.53 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1021/3051 [08:03<15:59,  2.12it/s, est. speed input: 69147.78 toks/s, output: 2.11 toks/s]
Processed prompts:  33%|███▎      | 1022/3051 [08:04<15:57,  2.12it/s, est. speed input: 69148.28 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1023/3051 [08:04<15:57,  2.12it/s, est. speed input: 69148.57 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1024/3051 [08:05<15:58,  2.11it/s, est. speed input: 69148.37 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1025/3051 [08:05<15:57,  2.12it/s, est. speed input: 69148.75 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1026/3051 [08:06<15:55,  2.12it/s, est. speed input: 69149.22 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1027/3051 [08:06<15:56,  2.12it/s, est. speed input: 69149.08 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1028/3051 [08:07<15:55,  2.12it/s, est. speed input: 69149.51 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▎      | 1029/3051 [08:07<15:53,  2.12it/s, est. speed input: 69149.94 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1030/3051 [08:08<15:53,  2.12it/s, est. speed input: 69150.23 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1031/3051 [08:08<15:54,  2.12it/s, est. speed input: 69150.14 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1032/3051 [08:09<15:54,  2.11it/s, est. speed input: 69150.22 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1033/3051 [08:09<15:52,  2.12it/s, est. speed input: 69150.72 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1034/3051 [08:09<15:52,  2.12it/s, est. speed input: 69150.99 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1035/3051 [08:10<16:35,  2.02it/s, est. speed input: 69140.93 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1036/3051 [08:10<16:20,  2.05it/s, est. speed input: 69141.48 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1037/3051 [08:11<16:10,  2.08it/s, est. speed input: 69141.97 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1038/3051 [08:11<16:03,  2.09it/s, est. speed input: 69142.40 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1039/3051 [08:12<15:58,  2.10it/s, est. speed input: 69142.74 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1040/3051 [08:12<15:54,  2.11it/s, est. speed input: 69143.14 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1041/3051 [08:13<15:55,  2.10it/s, est. speed input: 69142.75 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1042/3051 [08:13<15:52,  2.11it/s, est. speed input: 69143.09 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1043/3051 [08:14<15:50,  2.11it/s, est. speed input: 69143.56 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1044/3051 [08:14<15:48,  2.12it/s, est. speed input: 69144.02 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1045/3051 [08:15<15:50,  2.11it/s, est. speed input: 69143.64 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1046/3051 [08:15<15:48,  2.11it/s, est. speed input: 69143.89 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1047/3051 [08:16<15:46,  2.12it/s, est. speed input: 69144.36 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1048/3051 [08:16<15:45,  2.12it/s, est. speed input: 69144.69 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1049/3051 [08:17<15:45,  2.12it/s, est. speed input: 69144.86 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1050/3051 [08:17<15:43,  2.12it/s, est. speed input: 69145.33 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1051/3051 [08:18<15:43,  2.12it/s, est. speed input: 69145.54 toks/s, output: 2.11 toks/s]
Processed prompts:  34%|███▍      | 1052/3051 [08:18<15:43,  2.12it/s, est. speed input: 69145.88 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1053/3051 [08:19<15:41,  2.12it/s, est. speed input: 69146.41 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1054/3051 [08:19<15:43,  2.12it/s, est. speed input: 69146.26 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1055/3051 [08:19<15:43,  2.12it/s, est. speed input: 69146.31 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1056/3051 [08:20<15:41,  2.12it/s, est. speed input: 69146.75 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1057/3051 [08:20<15:40,  2.12it/s, est. speed input: 69147.17 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1058/3051 [08:21<15:39,  2.12it/s, est. speed input: 69147.58 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1059/3051 [08:21<15:39,  2.12it/s, est. speed input: 69147.91 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1060/3051 [08:22<15:38,  2.12it/s, est. speed input: 69148.38 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1061/3051 [08:22<15:40,  2.12it/s, est. speed input: 69148.17 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1062/3051 [08:23<15:39,  2.12it/s, est. speed input: 69148.46 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1063/3051 [08:23<15:38,  2.12it/s, est. speed input: 69148.86 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1064/3051 [08:24<15:37,  2.12it/s, est. speed input: 69149.08 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1065/3051 [08:24<15:38,  2.12it/s, est. speed input: 69149.18 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1066/3051 [08:25<15:36,  2.12it/s, est. speed input: 69149.56 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▍      | 1067/3051 [08:25<15:35,  2.12it/s, est. speed input: 69149.97 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1068/3051 [08:26<15:35,  2.12it/s, est. speed input: 69150.17 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1069/3051 [08:26<15:35,  2.12it/s, est. speed input: 69150.36 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1070/3051 [08:27<15:39,  2.11it/s, est. speed input: 69149.72 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1071/3051 [08:27<15:37,  2.11it/s, est. speed input: 69149.99 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1072/3051 [08:27<15:35,  2.12it/s, est. speed input: 69150.34 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1073/3051 [08:28<15:33,  2.12it/s, est. speed input: 69150.79 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1074/3051 [08:28<15:36,  2.11it/s, est. speed input: 69150.25 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1075/3051 [08:29<15:35,  2.11it/s, est. speed input: 69150.35 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1076/3051 [08:29<15:34,  2.11it/s, est. speed input: 69150.61 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1077/3051 [08:30<15:33,  2.11it/s, est. speed input: 69150.73 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1078/3051 [08:30<15:33,  2.11it/s, est. speed input: 69150.80 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1079/3051 [08:31<15:32,  2.11it/s, est. speed input: 69150.98 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1080/3051 [08:31<15:32,  2.11it/s, est. speed input: 69151.14 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1081/3051 [08:32<15:31,  2.11it/s, est. speed input: 69151.26 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1082/3051 [08:32<15:31,  2.11it/s, est. speed input: 69151.42 toks/s, output: 2.11 toks/s]
Processed prompts:  35%|███▌      | 1083/3051 [08:33<15:31,  2.11it/s, est. speed input: 69151.31 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1084/3051 [08:33<15:31,  2.11it/s, est. speed input: 69151.30 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1085/3051 [08:34<15:30,  2.11it/s, est. speed input: 69151.42 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1086/3051 [08:34<15:29,  2.11it/s, est. speed input: 69151.58 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1087/3051 [08:35<15:29,  2.11it/s, est. speed input: 69151.73 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1088/3051 [08:35<15:30,  2.11it/s, est. speed input: 69151.47 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1089/3051 [08:36<15:28,  2.11it/s, est. speed input: 69151.67 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1090/3051 [08:36<15:27,  2.11it/s, est. speed input: 69151.85 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1091/3051 [08:36<15:27,  2.11it/s, est. speed input: 69151.97 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1092/3051 [08:37<15:27,  2.11it/s, est. speed input: 69151.94 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1093/3051 [08:37<15:26,  2.11it/s, est. speed input: 69152.12 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1094/3051 [08:38<15:26,  2.11it/s, est. speed input: 69152.11 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1095/3051 [08:38<15:25,  2.11it/s, est. speed input: 69152.22 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1096/3051 [08:39<15:24,  2.11it/s, est. speed input: 69152.40 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1097/3051 [08:39<15:23,  2.12it/s, est. speed input: 69152.65 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1098/3051 [08:40<15:23,  2.12it/s, est. speed input: 69152.82 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1099/3051 [08:40<15:23,  2.11it/s, est. speed input: 69152.87 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1100/3051 [08:41<15:22,  2.12it/s, est. speed input: 69153.04 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1101/3051 [08:41<15:21,  2.12it/s, est. speed input: 69153.23 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1102/3051 [08:42<15:21,  2.12it/s, est. speed input: 69153.38 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1103/3051 [08:42<15:21,  2.11it/s, est. speed input: 69153.45 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1104/3051 [08:43<15:22,  2.11it/s, est. speed input: 69153.20 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▌      | 1105/3051 [08:43<15:20,  2.11it/s, est. speed input: 69153.45 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1106/3051 [08:44<15:19,  2.12it/s, est. speed input: 69153.79 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1107/3051 [08:44<15:19,  2.11it/s, est. speed input: 69153.78 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1108/3051 [08:45<15:17,  2.12it/s, est. speed input: 69154.15 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1109/3051 [08:45<15:18,  2.11it/s, est. speed input: 69154.09 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1110/3051 [08:45<15:16,  2.12it/s, est. speed input: 69154.45 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1111/3051 [08:46<15:16,  2.12it/s, est. speed input: 69154.65 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1112/3051 [08:46<15:15,  2.12it/s, est. speed input: 69154.98 toks/s, output: 2.11 toks/s]
Processed prompts:  36%|███▋      | 1113/3051 [08:47<15:15,  2.12it/s, est. speed input: 69155.04 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1114/3051 [08:47<15:15,  2.12it/s, est. speed input: 69155.13 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1115/3051 [08:48<15:13,  2.12it/s, est. speed input: 69155.54 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1116/3051 [08:48<15:16,  2.11it/s, est. speed input: 69155.11 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1117/3051 [08:49<15:14,  2.11it/s, est. speed input: 69155.38 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1118/3051 [08:49<15:13,  2.12it/s, est. speed input: 69155.76 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1119/3051 [08:50<15:11,  2.12it/s, est. speed input: 69156.12 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1120/3051 [08:50<15:11,  2.12it/s, est. speed input: 69156.38 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1121/3051 [08:51<15:10,  2.12it/s, est. speed input: 69156.65 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1122/3051 [08:51<15:09,  2.12it/s, est. speed input: 69156.97 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1123/3051 [08:52<15:10,  2.12it/s, est. speed input: 69157.07 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1124/3051 [08:52<15:09,  2.12it/s, est. speed input: 69157.34 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1125/3051 [08:53<15:08,  2.12it/s, est. speed input: 69157.67 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1126/3051 [08:53<15:08,  2.12it/s, est. speed input: 69157.83 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1127/3051 [08:53<15:08,  2.12it/s, est. speed input: 69158.00 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1128/3051 [08:54<15:07,  2.12it/s, est. speed input: 69158.26 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1129/3051 [08:54<15:07,  2.12it/s, est. speed input: 69158.38 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1130/3051 [08:55<15:08,  2.11it/s, est. speed input: 69158.25 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1131/3051 [08:55<15:07,  2.12it/s, est. speed input: 69158.47 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1132/3051 [08:56<15:06,  2.12it/s, est. speed input: 69158.67 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1133/3051 [08:56<15:06,  2.12it/s, est. speed input: 69158.89 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1134/3051 [08:57<15:05,  2.12it/s, est. speed input: 69159.11 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1135/3051 [08:57<15:04,  2.12it/s, est. speed input: 69159.32 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1136/3051 [08:58<15:04,  2.12it/s, est. speed input: 69159.41 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1137/3051 [08:58<15:04,  2.12it/s, est. speed input: 69159.58 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1138/3051 [08:59<15:04,  2.12it/s, est. speed input: 69159.67 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1139/3051 [08:59<15:03,  2.12it/s, est. speed input: 69159.86 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1140/3051 [09:00<15:03,  2.12it/s, est. speed input: 69160.03 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1141/3051 [09:00<15:02,  2.12it/s, est. speed input: 69160.27 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1142/3051 [09:01<15:02,  2.11it/s, est. speed input: 69160.25 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1143/3051 [09:01<15:02,  2.12it/s, est. speed input: 69160.38 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 1144/3051 [09:02<15:01,  2.11it/s, est. speed input: 69160.44 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1145/3051 [09:02<15:01,  2.11it/s, est. speed input: 69160.52 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1146/3051 [09:02<15:01,  2.11it/s, est. speed input: 69160.63 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1147/3051 [09:03<15:00,  2.12it/s, est. speed input: 69160.85 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1148/3051 [09:03<14:59,  2.11it/s, est. speed input: 69160.94 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1149/3051 [09:04<15:02,  2.11it/s, est. speed input: 69160.34 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1150/3051 [09:04<15:01,  2.11it/s, est. speed input: 69160.41 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1151/3051 [09:05<15:00,  2.11it/s, est. speed input: 69160.54 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1152/3051 [09:05<14:59,  2.11it/s, est. speed input: 69160.69 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1153/3051 [09:06<15:00,  2.11it/s, est. speed input: 69160.30 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1154/3051 [09:06<14:58,  2.11it/s, est. speed input: 69160.57 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1155/3051 [09:07<14:59,  2.11it/s, est. speed input: 69160.22 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1156/3051 [09:07<14:58,  2.11it/s, est. speed input: 69160.17 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1157/3051 [09:08<14:57,  2.11it/s, est. speed input: 69160.27 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1158/3051 [09:08<14:56,  2.11it/s, est. speed input: 69160.49 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1159/3051 [09:09<14:55,  2.11it/s, est. speed input: 69160.64 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1160/3051 [09:09<14:56,  2.11it/s, est. speed input: 69160.41 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1161/3051 [09:10<14:57,  2.10it/s, est. speed input: 69159.89 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1162/3051 [09:10<14:55,  2.11it/s, est. speed input: 69160.08 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1163/3051 [09:11<14:54,  2.11it/s, est. speed input: 69160.26 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1164/3051 [09:11<14:53,  2.11it/s, est. speed input: 69160.39 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1165/3051 [09:11<14:51,  2.11it/s, est. speed input: 69160.64 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1166/3051 [09:12<14:52,  2.11it/s, est. speed input: 69160.51 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1167/3051 [09:12<14:51,  2.11it/s, est. speed input: 69160.64 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1168/3051 [09:13<14:51,  2.11it/s, est. speed input: 69160.70 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1169/3051 [09:13<14:50,  2.11it/s, est. speed input: 69160.79 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1170/3051 [09:14<14:49,  2.11it/s, est. speed input: 69161.01 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1171/3051 [09:14<14:51,  2.11it/s, est. speed input: 69160.49 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1172/3051 [09:15<14:50,  2.11it/s, est. speed input: 69160.69 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1173/3051 [09:15<14:48,  2.11it/s, est. speed input: 69160.86 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 1174/3051 [09:16<14:47,  2.11it/s, est. speed input: 69161.05 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1175/3051 [09:16<14:46,  2.12it/s, est. speed input: 69161.25 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1176/3051 [09:17<14:46,  2.12it/s, est. speed input: 69161.39 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1177/3051 [09:17<14:49,  2.11it/s, est. speed input: 69160.83 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1178/3051 [09:18<14:47,  2.11it/s, est. speed input: 69161.00 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1179/3051 [09:18<14:45,  2.11it/s, est. speed input: 69161.29 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1180/3051 [09:19<14:45,  2.11it/s, est. speed input: 69161.35 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1181/3051 [09:19<14:44,  2.11it/s, est. speed input: 69161.51 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 1182/3051 [09:20<14:43,  2.11it/s, est. speed input: 69161.61 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1183/3051 [09:20<14:43,  2.12it/s, est. speed input: 69161.81 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1184/3051 [09:20<14:42,  2.12it/s, est. speed input: 69162.00 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1185/3051 [09:21<14:42,  2.11it/s, est. speed input: 69162.03 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1186/3051 [09:21<14:41,  2.12it/s, est. speed input: 69162.24 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1187/3051 [09:22<14:41,  2.12it/s, est. speed input: 69162.32 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1188/3051 [09:22<14:40,  2.12it/s, est. speed input: 69162.56 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1189/3051 [09:23<14:39,  2.12it/s, est. speed input: 69162.74 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1190/3051 [09:23<14:39,  2.12it/s, est. speed input: 69162.91 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1191/3051 [09:24<14:38,  2.12it/s, est. speed input: 69163.08 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1192/3051 [09:24<14:38,  2.12it/s, est. speed input: 69163.23 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1193/3051 [09:25<14:37,  2.12it/s, est. speed input: 69163.49 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1194/3051 [09:25<14:37,  2.12it/s, est. speed input: 69163.44 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1195/3051 [09:26<14:37,  2.11it/s, est. speed input: 69163.52 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1196/3051 [09:26<14:36,  2.12it/s, est. speed input: 69163.75 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1197/3051 [09:27<14:36,  2.12it/s, est. speed input: 69163.83 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1198/3051 [09:27<14:36,  2.12it/s, est. speed input: 69163.95 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1199/3051 [09:28<14:35,  2.11it/s, est. speed input: 69164.03 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1200/3051 [09:28<14:35,  2.11it/s, est. speed input: 69164.06 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1201/3051 [09:28<14:35,  2.11it/s, est. speed input: 69164.11 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1202/3051 [09:29<14:34,  2.12it/s, est. speed input: 69164.34 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1203/3051 [09:29<14:33,  2.12it/s, est. speed input: 69164.47 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1204/3051 [09:30<14:35,  2.11it/s, est. speed input: 69164.10 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 1205/3051 [09:30<14:34,  2.11it/s, est. speed input: 69164.21 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1206/3051 [09:31<14:32,  2.11it/s, est. speed input: 69164.46 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1207/3051 [09:31<14:32,  2.11it/s, est. speed input: 69164.61 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1208/3051 [09:32<14:31,  2.11it/s, est. speed input: 69164.71 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1209/3051 [09:32<14:30,  2.12it/s, est. speed input: 69164.94 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1210/3051 [09:33<14:29,  2.12it/s, est. speed input: 69165.12 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1211/3051 [09:33<14:30,  2.11it/s, est. speed input: 69165.12 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1212/3051 [09:34<14:29,  2.11it/s, est. speed input: 69165.21 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1213/3051 [09:34<14:28,  2.12it/s, est. speed input: 69165.38 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1214/3051 [09:35<14:28,  2.11it/s, est. speed input: 69165.44 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1215/3051 [09:35<14:27,  2.12it/s, est. speed input: 69165.63 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1216/3051 [09:36<14:28,  2.11it/s, est. speed input: 69165.54 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1217/3051 [09:36<14:27,  2.11it/s, est. speed input: 69165.68 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1218/3051 [09:37<14:26,  2.12it/s, est. speed input: 69165.87 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1219/3051 [09:37<14:28,  2.11it/s, est. speed input: 69165.40 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 1220/3051 [09:37<14:26,  2.11it/s, est. speed input: 69165.79 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1221/3051 [09:38<14:24,  2.12it/s, est. speed input: 69166.08 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1222/3051 [09:38<14:23,  2.12it/s, est. speed input: 69166.44 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1223/3051 [09:39<14:23,  2.12it/s, est. speed input: 69166.55 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1224/3051 [09:39<14:23,  2.12it/s, est. speed input: 69166.61 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1225/3051 [09:40<14:22,  2.12it/s, est. speed input: 69166.74 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1226/3051 [09:40<14:23,  2.11it/s, est. speed input: 69166.74 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1227/3051 [09:41<14:23,  2.11it/s, est. speed input: 69166.73 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1228/3051 [09:41<14:22,  2.11it/s, est. speed input: 69166.79 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1229/3051 [09:42<14:21,  2.11it/s, est. speed input: 69166.95 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1230/3051 [09:42<14:20,  2.12it/s, est. speed input: 69167.11 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1231/3051 [09:43<14:20,  2.11it/s, est. speed input: 69167.21 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1232/3051 [09:43<14:22,  2.11it/s, est. speed input: 69166.81 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1233/3051 [09:44<14:21,  2.11it/s, est. speed input: 69166.82 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1234/3051 [09:44<14:20,  2.11it/s, est. speed input: 69166.95 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|████      | 1235/3051 [09:45<14:20,  2.11it/s, est. speed input: 69166.78 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1236/3051 [09:45<14:20,  2.11it/s, est. speed input: 69166.63 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1237/3051 [09:46<14:19,  2.11it/s, est. speed input: 69166.76 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1238/3051 [09:46<14:18,  2.11it/s, est. speed input: 69166.93 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1239/3051 [09:46<14:17,  2.11it/s, est. speed input: 69166.98 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1240/3051 [09:47<14:18,  2.11it/s, est. speed input: 69166.82 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1241/3051 [09:47<14:16,  2.11it/s, est. speed input: 69167.01 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1242/3051 [09:48<14:15,  2.11it/s, est. speed input: 69167.17 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1243/3051 [09:48<14:16,  2.11it/s, est. speed input: 69167.10 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1244/3051 [09:49<14:15,  2.11it/s, est. speed input: 69167.23 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1245/3051 [09:49<14:14,  2.11it/s, est. speed input: 69167.27 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1246/3051 [09:50<14:14,  2.11it/s, est. speed input: 69167.29 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1247/3051 [09:50<14:13,  2.11it/s, est. speed input: 69167.44 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1248/3051 [09:51<14:13,  2.11it/s, est. speed input: 69167.34 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1249/3051 [09:51<14:14,  2.11it/s, est. speed input: 69167.21 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1250/3051 [09:52<14:13,  2.11it/s, est. speed input: 69167.28 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1251/3051 [09:52<14:11,  2.11it/s, est. speed input: 69167.44 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1252/3051 [09:53<14:10,  2.11it/s, est. speed input: 69167.60 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1253/3051 [09:53<14:10,  2.11it/s, est. speed input: 69167.68 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1254/3051 [09:54<14:09,  2.11it/s, est. speed input: 69167.79 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1255/3051 [09:54<14:09,  2.11it/s, est. speed input: 69167.83 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1256/3051 [09:55<14:09,  2.11it/s, est. speed input: 69167.92 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1257/3051 [09:55<14:08,  2.11it/s, est. speed input: 69168.02 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████      | 1258/3051 [09:55<14:08,  2.11it/s, est. speed input: 69168.06 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1259/3051 [09:56<14:07,  2.11it/s, est. speed input: 69168.10 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1260/3051 [09:56<14:06,  2.11it/s, est. speed input: 69168.30 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1261/3051 [09:57<14:07,  2.11it/s, est. speed input: 69168.17 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1262/3051 [09:57<14:06,  2.11it/s, est. speed input: 69168.29 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1263/3051 [09:58<14:05,  2.11it/s, est. speed input: 69168.41 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1264/3051 [09:58<14:08,  2.11it/s, est. speed input: 69167.88 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1265/3051 [09:59<14:07,  2.11it/s, est. speed input: 69167.88 toks/s, output: 2.11 toks/s]
Processed prompts:  41%|████▏     | 1266/3051 [09:59<14:05,  2.11it/s, est. speed input: 69168.12 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1267/3051 [10:00<14:04,  2.11it/s, est. speed input: 69168.12 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1268/3051 [10:00<14:03,  2.11it/s, est. speed input: 69168.27 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1269/3051 [10:01<14:03,  2.11it/s, est. speed input: 69168.34 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1270/3051 [10:01<14:50,  2.00it/s, est. speed input: 69158.20 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1271/3051 [10:02<14:35,  2.03it/s, est. speed input: 69158.36 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1272/3051 [10:02<14:23,  2.06it/s, est. speed input: 69158.61 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1273/3051 [10:03<14:16,  2.08it/s, est. speed input: 69158.80 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1274/3051 [10:03<14:10,  2.09it/s, est. speed input: 69159.01 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1275/3051 [10:04<14:06,  2.10it/s, est. speed input: 69159.28 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1276/3051 [10:04<14:04,  2.10it/s, est. speed input: 69159.17 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1277/3051 [10:05<14:02,  2.10it/s, est. speed input: 69159.25 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1278/3051 [10:05<14:00,  2.11it/s, est. speed input: 69159.43 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1279/3051 [10:05<13:59,  2.11it/s, est. speed input: 69159.59 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1280/3051 [10:06<13:57,  2.11it/s, est. speed input: 69159.85 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1281/3051 [10:06<13:56,  2.11it/s, est. speed input: 69160.01 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1282/3051 [10:07<13:57,  2.11it/s, est. speed input: 69159.79 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1283/3051 [10:07<13:57,  2.11it/s, est. speed input: 69159.91 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1284/3051 [10:08<13:55,  2.11it/s, est. speed input: 69160.15 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1285/3051 [10:08<13:54,  2.12it/s, est. speed input: 69160.38 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1286/3051 [10:09<13:53,  2.12it/s, est. speed input: 69160.54 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1287/3051 [10:09<13:53,  2.12it/s, est. speed input: 69160.70 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1288/3051 [10:10<13:53,  2.12it/s, est. speed input: 69160.80 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1289/3051 [10:10<13:52,  2.12it/s, est. speed input: 69161.05 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1290/3051 [10:11<13:51,  2.12it/s, est. speed input: 69161.23 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1291/3051 [10:11<13:51,  2.12it/s, est. speed input: 69161.40 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1292/3051 [10:12<13:51,  2.11it/s, est. speed input: 69161.35 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1293/3051 [10:12<13:51,  2.11it/s, est. speed input: 69161.39 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1294/3051 [10:13<13:50,  2.12it/s, est. speed input: 69161.65 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1295/3051 [10:13<13:49,  2.12it/s, est. speed input: 69161.82 toks/s, output: 2.11 toks/s]
Processed prompts:  42%|████▏     | 1296/3051 [10:14<13:48,  2.12it/s, est. speed input: 69162.02 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1297/3051 [10:14<13:48,  2.12it/s, est. speed input: 69162.26 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1298/3051 [10:14<13:47,  2.12it/s, est. speed input: 69162.55 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1299/3051 [10:15<13:46,  2.12it/s, est. speed input: 69162.71 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1300/3051 [10:15<13:46,  2.12it/s, est. speed input: 69162.92 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1301/3051 [10:16<13:46,  2.12it/s, est. speed input: 69163.07 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1302/3051 [10:16<13:46,  2.12it/s, est. speed input: 69163.20 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1303/3051 [10:17<13:45,  2.12it/s, est. speed input: 69163.38 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1304/3051 [10:17<13:44,  2.12it/s, est. speed input: 69163.56 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1305/3051 [10:18<13:44,  2.12it/s, est. speed input: 69163.77 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1306/3051 [10:18<13:44,  2.12it/s, est. speed input: 69163.85 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1307/3051 [10:19<13:43,  2.12it/s, est. speed input: 69164.00 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1308/3051 [10:19<13:43,  2.12it/s, est. speed input: 69164.12 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1309/3051 [10:20<13:44,  2.11it/s, est. speed input: 69164.04 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1310/3051 [10:20<13:42,  2.12it/s, est. speed input: 69164.27 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1311/3051 [10:21<13:42,  2.12it/s, est. speed input: 69164.42 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1312/3051 [10:21<13:41,  2.12it/s, est. speed input: 69164.63 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1313/3051 [10:22<13:40,  2.12it/s, est. speed input: 69164.81 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1314/3051 [10:22<13:40,  2.12it/s, est. speed input: 69165.01 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1315/3051 [10:23<13:39,  2.12it/s, est. speed input: 69165.18 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1316/3051 [10:23<13:39,  2.12it/s, est. speed input: 69165.36 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1317/3051 [10:23<13:38,  2.12it/s, est. speed input: 69165.53 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1318/3051 [10:24<13:39,  2.12it/s, est. speed input: 69165.49 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1319/3051 [10:24<13:39,  2.11it/s, est. speed input: 69165.57 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1320/3051 [10:25<13:38,  2.12it/s, est. speed input: 69165.70 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1321/3051 [10:25<13:37,  2.12it/s, est. speed input: 69165.93 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1322/3051 [10:26<13:36,  2.12it/s, est. speed input: 69166.10 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1323/3051 [10:26<13:36,  2.12it/s, est. speed input: 69166.28 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1324/3051 [10:27<13:35,  2.12it/s, est. speed input: 69166.40 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1325/3051 [10:27<13:35,  2.12it/s, est. speed input: 69166.57 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1326/3051 [10:28<13:34,  2.12it/s, est. speed input: 69166.70 toks/s, output: 2.11 toks/s]
Processed prompts:  43%|████▎     | 1327/3051 [10:28<13:34,  2.12it/s, est. speed input: 69166.87 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1328/3051 [10:29<13:33,  2.12it/s, est. speed input: 69167.05 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1329/3051 [10:29<13:33,  2.12it/s, est. speed input: 69167.23 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1330/3051 [10:30<13:32,  2.12it/s, est. speed input: 69167.46 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1331/3051 [10:30<13:31,  2.12it/s, est. speed input: 69167.69 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1332/3051 [10:31<13:31,  2.12it/s, est. speed input: 69167.83 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1333/3051 [10:31<13:31,  2.12it/s, est. speed input: 69168.02 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▎     | 1334/3051 [10:31<13:30,  2.12it/s, est. speed input: 69168.23 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1335/3051 [10:32<13:29,  2.12it/s, est. speed input: 69168.47 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1336/3051 [10:32<13:29,  2.12it/s, est. speed input: 69168.58 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1337/3051 [10:33<13:29,  2.12it/s, est. speed input: 69168.76 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1338/3051 [10:33<13:28,  2.12it/s, est. speed input: 69168.88 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1339/3051 [10:34<13:28,  2.12it/s, est. speed input: 69169.05 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1340/3051 [10:34<13:27,  2.12it/s, est. speed input: 69169.25 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1341/3051 [10:35<13:27,  2.12it/s, est. speed input: 69169.46 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1342/3051 [10:35<13:26,  2.12it/s, est. speed input: 69169.65 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1343/3051 [10:36<13:26,  2.12it/s, est. speed input: 69169.79 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1344/3051 [10:36<13:26,  2.12it/s, est. speed input: 69169.79 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1345/3051 [10:37<13:26,  2.12it/s, est. speed input: 69169.86 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1346/3051 [10:37<13:25,  2.12it/s, est. speed input: 69170.03 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1347/3051 [10:38<13:25,  2.12it/s, est. speed input: 69170.19 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1348/3051 [10:38<13:24,  2.12it/s, est. speed input: 69170.35 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1349/3051 [10:39<13:23,  2.12it/s, est. speed input: 69170.52 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1350/3051 [10:39<13:23,  2.12it/s, est. speed input: 69170.67 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1351/3051 [10:40<13:22,  2.12it/s, est. speed input: 69170.82 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1352/3051 [10:40<13:22,  2.12it/s, est. speed input: 69170.89 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1353/3051 [10:40<13:22,  2.12it/s, est. speed input: 69171.08 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1354/3051 [10:41<13:21,  2.12it/s, est. speed input: 69171.16 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1355/3051 [10:41<13:21,  2.12it/s, est. speed input: 69171.37 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1356/3051 [10:42<13:20,  2.12it/s, est. speed input: 69171.57 toks/s, output: 2.11 toks/s]
Processed prompts:  44%|████▍     | 1357/3051 [10:42<13:23,  2.11it/s, est. speed input: 69171.06 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1358/3051 [10:43<13:21,  2.11it/s, est. speed input: 69171.20 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1359/3051 [10:43<13:20,  2.11it/s, est. speed input: 69171.38 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1360/3051 [10:44<13:32,  2.08it/s, est. speed input: 69168.80 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1361/3051 [10:44<13:27,  2.09it/s, est. speed input: 69168.98 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1362/3051 [10:45<13:24,  2.10it/s, est. speed input: 69169.21 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1363/3051 [10:45<13:21,  2.11it/s, est. speed input: 69169.37 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1364/3051 [10:46<13:20,  2.11it/s, est. speed input: 69169.38 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1365/3051 [10:46<13:19,  2.11it/s, est. speed input: 69169.47 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1366/3051 [10:47<13:18,  2.11it/s, est. speed input: 69169.56 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1367/3051 [10:47<13:17,  2.11it/s, est. speed input: 69169.70 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1368/3051 [10:48<13:16,  2.11it/s, est. speed input: 69169.85 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1369/3051 [10:48<13:15,  2.12it/s, est. speed input: 69170.04 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1370/3051 [10:49<13:14,  2.11it/s, est. speed input: 69170.10 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1371/3051 [10:49<13:15,  2.11it/s, est. speed input: 69170.06 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▍     | 1372/3051 [10:49<13:13,  2.11it/s, est. speed input: 69170.25 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1373/3051 [10:50<13:13,  2.12it/s, est. speed input: 69170.40 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1374/3051 [10:50<13:12,  2.12it/s, est. speed input: 69170.58 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1375/3051 [10:51<13:18,  2.10it/s, est. speed input: 69169.25 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1376/3051 [10:51<13:29,  2.07it/s, est. speed input: 69166.63 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1377/3051 [10:52<13:23,  2.08it/s, est. speed input: 69166.71 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1378/3051 [10:52<13:20,  2.09it/s, est. speed input: 69166.60 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1379/3051 [10:53<13:16,  2.10it/s, est. speed input: 69166.77 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1380/3051 [10:53<13:14,  2.10it/s, est. speed input: 69166.89 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1381/3051 [10:54<13:12,  2.11it/s, est. speed input: 69167.07 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1382/3051 [10:54<13:11,  2.11it/s, est. speed input: 69167.13 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1383/3051 [10:55<13:10,  2.11it/s, est. speed input: 69167.21 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1384/3051 [10:55<13:09,  2.11it/s, est. speed input: 69167.34 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1385/3051 [10:56<13:07,  2.11it/s, est. speed input: 69167.53 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1386/3051 [10:56<13:07,  2.11it/s, est. speed input: 69167.64 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1387/3051 [10:57<13:06,  2.12it/s, est. speed input: 69167.78 toks/s, output: 2.11 toks/s]
Processed prompts:  45%|████▌     | 1388/3051 [10:57<13:05,  2.12it/s, est. speed input: 69167.97 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1389/3051 [10:58<13:05,  2.12it/s, est. speed input: 69168.05 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1390/3051 [10:58<13:06,  2.11it/s, est. speed input: 69167.81 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1391/3051 [10:58<13:05,  2.11it/s, est. speed input: 69167.98 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1392/3051 [10:59<13:04,  2.12it/s, est. speed input: 69168.19 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1393/3051 [10:59<13:04,  2.11it/s, est. speed input: 69168.22 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1394/3051 [11:00<13:03,  2.12it/s, est. speed input: 69168.44 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1395/3051 [11:00<13:04,  2.11it/s, est. speed input: 69168.08 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1396/3051 [11:01<13:03,  2.11it/s, est. speed input: 69168.25 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1397/3051 [11:01<13:02,  2.11it/s, est. speed input: 69168.39 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1398/3051 [11:02<13:01,  2.11it/s, est. speed input: 69168.56 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1399/3051 [11:02<13:00,  2.12it/s, est. speed input: 69168.70 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1400/3051 [11:03<13:00,  2.12it/s, est. speed input: 69168.87 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1401/3051 [11:03<12:59,  2.12it/s, est. speed input: 69169.02 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1402/3051 [11:04<12:59,  2.12it/s, est. speed input: 69169.13 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1403/3051 [11:04<12:59,  2.11it/s, est. speed input: 69169.11 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1404/3051 [11:05<12:58,  2.12it/s, est. speed input: 69169.30 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1405/3051 [11:05<13:00,  2.11it/s, est. speed input: 69168.98 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1406/3051 [11:06<12:59,  2.11it/s, est. speed input: 69169.03 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1407/3051 [11:06<12:58,  2.11it/s, est. speed input: 69169.17 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1408/3051 [11:07<12:57,  2.11it/s, est. speed input: 69169.27 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1409/3051 [11:07<12:56,  2.11it/s, est. speed input: 69169.34 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1410/3051 [11:07<12:56,  2.11it/s, est. speed input: 69169.47 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▌     | 1411/3051 [11:08<12:56,  2.11it/s, est. speed input: 69169.33 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1412/3051 [11:08<12:55,  2.11it/s, est. speed input: 69169.43 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1413/3051 [11:09<12:54,  2.11it/s, est. speed input: 69169.61 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1414/3051 [11:09<12:53,  2.12it/s, est. speed input: 69169.77 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1415/3051 [11:10<12:53,  2.12it/s, est. speed input: 69169.95 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1416/3051 [11:10<12:52,  2.12it/s, est. speed input: 69170.04 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1417/3051 [11:11<12:52,  2.12it/s, est. speed input: 69170.17 toks/s, output: 2.11 toks/s]
Processed prompts:  46%|████▋     | 1418/3051 [11:11<12:51,  2.12it/s, est. speed input: 69170.31 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1419/3051 [11:12<12:50,  2.12it/s, est. speed input: 69170.47 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1420/3051 [11:12<12:50,  2.12it/s, est. speed input: 69170.65 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1421/3051 [11:13<12:49,  2.12it/s, est. speed input: 69170.77 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1422/3051 [11:13<12:49,  2.12it/s, est. speed input: 69170.88 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1423/3051 [11:14<12:49,  2.12it/s, est. speed input: 69171.04 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1424/3051 [11:14<12:48,  2.12it/s, est. speed input: 69171.14 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1425/3051 [11:15<12:48,  2.11it/s, est. speed input: 69171.12 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1426/3051 [11:15<12:48,  2.12it/s, est. speed input: 69171.25 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1427/3051 [11:16<12:47,  2.12it/s, est. speed input: 69171.41 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1428/3051 [11:16<12:46,  2.12it/s, est. speed input: 69171.54 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1429/3051 [11:16<12:46,  2.12it/s, est. speed input: 69171.65 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1430/3051 [11:17<12:45,  2.12it/s, est. speed input: 69171.83 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1431/3051 [11:17<12:45,  2.12it/s, est. speed input: 69172.00 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1432/3051 [11:18<12:44,  2.12it/s, est. speed input: 69172.10 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1433/3051 [11:18<12:44,  2.12it/s, est. speed input: 69172.21 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1434/3051 [11:19<12:43,  2.12it/s, est. speed input: 69172.34 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1435/3051 [11:19<12:43,  2.12it/s, est. speed input: 69172.43 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1436/3051 [11:20<12:43,  2.12it/s, est. speed input: 69172.56 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1437/3051 [11:20<12:42,  2.12it/s, est. speed input: 69172.67 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1438/3051 [11:21<12:42,  2.12it/s, est. speed input: 69172.79 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1439/3051 [11:21<12:41,  2.12it/s, est. speed input: 69172.96 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1440/3051 [11:22<12:40,  2.12it/s, est. speed input: 69173.15 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1441/3051 [11:22<12:40,  2.12it/s, est. speed input: 69173.28 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1442/3051 [11:23<12:40,  2.12it/s, est. speed input: 69173.39 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1443/3051 [11:23<12:39,  2.12it/s, est. speed input: 69173.52 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1444/3051 [11:24<12:39,  2.12it/s, est. speed input: 69173.62 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1445/3051 [11:24<12:38,  2.12it/s, est. speed input: 69173.84 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1446/3051 [11:24<12:37,  2.12it/s, est. speed input: 69174.02 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1447/3051 [11:25<12:37,  2.12it/s, est. speed input: 69174.15 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1448/3051 [11:25<12:37,  2.12it/s, est. speed input: 69174.24 toks/s, output: 2.11 toks/s]
Processed prompts:  47%|████▋     | 1449/3051 [11:26<12:37,  2.12it/s, est. speed input: 69174.23 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1450/3051 [11:26<12:36,  2.12it/s, est. speed input: 69174.43 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1451/3051 [11:27<12:36,  2.11it/s, est. speed input: 69174.39 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1452/3051 [11:27<12:36,  2.11it/s, est. speed input: 69174.49 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1453/3051 [11:28<12:35,  2.12it/s, est. speed input: 69174.63 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1454/3051 [11:28<12:35,  2.11it/s, est. speed input: 69174.59 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1455/3051 [11:29<12:34,  2.11it/s, est. speed input: 69174.72 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1456/3051 [11:29<12:34,  2.11it/s, est. speed input: 69174.83 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1457/3051 [11:30<12:33,  2.12it/s, est. speed input: 69175.00 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1458/3051 [11:30<12:32,  2.12it/s, est. speed input: 69175.19 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1459/3051 [11:31<12:31,  2.12it/s, est. speed input: 69175.33 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1460/3051 [11:31<12:31,  2.12it/s, est. speed input: 69175.45 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1461/3051 [11:32<12:33,  2.11it/s, est. speed input: 69175.16 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1462/3051 [11:32<12:32,  2.11it/s, est. speed input: 69175.25 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1463/3051 [11:33<12:31,  2.11it/s, est. speed input: 69175.37 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1464/3051 [11:33<12:30,  2.11it/s, est. speed input: 69175.44 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1465/3051 [11:33<12:30,  2.11it/s, est. speed input: 69175.50 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1466/3051 [11:34<12:29,  2.12it/s, est. speed input: 69175.70 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1467/3051 [11:34<12:28,  2.12it/s, est. speed input: 69175.80 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1468/3051 [11:35<12:28,  2.12it/s, est. speed input: 69175.91 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1469/3051 [11:35<12:27,  2.12it/s, est. speed input: 69175.98 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1470/3051 [11:36<12:27,  2.12it/s, est. speed input: 69176.14 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1471/3051 [11:36<12:26,  2.12it/s, est. speed input: 69176.24 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1472/3051 [11:37<12:25,  2.12it/s, est. speed input: 69176.42 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1473/3051 [11:37<12:26,  2.11it/s, est. speed input: 69176.39 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1474/3051 [11:38<12:25,  2.11it/s, est. speed input: 69176.46 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1475/3051 [11:38<12:25,  2.11it/s, est. speed input: 69176.56 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1476/3051 [11:39<12:25,  2.11it/s, est. speed input: 69176.44 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1477/3051 [11:39<12:24,  2.11it/s, est. speed input: 69176.53 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1478/3051 [11:40<12:24,  2.11it/s, est. speed input: 69176.63 toks/s, output: 2.11 toks/s]
Processed prompts:  48%|████▊     | 1479/3051 [11:40<12:23,  2.12it/s, est. speed input: 69176.80 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1480/3051 [11:41<12:22,  2.12it/s, est. speed input: 69176.93 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1481/3051 [11:41<12:22,  2.12it/s, est. speed input: 69177.02 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1482/3051 [11:41<12:21,  2.12it/s, est. speed input: 69177.14 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1483/3051 [11:42<12:20,  2.12it/s, est. speed input: 69177.27 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1484/3051 [11:42<12:20,  2.12it/s, est. speed input: 69177.41 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1485/3051 [11:43<12:19,  2.12it/s, est. speed input: 69177.58 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1486/3051 [11:43<12:19,  2.12it/s, est. speed input: 69177.64 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▊     | 1487/3051 [11:44<12:18,  2.12it/s, est. speed input: 69177.77 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1488/3051 [11:44<12:18,  2.12it/s, est. speed input: 69177.86 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1489/3051 [11:45<12:18,  2.12it/s, est. speed input: 69177.98 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1490/3051 [11:45<12:17,  2.12it/s, est. speed input: 69178.05 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1491/3051 [11:46<12:17,  2.12it/s, est. speed input: 69178.14 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1492/3051 [11:46<12:17,  2.11it/s, est. speed input: 69178.17 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1493/3051 [11:47<12:17,  2.11it/s, est. speed input: 69178.01 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1494/3051 [11:47<12:16,  2.11it/s, est. speed input: 69178.18 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1495/3051 [11:48<12:16,  2.11it/s, est. speed input: 69178.25 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1496/3051 [11:48<12:15,  2.11it/s, est. speed input: 69178.33 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1497/3051 [11:49<12:14,  2.11it/s, est. speed input: 69178.44 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1498/3051 [11:49<12:14,  2.12it/s, est. speed input: 69178.58 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1499/3051 [11:50<12:13,  2.11it/s, est. speed input: 69178.62 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1500/3051 [11:50<12:13,  2.12it/s, est. speed input: 69178.74 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1501/3051 [11:50<12:12,  2.12it/s, est. speed input: 69178.85 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1502/3051 [11:51<12:12,  2.12it/s, est. speed input: 69178.92 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1503/3051 [11:51<12:12,  2.11it/s, est. speed input: 69178.94 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1504/3051 [11:52<12:11,  2.11it/s, est. speed input: 69179.03 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1505/3051 [11:52<12:10,  2.12it/s, est. speed input: 69179.21 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1506/3051 [11:53<12:10,  2.12it/s, est. speed input: 69179.27 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1507/3051 [11:53<12:09,  2.12it/s, est. speed input: 69179.41 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1508/3051 [11:54<12:09,  2.12it/s, est. speed input: 69179.50 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1509/3051 [11:54<12:09,  2.11it/s, est. speed input: 69179.48 toks/s, output: 2.11 toks/s]
Processed prompts:  49%|████▉     | 1510/3051 [11:55<12:10,  2.11it/s, est. speed input: 69179.30 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1511/3051 [11:55<12:08,  2.11it/s, est. speed input: 69179.41 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1512/3051 [11:56<12:08,  2.11it/s, est. speed input: 69179.51 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1513/3051 [11:56<12:07,  2.12it/s, est. speed input: 69179.69 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1514/3051 [11:57<12:06,  2.12it/s, est. speed input: 69179.81 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1515/3051 [11:57<12:07,  2.11it/s, est. speed input: 69179.66 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1516/3051 [11:58<12:07,  2.11it/s, est. speed input: 69179.57 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1517/3051 [11:58<12:06,  2.11it/s, est. speed input: 69179.69 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1518/3051 [11:59<12:04,  2.11it/s, est. speed input: 69179.88 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1519/3051 [11:59<12:04,  2.12it/s, est. speed input: 69180.02 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1520/3051 [11:59<12:03,  2.12it/s, est. speed input: 69180.14 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1521/3051 [12:00<12:03,  2.12it/s, est. speed input: 69180.23 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1522/3051 [12:00<12:02,  2.12it/s, est. speed input: 69180.34 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1523/3051 [12:01<12:02,  2.12it/s, est. speed input: 69180.43 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1524/3051 [12:01<12:01,  2.12it/s, est. speed input: 69180.60 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|████▉     | 1525/3051 [12:02<12:01,  2.11it/s, est. speed input: 69180.56 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1526/3051 [12:02<12:01,  2.11it/s, est. speed input: 69180.65 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1527/3051 [12:03<12:00,  2.12it/s, est. speed input: 69180.80 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1528/3051 [12:03<12:02,  2.11it/s, est. speed input: 69180.43 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1529/3051 [12:04<12:01,  2.11it/s, est. speed input: 69180.47 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1530/3051 [12:04<11:59,  2.11it/s, est. speed input: 69180.67 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1531/3051 [12:05<11:59,  2.11it/s, est. speed input: 69180.74 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1532/3051 [12:05<11:58,  2.11it/s, est. speed input: 69180.84 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1533/3051 [12:06<11:57,  2.11it/s, est. speed input: 69180.94 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1534/3051 [12:06<11:57,  2.12it/s, est. speed input: 69181.06 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1535/3051 [12:07<11:56,  2.12it/s, est. speed input: 69181.23 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1536/3051 [12:07<11:55,  2.12it/s, est. speed input: 69181.38 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1537/3051 [12:08<11:55,  2.12it/s, est. speed input: 69181.46 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1538/3051 [12:08<11:55,  2.11it/s, est. speed input: 69181.36 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1539/3051 [12:08<11:54,  2.12it/s, est. speed input: 69181.52 toks/s, output: 2.11 toks/s]
Processed prompts:  50%|█████     | 1540/3051 [12:09<11:54,  2.12it/s, est. speed input: 69181.66 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1541/3051 [12:09<11:53,  2.12it/s, est. speed input: 69181.80 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1542/3051 [12:10<11:53,  2.11it/s, est. speed input: 69181.80 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1543/3051 [12:10<11:52,  2.12it/s, est. speed input: 69181.93 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1544/3051 [12:11<11:52,  2.12it/s, est. speed input: 69182.08 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1545/3051 [12:11<11:53,  2.11it/s, est. speed input: 69181.74 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1546/3051 [12:12<11:52,  2.11it/s, est. speed input: 69181.92 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1547/3051 [12:12<11:51,  2.11it/s, est. speed input: 69182.04 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1548/3051 [12:13<11:50,  2.11it/s, est. speed input: 69182.13 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1549/3051 [12:13<11:50,  2.11it/s, est. speed input: 69182.19 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1550/3051 [12:14<11:49,  2.12it/s, est. speed input: 69182.36 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1551/3051 [12:14<11:49,  2.12it/s, est. speed input: 69182.43 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1552/3051 [12:15<11:48,  2.12it/s, est. speed input: 69182.53 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1553/3051 [12:15<11:48,  2.11it/s, est. speed input: 69182.55 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1554/3051 [12:16<11:47,  2.12it/s, est. speed input: 69182.71 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1555/3051 [12:16<11:47,  2.12it/s, est. speed input: 69182.80 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1556/3051 [12:16<11:46,  2.12it/s, est. speed input: 69182.92 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1557/3051 [12:17<11:46,  2.11it/s, est. speed input: 69182.92 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1558/3051 [12:17<11:46,  2.11it/s, est. speed input: 69182.99 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1559/3051 [12:18<11:45,  2.11it/s, est. speed input: 69183.08 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1560/3051 [12:18<11:50,  2.10it/s, est. speed input: 69182.08 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1561/3051 [12:19<11:47,  2.11it/s, est. speed input: 69182.22 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1562/3051 [12:19<11:46,  2.11it/s, est. speed input: 69182.28 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████     | 1563/3051 [12:20<11:44,  2.11it/s, est. speed input: 69182.45 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1564/3051 [12:20<11:43,  2.11it/s, est. speed input: 69182.61 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1565/3051 [12:21<11:42,  2.11it/s, est. speed input: 69182.70 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1566/3051 [12:21<11:42,  2.12it/s, est. speed input: 69182.82 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1567/3051 [12:22<11:41,  2.12it/s, est. speed input: 69182.97 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1568/3051 [12:22<11:41,  2.12it/s, est. speed input: 69183.02 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1569/3051 [12:23<11:40,  2.12it/s, est. speed input: 69183.15 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1570/3051 [12:23<11:39,  2.12it/s, est. speed input: 69183.23 toks/s, output: 2.11 toks/s]
Processed prompts:  51%|█████▏    | 1571/3051 [12:24<11:39,  2.12it/s, est. speed input: 69183.37 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1572/3051 [12:24<11:38,  2.12it/s, est. speed input: 69183.50 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1573/3051 [12:25<11:38,  2.12it/s, est. speed input: 69183.58 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1574/3051 [12:25<11:37,  2.12it/s, est. speed input: 69183.68 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1575/3051 [12:25<11:37,  2.12it/s, est. speed input: 69183.75 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1576/3051 [12:26<11:36,  2.12it/s, est. speed input: 69183.91 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1577/3051 [12:26<11:36,  2.12it/s, est. speed input: 69184.04 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1578/3051 [12:27<11:36,  2.12it/s, est. speed input: 69184.05 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1579/3051 [12:27<11:35,  2.12it/s, est. speed input: 69184.16 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1580/3051 [12:28<11:35,  2.12it/s, est. speed input: 69184.22 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1581/3051 [12:28<11:34,  2.12it/s, est. speed input: 69184.35 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1582/3051 [12:29<11:33,  2.12it/s, est. speed input: 69184.52 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1583/3051 [12:29<11:35,  2.11it/s, est. speed input: 69184.23 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1584/3051 [12:30<11:34,  2.11it/s, est. speed input: 69184.34 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1585/3051 [12:30<11:33,  2.11it/s, est. speed input: 69184.47 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1586/3051 [12:31<11:35,  2.11it/s, est. speed input: 69184.06 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1587/3051 [12:31<11:33,  2.11it/s, est. speed input: 69184.21 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1588/3051 [12:32<11:32,  2.11it/s, est. speed input: 69184.31 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1589/3051 [12:32<11:31,  2.11it/s, est. speed input: 69184.41 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1590/3051 [12:33<11:30,  2.11it/s, est. speed input: 69184.51 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1591/3051 [12:33<11:30,  2.11it/s, est. speed input: 69184.54 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1592/3051 [12:34<11:30,  2.11it/s, est. speed input: 69184.63 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1593/3051 [12:34<11:29,  2.11it/s, est. speed input: 69184.69 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1594/3051 [12:34<11:29,  2.11it/s, est. speed input: 69184.74 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1595/3051 [12:35<11:28,  2.12it/s, est. speed input: 69184.86 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1596/3051 [12:35<11:27,  2.12it/s, est. speed input: 69184.97 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1597/3051 [12:36<11:27,  2.12it/s, est. speed input: 69185.11 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1598/3051 [12:36<11:26,  2.12it/s, est. speed input: 69185.21 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1599/3051 [12:37<11:26,  2.12it/s, est. speed input: 69185.33 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1600/3051 [12:37<11:25,  2.12it/s, est. speed input: 69185.47 toks/s, output: 2.11 toks/s]
Processed prompts:  52%|█████▏    | 1601/3051 [12:38<11:25,  2.12it/s, est. speed input: 69185.53 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1602/3051 [12:38<11:24,  2.12it/s, est. speed input: 69185.65 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1603/3051 [12:39<11:24,  2.12it/s, est. speed input: 69185.78 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1604/3051 [12:39<11:23,  2.12it/s, est. speed input: 69185.87 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1605/3051 [12:40<11:23,  2.12it/s, est. speed input: 69185.97 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1606/3051 [12:40<11:22,  2.12it/s, est. speed input: 69186.12 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1607/3051 [12:41<11:22,  2.12it/s, est. speed input: 69186.24 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1608/3051 [12:41<11:21,  2.12it/s, est. speed input: 69186.36 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1609/3051 [12:42<11:21,  2.12it/s, est. speed input: 69186.38 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1610/3051 [12:42<11:22,  2.11it/s, est. speed input: 69186.21 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1611/3051 [12:43<11:21,  2.11it/s, est. speed input: 69186.31 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1612/3051 [12:43<11:20,  2.11it/s, est. speed input: 69186.41 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1613/3051 [12:43<11:20,  2.11it/s, est. speed input: 69186.43 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1614/3051 [12:44<11:19,  2.12it/s, est. speed input: 69186.61 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1615/3051 [12:44<11:19,  2.11it/s, est. speed input: 69186.56 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1616/3051 [12:45<11:18,  2.11it/s, est. speed input: 69186.65 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1617/3051 [12:45<11:17,  2.12it/s, est. speed input: 69186.76 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1618/3051 [12:46<11:17,  2.12it/s, est. speed input: 69186.86 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1619/3051 [12:46<11:16,  2.12it/s, est. speed input: 69186.98 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1620/3051 [12:47<11:16,  2.12it/s, est. speed input: 69187.06 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1621/3051 [12:47<11:15,  2.12it/s, est. speed input: 69187.22 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1622/3051 [12:48<11:15,  2.12it/s, est. speed input: 69187.29 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1623/3051 [12:48<11:14,  2.12it/s, est. speed input: 69187.39 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1624/3051 [12:49<11:14,  2.12it/s, est. speed input: 69187.48 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1625/3051 [12:49<11:13,  2.12it/s, est. speed input: 69187.62 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1626/3051 [12:50<11:13,  2.12it/s, est. speed input: 69187.73 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1627/3051 [12:50<11:13,  2.12it/s, est. speed input: 69187.77 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1628/3051 [12:51<11:12,  2.12it/s, est. speed input: 69187.89 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1629/3051 [12:51<11:11,  2.12it/s, est. speed input: 69188.02 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1630/3051 [12:51<11:11,  2.12it/s, est. speed input: 69188.12 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1631/3051 [12:52<11:10,  2.12it/s, est. speed input: 69188.27 toks/s, output: 2.11 toks/s]
Processed prompts:  53%|█████▎    | 1632/3051 [12:52<11:10,  2.12it/s, est. speed input: 69188.38 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1633/3051 [12:53<11:09,  2.12it/s, est. speed input: 69188.45 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1634/3051 [12:53<11:09,  2.12it/s, est. speed input: 69188.63 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1635/3051 [12:54<11:08,  2.12it/s, est. speed input: 69188.76 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1636/3051 [12:54<11:08,  2.12it/s, est. speed input: 69188.89 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1637/3051 [12:55<11:07,  2.12it/s, est. speed input: 69188.99 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1638/3051 [12:55<11:08,  2.11it/s, est. speed input: 69188.90 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▎    | 1639/3051 [12:56<11:07,  2.11it/s, est. speed input: 69188.98 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1640/3051 [12:56<11:06,  2.12it/s, est. speed input: 69189.11 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1641/3051 [12:57<11:06,  2.12it/s, est. speed input: 69189.18 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1642/3051 [12:57<11:05,  2.12it/s, est. speed input: 69189.28 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1643/3051 [12:58<11:05,  2.12it/s, est. speed input: 69189.40 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1644/3051 [12:58<11:04,  2.12it/s, est. speed input: 69189.55 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1645/3051 [12:59<11:03,  2.12it/s, est. speed input: 69189.70 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1646/3051 [12:59<11:03,  2.12it/s, est. speed input: 69189.82 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1647/3051 [13:00<11:03,  2.12it/s, est. speed input: 69189.92 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1648/3051 [13:00<11:02,  2.12it/s, est. speed input: 69190.00 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1649/3051 [13:00<11:02,  2.12it/s, est. speed input: 69190.00 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1650/3051 [13:01<11:02,  2.12it/s, est. speed input: 69190.14 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1651/3051 [13:01<11:02,  2.11it/s, est. speed input: 69190.04 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1652/3051 [13:02<11:01,  2.11it/s, est. speed input: 69190.15 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1653/3051 [13:02<11:01,  2.11it/s, est. speed input: 69190.24 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1654/3051 [13:03<11:00,  2.12it/s, est. speed input: 69190.37 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1655/3051 [13:03<10:59,  2.12it/s, est. speed input: 69190.50 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1656/3051 [13:04<10:59,  2.12it/s, est. speed input: 69190.61 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1657/3051 [13:04<10:58,  2.12it/s, est. speed input: 69190.68 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1658/3051 [13:05<10:58,  2.12it/s, est. speed input: 69190.78 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1659/3051 [13:05<10:57,  2.12it/s, est. speed input: 69190.84 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1660/3051 [13:06<10:57,  2.12it/s, est. speed input: 69190.96 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1661/3051 [13:06<10:57,  2.12it/s, est. speed input: 69190.99 toks/s, output: 2.11 toks/s]
Processed prompts:  54%|█████▍    | 1662/3051 [13:07<10:56,  2.12it/s, est. speed input: 69191.07 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1663/3051 [13:07<10:55,  2.12it/s, est. speed input: 69191.20 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1664/3051 [13:08<10:55,  2.12it/s, est. speed input: 69191.29 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1665/3051 [13:08<10:55,  2.12it/s, est. speed input: 69191.33 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1666/3051 [13:08<10:54,  2.11it/s, est. speed input: 69191.36 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1667/3051 [13:09<10:54,  2.12it/s, est. speed input: 69191.46 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1668/3051 [13:09<10:53,  2.12it/s, est. speed input: 69191.60 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1669/3051 [13:10<10:53,  2.12it/s, est. speed input: 69191.69 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1670/3051 [13:10<10:52,  2.12it/s, est. speed input: 69191.77 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1671/3051 [13:11<10:51,  2.12it/s, est. speed input: 69191.91 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1672/3051 [13:11<10:51,  2.12it/s, est. speed input: 69192.01 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1673/3051 [13:12<10:50,  2.12it/s, est. speed input: 69192.14 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1674/3051 [13:12<10:50,  2.12it/s, est. speed input: 69192.14 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1675/3051 [13:13<10:50,  2.11it/s, est. speed input: 69192.17 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1676/3051 [13:13<10:49,  2.12it/s, est. speed input: 69192.31 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1677/3051 [13:14<10:50,  2.11it/s, est. speed input: 69192.21 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▍    | 1678/3051 [13:14<10:49,  2.11it/s, est. speed input: 69192.25 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1679/3051 [13:15<10:49,  2.11it/s, est. speed input: 69192.29 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1680/3051 [13:15<10:48,  2.12it/s, est. speed input: 69192.47 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1681/3051 [13:16<10:47,  2.12it/s, est. speed input: 69192.58 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1682/3051 [13:16<10:47,  2.12it/s, est. speed input: 69192.62 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1683/3051 [13:17<10:46,  2.12it/s, est. speed input: 69192.73 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1684/3051 [13:17<10:45,  2.12it/s, est. speed input: 69192.83 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1685/3051 [13:17<10:45,  2.12it/s, est. speed input: 69192.89 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1686/3051 [13:18<10:45,  2.12it/s, est. speed input: 69192.97 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1687/3051 [13:18<10:44,  2.12it/s, est. speed input: 69193.04 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1688/3051 [13:19<10:44,  2.12it/s, est. speed input: 69193.11 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1689/3051 [13:19<10:43,  2.12it/s, est. speed input: 69193.21 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1690/3051 [13:20<10:43,  2.11it/s, est. speed input: 69193.18 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1691/3051 [13:20<10:43,  2.11it/s, est. speed input: 69193.26 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1692/3051 [13:21<10:42,  2.12it/s, est. speed input: 69193.35 toks/s, output: 2.11 toks/s]
Processed prompts:  55%|█████▌    | 1693/3051 [13:21<10:41,  2.12it/s, est. speed input: 69193.46 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1694/3051 [13:22<10:41,  2.12it/s, est. speed input: 69193.51 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1695/3051 [13:22<10:40,  2.12it/s, est. speed input: 69193.66 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1696/3051 [13:23<10:40,  2.12it/s, est. speed input: 69193.72 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1697/3051 [13:23<10:39,  2.12it/s, est. speed input: 69193.79 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1698/3051 [13:24<10:39,  2.12it/s, est. speed input: 69193.91 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1699/3051 [13:24<10:38,  2.12it/s, est. speed input: 69193.97 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1700/3051 [13:25<10:38,  2.12it/s, est. speed input: 69194.11 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1701/3051 [13:25<10:37,  2.12it/s, est. speed input: 69194.25 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1702/3051 [13:26<10:37,  2.12it/s, est. speed input: 69194.38 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1703/3051 [13:26<10:36,  2.12it/s, est. speed input: 69194.48 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1704/3051 [13:26<10:37,  2.11it/s, est. speed input: 69194.38 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1705/3051 [13:27<10:36,  2.11it/s, est. speed input: 69194.44 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1706/3051 [13:27<10:35,  2.12it/s, est. speed input: 69194.56 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1707/3051 [13:28<10:35,  2.12it/s, est. speed input: 69194.70 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1708/3051 [13:28<10:34,  2.12it/s, est. speed input: 69194.82 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1709/3051 [13:29<10:34,  2.12it/s, est. speed input: 69194.90 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1710/3051 [13:29<10:33,  2.12it/s, est. speed input: 69195.02 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1711/3051 [13:30<10:32,  2.12it/s, est. speed input: 69195.15 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1712/3051 [13:30<10:32,  2.12it/s, est. speed input: 69195.24 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1713/3051 [13:31<10:34,  2.11it/s, est. speed input: 69194.81 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1714/3051 [13:31<10:33,  2.11it/s, est. speed input: 69194.95 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1715/3051 [13:32<10:32,  2.11it/s, est. speed input: 69195.05 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▌    | 1716/3051 [13:32<10:31,  2.11it/s, est. speed input: 69195.18 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1717/3051 [13:33<10:30,  2.12it/s, est. speed input: 69195.27 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1718/3051 [13:33<10:30,  2.11it/s, est. speed input: 69195.32 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1719/3051 [13:34<10:29,  2.12it/s, est. speed input: 69195.43 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1720/3051 [13:34<10:28,  2.12it/s, est. speed input: 69195.57 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1721/3051 [13:34<10:28,  2.12it/s, est. speed input: 69195.64 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1722/3051 [13:35<10:27,  2.12it/s, est. speed input: 69195.75 toks/s, output: 2.11 toks/s]
Processed prompts:  56%|█████▋    | 1723/3051 [13:35<10:27,  2.12it/s, est. speed input: 69195.77 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1724/3051 [13:36<10:27,  2.12it/s, est. speed input: 69195.85 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1725/3051 [13:36<10:26,  2.12it/s, est. speed input: 69195.94 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1726/3051 [13:37<10:27,  2.11it/s, est. speed input: 69195.79 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1727/3051 [13:37<10:26,  2.11it/s, est. speed input: 69195.90 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1728/3051 [13:38<10:25,  2.12it/s, est. speed input: 69196.03 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1729/3051 [13:38<10:24,  2.12it/s, est. speed input: 69196.11 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1730/3051 [13:39<10:24,  2.12it/s, est. speed input: 69196.27 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1731/3051 [13:39<10:23,  2.12it/s, est. speed input: 69196.34 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1732/3051 [13:40<10:23,  2.12it/s, est. speed input: 69196.41 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1733/3051 [13:40<10:22,  2.12it/s, est. speed input: 69196.49 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1734/3051 [13:41<10:22,  2.12it/s, est. speed input: 69196.55 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1735/3051 [13:41<10:21,  2.12it/s, est. speed input: 69196.64 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1736/3051 [13:42<10:23,  2.11it/s, est. speed input: 69196.36 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1737/3051 [13:42<10:22,  2.11it/s, est. speed input: 69196.42 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1738/3051 [13:43<10:21,  2.11it/s, est. speed input: 69196.52 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1739/3051 [13:43<10:20,  2.11it/s, est. speed input: 69196.64 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1740/3051 [13:43<10:20,  2.11it/s, est. speed input: 69196.65 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1741/3051 [13:44<10:19,  2.12it/s, est. speed input: 69196.80 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1742/3051 [13:44<10:18,  2.12it/s, est. speed input: 69196.89 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1743/3051 [13:45<10:18,  2.11it/s, est. speed input: 69196.87 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1744/3051 [13:45<10:18,  2.11it/s, est. speed input: 69196.89 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1745/3051 [13:46<10:17,  2.12it/s, est. speed input: 69197.03 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1746/3051 [13:46<10:17,  2.11it/s, est. speed input: 69196.98 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1747/3051 [13:47<10:16,  2.11it/s, est. speed input: 69197.07 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1748/3051 [13:47<10:15,  2.12it/s, est. speed input: 69197.19 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1749/3051 [13:48<10:15,  2.12it/s, est. speed input: 69197.30 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1750/3051 [13:48<10:14,  2.12it/s, est. speed input: 69197.45 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1751/3051 [13:49<10:13,  2.12it/s, est. speed input: 69197.56 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1752/3051 [13:49<10:13,  2.12it/s, est. speed input: 69197.63 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1753/3051 [13:50<10:12,  2.12it/s, est. speed input: 69197.78 toks/s, output: 2.11 toks/s]
Processed prompts:  57%|█████▋    | 1754/3051 [13:50<10:12,  2.12it/s, est. speed input: 69197.90 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1755/3051 [13:51<10:12,  2.12it/s, est. speed input: 69197.89 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1756/3051 [13:51<10:11,  2.12it/s, est. speed input: 69197.98 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1757/3051 [13:52<10:11,  2.12it/s, est. speed input: 69198.11 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1758/3051 [13:52<10:10,  2.12it/s, est. speed input: 69198.19 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1759/3051 [13:52<10:10,  2.12it/s, est. speed input: 69198.20 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1760/3051 [13:53<10:11,  2.11it/s, est. speed input: 69198.10 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1761/3051 [13:53<10:10,  2.11it/s, est. speed input: 69198.15 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1762/3051 [13:54<10:09,  2.11it/s, est. speed input: 69198.23 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1763/3051 [13:54<10:09,  2.11it/s, est. speed input: 69198.24 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1764/3051 [13:55<10:08,  2.11it/s, est. speed input: 69198.31 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1765/3051 [13:55<10:08,  2.11it/s, est. speed input: 69198.25 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1766/3051 [13:56<10:08,  2.11it/s, est. speed input: 69198.29 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1767/3051 [13:56<10:07,  2.11it/s, est. speed input: 69198.42 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1768/3051 [13:57<10:07,  2.11it/s, est. speed input: 69198.20 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1769/3051 [13:57<10:07,  2.11it/s, est. speed input: 69198.14 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1770/3051 [13:58<10:06,  2.11it/s, est. speed input: 69198.21 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1771/3051 [13:58<10:05,  2.11it/s, est. speed input: 69198.39 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1772/3051 [13:59<10:04,  2.12it/s, est. speed input: 69198.52 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1773/3051 [13:59<10:04,  2.11it/s, est. speed input: 69198.54 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1774/3051 [14:00<10:03,  2.12it/s, est. speed input: 69198.62 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1775/3051 [14:00<10:03,  2.12it/s, est. speed input: 69198.67 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1776/3051 [14:00<10:02,  2.12it/s, est. speed input: 69198.83 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1777/3051 [14:01<10:01,  2.12it/s, est. speed input: 69198.95 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1778/3051 [14:01<10:03,  2.11it/s, est. speed input: 69198.67 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1779/3051 [14:02<10:02,  2.11it/s, est. speed input: 69198.67 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1780/3051 [14:02<10:01,  2.11it/s, est. speed input: 69198.80 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1781/3051 [14:03<10:01,  2.11it/s, est. speed input: 69198.65 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1782/3051 [14:03<10:00,  2.11it/s, est. speed input: 69198.76 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1783/3051 [14:04<09:59,  2.11it/s, est. speed input: 69198.86 toks/s, output: 2.11 toks/s]
Processed prompts:  58%|█████▊    | 1784/3051 [14:04<09:59,  2.12it/s, est. speed input: 69198.96 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1785/3051 [14:05<09:58,  2.12it/s, est. speed input: 69199.11 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1786/3051 [14:05<09:57,  2.12it/s, est. speed input: 69199.21 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1787/3051 [14:06<09:57,  2.12it/s, est. speed input: 69199.27 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1788/3051 [14:06<09:57,  2.11it/s, est. speed input: 69199.15 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1789/3051 [14:07<09:56,  2.11it/s, est. speed input: 69199.27 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1790/3051 [14:07<09:56,  2.12it/s, est. speed input: 69199.37 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1791/3051 [14:08<09:55,  2.12it/s, est. speed input: 69199.48 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▊    | 1792/3051 [14:08<09:54,  2.12it/s, est. speed input: 69199.60 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1793/3051 [14:09<09:54,  2.12it/s, est. speed input: 69199.69 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1794/3051 [14:09<09:53,  2.12it/s, est. speed input: 69199.79 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1795/3051 [14:09<09:53,  2.12it/s, est. speed input: 69199.78 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1796/3051 [14:10<09:53,  2.12it/s, est. speed input: 69199.91 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1797/3051 [14:10<09:52,  2.12it/s, est. speed input: 69199.99 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1798/3051 [14:11<09:52,  2.12it/s, est. speed input: 69200.08 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1799/3051 [14:11<09:51,  2.12it/s, est. speed input: 69200.22 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1800/3051 [14:12<09:53,  2.11it/s, est. speed input: 69199.83 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1801/3051 [14:12<09:52,  2.11it/s, est. speed input: 69199.89 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1802/3051 [14:13<09:50,  2.11it/s, est. speed input: 69200.03 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1803/3051 [14:13<09:50,  2.11it/s, est. speed input: 69200.03 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1804/3051 [14:14<09:50,  2.11it/s, est. speed input: 69200.08 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1805/3051 [14:14<09:49,  2.11it/s, est. speed input: 69200.16 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1806/3051 [14:15<09:48,  2.11it/s, est. speed input: 69200.22 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1807/3051 [14:15<09:47,  2.12it/s, est. speed input: 69200.35 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1808/3051 [14:16<09:48,  2.11it/s, est. speed input: 69200.27 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1809/3051 [14:16<09:47,  2.11it/s, est. speed input: 69200.38 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1810/3051 [14:17<09:46,  2.11it/s, est. speed input: 69200.43 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1811/3051 [14:17<09:46,  2.11it/s, est. speed input: 69200.42 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1812/3051 [14:18<09:45,  2.12it/s, est. speed input: 69200.54 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1813/3051 [14:18<09:45,  2.12it/s, est. speed input: 69200.63 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1814/3051 [14:18<09:44,  2.12it/s, est. speed input: 69200.71 toks/s, output: 2.11 toks/s]
Processed prompts:  59%|█████▉    | 1815/3051 [14:19<09:43,  2.12it/s, est. speed input: 69200.82 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1816/3051 [14:19<09:43,  2.12it/s, est. speed input: 69200.92 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1817/3051 [14:20<09:43,  2.12it/s, est. speed input: 69200.93 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1818/3051 [14:20<09:44,  2.11it/s, est. speed input: 69200.67 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1819/3051 [14:21<09:43,  2.11it/s, est. speed input: 69200.78 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1820/3051 [14:21<09:42,  2.11it/s, est. speed input: 69200.82 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1821/3051 [14:22<09:42,  2.11it/s, est. speed input: 69200.82 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1822/3051 [14:22<09:41,  2.11it/s, est. speed input: 69200.88 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1823/3051 [14:23<09:40,  2.12it/s, est. speed input: 69201.02 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1824/3051 [14:23<09:39,  2.12it/s, est. speed input: 69201.10 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1825/3051 [14:24<09:39,  2.12it/s, est. speed input: 69201.18 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1826/3051 [14:24<09:39,  2.11it/s, est. speed input: 69201.15 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1827/3051 [14:25<09:38,  2.11it/s, est. speed input: 69201.20 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1828/3051 [14:25<09:38,  2.11it/s, est. speed input: 69201.26 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1829/3051 [14:26<09:37,  2.12it/s, est. speed input: 69201.35 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|█████▉    | 1830/3051 [14:26<09:36,  2.12it/s, est. speed input: 69201.47 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1831/3051 [14:27<09:36,  2.12it/s, est. speed input: 69201.53 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1832/3051 [14:27<09:36,  2.12it/s, est. speed input: 69201.62 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1833/3051 [14:27<09:35,  2.12it/s, est. speed input: 69201.63 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1834/3051 [14:28<09:35,  2.12it/s, est. speed input: 69201.74 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1835/3051 [14:28<09:34,  2.12it/s, est. speed input: 69201.86 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1836/3051 [14:29<09:34,  2.12it/s, est. speed input: 69201.92 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1837/3051 [14:29<09:33,  2.12it/s, est. speed input: 69202.01 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1838/3051 [14:30<09:33,  2.12it/s, est. speed input: 69202.08 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1839/3051 [14:30<09:32,  2.12it/s, est. speed input: 69202.13 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1840/3051 [14:31<09:33,  2.11it/s, est. speed input: 69201.88 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1841/3051 [14:31<09:33,  2.11it/s, est. speed input: 69201.92 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1842/3051 [14:32<09:32,  2.11it/s, est. speed input: 69201.99 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1843/3051 [14:32<09:31,  2.11it/s, est. speed input: 69202.07 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1844/3051 [14:33<09:30,  2.12it/s, est. speed input: 69202.19 toks/s, output: 2.11 toks/s]
Processed prompts:  60%|██████    | 1845/3051 [14:33<09:30,  2.12it/s, est. speed input: 69202.27 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1846/3051 [14:34<09:29,  2.11it/s, est. speed input: 69202.28 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1847/3051 [14:34<09:29,  2.12it/s, est. speed input: 69202.35 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1848/3051 [14:35<09:28,  2.12it/s, est. speed input: 69202.41 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1849/3051 [14:35<09:28,  2.12it/s, est. speed input: 69202.49 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1850/3051 [14:35<09:27,  2.12it/s, est. speed input: 69202.59 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1851/3051 [14:36<09:26,  2.12it/s, est. speed input: 69202.72 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1852/3051 [14:36<09:28,  2.11it/s, est. speed input: 69202.35 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1853/3051 [14:37<09:27,  2.11it/s, est. speed input: 69202.50 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1854/3051 [14:37<09:25,  2.11it/s, est. speed input: 69202.65 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1855/3051 [14:38<09:25,  2.11it/s, est. speed input: 69202.62 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1856/3051 [14:38<09:25,  2.11it/s, est. speed input: 69202.60 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1857/3051 [14:39<09:24,  2.11it/s, est. speed input: 69202.70 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1858/3051 [14:39<09:24,  2.11it/s, est. speed input: 69202.70 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1859/3051 [14:40<09:23,  2.11it/s, est. speed input: 69202.74 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1860/3051 [14:40<09:23,  2.11it/s, est. speed input: 69202.79 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1861/3051 [14:41<09:22,  2.12it/s, est. speed input: 69202.89 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1862/3051 [14:41<09:21,  2.12it/s, est. speed input: 69203.02 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1863/3051 [14:42<09:21,  2.12it/s, est. speed input: 69203.14 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1864/3051 [14:42<09:20,  2.12it/s, est. speed input: 69203.24 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1865/3051 [14:43<09:20,  2.12it/s, est. speed input: 69203.22 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1866/3051 [14:43<09:20,  2.11it/s, est. speed input: 69203.23 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1867/3051 [14:44<09:19,  2.12it/s, est. speed input: 69203.31 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████    | 1868/3051 [14:44<09:19,  2.11it/s, est. speed input: 69203.28 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1869/3051 [14:44<09:19,  2.11it/s, est. speed input: 69203.34 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1870/3051 [14:45<09:18,  2.11it/s, est. speed input: 69203.40 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1871/3051 [14:45<09:17,  2.12it/s, est. speed input: 69203.51 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1872/3051 [14:46<09:17,  2.11it/s, est. speed input: 69203.53 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1873/3051 [14:46<09:17,  2.11it/s, est. speed input: 69203.51 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1874/3051 [14:47<09:16,  2.12it/s, est. speed input: 69203.66 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1875/3051 [14:47<09:15,  2.12it/s, est. speed input: 69203.72 toks/s, output: 2.11 toks/s]
Processed prompts:  61%|██████▏   | 1876/3051 [14:48<09:15,  2.12it/s, est. speed input: 69203.80 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1877/3051 [14:48<09:14,  2.12it/s, est. speed input: 69203.90 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1878/3051 [14:49<09:14,  2.12it/s, est. speed input: 69203.93 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1879/3051 [14:49<09:13,  2.12it/s, est. speed input: 69204.02 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1880/3051 [14:50<09:13,  2.12it/s, est. speed input: 69204.10 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1881/3051 [14:50<09:13,  2.11it/s, est. speed input: 69203.95 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1882/3051 [14:51<09:12,  2.11it/s, est. speed input: 69204.06 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1883/3051 [14:51<09:12,  2.12it/s, est. speed input: 69204.16 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1884/3051 [14:52<09:11,  2.12it/s, est. speed input: 69204.24 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1885/3051 [14:52<09:11,  2.12it/s, est. speed input: 69204.31 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1886/3051 [14:53<09:10,  2.12it/s, est. speed input: 69204.40 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1887/3051 [14:53<09:10,  2.12it/s, est. speed input: 69204.44 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1888/3051 [14:53<09:09,  2.12it/s, est. speed input: 69204.50 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1889/3051 [14:54<09:09,  2.12it/s, est. speed input: 69204.58 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1890/3051 [14:54<09:08,  2.12it/s, est. speed input: 69204.69 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1891/3051 [14:55<09:08,  2.12it/s, est. speed input: 69204.71 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1892/3051 [14:55<09:08,  2.11it/s, est. speed input: 69204.64 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1893/3051 [14:56<09:07,  2.11it/s, est. speed input: 69204.69 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1894/3051 [14:56<09:06,  2.12it/s, est. speed input: 69204.81 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1895/3051 [14:57<09:06,  2.11it/s, est. speed input: 69204.80 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1896/3051 [14:57<09:06,  2.11it/s, est. speed input: 69204.84 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1897/3051 [14:58<09:05,  2.12it/s, est. speed input: 69204.94 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1898/3051 [14:58<09:05,  2.11it/s, est. speed input: 69204.97 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1899/3051 [14:59<09:04,  2.11it/s, est. speed input: 69205.02 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1900/3051 [14:59<09:04,  2.11it/s, est. speed input: 69205.05 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1901/3051 [15:00<09:03,  2.12it/s, est. speed input: 69205.13 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1902/3051 [15:00<09:04,  2.11it/s, est. speed input: 69204.88 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1903/3051 [15:01<09:03,  2.11it/s, est. speed input: 69205.03 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1904/3051 [15:01<09:02,  2.11it/s, est. speed input: 69205.11 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1905/3051 [15:01<09:01,  2.11it/s, est. speed input: 69205.17 toks/s, output: 2.11 toks/s]
Processed prompts:  62%|██████▏   | 1906/3051 [15:02<09:01,  2.12it/s, est. speed input: 69205.24 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1907/3051 [15:02<09:00,  2.11it/s, est. speed input: 69205.29 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1908/3051 [15:03<09:00,  2.11it/s, est. speed input: 69205.32 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1909/3051 [15:03<08:59,  2.12it/s, est. speed input: 69205.42 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1910/3051 [15:04<08:59,  2.12it/s, est. speed input: 69205.47 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1911/3051 [15:04<08:58,  2.12it/s, est. speed input: 69205.53 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1912/3051 [15:05<08:58,  2.11it/s, est. speed input: 69205.52 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1913/3051 [15:05<08:57,  2.12it/s, est. speed input: 69205.64 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1914/3051 [15:06<08:57,  2.12it/s, est. speed input: 69205.69 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1915/3051 [15:06<08:58,  2.11it/s, est. speed input: 69205.37 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1916/3051 [15:07<08:57,  2.11it/s, est. speed input: 69205.41 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1917/3051 [15:07<08:56,  2.11it/s, est. speed input: 69205.48 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1918/3051 [15:08<08:56,  2.11it/s, est. speed input: 69205.57 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1919/3051 [15:08<08:55,  2.12it/s, est. speed input: 69205.70 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1920/3051 [15:09<08:54,  2.11it/s, est. speed input: 69205.68 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1921/3051 [15:09<08:54,  2.11it/s, est. speed input: 69205.70 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1922/3051 [15:10<08:54,  2.11it/s, est. speed input: 69205.75 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1923/3051 [15:10<08:53,  2.12it/s, est. speed input: 69205.86 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1924/3051 [15:10<08:52,  2.12it/s, est. speed input: 69205.94 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1925/3051 [15:11<08:52,  2.12it/s, est. speed input: 69206.01 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1926/3051 [15:11<08:51,  2.12it/s, est. speed input: 69206.05 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1927/3051 [15:12<08:51,  2.12it/s, est. speed input: 69206.10 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1928/3051 [15:12<08:51,  2.11it/s, est. speed input: 69205.95 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1929/3051 [15:13<08:50,  2.11it/s, est. speed input: 69206.07 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1930/3051 [15:13<08:50,  2.12it/s, est. speed input: 69206.17 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1931/3051 [15:14<08:49,  2.12it/s, est. speed input: 69206.28 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1932/3051 [15:14<08:48,  2.12it/s, est. speed input: 69206.36 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1933/3051 [15:15<08:48,  2.12it/s, est. speed input: 69206.44 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1934/3051 [15:15<08:47,  2.12it/s, est. speed input: 69206.60 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1935/3051 [15:16<08:46,  2.12it/s, est. speed input: 69206.72 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1936/3051 [15:16<08:46,  2.12it/s, est. speed input: 69206.77 toks/s, output: 2.11 toks/s]
Processed prompts:  63%|██████▎   | 1937/3051 [15:17<08:46,  2.11it/s, est. speed input: 69206.70 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1938/3051 [15:17<08:46,  2.12it/s, est. speed input: 69206.77 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1939/3051 [15:18<08:45,  2.12it/s, est. speed input: 69206.87 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1940/3051 [15:18<08:44,  2.12it/s, est. speed input: 69206.97 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1941/3051 [15:19<08:44,  2.11it/s, est. speed input: 69206.93 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1942/3051 [15:19<08:44,  2.11it/s, est. speed input: 69207.00 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1943/3051 [15:19<08:43,  2.12it/s, est. speed input: 69207.06 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1944/3051 [15:20<08:43,  2.12it/s, est. speed input: 69207.17 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▎   | 1945/3051 [15:20<08:42,  2.12it/s, est. speed input: 69207.23 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1946/3051 [15:21<08:41,  2.12it/s, est. speed input: 69207.36 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1947/3051 [15:21<08:41,  2.12it/s, est. speed input: 69207.40 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1948/3051 [15:22<08:41,  2.12it/s, est. speed input: 69207.49 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1949/3051 [15:22<08:40,  2.12it/s, est. speed input: 69207.52 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1950/3051 [15:23<08:40,  2.12it/s, est. speed input: 69207.57 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1951/3051 [15:23<08:39,  2.12it/s, est. speed input: 69207.67 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1952/3051 [15:24<08:39,  2.12it/s, est. speed input: 69207.77 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1953/3051 [15:24<08:38,  2.12it/s, est. speed input: 69207.87 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1954/3051 [15:25<08:38,  2.12it/s, est. speed input: 69207.99 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1955/3051 [15:25<08:37,  2.12it/s, est. speed input: 69208.10 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1956/3051 [15:26<08:37,  2.11it/s, est. speed input: 69208.00 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1957/3051 [15:26<08:37,  2.11it/s, est. speed input: 69208.04 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1958/3051 [15:27<08:36,  2.12it/s, est. speed input: 69208.18 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1959/3051 [15:27<08:37,  2.11it/s, est. speed input: 69207.86 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1960/3051 [15:28<08:36,  2.11it/s, est. speed input: 69207.88 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1961/3051 [15:28<08:35,  2.11it/s, est. speed input: 69207.99 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1962/3051 [15:28<08:35,  2.11it/s, est. speed input: 69208.08 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1963/3051 [15:29<08:34,  2.11it/s, est. speed input: 69208.10 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1964/3051 [15:29<08:34,  2.11it/s, est. speed input: 69208.17 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1965/3051 [15:30<08:33,  2.12it/s, est. speed input: 69208.24 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1966/3051 [15:30<08:32,  2.12it/s, est. speed input: 69208.35 toks/s, output: 2.11 toks/s]
Processed prompts:  64%|██████▍   | 1967/3051 [15:31<08:31,  2.12it/s, est. speed input: 69208.49 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1968/3051 [15:31<08:31,  2.12it/s, est. speed input: 69208.58 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1969/3051 [15:32<08:31,  2.12it/s, est. speed input: 69208.59 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1970/3051 [15:32<08:30,  2.12it/s, est. speed input: 69208.65 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1971/3051 [15:33<08:30,  2.12it/s, est. speed input: 69208.72 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1972/3051 [15:33<08:29,  2.12it/s, est. speed input: 69208.80 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1973/3051 [15:34<08:29,  2.12it/s, est. speed input: 69208.88 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1974/3051 [15:34<08:28,  2.12it/s, est. speed input: 69208.98 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1975/3051 [15:35<08:28,  2.12it/s, est. speed input: 69209.05 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1976/3051 [15:35<08:27,  2.12it/s, est. speed input: 69209.11 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1977/3051 [15:36<08:27,  2.12it/s, est. speed input: 69209.21 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1978/3051 [15:36<08:26,  2.12it/s, est. speed input: 69209.32 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1979/3051 [15:36<08:26,  2.12it/s, est. speed input: 69209.41 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1980/3051 [15:37<08:25,  2.12it/s, est. speed input: 69209.46 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1981/3051 [15:37<08:25,  2.12it/s, est. speed input: 69209.53 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1982/3051 [15:38<08:25,  2.12it/s, est. speed input: 69209.60 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▍   | 1983/3051 [15:38<08:25,  2.11it/s, est. speed input: 69209.40 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1984/3051 [15:39<08:24,  2.11it/s, est. speed input: 69209.47 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1985/3051 [15:39<08:23,  2.12it/s, est. speed input: 69209.61 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1986/3051 [15:40<08:23,  2.12it/s, est. speed input: 69209.66 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1987/3051 [15:40<08:22,  2.12it/s, est. speed input: 69209.74 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1988/3051 [15:41<08:23,  2.11it/s, est. speed input: 69209.65 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1989/3051 [15:41<08:22,  2.12it/s, est. speed input: 69209.78 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1990/3051 [15:42<08:21,  2.12it/s, est. speed input: 69209.88 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1991/3051 [15:42<08:20,  2.12it/s, est. speed input: 69209.97 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1992/3051 [15:43<08:20,  2.12it/s, est. speed input: 69210.07 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1993/3051 [15:43<08:19,  2.12it/s, est. speed input: 69210.16 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1994/3051 [15:44<08:19,  2.12it/s, est. speed input: 69210.22 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1995/3051 [15:44<08:19,  2.12it/s, est. speed input: 69210.23 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1996/3051 [15:45<08:18,  2.12it/s, est. speed input: 69210.32 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1997/3051 [15:45<08:17,  2.12it/s, est. speed input: 69210.45 toks/s, output: 2.11 toks/s]
Processed prompts:  65%|██████▌   | 1998/3051 [15:45<08:17,  2.12it/s, est. speed input: 69210.52 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 1999/3051 [15:46<08:16,  2.12it/s, est. speed input: 69210.60 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2000/3051 [15:46<08:16,  2.12it/s, est. speed input: 69210.67 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2001/3051 [15:47<08:16,  2.12it/s, est. speed input: 69210.71 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2002/3051 [15:47<08:16,  2.11it/s, est. speed input: 69210.50 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2003/3051 [15:48<08:16,  2.11it/s, est. speed input: 69210.53 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2004/3051 [15:48<08:15,  2.11it/s, est. speed input: 69210.63 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2005/3051 [15:49<08:15,  2.11it/s, est. speed input: 69210.57 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2006/3051 [15:49<08:14,  2.11it/s, est. speed input: 69210.66 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2007/3051 [15:50<08:13,  2.11it/s, est. speed input: 69210.73 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2008/3051 [15:50<08:13,  2.11it/s, est. speed input: 69210.76 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2009/3051 [15:51<08:12,  2.11it/s, est. speed input: 69210.78 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2010/3051 [15:51<08:12,  2.11it/s, est. speed input: 69210.85 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2011/3051 [15:52<08:11,  2.11it/s, est. speed input: 69210.87 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2012/3051 [15:52<08:11,  2.12it/s, est. speed input: 69210.96 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2013/3051 [15:53<08:10,  2.12it/s, est. speed input: 69211.05 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2014/3051 [15:53<08:09,  2.12it/s, est. speed input: 69211.13 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2015/3051 [15:54<08:09,  2.12it/s, est. speed input: 69211.21 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2016/3051 [15:54<08:09,  2.12it/s, est. speed input: 69211.25 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2017/3051 [15:54<08:10,  2.11it/s, est. speed input: 69210.83 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2018/3051 [15:55<08:09,  2.11it/s, est. speed input: 69210.90 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2019/3051 [15:55<08:08,  2.11it/s, est. speed input: 69210.96 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2020/3051 [15:56<08:07,  2.11it/s, est. speed input: 69211.02 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▌   | 2021/3051 [15:56<08:07,  2.11it/s, est. speed input: 69211.13 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2022/3051 [15:57<08:06,  2.12it/s, est. speed input: 69211.23 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2023/3051 [15:57<08:06,  2.12it/s, est. speed input: 69211.26 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2024/3051 [15:58<08:05,  2.12it/s, est. speed input: 69211.33 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2025/3051 [15:58<08:05,  2.11it/s, est. speed input: 69211.30 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2026/3051 [15:59<08:04,  2.11it/s, est. speed input: 69211.35 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2027/3051 [15:59<08:04,  2.11it/s, est. speed input: 69211.39 toks/s, output: 2.11 toks/s]
Processed prompts:  66%|██████▋   | 2028/3051 [16:00<08:03,  2.12it/s, est. speed input: 69211.49 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2029/3051 [16:00<08:02,  2.12it/s, est. speed input: 69211.60 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2030/3051 [16:01<08:04,  2.11it/s, est. speed input: 69211.18 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2031/3051 [16:01<08:03,  2.11it/s, est. speed input: 69211.28 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2032/3051 [16:02<08:02,  2.11it/s, est. speed input: 69211.37 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2033/3051 [16:02<08:01,  2.11it/s, est. speed input: 69211.48 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2034/3051 [16:02<08:00,  2.12it/s, est. speed input: 69211.59 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2035/3051 [16:03<08:00,  2.12it/s, est. speed input: 69211.65 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2036/3051 [16:03<07:59,  2.12it/s, est. speed input: 69211.75 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2037/3051 [16:04<07:59,  2.12it/s, est. speed input: 69211.81 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2038/3051 [16:04<07:59,  2.11it/s, est. speed input: 69211.64 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2039/3051 [16:05<07:58,  2.11it/s, est. speed input: 69211.73 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2040/3051 [16:05<07:58,  2.11it/s, est. speed input: 69211.80 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2041/3051 [16:06<07:57,  2.12it/s, est. speed input: 69211.88 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2042/3051 [16:06<07:56,  2.12it/s, est. speed input: 69211.98 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2043/3051 [16:07<07:57,  2.11it/s, est. speed input: 69211.82 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2044/3051 [16:07<07:56,  2.11it/s, est. speed input: 69211.90 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2045/3051 [16:08<07:55,  2.12it/s, est. speed input: 69212.03 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2046/3051 [16:08<07:55,  2.12it/s, est. speed input: 69212.09 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2047/3051 [16:09<07:54,  2.12it/s, est. speed input: 69212.21 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2048/3051 [16:09<07:53,  2.12it/s, est. speed input: 69212.37 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2049/3051 [16:10<07:53,  2.12it/s, est. speed input: 69212.46 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2050/3051 [16:10<07:53,  2.12it/s, est. speed input: 69212.40 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2051/3051 [16:11<07:52,  2.12it/s, est. speed input: 69212.52 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2052/3051 [16:11<07:52,  2.11it/s, est. speed input: 69212.48 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2053/3051 [16:11<07:52,  2.11it/s, est. speed input: 69212.47 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2054/3051 [16:12<07:51,  2.11it/s, est. speed input: 69212.51 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2055/3051 [16:12<07:50,  2.12it/s, est. speed input: 69212.62 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2056/3051 [16:13<07:50,  2.12it/s, est. speed input: 69212.73 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2057/3051 [16:13<07:50,  2.11it/s, est. speed input: 69212.50 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2058/3051 [16:14<07:49,  2.11it/s, est. speed input: 69212.61 toks/s, output: 2.11 toks/s]
Processed prompts:  67%|██████▋   | 2059/3051 [16:14<07:49,  2.11it/s, est. speed input: 69212.70 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2060/3051 [16:15<07:48,  2.12it/s, est. speed input: 69212.78 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2061/3051 [16:15<07:47,  2.12it/s, est. speed input: 69212.84 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2062/3051 [16:16<07:47,  2.12it/s, est. speed input: 69212.90 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2063/3051 [16:16<07:47,  2.11it/s, est. speed input: 69212.75 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2064/3051 [16:17<07:47,  2.11it/s, est. speed input: 69212.82 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2065/3051 [16:17<07:46,  2.11it/s, est. speed input: 69212.86 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2066/3051 [16:18<07:45,  2.11it/s, est. speed input: 69212.95 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2067/3051 [16:18<07:45,  2.12it/s, est. speed input: 69213.03 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2068/3051 [16:19<07:44,  2.12it/s, est. speed input: 69213.09 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2069/3051 [16:19<07:43,  2.12it/s, est. speed input: 69213.18 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2070/3051 [16:20<07:44,  2.11it/s, est. speed input: 69213.10 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2071/3051 [16:20<07:43,  2.11it/s, est. speed input: 69213.17 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2072/3051 [16:20<07:42,  2.12it/s, est. speed input: 69213.28 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2073/3051 [16:21<07:42,  2.12it/s, est. speed input: 69213.35 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2074/3051 [16:21<07:41,  2.12it/s, est. speed input: 69213.42 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2075/3051 [16:22<07:41,  2.12it/s, est. speed input: 69213.48 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2076/3051 [16:22<07:41,  2.11it/s, est. speed input: 69213.38 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2077/3051 [16:23<07:40,  2.11it/s, est. speed input: 69213.39 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2078/3051 [16:23<07:40,  2.11it/s, est. speed input: 69213.44 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2079/3051 [16:24<07:39,  2.11it/s, est. speed input: 69213.49 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2080/3051 [16:24<07:39,  2.11it/s, est. speed input: 69213.42 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2081/3051 [16:25<07:38,  2.11it/s, est. speed input: 69213.50 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2082/3051 [16:25<07:38,  2.12it/s, est. speed input: 69213.59 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2083/3051 [16:26<07:37,  2.11it/s, est. speed input: 69213.58 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2084/3051 [16:26<07:37,  2.12it/s, est. speed input: 69213.70 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2085/3051 [16:27<07:37,  2.11it/s, est. speed input: 69213.64 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2086/3051 [16:27<07:36,  2.11it/s, est. speed input: 69213.73 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2087/3051 [16:28<07:35,  2.12it/s, est. speed input: 69213.80 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2088/3051 [16:28<07:35,  2.12it/s, est. speed input: 69213.87 toks/s, output: 2.11 toks/s]
Processed prompts:  68%|██████▊   | 2089/3051 [16:28<07:34,  2.12it/s, est. speed input: 69213.96 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2090/3051 [16:29<07:33,  2.12it/s, est. speed input: 69214.06 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2091/3051 [16:29<07:33,  2.12it/s, est. speed input: 69214.09 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2092/3051 [16:30<07:34,  2.11it/s, est. speed input: 69213.84 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2093/3051 [16:33<20:35,  1.29s/it, est. speed input: 69024.39 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2094/3051 [16:34<16:38,  1.04s/it, est. speed input: 69024.76 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2095/3051 [16:34<13:53,  1.15it/s, est. speed input: 69025.08 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2096/3051 [16:35<11:57,  1.33it/s, est. speed input: 69025.42 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▊   | 2097/3051 [16:35<10:36,  1.50it/s, est. speed input: 69025.70 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2098/3051 [16:35<09:39,  1.65it/s, est. speed input: 69026.03 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2099/3051 [16:36<08:59,  1.77it/s, est. speed input: 69026.38 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2100/3051 [16:36<08:31,  1.86it/s, est. speed input: 69026.72 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2101/3051 [16:37<08:11,  1.93it/s, est. speed input: 69027.01 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2102/3051 [16:37<07:57,  1.99it/s, est. speed input: 69027.31 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2103/3051 [16:38<07:47,  2.03it/s, est. speed input: 69027.60 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2104/3051 [16:38<07:40,  2.06it/s, est. speed input: 69027.92 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2105/3051 [16:39<07:35,  2.08it/s, est. speed input: 69028.23 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2106/3051 [16:39<07:31,  2.09it/s, est. speed input: 69028.55 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2107/3051 [16:40<07:29,  2.10it/s, est. speed input: 69028.86 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2108/3051 [16:40<07:27,  2.11it/s, est. speed input: 69029.16 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2109/3051 [16:41<07:25,  2.11it/s, est. speed input: 69029.41 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2110/3051 [16:41<07:24,  2.12it/s, est. speed input: 69029.73 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2111/3051 [16:42<07:23,  2.12it/s, est. speed input: 69030.02 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2112/3051 [16:42<07:22,  2.12it/s, est. speed input: 69030.31 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2113/3051 [16:43<07:21,  2.12it/s, est. speed input: 69030.61 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2114/3051 [16:43<07:21,  2.12it/s, est. speed input: 69030.93 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2115/3051 [16:43<07:20,  2.13it/s, est. speed input: 69031.27 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2116/3051 [16:44<07:19,  2.13it/s, est. speed input: 69031.54 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2117/3051 [16:44<07:19,  2.13it/s, est. speed input: 69031.83 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2118/3051 [16:45<07:19,  2.13it/s, est. speed input: 69032.11 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2119/3051 [16:45<07:18,  2.13it/s, est. speed input: 69032.40 toks/s, output: 2.11 toks/s]
Processed prompts:  69%|██████▉   | 2120/3051 [16:46<07:17,  2.13it/s, est. speed input: 69032.71 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2121/3051 [16:46<07:17,  2.13it/s, est. speed input: 69033.01 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2122/3051 [16:47<07:17,  2.12it/s, est. speed input: 69033.25 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2123/3051 [16:47<07:16,  2.13it/s, est. speed input: 69033.54 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2124/3051 [16:48<07:16,  2.12it/s, est. speed input: 69033.78 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2125/3051 [16:48<07:15,  2.12it/s, est. speed input: 69034.06 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2126/3051 [16:49<07:15,  2.12it/s, est. speed input: 69034.33 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2127/3051 [16:49<07:14,  2.12it/s, est. speed input: 69034.60 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2128/3051 [16:50<07:14,  2.13it/s, est. speed input: 69034.93 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2129/3051 [16:50<07:13,  2.13it/s, est. speed input: 69035.19 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2130/3051 [16:51<07:13,  2.12it/s, est. speed input: 69035.46 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2131/3051 [16:51<07:13,  2.12it/s, est. speed input: 69035.70 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2132/3051 [16:51<07:12,  2.12it/s, est. speed input: 69036.01 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2133/3051 [16:52<07:12,  2.12it/s, est. speed input: 69036.18 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2134/3051 [16:52<07:12,  2.12it/s, est. speed input: 69036.34 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|██████▉   | 2135/3051 [16:53<07:12,  2.12it/s, est. speed input: 69036.46 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2136/3051 [16:53<07:11,  2.12it/s, est. speed input: 69036.61 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2137/3051 [16:54<07:11,  2.12it/s, est. speed input: 69036.75 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2138/3051 [16:54<07:11,  2.12it/s, est. speed input: 69036.94 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2139/3051 [16:55<07:10,  2.12it/s, est. speed input: 69037.11 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2140/3051 [16:55<07:10,  2.12it/s, est. speed input: 69037.30 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2141/3051 [16:56<07:09,  2.12it/s, est. speed input: 69037.46 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2142/3051 [16:56<07:09,  2.12it/s, est. speed input: 69037.62 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2143/3051 [16:57<07:08,  2.12it/s, est. speed input: 69037.84 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2144/3051 [16:57<07:08,  2.12it/s, est. speed input: 69037.98 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2145/3051 [16:58<07:07,  2.12it/s, est. speed input: 69038.11 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2146/3051 [16:58<07:07,  2.12it/s, est. speed input: 69038.28 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2147/3051 [16:59<07:06,  2.12it/s, est. speed input: 69038.46 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2148/3051 [16:59<07:07,  2.11it/s, est. speed input: 69038.40 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2149/3051 [16:59<07:06,  2.11it/s, est. speed input: 69038.53 toks/s, output: 2.11 toks/s]
Processed prompts:  70%|███████   | 2150/3051 [17:00<07:05,  2.12it/s, est. speed input: 69038.70 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2151/3051 [17:00<07:05,  2.12it/s, est. speed input: 69038.91 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2152/3051 [17:01<07:04,  2.12it/s, est. speed input: 69039.03 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2153/3051 [17:01<07:04,  2.12it/s, est. speed input: 69039.17 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2154/3051 [17:02<07:03,  2.12it/s, est. speed input: 69039.32 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2155/3051 [17:02<07:04,  2.11it/s, est. speed input: 69039.13 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2156/3051 [17:03<07:03,  2.11it/s, est. speed input: 69039.29 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2157/3051 [17:03<07:03,  2.11it/s, est. speed input: 69039.43 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2158/3051 [17:04<07:02,  2.11it/s, est. speed input: 69039.57 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2159/3051 [17:04<07:01,  2.11it/s, est. speed input: 69039.72 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2160/3051 [17:05<07:01,  2.12it/s, est. speed input: 69039.87 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2161/3051 [17:05<07:00,  2.12it/s, est. speed input: 69040.05 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2162/3051 [17:06<06:59,  2.12it/s, est. speed input: 69040.23 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2163/3051 [17:06<06:59,  2.12it/s, est. speed input: 69040.36 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2164/3051 [17:07<06:58,  2.12it/s, est. speed input: 69040.52 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2165/3051 [17:07<06:58,  2.12it/s, est. speed input: 69040.62 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2166/3051 [17:08<06:58,  2.12it/s, est. speed input: 69040.79 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2167/3051 [17:08<06:58,  2.11it/s, est. speed input: 69040.80 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2168/3051 [17:08<06:57,  2.11it/s, est. speed input: 69040.90 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2169/3051 [17:09<06:56,  2.12it/s, est. speed input: 69041.07 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2170/3051 [17:09<06:56,  2.11it/s, est. speed input: 69041.15 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2171/3051 [17:10<06:56,  2.12it/s, est. speed input: 69041.30 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2172/3051 [17:10<06:55,  2.12it/s, est. speed input: 69041.43 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████   | 2173/3051 [17:11<06:55,  2.12it/s, est. speed input: 69041.54 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2174/3051 [17:11<06:54,  2.11it/s, est. speed input: 69041.66 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2175/3051 [17:12<06:54,  2.11it/s, est. speed input: 69041.68 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2176/3051 [17:12<06:53,  2.11it/s, est. speed input: 69041.86 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2177/3051 [17:13<06:54,  2.11it/s, est. speed input: 69041.69 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2178/3051 [17:13<06:53,  2.11it/s, est. speed input: 69041.84 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2179/3051 [17:14<06:52,  2.11it/s, est. speed input: 69041.98 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2180/3051 [17:14<06:51,  2.11it/s, est. speed input: 69042.16 toks/s, output: 2.11 toks/s]
Processed prompts:  71%|███████▏  | 2181/3051 [17:15<06:51,  2.12it/s, est. speed input: 69042.30 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2182/3051 [17:15<06:50,  2.12it/s, est. speed input: 69042.51 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2183/3051 [17:16<06:50,  2.12it/s, est. speed input: 69042.62 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2184/3051 [17:16<06:49,  2.12it/s, est. speed input: 69042.77 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2185/3051 [17:17<06:49,  2.12it/s, est. speed input: 69042.89 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2186/3051 [17:17<06:51,  2.10it/s, est. speed input: 69042.36 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2187/3051 [17:17<06:50,  2.11it/s, est. speed input: 69042.49 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2188/3051 [17:18<06:49,  2.11it/s, est. speed input: 69042.63 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2189/3051 [17:18<06:48,  2.11it/s, est. speed input: 69042.77 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2190/3051 [17:19<06:47,  2.11it/s, est. speed input: 69042.85 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2191/3051 [17:19<06:47,  2.11it/s, est. speed input: 69042.95 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2192/3051 [17:20<06:46,  2.11it/s, est. speed input: 69043.14 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2193/3051 [17:20<06:46,  2.11it/s, est. speed input: 69043.08 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2194/3051 [17:21<06:45,  2.11it/s, est. speed input: 69043.18 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2195/3051 [17:21<06:44,  2.11it/s, est. speed input: 69043.34 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2196/3051 [17:22<06:44,  2.12it/s, est. speed input: 69043.52 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2197/3051 [17:22<06:43,  2.12it/s, est. speed input: 69043.68 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2198/3051 [17:23<06:42,  2.12it/s, est. speed input: 69043.84 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2199/3051 [17:23<06:42,  2.12it/s, est. speed input: 69043.98 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2200/3051 [17:24<06:42,  2.11it/s, est. speed input: 69044.04 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2201/3051 [17:24<06:42,  2.11it/s, est. speed input: 69044.12 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2202/3051 [17:25<06:41,  2.12it/s, est. speed input: 69044.27 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2203/3051 [17:25<06:40,  2.12it/s, est. speed input: 69044.39 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2204/3051 [17:26<06:40,  2.12it/s, est. speed input: 69044.51 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2205/3051 [17:26<06:39,  2.12it/s, est. speed input: 69044.67 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2206/3051 [17:26<06:39,  2.12it/s, est. speed input: 69044.82 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2207/3051 [17:27<06:38,  2.12it/s, est. speed input: 69044.99 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2208/3051 [17:27<06:38,  2.12it/s, est. speed input: 69045.12 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2209/3051 [17:28<06:37,  2.12it/s, est. speed input: 69045.24 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2210/3051 [17:28<06:37,  2.12it/s, est. speed input: 69045.32 toks/s, output: 2.11 toks/s]
Processed prompts:  72%|███████▏  | 2211/3051 [17:29<06:37,  2.12it/s, est. speed input: 69045.46 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2212/3051 [17:29<06:37,  2.11it/s, est. speed input: 69045.44 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2213/3051 [17:30<06:36,  2.11it/s, est. speed input: 69045.53 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2214/3051 [17:30<06:35,  2.11it/s, est. speed input: 69045.67 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2215/3051 [17:31<06:35,  2.12it/s, est. speed input: 69045.85 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2216/3051 [17:31<06:34,  2.12it/s, est. speed input: 69045.97 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2217/3051 [17:32<06:34,  2.11it/s, est. speed input: 69046.04 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2218/3051 [17:32<06:34,  2.11it/s, est. speed input: 69046.00 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2219/3051 [17:33<06:33,  2.11it/s, est. speed input: 69046.12 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2220/3051 [17:33<06:33,  2.11it/s, est. speed input: 69046.28 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2221/3051 [17:34<06:32,  2.11it/s, est. speed input: 69046.42 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2222/3051 [17:34<06:31,  2.12it/s, est. speed input: 69046.56 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2223/3051 [17:34<06:32,  2.11it/s, est. speed input: 69046.40 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2224/3051 [17:35<06:31,  2.11it/s, est. speed input: 69046.54 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2225/3051 [17:35<06:30,  2.11it/s, est. speed input: 69046.65 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2226/3051 [17:36<06:30,  2.11it/s, est. speed input: 69046.78 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2227/3051 [17:36<06:29,  2.11it/s, est. speed input: 69046.92 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2228/3051 [17:37<06:30,  2.11it/s, est. speed input: 69046.79 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2229/3051 [17:37<06:29,  2.11it/s, est. speed input: 69046.89 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2230/3051 [17:38<06:28,  2.11it/s, est. speed input: 69047.01 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2231/3051 [17:38<06:32,  2.09it/s, est. speed input: 69045.87 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2232/3051 [17:39<06:30,  2.10it/s, est. speed input: 69046.06 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2233/3051 [17:39<06:28,  2.10it/s, est. speed input: 69046.23 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2234/3051 [17:40<06:27,  2.11it/s, est. speed input: 69046.37 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2235/3051 [17:40<06:26,  2.11it/s, est. speed input: 69046.49 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2236/3051 [17:41<06:26,  2.11it/s, est. speed input: 69046.55 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2237/3051 [17:41<06:25,  2.11it/s, est. speed input: 69046.67 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2238/3051 [17:42<06:24,  2.11it/s, est. speed input: 69046.77 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2239/3051 [17:42<06:24,  2.11it/s, est. speed input: 69046.85 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2240/3051 [17:43<06:23,  2.11it/s, est. speed input: 69046.93 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2241/3051 [17:43<06:23,  2.11it/s, est. speed input: 69047.08 toks/s, output: 2.11 toks/s]
Processed prompts:  73%|███████▎  | 2242/3051 [17:43<06:22,  2.12it/s, est. speed input: 69047.23 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2243/3051 [17:44<06:21,  2.12it/s, est. speed input: 69047.37 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2244/3051 [17:44<06:21,  2.12it/s, est. speed input: 69047.52 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2245/3051 [17:45<06:21,  2.12it/s, est. speed input: 69047.60 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2246/3051 [17:45<06:20,  2.11it/s, est. speed input: 69047.66 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2247/3051 [17:46<06:20,  2.11it/s, est. speed input: 69047.77 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2248/3051 [17:46<06:20,  2.11it/s, est. speed input: 69047.70 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2249/3051 [17:47<06:19,  2.11it/s, est. speed input: 69047.84 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▎  | 2250/3051 [17:47<06:19,  2.11it/s, est. speed input: 69047.95 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2251/3051 [17:48<06:18,  2.11it/s, est. speed input: 69047.94 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2252/3051 [17:48<06:18,  2.11it/s, est. speed input: 69047.99 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2253/3051 [17:49<06:17,  2.11it/s, est. speed input: 69048.16 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2254/3051 [17:49<06:17,  2.11it/s, est. speed input: 69048.15 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2255/3051 [17:50<06:16,  2.11it/s, est. speed input: 69048.26 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2256/3051 [17:50<06:16,  2.11it/s, est. speed input: 69048.39 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2257/3051 [17:51<06:15,  2.11it/s, est. speed input: 69048.51 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2258/3051 [17:51<06:15,  2.11it/s, est. speed input: 69048.50 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2259/3051 [17:52<06:14,  2.11it/s, est. speed input: 69048.66 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2260/3051 [17:52<06:14,  2.11it/s, est. speed input: 69048.74 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2261/3051 [17:52<06:13,  2.11it/s, est. speed input: 69048.82 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2262/3051 [17:53<06:13,  2.11it/s, est. speed input: 69048.75 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2263/3051 [17:53<06:13,  2.11it/s, est. speed input: 69048.85 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2264/3051 [17:54<06:12,  2.11it/s, est. speed input: 69048.94 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2265/3051 [17:54<06:11,  2.11it/s, est. speed input: 69049.07 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2266/3051 [17:55<06:11,  2.11it/s, est. speed input: 69049.16 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2267/3051 [17:55<06:12,  2.11it/s, est. speed input: 69048.92 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2268/3051 [17:56<06:11,  2.11it/s, est. speed input: 69049.06 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2269/3051 [17:56<06:10,  2.11it/s, est. speed input: 69049.21 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2270/3051 [17:57<06:09,  2.11it/s, est. speed input: 69049.30 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2271/3051 [17:57<06:09,  2.11it/s, est. speed input: 69049.42 toks/s, output: 2.11 toks/s]
Processed prompts:  74%|███████▍  | 2272/3051 [17:58<06:08,  2.11it/s, est. speed input: 69049.55 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2273/3051 [17:58<06:07,  2.11it/s, est. speed input: 69049.68 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2274/3051 [17:59<06:07,  2.11it/s, est. speed input: 69049.76 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2275/3051 [17:59<06:06,  2.12it/s, est. speed input: 69049.93 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2276/3051 [18:00<06:06,  2.12it/s, est. speed input: 69050.03 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2277/3051 [18:00<06:06,  2.11it/s, est. speed input: 69049.86 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2278/3051 [18:01<06:06,  2.11it/s, est. speed input: 69049.99 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2279/3051 [18:01<06:05,  2.11it/s, est. speed input: 69050.12 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2280/3051 [18:01<06:04,  2.11it/s, est. speed input: 69050.21 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2281/3051 [18:02<06:04,  2.11it/s, est. speed input: 69050.27 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2282/3051 [18:02<06:03,  2.11it/s, est. speed input: 69050.42 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2283/3051 [18:03<06:03,  2.11it/s, est. speed input: 69050.45 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2284/3051 [18:03<06:02,  2.11it/s, est. speed input: 69050.59 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2285/3051 [18:04<06:02,  2.11it/s, est. speed input: 69050.73 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2286/3051 [18:04<06:01,  2.12it/s, est. speed input: 69050.84 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2287/3051 [18:05<06:01,  2.11it/s, est. speed input: 69050.95 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▍  | 2288/3051 [18:05<06:00,  2.12it/s, est. speed input: 69051.06 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2289/3051 [18:06<06:00,  2.11it/s, est. speed input: 69051.16 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2290/3051 [18:06<05:59,  2.11it/s, est. speed input: 69051.25 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2291/3051 [18:07<05:59,  2.11it/s, est. speed input: 69051.32 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2292/3051 [18:07<05:59,  2.11it/s, est. speed input: 69051.43 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2293/3051 [18:08<05:59,  2.11it/s, est. speed input: 69051.34 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2294/3051 [18:08<05:58,  2.11it/s, est. speed input: 69051.48 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2295/3051 [18:09<05:57,  2.11it/s, est. speed input: 69051.57 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2296/3051 [18:09<05:58,  2.11it/s, est. speed input: 69051.36 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2297/3051 [18:10<05:57,  2.11it/s, est. speed input: 69051.48 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2298/3051 [18:10<05:56,  2.11it/s, est. speed input: 69051.65 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2299/3051 [18:10<05:55,  2.11it/s, est. speed input: 69051.78 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2300/3051 [18:11<05:55,  2.11it/s, est. speed input: 69051.86 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2301/3051 [18:11<05:54,  2.11it/s, est. speed input: 69052.01 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2302/3051 [18:12<05:54,  2.11it/s, est. speed input: 69052.01 toks/s, output: 2.11 toks/s]
Processed prompts:  75%|███████▌  | 2303/3051 [18:12<05:54,  2.11it/s, est. speed input: 69052.09 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2304/3051 [18:13<05:53,  2.11it/s, est. speed input: 69052.23 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2305/3051 [18:13<05:52,  2.12it/s, est. speed input: 69052.37 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2306/3051 [18:14<05:52,  2.11it/s, est. speed input: 69052.37 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2307/3051 [18:14<05:52,  2.11it/s, est. speed input: 69052.43 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2308/3051 [18:15<05:51,  2.11it/s, est. speed input: 69052.46 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2309/3051 [18:15<05:51,  2.11it/s, est. speed input: 69052.52 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2310/3051 [18:16<05:50,  2.11it/s, est. speed input: 69052.63 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2311/3051 [18:16<05:50,  2.11it/s, est. speed input: 69052.74 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2312/3051 [18:17<05:49,  2.11it/s, est. speed input: 69052.77 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2313/3051 [18:17<05:49,  2.11it/s, est. speed input: 69052.83 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2314/3051 [18:18<05:48,  2.11it/s, est. speed input: 69052.95 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2315/3051 [18:18<05:48,  2.11it/s, est. speed input: 69052.93 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2316/3051 [18:19<05:47,  2.11it/s, est. speed input: 69053.09 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2317/3051 [18:19<05:47,  2.11it/s, est. speed input: 69053.00 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2318/3051 [18:19<05:47,  2.11it/s, est. speed input: 69053.08 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2319/3051 [18:20<05:46,  2.11it/s, est. speed input: 69053.23 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2320/3051 [18:20<05:45,  2.11it/s, est. speed input: 69053.41 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2321/3051 [18:21<05:45,  2.11it/s, est. speed input: 69053.50 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2322/3051 [18:21<05:44,  2.11it/s, est. speed input: 69053.61 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2323/3051 [18:22<05:44,  2.12it/s, est. speed input: 69053.75 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2324/3051 [18:22<05:43,  2.12it/s, est. speed input: 69053.86 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2325/3051 [18:23<05:43,  2.11it/s, est. speed input: 69053.82 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▌  | 2326/3051 [18:23<05:42,  2.11it/s, est. speed input: 69053.97 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2327/3051 [18:24<05:42,  2.11it/s, est. speed input: 69054.00 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2328/3051 [18:24<05:42,  2.11it/s, est. speed input: 69054.00 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2329/3051 [18:25<05:41,  2.11it/s, est. speed input: 69054.09 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2330/3051 [18:25<05:41,  2.11it/s, est. speed input: 69054.25 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2331/3051 [18:26<05:40,  2.11it/s, est. speed input: 69054.38 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2332/3051 [18:26<05:40,  2.11it/s, est. speed input: 69054.42 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2333/3051 [18:27<05:39,  2.11it/s, est. speed input: 69054.53 toks/s, output: 2.11 toks/s]
Processed prompts:  76%|███████▋  | 2334/3051 [18:27<05:39,  2.11it/s, est. speed input: 69054.63 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2335/3051 [18:28<05:38,  2.11it/s, est. speed input: 69054.65 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2336/3051 [18:28<05:38,  2.11it/s, est. speed input: 69054.80 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2337/3051 [18:28<05:37,  2.11it/s, est. speed input: 69054.89 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2338/3051 [18:29<05:38,  2.11it/s, est. speed input: 69054.69 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2339/3051 [18:29<05:37,  2.11it/s, est. speed input: 69054.83 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2340/3051 [18:30<05:36,  2.11it/s, est. speed input: 69054.96 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2341/3051 [18:30<05:35,  2.11it/s, est. speed input: 69055.07 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2342/3051 [18:31<05:35,  2.11it/s, est. speed input: 69055.19 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2343/3051 [18:31<05:34,  2.12it/s, est. speed input: 69055.33 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2344/3051 [18:32<05:34,  2.11it/s, est. speed input: 69055.43 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2345/3051 [18:32<05:33,  2.12it/s, est. speed input: 69055.55 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2346/3051 [18:33<05:33,  2.11it/s, est. speed input: 69055.64 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2347/3051 [18:33<05:32,  2.12it/s, est. speed input: 69055.78 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2348/3051 [18:34<05:32,  2.12it/s, est. speed input: 69055.91 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2349/3051 [18:34<05:31,  2.12it/s, est. speed input: 69056.05 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2350/3051 [18:35<05:31,  2.12it/s, est. speed input: 69056.17 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2351/3051 [18:35<05:30,  2.12it/s, est. speed input: 69056.31 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2352/3051 [18:36<05:30,  2.12it/s, est. speed input: 69056.37 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2353/3051 [18:36<05:29,  2.12it/s, est. speed input: 69056.51 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2354/3051 [18:36<05:29,  2.12it/s, est. speed input: 69056.64 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2355/3051 [18:37<05:29,  2.11it/s, est. speed input: 69056.61 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2356/3051 [18:37<05:28,  2.11it/s, est. speed input: 69056.74 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2357/3051 [18:38<05:28,  2.11it/s, est. speed input: 69056.84 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2358/3051 [18:38<05:27,  2.11it/s, est. speed input: 69056.92 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2359/3051 [18:39<05:27,  2.11it/s, est. speed input: 69057.04 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2360/3051 [18:39<05:27,  2.11it/s, est. speed input: 69057.02 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2361/3051 [18:40<05:26,  2.11it/s, est. speed input: 69057.03 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2362/3051 [18:40<05:26,  2.11it/s, est. speed input: 69057.15 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2363/3051 [18:41<05:25,  2.11it/s, est. speed input: 69057.25 toks/s, output: 2.11 toks/s]
Processed prompts:  77%|███████▋  | 2364/3051 [18:41<05:24,  2.11it/s, est. speed input: 69057.38 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2365/3051 [18:42<05:24,  2.11it/s, est. speed input: 69057.48 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2366/3051 [18:42<05:24,  2.11it/s, est. speed input: 69057.52 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2367/3051 [18:43<05:23,  2.11it/s, est. speed input: 69057.56 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2368/3051 [18:43<05:23,  2.11it/s, est. speed input: 69057.55 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2369/3051 [18:44<05:22,  2.11it/s, est. speed input: 69057.68 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2370/3051 [18:44<05:22,  2.11it/s, est. speed input: 69057.78 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2371/3051 [18:45<05:22,  2.11it/s, est. speed input: 69057.62 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2372/3051 [18:45<05:21,  2.11it/s, est. speed input: 69057.74 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2373/3051 [18:45<05:21,  2.11it/s, est. speed input: 69057.82 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2374/3051 [18:46<05:20,  2.11it/s, est. speed input: 69057.86 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2375/3051 [18:46<05:19,  2.11it/s, est. speed input: 69057.99 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2376/3051 [18:47<05:19,  2.11it/s, est. speed input: 69058.04 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2377/3051 [18:47<05:19,  2.11it/s, est. speed input: 69057.85 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2378/3051 [18:48<05:18,  2.11it/s, est. speed input: 69057.99 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2379/3051 [18:48<05:18,  2.11it/s, est. speed input: 69058.15 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2380/3051 [18:49<05:17,  2.11it/s, est. speed input: 69058.12 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2381/3051 [18:49<05:17,  2.11it/s, est. speed input: 69058.25 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2382/3051 [18:50<05:16,  2.11it/s, est. speed input: 69058.28 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2383/3051 [18:50<05:16,  2.11it/s, est. speed input: 69058.35 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2384/3051 [18:51<05:15,  2.11it/s, est. speed input: 69058.46 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2385/3051 [18:51<05:28,  2.03it/s, est. speed input: 69054.43 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2386/3051 [18:52<05:23,  2.05it/s, est. speed input: 69054.66 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2387/3051 [18:52<05:20,  2.07it/s, est. speed input: 69054.78 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2388/3051 [18:53<05:17,  2.09it/s, est. speed input: 69054.93 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2389/3051 [18:53<05:15,  2.10it/s, est. speed input: 69055.06 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2390/3051 [18:54<05:14,  2.10it/s, est. speed input: 69055.20 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2391/3051 [18:54<05:13,  2.11it/s, est. speed input: 69055.32 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2392/3051 [18:55<05:12,  2.11it/s, est. speed input: 69055.51 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2393/3051 [18:55<05:11,  2.11it/s, est. speed input: 69055.68 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2394/3051 [18:55<05:10,  2.12it/s, est. speed input: 69055.84 toks/s, output: 2.11 toks/s]
Processed prompts:  78%|███████▊  | 2395/3051 [18:56<05:10,  2.11it/s, est. speed input: 69055.93 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2396/3051 [18:56<05:09,  2.12it/s, est. speed input: 69056.06 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2397/3051 [18:57<05:09,  2.12it/s, est. speed input: 69056.21 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2398/3051 [18:57<05:08,  2.12it/s, est. speed input: 69056.35 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2399/3051 [18:58<05:07,  2.12it/s, est. speed input: 69056.49 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2400/3051 [18:58<05:07,  2.12it/s, est. speed input: 69056.62 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2401/3051 [18:59<05:07,  2.12it/s, est. speed input: 69056.72 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▊  | 2402/3051 [18:59<05:07,  2.11it/s, est. speed input: 69056.64 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2403/3051 [19:00<05:06,  2.11it/s, est. speed input: 69056.78 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2404/3051 [19:00<05:05,  2.12it/s, est. speed input: 69056.96 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2405/3051 [19:01<05:05,  2.12it/s, est. speed input: 69057.04 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2406/3051 [19:01<05:04,  2.12it/s, est. speed input: 69057.14 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2407/3051 [19:02<05:04,  2.12it/s, est. speed input: 69057.31 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2408/3051 [19:02<05:03,  2.12it/s, est. speed input: 69057.46 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2409/3051 [19:03<05:03,  2.12it/s, est. speed input: 69057.59 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2410/3051 [19:03<05:02,  2.12it/s, est. speed input: 69057.74 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2411/3051 [19:04<05:02,  2.12it/s, est. speed input: 69057.89 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2412/3051 [19:04<05:01,  2.12it/s, est. speed input: 69057.99 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2413/3051 [19:04<05:01,  2.12it/s, est. speed input: 69058.11 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2414/3051 [19:05<05:01,  2.11it/s, est. speed input: 69057.97 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2415/3051 [19:05<05:01,  2.11it/s, est. speed input: 69058.05 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2416/3051 [19:06<05:00,  2.11it/s, est. speed input: 69058.16 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2417/3051 [19:06<04:59,  2.11it/s, est. speed input: 69058.32 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2418/3051 [19:07<04:59,  2.12it/s, est. speed input: 69058.48 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2419/3051 [19:07<04:58,  2.12it/s, est. speed input: 69058.60 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2420/3051 [19:08<04:58,  2.12it/s, est. speed input: 69058.74 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2421/3051 [19:08<04:57,  2.12it/s, est. speed input: 69058.84 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2422/3051 [19:09<04:57,  2.12it/s, est. speed input: 69058.97 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2423/3051 [19:09<04:56,  2.12it/s, est. speed input: 69059.09 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2424/3051 [19:10<04:56,  2.11it/s, est. speed input: 69059.04 toks/s, output: 2.11 toks/s]
Processed prompts:  79%|███████▉  | 2425/3051 [19:10<04:56,  2.11it/s, est. speed input: 69059.14 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2426/3051 [19:11<04:55,  2.11it/s, est. speed input: 69059.28 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2427/3051 [19:11<04:55,  2.11it/s, est. speed input: 69059.31 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2428/3051 [19:12<04:54,  2.11it/s, est. speed input: 69059.38 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2429/3051 [19:12<04:54,  2.11it/s, est. speed input: 69059.50 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2430/3051 [19:13<04:53,  2.12it/s, est. speed input: 69059.64 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2431/3051 [19:13<04:52,  2.12it/s, est. speed input: 69059.81 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2432/3051 [19:13<04:52,  2.12it/s, est. speed input: 69059.94 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2433/3051 [19:14<04:51,  2.12it/s, est. speed input: 69060.08 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2434/3051 [19:14<04:51,  2.12it/s, est. speed input: 69060.18 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2435/3051 [19:15<04:50,  2.12it/s, est. speed input: 69060.33 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2436/3051 [19:15<04:50,  2.12it/s, est. speed input: 69060.41 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2437/3051 [19:16<04:50,  2.12it/s, est. speed input: 69060.54 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2438/3051 [19:16<04:49,  2.12it/s, est. speed input: 69060.63 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2439/3051 [19:17<04:49,  2.12it/s, est. speed input: 69060.74 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|███████▉  | 2440/3051 [19:17<04:48,  2.12it/s, est. speed input: 69060.84 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2441/3051 [19:18<04:49,  2.11it/s, est. speed input: 69060.65 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2442/3051 [19:18<04:48,  2.11it/s, est. speed input: 69060.79 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2443/3051 [19:19<04:47,  2.11it/s, est. speed input: 69060.94 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2444/3051 [19:19<04:47,  2.11it/s, est. speed input: 69060.92 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2445/3051 [19:20<04:46,  2.11it/s, est. speed input: 69061.03 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2446/3051 [19:20<04:46,  2.11it/s, est. speed input: 69061.19 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2447/3051 [19:21<04:45,  2.11it/s, est. speed input: 69061.25 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2448/3051 [19:21<04:45,  2.12it/s, est. speed input: 69061.39 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2449/3051 [19:21<04:44,  2.12it/s, est. speed input: 69061.53 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2450/3051 [19:22<04:44,  2.11it/s, est. speed input: 69061.56 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2451/3051 [19:22<04:43,  2.12it/s, est. speed input: 69061.69 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2452/3051 [19:23<04:43,  2.12it/s, est. speed input: 69061.80 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2453/3051 [19:23<04:42,  2.12it/s, est. speed input: 69061.95 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2454/3051 [19:24<04:42,  2.12it/s, est. speed input: 69062.09 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2455/3051 [19:24<04:41,  2.12it/s, est. speed input: 69062.25 toks/s, output: 2.11 toks/s]
Processed prompts:  80%|████████  | 2456/3051 [19:25<04:41,  2.12it/s, est. speed input: 69062.34 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2457/3051 [19:25<04:40,  2.12it/s, est. speed input: 69062.45 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2458/3051 [19:26<04:40,  2.12it/s, est. speed input: 69062.57 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2459/3051 [19:26<04:41,  2.11it/s, est. speed input: 69062.19 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2460/3051 [19:27<04:40,  2.11it/s, est. speed input: 69062.30 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2461/3051 [19:27<04:39,  2.11it/s, est. speed input: 69062.41 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2462/3051 [19:28<04:38,  2.11it/s, est. speed input: 69062.50 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2463/3051 [19:28<04:38,  2.11it/s, est. speed input: 69062.61 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2464/3051 [19:29<04:37,  2.12it/s, est. speed input: 69062.77 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2465/3051 [19:29<04:36,  2.12it/s, est. speed input: 69062.90 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2466/3051 [19:30<04:36,  2.12it/s, est. speed input: 69063.02 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2467/3051 [19:30<04:35,  2.12it/s, est. speed input: 69063.14 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2468/3051 [19:30<04:35,  2.12it/s, est. speed input: 69063.26 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2469/3051 [19:31<04:34,  2.12it/s, est. speed input: 69063.39 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2470/3051 [19:31<04:34,  2.12it/s, est. speed input: 69063.52 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2471/3051 [19:32<04:33,  2.12it/s, est. speed input: 69063.65 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2472/3051 [19:32<04:33,  2.12it/s, est. speed input: 69063.79 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2473/3051 [19:33<04:32,  2.12it/s, est. speed input: 69063.93 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2474/3051 [19:33<04:32,  2.12it/s, est. speed input: 69064.04 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2475/3051 [19:34<04:32,  2.12it/s, est. speed input: 69064.10 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2476/3051 [19:34<04:31,  2.12it/s, est. speed input: 69064.23 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2477/3051 [19:35<04:31,  2.12it/s, est. speed input: 69064.36 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████  | 2478/3051 [19:35<04:30,  2.12it/s, est. speed input: 69064.51 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2479/3051 [19:36<04:30,  2.12it/s, est. speed input: 69064.62 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2480/3051 [19:36<04:29,  2.12it/s, est. speed input: 69064.76 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2481/3051 [19:37<04:29,  2.12it/s, est. speed input: 69064.87 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2482/3051 [19:37<04:28,  2.12it/s, est. speed input: 69065.00 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2483/3051 [19:38<04:28,  2.12it/s, est. speed input: 69065.14 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2484/3051 [19:38<04:27,  2.12it/s, est. speed input: 69065.28 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2485/3051 [19:39<04:27,  2.12it/s, est. speed input: 69065.44 toks/s, output: 2.11 toks/s]
Processed prompts:  81%|████████▏ | 2486/3051 [19:39<04:26,  2.12it/s, est. speed input: 69065.59 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2487/3051 [19:39<04:26,  2.12it/s, est. speed input: 69065.71 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2488/3051 [19:40<04:25,  2.12it/s, est. speed input: 69065.83 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2489/3051 [19:40<04:26,  2.11it/s, est. speed input: 69065.66 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2490/3051 [19:41<04:25,  2.11it/s, est. speed input: 69065.76 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2491/3051 [19:41<04:24,  2.11it/s, est. speed input: 69065.89 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2492/3051 [19:42<04:24,  2.11it/s, est. speed input: 69065.98 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2493/3051 [19:42<04:23,  2.11it/s, est. speed input: 69066.10 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2494/3051 [19:43<04:23,  2.11it/s, est. speed input: 69066.20 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2495/3051 [19:43<04:22,  2.12it/s, est. speed input: 69066.31 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2496/3051 [19:44<04:22,  2.12it/s, est. speed input: 69066.44 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2497/3051 [19:44<04:21,  2.12it/s, est. speed input: 69066.54 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2498/3051 [19:45<04:21,  2.12it/s, est. speed input: 69066.61 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2499/3051 [19:45<04:20,  2.11it/s, est. speed input: 69066.70 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2500/3051 [19:46<04:20,  2.11it/s, est. speed input: 69066.80 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2501/3051 [19:46<04:20,  2.12it/s, est. speed input: 69066.90 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2502/3051 [19:47<04:19,  2.12it/s, est. speed input: 69067.01 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2503/3051 [19:47<04:19,  2.12it/s, est. speed input: 69067.11 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2504/3051 [19:47<04:18,  2.12it/s, est. speed input: 69067.20 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2505/3051 [19:48<04:18,  2.12it/s, est. speed input: 69067.31 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2506/3051 [19:48<04:17,  2.12it/s, est. speed input: 69067.42 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2507/3051 [19:49<04:17,  2.12it/s, est. speed input: 69067.54 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2508/3051 [19:49<04:16,  2.12it/s, est. speed input: 69067.67 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2509/3051 [19:50<04:16,  2.12it/s, est. speed input: 69067.77 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2510/3051 [19:50<04:15,  2.12it/s, est. speed input: 69067.89 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2511/3051 [19:51<04:15,  2.12it/s, est. speed input: 69068.00 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2512/3051 [19:51<04:14,  2.12it/s, est. speed input: 69068.14 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2513/3051 [19:52<04:14,  2.12it/s, est. speed input: 69068.24 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2514/3051 [19:52<04:13,  2.12it/s, est. speed input: 69068.34 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2515/3051 [19:53<04:13,  2.12it/s, est. speed input: 69068.44 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2516/3051 [19:53<04:12,  2.12it/s, est. speed input: 69068.50 toks/s, output: 2.11 toks/s]
Processed prompts:  82%|████████▏ | 2517/3051 [19:54<04:12,  2.12it/s, est. speed input: 69068.62 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2518/3051 [19:54<04:11,  2.12it/s, est. speed input: 69068.71 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2519/3051 [19:55<04:11,  2.11it/s, est. speed input: 69068.73 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2520/3051 [19:55<04:11,  2.11it/s, est. speed input: 69068.80 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2521/3051 [19:56<04:10,  2.11it/s, est. speed input: 69068.92 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2522/3051 [19:56<04:10,  2.12it/s, est. speed input: 69069.07 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2523/3051 [19:56<04:09,  2.12it/s, est. speed input: 69069.18 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2524/3051 [19:57<04:09,  2.11it/s, est. speed input: 69069.16 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2525/3051 [19:57<04:08,  2.11it/s, est. speed input: 69069.25 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2526/3051 [19:58<04:08,  2.12it/s, est. speed input: 69069.39 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2527/3051 [19:58<04:07,  2.12it/s, est. speed input: 69069.48 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2528/3051 [19:59<04:07,  2.12it/s, est. speed input: 69069.60 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2529/3051 [19:59<04:06,  2.11it/s, est. speed input: 69069.66 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2530/3051 [20:00<04:06,  2.11it/s, est. speed input: 69069.58 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2531/3051 [20:00<04:06,  2.11it/s, est. speed input: 69069.69 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2532/3051 [20:01<04:05,  2.11it/s, est. speed input: 69069.81 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2533/3051 [20:01<04:04,  2.11it/s, est. speed input: 69069.92 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2534/3051 [20:02<04:04,  2.11it/s, est. speed input: 69070.00 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2535/3051 [20:02<04:03,  2.12it/s, est. speed input: 69070.13 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2536/3051 [20:03<04:03,  2.12it/s, est. speed input: 69070.25 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2537/3051 [20:03<04:03,  2.12it/s, est. speed input: 69070.32 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2538/3051 [20:04<04:02,  2.11it/s, est. speed input: 69070.39 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2539/3051 [20:04<04:02,  2.11it/s, est. speed input: 69070.45 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2540/3051 [20:05<04:01,  2.12it/s, est. speed input: 69070.58 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2541/3051 [20:05<04:01,  2.12it/s, est. speed input: 69070.67 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2542/3051 [20:05<04:00,  2.11it/s, est. speed input: 69070.65 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2543/3051 [20:06<04:00,  2.11it/s, est. speed input: 69070.73 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2544/3051 [20:06<03:59,  2.11it/s, est. speed input: 69070.82 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2545/3051 [20:07<03:59,  2.11it/s, est. speed input: 69070.90 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2546/3051 [20:07<03:58,  2.12it/s, est. speed input: 69071.04 toks/s, output: 2.11 toks/s]
Processed prompts:  83%|████████▎ | 2547/3051 [20:08<03:58,  2.11it/s, est. speed input: 69071.10 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2548/3051 [20:08<03:57,  2.11it/s, est. speed input: 69071.19 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2549/3051 [20:09<03:57,  2.12it/s, est. speed input: 69071.32 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2550/3051 [20:09<03:56,  2.12it/s, est. speed input: 69071.43 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2551/3051 [20:10<03:56,  2.12it/s, est. speed input: 69071.55 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2552/3051 [20:10<03:55,  2.12it/s, est. speed input: 69071.66 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2553/3051 [20:11<03:55,  2.12it/s, est. speed input: 69071.75 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2554/3051 [20:11<03:54,  2.12it/s, est. speed input: 69071.87 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▎ | 2555/3051 [20:12<03:54,  2.12it/s, est. speed input: 69072.00 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2556/3051 [20:12<03:54,  2.11it/s, est. speed input: 69072.01 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2557/3051 [20:13<03:53,  2.11it/s, est. speed input: 69071.97 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2558/3051 [20:13<03:53,  2.11it/s, est. speed input: 69072.08 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2559/3051 [20:13<03:52,  2.11it/s, est. speed input: 69072.19 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2560/3051 [20:14<03:52,  2.11it/s, est. speed input: 69072.27 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2561/3051 [20:14<03:51,  2.12it/s, est. speed input: 69072.40 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2562/3051 [20:15<03:51,  2.12it/s, est. speed input: 69072.54 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2563/3051 [20:15<03:50,  2.12it/s, est. speed input: 69072.61 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2564/3051 [20:16<03:50,  2.11it/s, est. speed input: 69072.65 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2565/3051 [20:16<03:49,  2.11it/s, est. speed input: 69072.70 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2566/3051 [20:17<03:49,  2.11it/s, est. speed input: 69072.80 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2567/3051 [20:17<03:49,  2.11it/s, est. speed input: 69072.85 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2568/3051 [20:18<03:48,  2.11it/s, est. speed input: 69072.96 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2569/3051 [20:18<03:48,  2.11it/s, est. speed input: 69072.99 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2570/3051 [20:19<03:47,  2.11it/s, est. speed input: 69073.05 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2571/3051 [20:19<03:47,  2.11it/s, est. speed input: 69073.15 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2572/3051 [20:20<03:46,  2.11it/s, est. speed input: 69073.23 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2573/3051 [20:20<03:46,  2.11it/s, est. speed input: 69073.35 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2574/3051 [20:21<03:45,  2.11it/s, est. speed input: 69073.42 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2575/3051 [20:21<03:45,  2.12it/s, est. speed input: 69073.53 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2576/3051 [20:22<03:44,  2.12it/s, est. speed input: 69073.65 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2577/3051 [20:22<03:43,  2.12it/s, est. speed input: 69073.78 toks/s, output: 2.11 toks/s]
Processed prompts:  84%|████████▍ | 2578/3051 [20:22<03:43,  2.12it/s, est. speed input: 69073.89 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2579/3051 [20:23<03:43,  2.11it/s, est. speed input: 69073.89 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2580/3051 [20:23<03:42,  2.11it/s, est. speed input: 69073.97 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2581/3051 [20:24<03:42,  2.11it/s, est. speed input: 69074.05 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2582/3051 [20:24<03:41,  2.11it/s, est. speed input: 69074.13 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2583/3051 [20:25<03:41,  2.11it/s, est. speed input: 69074.24 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2584/3051 [20:25<03:41,  2.11it/s, est. speed input: 69074.25 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2585/3051 [20:26<03:40,  2.11it/s, est. speed input: 69074.31 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2586/3051 [20:26<03:40,  2.11it/s, est. speed input: 69074.37 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2587/3051 [20:27<03:39,  2.11it/s, est. speed input: 69074.51 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2588/3051 [20:27<03:38,  2.11it/s, est. speed input: 69074.60 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2589/3051 [20:28<03:38,  2.12it/s, est. speed input: 69074.70 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2590/3051 [20:28<03:37,  2.12it/s, est. speed input: 69074.80 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2591/3051 [20:29<03:37,  2.12it/s, est. speed input: 69074.90 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2592/3051 [20:29<03:36,  2.12it/s, est. speed input: 69074.98 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 2593/3051 [20:30<03:36,  2.12it/s, est. speed input: 69075.11 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2594/3051 [20:30<03:35,  2.12it/s, est. speed input: 69075.25 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2595/3051 [20:31<03:35,  2.12it/s, est. speed input: 69075.32 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2596/3051 [20:31<03:35,  2.12it/s, est. speed input: 69075.41 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2597/3051 [20:31<03:34,  2.11it/s, est. speed input: 69075.37 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2598/3051 [20:32<03:34,  2.11it/s, est. speed input: 69075.48 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2599/3051 [20:32<03:33,  2.11it/s, est. speed input: 69075.56 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2600/3051 [20:33<03:33,  2.12it/s, est. speed input: 69075.71 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2601/3051 [20:33<03:32,  2.12it/s, est. speed input: 69075.80 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2602/3051 [20:34<03:32,  2.11it/s, est. speed input: 69075.83 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2603/3051 [20:34<03:31,  2.11it/s, est. speed input: 69075.91 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2604/3051 [20:35<03:31,  2.12it/s, est. speed input: 69076.03 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2605/3051 [20:35<03:30,  2.12it/s, est. speed input: 69076.16 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2606/3051 [20:36<03:30,  2.11it/s, est. speed input: 69076.19 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2607/3051 [20:36<03:29,  2.11it/s, est. speed input: 69076.27 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 2608/3051 [20:37<03:29,  2.12it/s, est. speed input: 69076.38 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2609/3051 [20:37<03:29,  2.11it/s, est. speed input: 69076.45 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2610/3051 [20:38<03:28,  2.11it/s, est. speed input: 69076.54 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2611/3051 [20:38<03:27,  2.12it/s, est. speed input: 69076.65 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2612/3051 [20:39<03:27,  2.11it/s, est. speed input: 69076.67 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2613/3051 [20:39<03:27,  2.11it/s, est. speed input: 69076.71 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2614/3051 [20:40<03:26,  2.11it/s, est. speed input: 69076.83 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2615/3051 [20:40<03:26,  2.11it/s, est. speed input: 69076.92 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2616/3051 [20:40<03:25,  2.12it/s, est. speed input: 69077.02 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2617/3051 [20:41<03:25,  2.12it/s, est. speed input: 69077.14 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2618/3051 [20:41<03:24,  2.12it/s, est. speed input: 69077.22 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2619/3051 [20:42<03:24,  2.12it/s, est. speed input: 69077.32 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2620/3051 [20:42<03:23,  2.12it/s, est. speed input: 69077.40 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2621/3051 [20:43<03:23,  2.11it/s, est. speed input: 69077.46 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2622/3051 [20:43<03:23,  2.11it/s, est. speed input: 69077.44 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2623/3051 [20:44<03:22,  2.11it/s, est. speed input: 69077.52 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2624/3051 [20:44<03:22,  2.11it/s, est. speed input: 69077.62 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2625/3051 [20:45<03:21,  2.11it/s, est. speed input: 69077.71 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2626/3051 [20:45<03:20,  2.11it/s, est. speed input: 69077.81 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2627/3051 [20:46<03:20,  2.12it/s, est. speed input: 69077.94 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2628/3051 [20:46<03:19,  2.12it/s, est. speed input: 69078.03 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2629/3051 [20:47<03:19,  2.12it/s, est. speed input: 69078.12 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2630/3051 [20:47<03:19,  2.11it/s, est. speed input: 69078.18 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 2631/3051 [20:48<03:18,  2.11it/s, est. speed input: 69078.26 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2632/3051 [20:48<03:18,  2.11it/s, est. speed input: 69078.31 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2633/3051 [20:48<03:17,  2.11it/s, est. speed input: 69078.42 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2634/3051 [20:49<03:17,  2.11it/s, est. speed input: 69078.45 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2635/3051 [20:49<03:16,  2.11it/s, est. speed input: 69078.55 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2636/3051 [20:50<03:16,  2.11it/s, est. speed input: 69078.64 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2637/3051 [20:50<03:15,  2.12it/s, est. speed input: 69078.76 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2638/3051 [20:51<03:15,  2.12it/s, est. speed input: 69078.86 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 2639/3051 [20:51<03:14,  2.11it/s, est. speed input: 69078.86 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2640/3051 [20:52<03:14,  2.11it/s, est. speed input: 69078.90 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2641/3051 [20:52<03:14,  2.11it/s, est. speed input: 69078.96 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2642/3051 [20:53<03:13,  2.11it/s, est. speed input: 69079.10 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2643/3051 [20:53<03:12,  2.11it/s, est. speed input: 69079.19 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2644/3051 [20:54<03:12,  2.11it/s, est. speed input: 69079.15 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2645/3051 [20:54<03:12,  2.11it/s, est. speed input: 69079.25 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2646/3051 [20:55<03:11,  2.11it/s, est. speed input: 69079.36 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2647/3051 [20:55<03:11,  2.11it/s, est. speed input: 69079.33 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2648/3051 [20:56<03:10,  2.11it/s, est. speed input: 69079.44 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2649/3051 [20:56<03:10,  2.12it/s, est. speed input: 69079.58 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2650/3051 [20:57<03:09,  2.12it/s, est. speed input: 69079.67 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2651/3051 [20:57<03:09,  2.11it/s, est. speed input: 69079.71 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2652/3051 [20:57<03:08,  2.11it/s, est. speed input: 69079.81 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2653/3051 [20:58<03:08,  2.12it/s, est. speed input: 69079.93 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2654/3051 [20:58<03:07,  2.12it/s, est. speed input: 69080.06 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2655/3051 [20:59<03:07,  2.12it/s, est. speed input: 69080.15 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2656/3051 [20:59<03:06,  2.12it/s, est. speed input: 69080.28 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2657/3051 [21:00<03:06,  2.12it/s, est. speed input: 69080.34 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2658/3051 [21:00<03:05,  2.12it/s, est. speed input: 69080.43 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2659/3051 [21:01<03:05,  2.12it/s, est. speed input: 69080.54 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2660/3051 [21:01<03:04,  2.12it/s, est. speed input: 69080.61 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2661/3051 [21:02<03:04,  2.11it/s, est. speed input: 69080.64 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2662/3051 [21:02<03:03,  2.11it/s, est. speed input: 69080.73 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2663/3051 [21:03<03:03,  2.11it/s, est. speed input: 69080.81 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2664/3051 [21:03<03:02,  2.12it/s, est. speed input: 69080.93 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2665/3051 [21:04<03:02,  2.12it/s, est. speed input: 69081.05 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2666/3051 [21:04<03:01,  2.12it/s, est. speed input: 69081.16 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2667/3051 [21:05<03:01,  2.12it/s, est. speed input: 69081.29 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2668/3051 [21:05<03:01,  2.12it/s, est. speed input: 69081.32 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 2669/3051 [21:06<03:00,  2.12it/s, est. speed input: 69081.42 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2670/3051 [21:06<03:00,  2.12it/s, est. speed input: 69081.49 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2671/3051 [21:06<02:59,  2.12it/s, est. speed input: 69081.60 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2672/3051 [21:07<02:59,  2.12it/s, est. speed input: 69081.68 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2673/3051 [21:07<02:58,  2.11it/s, est. speed input: 69081.67 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2674/3051 [21:08<02:58,  2.11it/s, est. speed input: 69081.67 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2675/3051 [21:08<02:57,  2.11it/s, est. speed input: 69081.79 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2676/3051 [21:09<02:57,  2.11it/s, est. speed input: 69081.82 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2677/3051 [21:09<02:56,  2.11it/s, est. speed input: 69081.96 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2678/3051 [21:10<02:56,  2.12it/s, est. speed input: 69082.05 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2679/3051 [21:10<02:55,  2.12it/s, est. speed input: 69082.18 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2680/3051 [21:11<02:55,  2.12it/s, est. speed input: 69082.29 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2681/3051 [21:11<02:55,  2.11it/s, est. speed input: 69082.26 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2682/3051 [21:12<02:54,  2.11it/s, est. speed input: 69082.33 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2683/3051 [21:12<02:54,  2.11it/s, est. speed input: 69082.44 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2684/3051 [21:13<02:53,  2.11it/s, est. speed input: 69082.51 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2685/3051 [21:13<02:53,  2.12it/s, est. speed input: 69082.63 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2686/3051 [21:14<02:52,  2.11it/s, est. speed input: 69082.63 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2687/3051 [21:14<02:52,  2.11it/s, est. speed input: 69082.72 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2688/3051 [21:14<02:51,  2.11it/s, est. speed input: 69082.82 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2689/3051 [21:15<02:51,  2.11it/s, est. speed input: 69082.80 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2690/3051 [21:15<02:50,  2.11it/s, est. speed input: 69082.90 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2691/3051 [21:16<02:50,  2.11it/s, est. speed input: 69082.96 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2692/3051 [21:16<02:49,  2.12it/s, est. speed input: 69083.09 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2693/3051 [21:17<02:49,  2.12it/s, est. speed input: 69083.18 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2694/3051 [21:17<02:48,  2.12it/s, est. speed input: 69083.28 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2695/3051 [21:18<02:48,  2.12it/s, est. speed input: 69083.37 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2696/3051 [21:18<02:47,  2.11it/s, est. speed input: 69083.37 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2697/3051 [21:19<02:47,  2.11it/s, est. speed input: 69083.44 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2698/3051 [21:19<02:46,  2.11it/s, est. speed input: 69083.53 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2699/3051 [21:20<02:46,  2.11it/s, est. speed input: 69083.55 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 2700/3051 [21:20<02:46,  2.11it/s, est. speed input: 69083.59 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2701/3051 [21:21<02:45,  2.11it/s, est. speed input: 69083.69 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2702/3051 [21:21<02:45,  2.11it/s, est. speed input: 69083.78 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2703/3051 [21:22<02:44,  2.11it/s, est. speed input: 69083.84 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2704/3051 [21:22<02:44,  2.11it/s, est. speed input: 69083.95 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2705/3051 [21:23<02:43,  2.12it/s, est. speed input: 69084.05 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2706/3051 [21:23<02:43,  2.12it/s, est. speed input: 69084.14 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 2707/3051 [21:23<02:42,  2.12it/s, est. speed input: 69084.25 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2708/3051 [21:24<02:42,  2.12it/s, est. speed input: 69084.36 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2709/3051 [21:24<02:42,  2.11it/s, est. speed input: 69084.23 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2710/3051 [21:25<02:41,  2.11it/s, est. speed input: 69084.38 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2711/3051 [21:25<02:40,  2.11it/s, est. speed input: 69084.47 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2712/3051 [21:26<02:40,  2.12it/s, est. speed input: 69084.58 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2713/3051 [21:26<02:39,  2.11it/s, est. speed input: 69084.61 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2714/3051 [21:27<02:39,  2.11it/s, est. speed input: 69084.70 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2715/3051 [21:27<02:38,  2.11it/s, est. speed input: 69084.79 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2716/3051 [21:28<02:38,  2.11it/s, est. speed input: 69084.81 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2717/3051 [21:28<02:37,  2.11it/s, est. speed input: 69084.92 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2718/3051 [21:29<02:37,  2.11it/s, est. speed input: 69085.00 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2719/3051 [21:29<02:36,  2.12it/s, est. speed input: 69085.12 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2720/3051 [21:30<02:36,  2.12it/s, est. speed input: 69085.24 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2721/3051 [21:30<02:35,  2.12it/s, est. speed input: 69085.35 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2722/3051 [21:31<02:35,  2.12it/s, est. speed input: 69085.44 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2723/3051 [21:31<02:34,  2.12it/s, est. speed input: 69085.52 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2724/3051 [21:32<02:34,  2.12it/s, est. speed input: 69085.60 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2725/3051 [21:32<02:34,  2.12it/s, est. speed input: 69085.68 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2726/3051 [21:32<02:33,  2.11it/s, est. speed input: 69085.71 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2727/3051 [21:33<02:33,  2.11it/s, est. speed input: 69085.77 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2728/3051 [21:33<02:32,  2.11it/s, est. speed input: 69085.82 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2729/3051 [21:34<02:32,  2.11it/s, est. speed input: 69085.81 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 2730/3051 [21:34<02:31,  2.11it/s, est. speed input: 69085.93 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2731/3051 [21:35<02:31,  2.11it/s, est. speed input: 69085.95 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2732/3051 [21:35<02:30,  2.11it/s, est. speed input: 69086.04 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2733/3051 [21:36<02:30,  2.11it/s, est. speed input: 69086.10 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2734/3051 [21:36<02:29,  2.11it/s, est. speed input: 69086.21 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2735/3051 [21:37<02:29,  2.12it/s, est. speed input: 69086.30 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2736/3051 [21:37<02:28,  2.12it/s, est. speed input: 69086.43 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2737/3051 [21:38<02:28,  2.12it/s, est. speed input: 69086.51 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2738/3051 [21:38<02:27,  2.12it/s, est. speed input: 69086.57 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2739/3051 [21:39<02:27,  2.12it/s, est. speed input: 69086.64 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2740/3051 [21:39<02:27,  2.11it/s, est. speed input: 69086.68 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2741/3051 [21:40<02:26,  2.11it/s, est. speed input: 69086.79 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2742/3051 [21:40<02:26,  2.12it/s, est. speed input: 69086.87 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2743/3051 [21:41<02:25,  2.12it/s, est. speed input: 69086.98 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2744/3051 [21:41<02:25,  2.12it/s, est. speed input: 69087.10 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 2745/3051 [21:41<02:24,  2.12it/s, est. speed input: 69087.14 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2746/3051 [21:42<02:24,  2.11it/s, est. speed input: 69087.21 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2747/3051 [21:42<02:23,  2.11it/s, est. speed input: 69087.29 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2748/3051 [21:43<02:23,  2.11it/s, est. speed input: 69087.37 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2749/3051 [21:43<02:22,  2.11it/s, est. speed input: 69087.45 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2750/3051 [21:44<02:22,  2.12it/s, est. speed input: 69087.58 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2751/3051 [21:44<02:21,  2.11it/s, est. speed input: 69087.54 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2752/3051 [21:45<02:21,  2.11it/s, est. speed input: 69087.59 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2753/3051 [21:45<02:21,  2.11it/s, est. speed input: 69087.66 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2754/3051 [21:46<02:20,  2.11it/s, est. speed input: 69087.68 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2755/3051 [21:46<02:20,  2.11it/s, est. speed input: 69087.71 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2756/3051 [21:47<02:19,  2.11it/s, est. speed input: 69087.78 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2757/3051 [21:47<02:19,  2.11it/s, est. speed input: 69087.74 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2758/3051 [21:48<02:18,  2.11it/s, est. speed input: 69087.79 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2759/3051 [21:48<02:18,  2.11it/s, est. speed input: 69087.87 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2760/3051 [21:49<02:17,  2.11it/s, est. speed input: 69087.96 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 2761/3051 [21:49<02:17,  2.11it/s, est. speed input: 69088.06 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2762/3051 [21:49<02:16,  2.11it/s, est. speed input: 69088.14 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2763/3051 [21:50<02:16,  2.12it/s, est. speed input: 69088.27 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2764/3051 [21:50<02:15,  2.11it/s, est. speed input: 69088.31 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2765/3051 [21:51<02:15,  2.11it/s, est. speed input: 69088.37 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2766/3051 [21:51<02:14,  2.11it/s, est. speed input: 69088.44 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2767/3051 [21:52<02:14,  2.11it/s, est. speed input: 69088.47 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2768/3051 [21:52<02:13,  2.11it/s, est. speed input: 69088.59 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2769/3051 [21:53<02:13,  2.12it/s, est. speed input: 69088.67 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2770/3051 [21:53<02:12,  2.12it/s, est. speed input: 69088.76 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2771/3051 [21:54<02:12,  2.12it/s, est. speed input: 69088.88 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2772/3051 [21:54<02:11,  2.12it/s, est. speed input: 69088.97 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2773/3051 [21:55<02:11,  2.12it/s, est. speed input: 69089.06 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2774/3051 [21:55<02:11,  2.11it/s, est. speed input: 69088.86 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2775/3051 [21:56<02:10,  2.11it/s, est. speed input: 69088.99 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2776/3051 [21:56<02:10,  2.11it/s, est. speed input: 69089.08 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2777/3051 [21:57<02:09,  2.11it/s, est. speed input: 69089.16 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2778/3051 [21:57<02:09,  2.11it/s, est. speed input: 69089.26 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2779/3051 [21:58<02:08,  2.12it/s, est. speed input: 69089.34 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2780/3051 [21:58<02:08,  2.12it/s, est. speed input: 69089.44 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2781/3051 [21:58<02:07,  2.12it/s, est. speed input: 69089.52 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2782/3051 [21:59<02:07,  2.12it/s, est. speed input: 69089.61 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2783/3051 [21:59<02:06,  2.12it/s, est. speed input: 69089.72 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 2784/3051 [22:00<02:06,  2.12it/s, est. speed input: 69089.79 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2785/3051 [22:00<02:05,  2.12it/s, est. speed input: 69089.88 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2786/3051 [22:01<02:05,  2.11it/s, est. speed input: 69089.82 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2787/3051 [22:01<02:04,  2.11it/s, est. speed input: 69089.88 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2788/3051 [22:02<02:04,  2.11it/s, est. speed input: 69089.97 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2789/3051 [22:02<02:03,  2.11it/s, est. speed input: 69090.05 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2790/3051 [22:03<02:03,  2.12it/s, est. speed input: 69090.23 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 2791/3051 [22:03<02:02,  2.12it/s, est. speed input: 69090.31 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2792/3051 [22:04<02:02,  2.12it/s, est. speed input: 69090.36 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2793/3051 [22:04<02:01,  2.12it/s, est. speed input: 69090.47 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2794/3051 [22:05<02:01,  2.12it/s, est. speed input: 69090.59 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2795/3051 [22:05<02:00,  2.12it/s, est. speed input: 69090.70 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2796/3051 [22:06<02:00,  2.11it/s, est. speed input: 69090.66 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2797/3051 [22:06<02:00,  2.11it/s, est. speed input: 69090.71 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2798/3051 [22:07<01:59,  2.11it/s, est. speed input: 69090.80 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2799/3051 [22:07<01:59,  2.11it/s, est. speed input: 69090.89 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2800/3051 [22:07<01:58,  2.12it/s, est. speed input: 69090.99 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2801/3051 [22:08<01:58,  2.12it/s, est. speed input: 69091.07 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2802/3051 [22:08<01:57,  2.12it/s, est. speed input: 69091.17 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2803/3051 [22:09<01:57,  2.12it/s, est. speed input: 69091.25 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2804/3051 [22:09<01:56,  2.11it/s, est. speed input: 69091.29 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2805/3051 [22:10<01:56,  2.11it/s, est. speed input: 69091.34 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2806/3051 [22:10<01:55,  2.12it/s, est. speed input: 69091.44 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2807/3051 [22:11<01:55,  2.12it/s, est. speed input: 69091.55 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2808/3051 [22:11<01:54,  2.12it/s, est. speed input: 69091.62 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2809/3051 [22:12<01:54,  2.11it/s, est. speed input: 69091.67 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2810/3051 [22:12<01:53,  2.11it/s, est. speed input: 69091.74 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2811/3051 [22:13<01:53,  2.12it/s, est. speed input: 69091.85 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2812/3051 [22:13<01:52,  2.12it/s, est. speed input: 69091.94 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2813/3051 [22:14<01:52,  2.12it/s, est. speed input: 69092.00 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2814/3051 [22:14<01:52,  2.12it/s, est. speed input: 69092.08 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2815/3051 [22:15<01:51,  2.12it/s, est. speed input: 69092.17 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2816/3051 [22:15<01:51,  2.11it/s, est. speed input: 69092.23 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2817/3051 [22:16<01:50,  2.12it/s, est. speed input: 69092.34 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2818/3051 [22:16<01:50,  2.12it/s, est. speed input: 69092.41 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2819/3051 [22:16<01:49,  2.12it/s, est. speed input: 69092.52 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2820/3051 [22:17<01:49,  2.12it/s, est. speed input: 69092.57 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2821/3051 [22:17<01:48,  2.11it/s, est. speed input: 69092.64 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 2822/3051 [22:18<01:48,  2.12it/s, est. speed input: 69092.74 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2823/3051 [22:18<01:47,  2.11it/s, est. speed input: 69092.77 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2824/3051 [22:19<01:47,  2.11it/s, est. speed input: 69092.86 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2825/3051 [22:19<01:46,  2.12it/s, est. speed input: 69092.95 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2826/3051 [22:20<01:46,  2.12it/s, est. speed input: 69093.07 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2827/3051 [22:20<01:45,  2.12it/s, est. speed input: 69093.14 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2828/3051 [22:21<01:45,  2.11it/s, est. speed input: 69093.13 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2829/3051 [22:21<01:45,  2.11it/s, est. speed input: 69093.20 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2830/3051 [22:22<01:44,  2.11it/s, est. speed input: 69093.26 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2831/3051 [22:22<01:44,  2.11it/s, est. speed input: 69093.35 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2832/3051 [22:23<01:43,  2.12it/s, est. speed input: 69093.45 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2833/3051 [22:23<01:43,  2.11it/s, est. speed input: 69093.44 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2834/3051 [22:24<01:42,  2.11it/s, est. speed input: 69093.50 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2835/3051 [22:24<01:42,  2.11it/s, est. speed input: 69093.61 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2836/3051 [22:24<01:41,  2.11it/s, est. speed input: 69093.68 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2837/3051 [22:25<01:41,  2.12it/s, est. speed input: 69093.77 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2838/3051 [22:25<01:40,  2.11it/s, est. speed input: 69093.70 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2839/3051 [22:26<01:40,  2.11it/s, est. speed input: 69093.79 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2840/3051 [22:26<01:39,  2.11it/s, est. speed input: 69093.88 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2841/3051 [22:27<01:39,  2.11it/s, est. speed input: 69093.94 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2842/3051 [22:27<01:38,  2.11it/s, est. speed input: 69094.01 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2843/3051 [22:28<01:38,  2.11it/s, est. speed input: 69094.05 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2844/3051 [22:28<01:37,  2.11it/s, est. speed input: 69094.16 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2845/3051 [22:29<01:37,  2.11it/s, est. speed input: 69094.23 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2846/3051 [22:29<01:36,  2.12it/s, est. speed input: 69094.35 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2847/3051 [22:30<01:36,  2.12it/s, est. speed input: 69094.46 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2848/3051 [22:30<01:36,  2.11it/s, est. speed input: 69094.43 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2849/3051 [22:31<01:35,  2.11it/s, est. speed input: 69094.46 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2850/3051 [22:31<01:35,  2.11it/s, est. speed input: 69094.54 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2851/3051 [22:32<01:34,  2.11it/s, est. speed input: 69094.32 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 2852/3051 [22:32<01:34,  2.11it/s, est. speed input: 69094.38 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2853/3051 [22:33<01:33,  2.11it/s, est. speed input: 69094.48 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2854/3051 [22:33<01:33,  2.11it/s, est. speed input: 69094.56 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2855/3051 [22:33<01:32,  2.11it/s, est. speed input: 69094.64 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2856/3051 [22:34<01:32,  2.11it/s, est. speed input: 69094.75 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2857/3051 [22:34<01:31,  2.12it/s, est. speed input: 69094.88 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2858/3051 [22:35<01:31,  2.11it/s, est. speed input: 69094.91 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2859/3051 [22:35<01:30,  2.11it/s, est. speed input: 69094.87 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 2860/3051 [22:36<01:30,  2.11it/s, est. speed input: 69094.92 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2861/3051 [22:36<01:29,  2.11it/s, est. speed input: 69095.04 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2862/3051 [22:37<01:29,  2.12it/s, est. speed input: 69095.14 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2863/3051 [22:37<01:28,  2.12it/s, est. speed input: 69095.25 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2864/3051 [22:38<01:28,  2.12it/s, est. speed input: 69095.36 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2865/3051 [22:38<01:27,  2.12it/s, est. speed input: 69095.47 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2866/3051 [22:39<01:27,  2.12it/s, est. speed input: 69095.58 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2867/3051 [22:39<01:26,  2.12it/s, est. speed input: 69095.66 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2868/3051 [22:40<01:26,  2.12it/s, est. speed input: 69095.73 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2869/3051 [22:40<01:25,  2.12it/s, est. speed input: 69095.89 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2870/3051 [22:41<01:26,  2.09it/s, est. speed input: 69094.82 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2871/3051 [22:41<01:25,  2.10it/s, est. speed input: 69094.93 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2872/3051 [22:42<01:25,  2.10it/s, est. speed input: 69095.05 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2873/3051 [22:42<01:24,  2.10it/s, est. speed input: 69094.86 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2874/3051 [22:42<01:24,  2.10it/s, est. speed input: 69094.94 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2875/3051 [22:43<01:23,  2.11it/s, est. speed input: 69095.06 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2876/3051 [22:43<01:22,  2.11it/s, est. speed input: 69095.17 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2877/3051 [22:44<01:22,  2.11it/s, est. speed input: 69095.30 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2878/3051 [22:44<01:21,  2.11it/s, est. speed input: 69095.35 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2879/3051 [22:45<01:21,  2.12it/s, est. speed input: 69095.46 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2880/3051 [22:45<01:20,  2.12it/s, est. speed input: 69095.55 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2881/3051 [22:46<01:20,  2.12it/s, est. speed input: 69095.64 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2882/3051 [22:46<01:19,  2.12it/s, est. speed input: 69095.74 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 2883/3051 [22:47<01:19,  2.12it/s, est. speed input: 69095.87 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2884/3051 [22:47<01:18,  2.12it/s, est. speed input: 69095.98 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2885/3051 [22:48<01:18,  2.12it/s, est. speed input: 69096.04 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2886/3051 [22:48<01:18,  2.11it/s, est. speed input: 69095.96 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2887/3051 [22:49<01:17,  2.11it/s, est. speed input: 69096.01 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2888/3051 [22:49<01:17,  2.11it/s, est. speed input: 69096.09 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2889/3051 [22:50<01:16,  2.11it/s, est. speed input: 69096.20 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2890/3051 [22:50<01:16,  2.12it/s, est. speed input: 69096.29 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2891/3051 [22:51<01:15,  2.11it/s, est. speed input: 69096.34 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2892/3051 [22:51<01:15,  2.11it/s, est. speed input: 69096.38 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2893/3051 [22:51<01:14,  2.12it/s, est. speed input: 69096.49 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2894/3051 [22:52<01:14,  2.12it/s, est. speed input: 69096.56 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2895/3051 [22:52<01:13,  2.12it/s, est. speed input: 69096.63 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2896/3051 [22:53<01:13,  2.12it/s, est. speed input: 69096.75 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2897/3051 [22:53<01:12,  2.11it/s, est. speed input: 69096.77 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 2898/3051 [22:54<01:12,  2.11it/s, est. speed input: 69096.84 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2899/3051 [22:54<01:11,  2.12it/s, est. speed input: 69096.92 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2900/3051 [22:55<01:11,  2.12it/s, est. speed input: 69096.99 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2901/3051 [22:55<01:10,  2.11it/s, est. speed input: 69097.00 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2902/3051 [22:56<01:10,  2.11it/s, est. speed input: 69097.04 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2903/3051 [22:56<01:10,  2.11it/s, est. speed input: 69097.11 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2904/3051 [22:57<01:09,  2.11it/s, est. speed input: 69097.19 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2905/3051 [22:57<01:09,  2.12it/s, est. speed input: 69097.28 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2906/3051 [22:58<01:08,  2.11it/s, est. speed input: 69097.34 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2907/3051 [22:58<01:08,  2.11it/s, est. speed input: 69097.41 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2908/3051 [22:59<01:07,  2.12it/s, est. speed input: 69097.51 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2909/3051 [22:59<01:07,  2.12it/s, est. speed input: 69097.59 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2910/3051 [23:00<01:06,  2.12it/s, est. speed input: 69097.66 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2911/3051 [23:00<01:06,  2.12it/s, est. speed input: 69097.74 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2912/3051 [23:00<01:05,  2.12it/s, est. speed input: 69097.84 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 2913/3051 [23:01<01:05,  2.12it/s, est. speed input: 69097.90 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2914/3051 [23:01<01:04,  2.12it/s, est. speed input: 69097.97 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2915/3051 [23:02<01:04,  2.12it/s, est. speed input: 69098.08 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2916/3051 [23:02<01:03,  2.12it/s, est. speed input: 69098.17 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2917/3051 [23:03<01:03,  2.12it/s, est. speed input: 69098.23 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2918/3051 [23:03<01:02,  2.12it/s, est. speed input: 69098.29 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2919/3051 [23:04<01:02,  2.12it/s, est. speed input: 69098.39 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2920/3051 [23:04<01:01,  2.12it/s, est. speed input: 69098.49 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2921/3051 [23:05<01:01,  2.12it/s, est. speed input: 69098.60 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2922/3051 [23:05<01:00,  2.12it/s, est. speed input: 69098.68 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2923/3051 [23:06<01:00,  2.12it/s, est. speed input: 69098.73 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2924/3051 [23:06<01:00,  2.12it/s, est. speed input: 69098.81 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2925/3051 [23:07<00:59,  2.12it/s, est. speed input: 69098.90 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2926/3051 [23:07<00:59,  2.12it/s, est. speed input: 69099.00 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2927/3051 [23:08<00:58,  2.12it/s, est. speed input: 69099.09 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2928/3051 [23:08<00:58,  2.12it/s, est. speed input: 69099.19 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2929/3051 [23:08<00:57,  2.12it/s, est. speed input: 69099.26 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2930/3051 [23:09<00:57,  2.12it/s, est. speed input: 69099.35 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2931/3051 [23:09<00:56,  2.12it/s, est. speed input: 69099.46 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2932/3051 [23:10<00:56,  2.12it/s, est. speed input: 69099.55 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2933/3051 [23:10<00:55,  2.12it/s, est. speed input: 69099.59 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2934/3051 [23:11<00:55,  2.12it/s, est. speed input: 69099.67 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2935/3051 [23:11<00:54,  2.12it/s, est. speed input: 69099.76 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 2936/3051 [23:12<00:54,  2.12it/s, est. speed input: 69099.84 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2937/3051 [23:12<00:53,  2.12it/s, est. speed input: 69099.94 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2938/3051 [23:13<00:53,  2.12it/s, est. speed input: 69100.02 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2939/3051 [23:13<00:52,  2.12it/s, est. speed input: 69100.09 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2940/3051 [23:14<00:52,  2.12it/s, est. speed input: 69100.16 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2941/3051 [23:14<00:51,  2.12it/s, est. speed input: 69100.26 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2942/3051 [23:15<00:51,  2.12it/s, est. speed input: 69100.35 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2943/3051 [23:15<00:51,  2.12it/s, est. speed input: 69100.44 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 2944/3051 [23:16<00:50,  2.12it/s, est. speed input: 69100.52 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2945/3051 [23:16<00:50,  2.12it/s, est. speed input: 69100.63 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2946/3051 [23:17<00:49,  2.12it/s, est. speed input: 69100.70 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2947/3051 [23:17<00:49,  2.12it/s, est. speed input: 69100.77 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2948/3051 [23:17<00:48,  2.12it/s, est. speed input: 69100.85 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2949/3051 [23:18<00:48,  2.12it/s, est. speed input: 69100.95 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2950/3051 [23:18<00:47,  2.12it/s, est. speed input: 69101.03 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2951/3051 [23:19<00:47,  2.11it/s, est. speed input: 69100.98 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2952/3051 [23:19<00:46,  2.11it/s, est. speed input: 69101.09 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2953/3051 [23:20<00:46,  2.11it/s, est. speed input: 69101.17 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2954/3051 [23:20<00:45,  2.12it/s, est. speed input: 69101.29 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2955/3051 [23:21<00:45,  2.12it/s, est. speed input: 69101.38 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2956/3051 [23:21<00:44,  2.12it/s, est. speed input: 69101.42 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2957/3051 [23:22<00:44,  2.11it/s, est. speed input: 69101.47 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2958/3051 [23:22<00:43,  2.12it/s, est. speed input: 69101.56 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2959/3051 [23:23<00:43,  2.12it/s, est. speed input: 69101.65 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2960/3051 [23:23<00:43,  2.12it/s, est. speed input: 69101.71 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2961/3051 [23:24<00:42,  2.11it/s, est. speed input: 69101.74 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2962/3051 [23:24<00:42,  2.11it/s, est. speed input: 69101.81 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2963/3051 [23:25<00:41,  2.12it/s, est. speed input: 69101.93 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2964/3051 [23:25<00:41,  2.12it/s, est. speed input: 69101.99 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2965/3051 [23:25<00:40,  2.12it/s, est. speed input: 69102.07 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2966/3051 [23:26<00:40,  2.12it/s, est. speed input: 69102.14 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2967/3051 [23:26<00:39,  2.12it/s, est. speed input: 69102.24 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2968/3051 [23:27<00:39,  2.11it/s, est. speed input: 69102.21 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2969/3051 [23:27<00:38,  2.11it/s, est. speed input: 69102.26 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2970/3051 [23:28<00:38,  2.11it/s, est. speed input: 69102.33 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2971/3051 [23:28<00:37,  2.11it/s, est. speed input: 69102.42 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2972/3051 [23:29<00:37,  2.11it/s, est. speed input: 69102.49 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2973/3051 [23:29<00:36,  2.12it/s, est. speed input: 69102.58 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 2974/3051 [23:30<00:36,  2.11it/s, est. speed input: 69102.62 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2975/3051 [23:30<00:35,  2.11it/s, est. speed input: 69102.69 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2976/3051 [23:31<00:35,  2.12it/s, est. speed input: 69102.77 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2977/3051 [23:31<00:34,  2.12it/s, est. speed input: 69102.87 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2978/3051 [23:32<00:34,  2.12it/s, est. speed input: 69102.95 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2979/3051 [23:32<00:34,  2.12it/s, est. speed input: 69103.01 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2980/3051 [23:33<00:33,  2.12it/s, est. speed input: 69103.07 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2981/3051 [23:33<00:33,  2.11it/s, est. speed input: 69103.02 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2982/3051 [23:34<00:32,  2.11it/s, est. speed input: 69103.08 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2983/3051 [23:34<00:32,  2.11it/s, est. speed input: 69103.19 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2984/3051 [23:34<00:31,  2.12it/s, est. speed input: 69103.30 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2985/3051 [23:35<00:31,  2.12it/s, est. speed input: 69103.39 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2986/3051 [23:35<00:30,  2.12it/s, est. speed input: 69103.47 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2987/3051 [23:36<00:30,  2.12it/s, est. speed input: 69103.55 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2988/3051 [23:36<00:29,  2.12it/s, est. speed input: 69103.66 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2989/3051 [23:37<00:29,  2.12it/s, est. speed input: 69103.73 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2990/3051 [23:37<00:28,  2.11it/s, est. speed input: 69103.71 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2991/3051 [23:38<00:28,  2.11it/s, est. speed input: 69103.80 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2992/3051 [23:38<00:27,  2.12it/s, est. speed input: 69103.87 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2993/3051 [23:39<00:27,  2.11it/s, est. speed input: 69103.93 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2994/3051 [23:39<00:26,  2.12it/s, est. speed input: 69104.04 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2995/3051 [23:40<00:26,  2.12it/s, est. speed input: 69104.12 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2996/3051 [23:40<00:26,  2.12it/s, est. speed input: 69104.17 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2997/3051 [23:41<00:25,  2.11it/s, est. speed input: 69104.19 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2998/3051 [23:41<00:25,  2.11it/s, est. speed input: 69104.26 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 2999/3051 [23:42<00:24,  2.12it/s, est. speed input: 69104.36 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3000/3051 [23:42<00:24,  2.11it/s, est. speed input: 69104.19 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3001/3051 [23:43<00:23,  2.11it/s, est. speed input: 69104.16 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3002/3051 [23:43<00:23,  2.11it/s, est. speed input: 69104.24 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3003/3051 [23:43<00:22,  2.11it/s, est. speed input: 69104.32 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3004/3051 [23:44<00:22,  2.11it/s, est. speed input: 69104.39 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 3005/3051 [23:44<00:21,  2.11it/s, est. speed input: 69104.49 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3006/3051 [23:45<00:21,  2.11it/s, est. speed input: 69104.55 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3007/3051 [23:45<00:20,  2.11it/s, est. speed input: 69104.59 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3008/3051 [23:46<00:20,  2.12it/s, est. speed input: 69104.69 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3009/3051 [23:46<00:19,  2.12it/s, est. speed input: 69104.79 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3010/3051 [23:47<00:19,  2.12it/s, est. speed input: 69104.89 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3011/3051 [23:47<00:18,  2.12it/s, est. speed input: 69104.96 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 3012/3051 [23:48<00:18,  2.12it/s, est. speed input: 69105.03 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3013/3051 [23:48<00:17,  2.11it/s, est. speed input: 69105.03 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3014/3051 [23:49<00:17,  2.11it/s, est. speed input: 69105.11 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3015/3051 [23:49<00:17,  2.11it/s, est. speed input: 69105.18 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3016/3051 [23:50<00:16,  2.11it/s, est. speed input: 69105.12 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3017/3051 [23:50<00:16,  2.11it/s, est. speed input: 69105.21 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3018/3051 [23:51<00:15,  2.11it/s, est. speed input: 69105.32 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3019/3051 [23:51<00:15,  2.12it/s, est. speed input: 69105.41 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3020/3051 [23:52<00:14,  2.11it/s, est. speed input: 69105.43 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3021/3051 [23:52<00:14,  2.11it/s, est. speed input: 69105.49 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3022/3051 [23:52<00:13,  2.11it/s, est. speed input: 69105.54 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3023/3051 [23:53<00:13,  2.11it/s, est. speed input: 69105.60 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3024/3051 [23:53<00:12,  2.11it/s, est. speed input: 69105.67 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3025/3051 [23:54<00:12,  2.12it/s, est. speed input: 69105.76 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3026/3051 [23:54<00:11,  2.11it/s, est. speed input: 69105.72 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3027/3051 [23:55<00:11,  2.11it/s, est. speed input: 69105.81 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3028/3051 [23:55<00:10,  2.11it/s, est. speed input: 69105.87 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3029/3051 [23:56<00:10,  2.12it/s, est. speed input: 69105.98 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3030/3051 [23:56<00:09,  2.12it/s, est. speed input: 69106.04 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3031/3051 [23:57<00:09,  2.12it/s, est. speed input: 69106.12 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3032/3051 [23:57<00:08,  2.11it/s, est. speed input: 69106.17 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3033/3051 [23:58<00:08,  2.11it/s, est. speed input: 69106.19 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3034/3051 [23:58<00:08,  2.11it/s, est. speed input: 69106.19 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 3035/3051 [23:59<00:07,  2.11it/s, est. speed input: 69106.27 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3036/3051 [23:59<00:07,  2.11it/s, est. speed input: 69106.36 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3037/3051 [24:00<00:06,  2.11it/s, est. speed input: 69106.41 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3038/3051 [24:00<00:06,  2.12it/s, est. speed input: 69106.54 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3039/3051 [24:00<00:05,  2.11it/s, est. speed input: 69106.51 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3040/3051 [24:01<00:05,  2.11it/s, est. speed input: 69106.59 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3041/3051 [24:01<00:04,  2.11it/s, est. speed input: 69106.59 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3042/3051 [24:02<00:04,  2.11it/s, est. speed input: 69106.53 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3043/3051 [24:02<00:03,  2.10it/s, est. speed input: 69106.28 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3044/3051 [24:03<00:03,  2.09it/s, est. speed input: 69105.59 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3045/3051 [24:03<00:02,  2.09it/s, est. speed input: 69105.63 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3046/3051 [24:04<00:02,  2.10it/s, est. speed input: 69105.73 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3047/3051 [24:04<00:01,  2.10it/s, est. speed input: 69105.78 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3048/3051 [24:05<00:01,  2.11it/s, est. speed input: 69105.85 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3049/3051 [24:05<00:00,  2.11it/s, est. speed input: 69105.92 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 3050/3051 [24:06<00:00,  2.11it/s, est. speed input: 69106.01 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 3051/3051 [24:06<00:00,  2.11it/s, est. speed input: 69106.12 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 3051/3051 [24:06<00:00,  2.11it/s, est. speed input: 69106.12 toks/s, output: 2.11 toks/s]
Saved activation counts to new_activations/activation.en.train.q2.5-ga.pt
Processed 99975168 tokens total

Loading IDs from ./ids/q2.5/id.ga.train.qwen2.5-0.5 (lang: ga)...
ID shape: torch.Size([65506066])
Processing 1999 sequences of length 32768
Running inference...

Processed prompts:   0%|          | 0/1999 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts:   0%|          | 1/1999 [00:00<16:29,  2.02it/s, est. speed input: 66178.58 toks/s, output: 2.02 toks/s]
Processed prompts:   0%|          | 2/1999 [00:00<16:39,  2.00it/s, est. speed input: 65578.01 toks/s, output: 2.00 toks/s]
Processed prompts:   0%|          | 3/1999 [00:01<16:11,  2.05it/s, est. speed input: 66897.07 toks/s, output: 2.04 toks/s]
Processed prompts:   0%|          | 4/1999 [00:01<15:58,  2.08it/s, est. speed input: 67553.17 toks/s, output: 2.06 toks/s]
Processed prompts:   0%|          | 5/1999 [00:02<15:52,  2.09it/s, est. speed input: 67905.66 toks/s, output: 2.07 toks/s]
Processed prompts:   0%|          | 6/1999 [00:02<15:46,  2.10it/s, est. speed input: 68189.28 toks/s, output: 2.08 toks/s]
Processed prompts:   0%|          | 7/1999 [00:03<15:42,  2.11it/s, est. speed input: 68417.75 toks/s, output: 2.09 toks/s]
Processed prompts:   0%|          | 8/1999 [00:03<15:39,  2.12it/s, est. speed input: 68584.39 toks/s, output: 2.09 toks/s]
Processed prompts:   0%|          | 9/1999 [00:04<15:37,  2.12it/s, est. speed input: 68721.16 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 10/1999 [00:04<15:36,  2.12it/s, est. speed input: 68828.26 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 11/1999 [00:05<15:34,  2.13it/s, est. speed input: 68917.80 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 12/1999 [00:05<15:34,  2.13it/s, est. speed input: 68975.83 toks/s, output: 2.10 toks/s]
Processed prompts:   1%|          | 13/1999 [00:06<15:33,  2.13it/s, est. speed input: 69039.54 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 14/1999 [00:06<15:32,  2.13it/s, est. speed input: 69096.73 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 15/1999 [00:07<15:32,  2.13it/s, est. speed input: 69139.73 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 16/1999 [00:07<15:31,  2.13it/s, est. speed input: 69178.90 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 17/1999 [00:08<15:31,  2.13it/s, est. speed input: 69210.97 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 18/1999 [00:08<15:30,  2.13it/s, est. speed input: 69244.31 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 19/1999 [00:08<15:29,  2.13it/s, est. speed input: 69272.77 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 20/1999 [00:09<15:29,  2.13it/s, est. speed input: 69293.75 toks/s, output: 2.11 toks/s]
Processed prompts:   1%|          | 21/1999 [00:09<15:29,  2.13it/s, est. speed input: 69315.11 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 22/1999 [00:10<15:28,  2.13it/s, est. speed input: 69333.19 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 23/1999 [00:10<15:28,  2.13it/s, est. speed input: 69355.53 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|          | 24/1999 [00:11<15:27,  2.13it/s, est. speed input: 69372.75 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 25/1999 [00:11<15:27,  2.13it/s, est. speed input: 69387.00 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 26/1999 [00:12<15:27,  2.13it/s, est. speed input: 69398.53 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 27/1999 [00:12<15:26,  2.13it/s, est. speed input: 69409.89 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 28/1999 [00:13<15:26,  2.13it/s, est. speed input: 69422.60 toks/s, output: 2.12 toks/s]
Processed prompts:   1%|▏         | 29/1999 [00:13<15:25,  2.13it/s, est. speed input: 69433.07 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 30/1999 [00:14<15:25,  2.13it/s, est. speed input: 69442.95 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 31/1999 [00:14<15:24,  2.13it/s, est. speed input: 69450.72 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 32/1999 [00:15<15:24,  2.13it/s, est. speed input: 69456.41 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 33/1999 [00:15<15:24,  2.13it/s, est. speed input: 69465.80 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 34/1999 [00:16<15:24,  2.13it/s, est. speed input: 69468.69 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 35/1999 [00:16<15:25,  2.12it/s, est. speed input: 69461.70 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 36/1999 [00:16<15:25,  2.12it/s, est. speed input: 69458.87 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 37/1999 [00:17<15:25,  2.12it/s, est. speed input: 69458.60 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 38/1999 [00:17<15:24,  2.12it/s, est. speed input: 69461.19 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 39/1999 [00:18<15:23,  2.12it/s, est. speed input: 69463.01 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 40/1999 [00:18<15:22,  2.12it/s, est. speed input: 69469.34 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 41/1999 [00:19<15:21,  2.12it/s, est. speed input: 69474.69 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 42/1999 [00:19<15:20,  2.13it/s, est. speed input: 69479.97 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 43/1999 [00:20<15:20,  2.13it/s, est. speed input: 69484.10 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 44/1999 [00:20<15:19,  2.13it/s, est. speed input: 69490.67 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 45/1999 [00:21<15:18,  2.13it/s, est. speed input: 69495.64 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 46/1999 [00:21<15:18,  2.13it/s, est. speed input: 69499.64 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 47/1999 [00:22<15:17,  2.13it/s, est. speed input: 69505.35 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 48/1999 [00:22<15:16,  2.13it/s, est. speed input: 69510.80 toks/s, output: 2.12 toks/s]
Processed prompts:   2%|▏         | 49/1999 [00:23<15:15,  2.13it/s, est. speed input: 69517.46 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 50/1999 [00:23<15:15,  2.13it/s, est. speed input: 69522.99 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 51/1999 [00:24<15:15,  2.13it/s, est. speed input: 69526.08 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 52/1999 [00:24<15:15,  2.13it/s, est. speed input: 69528.69 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 53/1999 [00:24<15:14,  2.13it/s, est. speed input: 69532.12 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 54/1999 [00:25<15:14,  2.13it/s, est. speed input: 69535.17 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 55/1999 [00:25<15:13,  2.13it/s, est. speed input: 69537.59 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 56/1999 [00:26<15:13,  2.13it/s, est. speed input: 69540.56 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 57/1999 [00:26<15:13,  2.13it/s, est. speed input: 69543.01 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 58/1999 [00:27<15:12,  2.13it/s, est. speed input: 69545.07 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 59/1999 [00:27<15:12,  2.13it/s, est. speed input: 69547.73 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 60/1999 [00:28<15:11,  2.13it/s, est. speed input: 69549.39 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 61/1999 [00:28<15:11,  2.13it/s, est. speed input: 69551.13 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 62/1999 [00:29<15:10,  2.13it/s, est. speed input: 69553.50 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 63/1999 [00:29<15:10,  2.13it/s, est. speed input: 69556.04 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 64/1999 [00:30<15:10,  2.12it/s, est. speed input: 69553.74 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 65/1999 [00:30<15:10,  2.12it/s, est. speed input: 69553.27 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 66/1999 [00:31<15:10,  2.12it/s, est. speed input: 69553.42 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 67/1999 [00:31<15:10,  2.12it/s, est. speed input: 69552.65 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 68/1999 [00:32<15:09,  2.12it/s, est. speed input: 69552.72 toks/s, output: 2.12 toks/s]
Processed prompts:   3%|▎         | 69/1999 [00:32<15:09,  2.12it/s, est. speed input: 69552.39 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▎         | 70/1999 [00:32<15:09,  2.12it/s, est. speed input: 69550.33 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▎         | 71/1999 [00:33<15:09,  2.12it/s, est. speed input: 69547.47 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▎         | 72/1999 [00:33<15:08,  2.12it/s, est. speed input: 69547.60 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▎         | 73/1999 [00:34<15:08,  2.12it/s, est. speed input: 69546.34 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▎         | 74/1999 [00:34<15:09,  2.12it/s, est. speed input: 69540.95 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 75/1999 [00:35<15:10,  2.11it/s, est. speed input: 69534.03 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 76/1999 [00:35<15:09,  2.11it/s, est. speed input: 69529.68 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 77/1999 [00:36<15:09,  2.11it/s, est. speed input: 69526.38 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 78/1999 [00:36<15:09,  2.11it/s, est. speed input: 69521.33 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 79/1999 [00:37<15:08,  2.11it/s, est. speed input: 69519.51 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 80/1999 [00:37<15:07,  2.11it/s, est. speed input: 69515.91 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 81/1999 [00:38<15:07,  2.11it/s, est. speed input: 69513.54 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 82/1999 [00:38<15:06,  2.11it/s, est. speed input: 69510.00 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 83/1999 [00:39<15:06,  2.11it/s, est. speed input: 69507.34 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 84/1999 [00:39<15:05,  2.12it/s, est. speed input: 69506.34 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 85/1999 [00:40<15:04,  2.12it/s, est. speed input: 69505.04 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 86/1999 [00:40<15:03,  2.12it/s, est. speed input: 69503.22 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 87/1999 [00:41<15:03,  2.12it/s, est. speed input: 69502.62 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 88/1999 [00:41<15:02,  2.12it/s, est. speed input: 69500.82 toks/s, output: 2.12 toks/s]
Processed prompts:   4%|▍         | 89/1999 [00:41<15:03,  2.11it/s, est. speed input: 69495.27 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 90/1999 [00:42<15:02,  2.11it/s, est. speed input: 69493.91 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 91/1999 [00:42<15:01,  2.12it/s, est. speed input: 69493.70 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 92/1999 [00:43<15:00,  2.12it/s, est. speed input: 69494.01 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 93/1999 [00:43<14:59,  2.12it/s, est. speed input: 69493.35 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 94/1999 [00:44<14:59,  2.12it/s, est. speed input: 69491.96 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 95/1999 [00:44<14:59,  2.12it/s, est. speed input: 69490.66 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 96/1999 [00:45<14:59,  2.12it/s, est. speed input: 69487.84 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 97/1999 [00:45<15:00,  2.11it/s, est. speed input: 69481.53 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 98/1999 [00:46<15:00,  2.11it/s, est. speed input: 69478.03 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▍         | 99/1999 [00:46<15:00,  2.11it/s, est. speed input: 69474.66 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 100/1999 [00:47<14:59,  2.11it/s, est. speed input: 69471.82 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 101/1999 [00:47<14:58,  2.11it/s, est. speed input: 69469.38 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 102/1999 [00:48<14:57,  2.11it/s, est. speed input: 69469.09 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 103/1999 [00:48<14:56,  2.11it/s, est. speed input: 69468.11 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 104/1999 [00:49<14:55,  2.12it/s, est. speed input: 69467.45 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 105/1999 [00:49<14:54,  2.12it/s, est. speed input: 69467.18 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 106/1999 [00:50<14:54,  2.12it/s, est. speed input: 69466.34 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 107/1999 [00:50<14:53,  2.12it/s, est. speed input: 69465.45 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 108/1999 [00:50<14:52,  2.12it/s, est. speed input: 69466.11 toks/s, output: 2.12 toks/s]
Processed prompts:   5%|▌         | 109/1999 [00:51<14:52,  2.12it/s, est. speed input: 69465.51 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 110/1999 [00:51<14:51,  2.12it/s, est. speed input: 69465.30 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 111/1999 [00:52<14:51,  2.12it/s, est. speed input: 69463.73 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 112/1999 [00:52<14:51,  2.12it/s, est. speed input: 69462.38 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 113/1999 [00:53<14:50,  2.12it/s, est. speed input: 69462.77 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 114/1999 [00:53<14:49,  2.12it/s, est. speed input: 69462.32 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 115/1999 [00:54<14:49,  2.12it/s, est. speed input: 69462.28 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 116/1999 [00:54<14:49,  2.12it/s, est. speed input: 69461.46 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 117/1999 [00:55<14:48,  2.12it/s, est. speed input: 69460.43 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 118/1999 [00:55<14:48,  2.12it/s, est. speed input: 69460.07 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 119/1999 [00:56<14:47,  2.12it/s, est. speed input: 69459.64 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 120/1999 [00:56<14:47,  2.12it/s, est. speed input: 69459.21 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 121/1999 [00:57<14:46,  2.12it/s, est. speed input: 69459.51 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 122/1999 [00:57<14:46,  2.12it/s, est. speed input: 69458.90 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 123/1999 [00:58<14:45,  2.12it/s, est. speed input: 69458.75 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▌         | 124/1999 [00:58<14:45,  2.12it/s, est. speed input: 69458.16 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 125/1999 [00:58<14:44,  2.12it/s, est. speed input: 69457.67 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 126/1999 [00:59<14:44,  2.12it/s, est. speed input: 69456.98 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 127/1999 [00:59<14:43,  2.12it/s, est. speed input: 69456.87 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 128/1999 [01:00<14:43,  2.12it/s, est. speed input: 69456.26 toks/s, output: 2.12 toks/s]
Processed prompts:   6%|▋         | 129/1999 [01:00<14:42,  2.12it/s, est. speed input: 69456.08 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 130/1999 [01:01<14:42,  2.12it/s, est. speed input: 69456.17 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 131/1999 [01:01<14:41,  2.12it/s, est. speed input: 69455.68 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 132/1999 [01:02<14:41,  2.12it/s, est. speed input: 69455.24 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 133/1999 [01:02<14:40,  2.12it/s, est. speed input: 69454.88 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 134/1999 [01:03<14:40,  2.12it/s, est. speed input: 69454.68 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 135/1999 [01:03<14:39,  2.12it/s, est. speed input: 69454.30 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 136/1999 [01:04<14:39,  2.12it/s, est. speed input: 69453.60 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 137/1999 [01:04<14:39,  2.12it/s, est. speed input: 69452.75 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 138/1999 [01:05<14:38,  2.12it/s, est. speed input: 69452.34 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 139/1999 [01:05<14:38,  2.12it/s, est. speed input: 69451.75 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 140/1999 [01:06<14:38,  2.12it/s, est. speed input: 69450.31 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 141/1999 [01:06<14:38,  2.12it/s, est. speed input: 69449.11 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 142/1999 [01:06<14:37,  2.12it/s, est. speed input: 69448.83 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 143/1999 [01:07<14:36,  2.12it/s, est. speed input: 69448.23 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 144/1999 [01:07<14:36,  2.12it/s, est. speed input: 69447.29 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 145/1999 [01:08<14:36,  2.12it/s, est. speed input: 69446.75 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 146/1999 [01:08<14:35,  2.12it/s, est. speed input: 69446.09 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 147/1999 [01:09<14:34,  2.12it/s, est. speed input: 69445.70 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 148/1999 [01:09<14:34,  2.12it/s, est. speed input: 69445.05 toks/s, output: 2.12 toks/s]
Processed prompts:   7%|▋         | 149/1999 [01:10<14:34,  2.12it/s, est. speed input: 69444.15 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 150/1999 [01:10<14:33,  2.12it/s, est. speed input: 69443.60 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 151/1999 [01:11<14:32,  2.12it/s, est. speed input: 69443.45 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 152/1999 [01:11<14:32,  2.12it/s, est. speed input: 69442.90 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 153/1999 [01:12<14:32,  2.12it/s, est. speed input: 69441.97 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 154/1999 [01:12<14:32,  2.12it/s, est. speed input: 69440.79 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 155/1999 [01:13<14:31,  2.12it/s, est. speed input: 69440.61 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 156/1999 [01:13<14:30,  2.12it/s, est. speed input: 69439.78 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 157/1999 [01:14<14:30,  2.12it/s, est. speed input: 69439.37 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 158/1999 [01:14<14:29,  2.12it/s, est. speed input: 69439.16 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 159/1999 [01:15<14:29,  2.12it/s, est. speed input: 69438.73 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 160/1999 [01:15<14:28,  2.12it/s, est. speed input: 69438.33 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 161/1999 [01:15<14:28,  2.12it/s, est. speed input: 69437.82 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 162/1999 [01:16<14:28,  2.12it/s, est. speed input: 69437.01 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 163/1999 [01:16<14:28,  2.11it/s, est. speed input: 69435.14 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 164/1999 [01:17<14:29,  2.11it/s, est. speed input: 69431.44 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 165/1999 [01:17<14:28,  2.11it/s, est. speed input: 69430.38 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 166/1999 [01:18<14:27,  2.11it/s, est. speed input: 69429.55 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 167/1999 [01:18<14:26,  2.11it/s, est. speed input: 69429.35 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 168/1999 [01:19<14:25,  2.11it/s, est. speed input: 69428.72 toks/s, output: 2.12 toks/s]
Processed prompts:   8%|▊         | 169/1999 [01:19<14:25,  2.11it/s, est. speed input: 69427.95 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 170/1999 [01:20<14:24,  2.11it/s, est. speed input: 69427.41 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 171/1999 [01:20<14:24,  2.11it/s, est. speed input: 69426.34 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 172/1999 [01:21<14:23,  2.11it/s, est. speed input: 69425.71 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 173/1999 [01:21<14:23,  2.11it/s, est. speed input: 69424.32 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▊         | 174/1999 [01:22<14:23,  2.11it/s, est. speed input: 69423.88 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 175/1999 [01:22<14:22,  2.11it/s, est. speed input: 69423.35 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 176/1999 [01:23<14:22,  2.11it/s, est. speed input: 69422.48 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 177/1999 [01:23<14:21,  2.12it/s, est. speed input: 69422.33 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 178/1999 [01:24<14:20,  2.12it/s, est. speed input: 69421.82 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 179/1999 [01:24<14:20,  2.12it/s, est. speed input: 69421.11 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 180/1999 [01:24<14:20,  2.12it/s, est. speed input: 69420.31 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 181/1999 [01:25<14:19,  2.12it/s, est. speed input: 69420.00 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 182/1999 [01:25<14:19,  2.11it/s, est. speed input: 69418.61 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 183/1999 [01:26<14:20,  2.11it/s, est. speed input: 69415.35 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 184/1999 [01:26<14:20,  2.11it/s, est. speed input: 69413.56 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 185/1999 [01:27<14:19,  2.11it/s, est. speed input: 69411.92 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 186/1999 [01:27<14:19,  2.11it/s, est. speed input: 69410.20 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 187/1999 [01:28<14:20,  2.11it/s, est. speed input: 69406.39 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 188/1999 [01:28<14:19,  2.11it/s, est. speed input: 69405.41 toks/s, output: 2.12 toks/s]
Processed prompts:   9%|▉         | 189/1999 [01:29<14:17,  2.11it/s, est. speed input: 69404.81 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 190/1999 [01:29<14:16,  2.11it/s, est. speed input: 69404.50 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 191/1999 [01:30<14:15,  2.11it/s, est. speed input: 69403.93 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 192/1999 [01:30<14:14,  2.11it/s, est. speed input: 69403.94 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 193/1999 [01:31<14:14,  2.11it/s, est. speed input: 69403.06 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 194/1999 [01:31<14:13,  2.11it/s, est. speed input: 69402.75 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 195/1999 [01:32<14:13,  2.11it/s, est. speed input: 69402.13 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 196/1999 [01:32<14:12,  2.12it/s, est. speed input: 69402.22 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 197/1999 [01:33<14:11,  2.12it/s, est. speed input: 69401.93 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 198/1999 [01:33<14:11,  2.12it/s, est. speed input: 69401.25 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|▉         | 199/1999 [01:33<14:10,  2.12it/s, est. speed input: 69401.14 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 200/1999 [01:34<14:10,  2.12it/s, est. speed input: 69401.11 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 201/1999 [01:34<14:09,  2.12it/s, est. speed input: 69400.96 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 202/1999 [01:35<14:09,  2.12it/s, est. speed input: 69400.72 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 203/1999 [01:35<14:08,  2.12it/s, est. speed input: 69400.27 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 204/1999 [01:36<14:07,  2.12it/s, est. speed input: 69400.33 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 205/1999 [01:36<14:07,  2.12it/s, est. speed input: 69400.08 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 206/1999 [01:37<14:07,  2.12it/s, est. speed input: 69399.10 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 207/1999 [01:37<14:07,  2.12it/s, est. speed input: 69398.69 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 208/1999 [01:38<14:06,  2.12it/s, est. speed input: 69398.41 toks/s, output: 2.12 toks/s]
Processed prompts:  10%|█         | 209/1999 [01:38<14:05,  2.12it/s, est. speed input: 69398.60 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 210/1999 [01:39<14:05,  2.12it/s, est. speed input: 69398.28 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 211/1999 [01:39<14:04,  2.12it/s, est. speed input: 69397.93 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 212/1999 [01:40<14:04,  2.12it/s, est. speed input: 69397.69 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 213/1999 [01:40<14:03,  2.12it/s, est. speed input: 69397.60 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 214/1999 [01:41<14:03,  2.12it/s, est. speed input: 69397.32 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 215/1999 [01:41<14:03,  2.12it/s, est. speed input: 69396.92 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 216/1999 [01:41<14:02,  2.12it/s, est. speed input: 69396.89 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 217/1999 [01:42<14:01,  2.12it/s, est. speed input: 69396.80 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 218/1999 [01:42<14:01,  2.12it/s, est. speed input: 69396.70 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 219/1999 [01:43<14:01,  2.12it/s, est. speed input: 69395.92 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 220/1999 [01:43<14:00,  2.12it/s, est. speed input: 69395.60 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 221/1999 [01:44<13:59,  2.12it/s, est. speed input: 69395.91 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 222/1999 [01:44<13:59,  2.12it/s, est. speed input: 69395.58 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 223/1999 [01:45<13:59,  2.12it/s, est. speed input: 69395.35 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█         | 224/1999 [01:45<13:58,  2.12it/s, est. speed input: 69395.06 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 225/1999 [01:46<13:58,  2.12it/s, est. speed input: 69394.91 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 226/1999 [01:46<13:57,  2.12it/s, est. speed input: 69395.04 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 227/1999 [01:47<13:57,  2.11it/s, est. speed input: 69393.77 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 228/1999 [01:47<13:57,  2.11it/s, est. speed input: 69393.36 toks/s, output: 2.12 toks/s]
Processed prompts:  11%|█▏        | 229/1999 [01:48<13:56,  2.12it/s, est. speed input: 69393.15 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 230/1999 [01:48<13:56,  2.12it/s, est. speed input: 69392.74 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 231/1999 [01:49<13:55,  2.12it/s, est. speed input: 69392.37 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 232/1999 [01:49<13:55,  2.12it/s, est. speed input: 69392.17 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 233/1999 [01:50<13:54,  2.12it/s, est. speed input: 69391.84 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 234/1999 [01:50<13:54,  2.12it/s, est. speed input: 69391.69 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 235/1999 [01:50<13:53,  2.12it/s, est. speed input: 69391.45 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 236/1999 [01:51<13:53,  2.11it/s, est. speed input: 69390.55 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 237/1999 [01:51<13:53,  2.11it/s, est. speed input: 69389.44 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 238/1999 [01:52<13:52,  2.11it/s, est. speed input: 69389.46 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 239/1999 [01:52<13:51,  2.12it/s, est. speed input: 69389.51 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 240/1999 [01:53<13:51,  2.12it/s, est. speed input: 69389.39 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 241/1999 [01:53<13:50,  2.12it/s, est. speed input: 69389.62 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 242/1999 [01:54<13:51,  2.11it/s, est. speed input: 69387.98 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 243/1999 [01:54<13:51,  2.11it/s, est. speed input: 69386.85 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 244/1999 [01:55<13:50,  2.11it/s, est. speed input: 69386.82 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 245/1999 [01:55<13:49,  2.12it/s, est. speed input: 69386.89 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 246/1999 [01:56<13:49,  2.11it/s, est. speed input: 69386.33 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 247/1999 [01:56<13:49,  2.11it/s, est. speed input: 69384.93 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 248/1999 [01:57<13:48,  2.11it/s, est. speed input: 69384.82 toks/s, output: 2.12 toks/s]
Processed prompts:  12%|█▏        | 249/1999 [01:57<13:47,  2.11it/s, est. speed input: 69384.85 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 250/1999 [01:58<13:47,  2.11it/s, est. speed input: 69384.50 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 251/1999 [01:58<13:46,  2.11it/s, est. speed input: 69384.20 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 252/1999 [01:59<13:45,  2.12it/s, est. speed input: 69384.02 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 253/1999 [01:59<13:45,  2.12it/s, est. speed input: 69384.11 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 254/1999 [01:59<13:44,  2.12it/s, est. speed input: 69383.66 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 255/1999 [02:00<13:44,  2.12it/s, est. speed input: 69383.49 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 256/1999 [02:00<13:43,  2.12it/s, est. speed input: 69383.13 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 257/1999 [02:01<13:43,  2.12it/s, est. speed input: 69382.78 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 258/1999 [02:01<13:43,  2.11it/s, est. speed input: 69382.38 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 259/1999 [02:02<13:42,  2.12it/s, est. speed input: 69382.58 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 260/1999 [02:02<13:41,  2.12it/s, est. speed input: 69382.42 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 261/1999 [02:03<13:41,  2.12it/s, est. speed input: 69382.18 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 262/1999 [02:03<13:40,  2.12it/s, est. speed input: 69382.04 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 263/1999 [02:04<13:41,  2.11it/s, est. speed input: 69381.09 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 264/1999 [02:04<13:40,  2.11it/s, est. speed input: 69381.02 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 265/1999 [02:05<13:39,  2.12it/s, est. speed input: 69381.03 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 266/1999 [02:05<13:38,  2.12it/s, est. speed input: 69380.98 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 267/1999 [02:06<13:38,  2.12it/s, est. speed input: 69380.82 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 268/1999 [02:06<13:37,  2.12it/s, est. speed input: 69380.78 toks/s, output: 2.12 toks/s]
Processed prompts:  13%|█▎        | 269/1999 [02:07<13:37,  2.12it/s, est. speed input: 69380.56 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 270/1999 [02:07<13:36,  2.12it/s, est. speed input: 69380.68 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 271/1999 [02:08<13:52,  2.07it/s, est. speed input: 69363.30 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 272/1999 [02:08<13:47,  2.09it/s, est. speed input: 69363.60 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 273/1999 [02:08<13:43,  2.10it/s, est. speed input: 69363.60 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▎        | 274/1999 [02:09<13:40,  2.10it/s, est. speed input: 69363.61 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 275/1999 [02:09<13:38,  2.11it/s, est. speed input: 69363.67 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 276/1999 [02:10<13:36,  2.11it/s, est. speed input: 69363.45 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 277/1999 [02:10<13:35,  2.11it/s, est. speed input: 69363.21 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 278/1999 [02:11<13:34,  2.11it/s, est. speed input: 69363.31 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 279/1999 [02:11<13:33,  2.11it/s, est. speed input: 69363.09 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 280/1999 [02:12<13:32,  2.12it/s, est. speed input: 69363.34 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 281/1999 [02:12<13:31,  2.12it/s, est. speed input: 69363.47 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 282/1999 [02:13<13:31,  2.12it/s, est. speed input: 69363.58 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 283/1999 [02:13<13:31,  2.12it/s, est. speed input: 69363.19 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 284/1999 [02:14<13:30,  2.12it/s, est. speed input: 69363.25 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 285/1999 [02:14<13:30,  2.12it/s, est. speed input: 69362.94 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 286/1999 [02:15<13:29,  2.12it/s, est. speed input: 69363.00 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 287/1999 [02:15<13:29,  2.12it/s, est. speed input: 69362.60 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 288/1999 [02:16<13:28,  2.12it/s, est. speed input: 69362.36 toks/s, output: 2.12 toks/s]
Processed prompts:  14%|█▍        | 289/1999 [02:16<13:28,  2.12it/s, est. speed input: 69362.32 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 290/1999 [02:17<13:27,  2.12it/s, est. speed input: 69362.12 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 291/1999 [02:17<13:27,  2.12it/s, est. speed input: 69362.05 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 292/1999 [02:17<13:26,  2.12it/s, est. speed input: 69362.13 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 293/1999 [02:18<13:26,  2.12it/s, est. speed input: 69362.01 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 294/1999 [02:18<13:27,  2.11it/s, est. speed input: 69360.56 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 295/1999 [02:19<13:26,  2.11it/s, est. speed input: 69360.43 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 296/1999 [02:19<13:25,  2.11it/s, est. speed input: 69360.15 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 297/1999 [02:20<13:25,  2.11it/s, est. speed input: 69359.85 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 298/1999 [02:20<13:24,  2.11it/s, est. speed input: 69359.30 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▍        | 299/1999 [02:21<13:24,  2.11it/s, est. speed input: 69359.32 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 300/1999 [02:21<13:23,  2.11it/s, est. speed input: 69359.27 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 301/1999 [02:22<13:22,  2.12it/s, est. speed input: 69359.24 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 302/1999 [02:22<13:22,  2.12it/s, est. speed input: 69359.18 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 303/1999 [02:23<13:21,  2.12it/s, est. speed input: 69359.23 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 304/1999 [02:23<13:21,  2.12it/s, est. speed input: 69359.14 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 305/1999 [02:24<13:20,  2.12it/s, est. speed input: 69359.16 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 306/1999 [02:24<13:20,  2.12it/s, est. speed input: 69358.98 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 307/1999 [02:25<13:19,  2.12it/s, est. speed input: 69358.86 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 308/1999 [02:25<13:19,  2.12it/s, est. speed input: 69358.73 toks/s, output: 2.12 toks/s]
Processed prompts:  15%|█▌        | 309/1999 [02:25<13:18,  2.12it/s, est. speed input: 69358.69 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 310/1999 [02:26<13:18,  2.12it/s, est. speed input: 69358.63 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 311/1999 [02:26<13:18,  2.11it/s, est. speed input: 69358.16 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 312/1999 [02:27<13:17,  2.12it/s, est. speed input: 69358.23 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 313/1999 [02:27<13:16,  2.12it/s, est. speed input: 69358.24 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 314/1999 [02:28<13:16,  2.12it/s, est. speed input: 69358.14 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 315/1999 [02:28<13:15,  2.12it/s, est. speed input: 69358.16 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 316/1999 [02:29<13:17,  2.11it/s, est. speed input: 69356.01 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 317/1999 [02:29<13:16,  2.11it/s, est. speed input: 69356.11 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 318/1999 [02:30<13:15,  2.11it/s, est. speed input: 69356.19 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 319/1999 [02:30<13:14,  2.11it/s, est. speed input: 69356.22 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 320/1999 [02:31<13:13,  2.12it/s, est. speed input: 69356.17 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 321/1999 [02:31<13:13,  2.12it/s, est. speed input: 69356.17 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 322/1999 [02:32<13:12,  2.12it/s, est. speed input: 69356.09 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 323/1999 [02:32<13:12,  2.12it/s, est. speed input: 69355.78 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▌        | 324/1999 [02:33<13:14,  2.11it/s, est. speed input: 69353.22 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 325/1999 [02:33<13:13,  2.11it/s, est. speed input: 69353.02 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 326/1999 [02:34<13:12,  2.11it/s, est. speed input: 69352.96 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 327/1999 [02:34<13:11,  2.11it/s, est. speed input: 69352.96 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 328/1999 [02:34<13:10,  2.11it/s, est. speed input: 69352.76 toks/s, output: 2.12 toks/s]
Processed prompts:  16%|█▋        | 329/1999 [02:35<13:09,  2.12it/s, est. speed input: 69353.02 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 330/1999 [02:35<13:09,  2.11it/s, est. speed input: 69352.58 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 331/1999 [02:36<13:08,  2.12it/s, est. speed input: 69352.75 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 332/1999 [02:36<13:07,  2.12it/s, est. speed input: 69352.74 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 333/1999 [02:37<13:07,  2.12it/s, est. speed input: 69352.96 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 334/1999 [02:37<13:06,  2.12it/s, est. speed input: 69353.19 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 335/1999 [02:38<13:06,  2.12it/s, est. speed input: 69353.17 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 336/1999 [02:38<13:06,  2.12it/s, est. speed input: 69352.64 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 337/1999 [02:39<13:05,  2.11it/s, est. speed input: 69352.43 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 338/1999 [02:39<13:04,  2.12it/s, est. speed input: 69352.60 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 339/1999 [02:40<13:06,  2.11it/s, est. speed input: 69351.04 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 340/1999 [02:40<13:05,  2.11it/s, est. speed input: 69351.18 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 341/1999 [02:41<13:04,  2.11it/s, est. speed input: 69351.26 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 342/1999 [02:41<13:04,  2.11it/s, est. speed input: 69349.94 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 343/1999 [02:42<13:04,  2.11it/s, est. speed input: 69349.76 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 344/1999 [02:42<13:03,  2.11it/s, est. speed input: 69349.49 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 345/1999 [02:43<13:02,  2.11it/s, est. speed input: 69349.48 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 346/1999 [02:43<13:01,  2.11it/s, est. speed input: 69349.65 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 347/1999 [02:43<13:00,  2.12it/s, est. speed input: 69349.65 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 348/1999 [02:44<13:00,  2.12it/s, est. speed input: 69349.63 toks/s, output: 2.12 toks/s]
Processed prompts:  17%|█▋        | 349/1999 [02:44<12:59,  2.12it/s, est. speed input: 69349.74 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 350/1999 [02:45<12:59,  2.12it/s, est. speed input: 69349.74 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 351/1999 [02:45<12:58,  2.12it/s, est. speed input: 69349.73 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 352/1999 [02:46<12:58,  2.11it/s, est. speed input: 69349.32 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 353/1999 [02:46<12:58,  2.12it/s, est. speed input: 69349.32 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 354/1999 [02:47<12:57,  2.12it/s, est. speed input: 69349.30 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 355/1999 [02:47<12:57,  2.12it/s, est. speed input: 69349.27 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 356/1999 [02:48<12:56,  2.12it/s, est. speed input: 69349.25 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 357/1999 [02:48<12:55,  2.12it/s, est. speed input: 69349.25 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 358/1999 [02:49<12:55,  2.12it/s, est. speed input: 69349.12 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 359/1999 [02:49<12:55,  2.12it/s, est. speed input: 69349.18 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 360/1999 [02:50<12:54,  2.12it/s, est. speed input: 69349.04 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 361/1999 [02:50<12:53,  2.12it/s, est. speed input: 69349.19 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 362/1999 [02:51<12:54,  2.11it/s, est. speed input: 69348.76 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 363/1999 [02:51<12:53,  2.12it/s, est. speed input: 69348.68 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 364/1999 [02:51<12:52,  2.12it/s, est. speed input: 69348.60 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 365/1999 [02:52<12:52,  2.12it/s, est. speed input: 69348.82 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 366/1999 [02:52<12:52,  2.12it/s, est. speed input: 69348.50 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 367/1999 [02:53<12:51,  2.12it/s, est. speed input: 69348.59 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 368/1999 [02:53<12:50,  2.12it/s, est. speed input: 69348.59 toks/s, output: 2.12 toks/s]
Processed prompts:  18%|█▊        | 369/1999 [02:54<12:50,  2.12it/s, est. speed input: 69348.40 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 370/1999 [02:54<12:49,  2.12it/s, est. speed input: 69348.43 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 371/1999 [02:55<12:49,  2.12it/s, est. speed input: 69348.60 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 372/1999 [02:55<12:48,  2.12it/s, est. speed input: 69348.68 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 373/1999 [02:56<12:48,  2.12it/s, est. speed input: 69348.78 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▊        | 374/1999 [02:56<12:47,  2.12it/s, est. speed input: 69348.96 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 375/1999 [02:57<12:49,  2.11it/s, est. speed input: 69347.11 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 376/1999 [02:57<12:48,  2.11it/s, est. speed input: 69346.86 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 377/1999 [02:58<12:47,  2.11it/s, est. speed input: 69346.81 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 378/1999 [02:58<12:46,  2.11it/s, est. speed input: 69346.71 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 379/1999 [02:59<12:46,  2.11it/s, est. speed input: 69346.80 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 380/1999 [02:59<12:44,  2.12it/s, est. speed input: 69347.25 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 381/1999 [03:00<12:44,  2.12it/s, est. speed input: 69347.13 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 382/1999 [03:00<12:44,  2.11it/s, est. speed input: 69346.68 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 383/1999 [03:00<12:43,  2.12it/s, est. speed input: 69346.74 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 384/1999 [03:01<12:43,  2.12it/s, est. speed input: 69346.59 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 385/1999 [03:01<12:42,  2.12it/s, est. speed input: 69346.64 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 386/1999 [03:02<12:42,  2.12it/s, est. speed input: 69346.76 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 387/1999 [03:02<12:41,  2.12it/s, est. speed input: 69346.80 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 388/1999 [03:03<12:41,  2.12it/s, est. speed input: 69346.49 toks/s, output: 2.12 toks/s]
Processed prompts:  19%|█▉        | 389/1999 [03:03<12:41,  2.11it/s, est. speed input: 69346.04 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 390/1999 [03:04<12:40,  2.11it/s, est. speed input: 69346.03 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 391/1999 [03:04<12:40,  2.12it/s, est. speed input: 69346.05 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 392/1999 [03:05<12:39,  2.12it/s, est. speed input: 69346.04 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 393/1999 [03:05<12:38,  2.12it/s, est. speed input: 69346.23 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 394/1999 [03:06<12:38,  2.12it/s, est. speed input: 69346.24 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 395/1999 [03:06<12:38,  2.12it/s, est. speed input: 69346.12 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 396/1999 [03:07<12:37,  2.12it/s, est. speed input: 69345.94 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 397/1999 [03:07<12:37,  2.12it/s, est. speed input: 69346.02 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 398/1999 [03:08<12:36,  2.12it/s, est. speed input: 69346.08 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|█▉        | 399/1999 [03:08<12:35,  2.12it/s, est. speed input: 69346.30 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 400/1999 [03:09<12:53,  2.07it/s, est. speed input: 69332.16 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 401/1999 [03:09<12:47,  2.08it/s, est. speed input: 69332.66 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 402/1999 [03:09<12:42,  2.09it/s, est. speed input: 69333.23 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 403/1999 [03:10<12:39,  2.10it/s, est. speed input: 69333.62 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 404/1999 [03:10<12:36,  2.11it/s, est. speed input: 69333.99 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 405/1999 [03:11<12:34,  2.11it/s, est. speed input: 69334.38 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 406/1999 [03:11<12:32,  2.12it/s, est. speed input: 69335.10 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 407/1999 [03:12<12:32,  2.11it/s, est. speed input: 69334.72 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 408/1999 [03:12<12:32,  2.12it/s, est. speed input: 69334.80 toks/s, output: 2.12 toks/s]
Processed prompts:  20%|██        | 409/1999 [03:13<12:30,  2.12it/s, est. speed input: 69335.33 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 410/1999 [03:13<12:29,  2.12it/s, est. speed input: 69335.81 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 411/1999 [03:14<12:30,  2.12it/s, est. speed input: 69335.33 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 412/1999 [03:14<12:29,  2.12it/s, est. speed input: 69335.57 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 413/1999 [03:15<12:30,  2.11it/s, est. speed input: 69334.68 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 414/1999 [03:15<12:29,  2.11it/s, est. speed input: 69334.53 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 415/1999 [03:16<12:28,  2.12it/s, est. speed input: 69335.29 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 416/1999 [03:16<12:28,  2.12it/s, est. speed input: 69334.97 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 417/1999 [03:17<12:27,  2.12it/s, est. speed input: 69334.96 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 418/1999 [03:17<12:26,  2.12it/s, est. speed input: 69335.44 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 419/1999 [03:18<12:25,  2.12it/s, est. speed input: 69336.12 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 420/1999 [03:18<12:26,  2.12it/s, est. speed input: 69335.25 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 421/1999 [03:18<12:25,  2.12it/s, est. speed input: 69335.52 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 422/1999 [03:19<12:24,  2.12it/s, est. speed input: 69336.04 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 423/1999 [03:19<12:24,  2.12it/s, est. speed input: 69336.07 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██        | 424/1999 [03:20<12:23,  2.12it/s, est. speed input: 69336.37 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 425/1999 [03:20<12:22,  2.12it/s, est. speed input: 69336.66 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 426/1999 [03:21<12:21,  2.12it/s, est. speed input: 69337.28 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 427/1999 [03:21<12:21,  2.12it/s, est. speed input: 69337.61 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 428/1999 [03:22<12:20,  2.12it/s, est. speed input: 69338.19 toks/s, output: 2.12 toks/s]
Processed prompts:  21%|██▏       | 429/1999 [03:22<12:19,  2.12it/s, est. speed input: 69338.82 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 430/1999 [03:23<12:19,  2.12it/s, est. speed input: 69339.02 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 431/1999 [03:23<12:20,  2.12it/s, est. speed input: 69338.76 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 432/1999 [03:24<12:19,  2.12it/s, est. speed input: 69339.02 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 433/1999 [03:24<12:18,  2.12it/s, est. speed input: 69339.70 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 434/1999 [03:25<12:17,  2.12it/s, est. speed input: 69340.13 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 435/1999 [03:25<12:18,  2.12it/s, est. speed input: 69339.93 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 436/1999 [03:26<12:18,  2.12it/s, est. speed input: 69339.37 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 437/1999 [03:26<12:18,  2.12it/s, est. speed input: 69339.20 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 438/1999 [03:26<12:17,  2.12it/s, est. speed input: 69339.27 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 439/1999 [03:27<12:17,  2.11it/s, est. speed input: 69339.01 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 440/1999 [03:27<12:16,  2.12it/s, est. speed input: 69339.21 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 441/1999 [03:28<12:16,  2.12it/s, est. speed input: 69339.02 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 442/1999 [03:28<12:15,  2.12it/s, est. speed input: 69339.43 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 443/1999 [03:29<12:15,  2.12it/s, est. speed input: 69339.11 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 444/1999 [03:29<12:15,  2.12it/s, est. speed input: 69339.00 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 445/1999 [03:30<12:14,  2.12it/s, est. speed input: 69339.21 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 446/1999 [03:30<12:12,  2.12it/s, est. speed input: 69339.93 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 447/1999 [03:31<12:12,  2.12it/s, est. speed input: 69340.28 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 448/1999 [03:31<12:11,  2.12it/s, est. speed input: 69340.77 toks/s, output: 2.12 toks/s]
Processed prompts:  22%|██▏       | 449/1999 [03:32<12:11,  2.12it/s, est. speed input: 69340.65 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 450/1999 [03:32<12:11,  2.12it/s, est. speed input: 69340.67 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 451/1999 [03:33<12:10,  2.12it/s, est. speed input: 69341.02 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 452/1999 [03:33<12:09,  2.12it/s, est. speed input: 69341.65 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 453/1999 [03:34<12:08,  2.12it/s, est. speed input: 69342.06 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 454/1999 [03:34<12:08,  2.12it/s, est. speed input: 69342.16 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 455/1999 [03:35<12:08,  2.12it/s, est. speed input: 69342.39 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 456/1999 [03:35<12:09,  2.12it/s, est. speed input: 69341.76 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 457/1999 [03:35<12:09,  2.11it/s, est. speed input: 69341.13 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 458/1999 [03:36<12:08,  2.12it/s, est. speed input: 69341.64 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 459/1999 [03:36<12:07,  2.12it/s, est. speed input: 69341.71 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 460/1999 [03:37<12:07,  2.12it/s, est. speed input: 69341.81 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 461/1999 [03:37<12:06,  2.12it/s, est. speed input: 69342.13 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 462/1999 [03:38<12:05,  2.12it/s, est. speed input: 69342.16 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 463/1999 [03:38<12:05,  2.12it/s, est. speed input: 69342.19 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 464/1999 [03:39<12:04,  2.12it/s, est. speed input: 69342.47 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 465/1999 [03:39<12:07,  2.11it/s, est. speed input: 69340.14 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 466/1999 [03:40<12:05,  2.11it/s, est. speed input: 69340.48 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 467/1999 [03:40<12:04,  2.11it/s, est. speed input: 69340.64 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 468/1999 [03:41<12:03,  2.12it/s, est. speed input: 69341.16 toks/s, output: 2.12 toks/s]
Processed prompts:  23%|██▎       | 469/1999 [03:41<12:04,  2.11it/s, est. speed input: 69340.07 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 470/1999 [03:42<12:03,  2.11it/s, est. speed input: 69340.39 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 471/1999 [03:42<12:01,  2.12it/s, est. speed input: 69340.97 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 472/1999 [03:43<12:01,  2.12it/s, est. speed input: 69341.12 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 473/1999 [03:43<12:00,  2.12it/s, est. speed input: 69341.24 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▎       | 474/1999 [03:43<11:59,  2.12it/s, est. speed input: 69341.63 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 475/1999 [03:44<12:00,  2.12it/s, est. speed input: 69341.15 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 476/1999 [03:44<11:59,  2.12it/s, est. speed input: 69341.24 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 477/1999 [03:45<11:58,  2.12it/s, est. speed input: 69341.69 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 478/1999 [03:45<12:00,  2.11it/s, est. speed input: 69340.46 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 479/1999 [03:46<11:59,  2.11it/s, est. speed input: 69340.17 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 480/1999 [03:46<11:58,  2.12it/s, est. speed input: 69340.67 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 481/1999 [03:47<11:56,  2.12it/s, est. speed input: 69341.03 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 482/1999 [03:47<11:56,  2.12it/s, est. speed input: 69341.14 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 483/1999 [03:48<11:57,  2.11it/s, est. speed input: 69340.22 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 484/1999 [03:48<11:56,  2.12it/s, est. speed input: 69340.66 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 485/1999 [03:49<11:54,  2.12it/s, est. speed input: 69341.23 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 486/1999 [03:49<11:53,  2.12it/s, est. speed input: 69341.81 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 487/1999 [03:50<11:52,  2.12it/s, est. speed input: 69342.24 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 488/1999 [03:50<11:51,  2.12it/s, est. speed input: 69342.84 toks/s, output: 2.12 toks/s]
Processed prompts:  24%|██▍       | 489/1999 [03:51<11:52,  2.12it/s, est. speed input: 69342.54 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 490/1999 [03:51<11:52,  2.12it/s, est. speed input: 69342.69 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 491/1999 [03:52<11:51,  2.12it/s, est. speed input: 69343.15 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 492/1999 [03:52<11:51,  2.12it/s, est. speed input: 69343.23 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 493/1999 [03:52<11:50,  2.12it/s, est. speed input: 69343.48 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 494/1999 [03:53<11:49,  2.12it/s, est. speed input: 69344.06 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 495/1999 [03:53<11:49,  2.12it/s, est. speed input: 69344.07 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 496/1999 [03:54<11:50,  2.12it/s, est. speed input: 69343.55 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 497/1999 [03:54<11:49,  2.12it/s, est. speed input: 69343.89 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 498/1999 [03:55<11:48,  2.12it/s, est. speed input: 69344.27 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▍       | 499/1999 [03:55<11:47,  2.12it/s, est. speed input: 69344.87 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 500/1999 [03:56<11:46,  2.12it/s, est. speed input: 69345.04 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 501/1999 [03:56<11:47,  2.12it/s, est. speed input: 69344.80 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 502/1999 [03:57<11:47,  2.12it/s, est. speed input: 69344.60 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 503/1999 [03:57<11:46,  2.12it/s, est. speed input: 69344.59 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 504/1999 [03:58<11:45,  2.12it/s, est. speed input: 69344.85 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 505/1999 [03:58<11:48,  2.11it/s, est. speed input: 69343.27 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 506/1999 [03:59<11:46,  2.11it/s, est. speed input: 69343.67 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 507/1999 [03:59<11:45,  2.12it/s, est. speed input: 69344.07 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 508/1999 [04:00<11:44,  2.12it/s, est. speed input: 69344.31 toks/s, output: 2.12 toks/s]
Processed prompts:  25%|██▌       | 509/1999 [04:00<11:43,  2.12it/s, est. speed input: 69344.61 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 510/1999 [04:00<11:42,  2.12it/s, est. speed input: 69344.89 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 511/1999 [04:01<11:41,  2.12it/s, est. speed input: 69345.41 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 512/1999 [04:01<11:41,  2.12it/s, est. speed input: 69345.52 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 513/1999 [04:02<11:41,  2.12it/s, est. speed input: 69345.73 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 514/1999 [04:02<11:40,  2.12it/s, est. speed input: 69345.94 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 515/1999 [04:03<11:39,  2.12it/s, est. speed input: 69346.26 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 516/1999 [04:03<11:39,  2.12it/s, est. speed input: 69346.40 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 517/1999 [04:04<11:38,  2.12it/s, est. speed input: 69346.79 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 518/1999 [04:04<11:39,  2.12it/s, est. speed input: 69346.63 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 519/1999 [04:05<11:38,  2.12it/s, est. speed input: 69346.71 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 520/1999 [04:05<11:38,  2.12it/s, est. speed input: 69346.91 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 521/1999 [04:06<11:38,  2.11it/s, est. speed input: 69346.26 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 522/1999 [04:06<11:38,  2.12it/s, est. speed input: 69346.25 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 523/1999 [04:07<11:37,  2.12it/s, est. speed input: 69346.50 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▌       | 524/1999 [04:07<11:36,  2.12it/s, est. speed input: 69346.71 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 525/1999 [04:08<11:35,  2.12it/s, est. speed input: 69346.99 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 526/1999 [04:08<11:35,  2.12it/s, est. speed input: 69346.91 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 527/1999 [04:09<11:35,  2.12it/s, est. speed input: 69346.95 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 528/1999 [04:09<11:37,  2.11it/s, est. speed input: 69345.38 toks/s, output: 2.12 toks/s]
Processed prompts:  26%|██▋       | 529/1999 [04:09<11:36,  2.11it/s, est. speed input: 69345.43 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 530/1999 [04:10<11:34,  2.12it/s, est. speed input: 69345.84 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 531/1999 [04:10<11:33,  2.12it/s, est. speed input: 69346.03 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 532/1999 [04:11<11:33,  2.12it/s, est. speed input: 69346.13 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 533/1999 [04:11<11:32,  2.12it/s, est. speed input: 69346.28 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 534/1999 [04:12<11:31,  2.12it/s, est. speed input: 69346.61 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 535/1999 [04:12<11:31,  2.12it/s, est. speed input: 69346.40 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 536/1999 [04:13<11:30,  2.12it/s, est. speed input: 69346.67 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 537/1999 [04:13<11:29,  2.12it/s, est. speed input: 69347.01 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 538/1999 [04:14<11:29,  2.12it/s, est. speed input: 69347.30 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 539/1999 [04:14<11:28,  2.12it/s, est. speed input: 69347.52 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 540/1999 [04:15<12:07,  2.01it/s, est. speed input: 69323.65 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 541/1999 [04:15<11:54,  2.04it/s, est. speed input: 69324.11 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 542/1999 [04:16<11:46,  2.06it/s, est. speed input: 69324.23 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 543/1999 [04:16<11:40,  2.08it/s, est. speed input: 69324.09 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 544/1999 [04:17<11:36,  2.09it/s, est. speed input: 69324.17 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 545/1999 [04:17<11:33,  2.10it/s, est. speed input: 69324.23 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 546/1999 [04:18<11:30,  2.10it/s, est. speed input: 69324.25 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 547/1999 [04:18<11:29,  2.11it/s, est. speed input: 69324.07 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 548/1999 [04:19<11:27,  2.11it/s, est. speed input: 69324.25 toks/s, output: 2.12 toks/s]
Processed prompts:  27%|██▋       | 549/1999 [04:19<11:26,  2.11it/s, est. speed input: 69324.35 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 550/1999 [04:19<11:25,  2.11it/s, est. speed input: 69324.36 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 551/1999 [04:20<11:25,  2.11it/s, est. speed input: 69324.36 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 552/1999 [04:20<11:24,  2.11it/s, est. speed input: 69324.24 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 553/1999 [04:21<11:24,  2.11it/s, est. speed input: 69323.87 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 554/1999 [04:21<11:23,  2.11it/s, est. speed input: 69323.76 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 555/1999 [04:22<11:23,  2.11it/s, est. speed input: 69323.85 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 556/1999 [04:22<11:22,  2.11it/s, est. speed input: 69323.83 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 557/1999 [04:23<11:21,  2.11it/s, est. speed input: 69323.83 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 558/1999 [04:23<11:21,  2.12it/s, est. speed input: 69323.96 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 559/1999 [04:24<11:20,  2.12it/s, est. speed input: 69323.84 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 560/1999 [04:24<11:20,  2.11it/s, est. speed input: 69323.65 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 561/1999 [04:25<11:20,  2.11it/s, est. speed input: 69323.64 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 562/1999 [04:25<11:19,  2.11it/s, est. speed input: 69323.57 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 563/1999 [04:26<11:18,  2.12it/s, est. speed input: 69323.65 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 564/1999 [04:26<11:18,  2.12it/s, est. speed input: 69323.69 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 565/1999 [04:27<11:18,  2.11it/s, est. speed input: 69323.40 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 566/1999 [04:27<11:17,  2.11it/s, est. speed input: 69323.38 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 567/1999 [04:28<11:17,  2.11it/s, est. speed input: 69323.32 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 568/1999 [04:28<11:16,  2.12it/s, est. speed input: 69323.46 toks/s, output: 2.12 toks/s]
Processed prompts:  28%|██▊       | 569/1999 [04:28<11:16,  2.11it/s, est. speed input: 69323.22 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 570/1999 [04:29<11:15,  2.11it/s, est. speed input: 69323.14 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 571/1999 [04:29<11:15,  2.12it/s, est. speed input: 69323.24 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 572/1999 [04:30<11:14,  2.12it/s, est. speed input: 69323.41 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 573/1999 [04:30<11:13,  2.12it/s, est. speed input: 69323.49 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▊       | 574/1999 [04:31<11:13,  2.12it/s, est. speed input: 69323.66 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 575/1999 [04:31<11:13,  2.12it/s, est. speed input: 69323.50 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 576/1999 [04:32<11:12,  2.12it/s, est. speed input: 69323.49 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 577/1999 [04:32<11:12,  2.12it/s, est. speed input: 69323.56 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 578/1999 [04:33<11:11,  2.12it/s, est. speed input: 69323.52 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 579/1999 [04:33<11:10,  2.12it/s, est. speed input: 69323.64 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 580/1999 [04:34<11:10,  2.12it/s, est. speed input: 69323.61 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 581/1999 [04:34<11:10,  2.12it/s, est. speed input: 69323.66 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 582/1999 [04:35<11:10,  2.11it/s, est. speed input: 69323.40 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 583/1999 [04:35<11:09,  2.12it/s, est. speed input: 69323.53 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 584/1999 [04:36<11:08,  2.12it/s, est. speed input: 69323.45 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 585/1999 [04:36<11:08,  2.12it/s, est. speed input: 69323.61 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 586/1999 [04:36<11:07,  2.12it/s, est. speed input: 69323.63 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 587/1999 [04:37<11:07,  2.12it/s, est. speed input: 69323.50 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 588/1999 [04:37<11:06,  2.12it/s, est. speed input: 69323.55 toks/s, output: 2.12 toks/s]
Processed prompts:  29%|██▉       | 589/1999 [04:38<11:06,  2.12it/s, est. speed input: 69323.56 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 590/1999 [04:38<11:05,  2.12it/s, est. speed input: 69323.57 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 591/1999 [04:39<11:05,  2.12it/s, est. speed input: 69323.51 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 592/1999 [04:39<11:04,  2.12it/s, est. speed input: 69323.61 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 593/1999 [04:40<11:04,  2.12it/s, est. speed input: 69323.59 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 594/1999 [04:40<11:04,  2.12it/s, est. speed input: 69323.63 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 595/1999 [04:41<11:03,  2.12it/s, est. speed input: 69323.57 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 596/1999 [04:41<11:03,  2.12it/s, est. speed input: 69323.66 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 597/1999 [04:42<11:02,  2.12it/s, est. speed input: 69323.72 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 598/1999 [04:42<11:02,  2.12it/s, est. speed input: 69323.76 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|██▉       | 599/1999 [04:43<11:01,  2.12it/s, est. speed input: 69323.84 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 600/1999 [04:43<11:01,  2.12it/s, est. speed input: 69323.79 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 601/1999 [04:44<11:00,  2.12it/s, est. speed input: 69323.87 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 602/1999 [04:44<11:00,  2.12it/s, est. speed input: 69323.85 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 603/1999 [04:45<10:59,  2.12it/s, est. speed input: 69323.96 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 604/1999 [04:45<11:01,  2.11it/s, est. speed input: 69322.89 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 605/1999 [04:45<11:00,  2.11it/s, est. speed input: 69322.64 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 606/1999 [04:46<10:59,  2.11it/s, est. speed input: 69322.68 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 607/1999 [04:46<10:58,  2.11it/s, est. speed input: 69322.79 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 608/1999 [04:47<10:57,  2.11it/s, est. speed input: 69322.84 toks/s, output: 2.12 toks/s]
Processed prompts:  30%|███       | 609/1999 [04:47<10:56,  2.12it/s, est. speed input: 69323.06 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 610/1999 [04:48<10:56,  2.12it/s, est. speed input: 69323.10 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 611/1999 [04:48<10:56,  2.12it/s, est. speed input: 69323.04 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 612/1999 [04:49<10:55,  2.12it/s, est. speed input: 69322.99 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 613/1999 [04:49<10:54,  2.12it/s, est. speed input: 69323.19 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 614/1999 [04:50<10:55,  2.11it/s, est. speed input: 69322.83 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 615/1999 [04:50<10:54,  2.12it/s, est. speed input: 69323.01 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 616/1999 [04:51<10:53,  2.12it/s, est. speed input: 69323.23 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 617/1999 [04:51<10:54,  2.11it/s, est. speed input: 69322.63 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 618/1999 [04:52<10:53,  2.11it/s, est. speed input: 69322.57 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 619/1999 [04:52<10:52,  2.11it/s, est. speed input: 69322.70 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 620/1999 [04:53<10:51,  2.12it/s, est. speed input: 69322.72 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 621/1999 [04:53<10:51,  2.12it/s, est. speed input: 69322.75 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 622/1999 [04:54<10:50,  2.12it/s, est. speed input: 69322.85 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 623/1999 [04:54<10:50,  2.12it/s, est. speed input: 69322.90 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███       | 624/1999 [04:54<10:50,  2.11it/s, est. speed input: 69322.53 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 625/1999 [04:55<10:49,  2.11it/s, est. speed input: 69322.48 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 626/1999 [04:55<10:49,  2.11it/s, est. speed input: 69322.51 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 627/1999 [04:56<10:49,  2.11it/s, est. speed input: 69322.11 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 628/1999 [04:56<10:48,  2.11it/s, est. speed input: 69322.16 toks/s, output: 2.12 toks/s]
Processed prompts:  31%|███▏      | 629/1999 [04:57<10:47,  2.11it/s, est. speed input: 69322.18 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 630/1999 [04:57<10:47,  2.12it/s, est. speed input: 69322.37 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 631/1999 [04:58<10:46,  2.12it/s, est. speed input: 69322.35 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 632/1999 [04:58<10:45,  2.12it/s, est. speed input: 69322.46 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 633/1999 [04:59<10:45,  2.12it/s, est. speed input: 69322.53 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 634/1999 [04:59<10:44,  2.12it/s, est. speed input: 69322.55 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 635/1999 [05:00<10:44,  2.12it/s, est. speed input: 69322.65 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 636/1999 [05:00<10:43,  2.12it/s, est. speed input: 69322.69 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 637/1999 [05:01<10:43,  2.12it/s, est. speed input: 69322.78 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 638/1999 [05:01<10:42,  2.12it/s, est. speed input: 69322.86 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 639/1999 [05:02<10:42,  2.12it/s, est. speed input: 69322.98 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 640/1999 [05:02<10:43,  2.11it/s, est. speed input: 69322.04 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 641/1999 [05:02<10:43,  2.11it/s, est. speed input: 69321.91 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 642/1999 [05:03<10:42,  2.11it/s, est. speed input: 69321.89 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 643/1999 [05:03<10:41,  2.11it/s, est. speed input: 69321.95 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 644/1999 [05:04<10:40,  2.11it/s, est. speed input: 69321.88 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 645/1999 [05:04<10:40,  2.11it/s, est. speed input: 69321.86 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 646/1999 [05:05<10:39,  2.11it/s, est. speed input: 69321.88 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 647/1999 [05:05<10:38,  2.12it/s, est. speed input: 69322.04 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 648/1999 [05:06<10:38,  2.12it/s, est. speed input: 69322.16 toks/s, output: 2.12 toks/s]
Processed prompts:  32%|███▏      | 649/1999 [05:06<10:37,  2.12it/s, est. speed input: 69322.22 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 650/1999 [05:07<10:37,  2.12it/s, est. speed input: 69322.22 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 651/1999 [05:07<10:37,  2.12it/s, est. speed input: 69322.13 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 652/1999 [05:08<10:36,  2.12it/s, est. speed input: 69322.18 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 653/1999 [05:08<10:36,  2.11it/s, est. speed input: 69322.04 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 654/1999 [05:09<10:36,  2.11it/s, est. speed input: 69321.68 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 655/1999 [05:09<10:35,  2.11it/s, est. speed input: 69321.75 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 656/1999 [05:10<10:34,  2.12it/s, est. speed input: 69321.84 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 657/1999 [05:10<10:34,  2.11it/s, est. speed input: 69321.76 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 658/1999 [05:11<10:34,  2.11it/s, est. speed input: 69321.70 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 659/1999 [05:11<10:34,  2.11it/s, est. speed input: 69321.26 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 660/1999 [05:11<10:33,  2.11it/s, est. speed input: 69321.14 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 661/1999 [05:12<10:32,  2.11it/s, est. speed input: 69321.21 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 662/1999 [05:12<10:32,  2.12it/s, est. speed input: 69321.38 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 663/1999 [05:13<10:31,  2.11it/s, est. speed input: 69321.30 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 664/1999 [05:13<10:32,  2.11it/s, est. speed input: 69320.67 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 665/1999 [05:14<10:31,  2.11it/s, est. speed input: 69320.80 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 666/1999 [05:14<10:30,  2.11it/s, est. speed input: 69320.91 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 667/1999 [05:15<10:29,  2.12it/s, est. speed input: 69320.99 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 668/1999 [05:15<10:29,  2.12it/s, est. speed input: 69320.94 toks/s, output: 2.12 toks/s]
Processed prompts:  33%|███▎      | 669/1999 [05:16<10:28,  2.12it/s, est. speed input: 69321.02 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 670/1999 [05:16<10:28,  2.12it/s, est. speed input: 69321.01 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 671/1999 [05:17<10:27,  2.12it/s, est. speed input: 69321.08 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 672/1999 [05:17<10:27,  2.12it/s, est. speed input: 69321.03 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 673/1999 [05:18<10:26,  2.12it/s, est. speed input: 69321.05 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▎      | 674/1999 [05:18<10:26,  2.11it/s, est. speed input: 69320.92 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 675/1999 [05:19<10:26,  2.11it/s, est. speed input: 69320.83 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 676/1999 [05:19<10:25,  2.11it/s, est. speed input: 69320.84 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 677/1999 [05:20<10:24,  2.12it/s, est. speed input: 69321.01 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 678/1999 [05:20<10:24,  2.12it/s, est. speed input: 69320.99 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 679/1999 [05:20<10:23,  2.12it/s, est. speed input: 69320.94 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 680/1999 [05:21<10:23,  2.12it/s, est. speed input: 69320.97 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 681/1999 [05:21<10:23,  2.11it/s, est. speed input: 69320.87 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 682/1999 [05:22<10:22,  2.11it/s, est. speed input: 69320.83 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 683/1999 [05:22<10:22,  2.12it/s, est. speed input: 69320.84 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 684/1999 [05:23<10:21,  2.12it/s, est. speed input: 69320.82 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 685/1999 [05:23<10:21,  2.12it/s, est. speed input: 69320.86 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 686/1999 [05:24<10:20,  2.12it/s, est. speed input: 69320.90 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 687/1999 [05:24<10:20,  2.12it/s, est. speed input: 69320.92 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 688/1999 [05:25<10:28,  2.09it/s, est. speed input: 69316.03 toks/s, output: 2.12 toks/s]
Processed prompts:  34%|███▍      | 689/1999 [05:25<10:25,  2.10it/s, est. speed input: 69316.16 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 690/1999 [05:26<10:22,  2.10it/s, est. speed input: 69316.21 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 691/1999 [05:26<10:21,  2.11it/s, est. speed input: 69316.31 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 692/1999 [05:27<10:19,  2.11it/s, est. speed input: 69316.27 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 693/1999 [05:27<10:18,  2.11it/s, est. speed input: 69316.33 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 694/1999 [05:28<10:17,  2.11it/s, est. speed input: 69316.30 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 695/1999 [05:28<10:17,  2.11it/s, est. speed input: 69316.33 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 696/1999 [05:29<10:16,  2.11it/s, est. speed input: 69316.32 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 697/1999 [05:29<10:15,  2.11it/s, est. speed input: 69316.31 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 698/1999 [05:29<10:15,  2.11it/s, est. speed input: 69316.05 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▍      | 699/1999 [05:30<10:15,  2.11it/s, est. speed input: 69315.87 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 700/1999 [05:30<10:14,  2.11it/s, est. speed input: 69315.87 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 701/1999 [05:31<10:14,  2.11it/s, est. speed input: 69315.86 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 702/1999 [05:31<10:13,  2.12it/s, est. speed input: 69315.96 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 703/1999 [05:32<10:12,  2.12it/s, est. speed input: 69315.96 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 704/1999 [05:32<10:12,  2.12it/s, est. speed input: 69316.08 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 705/1999 [05:33<10:11,  2.12it/s, est. speed input: 69316.01 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 706/1999 [05:33<10:11,  2.12it/s, est. speed input: 69316.14 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 707/1999 [05:34<10:10,  2.12it/s, est. speed input: 69316.20 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 708/1999 [05:34<10:09,  2.12it/s, est. speed input: 69316.35 toks/s, output: 2.12 toks/s]
Processed prompts:  35%|███▌      | 709/1999 [05:35<10:09,  2.12it/s, est. speed input: 69316.35 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 710/1999 [05:35<10:09,  2.12it/s, est. speed input: 69316.32 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 711/1999 [05:36<10:08,  2.12it/s, est. speed input: 69316.36 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 712/1999 [05:36<10:08,  2.11it/s, est. speed input: 69316.22 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 713/1999 [05:37<10:08,  2.11it/s, est. speed input: 69316.15 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 714/1999 [05:37<10:07,  2.12it/s, est. speed input: 69316.24 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 715/1999 [05:38<10:06,  2.12it/s, est. speed input: 69316.26 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 716/1999 [05:38<10:06,  2.12it/s, est. speed input: 69316.22 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 717/1999 [05:38<10:05,  2.12it/s, est. speed input: 69316.26 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 718/1999 [05:39<10:05,  2.12it/s, est. speed input: 69316.28 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 719/1999 [05:39<10:05,  2.12it/s, est. speed input: 69316.30 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 720/1999 [05:40<10:19,  2.06it/s, est. speed input: 69308.38 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 721/1999 [05:40<10:13,  2.08it/s, est. speed input: 69308.72 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 722/1999 [05:41<10:10,  2.09it/s, est. speed input: 69308.73 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 723/1999 [05:41<10:07,  2.10it/s, est. speed input: 69308.78 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▌      | 724/1999 [05:42<10:05,  2.10it/s, est. speed input: 69308.94 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 725/1999 [05:42<10:04,  2.11it/s, est. speed input: 69308.92 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 726/1999 [05:43<10:03,  2.11it/s, est. speed input: 69308.90 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 727/1999 [05:43<10:02,  2.11it/s, est. speed input: 69308.85 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 728/1999 [05:44<10:01,  2.11it/s, est. speed input: 69308.79 toks/s, output: 2.12 toks/s]
Processed prompts:  36%|███▋      | 729/1999 [05:44<10:00,  2.11it/s, est. speed input: 69308.88 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 730/1999 [05:45<10:01,  2.11it/s, est. speed input: 69308.41 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 731/1999 [05:45<10:00,  2.11it/s, est. speed input: 69308.33 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 732/1999 [05:46<09:59,  2.11it/s, est. speed input: 69308.45 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 733/1999 [05:46<09:58,  2.11it/s, est. speed input: 69308.44 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 734/1999 [05:47<09:58,  2.11it/s, est. speed input: 69308.50 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 735/1999 [05:47<09:57,  2.12it/s, est. speed input: 69308.54 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 736/1999 [05:47<09:57,  2.12it/s, est. speed input: 69308.55 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 737/1999 [05:48<09:58,  2.11it/s, est. speed input: 69307.65 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 738/1999 [05:48<09:57,  2.11it/s, est. speed input: 69307.65 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 739/1999 [05:49<09:56,  2.11it/s, est. speed input: 69307.53 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 740/1999 [05:49<09:55,  2.11it/s, est. speed input: 69307.65 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 741/1999 [05:50<09:54,  2.11it/s, est. speed input: 69307.78 toks/s, output: 2.12 toks/s]
Processed prompts:  37%|███▋      | 742/1999 [05:50<10:13,  2.05it/s, est. speed input: 69297.94 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 743/1999 [05:51<10:06,  2.07it/s, est. speed input: 69298.41 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 744/1999 [05:51<10:01,  2.09it/s, est. speed input: 69298.84 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 745/1999 [05:52<09:57,  2.10it/s, est. speed input: 69299.26 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 746/1999 [05:52<09:55,  2.10it/s, est. speed input: 69299.31 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 747/1999 [05:53<09:53,  2.11it/s, est. speed input: 69299.77 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 748/1999 [05:53<09:53,  2.11it/s, est. speed input: 69299.20 toks/s, output: 2.11 toks/s]
Processed prompts:  37%|███▋      | 749/1999 [05:54<09:52,  2.11it/s, est. speed input: 69299.11 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 750/1999 [05:54<09:51,  2.11it/s, est. speed input: 69299.28 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 751/1999 [05:55<09:49,  2.12it/s, est. speed input: 69299.67 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 752/1999 [05:55<09:49,  2.12it/s, est. speed input: 69299.84 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 753/1999 [05:56<09:49,  2.12it/s, est. speed input: 69299.70 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 754/1999 [05:56<09:47,  2.12it/s, est. speed input: 69300.06 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 755/1999 [05:56<09:49,  2.11it/s, est. speed input: 69299.35 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 756/1999 [05:57<09:47,  2.11it/s, est. speed input: 69299.65 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 757/1999 [05:57<09:46,  2.12it/s, est. speed input: 69300.02 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 758/1999 [05:58<09:45,  2.12it/s, est. speed input: 69300.46 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 759/1999 [05:58<09:45,  2.12it/s, est. speed input: 69300.56 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 760/1999 [05:59<09:44,  2.12it/s, est. speed input: 69300.84 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 761/1999 [05:59<09:43,  2.12it/s, est. speed input: 69301.31 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 762/1999 [06:00<09:43,  2.12it/s, est. speed input: 69301.59 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 763/1999 [06:00<09:42,  2.12it/s, est. speed input: 69301.83 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 764/1999 [06:01<09:42,  2.12it/s, est. speed input: 69302.08 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 765/1999 [06:01<09:41,  2.12it/s, est. speed input: 69302.26 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 766/1999 [06:02<09:41,  2.12it/s, est. speed input: 69302.38 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 767/1999 [06:02<09:41,  2.12it/s, est. speed input: 69302.65 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 768/1999 [06:03<09:41,  2.12it/s, est. speed input: 69302.25 toks/s, output: 2.11 toks/s]
Processed prompts:  38%|███▊      | 769/1999 [06:03<09:40,  2.12it/s, est. speed input: 69302.60 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 770/1999 [06:04<09:40,  2.12it/s, est. speed input: 69302.85 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 771/1999 [06:04<09:40,  2.11it/s, est. speed input: 69302.42 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 772/1999 [06:05<09:39,  2.12it/s, est. speed input: 69302.61 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 773/1999 [06:05<09:39,  2.12it/s, est. speed input: 69302.79 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▊      | 774/1999 [06:05<09:38,  2.12it/s, est. speed input: 69303.21 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 775/1999 [06:06<09:38,  2.12it/s, est. speed input: 69302.97 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 776/1999 [06:06<09:37,  2.12it/s, est. speed input: 69303.11 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 777/1999 [06:07<09:37,  2.12it/s, est. speed input: 69303.31 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 778/1999 [06:07<09:37,  2.11it/s, est. speed input: 69302.92 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 779/1999 [06:08<09:36,  2.12it/s, est. speed input: 69303.01 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 780/1999 [06:08<09:35,  2.12it/s, est. speed input: 69303.37 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 781/1999 [06:09<09:34,  2.12it/s, est. speed input: 69303.83 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 782/1999 [06:09<09:34,  2.12it/s, est. speed input: 69304.02 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 783/1999 [06:10<09:33,  2.12it/s, est. speed input: 69304.34 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 784/1999 [06:10<09:32,  2.12it/s, est. speed input: 69304.65 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 785/1999 [06:11<09:32,  2.12it/s, est. speed input: 69304.93 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 786/1999 [06:11<09:33,  2.12it/s, est. speed input: 69304.38 toks/s, output: 2.12 toks/s]
Processed prompts:  39%|███▉      | 787/1999 [06:12<09:33,  2.11it/s, est. speed input: 69303.98 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 788/1999 [06:12<09:32,  2.11it/s, est. speed input: 69304.07 toks/s, output: 2.11 toks/s]
Processed prompts:  39%|███▉      | 789/1999 [06:13<09:31,  2.12it/s, est. speed input: 69304.43 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 790/1999 [06:13<09:30,  2.12it/s, est. speed input: 69304.84 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 791/1999 [06:13<09:31,  2.11it/s, est. speed input: 69304.28 toks/s, output: 2.11 toks/s]
Processed prompts:  40%|███▉      | 792/1999 [06:14<09:30,  2.12it/s, est. speed input: 69304.57 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 793/1999 [06:14<09:29,  2.12it/s, est. speed input: 69304.84 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 794/1999 [06:15<09:29,  2.12it/s, est. speed input: 69304.90 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 795/1999 [06:15<09:28,  2.12it/s, est. speed input: 69305.06 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 796/1999 [06:16<09:27,  2.12it/s, est. speed input: 69305.32 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 797/1999 [06:16<09:27,  2.12it/s, est. speed input: 69305.58 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 798/1999 [06:17<09:26,  2.12it/s, est. speed input: 69305.78 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|███▉      | 799/1999 [06:17<09:26,  2.12it/s, est. speed input: 69305.94 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 800/1999 [06:18<09:25,  2.12it/s, est. speed input: 69306.11 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 801/1999 [06:18<09:25,  2.12it/s, est. speed input: 69305.93 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 802/1999 [06:19<09:25,  2.12it/s, est. speed input: 69306.11 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 803/1999 [06:19<09:24,  2.12it/s, est. speed input: 69306.37 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 804/1999 [06:20<09:23,  2.12it/s, est. speed input: 69306.63 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 805/1999 [06:20<09:23,  2.12it/s, est. speed input: 69306.74 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 806/1999 [06:21<09:23,  2.12it/s, est. speed input: 69306.82 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 807/1999 [06:21<09:23,  2.12it/s, est. speed input: 69306.59 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 808/1999 [06:22<09:22,  2.12it/s, est. speed input: 69306.63 toks/s, output: 2.12 toks/s]
Processed prompts:  40%|████      | 809/1999 [06:22<09:21,  2.12it/s, est. speed input: 69306.96 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 810/1999 [06:22<09:22,  2.11it/s, est. speed input: 69306.41 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 811/1999 [06:23<09:22,  2.11it/s, est. speed input: 69306.02 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 812/1999 [06:23<09:21,  2.11it/s, est. speed input: 69306.12 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 813/1999 [06:24<09:20,  2.12it/s, est. speed input: 69306.42 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 814/1999 [06:24<09:19,  2.12it/s, est. speed input: 69306.60 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 815/1999 [06:25<09:20,  2.11it/s, est. speed input: 69305.90 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 816/1999 [06:25<09:19,  2.11it/s, est. speed input: 69306.25 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 817/1999 [06:26<09:18,  2.12it/s, est. speed input: 69306.52 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 818/1999 [06:26<09:17,  2.12it/s, est. speed input: 69306.59 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 819/1999 [06:27<09:17,  2.12it/s, est. speed input: 69306.74 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 820/1999 [06:27<09:15,  2.12it/s, est. speed input: 69307.25 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 821/1999 [06:28<09:15,  2.12it/s, est. speed input: 69307.48 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 822/1999 [06:28<09:16,  2.12it/s, est. speed input: 69307.01 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 823/1999 [06:29<09:15,  2.12it/s, est. speed input: 69307.00 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████      | 824/1999 [06:29<09:15,  2.12it/s, est. speed input: 69307.07 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 825/1999 [06:30<09:14,  2.12it/s, est. speed input: 69307.25 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 826/1999 [06:30<09:13,  2.12it/s, est. speed input: 69307.71 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 827/1999 [06:30<09:12,  2.12it/s, est. speed input: 69307.92 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 828/1999 [06:31<09:12,  2.12it/s, est. speed input: 69308.24 toks/s, output: 2.12 toks/s]
Processed prompts:  41%|████▏     | 829/1999 [06:31<09:11,  2.12it/s, est. speed input: 69308.33 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 830/1999 [06:32<09:11,  2.12it/s, est. speed input: 69308.52 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 831/1999 [06:32<09:11,  2.12it/s, est. speed input: 69308.58 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 832/1999 [06:33<09:10,  2.12it/s, est. speed input: 69308.89 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 833/1999 [06:33<09:10,  2.12it/s, est. speed input: 69308.94 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 834/1999 [06:34<09:09,  2.12it/s, est. speed input: 69309.06 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 835/1999 [06:34<09:09,  2.12it/s, est. speed input: 69309.24 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 836/1999 [06:35<09:08,  2.12it/s, est. speed input: 69309.58 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 837/1999 [06:35<09:07,  2.12it/s, est. speed input: 69309.80 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 838/1999 [06:36<09:07,  2.12it/s, est. speed input: 69310.11 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 839/1999 [06:36<09:06,  2.12it/s, est. speed input: 69310.33 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 840/1999 [06:37<09:06,  2.12it/s, est. speed input: 69310.63 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 841/1999 [06:37<09:06,  2.12it/s, est. speed input: 69310.33 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 842/1999 [06:38<09:06,  2.12it/s, est. speed input: 69310.51 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 843/1999 [06:38<09:05,  2.12it/s, est. speed input: 69310.85 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 844/1999 [06:39<09:04,  2.12it/s, est. speed input: 69310.96 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 845/1999 [06:39<09:04,  2.12it/s, est. speed input: 69311.16 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 846/1999 [06:39<09:03,  2.12it/s, est. speed input: 69311.45 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 847/1999 [06:40<09:03,  2.12it/s, est. speed input: 69311.71 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 848/1999 [06:40<09:02,  2.12it/s, est. speed input: 69311.78 toks/s, output: 2.12 toks/s]
Processed prompts:  42%|████▏     | 849/1999 [06:41<09:02,  2.12it/s, est. speed input: 69312.09 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 850/1999 [06:41<09:03,  2.11it/s, est. speed input: 69311.50 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 851/1999 [06:42<09:02,  2.11it/s, est. speed input: 69311.49 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 852/1999 [06:42<09:01,  2.12it/s, est. speed input: 69311.68 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 853/1999 [06:43<09:01,  2.12it/s, est. speed input: 69311.73 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 854/1999 [06:43<09:00,  2.12it/s, est. speed input: 69311.81 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 855/1999 [06:44<09:00,  2.12it/s, est. speed input: 69312.04 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 856/1999 [06:44<08:59,  2.12it/s, est. speed input: 69312.07 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 857/1999 [06:45<09:00,  2.11it/s, est. speed input: 69311.80 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 858/1999 [06:45<08:59,  2.12it/s, est. speed input: 69311.88 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 859/1999 [06:46<08:58,  2.12it/s, est. speed input: 69312.15 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 860/1999 [06:46<08:58,  2.12it/s, est. speed input: 69312.16 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 861/1999 [06:47<08:57,  2.12it/s, est. speed input: 69312.33 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 862/1999 [06:47<08:56,  2.12it/s, est. speed input: 69312.58 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 863/1999 [06:47<08:55,  2.12it/s, est. speed input: 69312.87 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 864/1999 [06:48<08:55,  2.12it/s, est. speed input: 69313.03 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 865/1999 [06:48<08:54,  2.12it/s, est. speed input: 69313.44 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 866/1999 [06:49<08:55,  2.12it/s, est. speed input: 69312.97 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 867/1999 [06:49<08:55,  2.11it/s, est. speed input: 69312.86 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 868/1999 [06:50<08:54,  2.12it/s, est. speed input: 69313.04 toks/s, output: 2.12 toks/s]
Processed prompts:  43%|████▎     | 869/1999 [06:50<08:53,  2.12it/s, est. speed input: 69313.35 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 870/1999 [06:51<08:52,  2.12it/s, est. speed input: 69313.43 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 871/1999 [06:51<08:53,  2.12it/s, est. speed input: 69313.16 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 872/1999 [06:52<08:52,  2.12it/s, est. speed input: 69313.27 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 873/1999 [06:52<08:51,  2.12it/s, est. speed input: 69313.39 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▎     | 874/1999 [06:53<08:51,  2.12it/s, est. speed input: 69313.60 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 875/1999 [06:53<08:50,  2.12it/s, est. speed input: 69313.70 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 876/1999 [06:54<08:50,  2.12it/s, est. speed input: 69313.65 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 877/1999 [06:54<08:50,  2.12it/s, est. speed input: 69313.70 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 878/1999 [06:55<08:49,  2.12it/s, est. speed input: 69313.86 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 879/1999 [06:55<08:48,  2.12it/s, est. speed input: 69314.04 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 880/1999 [06:56<08:48,  2.12it/s, est. speed input: 69314.00 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 881/1999 [06:56<08:47,  2.12it/s, est. speed input: 69314.19 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 882/1999 [06:56<08:47,  2.12it/s, est. speed input: 69314.42 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 883/1999 [06:57<08:46,  2.12it/s, est. speed input: 69314.76 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 884/1999 [06:57<08:46,  2.12it/s, est. speed input: 69314.68 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 885/1999 [06:58<08:45,  2.12it/s, est. speed input: 69314.83 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 886/1999 [06:58<08:44,  2.12it/s, est. speed input: 69315.13 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 887/1999 [06:59<08:44,  2.12it/s, est. speed input: 69315.35 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 888/1999 [06:59<08:43,  2.12it/s, est. speed input: 69315.61 toks/s, output: 2.12 toks/s]
Processed prompts:  44%|████▍     | 889/1999 [07:00<08:44,  2.12it/s, est. speed input: 69315.09 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 890/1999 [07:00<08:44,  2.12it/s, est. speed input: 69315.11 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 891/1999 [07:01<08:43,  2.12it/s, est. speed input: 69315.35 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 892/1999 [07:01<08:42,  2.12it/s, est. speed input: 69315.36 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 893/1999 [07:02<08:42,  2.12it/s, est. speed input: 69315.47 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 894/1999 [07:02<08:41,  2.12it/s, est. speed input: 69315.63 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 895/1999 [07:03<08:40,  2.12it/s, est. speed input: 69315.91 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 896/1999 [07:03<08:41,  2.12it/s, est. speed input: 69315.62 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 897/1999 [07:04<08:40,  2.12it/s, est. speed input: 69315.75 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 898/1999 [07:04<08:39,  2.12it/s, est. speed input: 69315.98 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▍     | 899/1999 [07:04<08:39,  2.12it/s, est. speed input: 69315.90 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 900/1999 [07:05<08:38,  2.12it/s, est. speed input: 69316.06 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 901/1999 [07:05<08:38,  2.12it/s, est. speed input: 69316.26 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 902/1999 [07:06<08:37,  2.12it/s, est. speed input: 69316.51 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 903/1999 [07:06<08:36,  2.12it/s, est. speed input: 69316.71 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 904/1999 [07:07<08:36,  2.12it/s, est. speed input: 69316.82 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 905/1999 [07:07<08:38,  2.11it/s, est. speed input: 69315.79 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 906/1999 [07:08<08:36,  2.11it/s, est. speed input: 69316.17 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 907/1999 [07:08<08:35,  2.12it/s, est. speed input: 69316.46 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 908/1999 [07:09<08:34,  2.12it/s, est. speed input: 69316.71 toks/s, output: 2.12 toks/s]
Processed prompts:  45%|████▌     | 909/1999 [07:09<08:35,  2.12it/s, est. speed input: 69316.44 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 910/1999 [07:10<08:34,  2.12it/s, est. speed input: 69316.40 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 911/1999 [07:10<08:34,  2.12it/s, est. speed input: 69316.54 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 912/1999 [07:11<08:33,  2.12it/s, est. speed input: 69316.46 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 913/1999 [07:11<08:33,  2.11it/s, est. speed input: 69316.30 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 914/1999 [07:12<08:32,  2.12it/s, est. speed input: 69316.57 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 915/1999 [07:12<08:32,  2.12it/s, est. speed input: 69316.65 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 916/1999 [07:13<08:31,  2.12it/s, est. speed input: 69316.83 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 917/1999 [07:13<08:30,  2.12it/s, est. speed input: 69317.07 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 918/1999 [07:13<08:29,  2.12it/s, est. speed input: 69317.28 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 919/1999 [07:14<08:29,  2.12it/s, est. speed input: 69317.36 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 920/1999 [07:14<08:29,  2.12it/s, est. speed input: 69317.46 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 921/1999 [07:15<08:29,  2.12it/s, est. speed input: 69317.20 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 922/1999 [07:15<08:30,  2.11it/s, est. speed input: 69316.74 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 923/1999 [07:16<08:29,  2.11it/s, est. speed input: 69316.79 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▌     | 924/1999 [07:16<08:28,  2.12it/s, est. speed input: 69317.01 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 925/1999 [07:17<08:27,  2.12it/s, est. speed input: 69316.97 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 926/1999 [07:17<08:26,  2.12it/s, est. speed input: 69317.11 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 927/1999 [07:18<08:26,  2.12it/s, est. speed input: 69317.32 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 928/1999 [07:18<08:25,  2.12it/s, est. speed input: 69317.68 toks/s, output: 2.12 toks/s]
Processed prompts:  46%|████▋     | 929/1999 [07:19<08:25,  2.12it/s, est. speed input: 69317.67 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 930/1999 [07:19<08:24,  2.12it/s, est. speed input: 69317.77 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 931/1999 [07:20<08:23,  2.12it/s, est. speed input: 69317.98 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 932/1999 [07:20<08:23,  2.12it/s, est. speed input: 69318.14 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 933/1999 [07:21<08:22,  2.12it/s, est. speed input: 69318.29 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 934/1999 [07:21<08:22,  2.12it/s, est. speed input: 69318.51 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 935/1999 [07:21<08:22,  2.12it/s, est. speed input: 69318.34 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 936/1999 [07:22<08:22,  2.11it/s, est. speed input: 69318.09 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 937/1999 [07:22<08:21,  2.12it/s, est. speed input: 69318.49 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 938/1999 [07:23<08:21,  2.12it/s, est. speed input: 69318.37 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 939/1999 [07:23<08:20,  2.12it/s, est. speed input: 69318.42 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 940/1999 [07:24<08:20,  2.12it/s, est. speed input: 69318.57 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 941/1999 [07:24<08:19,  2.12it/s, est. speed input: 69318.78 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 942/1999 [07:25<08:19,  2.12it/s, est. speed input: 69318.77 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 943/1999 [07:25<08:18,  2.12it/s, est. speed input: 69319.08 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 944/1999 [07:26<08:17,  2.12it/s, est. speed input: 69319.24 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 945/1999 [07:26<08:17,  2.12it/s, est. speed input: 69319.25 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 946/1999 [07:27<08:17,  2.12it/s, est. speed input: 69319.37 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 947/1999 [07:27<08:16,  2.12it/s, est. speed input: 69319.63 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 948/1999 [07:28<08:15,  2.12it/s, est. speed input: 69319.80 toks/s, output: 2.12 toks/s]
Processed prompts:  47%|████▋     | 949/1999 [07:28<08:15,  2.12it/s, est. speed input: 69319.84 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 950/1999 [07:29<08:14,  2.12it/s, est. speed input: 69320.12 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 951/1999 [07:29<08:14,  2.12it/s, est. speed input: 69320.21 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 952/1999 [07:30<08:14,  2.12it/s, est. speed input: 69320.27 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 953/1999 [07:30<08:13,  2.12it/s, est. speed input: 69320.52 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 954/1999 [07:30<08:13,  2.12it/s, est. speed input: 69320.60 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 955/1999 [07:31<08:12,  2.12it/s, est. speed input: 69320.73 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 956/1999 [07:31<08:12,  2.12it/s, est. speed input: 69320.53 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 957/1999 [07:32<08:12,  2.12it/s, est. speed input: 69320.72 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 958/1999 [07:32<08:11,  2.12it/s, est. speed input: 69320.85 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 959/1999 [07:33<08:11,  2.12it/s, est. speed input: 69320.90 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 960/1999 [07:33<08:10,  2.12it/s, est. speed input: 69321.05 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 961/1999 [07:34<08:10,  2.12it/s, est. speed input: 69320.95 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 962/1999 [07:34<08:09,  2.12it/s, est. speed input: 69320.95 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 963/1999 [07:35<08:09,  2.12it/s, est. speed input: 69321.13 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 964/1999 [07:35<08:08,  2.12it/s, est. speed input: 69321.40 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 965/1999 [07:36<08:07,  2.12it/s, est. speed input: 69321.73 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 966/1999 [07:36<08:07,  2.12it/s, est. speed input: 69321.74 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 967/1999 [07:37<08:07,  2.12it/s, est. speed input: 69321.44 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 968/1999 [07:37<08:07,  2.12it/s, est. speed input: 69321.44 toks/s, output: 2.12 toks/s]
Processed prompts:  48%|████▊     | 969/1999 [07:38<08:06,  2.12it/s, est. speed input: 69321.64 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 970/1999 [07:38<08:05,  2.12it/s, est. speed input: 69321.90 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 971/1999 [07:38<08:05,  2.12it/s, est. speed input: 69321.97 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 972/1999 [07:39<08:05,  2.11it/s, est. speed input: 69321.60 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 973/1999 [07:39<08:04,  2.12it/s, est. speed input: 69321.71 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▊     | 974/1999 [07:40<08:03,  2.12it/s, est. speed input: 69322.00 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 975/1999 [07:40<08:03,  2.12it/s, est. speed input: 69322.16 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 976/1999 [07:41<08:02,  2.12it/s, est. speed input: 69322.41 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 977/1999 [07:41<08:02,  2.12it/s, est. speed input: 69322.50 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 978/1999 [07:42<08:02,  2.12it/s, est. speed input: 69322.48 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 979/1999 [07:42<08:02,  2.11it/s, est. speed input: 69322.04 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 980/1999 [07:43<08:01,  2.12it/s, est. speed input: 69322.22 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 981/1999 [07:43<08:00,  2.12it/s, est. speed input: 69322.45 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 982/1999 [07:44<07:59,  2.12it/s, est. speed input: 69322.67 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 983/1999 [07:44<07:59,  2.12it/s, est. speed input: 69322.91 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 984/1999 [07:45<08:00,  2.11it/s, est. speed input: 69322.20 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 985/1999 [07:45<07:59,  2.12it/s, est. speed input: 69322.50 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 986/1999 [07:46<07:58,  2.12it/s, est. speed input: 69322.71 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 987/1999 [07:46<07:57,  2.12it/s, est. speed input: 69322.95 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 988/1999 [07:47<07:57,  2.12it/s, est. speed input: 69322.74 toks/s, output: 2.12 toks/s]
Processed prompts:  49%|████▉     | 989/1999 [07:47<07:57,  2.12it/s, est. speed input: 69322.89 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 990/1999 [07:47<07:59,  2.10it/s, est. speed input: 69321.30 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 991/1999 [07:48<07:57,  2.11it/s, est. speed input: 69321.64 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 992/1999 [07:48<07:56,  2.11it/s, est. speed input: 69321.86 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 993/1999 [07:49<07:55,  2.12it/s, est. speed input: 69322.18 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 994/1999 [07:49<07:56,  2.11it/s, est. speed input: 69321.58 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 995/1999 [07:50<07:55,  2.11it/s, est. speed input: 69321.69 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 996/1999 [07:50<07:53,  2.12it/s, est. speed input: 69321.96 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 997/1999 [07:51<07:53,  2.12it/s, est. speed input: 69322.17 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 998/1999 [07:51<07:54,  2.11it/s, est. speed input: 69321.50 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|████▉     | 999/1999 [07:52<07:52,  2.12it/s, est. speed input: 69321.79 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1000/1999 [07:52<07:51,  2.12it/s, est. speed input: 69322.02 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1001/1999 [07:53<07:51,  2.12it/s, est. speed input: 69322.00 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1002/1999 [07:53<07:50,  2.12it/s, est. speed input: 69322.33 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1003/1999 [07:54<07:50,  2.12it/s, est. speed input: 69322.04 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1004/1999 [07:54<07:50,  2.11it/s, est. speed input: 69321.96 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1005/1999 [07:55<07:49,  2.12it/s, est. speed input: 69322.20 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1006/1999 [07:55<07:48,  2.12it/s, est. speed input: 69322.57 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1007/1999 [07:56<07:49,  2.11it/s, est. speed input: 69322.10 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1008/1999 [07:56<07:48,  2.12it/s, est. speed input: 69322.27 toks/s, output: 2.12 toks/s]
Processed prompts:  50%|█████     | 1009/1999 [07:56<07:47,  2.12it/s, est. speed input: 69322.52 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1010/1999 [07:57<07:46,  2.12it/s, est. speed input: 69322.74 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1011/1999 [07:57<07:45,  2.12it/s, est. speed input: 69322.98 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1012/1999 [07:58<07:45,  2.12it/s, est. speed input: 69323.19 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1013/1999 [07:58<07:44,  2.12it/s, est. speed input: 69323.35 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1014/1999 [07:59<07:45,  2.12it/s, est. speed input: 69323.23 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1015/1999 [07:59<07:44,  2.12it/s, est. speed input: 69323.38 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1016/1999 [08:00<07:43,  2.12it/s, est. speed input: 69323.66 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1017/1999 [08:00<07:43,  2.12it/s, est. speed input: 69323.82 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1018/1999 [08:01<07:43,  2.11it/s, est. speed input: 69323.35 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1019/1999 [08:01<07:42,  2.12it/s, est. speed input: 69323.57 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1020/1999 [08:02<07:42,  2.12it/s, est. speed input: 69323.74 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1021/1999 [08:02<07:41,  2.12it/s, est. speed input: 69323.87 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1022/1999 [08:03<07:40,  2.12it/s, est. speed input: 69324.11 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1023/1999 [08:03<07:41,  2.11it/s, est. speed input: 69323.57 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████     | 1024/1999 [08:04<07:40,  2.12it/s, est. speed input: 69323.66 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1025/1999 [08:04<07:40,  2.12it/s, est. speed input: 69323.83 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1026/1999 [08:04<07:39,  2.12it/s, est. speed input: 69323.93 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1027/1999 [08:05<07:38,  2.12it/s, est. speed input: 69324.06 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1028/1999 [08:05<07:38,  2.12it/s, est. speed input: 69324.19 toks/s, output: 2.12 toks/s]
Processed prompts:  51%|█████▏    | 1029/1999 [08:06<07:37,  2.12it/s, est. speed input: 69324.40 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1030/1999 [08:06<07:37,  2.12it/s, est. speed input: 69324.51 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1031/1999 [08:07<07:37,  2.12it/s, est. speed input: 69324.48 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1032/1999 [08:07<07:36,  2.12it/s, est. speed input: 69324.68 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1033/1999 [08:08<07:36,  2.12it/s, est. speed input: 69324.67 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1034/1999 [08:08<07:35,  2.12it/s, est. speed input: 69324.88 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1035/1999 [08:09<07:34,  2.12it/s, est. speed input: 69324.99 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1036/1999 [08:09<07:34,  2.12it/s, est. speed input: 69325.16 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1037/1999 [08:10<07:33,  2.12it/s, est. speed input: 69325.38 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1038/1999 [08:10<07:33,  2.12it/s, est. speed input: 69325.43 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1039/1999 [08:11<07:32,  2.12it/s, est. speed input: 69325.68 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1040/1999 [08:11<07:31,  2.12it/s, est. speed input: 69325.96 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1041/1999 [08:12<07:31,  2.12it/s, est. speed input: 69326.03 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1042/1999 [08:12<07:31,  2.12it/s, est. speed input: 69326.09 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1043/1999 [08:12<07:32,  2.11it/s, est. speed input: 69325.72 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1044/1999 [08:13<07:31,  2.11it/s, est. speed input: 69325.60 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1045/1999 [08:13<07:30,  2.12it/s, est. speed input: 69325.90 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1046/1999 [08:14<07:30,  2.12it/s, est. speed input: 69325.94 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1047/1999 [08:14<07:29,  2.12it/s, est. speed input: 69325.96 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1048/1999 [08:15<07:28,  2.12it/s, est. speed input: 69326.18 toks/s, output: 2.12 toks/s]
Processed prompts:  52%|█████▏    | 1049/1999 [08:15<07:28,  2.12it/s, est. speed input: 69326.34 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1050/1999 [08:16<07:29,  2.11it/s, est. speed input: 69325.84 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1051/1999 [08:16<07:28,  2.12it/s, est. speed input: 69325.99 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1052/1999 [08:17<07:27,  2.12it/s, est. speed input: 69326.27 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1053/1999 [08:17<07:26,  2.12it/s, est. speed input: 69326.32 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1054/1999 [08:18<07:26,  2.12it/s, est. speed input: 69326.23 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1055/1999 [08:18<07:25,  2.12it/s, est. speed input: 69326.44 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1056/1999 [08:19<07:24,  2.12it/s, est. speed input: 69326.62 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1057/1999 [08:19<07:24,  2.12it/s, est. speed input: 69326.69 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1058/1999 [08:20<07:23,  2.12it/s, est. speed input: 69326.90 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1059/1999 [08:20<07:24,  2.12it/s, est. speed input: 69326.58 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1060/1999 [08:21<07:23,  2.12it/s, est. speed input: 69326.60 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1061/1999 [08:21<07:23,  2.12it/s, est. speed input: 69326.71 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1062/1999 [08:21<07:22,  2.12it/s, est. speed input: 69326.96 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1063/1999 [08:22<07:21,  2.12it/s, est. speed input: 69327.22 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1064/1999 [08:22<07:21,  2.12it/s, est. speed input: 69327.21 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1065/1999 [08:23<07:20,  2.12it/s, est. speed input: 69327.41 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1066/1999 [08:23<07:20,  2.12it/s, est. speed input: 69327.26 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1067/1999 [08:24<07:19,  2.12it/s, est. speed input: 69327.45 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1068/1999 [08:24<07:19,  2.12it/s, est. speed input: 69327.48 toks/s, output: 2.12 toks/s]
Processed prompts:  53%|█████▎    | 1069/1999 [08:25<07:19,  2.11it/s, est. speed input: 69327.21 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1070/1999 [08:25<07:19,  2.11it/s, est. speed input: 69327.18 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1071/1999 [08:26<07:18,  2.12it/s, est. speed input: 69327.36 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1072/1999 [08:26<07:17,  2.12it/s, est. speed input: 69327.50 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1073/1999 [08:27<07:17,  2.12it/s, est. speed input: 69327.59 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▎    | 1074/1999 [08:27<07:16,  2.12it/s, est. speed input: 69327.67 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1075/1999 [08:28<07:16,  2.12it/s, est. speed input: 69327.76 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1076/1999 [08:28<07:15,  2.12it/s, est. speed input: 69327.90 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1077/1999 [08:29<07:14,  2.12it/s, est. speed input: 69328.06 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1078/1999 [08:29<07:14,  2.12it/s, est. speed input: 69328.28 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1079/1999 [08:29<07:13,  2.12it/s, est. speed input: 69328.45 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1080/1999 [08:30<07:13,  2.12it/s, est. speed input: 69328.75 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1081/1999 [08:30<07:12,  2.12it/s, est. speed input: 69328.90 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1082/1999 [08:31<07:13,  2.12it/s, est. speed input: 69328.61 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1083/1999 [08:31<07:12,  2.12it/s, est. speed input: 69328.77 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1084/1999 [08:32<07:11,  2.12it/s, est. speed input: 69329.01 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1085/1999 [08:32<07:11,  2.12it/s, est. speed input: 69329.14 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1086/1999 [08:33<07:11,  2.12it/s, est. speed input: 69328.97 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1087/1999 [08:33<07:10,  2.12it/s, est. speed input: 69329.26 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1088/1999 [08:34<07:09,  2.12it/s, est. speed input: 69329.37 toks/s, output: 2.12 toks/s]
Processed prompts:  54%|█████▍    | 1089/1999 [08:34<07:09,  2.12it/s, est. speed input: 69329.43 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1090/1999 [08:35<07:08,  2.12it/s, est. speed input: 69329.56 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1091/1999 [08:35<07:08,  2.12it/s, est. speed input: 69329.86 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1092/1999 [08:36<07:08,  2.12it/s, est. speed input: 69329.68 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1093/1999 [08:36<07:08,  2.12it/s, est. speed input: 69329.56 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1094/1999 [08:37<07:07,  2.12it/s, est. speed input: 69329.81 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1095/1999 [08:37<07:06,  2.12it/s, est. speed input: 69329.76 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1096/1999 [08:38<07:06,  2.12it/s, est. speed input: 69329.78 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1097/1999 [08:38<07:05,  2.12it/s, est. speed input: 69329.90 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1098/1999 [08:38<07:05,  2.12it/s, est. speed input: 69330.15 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▍    | 1099/1999 [08:39<07:04,  2.12it/s, est. speed input: 69330.28 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1100/1999 [08:39<07:03,  2.12it/s, est. speed input: 69330.49 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1101/1999 [08:40<07:03,  2.12it/s, est. speed input: 69330.82 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1102/1999 [08:40<07:02,  2.12it/s, est. speed input: 69330.93 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1103/1999 [08:41<07:02,  2.12it/s, est. speed input: 69330.99 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1104/1999 [08:41<07:01,  2.12it/s, est. speed input: 69331.19 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1105/1999 [08:42<07:01,  2.12it/s, est. speed input: 69331.40 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1106/1999 [08:42<07:00,  2.12it/s, est. speed input: 69331.54 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1107/1999 [08:43<07:00,  2.12it/s, est. speed input: 69331.67 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1108/1999 [08:43<07:03,  2.11it/s, est. speed input: 69330.29 toks/s, output: 2.12 toks/s]
Processed prompts:  55%|█████▌    | 1109/1999 [08:44<07:01,  2.11it/s, est. speed input: 69330.44 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1110/1999 [08:44<07:00,  2.12it/s, est. speed input: 69330.77 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1111/1999 [08:45<07:00,  2.11it/s, est. speed input: 69330.64 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1112/1999 [08:45<06:59,  2.11it/s, est. speed input: 69330.66 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1113/1999 [08:46<06:58,  2.12it/s, est. speed input: 69330.94 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1114/1999 [08:46<06:57,  2.12it/s, est. speed input: 69331.15 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1115/1999 [08:46<06:57,  2.12it/s, est. speed input: 69331.19 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1116/1999 [08:47<06:57,  2.12it/s, est. speed input: 69331.15 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1117/1999 [08:47<06:56,  2.12it/s, est. speed input: 69331.31 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1118/1999 [08:48<06:55,  2.12it/s, est. speed input: 69331.39 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1119/1999 [08:48<06:56,  2.11it/s, est. speed input: 69330.98 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1120/1999 [08:49<06:55,  2.12it/s, est. speed input: 69331.15 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1121/1999 [08:49<06:54,  2.12it/s, est. speed input: 69331.39 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1122/1999 [08:50<06:53,  2.12it/s, est. speed input: 69331.67 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1123/1999 [08:50<06:53,  2.12it/s, est. speed input: 69331.62 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▌    | 1124/1999 [08:51<06:52,  2.12it/s, est. speed input: 69331.78 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1125/1999 [08:51<06:53,  2.11it/s, est. speed input: 69331.45 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1126/1999 [08:52<06:52,  2.12it/s, est. speed input: 69331.61 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1127/1999 [08:52<06:51,  2.12it/s, est. speed input: 69331.78 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1128/1999 [08:53<06:51,  2.12it/s, est. speed input: 69331.93 toks/s, output: 2.12 toks/s]
Processed prompts:  56%|█████▋    | 1129/1999 [08:53<06:50,  2.12it/s, est. speed input: 69331.99 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1130/1999 [08:54<06:50,  2.12it/s, est. speed input: 69332.13 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1131/1999 [08:54<06:49,  2.12it/s, est. speed input: 69332.16 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1132/1999 [08:55<06:50,  2.11it/s, est. speed input: 69331.60 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1133/1999 [08:55<06:49,  2.11it/s, est. speed input: 69331.76 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1134/1999 [08:55<06:48,  2.12it/s, est. speed input: 69331.93 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1135/1999 [08:56<06:47,  2.12it/s, est. speed input: 69332.10 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1136/1999 [08:56<06:47,  2.12it/s, est. speed input: 69332.09 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1137/1999 [08:57<06:47,  2.12it/s, est. speed input: 69331.96 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1138/1999 [08:57<06:46,  2.12it/s, est. speed input: 69332.00 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1139/1999 [08:58<06:46,  2.12it/s, est. speed input: 69332.10 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1140/1999 [08:58<06:45,  2.12it/s, est. speed input: 69332.31 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1141/1999 [08:59<06:46,  2.11it/s, est. speed input: 69331.79 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1142/1999 [08:59<06:45,  2.11it/s, est. speed input: 69331.82 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1143/1999 [09:00<06:44,  2.12it/s, est. speed input: 69332.02 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1144/1999 [09:00<06:43,  2.12it/s, est. speed input: 69332.06 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1145/1999 [09:01<06:43,  2.12it/s, est. speed input: 69332.12 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1146/1999 [09:01<06:42,  2.12it/s, est. speed input: 69332.26 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1147/1999 [09:02<06:41,  2.12it/s, est. speed input: 69332.49 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1148/1999 [09:02<06:41,  2.12it/s, est. speed input: 69332.47 toks/s, output: 2.12 toks/s]
Processed prompts:  57%|█████▋    | 1149/1999 [09:03<06:41,  2.12it/s, est. speed input: 69332.48 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1150/1999 [09:03<06:40,  2.12it/s, est. speed input: 69332.64 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1151/1999 [09:03<06:40,  2.12it/s, est. speed input: 69332.82 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1152/1999 [09:04<06:39,  2.12it/s, est. speed input: 69332.96 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1153/1999 [09:04<06:38,  2.12it/s, est. speed input: 69333.13 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1154/1999 [09:05<06:38,  2.12it/s, est. speed input: 69333.33 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1155/1999 [09:05<06:39,  2.11it/s, est. speed input: 69332.81 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1156/1999 [09:06<06:38,  2.12it/s, est. speed input: 69332.96 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1157/1999 [09:06<06:38,  2.11it/s, est. speed input: 69332.54 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1158/1999 [09:07<06:40,  2.10it/s, est. speed input: 69331.54 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1159/1999 [09:07<06:42,  2.09it/s, est. speed input: 69329.79 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1160/1999 [09:08<06:40,  2.10it/s, est. speed input: 69329.74 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1161/1999 [09:08<06:38,  2.10it/s, est. speed input: 69329.69 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1162/1999 [09:09<06:37,  2.11it/s, est. speed input: 69329.79 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1163/1999 [09:09<06:36,  2.11it/s, est. speed input: 69329.95 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1164/1999 [09:10<06:35,  2.11it/s, est. speed input: 69329.95 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1165/1999 [09:10<06:35,  2.11it/s, est. speed input: 69329.65 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1166/1999 [09:11<06:33,  2.11it/s, est. speed input: 69329.87 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1167/1999 [09:11<06:33,  2.11it/s, est. speed input: 69329.80 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1168/1999 [09:12<06:33,  2.11it/s, est. speed input: 69329.41 toks/s, output: 2.12 toks/s]
Processed prompts:  58%|█████▊    | 1169/1999 [09:12<06:32,  2.11it/s, est. speed input: 69329.51 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1170/1999 [09:12<06:32,  2.11it/s, est. speed input: 69329.24 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1171/1999 [09:13<06:32,  2.11it/s, est. speed input: 69328.81 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1172/1999 [09:13<06:31,  2.11it/s, est. speed input: 69328.83 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1173/1999 [09:14<06:31,  2.11it/s, est. speed input: 69328.82 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▊    | 1174/1999 [09:14<06:30,  2.11it/s, est. speed input: 69328.80 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1175/1999 [09:15<06:29,  2.11it/s, est. speed input: 69328.73 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1176/1999 [09:15<06:30,  2.11it/s, est. speed input: 69328.18 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1177/1999 [09:16<06:29,  2.11it/s, est. speed input: 69328.25 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1178/1999 [09:16<06:28,  2.11it/s, est. speed input: 69328.29 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1179/1999 [09:17<06:27,  2.12it/s, est. speed input: 69328.46 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1180/1999 [09:17<06:27,  2.12it/s, est. speed input: 69328.40 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1181/1999 [09:18<06:27,  2.11it/s, est. speed input: 69327.79 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1182/1999 [09:18<06:27,  2.11it/s, est. speed input: 69327.44 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1183/1999 [09:19<06:26,  2.11it/s, est. speed input: 69327.54 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1184/1999 [09:19<06:25,  2.11it/s, est. speed input: 69327.58 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1185/1999 [09:20<06:25,  2.11it/s, est. speed input: 69327.56 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1186/1999 [09:20<06:24,  2.11it/s, est. speed input: 69327.56 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1187/1999 [09:21<06:24,  2.11it/s, est. speed input: 69327.13 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1188/1999 [09:21<06:24,  2.11it/s, est. speed input: 69327.02 toks/s, output: 2.12 toks/s]
Processed prompts:  59%|█████▉    | 1189/1999 [09:21<06:23,  2.11it/s, est. speed input: 69327.15 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1190/1999 [09:22<06:22,  2.12it/s, est. speed input: 69327.27 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1191/1999 [09:22<06:21,  2.12it/s, est. speed input: 69327.40 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1192/1999 [09:23<06:22,  2.11it/s, est. speed input: 69326.85 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1193/1999 [09:23<06:21,  2.11it/s, est. speed input: 69326.79 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1194/1999 [09:24<06:20,  2.11it/s, est. speed input: 69326.82 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1195/1999 [09:24<06:26,  2.08it/s, est. speed input: 69323.80 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1196/1999 [09:25<06:23,  2.09it/s, est. speed input: 69323.99 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1197/1999 [09:25<06:21,  2.10it/s, est. speed input: 69324.13 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1198/1999 [09:26<06:21,  2.10it/s, est. speed input: 69323.64 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|█████▉    | 1199/1999 [09:26<06:19,  2.11it/s, est. speed input: 69323.79 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1200/1999 [09:27<06:19,  2.11it/s, est. speed input: 69323.62 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1201/1999 [09:27<06:18,  2.11it/s, est. speed input: 69323.62 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1202/1999 [09:28<06:17,  2.11it/s, est. speed input: 69323.61 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1203/1999 [09:28<06:17,  2.11it/s, est. speed input: 69323.42 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1204/1999 [09:29<06:16,  2.11it/s, est. speed input: 69323.56 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1205/1999 [09:29<06:15,  2.11it/s, est. speed input: 69323.55 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1206/1999 [09:30<06:15,  2.11it/s, est. speed input: 69323.52 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1207/1999 [09:30<06:14,  2.11it/s, est. speed input: 69323.53 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1208/1999 [09:31<06:14,  2.11it/s, est. speed input: 69323.31 toks/s, output: 2.12 toks/s]
Processed prompts:  60%|██████    | 1209/1999 [09:31<06:14,  2.11it/s, est. speed input: 69323.18 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1210/1999 [09:31<06:13,  2.11it/s, est. speed input: 69323.27 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1211/1999 [09:32<06:12,  2.12it/s, est. speed input: 69323.45 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1212/1999 [09:32<06:11,  2.12it/s, est. speed input: 69323.41 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1213/1999 [09:33<06:11,  2.12it/s, est. speed input: 69323.48 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1214/1999 [09:33<06:11,  2.12it/s, est. speed input: 69323.42 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1215/1999 [09:34<06:11,  2.11it/s, est. speed input: 69323.18 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1216/1999 [09:34<06:10,  2.11it/s, est. speed input: 69323.04 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1217/1999 [09:35<06:09,  2.11it/s, est. speed input: 69323.15 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1218/1999 [09:35<06:09,  2.11it/s, est. speed input: 69322.95 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1219/1999 [09:36<06:08,  2.12it/s, est. speed input: 69323.10 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1220/1999 [09:36<06:08,  2.12it/s, est. speed input: 69323.14 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1221/1999 [09:37<06:07,  2.12it/s, est. speed input: 69323.26 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1222/1999 [09:37<06:07,  2.12it/s, est. speed input: 69323.18 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1223/1999 [09:38<06:06,  2.12it/s, est. speed input: 69323.22 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████    | 1224/1999 [09:38<06:06,  2.12it/s, est. speed input: 69323.24 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1225/1999 [09:39<06:06,  2.11it/s, est. speed input: 69322.74 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1226/1999 [09:39<06:05,  2.11it/s, est. speed input: 69322.77 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1227/1999 [09:39<06:04,  2.12it/s, est. speed input: 69322.94 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1228/1999 [09:40<06:04,  2.11it/s, est. speed input: 69322.86 toks/s, output: 2.12 toks/s]
Processed prompts:  61%|██████▏   | 1229/1999 [09:40<06:04,  2.12it/s, est. speed input: 69322.89 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1230/1999 [09:41<06:03,  2.12it/s, est. speed input: 69323.09 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1231/1999 [09:41<06:03,  2.11it/s, est. speed input: 69322.64 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1232/1999 [09:42<06:02,  2.11it/s, est. speed input: 69322.70 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1233/1999 [09:42<06:01,  2.12it/s, est. speed input: 69322.97 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1234/1999 [09:43<06:01,  2.12it/s, est. speed input: 69323.09 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1235/1999 [09:43<06:01,  2.11it/s, est. speed input: 69322.88 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1236/1999 [09:44<06:00,  2.12it/s, est. speed input: 69323.11 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1237/1999 [09:44<06:00,  2.12it/s, est. speed input: 69322.96 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1238/1999 [09:45<05:59,  2.12it/s, est. speed input: 69322.99 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1239/1999 [09:45<05:59,  2.12it/s, est. speed input: 69323.03 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1240/1999 [09:46<05:58,  2.12it/s, est. speed input: 69323.07 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1241/1999 [09:46<05:58,  2.12it/s, est. speed input: 69323.15 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1242/1999 [09:47<05:57,  2.12it/s, est. speed input: 69323.32 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1243/1999 [09:47<05:56,  2.12it/s, est. speed input: 69323.41 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1244/1999 [09:48<05:56,  2.12it/s, est. speed input: 69323.57 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1245/1999 [09:48<05:55,  2.12it/s, est. speed input: 69323.66 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1246/1999 [09:48<05:55,  2.12it/s, est. speed input: 69323.60 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1247/1999 [09:49<05:56,  2.11it/s, est. speed input: 69323.15 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1248/1999 [09:49<05:55,  2.11it/s, est. speed input: 69323.18 toks/s, output: 2.12 toks/s]
Processed prompts:  62%|██████▏   | 1249/1999 [09:50<05:54,  2.11it/s, est. speed input: 69323.25 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1250/1999 [09:50<05:54,  2.11it/s, est. speed input: 69323.23 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1251/1999 [09:51<05:53,  2.11it/s, est. speed input: 69323.17 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1252/1999 [09:51<05:53,  2.11it/s, est. speed input: 69323.13 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1253/1999 [09:52<05:52,  2.12it/s, est. speed input: 69323.19 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1254/1999 [09:52<05:52,  2.12it/s, est. speed input: 69323.17 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1255/1999 [09:53<05:52,  2.11it/s, est. speed input: 69323.02 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1256/1999 [09:53<05:51,  2.12it/s, est. speed input: 69323.10 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1257/1999 [09:54<05:50,  2.12it/s, est. speed input: 69323.27 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1258/1999 [09:54<05:50,  2.12it/s, est. speed input: 69323.30 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1259/1999 [09:55<05:49,  2.12it/s, est. speed input: 69323.29 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1260/1999 [09:55<05:49,  2.12it/s, est. speed input: 69323.33 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1261/1999 [09:56<05:48,  2.12it/s, est. speed input: 69323.32 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1262/1999 [09:56<05:48,  2.12it/s, est. speed input: 69323.35 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1263/1999 [09:56<05:47,  2.12it/s, est. speed input: 69323.38 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1264/1999 [09:57<05:48,  2.11it/s, est. speed input: 69322.72 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1265/1999 [09:57<05:47,  2.11it/s, est. speed input: 69322.69 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1266/1999 [09:58<05:46,  2.11it/s, est. speed input: 69322.80 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1267/1999 [09:58<05:45,  2.12it/s, est. speed input: 69322.97 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1268/1999 [09:59<05:45,  2.12it/s, est. speed input: 69322.93 toks/s, output: 2.12 toks/s]
Processed prompts:  63%|██████▎   | 1269/1999 [09:59<05:44,  2.12it/s, est. speed input: 69323.01 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1270/1999 [10:00<05:44,  2.12it/s, est. speed input: 69323.16 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1271/1999 [10:00<05:43,  2.12it/s, est. speed input: 69323.09 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1272/1999 [10:01<05:43,  2.12it/s, est. speed input: 69323.21 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1273/1999 [10:01<05:42,  2.12it/s, est. speed input: 69323.30 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▎   | 1274/1999 [10:02<05:42,  2.12it/s, est. speed input: 69323.32 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1275/1999 [10:02<05:41,  2.12it/s, est. speed input: 69323.47 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1276/1999 [10:03<05:41,  2.12it/s, est. speed input: 69323.38 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1277/1999 [10:03<05:41,  2.12it/s, est. speed input: 69323.37 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1278/1999 [10:04<05:40,  2.12it/s, est. speed input: 69323.43 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1279/1999 [10:04<05:39,  2.12it/s, est. speed input: 69323.60 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1280/1999 [10:05<05:39,  2.12it/s, est. speed input: 69323.46 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1281/1999 [10:05<05:40,  2.11it/s, est. speed input: 69323.11 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1282/1999 [10:05<05:39,  2.11it/s, est. speed input: 69323.24 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1283/1999 [10:06<05:39,  2.11it/s, est. speed input: 69323.00 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1284/1999 [10:06<05:38,  2.11it/s, est. speed input: 69323.05 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1285/1999 [10:07<05:37,  2.12it/s, est. speed input: 69323.29 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1286/1999 [10:07<05:37,  2.11it/s, est. speed input: 69322.89 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1287/1999 [10:08<05:36,  2.11it/s, est. speed input: 69323.04 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1288/1999 [10:08<05:35,  2.12it/s, est. speed input: 69323.21 toks/s, output: 2.12 toks/s]
Processed prompts:  64%|██████▍   | 1289/1999 [10:09<05:35,  2.12it/s, est. speed input: 69323.32 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1290/1999 [10:09<05:34,  2.12it/s, est. speed input: 69323.39 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1291/1999 [10:10<05:34,  2.12it/s, est. speed input: 69323.49 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1292/1999 [10:10<05:33,  2.12it/s, est. speed input: 69323.71 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1293/1999 [10:11<05:33,  2.12it/s, est. speed input: 69323.77 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1294/1999 [10:11<05:32,  2.12it/s, est. speed input: 69323.74 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1295/1999 [10:12<05:32,  2.12it/s, est. speed input: 69323.97 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1296/1999 [10:12<05:31,  2.12it/s, est. speed input: 69324.16 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1297/1999 [10:13<05:32,  2.11it/s, est. speed input: 69323.69 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1298/1999 [10:13<05:31,  2.12it/s, est. speed input: 69323.82 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▍   | 1299/1999 [10:14<05:30,  2.12it/s, est. speed input: 69324.01 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1300/1999 [10:14<05:29,  2.12it/s, est. speed input: 69324.08 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1301/1999 [10:14<05:29,  2.12it/s, est. speed input: 69324.31 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1302/1999 [10:15<05:29,  2.11it/s, est. speed input: 69323.77 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1303/1999 [10:15<05:28,  2.12it/s, est. speed input: 69323.98 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1304/1999 [10:16<05:28,  2.12it/s, est. speed input: 69324.07 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1305/1999 [10:16<05:27,  2.12it/s, est. speed input: 69324.23 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1306/1999 [10:17<05:26,  2.12it/s, est. speed input: 69324.37 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1307/1999 [10:17<05:26,  2.12it/s, est. speed input: 69324.48 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1308/1999 [10:18<05:25,  2.12it/s, est. speed input: 69324.65 toks/s, output: 2.12 toks/s]
Processed prompts:  65%|██████▌   | 1309/1999 [10:18<05:25,  2.12it/s, est. speed input: 69324.69 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1310/1999 [10:19<05:25,  2.12it/s, est. speed input: 69324.74 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1311/1999 [10:19<05:24,  2.12it/s, est. speed input: 69324.90 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1312/1999 [10:20<05:24,  2.12it/s, est. speed input: 69324.88 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1313/1999 [10:20<05:24,  2.12it/s, est. speed input: 69324.77 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1314/1999 [10:21<05:23,  2.12it/s, est. speed input: 69324.87 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1315/1999 [10:21<05:27,  2.09it/s, est. speed input: 69322.55 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1316/1999 [10:22<05:25,  2.10it/s, est. speed input: 69322.68 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1317/1999 [10:22<05:23,  2.11it/s, est. speed input: 69322.96 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1318/1999 [10:22<05:22,  2.11it/s, est. speed input: 69323.21 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1319/1999 [10:23<05:21,  2.12it/s, est. speed input: 69323.34 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1320/1999 [10:23<05:21,  2.11it/s, est. speed input: 69323.19 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1321/1999 [10:24<05:20,  2.11it/s, est. speed input: 69323.26 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1322/1999 [10:24<05:20,  2.11it/s, est. speed input: 69322.95 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1323/1999 [10:25<05:19,  2.11it/s, est. speed input: 69323.12 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▌   | 1324/1999 [10:25<05:18,  2.12it/s, est. speed input: 69323.27 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 1325/1999 [10:26<05:18,  2.11it/s, est. speed input: 69323.08 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 1326/1999 [10:26<05:18,  2.12it/s, est. speed input: 69323.12 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 1327/1999 [10:27<05:17,  2.12it/s, est. speed input: 69323.31 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 1328/1999 [10:27<05:16,  2.12it/s, est. speed input: 69323.49 toks/s, output: 2.12 toks/s]
Processed prompts:  66%|██████▋   | 1329/1999 [10:28<05:17,  2.11it/s, est. speed input: 69323.07 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1330/1999 [10:28<05:16,  2.12it/s, est. speed input: 69323.28 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1331/1999 [10:29<05:15,  2.12it/s, est. speed input: 69323.39 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1332/1999 [10:29<05:14,  2.12it/s, est. speed input: 69323.52 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1333/1999 [10:30<05:14,  2.12it/s, est. speed input: 69323.67 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1334/1999 [10:30<05:13,  2.12it/s, est. speed input: 69323.89 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1335/1999 [10:31<05:13,  2.12it/s, est. speed input: 69323.83 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1336/1999 [10:31<05:12,  2.12it/s, est. speed input: 69323.88 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1337/1999 [10:31<05:12,  2.12it/s, est. speed input: 69324.01 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1338/1999 [10:32<05:12,  2.12it/s, est. speed input: 69323.81 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1339/1999 [10:32<05:11,  2.12it/s, est. speed input: 69323.90 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1340/1999 [10:33<05:10,  2.12it/s, est. speed input: 69324.17 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1341/1999 [10:33<05:11,  2.11it/s, est. speed input: 69323.81 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1342/1999 [10:34<05:10,  2.11it/s, est. speed input: 69323.71 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1343/1999 [10:34<05:10,  2.12it/s, est. speed input: 69323.88 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1344/1999 [10:35<05:09,  2.12it/s, est. speed input: 69323.95 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1345/1999 [10:35<05:08,  2.12it/s, est. speed input: 69324.12 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1346/1999 [10:36<05:08,  2.12it/s, est. speed input: 69324.21 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1347/1999 [10:36<05:07,  2.12it/s, est. speed input: 69324.31 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1348/1999 [10:37<05:07,  2.12it/s, est. speed input: 69324.40 toks/s, output: 2.12 toks/s]
Processed prompts:  67%|██████▋   | 1349/1999 [10:37<05:07,  2.12it/s, est. speed input: 69324.27 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1350/1999 [10:38<05:06,  2.12it/s, est. speed input: 69324.23 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1351/1999 [10:38<05:06,  2.12it/s, est. speed input: 69324.38 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1352/1999 [10:39<05:05,  2.12it/s, est. speed input: 69324.30 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1353/1999 [10:39<05:04,  2.12it/s, est. speed input: 69324.53 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1354/1999 [10:40<05:04,  2.12it/s, est. speed input: 69324.63 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1355/1999 [10:40<05:03,  2.12it/s, est. speed input: 69324.72 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1356/1999 [10:40<05:03,  2.12it/s, est. speed input: 69324.83 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1357/1999 [10:41<05:16,  2.03it/s, est. speed input: 69317.16 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1358/1999 [10:41<05:12,  2.05it/s, est. speed input: 69317.31 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1359/1999 [10:42<05:08,  2.07it/s, est. speed input: 69317.44 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1360/1999 [10:42<05:06,  2.09it/s, est. speed input: 69317.60 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1361/1999 [10:43<05:03,  2.10it/s, est. speed input: 69317.83 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1362/1999 [10:43<05:02,  2.10it/s, est. speed input: 69317.75 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1363/1999 [10:44<05:01,  2.11it/s, est. speed input: 69317.72 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1364/1999 [10:44<05:01,  2.11it/s, est. speed input: 69317.76 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1365/1999 [10:45<05:00,  2.11it/s, est. speed input: 69317.92 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1366/1999 [10:45<04:59,  2.12it/s, est. speed input: 69318.09 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1367/1999 [10:46<04:59,  2.11it/s, est. speed input: 69317.62 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1368/1999 [10:46<04:58,  2.11it/s, est. speed input: 69317.67 toks/s, output: 2.12 toks/s]
Processed prompts:  68%|██████▊   | 1369/1999 [10:47<04:57,  2.12it/s, est. speed input: 69317.87 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▊   | 1370/1999 [10:47<04:56,  2.12it/s, est. speed input: 69318.09 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▊   | 1371/1999 [10:48<04:56,  2.12it/s, est. speed input: 69318.31 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▊   | 1372/1999 [10:48<04:55,  2.12it/s, est. speed input: 69318.50 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▊   | 1373/1999 [10:49<04:55,  2.12it/s, est. speed input: 69318.52 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▊   | 1374/1999 [10:49<04:54,  2.12it/s, est. speed input: 69318.58 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1375/1999 [10:49<04:54,  2.12it/s, est. speed input: 69318.66 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1376/1999 [10:50<04:54,  2.12it/s, est. speed input: 69318.44 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1377/1999 [10:50<04:53,  2.12it/s, est. speed input: 69318.53 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1378/1999 [10:51<04:53,  2.12it/s, est. speed input: 69318.66 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1379/1999 [10:51<04:52,  2.12it/s, est. speed input: 69318.78 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1380/1999 [10:52<04:52,  2.11it/s, est. speed input: 69318.50 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1381/1999 [10:52<04:52,  2.12it/s, est. speed input: 69318.58 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1382/1999 [10:53<04:51,  2.12it/s, est. speed input: 69318.76 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1383/1999 [10:53<04:51,  2.11it/s, est. speed input: 69318.35 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1384/1999 [10:54<04:51,  2.11it/s, est. speed input: 69318.37 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1385/1999 [10:54<04:50,  2.12it/s, est. speed input: 69318.48 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1386/1999 [10:55<04:49,  2.12it/s, est. speed input: 69318.62 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1387/1999 [10:55<04:49,  2.12it/s, est. speed input: 69318.62 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1388/1999 [10:56<04:48,  2.12it/s, est. speed input: 69318.65 toks/s, output: 2.12 toks/s]
Processed prompts:  69%|██████▉   | 1389/1999 [10:56<04:48,  2.12it/s, est. speed input: 69318.79 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|██████▉   | 1390/1999 [10:57<04:47,  2.12it/s, est. speed input: 69318.84 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|██████▉   | 1391/1999 [10:57<04:47,  2.12it/s, est. speed input: 69318.90 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|██████▉   | 1392/1999 [10:58<04:46,  2.12it/s, est. speed input: 69319.07 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|██████▉   | 1393/1999 [10:58<04:46,  2.11it/s, est. speed input: 69318.80 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|██████▉   | 1394/1999 [10:58<04:46,  2.12it/s, est. speed input: 69318.83 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|██████▉   | 1395/1999 [10:59<04:45,  2.12it/s, est. speed input: 69318.95 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|██████▉   | 1396/1999 [10:59<04:44,  2.12it/s, est. speed input: 69319.11 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|██████▉   | 1397/1999 [11:00<04:44,  2.12it/s, est. speed input: 69319.23 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|██████▉   | 1398/1999 [11:00<04:43,  2.12it/s, est. speed input: 69319.43 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|██████▉   | 1399/1999 [11:01<04:43,  2.12it/s, est. speed input: 69319.44 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|███████   | 1400/1999 [11:01<04:42,  2.12it/s, est. speed input: 69319.46 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|███████   | 1401/1999 [11:02<04:42,  2.12it/s, est. speed input: 69319.63 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|███████   | 1402/1999 [11:02<04:41,  2.12it/s, est. speed input: 69319.63 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|███████   | 1403/1999 [11:03<04:41,  2.12it/s, est. speed input: 69319.74 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|███████   | 1404/1999 [11:03<04:40,  2.12it/s, est. speed input: 69319.83 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|███████   | 1405/1999 [11:04<04:40,  2.12it/s, est. speed input: 69319.93 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|███████   | 1406/1999 [11:04<04:39,  2.12it/s, est. speed input: 69319.97 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|███████   | 1407/1999 [11:05<04:39,  2.12it/s, est. speed input: 69320.07 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|███████   | 1408/1999 [11:05<04:38,  2.12it/s, est. speed input: 69320.11 toks/s, output: 2.12 toks/s]
Processed prompts:  70%|███████   | 1409/1999 [11:06<04:38,  2.12it/s, est. speed input: 69319.97 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1410/1999 [11:06<04:38,  2.12it/s, est. speed input: 69320.12 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1411/1999 [11:06<04:37,  2.12it/s, est. speed input: 69320.29 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1412/1999 [11:07<04:37,  2.11it/s, est. speed input: 69319.98 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1413/1999 [11:07<04:37,  2.11it/s, est. speed input: 69319.94 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1414/1999 [11:08<04:36,  2.12it/s, est. speed input: 69320.11 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1415/1999 [11:08<04:35,  2.12it/s, est. speed input: 69320.15 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1416/1999 [11:09<04:35,  2.12it/s, est. speed input: 69320.26 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1417/1999 [11:09<04:34,  2.12it/s, est. speed input: 69320.33 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1418/1999 [11:10<04:34,  2.12it/s, est. speed input: 69320.52 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1419/1999 [11:10<04:33,  2.12it/s, est. speed input: 69320.58 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1420/1999 [11:11<04:34,  2.11it/s, est. speed input: 69320.17 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1421/1999 [11:11<04:33,  2.12it/s, est. speed input: 69320.36 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1422/1999 [11:12<04:32,  2.12it/s, est. speed input: 69320.52 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1423/1999 [11:12<04:31,  2.12it/s, est. speed input: 69320.68 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████   | 1424/1999 [11:13<04:31,  2.12it/s, est. speed input: 69320.79 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████▏  | 1425/1999 [11:13<04:31,  2.12it/s, est. speed input: 69320.55 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████▏  | 1426/1999 [11:14<04:30,  2.12it/s, est. speed input: 69320.64 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████▏  | 1427/1999 [11:14<04:30,  2.12it/s, est. speed input: 69320.63 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████▏  | 1428/1999 [11:15<04:30,  2.11it/s, est. speed input: 69320.54 toks/s, output: 2.12 toks/s]
Processed prompts:  71%|███████▏  | 1429/1999 [11:15<04:29,  2.12it/s, est. speed input: 69320.61 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1430/1999 [11:15<04:28,  2.12it/s, est. speed input: 69320.70 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1431/1999 [11:16<04:28,  2.12it/s, est. speed input: 69320.70 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1432/1999 [11:16<04:28,  2.11it/s, est. speed input: 69320.56 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1433/1999 [11:17<04:27,  2.11it/s, est. speed input: 69320.55 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1434/1999 [11:17<04:26,  2.12it/s, est. speed input: 69320.65 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1435/1999 [11:18<04:26,  2.12it/s, est. speed input: 69320.84 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1436/1999 [11:18<04:25,  2.12it/s, est. speed input: 69320.99 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1437/1999 [11:19<04:25,  2.12it/s, est. speed input: 69321.15 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1438/1999 [11:19<04:25,  2.11it/s, est. speed input: 69320.82 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1439/1999 [11:20<04:24,  2.11it/s, est. speed input: 69320.80 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1440/1999 [11:20<04:23,  2.12it/s, est. speed input: 69320.99 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1441/1999 [11:21<04:23,  2.12it/s, est. speed input: 69321.00 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1442/1999 [11:21<04:22,  2.12it/s, est. speed input: 69321.16 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1443/1999 [11:22<04:22,  2.12it/s, est. speed input: 69321.35 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1444/1999 [11:22<04:21,  2.12it/s, est. speed input: 69321.37 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1445/1999 [11:23<04:21,  2.12it/s, est. speed input: 69321.36 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1446/1999 [11:23<04:21,  2.11it/s, est. speed input: 69321.09 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1447/1999 [11:23<04:20,  2.12it/s, est. speed input: 69321.23 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1448/1999 [11:24<04:20,  2.12it/s, est. speed input: 69321.27 toks/s, output: 2.12 toks/s]
Processed prompts:  72%|███████▏  | 1449/1999 [11:24<04:19,  2.12it/s, est. speed input: 69321.27 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1450/1999 [11:25<04:19,  2.12it/s, est. speed input: 69321.41 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1451/1999 [11:25<04:19,  2.11it/s, est. speed input: 69321.16 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1452/1999 [11:26<04:18,  2.12it/s, est. speed input: 69321.22 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1453/1999 [11:26<04:18,  2.11it/s, est. speed input: 69321.13 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1454/1999 [11:27<04:18,  2.11it/s, est. speed input: 69320.93 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1455/1999 [11:27<04:17,  2.11it/s, est. speed input: 69321.04 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1456/1999 [11:28<04:16,  2.12it/s, est. speed input: 69321.18 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1457/1999 [11:28<04:15,  2.12it/s, est. speed input: 69321.33 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1458/1999 [11:29<04:15,  2.12it/s, est. speed input: 69321.30 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1459/1999 [11:29<04:15,  2.12it/s, est. speed input: 69321.28 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1460/1999 [11:30<04:14,  2.12it/s, est. speed input: 69321.43 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1461/1999 [11:30<04:14,  2.12it/s, est. speed input: 69321.37 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1462/1999 [11:31<04:13,  2.12it/s, est. speed input: 69321.43 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1463/1999 [11:31<04:13,  2.12it/s, est. speed input: 69321.55 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1464/1999 [11:32<04:12,  2.12it/s, est. speed input: 69321.69 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1465/1999 [11:32<04:12,  2.12it/s, est. speed input: 69321.72 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1466/1999 [11:32<04:11,  2.12it/s, est. speed input: 69321.94 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1467/1999 [11:33<04:11,  2.12it/s, est. speed input: 69321.66 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1468/1999 [11:33<04:10,  2.12it/s, est. speed input: 69321.91 toks/s, output: 2.12 toks/s]
Processed prompts:  73%|███████▎  | 1469/1999 [11:34<04:10,  2.12it/s, est. speed input: 69321.92 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▎  | 1470/1999 [11:34<04:09,  2.12it/s, est. speed input: 69322.04 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▎  | 1471/1999 [11:35<04:09,  2.12it/s, est. speed input: 69322.12 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▎  | 1472/1999 [11:35<04:08,  2.12it/s, est. speed input: 69322.14 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▎  | 1473/1999 [11:36<04:08,  2.12it/s, est. speed input: 69322.23 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▎  | 1474/1999 [11:36<04:07,  2.12it/s, est. speed input: 69322.36 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1475/1999 [11:37<04:07,  2.12it/s, est. speed input: 69322.50 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1476/1999 [11:37<04:06,  2.12it/s, est. speed input: 69322.55 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1477/1999 [11:38<04:06,  2.12it/s, est. speed input: 69322.60 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1478/1999 [11:38<04:05,  2.12it/s, est. speed input: 69322.69 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1479/1999 [11:39<04:05,  2.12it/s, est. speed input: 69322.77 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1480/1999 [11:39<04:05,  2.12it/s, est. speed input: 69322.67 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1481/1999 [11:40<04:05,  2.11it/s, est. speed input: 69322.49 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1482/1999 [11:40<04:04,  2.12it/s, est. speed input: 69322.65 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1483/1999 [11:40<04:03,  2.12it/s, est. speed input: 69322.75 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1484/1999 [11:41<04:03,  2.12it/s, est. speed input: 69322.80 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1485/1999 [11:41<04:02,  2.12it/s, est. speed input: 69322.87 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1486/1999 [11:42<04:02,  2.12it/s, est. speed input: 69323.03 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1487/1999 [11:42<04:01,  2.12it/s, est. speed input: 69323.03 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1488/1999 [11:43<04:01,  2.12it/s, est. speed input: 69323.11 toks/s, output: 2.12 toks/s]
Processed prompts:  74%|███████▍  | 1489/1999 [11:43<04:00,  2.12it/s, est. speed input: 69323.21 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▍  | 1490/1999 [11:44<04:04,  2.08it/s, est. speed input: 69320.66 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▍  | 1491/1999 [11:44<04:02,  2.10it/s, est. speed input: 69320.81 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▍  | 1492/1999 [11:45<04:00,  2.10it/s, est. speed input: 69321.01 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▍  | 1493/1999 [11:45<04:01,  2.10it/s, est. speed input: 69320.32 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▍  | 1494/1999 [11:46<03:59,  2.11it/s, est. speed input: 69320.49 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▍  | 1495/1999 [11:46<03:58,  2.11it/s, est. speed input: 69320.65 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▍  | 1496/1999 [11:47<03:57,  2.11it/s, est. speed input: 69320.75 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▍  | 1497/1999 [11:47<03:57,  2.12it/s, est. speed input: 69320.98 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▍  | 1498/1999 [11:48<03:56,  2.12it/s, est. speed input: 69321.20 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▍  | 1499/1999 [11:48<03:55,  2.12it/s, est. speed input: 69321.42 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▌  | 1500/1999 [11:49<03:55,  2.12it/s, est. speed input: 69321.61 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▌  | 1501/1999 [11:49<03:54,  2.12it/s, est. speed input: 69321.62 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▌  | 1502/1999 [11:49<03:54,  2.12it/s, est. speed input: 69321.73 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▌  | 1503/1999 [11:50<03:54,  2.11it/s, est. speed input: 69321.28 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▌  | 1504/1999 [11:50<03:54,  2.11it/s, est. speed input: 69321.28 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▌  | 1505/1999 [11:51<03:53,  2.12it/s, est. speed input: 69321.49 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▌  | 1506/1999 [11:51<03:52,  2.12it/s, est. speed input: 69321.68 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▌  | 1507/1999 [11:52<03:52,  2.12it/s, est. speed input: 69321.86 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▌  | 1508/1999 [11:52<03:51,  2.12it/s, est. speed input: 69321.89 toks/s, output: 2.12 toks/s]
Processed prompts:  75%|███████▌  | 1509/1999 [11:53<03:51,  2.12it/s, est. speed input: 69321.86 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1510/1999 [11:53<03:50,  2.12it/s, est. speed input: 69321.85 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1511/1999 [11:54<03:50,  2.12it/s, est. speed input: 69321.85 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1512/1999 [11:54<03:49,  2.12it/s, est. speed input: 69322.01 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1513/1999 [11:55<03:49,  2.12it/s, est. speed input: 69322.02 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1514/1999 [11:55<03:49,  2.12it/s, est. speed input: 69322.06 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1515/1999 [11:56<03:48,  2.12it/s, est. speed input: 69322.11 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1516/1999 [11:56<03:47,  2.12it/s, est. speed input: 69322.31 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1517/1999 [11:57<03:47,  2.12it/s, est. speed input: 69322.33 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1518/1999 [11:57<03:47,  2.12it/s, est. speed input: 69322.33 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1519/1999 [11:58<03:46,  2.12it/s, est. speed input: 69322.53 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1520/1999 [11:58<03:46,  2.12it/s, est. speed input: 69322.53 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1521/1999 [11:58<03:45,  2.12it/s, est. speed input: 69322.52 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1522/1999 [11:59<03:45,  2.12it/s, est. speed input: 69322.61 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1523/1999 [11:59<03:44,  2.12it/s, est. speed input: 69322.69 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▌  | 1524/1999 [12:00<03:44,  2.12it/s, est. speed input: 69322.79 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▋  | 1525/1999 [12:00<03:43,  2.12it/s, est. speed input: 69322.92 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▋  | 1526/1999 [12:01<03:43,  2.12it/s, est. speed input: 69322.69 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▋  | 1527/1999 [12:01<03:43,  2.11it/s, est. speed input: 69322.62 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▋  | 1528/1999 [12:02<03:42,  2.12it/s, est. speed input: 69322.74 toks/s, output: 2.12 toks/s]
Processed prompts:  76%|███████▋  | 1529/1999 [12:02<03:41,  2.12it/s, est. speed input: 69322.85 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1530/1999 [12:03<03:41,  2.12it/s, est. speed input: 69322.94 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1531/1999 [12:03<03:40,  2.12it/s, est. speed input: 69323.04 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1532/1999 [12:04<03:40,  2.12it/s, est. speed input: 69323.10 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1533/1999 [12:04<03:40,  2.12it/s, est. speed input: 69323.01 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1534/1999 [12:05<03:39,  2.12it/s, est. speed input: 69323.05 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1535/1999 [12:05<03:39,  2.12it/s, est. speed input: 69323.20 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1536/1999 [12:06<03:38,  2.12it/s, est. speed input: 69323.20 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1537/1999 [12:06<03:38,  2.12it/s, est. speed input: 69323.19 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1538/1999 [12:06<03:37,  2.12it/s, est. speed input: 69323.26 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1539/1999 [12:07<03:37,  2.11it/s, est. speed input: 69322.87 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1540/1999 [12:07<03:37,  2.11it/s, est. speed input: 69322.84 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1541/1999 [12:08<03:36,  2.12it/s, est. speed input: 69323.02 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1542/1999 [12:08<03:35,  2.12it/s, est. speed input: 69323.15 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1543/1999 [12:09<03:36,  2.11it/s, est. speed input: 69322.72 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1544/1999 [12:09<03:35,  2.11it/s, est. speed input: 69322.76 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1545/1999 [12:10<03:34,  2.12it/s, est. speed input: 69322.90 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1546/1999 [12:10<03:33,  2.12it/s, est. speed input: 69323.03 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1547/1999 [12:11<03:33,  2.12it/s, est. speed input: 69323.04 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1548/1999 [12:11<03:32,  2.12it/s, est. speed input: 69323.10 toks/s, output: 2.12 toks/s]
Processed prompts:  77%|███████▋  | 1549/1999 [12:12<03:32,  2.12it/s, est. speed input: 69323.24 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1550/1999 [12:12<03:31,  2.12it/s, est. speed input: 69323.28 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1551/1999 [12:13<03:31,  2.12it/s, est. speed input: 69323.34 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1552/1999 [12:13<03:31,  2.11it/s, est. speed input: 69322.89 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1553/1999 [12:14<03:31,  2.11it/s, est. speed input: 69322.84 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1554/1999 [12:14<03:30,  2.11it/s, est. speed input: 69322.95 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1555/1999 [12:15<03:30,  2.11it/s, est. speed input: 69322.89 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1556/1999 [12:15<03:29,  2.11it/s, est. speed input: 69322.92 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1557/1999 [12:15<03:28,  2.12it/s, est. speed input: 69323.12 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1558/1999 [12:16<03:28,  2.12it/s, est. speed input: 69323.27 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1559/1999 [12:16<03:27,  2.12it/s, est. speed input: 69323.30 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1560/1999 [12:17<03:27,  2.12it/s, est. speed input: 69323.28 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1561/1999 [12:17<03:26,  2.12it/s, est. speed input: 69323.40 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1562/1999 [12:18<03:26,  2.11it/s, est. speed input: 69323.21 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1563/1999 [12:18<03:26,  2.11it/s, est. speed input: 69323.19 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1564/1999 [12:19<03:25,  2.12it/s, est. speed input: 69323.29 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1565/1999 [12:19<03:24,  2.12it/s, est. speed input: 69323.38 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1566/1999 [12:20<03:24,  2.12it/s, est. speed input: 69323.49 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1567/1999 [12:20<03:23,  2.12it/s, est. speed input: 69323.59 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1568/1999 [12:21<03:23,  2.12it/s, est. speed input: 69323.73 toks/s, output: 2.12 toks/s]
Processed prompts:  78%|███████▊  | 1569/1999 [12:21<03:23,  2.11it/s, est. speed input: 69323.34 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▊  | 1570/1999 [12:22<03:22,  2.11it/s, est. speed input: 69323.34 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▊  | 1571/1999 [12:22<03:22,  2.11it/s, est. speed input: 69323.38 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▊  | 1572/1999 [12:23<03:21,  2.12it/s, est. speed input: 69323.45 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▊  | 1573/1999 [12:23<03:21,  2.12it/s, est. speed input: 69323.61 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▊  | 1574/1999 [12:23<03:20,  2.12it/s, est. speed input: 69323.72 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1575/1999 [12:24<03:19,  2.12it/s, est. speed input: 69323.92 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1576/1999 [12:24<03:19,  2.12it/s, est. speed input: 69323.99 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1577/1999 [12:25<03:19,  2.12it/s, est. speed input: 69324.07 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1578/1999 [12:25<03:19,  2.11it/s, est. speed input: 69323.65 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1579/1999 [12:26<03:18,  2.11it/s, est. speed input: 69323.57 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1580/1999 [12:26<03:18,  2.12it/s, est. speed input: 69323.70 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1581/1999 [12:27<03:17,  2.12it/s, est. speed input: 69323.80 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1582/1999 [12:27<03:16,  2.12it/s, est. speed input: 69323.83 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1583/1999 [12:28<03:16,  2.12it/s, est. speed input: 69323.88 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1584/1999 [12:28<03:16,  2.12it/s, est. speed input: 69323.86 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1585/1999 [12:29<03:15,  2.12it/s, est. speed input: 69323.91 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1586/1999 [12:29<03:15,  2.12it/s, est. speed input: 69323.91 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1587/1999 [12:30<03:14,  2.12it/s, est. speed input: 69324.02 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1588/1999 [12:30<03:14,  2.12it/s, est. speed input: 69324.08 toks/s, output: 2.12 toks/s]
Processed prompts:  79%|███████▉  | 1589/1999 [12:31<03:13,  2.12it/s, est. speed input: 69324.11 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|███████▉  | 1590/1999 [12:31<03:12,  2.12it/s, est. speed input: 69324.30 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|███████▉  | 1591/1999 [12:32<03:12,  2.12it/s, est. speed input: 69324.44 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|███████▉  | 1592/1999 [12:32<03:12,  2.12it/s, est. speed input: 69324.45 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|███████▉  | 1593/1999 [12:32<03:11,  2.12it/s, est. speed input: 69324.43 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|███████▉  | 1594/1999 [12:33<03:11,  2.12it/s, est. speed input: 69324.54 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|███████▉  | 1595/1999 [12:33<03:10,  2.12it/s, est. speed input: 69324.45 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|███████▉  | 1596/1999 [12:34<03:10,  2.12it/s, est. speed input: 69324.51 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|███████▉  | 1597/1999 [12:34<03:09,  2.12it/s, est. speed input: 69324.57 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|███████▉  | 1598/1999 [12:35<03:09,  2.12it/s, est. speed input: 69324.67 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|███████▉  | 1599/1999 [12:35<03:08,  2.12it/s, est. speed input: 69324.72 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|████████  | 1600/1999 [12:36<03:08,  2.12it/s, est. speed input: 69324.82 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|████████  | 1601/1999 [12:36<03:07,  2.12it/s, est. speed input: 69324.82 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|████████  | 1602/1999 [12:37<03:07,  2.12it/s, est. speed input: 69324.75 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|████████  | 1603/1999 [12:37<03:07,  2.12it/s, est. speed input: 69324.85 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|████████  | 1604/1999 [12:38<03:06,  2.12it/s, est. speed input: 69325.05 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|████████  | 1605/1999 [12:38<03:06,  2.12it/s, est. speed input: 69324.85 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|████████  | 1606/1999 [12:39<03:05,  2.12it/s, est. speed input: 69324.91 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|████████  | 1607/1999 [12:39<03:05,  2.12it/s, est. speed input: 69325.03 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|████████  | 1608/1999 [12:40<03:04,  2.12it/s, est. speed input: 69325.00 toks/s, output: 2.12 toks/s]
Processed prompts:  80%|████████  | 1609/1999 [12:40<03:04,  2.11it/s, est. speed input: 69324.82 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1610/1999 [12:41<03:03,  2.12it/s, est. speed input: 69324.89 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1611/1999 [12:41<03:03,  2.12it/s, est. speed input: 69325.05 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1612/1999 [12:41<03:02,  2.12it/s, est. speed input: 69325.07 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1613/1999 [12:42<03:02,  2.12it/s, est. speed input: 69325.10 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1614/1999 [12:42<03:01,  2.12it/s, est. speed input: 69325.19 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1615/1999 [12:43<03:01,  2.11it/s, est. speed input: 69324.97 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1616/1999 [12:43<03:01,  2.11it/s, est. speed input: 69324.94 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1617/1999 [12:44<03:00,  2.11it/s, est. speed input: 69324.95 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1618/1999 [12:44<03:00,  2.11it/s, est. speed input: 69324.94 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1619/1999 [12:45<02:59,  2.12it/s, est. speed input: 69324.99 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1620/1999 [12:45<02:59,  2.11it/s, est. speed input: 69324.92 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1621/1999 [12:46<02:58,  2.11it/s, est. speed input: 69324.70 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1622/1999 [12:46<02:58,  2.11it/s, est. speed input: 69324.81 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1623/1999 [12:47<02:57,  2.12it/s, est. speed input: 69324.91 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████  | 1624/1999 [12:47<02:57,  2.12it/s, est. speed input: 69324.85 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████▏ | 1625/1999 [12:48<02:56,  2.12it/s, est. speed input: 69324.83 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████▏ | 1626/1999 [12:48<02:56,  2.12it/s, est. speed input: 69324.88 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████▏ | 1627/1999 [12:49<02:55,  2.12it/s, est. speed input: 69325.05 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████▏ | 1628/1999 [12:49<02:55,  2.12it/s, est. speed input: 69324.98 toks/s, output: 2.12 toks/s]
Processed prompts:  81%|████████▏ | 1629/1999 [12:49<02:54,  2.12it/s, est. speed input: 69324.90 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1630/1999 [12:50<02:54,  2.11it/s, est. speed input: 69324.65 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1631/1999 [12:50<02:53,  2.12it/s, est. speed input: 69324.82 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1632/1999 [12:51<02:53,  2.12it/s, est. speed input: 69324.88 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1633/1999 [12:51<02:52,  2.12it/s, est. speed input: 69325.01 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1634/1999 [12:52<02:52,  2.12it/s, est. speed input: 69325.12 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1635/1999 [12:52<02:51,  2.12it/s, est. speed input: 69325.12 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1636/1999 [12:53<02:51,  2.12it/s, est. speed input: 69325.16 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1637/1999 [12:53<02:51,  2.11it/s, est. speed input: 69324.92 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1638/1999 [12:54<02:50,  2.12it/s, est. speed input: 69324.98 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1639/1999 [12:54<02:50,  2.11it/s, est. speed input: 69324.94 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1640/1999 [12:55<02:49,  2.11it/s, est. speed input: 69324.87 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1641/1999 [12:55<02:49,  2.11it/s, est. speed input: 69324.89 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1642/1999 [12:56<02:48,  2.12it/s, est. speed input: 69325.07 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1643/1999 [12:56<02:48,  2.12it/s, est. speed input: 69325.20 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1644/1999 [12:57<02:47,  2.12it/s, est. speed input: 69325.30 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1645/1999 [12:57<02:47,  2.12it/s, est. speed input: 69325.39 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1646/1999 [12:58<02:46,  2.12it/s, est. speed input: 69325.55 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1647/1999 [12:58<02:45,  2.12it/s, est. speed input: 69325.67 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1648/1999 [12:58<02:45,  2.12it/s, est. speed input: 69325.77 toks/s, output: 2.12 toks/s]
Processed prompts:  82%|████████▏ | 1649/1999 [12:59<02:45,  2.12it/s, est. speed input: 69325.79 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1650/1999 [12:59<02:44,  2.12it/s, est. speed input: 69325.64 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1651/1999 [13:00<02:44,  2.11it/s, est. speed input: 69325.54 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1652/1999 [13:00<02:43,  2.12it/s, est. speed input: 69325.61 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1653/1999 [13:01<02:43,  2.12it/s, est. speed input: 69325.63 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1654/1999 [13:01<02:42,  2.12it/s, est. speed input: 69325.78 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1655/1999 [13:02<02:42,  2.12it/s, est. speed input: 69325.73 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1656/1999 [13:02<02:42,  2.12it/s, est. speed input: 69325.78 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1657/1999 [13:03<02:41,  2.12it/s, est. speed input: 69325.81 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1658/1999 [13:03<02:41,  2.11it/s, est. speed input: 69325.70 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1659/1999 [13:04<02:40,  2.12it/s, est. speed input: 69325.77 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1660/1999 [13:04<02:40,  2.12it/s, est. speed input: 69325.81 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1661/1999 [13:05<02:39,  2.12it/s, est. speed input: 69325.86 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1662/1999 [13:05<02:38,  2.12it/s, est. speed input: 69326.06 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1663/1999 [13:06<02:38,  2.12it/s, est. speed input: 69326.15 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1664/1999 [13:06<02:38,  2.12it/s, est. speed input: 69326.15 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1665/1999 [13:06<02:37,  2.12it/s, est. speed input: 69326.25 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1666/1999 [13:07<02:37,  2.12it/s, est. speed input: 69326.36 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1667/1999 [13:07<02:37,  2.11it/s, est. speed input: 69326.04 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1668/1999 [13:08<02:36,  2.11it/s, est. speed input: 69326.07 toks/s, output: 2.12 toks/s]
Processed prompts:  83%|████████▎ | 1669/1999 [13:08<02:35,  2.12it/s, est. speed input: 69326.31 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▎ | 1670/1999 [13:09<02:35,  2.12it/s, est. speed input: 69326.32 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▎ | 1671/1999 [13:09<02:34,  2.12it/s, est. speed input: 69326.48 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▎ | 1672/1999 [13:10<02:34,  2.12it/s, est. speed input: 69326.63 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▎ | 1673/1999 [13:10<02:33,  2.12it/s, est. speed input: 69326.76 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▎ | 1674/1999 [13:11<02:33,  2.12it/s, est. speed input: 69326.89 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1675/1999 [13:11<02:32,  2.12it/s, est. speed input: 69326.86 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1676/1999 [13:12<02:32,  2.12it/s, est. speed input: 69326.66 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1677/1999 [13:12<02:32,  2.12it/s, est. speed input: 69326.75 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1678/1999 [13:13<02:31,  2.12it/s, est. speed input: 69326.70 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1679/1999 [13:13<02:31,  2.12it/s, est. speed input: 69326.72 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1680/1999 [13:14<02:30,  2.12it/s, est. speed input: 69326.76 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1681/1999 [13:14<02:30,  2.11it/s, est. speed input: 69326.65 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1682/1999 [13:15<02:29,  2.12it/s, est. speed input: 69326.69 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1683/1999 [13:15<02:29,  2.12it/s, est. speed input: 69326.74 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1684/1999 [13:15<02:28,  2.12it/s, est. speed input: 69326.80 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1685/1999 [13:16<02:28,  2.12it/s, est. speed input: 69326.88 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1686/1999 [13:16<02:27,  2.12it/s, est. speed input: 69326.77 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1687/1999 [13:17<02:27,  2.12it/s, est. speed input: 69326.76 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1688/1999 [13:17<02:26,  2.12it/s, est. speed input: 69326.83 toks/s, output: 2.12 toks/s]
Processed prompts:  84%|████████▍ | 1689/1999 [13:18<02:26,  2.12it/s, est. speed input: 69326.97 toks/s, output: 2.12 toks/s]
Processed prompts:  85%|████████▍ | 1690/1999 [13:18<02:25,  2.12it/s, est. speed input: 69326.99 toks/s, output: 2.12 toks/s]
Processed prompts:  85%|████████▍ | 1691/1999 [13:19<02:25,  2.12it/s, est. speed input: 69326.99 toks/s, output: 2.12 toks/s]
Processed prompts:  85%|████████▍ | 1692/1999 [13:19<02:25,  2.12it/s, est. speed input: 69326.98 toks/s, output: 2.12 toks/s]
Processed prompts:  85%|████████▍ | 1693/1999 [13:22<05:53,  1.15s/it, est. speed input: 69130.76 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 1694/1999 [13:22<04:49,  1.05it/s, est. speed input: 69131.12 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 1695/1999 [13:23<04:04,  1.24it/s, est. speed input: 69131.51 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 1696/1999 [13:23<03:33,  1.42it/s, est. speed input: 69131.88 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 1697/1999 [13:24<03:11,  1.58it/s, est. speed input: 69132.23 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 1698/1999 [13:24<02:56,  1.71it/s, est. speed input: 69132.58 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▍ | 1699/1999 [13:25<02:45,  1.82it/s, est. speed input: 69132.90 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 1700/1999 [13:25<02:37,  1.90it/s, est. speed input: 69133.21 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 1701/1999 [13:26<02:31,  1.96it/s, est. speed input: 69133.57 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 1702/1999 [13:26<02:27,  2.01it/s, est. speed input: 69133.90 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 1703/1999 [13:27<02:24,  2.04it/s, est. speed input: 69134.22 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 1704/1999 [13:27<02:22,  2.07it/s, est. speed input: 69134.51 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 1705/1999 [13:28<02:21,  2.08it/s, est. speed input: 69134.80 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 1706/1999 [13:28<02:19,  2.10it/s, est. speed input: 69135.10 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 1707/1999 [13:29<02:18,  2.10it/s, est. speed input: 69135.37 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 1708/1999 [13:29<02:17,  2.11it/s, est. speed input: 69135.67 toks/s, output: 2.11 toks/s]
Processed prompts:  85%|████████▌ | 1709/1999 [13:30<02:17,  2.12it/s, est. speed input: 69135.99 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1710/1999 [13:30<02:16,  2.12it/s, est. speed input: 69136.26 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1711/1999 [13:30<02:15,  2.12it/s, est. speed input: 69136.55 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1712/1999 [13:31<02:15,  2.12it/s, est. speed input: 69136.82 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1713/1999 [13:31<02:14,  2.12it/s, est. speed input: 69137.10 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1714/1999 [13:32<02:14,  2.12it/s, est. speed input: 69137.38 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1715/1999 [13:32<02:13,  2.12it/s, est. speed input: 69137.66 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1716/1999 [13:33<02:13,  2.12it/s, est. speed input: 69137.94 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1717/1999 [13:33<02:12,  2.12it/s, est. speed input: 69138.23 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1718/1999 [13:34<02:12,  2.12it/s, est. speed input: 69138.54 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1719/1999 [13:34<02:11,  2.13it/s, est. speed input: 69138.87 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1720/1999 [13:35<02:11,  2.13it/s, est. speed input: 69139.15 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1721/1999 [13:35<02:10,  2.12it/s, est. speed input: 69139.40 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1722/1999 [13:36<02:10,  2.12it/s, est. speed input: 69139.68 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1723/1999 [13:36<02:09,  2.13it/s, est. speed input: 69139.98 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▌ | 1724/1999 [13:37<02:09,  2.13it/s, est. speed input: 69140.31 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 1725/1999 [13:37<02:08,  2.13it/s, est. speed input: 69140.63 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 1726/1999 [13:38<02:08,  2.13it/s, est. speed input: 69140.87 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 1727/1999 [13:38<02:07,  2.13it/s, est. speed input: 69141.16 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 1728/1999 [13:38<02:07,  2.13it/s, est. speed input: 69141.46 toks/s, output: 2.11 toks/s]
Processed prompts:  86%|████████▋ | 1729/1999 [13:39<02:07,  2.13it/s, est. speed input: 69141.76 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1730/1999 [13:39<02:06,  2.12it/s, est. speed input: 69141.99 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1731/1999 [13:40<02:06,  2.12it/s, est. speed input: 69142.15 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1732/1999 [13:40<02:05,  2.12it/s, est. speed input: 69142.37 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1733/1999 [13:41<02:05,  2.12it/s, est. speed input: 69142.63 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1734/1999 [13:41<02:04,  2.12it/s, est. speed input: 69142.94 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1735/1999 [13:42<02:04,  2.12it/s, est. speed input: 69143.25 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1736/1999 [13:42<02:03,  2.12it/s, est. speed input: 69143.50 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1737/1999 [13:43<02:03,  2.12it/s, est. speed input: 69143.80 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1738/1999 [13:43<02:02,  2.12it/s, est. speed input: 69144.08 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1739/1999 [13:44<02:02,  2.12it/s, est. speed input: 69144.25 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1740/1999 [13:44<02:02,  2.12it/s, est. speed input: 69144.43 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1741/1999 [13:45<02:01,  2.12it/s, est. speed input: 69144.66 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1742/1999 [13:45<02:01,  2.12it/s, est. speed input: 69144.95 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1743/1999 [13:46<02:00,  2.12it/s, est. speed input: 69144.83 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1744/1999 [13:46<02:00,  2.12it/s, est. speed input: 69145.03 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1745/1999 [13:46<01:59,  2.12it/s, est. speed input: 69145.24 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1746/1999 [13:47<01:59,  2.12it/s, est. speed input: 69145.54 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1747/1999 [13:47<01:58,  2.12it/s, est. speed input: 69145.82 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1748/1999 [13:48<01:58,  2.12it/s, est. speed input: 69146.12 toks/s, output: 2.11 toks/s]
Processed prompts:  87%|████████▋ | 1749/1999 [13:48<01:57,  2.12it/s, est. speed input: 69146.41 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1750/1999 [13:49<01:57,  2.12it/s, est. speed input: 69146.65 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1751/1999 [13:49<01:56,  2.12it/s, est. speed input: 69146.94 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1752/1999 [13:50<01:57,  2.10it/s, est. speed input: 69145.42 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1753/1999 [13:50<01:56,  2.10it/s, est. speed input: 69145.62 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1754/1999 [13:51<01:56,  2.11it/s, est. speed input: 69145.92 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1755/1999 [13:51<01:55,  2.10it/s, est. speed input: 69145.56 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1756/1999 [13:52<01:55,  2.11it/s, est. speed input: 69145.85 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1757/1999 [13:52<01:54,  2.11it/s, est. speed input: 69146.12 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1758/1999 [13:53<01:53,  2.12it/s, est. speed input: 69146.43 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1759/1999 [13:53<01:53,  2.12it/s, est. speed input: 69146.71 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1760/1999 [13:54<01:52,  2.12it/s, est. speed input: 69146.87 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1761/1999 [13:54<01:52,  2.12it/s, est. speed input: 69147.16 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1762/1999 [13:54<01:52,  2.12it/s, est. speed input: 69147.02 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1763/1999 [13:55<01:51,  2.12it/s, est. speed input: 69147.09 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1764/1999 [13:55<01:51,  2.12it/s, est. speed input: 69147.27 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1765/1999 [13:56<01:50,  2.12it/s, est. speed input: 69147.46 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1766/1999 [13:56<01:49,  2.12it/s, est. speed input: 69147.65 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1767/1999 [13:57<01:49,  2.12it/s, est. speed input: 69147.86 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1768/1999 [13:57<01:48,  2.12it/s, est. speed input: 69148.05 toks/s, output: 2.11 toks/s]
Processed prompts:  88%|████████▊ | 1769/1999 [13:58<01:48,  2.11it/s, est. speed input: 69147.91 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 1770/1999 [13:58<01:48,  2.12it/s, est. speed input: 69148.14 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 1771/1999 [13:59<01:47,  2.12it/s, est. speed input: 69148.40 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 1772/1999 [13:59<01:47,  2.12it/s, est. speed input: 69148.59 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 1773/1999 [14:00<01:46,  2.12it/s, est. speed input: 69148.72 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▊ | 1774/1999 [14:00<01:46,  2.12it/s, est. speed input: 69148.86 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1775/1999 [14:01<01:45,  2.12it/s, est. speed input: 69148.88 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1776/1999 [14:01<01:45,  2.12it/s, est. speed input: 69148.97 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1777/1999 [14:02<01:44,  2.12it/s, est. speed input: 69149.18 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1778/1999 [14:02<01:44,  2.12it/s, est. speed input: 69149.39 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1779/1999 [14:03<01:43,  2.12it/s, est. speed input: 69149.50 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1780/1999 [14:03<01:43,  2.12it/s, est. speed input: 69149.79 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1781/1999 [14:03<01:42,  2.12it/s, est. speed input: 69150.02 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1782/1999 [14:04<01:42,  2.12it/s, est. speed input: 69149.90 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1783/1999 [14:04<01:42,  2.12it/s, est. speed input: 69150.01 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1784/1999 [14:05<01:41,  2.12it/s, est. speed input: 69150.16 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1785/1999 [14:05<01:41,  2.12it/s, est. speed input: 69150.36 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1786/1999 [14:06<01:40,  2.12it/s, est. speed input: 69150.53 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1787/1999 [14:06<01:40,  2.12it/s, est. speed input: 69150.76 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1788/1999 [14:07<01:39,  2.12it/s, est. speed input: 69150.96 toks/s, output: 2.11 toks/s]
Processed prompts:  89%|████████▉ | 1789/1999 [14:07<01:39,  2.12it/s, est. speed input: 69151.06 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 1790/1999 [14:08<01:38,  2.12it/s, est. speed input: 69151.18 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 1791/1999 [14:08<01:38,  2.12it/s, est. speed input: 69151.38 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 1792/1999 [14:09<01:37,  2.12it/s, est. speed input: 69151.44 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 1793/1999 [14:09<01:37,  2.12it/s, est. speed input: 69151.58 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 1794/1999 [14:10<01:36,  2.12it/s, est. speed input: 69151.73 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 1795/1999 [14:10<01:36,  2.12it/s, est. speed input: 69151.92 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 1796/1999 [14:11<01:35,  2.12it/s, est. speed input: 69152.02 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 1797/1999 [14:11<01:35,  2.12it/s, est. speed input: 69152.27 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 1798/1999 [14:11<01:34,  2.12it/s, est. speed input: 69152.29 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|████████▉ | 1799/1999 [14:12<01:34,  2.12it/s, est. speed input: 69152.34 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 1800/1999 [14:12<01:33,  2.12it/s, est. speed input: 69152.53 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 1801/1999 [14:13<01:33,  2.12it/s, est. speed input: 69152.73 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 1802/1999 [14:13<01:33,  2.12it/s, est. speed input: 69152.82 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 1803/1999 [14:14<01:32,  2.12it/s, est. speed input: 69152.98 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 1804/1999 [14:14<01:32,  2.12it/s, est. speed input: 69153.16 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 1805/1999 [14:15<01:31,  2.12it/s, est. speed input: 69153.35 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 1806/1999 [14:15<01:31,  2.12it/s, est. speed input: 69153.57 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 1807/1999 [14:16<01:30,  2.12it/s, est. speed input: 69153.71 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 1808/1999 [14:16<01:30,  2.12it/s, est. speed input: 69153.91 toks/s, output: 2.11 toks/s]
Processed prompts:  90%|█████████ | 1809/1999 [14:17<01:29,  2.12it/s, est. speed input: 69154.02 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1810/1999 [14:17<01:29,  2.12it/s, est. speed input: 69154.26 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1811/1999 [14:18<01:28,  2.12it/s, est. speed input: 69154.21 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1812/1999 [14:18<01:28,  2.12it/s, est. speed input: 69154.39 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1813/1999 [14:19<01:27,  2.12it/s, est. speed input: 69154.62 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1814/1999 [14:19<01:27,  2.12it/s, est. speed input: 69154.80 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1815/1999 [14:20<01:26,  2.12it/s, est. speed input: 69155.01 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1816/1999 [14:20<01:26,  2.12it/s, est. speed input: 69155.19 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1817/1999 [14:20<01:25,  2.12it/s, est. speed input: 69155.31 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1818/1999 [14:21<01:25,  2.11it/s, est. speed input: 69155.06 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1819/1999 [14:21<01:25,  2.11it/s, est. speed input: 69155.15 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1820/1999 [14:22<01:24,  2.12it/s, est. speed input: 69155.31 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1821/1999 [14:22<01:24,  2.11it/s, est. speed input: 69155.33 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1822/1999 [14:23<01:23,  2.12it/s, est. speed input: 69155.49 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1823/1999 [14:23<01:23,  2.12it/s, est. speed input: 69155.71 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████ | 1824/1999 [14:24<01:22,  2.11it/s, est. speed input: 69155.65 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 1825/1999 [14:24<01:22,  2.11it/s, est. speed input: 69155.68 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 1826/1999 [14:25<01:21,  2.12it/s, est. speed input: 69155.82 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 1827/1999 [14:25<01:21,  2.12it/s, est. speed input: 69155.95 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 1828/1999 [14:26<01:20,  2.12it/s, est. speed input: 69156.09 toks/s, output: 2.11 toks/s]
Processed prompts:  91%|█████████▏| 1829/1999 [14:26<01:20,  2.12it/s, est. speed input: 69156.26 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1830/1999 [14:27<01:19,  2.12it/s, est. speed input: 69156.25 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1831/1999 [14:27<01:19,  2.11it/s, est. speed input: 69156.22 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1832/1999 [14:28<01:18,  2.11it/s, est. speed input: 69156.35 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1833/1999 [14:28<01:18,  2.12it/s, est. speed input: 69156.55 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1834/1999 [14:28<01:17,  2.12it/s, est. speed input: 69156.64 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1835/1999 [14:29<01:17,  2.12it/s, est. speed input: 69156.82 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1836/1999 [14:29<01:16,  2.12it/s, est. speed input: 69157.02 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1837/1999 [14:30<01:16,  2.12it/s, est. speed input: 69157.03 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1838/1999 [14:30<01:16,  2.12it/s, est. speed input: 69157.15 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1839/1999 [14:31<01:15,  2.12it/s, est. speed input: 69157.24 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1840/1999 [14:31<01:15,  2.12it/s, est. speed input: 69157.43 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1841/1999 [14:32<01:14,  2.12it/s, est. speed input: 69157.64 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1842/1999 [14:32<01:14,  2.12it/s, est. speed input: 69157.76 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1843/1999 [14:33<01:13,  2.12it/s, est. speed input: 69157.98 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1844/1999 [14:33<01:13,  2.11it/s, est. speed input: 69157.70 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1845/1999 [14:34<01:12,  2.11it/s, est. speed input: 69157.81 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1846/1999 [14:34<01:12,  2.12it/s, est. speed input: 69158.04 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1847/1999 [14:35<01:11,  2.12it/s, est. speed input: 69158.20 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1848/1999 [14:35<01:11,  2.12it/s, est. speed input: 69158.40 toks/s, output: 2.11 toks/s]
Processed prompts:  92%|█████████▏| 1849/1999 [14:36<01:10,  2.12it/s, est. speed input: 69158.62 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1850/1999 [14:36<01:10,  2.12it/s, est. speed input: 69158.87 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1851/1999 [14:37<01:09,  2.12it/s, est. speed input: 69158.88 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1852/1999 [14:37<01:09,  2.12it/s, est. speed input: 69158.97 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1853/1999 [14:37<01:08,  2.12it/s, est. speed input: 69159.17 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1854/1999 [14:38<01:08,  2.12it/s, est. speed input: 69159.24 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1855/1999 [14:38<01:08,  2.12it/s, est. speed input: 69159.33 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1856/1999 [14:39<01:07,  2.12it/s, est. speed input: 69159.57 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1857/1999 [14:39<01:07,  2.11it/s, est. speed input: 69159.42 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1858/1999 [14:40<01:06,  2.11it/s, est. speed input: 69159.46 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1859/1999 [14:40<01:06,  2.12it/s, est. speed input: 69159.73 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1860/1999 [14:41<01:05,  2.12it/s, est. speed input: 69159.84 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1861/1999 [14:41<01:05,  2.12it/s, est. speed input: 69159.91 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1862/1999 [14:42<01:04,  2.12it/s, est. speed input: 69160.07 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1863/1999 [14:42<01:05,  2.08it/s, est. speed input: 69158.12 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1864/1999 [14:43<01:04,  2.10it/s, est. speed input: 69158.37 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1865/1999 [14:43<01:03,  2.10it/s, est. speed input: 69158.56 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1866/1999 [14:44<01:03,  2.10it/s, est. speed input: 69158.35 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1867/1999 [14:44<01:02,  2.11it/s, est. speed input: 69158.43 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1868/1999 [14:45<01:02,  2.11it/s, est. speed input: 69158.69 toks/s, output: 2.11 toks/s]
Processed prompts:  93%|█████████▎| 1869/1999 [14:45<01:01,  2.12it/s, est. speed input: 69158.93 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 1870/1999 [14:46<01:01,  2.11it/s, est. speed input: 69158.85 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 1871/1999 [14:46<01:00,  2.11it/s, est. speed input: 69158.93 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 1872/1999 [14:46<01:00,  2.12it/s, est. speed input: 69159.16 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 1873/1999 [14:47<00:59,  2.12it/s, est. speed input: 69159.29 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▎| 1874/1999 [14:47<00:59,  2.12it/s, est. speed input: 69159.43 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1875/1999 [14:48<00:58,  2.12it/s, est. speed input: 69159.64 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1876/1999 [14:48<00:57,  2.12it/s, est. speed input: 69159.90 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1877/1999 [14:49<00:57,  2.12it/s, est. speed input: 69160.00 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1878/1999 [14:49<00:57,  2.12it/s, est. speed input: 69160.19 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1879/1999 [14:50<00:56,  2.12it/s, est. speed input: 69160.43 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1880/1999 [14:50<00:56,  2.12it/s, est. speed input: 69160.57 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1881/1999 [14:51<00:55,  2.12it/s, est. speed input: 69160.72 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1882/1999 [14:51<00:55,  2.12it/s, est. speed input: 69160.92 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1883/1999 [14:52<00:54,  2.12it/s, est. speed input: 69161.00 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1884/1999 [14:52<00:54,  2.12it/s, est. speed input: 69160.92 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1885/1999 [14:53<00:53,  2.12it/s, est. speed input: 69161.11 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1886/1999 [14:53<00:53,  2.11it/s, est. speed input: 69160.94 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1887/1999 [14:54<00:52,  2.11it/s, est. speed input: 69161.03 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1888/1999 [14:54<00:52,  2.12it/s, est. speed input: 69161.27 toks/s, output: 2.11 toks/s]
Processed prompts:  94%|█████████▍| 1889/1999 [14:54<00:51,  2.12it/s, est. speed input: 69161.41 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 1890/1999 [14:55<00:51,  2.12it/s, est. speed input: 69161.59 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 1891/1999 [14:55<00:50,  2.12it/s, est. speed input: 69161.75 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 1892/1999 [14:56<00:50,  2.12it/s, est. speed input: 69161.83 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 1893/1999 [14:56<00:50,  2.12it/s, est. speed input: 69161.88 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 1894/1999 [14:57<00:49,  2.11it/s, est. speed input: 69161.77 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 1895/1999 [14:57<00:49,  2.12it/s, est. speed input: 69162.02 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 1896/1999 [14:58<00:48,  2.11it/s, est. speed input: 69161.88 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 1897/1999 [14:58<00:48,  2.11it/s, est. speed input: 69162.06 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 1898/1999 [14:59<00:47,  2.12it/s, est. speed input: 69162.30 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▍| 1899/1999 [14:59<00:47,  2.12it/s, est. speed input: 69162.27 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 1900/1999 [15:00<00:46,  2.12it/s, est. speed input: 69162.40 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 1901/1999 [15:00<00:46,  2.12it/s, est. speed input: 69162.58 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 1902/1999 [15:01<00:45,  2.12it/s, est. speed input: 69162.72 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 1903/1999 [15:01<00:45,  2.11it/s, est. speed input: 69162.50 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 1904/1999 [15:02<00:44,  2.11it/s, est. speed input: 69162.68 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 1905/1999 [15:02<00:44,  2.12it/s, est. speed input: 69162.92 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 1906/1999 [15:03<00:43,  2.12it/s, est. speed input: 69163.04 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 1907/1999 [15:03<00:43,  2.12it/s, est. speed input: 69163.10 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 1908/1999 [15:03<00:42,  2.12it/s, est. speed input: 69163.36 toks/s, output: 2.11 toks/s]
Processed prompts:  95%|█████████▌| 1909/1999 [15:04<00:42,  2.12it/s, est. speed input: 69163.51 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1910/1999 [15:04<00:42,  2.11it/s, est. speed input: 69163.39 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1911/1999 [15:05<00:41,  2.12it/s, est. speed input: 69163.60 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1912/1999 [15:05<00:41,  2.12it/s, est. speed input: 69163.67 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1913/1999 [15:06<00:40,  2.12it/s, est. speed input: 69163.78 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1914/1999 [15:06<00:40,  2.12it/s, est. speed input: 69163.88 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1915/1999 [15:07<00:39,  2.12it/s, est. speed input: 69164.11 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1916/1999 [15:07<00:39,  2.12it/s, est. speed input: 69164.19 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1917/1999 [15:08<00:38,  2.12it/s, est. speed input: 69164.31 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1918/1999 [15:08<00:38,  2.12it/s, est. speed input: 69164.43 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1919/1999 [15:09<00:37,  2.12it/s, est. speed input: 69164.47 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1920/1999 [15:09<00:37,  2.11it/s, est. speed input: 69164.47 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1921/1999 [15:10<00:36,  2.11it/s, est. speed input: 69164.53 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1922/1999 [15:10<00:36,  2.12it/s, est. speed input: 69164.66 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1923/1999 [15:11<00:35,  2.12it/s, est. speed input: 69164.73 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▌| 1924/1999 [15:11<00:35,  2.12it/s, est. speed input: 69164.83 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 1925/1999 [15:12<00:34,  2.12it/s, est. speed input: 69164.90 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 1926/1999 [15:12<00:34,  2.12it/s, est. speed input: 69165.05 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 1927/1999 [15:12<00:34,  2.12it/s, est. speed input: 69165.11 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 1928/1999 [15:13<00:33,  2.12it/s, est. speed input: 69165.19 toks/s, output: 2.11 toks/s]
Processed prompts:  96%|█████████▋| 1929/1999 [15:13<00:33,  2.12it/s, est. speed input: 69165.32 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1930/1999 [15:14<00:32,  2.12it/s, est. speed input: 69165.37 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1931/1999 [15:14<00:32,  2.12it/s, est. speed input: 69165.45 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1932/1999 [15:15<00:31,  2.12it/s, est. speed input: 69165.53 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1933/1999 [15:15<00:31,  2.12it/s, est. speed input: 69165.63 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1934/1999 [15:16<00:30,  2.12it/s, est. speed input: 69165.70 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1935/1999 [15:16<00:30,  2.12it/s, est. speed input: 69165.84 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1936/1999 [15:17<00:29,  2.12it/s, est. speed input: 69165.90 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1937/1999 [15:17<00:29,  2.12it/s, est. speed input: 69166.01 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1938/1999 [15:18<00:28,  2.12it/s, est. speed input: 69166.15 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1939/1999 [15:18<00:28,  2.11it/s, est. speed input: 69166.05 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1940/1999 [15:19<00:27,  2.11it/s, est. speed input: 69166.14 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1941/1999 [15:19<00:27,  2.11it/s, est. speed input: 69166.20 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1942/1999 [15:20<00:26,  2.11it/s, est. speed input: 69166.14 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1943/1999 [15:20<00:26,  2.11it/s, est. speed input: 69166.21 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1944/1999 [15:20<00:26,  2.11it/s, est. speed input: 69166.31 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1945/1999 [15:21<00:25,  2.11it/s, est. speed input: 69166.37 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1946/1999 [15:21<00:25,  2.11it/s, est. speed input: 69166.41 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1947/1999 [15:22<00:24,  2.11it/s, est. speed input: 69166.48 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1948/1999 [15:22<00:24,  2.12it/s, est. speed input: 69166.58 toks/s, output: 2.11 toks/s]
Processed prompts:  97%|█████████▋| 1949/1999 [15:23<00:23,  2.12it/s, est. speed input: 69166.74 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1950/1999 [15:23<00:23,  2.12it/s, est. speed input: 69166.84 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1951/1999 [15:24<00:22,  2.11it/s, est. speed input: 69166.82 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1952/1999 [15:24<00:22,  2.11it/s, est. speed input: 69166.88 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1953/1999 [15:25<00:21,  2.11it/s, est. speed input: 69166.93 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1954/1999 [15:25<00:21,  2.11it/s, est. speed input: 69166.56 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1955/1999 [15:26<00:20,  2.11it/s, est. speed input: 69166.53 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1956/1999 [15:26<00:20,  2.11it/s, est. speed input: 69166.66 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1957/1999 [15:27<00:19,  2.11it/s, est. speed input: 69166.55 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1958/1999 [15:27<00:19,  2.11it/s, est. speed input: 69166.61 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1959/1999 [15:28<00:18,  2.11it/s, est. speed input: 69166.68 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1960/1999 [15:28<00:18,  2.11it/s, est. speed input: 69166.76 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1961/1999 [15:29<00:17,  2.11it/s, est. speed input: 69166.83 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1962/1999 [15:29<00:17,  2.11it/s, est. speed input: 69166.87 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1963/1999 [15:29<00:17,  2.11it/s, est. speed input: 69166.97 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1964/1999 [15:30<00:16,  2.11it/s, est. speed input: 69166.94 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1965/1999 [15:30<00:16,  2.11it/s, est. speed input: 69166.93 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1966/1999 [15:31<00:15,  2.11it/s, est. speed input: 69167.04 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1967/1999 [15:31<00:15,  2.11it/s, est. speed input: 69166.90 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1968/1999 [15:32<00:14,  2.11it/s, est. speed input: 69166.89 toks/s, output: 2.11 toks/s]
Processed prompts:  98%|█████████▊| 1969/1999 [15:32<00:14,  2.11it/s, est. speed input: 69166.93 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 1970/1999 [15:33<00:13,  2.11it/s, est. speed input: 69167.02 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 1971/1999 [15:33<00:13,  2.11it/s, est. speed input: 69166.87 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 1972/1999 [15:34<00:12,  2.11it/s, est. speed input: 69166.94 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 1973/1999 [15:34<00:12,  2.11it/s, est. speed input: 69166.89 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▊| 1974/1999 [15:35<00:11,  2.11it/s, est. speed input: 69166.88 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1975/1999 [15:35<00:11,  2.11it/s, est. speed input: 69166.93 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1976/1999 [15:36<00:10,  2.11it/s, est. speed input: 69167.05 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1977/1999 [15:36<00:10,  2.11it/s, est. speed input: 69167.15 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1978/1999 [15:37<00:09,  2.12it/s, est. speed input: 69167.25 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1979/1999 [15:37<00:09,  2.12it/s, est. speed input: 69167.40 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1980/1999 [15:38<00:08,  2.12it/s, est. speed input: 69167.47 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1981/1999 [15:38<00:08,  2.11it/s, est. speed input: 69167.48 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1982/1999 [15:38<00:08,  2.11it/s, est. speed input: 69167.56 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1983/1999 [15:39<00:07,  2.11it/s, est. speed input: 69167.52 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1984/1999 [15:39<00:07,  2.11it/s, est. speed input: 69167.60 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1985/1999 [15:40<00:06,  2.12it/s, est. speed input: 69167.74 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1986/1999 [15:40<00:06,  2.11it/s, est. speed input: 69167.48 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1987/1999 [15:41<00:05,  2.11it/s, est. speed input: 69167.46 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1988/1999 [15:41<00:05,  2.11it/s, est. speed input: 69167.59 toks/s, output: 2.11 toks/s]
Processed prompts:  99%|█████████▉| 1989/1999 [15:42<00:04,  2.11it/s, est. speed input: 69167.69 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 1990/1999 [15:42<00:04,  2.11it/s, est. speed input: 69167.78 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 1991/1999 [15:43<00:03,  2.12it/s, est. speed input: 69167.90 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 1992/1999 [15:43<00:03,  2.12it/s, est. speed input: 69167.98 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 1993/1999 [15:44<00:02,  2.12it/s, est. speed input: 69168.05 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 1994/1999 [15:44<00:02,  2.11it/s, est. speed input: 69168.01 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 1995/1999 [15:45<00:01,  2.11it/s, est. speed input: 69168.08 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 1996/1999 [15:45<00:01,  2.11it/s, est. speed input: 69168.13 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 1997/1999 [15:46<00:00,  2.11it/s, est. speed input: 69168.09 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|█████████▉| 1998/1999 [15:46<00:00,  2.11it/s, est. speed input: 69168.20 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 1999/1999 [15:47<00:00,  2.11it/s, est. speed input: 69168.28 toks/s, output: 2.11 toks/s]
Processed prompts: 100%|██████████| 1999/1999 [15:47<00:00,  2.11it/s, est. speed input: 69168.28 toks/s, output: 2.11 toks/s]
Saved activation counts to new_activations/activation.ga.train.q2.5-ga.pt
Processed 65503232 tokens total

Activation analysis complete for model: q2.5-ga!

=== Processing model: /home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_1.5Bbasque_corpus_24092/checkpoint-2424, type: qwen ===
[rank0]: Traceback (most recent call last):
[rank0]:   File "/home/khanh/miniforge3/envs/sla/lib/python3.13/site-packages/vllm/transformers_utils/config.py", line 258, in get_config
[rank0]:     if is_gguf or file_or_path_exists(
[rank0]:                   ~~~~~~~~~~~~~~~~~~~^
[rank0]:             model, HF_CONFIG_NAME, revision=revision):
[rank0]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:   File "/home/khanh/miniforge3/envs/sla/lib/python3.13/site-packages/vllm/transformers_utils/config.py", line 169, in file_or_path_exists
[rank0]:     cached_filepath = try_to_load_from_cache(repo_id=model,
[rank0]:                                              filename=config_name,
[rank0]:                                              revision=revision)
[rank0]:   File "/home/khanh/miniforge3/envs/sla/lib/python3.13/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
[rank0]:     validate_repo_id(arg_value)
[rank0]:     ~~~~~~~~~~~~~~~~^^^^^^^^^^^
[rank0]:   File "/home/khanh/miniforge3/envs/sla/lib/python3.13/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
[rank0]:     raise HFValidationError(
[rank0]:     ...<2 lines>...
[rank0]:     )
[rank0]: huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_1.5Bbasque_corpus_24092/checkpoint-2424'. Use `repo_type` argument if needed.

[rank0]: The above exception was the direct cause of the following exception:

[rank0]: Traceback (most recent call last):
[rank0]:   File "/home/khanh/sla/lsn-analysis/activation_all.py", line 127, in <module>
[rank0]:     model = LLM(
[rank0]:         model=model_name,
[rank0]:     ...<2 lines>...
[rank0]:         trust_remote_code=True
[rank0]:     )
[rank0]:   File "/home/khanh/miniforge3/envs/sla/lib/python3.13/site-packages/vllm/utils.py", line 1096, in inner
[rank0]:     return fn(*args, **kwargs)
[rank0]:   File "/home/khanh/miniforge3/envs/sla/lib/python3.13/site-packages/vllm/entrypoints/llm.py", line 243, in __init__
[rank0]:     self.llm_engine = LLMEngine.from_engine_args(
[rank0]:                       ~~~~~~~~~~~~~~~~~~~~~~~~~~^
[rank0]:         engine_args=engine_args, usage_context=UsageContext.LLM_CLASS)
[rank0]:         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:   File "/home/khanh/miniforge3/envs/sla/lib/python3.13/site-packages/vllm/engine/llm_engine.py", line 514, in from_engine_args
[rank0]:     vllm_config = engine_args.create_engine_config(usage_context)
[rank0]:   File "/home/khanh/miniforge3/envs/sla/lib/python3.13/site-packages/vllm/engine/arg_utils.py", line 1137, in create_engine_config
[rank0]:     model_config = self.create_model_config()
[rank0]:   File "/home/khanh/miniforge3/envs/sla/lib/python3.13/site-packages/vllm/engine/arg_utils.py", line 1026, in create_model_config
[rank0]:     return ModelConfig(
[rank0]:         model=self.model,
[rank0]:     ...<35 lines>...
[rank0]:         model_impl=self.model_impl,
[rank0]:     )
[rank0]:   File "/home/khanh/miniforge3/envs/sla/lib/python3.13/site-packages/vllm/config.py", line 343, in __init__
[rank0]:     hf_config = get_config(self.hf_config_path or self.model,
[rank0]:                            trust_remote_code, revision, code_revision,
[rank0]:                            config_format)
[rank0]:   File "/home/khanh/miniforge3/envs/sla/lib/python3.13/site-packages/vllm/transformers_utils/config.py", line 283, in get_config
[rank0]:     raise ValueError(error_message) from e
[rank0]: ValueError: Invalid repository ID or local directory specified: '/home/khanh/sla/sla_cpt/qwen2.5-0.5b_english_wiki_1.5Bbasque_corpus_24092/checkpoint-2424'.
[rank0]: Please verify the following requirements:
[rank0]: 1. Provide a valid Hugging Face repository ID.
[rank0]: 2. Specify a local directory that contains a recognized configuration file.
[rank0]:    - For Hugging Face models: ensure the presence of a 'config.json'.
[rank0]:    - For Mistral models: ensure the presence of a 'params.json'.

[rank0]:[W926 15:17:03.990277284 ProcessGroupNCCL.cpp:1496] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
srun: error: gpu0: task 0: Exited with exit code 1