Buckets:

Mercity
/

Desicion-Model-Custom

Files
xet
Mercity/Desicion-Model-Custom / think-decide /logs /p5_notes.log
Pranav2748
2 days ago
download
raw
205 kB
{"selected": 32000, "by_source": {"proc/record_eligibility": 1781, "proc/taxonomy_routing": 1852, "proc/arithmetic": 1871, "proc/table_lookup": 1752, "proc/state_perturbation": 1813, "proc/multi_view_adjudication": 1818, "proc/policy_under_uncertainty": 1785, "proc/policy_applicability": 1744, "proc/record_aggregation": 1717, "proc/event_state_reconstruction": 1839, "proc/entity_belief_tracking": 1861, "proc/needle_retrieval": 1818, "proc/instruction_execution": 1819, "proc/goal_directed_planning": 1818, "proc/partial_observation_calibration": 1229, "proc/evidence_sufficiency": 1827, "proc/event_triggered_execution": 1853, "proc/graph_navigation": 1803}}
W1106 15:46:11.530000 38470 torch/_opaque_base.py:6] torch._opaque_base is deprecated, use torch._custom_class_base instead
W1106 15:46:11.531000 38470 torch/_library/opaque_object.py:288] register_opaque_type is deprecated, use register_custom_class instead
W1106 15:46:11.532000 38470 torch/_library/opaque_object.py:211] typ='value' is deprecated, use typ='constant' instead
(MainProcess pid=38470) INFO 11-06 15:46:18 [startup.py:96] non-default args: {'dtype': 'bfloat16', 'max_model_len': 8192, 'enable_prefix_caching': True, 'gpu_memory_utilization': 0.5, 'max_num_seqs': 256, 'disable_log_stats': True, 'model': '/root/hf/qwen3.5-9b'}
(MainProcess pid=38470) INFO 11-06 15:46:18 [model.py:744] Resolved architecture: Qwen3_5ForConditionalGeneration
(MainProcess pid=38470) INFO 11-06 15:46:18 [model.py:2170] Using max model len 8192
(MainProcess pid=38470) INFO 11-06 15:46:21 [scheduler.py:310] Chunked prefill is enabled with max_num_batched_tokens=8192.
(MainProcess pid=38470) INFO 11-06 15:46:21 [config.py:766] Mamba cache mode is set to 'align' for Qwen3_5ForConditionalGeneration by default when prefix caching is enabled
(MainProcess pid=38470) INFO 11-06 15:46:21 [kernel.py:480] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(MainProcess pid=38470) WARNING 11-06 15:46:32 [system_utils.py:154] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
W1106 15:46:38.454000 38829 torch/_opaque_base.py:6] torch._opaque_base is deprecated, use torch._custom_class_base instead
W1106 15:46:38.455000 38829 torch/_library/opaque_object.py:288] register_opaque_type is deprecated, use register_custom_class instead
W1106 15:46:38.456000 38829 torch/_library/opaque_object.py:211] typ='value' is deprecated, use typ='constant' instead
(EngineCore pid=38829) INFO 11-06 15:46:43 [core.py:139] Initializing a V1 LLM engine (v0.33.0) with config: model='/root/hf/qwen3.5-9b', speculative_config=None, tokenizer='/root/hf/qwen3.5-9b', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=8192, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=False, aux_output_config=AuxOutputConfig(enable_return_routed_experts=False, max_bytes=None), kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, per_request_spec_decode_metrics='none', kv_cache_metrics=False, kv_cache_metrics_sample=0.01, custom_histogram_buckets=None, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=/root/hf/qwen3.5-9b, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::qwen4_exp_ple_short_conv', 'vllm::qwen4_exp_qsa_with_output', 'vllm::qwen4_exp_qsa_prepare_with_output', 'vllm::linear_attention', 'vllm::qwen_gdn_attention_core', 'vllm::qwen_gdn_attention_core_fused_norm_packed', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'cudagraph_decoder_replay': True, 'decoder_replay_cudagraph_capture_sizes': [], 'decoder_replay_trim_threshold': 768, 'compile_sizes': [], 'compile_ranges_endpoints': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native']), enable_flashinfer_autotune=True, enable_rocm_segmented_attn_autotune=False, enable_cutedsl_warmup=True, enable_jit_warmup=True, monokernel=False, moe_backend='auto', sparse_indexer_topk_backend='auto', linear_backend='auto', linear_backend_per_quant=None)
(EngineCore pid=38829) INFO 11-06 15:46:46 [parallel_state.py:1929] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_1f6283b08ebb418aacd3f88553acd8ff backend=nccl
(EngineCore pid=38829) INFO 11-06 15:46:46 [parallel_state.py:2385] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, ETP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=38829) INFO 11-06 15:46:46 [gpu_worker.py:513] Using V2 Model Runner
(EngineCore pid=38829) INFO 11-06 15:46:47 [model_runner.py:399] Loading model from scratch...
(EngineCore pid=38829) INFO 11-06 15:46:48 [cuda.py:567] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(EngineCore pid=38829) INFO 11-06 15:46:48 [mm_encoder_attention.py:372] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(EngineCore pid=38829) INFO 11-06 15:46:48 [qwen_gdn_linear_attn.py:300] Using Triton/FLA GDN prefill kernel (requested=auto, head_k_dim=128).
(EngineCore pid=38829) INFO 11-06 15:46:49 [cuda.py:508] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=38829) INFO 11-06 15:46:49 [flash_attn.py:1122] Using FlashAttention version 2
(EngineCore pid=38829) INFO 11-06 15:46:50 [base_loader.py:82] Initializing model took 2.173478 seconds
(EngineCore pid=38829) INFO 11-06 15:46:50 [weight_utils.py:1144] Filesystem type for checkpoints: OVERLAY. Checkpoint size: 17.98 GiB. Available RAM: 85.64 GiB.
(EngineCore pid=38829) INFO 11-06 15:46:50 [weight_utils.py:1167] Auto-prefetch is disabled because the filesystem (OVERLAY) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.

Loading safetensors checkpoint shards:   0% Completed | 0/4 [00:00<?, ?it/s]

Loading safetensors checkpoint shards:  25% Completed | 1/4 [00:00<00:01,  1.60it/s]

Loading safetensors checkpoint shards:  50% Completed | 2/4 [00:01<00:01,  1.52it/s]
[transformers] Qwen3VL video processing does not apply the per-frame pixel cap the reference implementation (qwen-vl-utils) applies, so some videos cost far more tokens than they would there. In v5.22 the capped behavior will become the default and `cap_pixels_per_frame` will be removed. Pass `cap_pixels_per_frame=True` to adopt the reference behavior now, or `False` to keep the current behavior and silence this warning.

Loading safetensors checkpoint shards:  75% Completed | 3/4 [00:02<00:00,  1.38it/s]
(MainProcess pid=38470) INFO 11-06 15:46:52 [base.py:283] Multi-modal warmup completed in 19.656s
(EngineCore pid=38829) WARNING 11-06 15:46:52 [utils.py:333] Attempted to load weight layers.0.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:52 [utils.py:333] Attempted to load weight layers.1.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:52 [utils.py:333] Attempted to load weight layers.10.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:52 [utils.py:333] Attempted to load weight layers.12.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:52 [utils.py:333] Attempted to load weight layers.13.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:52 [utils.py:333] Attempted to load weight layers.14.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:52 [utils.py:333] Attempted to load weight layers.16.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:52 [utils.py:333] Attempted to load weight layers.17.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:52 [utils.py:333] Attempted to load weight layers.18.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.2.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.20.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.21.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.22.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.24.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.25.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.26.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.28.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.29.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.30.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.4.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.5.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.6.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:53 [utils.py:333] Attempted to load weight layers.8.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16
(EngineCore pid=38829) WARNING 11-06 15:46:54 [utils.py:333] Attempted to load weight layers.9.linear_attn.norm.weight with dtype torch.float32 into parameter with dtype torch.bfloat16

Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:04<00:00,  1.57s/it]

Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:04<00:00,  1.25s/it]

(EngineCore pid=38829) INFO 11-06 15:46:55 [default_loader.py:506] Loading weights took 5.13 seconds
(EngineCore pid=38829) INFO 11-06 15:46:56 [model_runner.py:428] Model loading took 17.57 GiB memory and 9.206754 seconds
(EngineCore pid=38829) INFO 11-06 15:46:56 [topk_topp_sampler.py:149] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=38829) INFO 11-06 15:46:56 [torch_utils.py:281] Reducing Torch threads from 255 to 1 for serving. Set OMP_NUM_THREADS in the external environment to override.
(EngineCore pid=38829) INFO 11-06 15:46:56 [interface.py:1108] Setting attention block size to 528 tokens to ensure that attention page size is >= mamba page size.
(EngineCore pid=38829) INFO 11-06 15:46:56 [interface.py:1132] Padding mamba page size by 0.76% to ensure that mamba page size and attention page size are exactly equal.
(EngineCore pid=38829) INFO 11-06 15:46:56 [utils.py:381] Using LBNHC KV cache layout.
(EngineCore pid=38829) INFO 11-06 15:47:03 [encoder_runner.py:131] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
(EngineCore pid=38829) INFO 11-06 15:47:15 [caching.py:342] reconstructed serializable fn from standalone compile artifacts. num_artifacts=13 num_submods=33
(EngineCore pid=38829) INFO 11-06 15:47:15 [decorators.py:326] [backbone] Directly load AOT compilation from path /root/.cache/vllm/torch_compile_cache/torch_aot_compile/570fe35d686514fcab15b2f5a7f595cad3825ba85c7952aa6d2232b00327468d/rank_0_0/model
(EngineCore pid=38829) INFO 11-06 15:47:15 [monitor.py:62] [backbone] torch.compile took 0.71 s in total
(EngineCore pid=38829) INFO 11-06 15:47:17 [monitor.py:91] [backbone] Initial profiling/warmup run took 1.68 s

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   2%|▏         | 1/51 [00:01<00:52,  1.05s/it]
Capturing CUDA graphs (PIECEWISE):   4%|▍         | 2/51 [00:01<00:44,  1.10it/s]
Capturing CUDA graphs (PIECEWISE):   6%|▌         | 3/51 [00:02<00:40,  1.20it/s]
Capturing CUDA graphs (PIECEWISE):   8%|▊         | 4/51 [00:03<00:37,  1.25it/s]
Capturing CUDA graphs (PIECEWISE):  10%|▉         | 5/51 [00:04<00:35,  1.29it/s]
Capturing CUDA graphs (PIECEWISE):  12%|█▏        | 6/51 [00:04<00:32,  1.38it/s]
Capturing CUDA graphs (PIECEWISE):  14%|█▎        | 7/51 [00:05<00:29,  1.48it/s]
Capturing CUDA graphs (PIECEWISE):  16%|█▌        | 8/51 [00:05<00:27,  1.56it/s]
Capturing CUDA graphs (PIECEWISE):  18%|█▊        | 9/51 [00:06<00:27,  1.54it/s]
Capturing CUDA graphs (PIECEWISE):  20%|█▉        | 10/51 [00:07<00:24,  1.65it/s]
Capturing CUDA graphs (PIECEWISE):  22%|██▏       | 11/51 [00:07<00:22,  1.75it/s]
Capturing CUDA graphs (PIECEWISE):  24%|██▎       | 12/51 [00:07<00:21,  1.84it/s]
Capturing CUDA graphs (PIECEWISE):  25%|██▌       | 13/51 [00:08<00:20,  1.84it/s]
Capturing CUDA graphs (PIECEWISE):  27%|██▋       | 14/51 [00:09<00:19,  1.89it/s]
Capturing CUDA graphs (PIECEWISE):  29%|██▉       | 15/51 [00:09<00:18,  2.00it/s]
Capturing CUDA graphs (PIECEWISE):  31%|███▏      | 16/51 [00:09<00:16,  2.16it/s]
Capturing CUDA graphs (PIECEWISE):  33%|███▎      | 17/51 [00:10<00:12,  2.67it/s]
Capturing CUDA graphs (PIECEWISE):  35%|███▌      | 18/51 [00:10<00:10,  3.17it/s]
Capturing CUDA graphs (PIECEWISE):  37%|███▋      | 19/51 [00:10<00:09,  3.41it/s]
Capturing CUDA graphs (PIECEWISE):  39%|███▉      | 20/51 [00:10<00:07,  3.98it/s]
Capturing CUDA graphs (PIECEWISE):  41%|████      | 21/51 [00:10<00:06,  4.47it/s]
Capturing CUDA graphs (PIECEWISE):  43%|████▎     | 22/51 [00:10<00:06,  4.52it/s]
Capturing CUDA graphs (PIECEWISE):  45%|████▌     | 23/51 [00:11<00:05,  4.97it/s]
Capturing CUDA graphs (PIECEWISE):  47%|████▋     | 24/51 [00:11<00:04,  5.41it/s]
Capturing CUDA graphs (PIECEWISE):  49%|████▉     | 25/51 [00:11<00:04,  5.40it/s]
Capturing CUDA graphs (PIECEWISE):  51%|█████     | 26/51 [00:11<00:04,  5.70it/s]
Capturing CUDA graphs (PIECEWISE):  53%|█████▎    | 27/51 [00:11<00:03,  6.02it/s]
Capturing CUDA graphs (PIECEWISE):  55%|█████▍    | 28/51 [00:11<00:03,  5.97it/s]
Capturing CUDA graphs (PIECEWISE):  57%|█████▋    | 29/51 [00:12<00:03,  6.17it/s]
Capturing CUDA graphs (PIECEWISE):  59%|█████▉    | 30/51 [00:12<00:03,  6.35it/s]
Capturing CUDA graphs (PIECEWISE):  61%|██████    | 31/51 [00:12<00:03,  6.24it/s]
Capturing CUDA graphs (PIECEWISE):  63%|██████▎   | 32/51 [00:12<00:03,  6.21it/s]
Capturing CUDA graphs (PIECEWISE):  65%|██████▍   | 33/51 [00:12<00:02,  6.54it/s]
Capturing CUDA graphs (PIECEWISE):  67%|██████▋   | 34/51 [00:12<00:02,  6.49it/s]
Capturing CUDA graphs (PIECEWISE):  69%|██████▊   | 35/51 [00:13<00:02,  6.23it/s]
Capturing CUDA graphs (PIECEWISE):  71%|███████   | 36/51 [00:13<00:02,  6.58it/s]
Capturing CUDA graphs (PIECEWISE):  73%|███████▎  | 37/51 [00:13<00:02,  6.72it/s]
Capturing CUDA graphs (PIECEWISE):  75%|███████▍  | 38/51 [00:13<00:01,  6.73it/s]
Capturing CUDA graphs (PIECEWISE):  76%|███████▋  | 39/51 [00:13<00:01,  6.66it/s]
Capturing CUDA graphs (PIECEWISE):  78%|███████▊  | 40/51 [00:13<00:01,  6.81it/s]
Capturing CUDA graphs (PIECEWISE):  80%|████████  | 41/51 [00:13<00:01,  6.89it/s]
Capturing CUDA graphs (PIECEWISE):  82%|████████▏ | 42/51 [00:14<00:01,  6.32it/s]
Capturing CUDA graphs (PIECEWISE):  84%|████████▍ | 43/51 [00:14<00:01,  6.41it/s]
Capturing CUDA graphs (PIECEWISE):  86%|████████▋ | 44/51 [00:14<00:01,  6.72it/s]
Capturing CUDA graphs (PIECEWISE):  88%|████████▊ | 45/51 [00:14<00:00,  6.92it/s]
Capturing CUDA graphs (PIECEWISE):  90%|█████████ | 46/51 [00:14<00:00,  5.95it/s]
Capturing CUDA graphs (PIECEWISE):  92%|█████████▏| 47/51 [00:14<00:00,  6.38it/s]
Capturing CUDA graphs (PIECEWISE):  94%|█████████▍| 48/51 [00:14<00:00,  6.86it/s]
Capturing CUDA graphs (PIECEWISE):  96%|█████████▌| 49/51 [00:15<00:00,  6.90it/s]
Capturing CUDA graphs (PIECEWISE):  98%|█████████▊| 50/51 [00:15<00:00,  6.86it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:15<00:00,  7.02it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:15<00:00,  3.32it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/2 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 2/2 [00:00<00:00, 14.30it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 2/2 [00:00<00:00, 14.27it/s]
(EngineCore pid=38829) INFO 11-06 15:47:36 [model_runner.py:1163] Graph capturing finished in 16 secs, took 0.43 GiB
(EngineCore pid=38829) INFO 11-06 15:47:36 [gpu_worker.py:772] Available KV cache memory: 19.42 GiB
(EngineCore pid=38829) INFO 11-06 15:47:36 [gpu_worker.py:787] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.5000 is equivalent to --gpu-memory-utilization=0.4940 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.5060. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
(EngineCore pid=38829) INFO 11-06 15:47:36 [kv_cache_utils.py:1147] GPU KV cache size: 448,698 tokens, Maximum concurrency for 8,192 tokens per request: 54.77x
(EngineCore pid=38829) INFO 11-06 15:47:36 [kv_cache_utils.py:865] kv cache group sizes (528, 528, 528, 528)
(EngineCore pid=38829) INFO 11-06 15:47:36 [kv_cache_utils.py:866] kv lcm block sizes 528
(EngineCore pid=38829) INFO 11-06 15:47:37 [kernel_warmup.py:179] JIT kernel warmup starting.
(EngineCore pid=38829) INFO 11-06 15:47:37 [kernel_warmup.py:192] JIT kernel warmup finished in 0.00s.
(EngineCore pid=38829) INFO 11-06 15:47:37 [qwen_triton_warmup.py:370] Warming up Qwen GDN Triton kernels for model_type=qwen3_5_text.
(EngineCore pid=38829) INFO 11-06 15:47:37 [qwen_vl_triton_warmup.py:57] Warmed position embedding and vision rotary kernels on grids=[(1, 16, 16), (1, 16, 2), (1, 2, 16), (1, 2, 2)].
(EngineCore pid=38829) INFO 11-06 15:47:37 [mamba_triton_warmup.py:42] Warmed Mamba batch_memcpy_kernel.

Capturing CUDA graphs (PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):   2%|▏         | 1/51 [00:00<00:42,  1.17it/s]
Capturing CUDA graphs (PIECEWISE):   4%|▍         | 2/51 [00:01<00:42,  1.16it/s]
Capturing CUDA graphs (PIECEWISE):   6%|▌         | 3/51 [00:02<00:38,  1.25it/s]
Capturing CUDA graphs (PIECEWISE):   8%|▊         | 4/51 [00:03<00:34,  1.37it/s]
Capturing CUDA graphs (PIECEWISE):  10%|▉         | 5/51 [00:03<00:33,  1.38it/s]
Capturing CUDA graphs (PIECEWISE):  12%|█▏        | 6/51 [00:04<00:31,  1.45it/s]
Capturing CUDA graphs (PIECEWISE):  14%|█▎        | 7/51 [00:04<00:28,  1.54it/s]
Capturing CUDA graphs (PIECEWISE):  16%|█▌        | 8/51 [00:05<00:26,  1.64it/s]
Capturing CUDA graphs (PIECEWISE):  18%|█▊        | 9/51 [00:06<00:24,  1.69it/s]
Capturing CUDA graphs (PIECEWISE):  20%|█▉        | 10/51 [00:06<00:23,  1.72it/s]
Capturing CUDA graphs (PIECEWISE):  22%|██▏       | 11/51 [00:07<00:22,  1.81it/s]
Capturing CUDA graphs (PIECEWISE):  24%|██▎       | 12/51 [00:07<00:20,  1.87it/s]
Capturing CUDA graphs (PIECEWISE):  25%|██▌       | 13/51 [00:08<00:20,  1.88it/s]
Capturing CUDA graphs (PIECEWISE):  27%|██▋       | 14/51 [00:08<00:18,  2.02it/s]
Capturing CUDA graphs (PIECEWISE):  29%|██▉       | 15/51 [00:08<00:16,  2.20it/s]
Capturing CUDA graphs (PIECEWISE):  31%|███▏      | 16/51 [00:09<00:15,  2.29it/s]
Capturing CUDA graphs (PIECEWISE):  33%|███▎      | 17/51 [00:09<00:11,  2.83it/s]
Capturing CUDA graphs (PIECEWISE):  35%|███▌      | 18/51 [00:09<00:09,  3.33it/s]
Capturing CUDA graphs (PIECEWISE):  37%|███▋      | 19/51 [00:09<00:08,  3.86it/s]
Capturing CUDA graphs (PIECEWISE):  39%|███▉      | 20/51 [00:09<00:06,  4.47it/s]
Capturing CUDA graphs (PIECEWISE):  41%|████      | 21/51 [00:10<00:06,  4.81it/s]
Capturing CUDA graphs (PIECEWISE):  43%|████▎     | 22/51 [00:10<00:05,  5.03it/s]
Capturing CUDA graphs (PIECEWISE):  45%|████▌     | 23/51 [00:10<00:05,  5.48it/s]
Capturing CUDA graphs (PIECEWISE):  47%|████▋     | 24/51 [00:10<00:04,  5.69it/s]
Capturing CUDA graphs (PIECEWISE):  49%|████▉     | 25/51 [00:10<00:04,  5.67it/s]
Capturing CUDA graphs (PIECEWISE):  51%|█████     | 26/51 [00:10<00:04,  5.87it/s]
Capturing CUDA graphs (PIECEWISE):  53%|█████▎    | 27/51 [00:11<00:03,  6.07it/s]
Capturing CUDA graphs (PIECEWISE):  55%|█████▍    | 28/51 [00:11<00:03,  5.99it/s]
Capturing CUDA graphs (PIECEWISE):  57%|█████▋    | 29/51 [00:11<00:03,  6.24it/s]
Capturing CUDA graphs (PIECEWISE):  59%|█████▉    | 30/51 [00:11<00:03,  6.39it/s]
Capturing CUDA graphs (PIECEWISE):  61%|██████    | 31/51 [00:11<00:03,  6.18it/s]
Capturing CUDA graphs (PIECEWISE):  63%|██████▎   | 32/51 [00:11<00:03,  6.31it/s]
Capturing CUDA graphs (PIECEWISE):  65%|██████▍   | 33/51 [00:11<00:02,  6.49it/s]
Capturing CUDA graphs (PIECEWISE):  67%|██████▋   | 34/51 [00:12<00:02,  6.73it/s]
Capturing CUDA graphs (PIECEWISE):  69%|██████▊   | 35/51 [00:12<00:02,  6.50it/s]
Capturing CUDA graphs (PIECEWISE):  71%|███████   | 36/51 [00:12<00:02,  6.74it/s]
Capturing CUDA graphs (PIECEWISE):  73%|███████▎  | 37/51 [00:12<00:02,  6.90it/s]
Capturing CUDA graphs (PIECEWISE):  75%|███████▍  | 38/51 [00:12<00:02,  6.42it/s]
Capturing CUDA graphs (PIECEWISE):  76%|███████▋  | 39/51 [00:12<00:02,  5.68it/s]
Capturing CUDA graphs (PIECEWISE):  78%|███████▊  | 40/51 [00:13<00:01,  6.10it/s]
Capturing CUDA graphs (PIECEWISE):  80%|████████  | 41/51 [00:13<00:01,  6.16it/s]
Capturing CUDA graphs (PIECEWISE):  82%|████████▏ | 42/51 [00:13<00:01,  6.48it/s]
Capturing CUDA graphs (PIECEWISE):  84%|████████▍ | 43/51 [00:13<00:01,  6.70it/s]
Capturing CUDA graphs (PIECEWISE):  86%|████████▋ | 44/51 [00:13<00:01,  6.76it/s]
Capturing CUDA graphs (PIECEWISE):  88%|████████▊ | 45/51 [00:13<00:00,  6.58it/s]
Capturing CUDA graphs (PIECEWISE):  90%|█████████ | 46/51 [00:13<00:00,  7.02it/s]
Capturing CUDA graphs (PIECEWISE):  92%|█████████▏| 47/51 [00:14<00:00,  7.19it/s]
Capturing CUDA graphs (PIECEWISE):  94%|█████████▍| 48/51 [00:14<00:00,  6.92it/s]
Capturing CUDA graphs (PIECEWISE):  96%|█████████▌| 49/51 [00:14<00:00,  7.10it/s]
Capturing CUDA graphs (PIECEWISE):  98%|█████████▊| 50/51 [00:14<00:00,  7.31it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:14<00:00,  7.14it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:14<00:00,  3.48it/s]

Capturing CUDA graphs (FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL):   6%|▌         | 2/35 [00:00<00:02, 12.37it/s]
Capturing CUDA graphs (FULL):  11%|█▏        | 4/35 [00:00<00:02, 12.86it/s]
Capturing CUDA graphs (FULL):  17%|█▋        | 6/35 [00:00<00:02, 12.55it/s]
Capturing CUDA graphs (FULL):  23%|██▎       | 8/35 [00:00<00:02, 12.54it/s]
Capturing CUDA graphs (FULL):  29%|██▊       | 10/35 [00:00<00:01, 12.96it/s]
Capturing CUDA graphs (FULL):  34%|███▍      | 12/35 [00:00<00:01, 13.06it/s]
Capturing CUDA graphs (FULL):  40%|████      | 14/35 [00:01<00:01, 13.13it/s]
Capturing CUDA graphs (FULL):  46%|████▌     | 16/35 [00:01<00:01, 13.55it/s]
Capturing CUDA graphs (FULL):  51%|█████▏    | 18/35 [00:01<00:01, 13.93it/s]
Capturing CUDA graphs (FULL):  57%|█████▋    | 20/35 [00:01<00:01, 13.92it/s]
Capturing CUDA graphs (FULL):  63%|██████▎   | 22/35 [00:01<00:00, 14.61it/s]
Capturing CUDA graphs (FULL):  69%|██████▊   | 24/35 [00:01<00:00, 15.21it/s]
Capturing CUDA graphs (FULL):  74%|███████▍  | 26/35 [00:01<00:00, 15.77it/s]
Capturing CUDA graphs (FULL):  80%|████████  | 28/35 [00:01<00:00, 15.83it/s]
Capturing CUDA graphs (FULL):  86%|████████▌ | 30/35 [00:02<00:00, 16.30it/s]
Capturing CUDA graphs (FULL):  91%|█████████▏| 32/35 [00:02<00:00, 16.94it/s]
Capturing CUDA graphs (FULL):  97%|█████████▋| 34/35 [00:02<00:00, 17.56it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 35/35 [00:02<00:00, 14.80it/s]
(EngineCore pid=38829) INFO 11-06 15:47:56 [model_runner.py:1163] Graph capturing finished in 18 secs, took 0.29 GiB
(EngineCore pid=38829) INFO 11-06 15:47:56 [gpu_worker.py:1003] CUDA graph pool memory: 0.29 GiB (actual), 0.48 GiB (estimated), difference: 0.19 GiB (66.7%).
(EngineCore pid=38829) INFO 11-06 15:47:56 [gpu_worker.py:1066] Free memory on device (75.97/79.25 GiB) on startup. Desired GPU memory utilization is (0.5, 39.63 GiB). Actual usage is 17.94 GiB for consumed memory (weights + non-torch), 2.27 GiB for peak activation, and 0.29 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=20386823680` (18.99 GiB) to fit into requested memory, or `--kv-cache-memory=59416854016` (55.34 GiB) to fully utilize gpu memory. Current kv cache memory in use is 19.42 GiB.
(EngineCore pid=38829) INFO 11-06 15:47:57 [jit_monitor.py:84] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
(EngineCore pid=38829) INFO 11-06 15:47:57 [core.py:405] init engine (profile, create kv cache, warmup model) took 61.04 s (compilation: 0.71 s)
(EngineCore pid=38829) INFO 11-06 15:47:57 [kernel.py:480] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(MainProcess pid=38470) INFO 11-06 15:47:58 [hf.py:736] Detected the chat template content format to be 'openai'. You can set `--chat-template-content-format` to override this.

Rendering prompts:   0%|          | 0/32000 [00:00<?, ?it/s]
Rendering prompts:   0%|          | 59/32000 [00:00<00:55, 570.98it/s]
Rendering prompts:   0%|          | 128/32000 [00:00<00:50, 635.54it/s]
Rendering prompts:   1%|          | 192/32000 [00:00<00:50, 625.53it/s]
Rendering prompts:   1%|          | 255/32000 [00:00<00:51, 612.00it/s]
Rendering prompts:   1%|          | 317/32000 [00:00<00:55, 573.96it/s]
Rendering prompts:   1%|          | 375/32000 [00:00<00:57, 552.26it/s]
Rendering prompts:   1%|▏         | 431/32000 [00:00<01:07, 466.43it/s]
Rendering prompts:   2%|▏         | 501/32000 [00:00<00:59, 528.08it/s]
Rendering prompts:   2%|▏         | 564/32000 [00:01<00:56, 555.88it/s]
Rendering prompts:   2%|▏         | 628/32000 [00:01<00:54, 579.16it/s]
Rendering prompts:   2%|▏         | 688/32000 [00:01<00:54, 575.22it/s]
Rendering prompts:   2%|▏         | 747/32000 [00:01<00:55, 566.56it/s]
Rendering prompts:   3%|▎         | 805/32000 [00:01<00:57, 542.39it/s]
Rendering prompts:   3%|▎         | 866/32000 [00:01<00:55, 560.64it/s]
Rendering prompts:   3%|▎         | 923/32000 [00:01<00:55, 560.36it/s]
Rendering prompts:   3%|▎         | 986/32000 [00:01<00:54, 572.59it/s]
Rendering prompts:   3%|▎         | 1049/32000 [00:01<00:52, 588.26it/s]
Rendering prompts:   3%|▎         | 1112/32000 [00:01<00:51, 599.36it/s]
Rendering prompts:   4%|▎         | 1179/32000 [00:02<00:49, 618.82it/s]
Rendering prompts:   4%|▍         | 1242/32000 [00:02<00:50, 610.28it/s]
Rendering prompts:   4%|▍         | 1304/32000 [00:02<00:51, 597.18it/s]
Rendering prompts:   4%|▍         | 1364/32000 [00:02<00:51, 596.42it/s]
Rendering prompts:   4%|▍         | 1429/32000 [00:02<00:50, 611.06it/s]
Rendering prompts:   5%|▍         | 1491/32000 [00:02<00:50, 601.98it/s]
Rendering prompts:   5%|▍         | 1559/32000 [00:02<00:48, 621.53it/s]
Rendering prompts:   5%|▌         | 1622/32000 [00:02<00:49, 619.30it/s]
Rendering prompts:   5%|▌         | 1684/32000 [00:02<00:51, 587.17it/s]
Rendering prompts:   5%|▌         | 1744/32000 [00:03<00:51, 585.82it/s]
Rendering prompts:   6%|▌         | 1804/32000 [00:03<00:51, 589.50it/s]
Rendering prompts:   6%|▌         | 1864/32000 [00:03<00:54, 551.27it/s]
Rendering prompts:   6%|▌         | 1921/32000 [00:03<00:54, 554.19it/s]
Rendering prompts:   6%|▌         | 1984/32000 [00:03<00:52, 575.41it/s]
Rendering prompts:   6%|▋         | 2047/32000 [00:03<00:50, 590.72it/s]
Rendering prompts:   7%|▋         | 2109/32000 [00:03<00:50, 593.84it/s]
Rendering prompts:   7%|▋         | 2169/32000 [00:03<00:51, 584.55it/s]
Rendering prompts:   7%|▋         | 2228/32000 [00:03<00:54, 542.07it/s]
Rendering prompts:   7%|▋         | 2283/32000 [00:03<00:55, 538.62it/s]
Rendering prompts:   7%|▋         | 2339/32000 [00:04<00:54, 541.00it/s]
Rendering prompts:   8%|▊         | 2400/32000 [00:04<00:52, 559.59it/s]
Rendering prompts:   8%|▊         | 2461/32000 [00:04<00:51, 573.18it/s]
Rendering prompts:   8%|▊         | 2521/32000 [00:04<00:50, 580.27it/s]
Rendering prompts:   8%|▊         | 2580/32000 [00:04<00:51, 573.88it/s]
Rendering prompts:   8%|▊         | 2640/32000 [00:04<00:50, 580.75it/s]
Rendering prompts:   8%|▊         | 2699/32000 [00:04<00:51, 568.35it/s]
Rendering prompts:   9%|▊         | 2760/32000 [00:04<00:50, 578.60it/s]
Rendering prompts:   9%|▉         | 2825/32000 [00:04<00:48, 599.10it/s]
Rendering prompts:   9%|▉         | 2886/32000 [00:05<00:51, 564.77it/s]
Rendering prompts:   9%|▉         | 2946/32000 [00:05<00:50, 574.40it/s]
Rendering prompts:   9%|▉         | 3004/32000 [00:05<00:50, 573.80it/s]
Rendering prompts:  10%|▉         | 3063/32000 [00:05<00:50, 575.12it/s]
Rendering prompts:  10%|▉         | 3121/32000 [00:05<00:50, 571.51it/s]
Rendering prompts:  10%|▉         | 3179/32000 [00:05<00:50, 566.04it/s]
Rendering prompts:  10%|█         | 3239/32000 [00:05<00:50, 575.02it/s]
Rendering prompts:  10%|█         | 3298/32000 [00:05<00:49, 578.13it/s]
Rendering prompts:  10%|█         | 3360/32000 [00:05<00:48, 588.37it/s]
Rendering prompts:  11%|█         | 3419/32000 [00:05<00:49, 582.29it/s]
Rendering prompts:  11%|█         | 3478/32000 [00:06<00:49, 576.58it/s]
Rendering prompts:  11%|█         | 3542/32000 [00:06<00:49, 578.14it/s]
Rendering prompts:  11%|█▏        | 3600/32000 [00:06<00:51, 553.83it/s]
Rendering prompts:  11%|█▏        | 3656/32000 [00:06<00:51, 553.48it/s]
Rendering prompts:  12%|█▏        | 3715/32000 [00:06<00:50, 563.47it/s]
Rendering prompts:  12%|█▏        | 3772/32000 [00:06<00:49, 565.27it/s]
Rendering prompts:  12%|█▏        | 3829/32000 [00:06<00:50, 561.04it/s]
Rendering prompts:  12%|█▏        | 3888/32000 [00:06<00:49, 567.45it/s]
Rendering prompts:  12%|█▏        | 3945/32000 [00:06<00:50, 559.52it/s]
Rendering prompts:  13%|█▎        | 4010/32000 [00:06<00:47, 584.92it/s]
Rendering prompts:  13%|█▎        | 4069/32000 [00:07<00:47, 581.98it/s]
Rendering prompts:  13%|█▎        | 4128/32000 [00:07<00:49, 559.99it/s]
Rendering prompts:  13%|█▎        | 4192/32000 [00:07<00:47, 580.97it/s]
Rendering prompts:  13%|█▎        | 4251/32000 [00:07<00:48, 568.95it/s]
Rendering prompts:  13%|█▎        | 4316/32000 [00:07<00:46, 589.41it/s]
Rendering prompts:  14%|█▎        | 4378/32000 [00:07<00:46, 597.11it/s]
Rendering prompts:  14%|█▍        | 4441/32000 [00:07<00:45, 606.13it/s]
Rendering prompts:  14%|█▍        | 4502/32000 [00:07<00:46, 589.49it/s]
Rendering prompts:  14%|█▍        | 4562/32000 [00:07<00:46, 591.68it/s]
Rendering prompts:  14%|█▍        | 4623/32000 [00:08<00:45, 596.73it/s]
Rendering prompts:  15%|█▍        | 4689/32000 [00:08<00:44, 613.30it/s]
Rendering prompts:  15%|█▍        | 4751/32000 [00:08<00:44, 609.86it/s]
Rendering prompts:  15%|█▌        | 4813/32000 [00:08<00:48, 557.88it/s]
Rendering prompts:  15%|█▌        | 4870/32000 [00:08<00:50, 541.40it/s]
Rendering prompts:  15%|█▌        | 4929/32000 [00:08<00:48, 553.70it/s]
Rendering prompts:  16%|█▌        | 4986/32000 [00:08<00:49, 550.27it/s]
Rendering prompts:  16%|█▌        | 5042/32000 [00:08<00:49, 548.52it/s]
Rendering prompts:  16%|█▌        | 5104/32000 [00:08<00:47, 562.61it/s]
Rendering prompts:  16%|█▌        | 5161/32000 [00:08<00:48, 552.82it/s]
Rendering prompts:  16%|█▋        | 5222/32000 [00:09<00:47, 567.92it/s]
Rendering prompts:  16%|█▋        | 5279/32000 [00:09<00:48, 553.61it/s]
Rendering prompts:  17%|█▋        | 5345/32000 [00:09<00:45, 584.03it/s]
Rendering prompts:  17%|█▋        | 5404/32000 [00:09<00:47, 565.13it/s]
Rendering prompts:  17%|█▋        | 5461/32000 [00:09<00:47, 553.37it/s]
Rendering prompts:  17%|█▋        | 5519/32000 [00:09<00:47, 559.88it/s]
Rendering prompts:  17%|█▋        | 5582/32000 [00:09<00:45, 579.76it/s]
Rendering prompts:  18%|█▊        | 5642/32000 [00:09<00:45, 582.04it/s]
Rendering prompts:  18%|█▊        | 5702/32000 [00:09<00:44, 585.83it/s]
Rendering prompts:  18%|█▊        | 5762/32000 [00:10<00:44, 588.98it/s]
Rendering prompts:  18%|█▊        | 5821/32000 [00:10<00:45, 581.27it/s]
Rendering prompts:  18%|█▊        | 5880/32000 [00:10<00:45, 573.59it/s]
Rendering prompts:  19%|█▊        | 5938/32000 [00:10<00:46, 564.42it/s]
Rendering prompts:  19%|█▊        | 5995/32000 [00:10<00:46, 555.28it/s]
Rendering prompts:  19%|█▉        | 6057/32000 [00:10<00:45, 571.47it/s]
Rendering prompts:  19%|█▉        | 6117/32000 [00:10<00:44, 578.67it/s]
Rendering prompts:  19%|█▉        | 6175/32000 [00:10<00:45, 573.56it/s]
Rendering prompts:  19%|█▉        | 6236/32000 [00:10<00:44, 583.97it/s]
Rendering prompts:  20%|█▉        | 6297/32000 [00:10<00:43, 591.50it/s]
Rendering prompts:  20%|█▉        | 6363/32000 [00:11<00:42, 608.95it/s]
Rendering prompts:  20%|██        | 6424/32000 [00:11<00:42, 600.13it/s]
Rendering prompts:  20%|██        | 6485/32000 [00:11<00:42, 595.31it/s]
Rendering prompts:  20%|██        | 6547/32000 [00:11<00:42, 600.65it/s]
Rendering prompts:  21%|██        | 6610/32000 [00:11<00:41, 607.85it/s]
Rendering prompts:  21%|██        | 6671/32000 [00:11<00:43, 584.32it/s]
Rendering prompts:  21%|██        | 6733/32000 [00:11<00:42, 593.74it/s]
Rendering prompts:  21%|██        | 6793/32000 [00:11<00:43, 577.68it/s]
Rendering prompts:  21%|██▏       | 6851/32000 [00:11<00:44, 571.30it/s]
Rendering prompts:  22%|██▏       | 6909/32000 [00:12<00:44, 561.54it/s]
Rendering prompts:  22%|██▏       | 6967/32000 [00:12<00:44, 563.40it/s]
Rendering prompts:  22%|██▏       | 7035/32000 [00:12<00:41, 595.36it/s]
Rendering prompts:  22%|██▏       | 7098/32000 [00:12<00:41, 604.71it/s]
Rendering prompts:  22%|██▏       | 7159/32000 [00:12<00:41, 600.24it/s]
Rendering prompts:  23%|██▎       | 7220/32000 [00:12<00:42, 581.18it/s]
Rendering prompts:  23%|██▎       | 7279/32000 [00:12<00:43, 569.27it/s]
Rendering prompts:  23%|██▎       | 7337/32000 [00:12<00:44, 550.67it/s]
Rendering prompts:  23%|██▎       | 7398/32000 [00:12<00:43, 565.98it/s]
Rendering prompts:  23%|██▎       | 7455/32000 [00:12<00:44, 553.55it/s]
Rendering prompts:  23%|██▎       | 7511/32000 [00:13<00:44, 549.70it/s]
Rendering prompts:  24%|██▎       | 7567/32000 [00:13<00:44, 543.70it/s]
Rendering prompts:  24%|██▍       | 7622/32000 [00:13<00:44, 542.85it/s]
Rendering prompts:  24%|██▍       | 7684/32000 [00:13<00:43, 564.59it/s]
Rendering prompts:  24%|██▍       | 7741/32000 [00:13<00:43, 560.09it/s]
Rendering prompts:  24%|██▍       | 7802/32000 [00:13<00:42, 573.19it/s]
Rendering prompts:  25%|██▍       | 7863/32000 [00:13<00:41, 583.10it/s]
Rendering prompts:  25%|██▍       | 7922/32000 [00:13<00:41, 583.85it/s]
Rendering prompts:  25%|██▍       | 7985/32000 [00:13<00:40, 597.30it/s]
Rendering prompts:  25%|██▌       | 8045/32000 [00:14<00:52, 457.48it/s]
Rendering prompts:  25%|██▌       | 8101/32000 [00:14<00:49, 481.44it/s]
Rendering prompts:  25%|██▌       | 8155/32000 [00:14<00:48, 494.69it/s]
Rendering prompts:  26%|██▌       | 8209/32000 [00:14<00:47, 505.69it/s]
Rendering prompts:  26%|██▌       | 8264/32000 [00:14<00:45, 516.92it/s]
Rendering prompts:  26%|██▌       | 8329/32000 [00:14<00:42, 551.13it/s]
Rendering prompts:  26%|██▌       | 8386/32000 [00:14<00:43, 547.21it/s]
Rendering prompts:  26%|██▋       | 8442/32000 [00:14<00:43, 543.98it/s]
Rendering prompts:  27%|██▋       | 8499/32000 [00:14<00:42, 550.68it/s]
Rendering prompts:  27%|██▋       | 8555/32000 [00:15<00:43, 541.33it/s]
Rendering prompts:  27%|██▋       | 8614/32000 [00:15<00:42, 554.43it/s]
Rendering prompts:  27%|██▋       | 8671/32000 [00:15<00:41, 558.55it/s]
Rendering prompts:  27%|██▋       | 8732/32000 [00:15<00:40, 572.50it/s]
Rendering prompts:  27%|██▋       | 8790/32000 [00:15<00:40, 570.91it/s]
Rendering prompts:  28%|██▊       | 8852/32000 [00:15<00:39, 583.84it/s]
Rendering prompts:  28%|██▊       | 8912/32000 [00:15<00:39, 587.87it/s]
Rendering prompts:  28%|██▊       | 8977/32000 [00:15<00:38, 601.44it/s]
Rendering prompts:  28%|██▊       | 9038/32000 [00:15<00:38, 599.92it/s]
Rendering prompts:  28%|██▊       | 9104/32000 [00:15<00:37, 617.45it/s]
Rendering prompts:  29%|██▊       | 9166/32000 [00:16<00:38, 591.42it/s]
Rendering prompts:  29%|██▉       | 9226/32000 [00:16<00:38, 588.22it/s]
Rendering prompts:  29%|██▉       | 9285/32000 [00:16<00:40, 565.13it/s]
Rendering prompts:  29%|██▉       | 9349/32000 [00:16<00:38, 585.50it/s]
Rendering prompts:  29%|██▉       | 9408/32000 [00:16<00:38, 579.86it/s]
Rendering prompts:  30%|██▉       | 9467/32000 [00:16<00:39, 564.33it/s]
Rendering prompts:  30%|██▉       | 9532/32000 [00:16<00:38, 588.64it/s]
Rendering prompts:  30%|██▉       | 9592/32000 [00:16<00:38, 577.53it/s]
Rendering prompts:  30%|███       | 9650/32000 [00:16<00:38, 575.24it/s]
Rendering prompts:  30%|███       | 9708/32000 [00:16<00:40, 556.72it/s]
Rendering prompts:  31%|███       | 9767/32000 [00:17<00:39, 564.93it/s]
Rendering prompts:  31%|███       | 9827/32000 [00:17<00:38, 571.74it/s]
Rendering prompts:  31%|███       | 9891/32000 [00:17<00:37, 590.66it/s]
Rendering prompts:  31%|███       | 9951/32000 [00:17<00:37, 590.92it/s]
Rendering prompts:  31%|███▏      | 10011/32000 [00:17<00:37, 583.40it/s]
Rendering prompts:  31%|███▏      | 10070/32000 [00:17<00:37, 584.63it/s]
Rendering prompts:  32%|███▏      | 10129/32000 [00:17<00:37, 579.46it/s]
Rendering prompts:  32%|███▏      | 10192/32000 [00:17<00:36, 591.00it/s]
Rendering prompts:  32%|███▏      | 10256/32000 [00:17<00:35, 605.30it/s]
Rendering prompts:  32%|███▏      | 10317/32000 [00:18<00:37, 583.87it/s]
Rendering prompts:  32%|███▏      | 10376/32000 [00:18<00:36, 585.57it/s]
Rendering prompts:  33%|███▎      | 10435/32000 [00:18<00:37, 580.31it/s]
Rendering prompts:  33%|███▎      | 10494/32000 [00:18<00:37, 575.61it/s]
Rendering prompts:  33%|███▎      | 10552/32000 [00:18<00:37, 565.44it/s]
Rendering prompts:  33%|███▎      | 10609/32000 [00:18<00:37, 563.34it/s]
Rendering prompts:  33%|███▎      | 10666/32000 [00:18<00:38, 559.68it/s]
Rendering prompts:  34%|███▎      | 10728/32000 [00:18<00:36, 575.65it/s]
Rendering prompts:  34%|███▎      | 10788/32000 [00:18<00:37, 571.43it/s]
Rendering prompts:  34%|███▍      | 10846/32000 [00:18<00:37, 567.85it/s]
Rendering prompts:  34%|███▍      | 10903/32000 [00:19<00:38, 551.38it/s]
Rendering prompts:  34%|███▍      | 10961/32000 [00:19<00:37, 559.10it/s]
Rendering prompts:  34%|███▍      | 11018/32000 [00:19<00:37, 556.75it/s]
Rendering prompts:  35%|███▍      | 11074/32000 [00:19<00:37, 551.69it/s]
Rendering prompts:  35%|███▍      | 11132/32000 [00:19<00:37, 558.74it/s]
Rendering prompts:  35%|███▍      | 11194/32000 [00:19<00:36, 574.67it/s]
Rendering prompts:  35%|███▌      | 11255/32000 [00:19<00:35, 583.09it/s]
Rendering prompts:  35%|███▌      | 11314/32000 [00:19<00:35, 580.88it/s]
Rendering prompts:  36%|███▌      | 11375/32000 [00:19<00:35, 588.32it/s]
Rendering prompts:  36%|███▌      | 11436/32000 [00:19<00:34, 592.10it/s]
Rendering prompts:  36%|███▌      | 11500/32000 [00:20<00:33, 604.14it/s]
Rendering prompts:  36%|███▌      | 11572/32000 [00:20<00:32, 637.59it/s]
Rendering prompts:  36%|███▋      | 11636/32000 [00:20<00:32, 630.10it/s]
Rendering prompts:  37%|███▋      | 11700/32000 [00:20<00:33, 613.93it/s]
Rendering prompts:  37%|███▋      | 11762/32000 [00:20<00:34, 583.96it/s]
Rendering prompts:  37%|███▋      | 11821/32000 [00:20<00:34, 580.97it/s]
Rendering prompts:  37%|███▋      | 11880/32000 [00:20<00:35, 568.86it/s]
Rendering prompts:  37%|███▋      | 11938/32000 [00:20<00:35, 565.64it/s]
Rendering prompts:  37%|███▋      | 11998/32000 [00:20<00:35, 571.47it/s]
Rendering prompts:  38%|███▊      | 12056/32000 [00:21<00:34, 569.99it/s]
Rendering prompts:  38%|███▊      | 12116/32000 [00:21<00:34, 577.85it/s]
Rendering prompts:  38%|███▊      | 12182/32000 [00:21<00:33, 600.29it/s]
Rendering prompts:  38%|███▊      | 12243/32000 [00:21<00:33, 583.41it/s]
Rendering prompts:  38%|███▊      | 12303/32000 [00:21<00:33, 584.50it/s]
Rendering prompts:  39%|███▊      | 12367/32000 [00:21<00:32, 597.68it/s]
Rendering prompts:  39%|███▉      | 12427/32000 [00:21<00:33, 591.56it/s]
Rendering prompts:  39%|███▉      | 12487/32000 [00:21<00:34, 557.59it/s]
Rendering prompts:  39%|███▉      | 12549/32000 [00:21<00:34, 562.77it/s]
Rendering prompts:  39%|███▉      | 12606/32000 [00:21<00:34, 564.66it/s]
Rendering prompts:  40%|███▉      | 12666/32000 [00:22<00:33, 572.94it/s]
Rendering prompts:  40%|███▉      | 12724/32000 [00:22<00:34, 562.69it/s]
Rendering prompts:  40%|███▉      | 12781/32000 [00:22<00:34, 558.15it/s]
Rendering prompts:  40%|████      | 12846/32000 [00:22<00:32, 583.80it/s]
Rendering prompts:  40%|████      | 12906/32000 [00:22<00:32, 586.86it/s]
Rendering prompts:  41%|████      | 12967/32000 [00:22<00:32, 593.62it/s]
Rendering prompts:  41%|████      | 13032/32000 [00:22<00:31, 609.97it/s]
Rendering prompts:  41%|████      | 13094/32000 [00:22<00:30, 610.58it/s]
Rendering prompts:  41%|████      | 13156/32000 [00:22<00:31, 591.86it/s]
Rendering prompts:  41%|████▏     | 13216/32000 [00:23<00:32, 574.96it/s]
Rendering prompts:  41%|████▏     | 13274/32000 [00:23<00:33, 566.84it/s]
Rendering prompts:  42%|████▏     | 13338/32000 [00:23<00:31, 586.97it/s]
Rendering prompts:  42%|████▏     | 13397/32000 [00:23<00:32, 581.12it/s]
Rendering prompts:  42%|████▏     | 13456/32000 [00:23<00:32, 571.86it/s]
Rendering prompts:  42%|████▏     | 13520/32000 [00:23<00:31, 591.54it/s]
Rendering prompts:  42%|████▏     | 13581/32000 [00:23<00:31, 594.15it/s]
Rendering prompts:  43%|████▎     | 13643/32000 [00:23<00:30, 601.52it/s]
Rendering prompts:  43%|████▎     | 13704/32000 [00:23<00:32, 569.43it/s]
Rendering prompts:  43%|████▎     | 13762/32000 [00:23<00:33, 538.85it/s]
Rendering prompts:  43%|████▎     | 13821/32000 [00:24<00:33, 550.64it/s]
Rendering prompts:  43%|████▎     | 13877/32000 [00:24<00:32, 552.60it/s]
Rendering prompts:  44%|████▎     | 13939/32000 [00:24<00:32, 562.16it/s]
Rendering prompts:  44%|████▎     | 13996/32000 [00:24<00:32, 562.32it/s]
Rendering prompts:  44%|████▍     | 14053/32000 [00:24<00:32, 559.72it/s]
Rendering prompts:  44%|████▍     | 14111/32000 [00:24<00:31, 565.05it/s]
Rendering prompts:  44%|████▍     | 14168/32000 [00:24<00:31, 560.42it/s]
Rendering prompts:  44%|████▍     | 14225/32000 [00:24<00:32, 539.81it/s]
Rendering prompts:  45%|████▍     | 14283/32000 [00:24<00:32, 548.84it/s]
Rendering prompts:  45%|████▍     | 14348/32000 [00:25<00:30, 575.95it/s]
Rendering prompts:  45%|████▌     | 14406/32000 [00:25<00:30, 574.03it/s]
Rendering prompts:  45%|████▌     | 14464/32000 [00:25<00:30, 570.48it/s]
Rendering prompts:  45%|████▌     | 14528/32000 [00:25<00:29, 590.04it/s]
Rendering prompts:  46%|████▌     | 14588/32000 [00:25<00:29, 583.11it/s]
Rendering prompts:  46%|████▌     | 14647/32000 [00:25<00:29, 579.34it/s]
Rendering prompts:  46%|████▌     | 14715/32000 [00:25<00:28, 605.33it/s]
Rendering prompts:  46%|████▌     | 14776/32000 [00:25<00:29, 584.60it/s]
Rendering prompts:  46%|████▋     | 14840/32000 [00:25<00:28, 598.29it/s]
Rendering prompts:  47%|████▋     | 14900/32000 [00:25<00:29, 586.83it/s]
Rendering prompts:  47%|████▋     | 14959/32000 [00:26<00:29, 576.31it/s]
Rendering prompts:  47%|████▋     | 15020/32000 [00:26<00:29, 585.03it/s]
Rendering prompts:  47%|████▋     | 15090/32000 [00:26<00:27, 614.54it/s]
Rendering prompts:  47%|████▋     | 15152/32000 [00:26<00:27, 607.97it/s]
Rendering prompts:  48%|████▊     | 15213/32000 [00:26<00:28, 592.80it/s]
Rendering prompts:  48%|████▊     | 15273/32000 [00:26<00:28, 585.69it/s]
Rendering prompts:  48%|████▊     | 15332/32000 [00:26<00:28, 576.95it/s]
Rendering prompts:  48%|████▊     | 15390/32000 [00:26<00:29, 562.76it/s]
Rendering prompts:  48%|████▊     | 15454/32000 [00:26<00:28, 583.76it/s]
Rendering prompts:  48%|████▊     | 15518/32000 [00:26<00:27, 599.04it/s]
Rendering prompts:  49%|████▊     | 15587/32000 [00:27<00:26, 625.24it/s]
Rendering prompts:  49%|████▉     | 15650/32000 [00:27<00:26, 608.89it/s]
Rendering prompts:  49%|████▉     | 15712/32000 [00:27<00:34, 476.49it/s]
Rendering prompts:  49%|████▉     | 15772/32000 [00:27<00:32, 505.82it/s]
Rendering prompts:  49%|████▉     | 15827/32000 [00:27<00:31, 513.04it/s]
Rendering prompts:  50%|████▉     | 15893/32000 [00:27<00:29, 549.64it/s]
Rendering prompts:  50%|████▉     | 15955/32000 [00:27<00:28, 564.06it/s]
Rendering prompts:  50%|█████     | 16022/32000 [00:27<00:26, 593.58it/s]
Rendering prompts:  50%|█████     | 16083/32000 [00:28<00:26, 589.54it/s]
Rendering prompts:  50%|█████     | 16149/32000 [00:28<00:26, 607.89it/s]
Rendering prompts:  51%|█████     | 16212/32000 [00:28<00:25, 612.86it/s]
Rendering prompts:  51%|█████     | 16277/32000 [00:28<00:25, 623.58it/s]
Rendering prompts:  51%|█████     | 16340/32000 [00:28<00:25, 604.91it/s]
Rendering prompts:  51%|█████▏    | 16401/32000 [00:28<00:26, 591.71it/s]
Rendering prompts:  51%|█████▏    | 16461/32000 [00:28<00:26, 578.90it/s]
Rendering prompts:  52%|█████▏    | 16522/32000 [00:28<00:26, 587.62it/s]
Rendering prompts:  52%|█████▏    | 16586/32000 [00:28<00:25, 601.77it/s]
Rendering prompts:  52%|█████▏    | 16647/32000 [00:28<00:26, 581.80it/s]
Rendering prompts:  52%|█████▏    | 16713/32000 [00:29<00:25, 603.18it/s]
Rendering prompts:  52%|█████▏    | 16777/32000 [00:29<00:24, 610.97it/s]
Rendering prompts:  53%|█████▎    | 16843/32000 [00:29<00:24, 624.89it/s]
Rendering prompts:  53%|█████▎    | 16906/32000 [00:29<00:24, 619.77it/s]
Rendering prompts:  53%|█████▎    | 16969/32000 [00:29<00:24, 618.16it/s]
Rendering prompts:  53%|█████▎    | 17031/32000 [00:29<00:24, 616.46it/s]
Rendering prompts:  53%|█████▎    | 17094/32000 [00:29<00:24, 619.93it/s]
Rendering prompts:  54%|█████▎    | 17157/32000 [00:29<00:24, 605.81it/s]
Rendering prompts:  54%|█████▍    | 17219/32000 [00:29<00:24, 607.53it/s]
Rendering prompts:  54%|█████▍    | 17285/32000 [00:29<00:23, 622.50it/s]
Rendering prompts:  54%|█████▍    | 17348/32000 [00:30<00:23, 616.58it/s]
Rendering prompts:  54%|█████▍    | 17410/32000 [00:30<00:24, 606.61it/s]
Rendering prompts:  55%|█████▍    | 17471/32000 [00:30<00:25, 578.95it/s]
Rendering prompts:  55%|█████▍    | 17534/32000 [00:30<00:24, 591.62it/s]
Rendering prompts:  55%|█████▍    | 17594/32000 [00:30<00:24, 578.54it/s]
Rendering prompts:  55%|█████▌    | 17659/32000 [00:30<00:23, 598.15it/s]
Rendering prompts:  55%|█████▌    | 17721/32000 [00:30<00:23, 603.01it/s]
Rendering prompts:  56%|█████▌    | 17783/32000 [00:30<00:23, 607.75it/s]
Rendering prompts:  56%|█████▌    | 17845/32000 [00:30<00:23, 608.73it/s]
Rendering prompts:  56%|█████▌    | 17908/32000 [00:31<00:22, 613.73it/s]
Rendering prompts:  56%|█████▌    | 17970/32000 [00:31<00:23, 603.06it/s]
Rendering prompts:  56%|█████▋    | 18031/32000 [00:31<00:23, 596.90it/s]
Rendering prompts:  57%|█████▋    | 18094/32000 [00:31<00:22, 606.38it/s]
Rendering prompts:  57%|█████▋    | 18155/32000 [00:31<00:23, 598.50it/s]
Rendering prompts:  57%|█████▋    | 18224/32000 [00:31<00:22, 620.46it/s]
Rendering prompts:  57%|█████▋    | 18289/32000 [00:31<00:21, 628.40it/s]
Rendering prompts:  57%|█████▋    | 18352/32000 [00:31<00:21, 624.83it/s]
Rendering prompts:  58%|█████▊    | 18417/32000 [00:31<00:21, 628.23it/s]
Rendering prompts:  58%|█████▊    | 18480/32000 [00:31<00:22, 613.65it/s]
Rendering prompts:  58%|█████▊    | 18551/32000 [00:32<00:21, 626.41it/s]
Rendering prompts:  58%|█████▊    | 18614/32000 [00:32<00:21, 610.75it/s]
Rendering prompts:  58%|█████▊    | 18676/32000 [00:32<00:22, 605.27it/s]
Rendering prompts:  59%|█████▊    | 18737/32000 [00:32<00:22, 602.29it/s]
Rendering prompts:  59%|█████▊    | 18798/32000 [00:32<00:22, 592.36it/s]
Rendering prompts:  59%|█████▉    | 18858/32000 [00:32<00:22, 575.45it/s]
Rendering prompts:  59%|█████▉    | 18922/32000 [00:32<00:22, 589.12it/s]
Rendering prompts:  59%|█████▉    | 18986/32000 [00:32<00:21, 602.31it/s]
Rendering prompts:  60%|█████▉    | 19048/32000 [00:32<00:21, 605.71it/s]
Rendering prompts:  60%|█████▉    | 19109/32000 [00:33<00:21, 595.11it/s]
Rendering prompts:  60%|█████▉    | 19169/32000 [00:33<00:21, 587.58it/s]
Rendering prompts:  60%|██████    | 19228/32000 [00:33<00:22, 564.14it/s]
Rendering prompts:  60%|██████    | 19285/32000 [00:33<00:23, 546.17it/s]
Rendering prompts:  60%|██████    | 19340/32000 [00:33<00:23, 528.82it/s]
Rendering prompts:  61%|██████    | 19400/32000 [00:33<00:23, 547.68it/s]
Rendering prompts:  61%|██████    | 19462/32000 [00:33<00:22, 567.71it/s]
Rendering prompts:  61%|██████    | 19527/32000 [00:33<00:21, 590.75it/s]
Rendering prompts:  61%|██████    | 19593/32000 [00:33<00:20, 608.12it/s]
Rendering prompts:  61%|██████▏   | 19656/32000 [00:33<00:20, 610.83it/s]
Rendering prompts:  62%|██████▏   | 19721/32000 [00:34<00:19, 619.03it/s]
Rendering prompts:  62%|██████▏   | 19784/32000 [00:34<00:20, 610.71it/s]
Rendering prompts:  62%|██████▏   | 19846/32000 [00:34<00:20, 604.86it/s]
Rendering prompts:  62%|██████▏   | 19907/32000 [00:34<00:20, 576.53it/s]
Rendering prompts:  62%|██████▏   | 19973/32000 [00:34<00:20, 599.64it/s]
Rendering prompts:  63%|██████▎   | 20034/32000 [00:34<00:20, 591.02it/s]
Rendering prompts:  63%|██████▎   | 20098/32000 [00:34<00:19, 602.71it/s]
Rendering prompts:  63%|██████▎   | 20159/32000 [00:34<00:19, 600.48it/s]
Rendering prompts:  63%|██████▎   | 20222/32000 [00:34<00:19, 609.06it/s]
Rendering prompts:  63%|██████▎   | 20284/32000 [00:35<00:21, 555.31it/s]
Rendering prompts:  64%|██████▎   | 20341/32000 [00:35<00:20, 556.89it/s]
Rendering prompts:  64%|██████▍   | 20405/32000 [00:35<00:20, 575.92it/s]
Rendering prompts:  64%|██████▍   | 20467/32000 [00:35<00:19, 585.43it/s]
Rendering prompts:  64%|██████▍   | 20526/32000 [00:35<00:20, 562.36it/s]
Rendering prompts:  64%|██████▍   | 20587/32000 [00:35<00:20, 570.28it/s]
Rendering prompts:  65%|██████▍   | 20647/32000 [00:35<00:19, 575.18it/s]
Rendering prompts:  65%|██████▍   | 20705/32000 [00:35<00:20, 548.92it/s]
Rendering prompts:  65%|██████▍   | 20766/32000 [00:35<00:19, 565.45it/s]
Rendering prompts:  65%|██████▌   | 20823/32000 [00:35<00:19, 564.52it/s]
Rendering prompts:  65%|██████▌   | 20880/32000 [00:36<00:20, 554.22it/s]
Rendering prompts:  65%|██████▌   | 20936/32000 [00:36<00:20, 549.13it/s]
Rendering prompts:  66%|██████▌   | 20992/32000 [00:36<00:20, 550.38it/s]
Rendering prompts:  66%|██████▌   | 21058/32000 [00:36<00:18, 580.93it/s]
Rendering prompts:  66%|██████▌   | 21118/32000 [00:36<00:18, 583.12it/s]
Rendering prompts:  66%|██████▌   | 21177/32000 [00:36<00:19, 552.48it/s]
Rendering prompts:  66%|██████▋   | 21241/32000 [00:36<00:18, 575.56it/s]
Rendering prompts:  67%|██████▋   | 21303/32000 [00:36<00:18, 586.89it/s]
Rendering prompts:  67%|██████▋   | 21367/32000 [00:36<00:17, 599.40it/s]
Rendering prompts:  67%|██████▋   | 21428/32000 [00:37<00:17, 600.90it/s]
Rendering prompts:  67%|██████▋   | 21489/32000 [00:37<00:17, 592.35it/s]
Rendering prompts:  67%|██████▋   | 21549/32000 [00:37<00:19, 547.59it/s]
Rendering prompts:  68%|██████▊   | 21610/32000 [00:37<00:18, 564.01it/s]
Rendering prompts:  68%|██████▊   | 21668/32000 [00:37<00:18, 558.07it/s]
Rendering prompts:  68%|██████▊   | 21729/32000 [00:37<00:17, 570.94it/s]
Rendering prompts:  68%|██████▊   | 21787/32000 [00:37<00:17, 569.33it/s]
Rendering prompts:  68%|██████▊   | 21846/32000 [00:37<00:17, 570.44it/s]
Rendering prompts:  68%|██████▊   | 21904/32000 [00:37<00:18, 547.19it/s]
Rendering prompts:  69%|██████▊   | 21963/32000 [00:37<00:17, 558.87it/s]
Rendering prompts:  69%|██████▉   | 22020/32000 [00:38<00:18, 547.54it/s]
Rendering prompts:  69%|██████▉   | 22077/32000 [00:38<00:17, 553.16it/s]
Rendering prompts:  69%|██████▉   | 22133/32000 [00:38<00:18, 547.18it/s]
Rendering prompts:  69%|██████▉   | 22188/32000 [00:38<00:18, 540.58it/s]
Rendering prompts:  70%|██████▉   | 22251/32000 [00:38<00:17, 564.94it/s]
Rendering prompts:  70%|██████▉   | 22318/32000 [00:38<00:16, 595.05it/s]
Rendering prompts:  70%|██████▉   | 22378/32000 [00:38<00:16, 569.12it/s]
Rendering prompts:  70%|███████   | 22438/32000 [00:38<00:16, 575.81it/s]
Rendering prompts:  70%|███████   | 22496/32000 [00:38<00:16, 572.66it/s]
Rendering prompts:  70%|███████   | 22559/32000 [00:39<00:16, 588.00it/s]
Rendering prompts:  71%|███████   | 22618/32000 [00:39<00:15, 588.20it/s]
Rendering prompts:  71%|███████   | 22677/32000 [00:39<00:16, 569.79it/s]
Rendering prompts:  71%|███████   | 22735/32000 [00:39<00:16, 558.91it/s]
Rendering prompts:  71%|███████   | 22793/32000 [00:39<00:16, 564.80it/s]
Rendering prompts:  71%|███████▏  | 22850/32000 [00:39<00:16, 549.78it/s]
Rendering prompts:  72%|███████▏  | 22911/32000 [00:39<00:16, 566.57it/s]
Rendering prompts:  72%|███████▏  | 22973/32000 [00:39<00:15, 580.47it/s]
Rendering prompts:  72%|███████▏  | 23032/32000 [00:39<00:15, 580.42it/s]
Rendering prompts:  72%|███████▏  | 23093/32000 [00:39<00:15, 579.71it/s]
Rendering prompts:  72%|███████▏  | 23160/32000 [00:40<00:14, 605.64it/s]
Rendering prompts:  73%|███████▎  | 23221/32000 [00:40<00:14, 590.51it/s]
Rendering prompts:  73%|███████▎  | 23286/32000 [00:40<00:14, 606.42it/s]
Rendering prompts:  73%|███████▎  | 23350/32000 [00:40<00:14, 612.68it/s]
Rendering prompts:  73%|███████▎  | 23412/32000 [00:40<00:13, 613.75it/s]
Rendering prompts:  73%|███████▎  | 23474/32000 [00:40<00:14, 605.85it/s]
Rendering prompts:  74%|███████▎  | 23539/32000 [00:40<00:13, 615.45it/s]
Rendering prompts:  74%|███████▍  | 23603/32000 [00:40<00:13, 612.08it/s]
Rendering prompts:  74%|███████▍  | 23665/32000 [00:40<00:13, 598.80it/s]
Rendering prompts:  74%|███████▍  | 23725/32000 [00:41<00:18, 449.01it/s]
Rendering prompts:  74%|███████▍  | 23786/32000 [00:41<00:16, 485.72it/s]
Rendering prompts:  75%|███████▍  | 23844/32000 [00:41<00:16, 507.60it/s]
Rendering prompts:  75%|███████▍  | 23903/32000 [00:41<00:15, 529.08it/s]
Rendering prompts:  75%|███████▍  | 23964/32000 [00:41<00:14, 549.94it/s]
Rendering prompts:  75%|███████▌  | 24022/32000 [00:41<00:14, 548.39it/s]
Rendering prompts:  75%|███████▌  | 24091/32000 [00:41<00:13, 586.13it/s]
Rendering prompts:  76%|███████▌  | 24163/32000 [00:41<00:12, 621.10it/s]
Rendering prompts:  76%|███████▌  | 24227/32000 [00:41<00:12, 609.40it/s]
Rendering prompts:  76%|███████▌  | 24295/32000 [00:42<00:12, 626.75it/s]
Rendering prompts:  76%|███████▌  | 24360/32000 [00:42<00:12, 633.22it/s]
Rendering prompts:  76%|███████▋  | 24424/32000 [00:42<00:12, 610.49it/s]
Rendering prompts:  77%|███████▋  | 24486/32000 [00:42<00:12, 589.31it/s]
Rendering prompts:  77%|███████▋  | 24555/32000 [00:42<00:12, 617.02it/s]
Rendering prompts:  77%|███████▋  | 24618/32000 [00:42<00:12, 601.07it/s]
Rendering prompts:  77%|███████▋  | 24682/32000 [00:42<00:11, 611.60it/s]
Rendering prompts:  77%|███████▋  | 24744/32000 [00:42<00:12, 598.14it/s]
Rendering prompts:  78%|███████▊  | 24805/32000 [00:42<00:12, 598.46it/s]
Rendering prompts:  78%|███████▊  | 24866/32000 [00:43<00:12, 581.95it/s]
Rendering prompts:  78%|███████▊  | 24925/32000 [00:43<00:12, 575.11it/s]
Rendering prompts:  78%|███████▊  | 24988/32000 [00:43<00:11, 587.97it/s]
Rendering prompts:  78%|███████▊  | 25048/32000 [00:43<00:11, 589.71it/s]
Rendering prompts:  78%|███████▊  | 25109/32000 [00:43<00:11, 594.57it/s]
Rendering prompts:  79%|███████▊  | 25184/32000 [00:43<00:10, 637.84it/s]
Rendering prompts:  79%|███████▉  | 25248/32000 [00:43<00:11, 593.62it/s]
Rendering prompts:  79%|███████▉  | 25309/32000 [00:43<00:11, 597.63it/s]
Rendering prompts:  79%|███████▉  | 25370/32000 [00:43<00:11, 593.08it/s]
Rendering prompts:  79%|███████▉  | 25430/32000 [00:43<00:11, 585.33it/s]
Rendering prompts:  80%|███████▉  | 25499/32000 [00:44<00:10, 611.42it/s]
Rendering prompts:  80%|███████▉  | 25564/32000 [00:44<00:10, 621.38it/s]
Rendering prompts:  80%|████████  | 25627/32000 [00:44<00:10, 599.37it/s]
Rendering prompts:  80%|████████  | 25689/32000 [00:44<00:10, 597.02it/s]
Rendering prompts:  80%|████████  | 25749/32000 [00:44<00:10, 592.91it/s]
Rendering prompts:  81%|████████  | 25812/32000 [00:44<00:10, 594.65it/s]
Rendering prompts:  81%|████████  | 25872/32000 [00:44<00:10, 558.78it/s]
Rendering prompts:  81%|████████  | 25939/32000 [00:44<00:10, 588.23it/s]
Rendering prompts:  81%|████████  | 25999/32000 [00:44<00:10, 578.64it/s]
Rendering prompts:  81%|████████▏ | 26065/32000 [00:45<00:09, 597.61it/s]
Rendering prompts:  82%|████████▏ | 26126/32000 [00:45<00:09, 587.96it/s]
Rendering prompts:  82%|████████▏ | 26186/32000 [00:45<00:09, 584.08it/s]
Rendering prompts:  82%|████████▏ | 26256/32000 [00:45<00:09, 616.95it/s]
Rendering prompts:  82%|████████▏ | 26318/32000 [00:45<00:09, 575.06it/s]
Rendering prompts:  82%|████████▏ | 26377/32000 [00:45<00:09, 575.07it/s]
Rendering prompts:  83%|████████▎ | 26435/32000 [00:45<00:10, 541.86it/s]
Rendering prompts:  83%|████████▎ | 26502/32000 [00:45<00:09, 576.83it/s]
Rendering prompts:  83%|████████▎ | 26561/32000 [00:45<00:09, 561.97it/s]
Rendering prompts:  83%|████████▎ | 26628/32000 [00:46<00:09, 591.84it/s]
Rendering prompts:  83%|████████▎ | 26688/32000 [00:46<00:09, 575.25it/s]
Rendering prompts:  84%|████████▎ | 26751/32000 [00:46<00:08, 588.03it/s]
Rendering prompts:  84%|████████▍ | 26817/32000 [00:46<00:08, 604.39it/s]
Rendering prompts:  84%|████████▍ | 26878/32000 [00:46<00:08, 597.68it/s]
Rendering prompts:  84%|████████▍ | 26940/32000 [00:46<00:08, 604.08it/s]
Rendering prompts:  84%|████████▍ | 27001/32000 [00:46<00:08, 590.74it/s]
Rendering prompts:  85%|████████▍ | 27061/32000 [00:46<00:08, 589.23it/s]
Rendering prompts:  85%|████████▍ | 27124/32000 [00:46<00:08, 596.94it/s]
Rendering prompts:  85%|████████▍ | 27189/32000 [00:46<00:07, 611.19it/s]
Rendering prompts:  85%|████████▌ | 27251/32000 [00:47<00:08, 590.97it/s]
Rendering prompts:  85%|████████▌ | 27311/32000 [00:47<00:08, 577.46it/s]
Rendering prompts:  86%|████████▌ | 27369/32000 [00:47<00:08, 576.67it/s]
Rendering prompts:  86%|████████▌ | 27431/32000 [00:47<00:07, 588.02it/s]
Rendering prompts:  86%|████████▌ | 27490/32000 [00:47<00:07, 577.67it/s]
Rendering prompts:  86%|████████▌ | 27552/32000 [00:47<00:07, 588.31it/s]
Rendering prompts:  86%|████████▋ | 27617/32000 [00:47<00:07, 604.81it/s]
Rendering prompts:  86%|████████▋ | 27678/32000 [00:47<00:07, 597.43it/s]
Rendering prompts:  87%|████████▋ | 27738/32000 [00:47<00:07, 593.51it/s]
Rendering prompts:  87%|████████▋ | 27798/32000 [00:47<00:07, 592.85it/s]
Rendering prompts:  87%|████████▋ | 27858/32000 [00:48<00:07, 574.11it/s]
Rendering prompts:  87%|████████▋ | 27920/32000 [00:48<00:06, 586.11it/s]
Rendering prompts:  87%|████████▋ | 27979/32000 [00:48<00:06, 586.17it/s]
Rendering prompts:  88%|████████▊ | 28038/32000 [00:48<00:06, 581.21it/s]
Rendering prompts:  88%|████████▊ | 28097/32000 [00:48<00:06, 566.03it/s]
Rendering prompts:  88%|████████▊ | 28161/32000 [00:48<00:06, 586.49it/s]
Rendering prompts:  88%|████████▊ | 28220/32000 [00:48<00:06, 585.58it/s]
Rendering prompts:  88%|████████▊ | 28279/32000 [00:48<00:06, 564.87it/s]
Rendering prompts:  89%|████████▊ | 28337/32000 [00:48<00:06, 569.00it/s]
Rendering prompts:  89%|████████▉ | 28403/32000 [00:49<00:06, 589.30it/s]
Rendering prompts:  89%|████████▉ | 28465/32000 [00:49<00:05, 595.83it/s]
Rendering prompts:  89%|████████▉ | 28525/32000 [00:49<00:05, 594.11it/s]
Rendering prompts:  89%|████████▉ | 28585/32000 [00:49<00:05, 588.62it/s]
Rendering prompts:  90%|████████▉ | 28644/32000 [00:49<00:05, 569.62it/s]
Rendering prompts:  90%|████████▉ | 28704/32000 [00:49<00:05, 577.67it/s]
Rendering prompts:  90%|████████▉ | 28771/32000 [00:49<00:05, 603.53it/s]
Rendering prompts:  90%|█████████ | 28842/32000 [00:49<00:05, 631.05it/s]
Rendering prompts:  90%|█████████ | 28906/32000 [00:49<00:04, 623.60it/s]
Rendering prompts:  91%|█████████ | 28969/32000 [00:49<00:05, 603.21it/s]
Rendering prompts:  91%|█████████ | 29030/32000 [00:50<00:04, 597.22it/s]
Rendering prompts:  91%|█████████ | 29093/32000 [00:50<00:04, 604.36it/s]
Rendering prompts:  91%|█████████ | 29165/32000 [00:50<00:04, 635.94it/s]
Rendering prompts:  91%|█████████▏| 29229/32000 [00:50<00:04, 629.98it/s]
Rendering prompts:  92%|█████████▏| 29293/32000 [00:50<00:04, 591.42it/s]
Rendering prompts:  92%|█████████▏| 29355/32000 [00:50<00:04, 595.57it/s]
Rendering prompts:  92%|█████████▏| 29423/32000 [00:50<00:04, 606.95it/s]
Rendering prompts:  92%|█████████▏| 29484/32000 [00:50<00:04, 601.05it/s]
Rendering prompts:  92%|█████████▏| 29550/32000 [00:50<00:03, 615.92it/s]
Rendering prompts:  93%|█████████▎| 29619/32000 [00:51<00:03, 636.37it/s]
Rendering prompts:  93%|█████████▎| 29683/32000 [00:51<00:03, 631.76it/s]
Rendering prompts:  93%|█████████▎| 29759/32000 [00:51<00:03, 667.40it/s]
Rendering prompts:  93%|█████████▎| 29826/32000 [00:51<00:03, 618.63it/s]
Rendering prompts:  93%|█████████▎| 29889/32000 [00:51<00:03, 585.42it/s]
Rendering prompts:  94%|█████████▎| 29949/32000 [00:51<00:03, 587.59it/s]
Rendering prompts:  94%|█████████▍| 30009/32000 [00:51<00:03, 583.85it/s]
Rendering prompts:  94%|█████████▍| 30068/32000 [00:51<00:03, 579.32it/s]
Rendering prompts:  94%|█████████▍| 30127/32000 [00:51<00:03, 577.26it/s]
Rendering prompts:  94%|█████████▍| 30187/32000 [00:51<00:03, 582.67it/s]
Rendering prompts:  95%|█████████▍| 30246/32000 [00:52<00:03, 578.76it/s]
Rendering prompts:  95%|█████████▍| 30305/32000 [00:52<00:02, 581.54it/s]
Rendering prompts:  95%|█████████▍| 30367/32000 [00:52<00:02, 591.86it/s]
Rendering prompts:  95%|█████████▌| 30433/32000 [00:52<00:02, 610.49it/s]
Rendering prompts:  95%|█████████▌| 30495/32000 [00:52<00:02, 602.71it/s]
Rendering prompts:  95%|█████████▌| 30556/32000 [00:52<00:02, 581.87it/s]
Rendering prompts:  96%|█████████▌| 30621/32000 [00:52<00:02, 600.44it/s]
Rendering prompts:  96%|█████████▌| 30682/32000 [00:52<00:02, 591.22it/s]
Rendering prompts:  96%|█████████▌| 30744/32000 [00:52<00:02, 597.32it/s]
Rendering prompts:  96%|█████████▋| 30804/32000 [00:53<00:02, 575.22it/s]
Rendering prompts:  96%|█████████▋| 30870/32000 [00:53<00:01, 598.15it/s]
Rendering prompts:  97%|█████████▋| 30931/32000 [00:53<00:01, 588.85it/s]
Rendering prompts:  97%|█████████▋| 30996/32000 [00:53<00:01, 606.44it/s]
Rendering prompts:  97%|█████████▋| 31057/32000 [00:53<00:01, 582.68it/s]
Rendering prompts:  97%|█████████▋| 31117/32000 [00:53<00:01, 584.94it/s]
Rendering prompts:  97%|█████████▋| 31176/32000 [00:53<00:01, 575.15it/s]
Rendering prompts:  98%|█████████▊| 31234/32000 [00:53<00:01, 561.40it/s]
Rendering prompts:  98%|█████████▊| 31294/32000 [00:53<00:01, 571.24it/s]
Rendering prompts:  98%|█████████▊| 31355/32000 [00:53<00:01, 582.21it/s]
Rendering prompts:  98%|█████████▊| 31414/32000 [00:54<00:01, 556.10it/s]
Rendering prompts:  98%|█████████▊| 31478/32000 [00:54<00:00, 579.01it/s]
Rendering prompts:  99%|█████████▊| 31537/32000 [00:54<00:00, 576.13it/s]
Rendering prompts:  99%|█████████▊| 31595/32000 [00:54<00:00, 574.71it/s]
Rendering prompts:  99%|█████████▉| 31653/32000 [00:54<00:00, 552.95it/s]
Rendering prompts:  99%|█████████▉| 31718/32000 [00:54<00:00, 579.80it/s]
Rendering prompts:  99%|█████████▉| 31782/32000 [00:54<00:00, 595.71it/s]
Rendering prompts: 100%|█████████▉| 31844/32000 [00:54<00:00, 602.12it/s]
Rendering prompts: 100%|█████████▉| 31906/32000 [00:54<00:00, 606.00it/s]
Rendering prompts: 100%|█████████▉| 31967/32000 [00:55<00:00, 577.53it/s]
Rendering prompts: 100%|██████████| 32000/32000 [00:55<00:00, 580.96it/s]

Processed prompts:   0%|          | 0/32000 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts:   0%|          | 1/32000 [00:00<54:52,  9.72it/s, est. speed input: 3296.19 toks/s, output: 456.80 toks/s]
Processed prompts:   0%|          | 79/32000 [00:00<03:02, 174.59it/s, est. speed input: 107855.86 toks/s, output: 14382.31 toks/s]
Processed prompts:   0%|          | 92/32000 [00:02<18:29, 28.76it/s, est. speed input: 24424.59 toks/s, output: 3270.31 toks/s]   
Processed prompts:   0%|          | 100/32000 [00:04<30:59, 17.15it/s, est. speed input: 17095.79 toks/s, output: 2205.60 toks/s]
Processed prompts:   0%|          | 105/32000 [00:04<38:01, 13.98it/s, est. speed input: 14766.18 toks/s, output: 1926.74 toks/s]
Processed prompts:   0%|          | 109/32000 [00:05<44:35, 11.92it/s, est. speed input: 13268.64 toks/s, output: 1760.11 toks/s]
Processed prompts:   0%|          | 112/32000 [00:05<44:20, 11.99it/s, est. speed input: 13130.17 toks/s, output: 1746.31 toks/s]
Processed prompts:   0%|          | 114/32000 [00:07<1:06:48,  7.96it/s, est. speed input: 11304.65 toks/s, output: 1514.36 toks/s]
Processed prompts:   0%|          | 117/32000 [00:07<1:12:57,  7.28it/s, est. speed input: 10667.93 toks/s, output: 1440.97 toks/s]
Processed prompts:   0%|          | 119/32000 [00:07<1:17:03,  6.90it/s, est. speed input: 10380.93 toks/s, output: 1399.14 toks/s]
Processed prompts:   0%|          | 125/32000 [00:08<1:04:51,  8.19it/s, est. speed input: 10386.38 toks/s, output: 1397.24 toks/s]
Processed prompts:   0%|          | 128/32000 [00:09<1:40:07,  5.31it/s, est. speed input: 9263.27 toks/s, output: 1242.95 toks/s] 
Processed prompts:   0%|          | 131/32000 [00:10<1:47:14,  4.95it/s, est. speed input: 8828.09 toks/s, output: 1188.78 toks/s]
Processed prompts:   0%|          | 135/32000 [00:10<1:21:56,  6.48it/s, est. speed input: 9055.92 toks/s, output: 1212.53 toks/s]
Processed prompts:   0%|          | 137/32000 [00:11<1:41:19,  5.24it/s, est. speed input: 8628.72 toks/s, output: 1157.11 toks/s]
Processed prompts:   0%|          | 138/32000 [00:11<1:41:01,  5.26it/s, est. speed input: 8561.94 toks/s, output: 1149.70 toks/s]
Processed prompts:   0%|          | 139/32000 [00:11<1:45:25,  5.04it/s, est. speed input: 8447.38 toks/s, output: 1135.89 toks/s]
Processed prompts:   0%|          | 143/32000 [00:12<1:24:49,  6.26it/s, est. speed input: 8513.25 toks/s, output: 1133.30 toks/s]
Processed prompts:   0%|          | 147/32000 [00:13<1:37:00,  5.47it/s, est. speed input: 8253.60 toks/s, output: 1093.59 toks/s]
Processed prompts:   0%|          | 148/32000 [00:13<1:56:57,  4.54it/s, est. speed input: 8032.15 toks/s, output: 1064.03 toks/s]
Processed prompts:   0%|          | 151/32000 [00:13<1:34:24,  5.62it/s, est. speed input: 8044.33 toks/s, output: 1068.47 toks/s]
Processed prompts:   0%|          | 153/32000 [00:14<2:08:15,  4.14it/s, est. speed input: 7623.05 toks/s, output: 1023.14 toks/s]
Processed prompts:   0%|          | 156/32000 [00:15<2:07:04,  4.18it/s, est. speed input: 7447.46 toks/s, output: 1001.86 toks/s]
Processed prompts:   0%|          | 159/32000 [00:15<1:35:48,  5.54it/s, est. speed input: 7547.86 toks/s, output: 1015.21 toks/s]
Processed prompts:   1%|          | 162/32000 [00:16<1:33:43,  5.66it/s, est. speed input: 7459.24 toks/s, output: 1007.78 toks/s]
Processed prompts:   1%|          | 165/32000 [00:16<1:31:45,  5.78it/s, est. speed input: 7432.76 toks/s, output: 1001.43 toks/s]
Processed prompts:   1%|          | 167/32000 [00:17<1:29:41,  5.91it/s, est. speed input: 7385.78 toks/s, output: 999.04 toks/s] 
Processed prompts:   1%|          | 169/32000 [00:18<2:13:34,  3.97it/s, est. speed input: 7003.79 toks/s, output: 958.96 toks/s]
Processed prompts:   1%|          | 172/32000 [00:18<1:40:48,  5.26it/s, est. speed input: 6999.61 toks/s, output: 969.47 toks/s]
Processed prompts:   1%|          | 173/32000 [00:18<2:11:34,  4.03it/s, est. speed input: 6802.73 toks/s, output: 945.72 toks/s]
Processed prompts:   1%|          | 174/32000 [00:19<2:15:39,  3.91it/s, est. speed input: 6738.12 toks/s, output: 938.90 toks/s]
Processed prompts:   1%|          | 175/32000 [00:19<2:08:20,  4.13it/s, est. speed input: 6693.23 toks/s, output: 933.71 toks/s]
Processed prompts:   1%|          | 176/32000 [00:20<3:01:47,  2.92it/s, est. speed input: 6527.44 toks/s, output: 908.44 toks/s]
Processed prompts:   1%|          | 177/32000 [00:20<2:49:01,  3.14it/s, est. speed input: 6519.66 toks/s, output: 904.80 toks/s]
Processed prompts:   1%|          | 178/32000 [00:20<2:20:18,  3.78it/s, est. speed input: 6539.62 toks/s, output: 907.23 toks/s]
Processed prompts:   1%|          | 180/32000 [00:20<2:03:12,  4.30it/s, est. speed input: 6505.85 toks/s, output: 904.66 toks/s]
Processed prompts:   1%|          | 182/32000 [00:21<2:19:05,  3.81it/s, est. speed input: 6401.40 toks/s, output: 891.52 toks/s]
Processed prompts:   1%|          | 184/32000 [00:21<1:53:02,  4.69it/s, est. speed input: 6421.10 toks/s, output: 891.22 toks/s]
Processed prompts:   1%|          | 185/32000 [00:22<2:08:26,  4.13it/s, est. speed input: 6352.85 toks/s, output: 883.33 toks/s]
Processed prompts:   1%|          | 186/32000 [00:22<2:08:27,  4.13it/s, est. speed input: 6313.39 toks/s, output: 880.49 toks/s]
Processed prompts:   1%|          | 188/32000 [00:22<1:29:31,  5.92it/s, est. speed input: 6365.64 toks/s, output: 885.98 toks/s]
Processed prompts:   1%|          | 189/32000 [00:22<2:15:12,  3.92it/s, est. speed input: 6247.43 toks/s, output: 871.17 toks/s]
Processed prompts:   1%|          | 192/32000 [00:23<2:04:44,  4.25it/s, est. speed input: 6148.12 toks/s, output: 862.49 toks/s]
Processed prompts:   1%|          | 193/32000 [00:23<2:07:52,  4.15it/s, est. speed input: 6099.41 toks/s, output: 856.06 toks/s]
Processed prompts:   1%|          | 194/32000 [00:24<2:26:13,  3.63it/s, est. speed input: 6010.33 toks/s, output: 848.45 toks/s]
Processed prompts:   1%|          | 197/32000 [00:24<1:39:56,  5.30it/s, est. speed input: 6029.60 toks/s, output: 854.46 toks/s]
Processed prompts:   1%|          | 199/32000 [00:25<2:00:38,  4.39it/s, est. speed input: 6059.82 toks/s, output: 845.08 toks/s]
Processed prompts:   1%|          | 200/32000 [00:25<2:21:40,  3.74it/s, est. speed input: 6006.48 toks/s, output: 836.96 toks/s]
Processed prompts:   1%|          | 205/32000 [00:26<1:28:19,  6.00it/s, est. speed input: 6081.90 toks/s, output: 845.74 toks/s]
Processed prompts:   1%|          | 206/32000 [00:26<2:17:01,  3.87it/s, est. speed input: 5934.69 toks/s, output: 825.97 toks/s]
Processed prompts:   1%|          | 208/32000 [00:27<1:59:17,  4.44it/s, est. speed input: 5917.76 toks/s, output: 825.50 toks/s]
Processed prompts:   1%|          | 209/32000 [00:27<1:48:15,  4.89it/s, est. speed input: 5925.81 toks/s, output: 827.96 toks/s]
Processed prompts:   1%|          | 210/32000 [00:27<2:27:40,  3.59it/s, est. speed input: 5815.75 toks/s, output: 816.59 toks/s]
Processed prompts:   1%|          | 214/32000 [00:28<2:16:25,  3.88it/s, est. speed input: 5703.88 toks/s, output: 806.35 toks/s]
Processed prompts:   1%|          | 215/32000 [00:29<2:31:08,  3.50it/s, est. speed input: 5644.70 toks/s, output: 797.34 toks/s]
Processed prompts:   1%|          | 218/32000 [00:29<1:47:23,  4.93it/s, est. speed input: 5689.20 toks/s, output: 801.63 toks/s]
Processed prompts:   1%|          | 219/32000 [00:29<2:06:16,  4.19it/s, est. speed input: 5648.51 toks/s, output: 796.28 toks/s]
Processed prompts:   1%|          | 223/32000 [00:30<2:03:33,  4.29it/s, est. speed input: 5526.88 toks/s, output: 787.11 toks/s]
Processed prompts:   1%|          | 224/32000 [00:31<2:07:42,  4.15it/s, est. speed input: 5498.61 toks/s, output: 785.25 toks/s]
Processed prompts:   1%|          | 226/32000 [00:31<1:47:42,  4.92it/s, est. speed input: 5526.62 toks/s, output: 785.49 toks/s]
Processed prompts:   1%|          | 228/32000 [00:31<1:48:11,  4.89it/s, est. speed input: 5493.65 toks/s, output: 782.82 toks/s]
Processed prompts:   1%|          | 229/32000 [00:32<2:07:24,  4.16it/s, est. speed input: 5460.16 toks/s, output: 778.45 toks/s]
Processed prompts:   1%|          | 232/32000 [00:32<1:28:10,  6.01it/s, est. speed input: 5511.45 toks/s, output: 784.64 toks/s]
Processed prompts:   1%|          | 233/32000 [00:32<1:57:44,  4.50it/s, est. speed input: 5437.78 toks/s, output: 778.32 toks/s]
Processed prompts:   1%|          | 235/32000 [00:33<1:48:55,  4.86it/s, est. speed input: 5408.62 toks/s, output: 778.21 toks/s]
Processed prompts:   1%|          | 236/32000 [00:33<2:06:26,  4.19it/s, est. speed input: 5367.23 toks/s, output: 774.41 toks/s]
Processed prompts:   1%|          | 237/32000 [00:34<3:25:14,  2.58it/s, est. speed input: 5244.31 toks/s, output: 755.41 toks/s]
Processed prompts:   1%|          | 239/32000 [00:34<2:18:25,  3.82it/s, est. speed input: 5250.89 toks/s, output: 757.26 toks/s]
Processed prompts:   1%|          | 240/32000 [00:35<2:36:53,  3.37it/s, est. speed input: 5193.03 toks/s, output: 750.85 toks/s]
Processed prompts:   1%|          | 241/32000 [00:35<2:28:13,  3.57it/s, est. speed input: 5183.21 toks/s, output: 748.36 toks/s]
Processed prompts:   1%|          | 242/32000 [00:35<3:01:54,  2.91it/s, est. speed input: 5120.91 toks/s, output: 742.54 toks/s]
Processed prompts:   1%|          | 245/32000 [00:35<1:46:40,  4.96it/s, est. speed input: 5149.92 toks/s, output: 747.29 toks/s]
Processed prompts:   1%|          | 247/32000 [00:36<2:19:36,  3.79it/s, est. speed input: 5065.36 toks/s, output: 737.55 toks/s]
Processed prompts:   1%|          | 248/32000 [00:36<2:11:55,  4.01it/s, est. speed input: 5052.51 toks/s, output: 739.21 toks/s]
Processed prompts:   1%|          | 253/32000 [00:38<2:08:28,  4.12it/s, est. speed input: 4995.00 toks/s, output: 733.37 toks/s]
Processed prompts:   1%|          | 257/32000 [00:38<1:23:02,  6.37it/s, est. speed input: 5081.39 toks/s, output: 740.40 toks/s]
Processed prompts:   1%|          | 259/32000 [00:39<2:01:43,  4.35it/s, est. speed input: 5009.47 toks/s, output: 729.65 toks/s]
Processed prompts:   1%|          | 263/32000 [00:39<1:30:57,  5.82it/s, est. speed input: 5032.19 toks/s, output: 734.25 toks/s]
Processed prompts:   1%|          | 265/32000 [00:40<1:45:21,  5.02it/s, est. speed input: 4976.85 toks/s, output: 730.51 toks/s]
Processed prompts:   1%|          | 267/32000 [00:41<2:20:48,  3.76it/s, est. speed input: 4882.13 toks/s, output: 720.31 toks/s]
Processed prompts:   1%|          | 268/32000 [00:41<2:08:41,  4.11it/s, est. speed input: 4894.15 toks/s, output: 720.56 toks/s]
Processed prompts:   1%|          | 270/32000 [00:41<1:49:59,  4.81it/s, est. speed input: 4889.93 toks/s, output: 723.54 toks/s]
Processed prompts:   1%|          | 272/32000 [00:41<1:54:10,  4.63it/s, est. speed input: 4865.65 toks/s, output: 719.28 toks/s]
Processed prompts:   1%|          | 274/32000 [00:42<1:36:12,  5.50it/s, est. speed input: 4883.55 toks/s, output: 722.30 toks/s]
Processed prompts:   1%|          | 277/32000 [00:42<1:18:50,  6.71it/s, est. speed input: 4916.19 toks/s, output: 723.85 toks/s]
Processed prompts:   1%|          | 278/32000 [00:43<1:58:53,  4.45it/s, est. speed input: 4849.20 toks/s, output: 715.37 toks/s]
Processed prompts:   1%|          | 279/32000 [00:43<2:20:08,  3.77it/s, est. speed input: 4805.53 toks/s, output: 711.13 toks/s]
Processed prompts:   1%|          | 281/32000 [00:43<1:53:04,  4.68it/s, est. speed input: 4812.31 toks/s, output: 714.04 toks/s]
Processed prompts:   1%|          | 285/32000 [00:44<1:38:41,  5.36it/s, est. speed input: 4945.72 toks/s, output: 718.03 toks/s]
Processed prompts:   1%|          | 288/32000 [00:44<1:31:00,  5.81it/s, est. speed input: 4944.65 toks/s, output: 718.28 toks/s]
Processed prompts:   1%|          | 290/32000 [00:45<2:06:25,  4.18it/s, est. speed input: 4867.54 toks/s, output: 708.05 toks/s]
Processed prompts:   1%|          | 291/32000 [00:45<2:01:31,  4.35it/s, est. speed input: 4860.09 toks/s, output: 707.51 toks/s]
Processed prompts:   1%|          | 292/32000 [00:46<2:43:13,  3.24it/s, est. speed input: 4825.10 toks/s, output: 701.68 toks/s]
Processed prompts:   1%|          | 295/32000 [00:46<1:40:53,  5.24it/s, est. speed input: 4929.86 toks/s, output: 706.26 toks/s]
Processed prompts:   1%|          | 297/32000 [00:47<2:49:29,  3.12it/s, est. speed input: 4831.72 toks/s, output: 692.50 toks/s]
Processed prompts:   1%|          | 299/32000 [00:48<2:13:31,  3.96it/s, est. speed input: 4850.92 toks/s, output: 694.13 toks/s]
Processed prompts:   1%|          | 300/32000 [00:48<3:03:34,  2.88it/s, est. speed input: 4778.16 toks/s, output: 685.98 toks/s]
Processed prompts:   1%|          | 302/32000 [00:49<2:23:23,  3.68it/s, est. speed input: 4774.47 toks/s, output: 687.10 toks/s]
Processed prompts:   1%|          | 303/32000 [00:49<2:32:11,  3.47it/s, est. speed input: 4747.72 toks/s, output: 683.64 toks/s]
Processed prompts:   1%|          | 305/32000 [00:49<2:22:37,  3.70it/s, est. speed input: 4739.29 toks/s, output: 681.80 toks/s]
Processed prompts:   1%|          | 309/32000 [00:50<1:26:26,  6.11it/s, est. speed input: 4774.02 toks/s, output: 688.75 toks/s]
Processed prompts:   1%|          | 310/32000 [00:50<2:09:41,  4.07it/s, est. speed input: 4715.50 toks/s, output: 683.69 toks/s]
Processed prompts:   1%|          | 314/32000 [00:51<1:23:55,  6.29it/s, est. speed input: 4786.26 toks/s, output: 690.26 toks/s]
Processed prompts:   1%|          | 319/32000 [00:51<1:10:02,  7.54it/s, est. speed input: 4809.70 toks/s, output: 694.90 toks/s]
Processed prompts:   1%|          | 320/32000 [00:52<1:40:35,  5.25it/s, est. speed input: 4767.67 toks/s, output: 688.48 toks/s]
Processed prompts:   1%|          | 321/32000 [00:52<1:48:08,  4.88it/s, est. speed input: 4756.34 toks/s, output: 686.86 toks/s]
Processed prompts:   1%|          | 324/32000 [00:53<2:02:47,  4.30it/s, est. speed input: 4776.99 toks/s, output: 681.54 toks/s]
Processed prompts:   1%|          | 326/32000 [00:53<2:03:35,  4.27it/s, est. speed input: 4763.73 toks/s, output: 680.15 toks/s]
Processed prompts:   1%|          | 329/32000 [00:54<1:48:29,  4.87it/s, est. speed input: 4806.32 toks/s, output: 682.61 toks/s]
Processed prompts:   1%|          | 331/32000 [00:55<2:15:47,  3.89it/s, est. speed input: 4765.84 toks/s, output: 676.64 toks/s]
Processed prompts:   1%|          | 332/32000 [00:55<3:00:14,  2.93it/s, est. speed input: 4706.32 toks/s, output: 669.11 toks/s]
Processed prompts:   1%|          | 336/32000 [00:56<1:56:51,  4.52it/s, est. speed input: 4722.44 toks/s, output: 673.00 toks/s]
Processed prompts:   1%|          | 338/32000 [00:56<2:00:17,  4.39it/s, est. speed input: 4693.41 toks/s, output: 670.39 toks/s]
Processed prompts:   1%|          | 339/32000 [00:57<1:59:23,  4.42it/s, est. speed input: 4679.49 toks/s, output: 669.70 toks/s]
Processed prompts:   1%|          | 340/32000 [00:57<2:43:09,  3.23it/s, est. speed input: 4648.84 toks/s, output: 665.82 toks/s]
Processed prompts:   1%|          | 343/32000 [00:57<1:47:55,  4.89it/s, est. speed input: 4667.63 toks/s, output: 668.86 toks/s]
Processed prompts:   1%|          | 345/32000 [00:58<2:32:44,  3.45it/s, est. speed input: 4619.53 toks/s, output: 663.31 toks/s]
Processed prompts:   1%|          | 350/32000 [00:59<1:54:28,  4.61it/s, est. speed input: 4675.61 toks/s, output: 666.06 toks/s]
Processed prompts:   1%|          | 351/32000 [01:00<2:10:35,  4.04it/s, est. speed input: 4669.87 toks/s, output: 663.54 toks/s]
Processed prompts:   1%|          | 352/32000 [01:00<2:05:13,  4.21it/s, est. speed input: 4675.44 toks/s, output: 665.79 toks/s]
Processed prompts:   1%|          | 355/32000 [01:00<1:59:47,  4.40it/s, est. speed input: 4646.99 toks/s, output: 663.52 toks/s]
Processed prompts:   1%|          | 358/32000 [01:01<1:44:26,  5.05it/s, est. speed input: 4671.76 toks/s, output: 663.87 toks/s]
Processed prompts:   1%|          | 359/32000 [01:01<2:13:13,  3.96it/s, est. speed input: 4644.12 toks/s, output: 660.06 toks/s]
Processed prompts:   1%|          | 362/32000 [01:02<1:35:32,  5.52it/s, est. speed input: 4654.87 toks/s, output: 662.95 toks/s]
Processed prompts:   1%|          | 365/32000 [01:02<1:29:11,  5.91it/s, est. speed input: 4671.52 toks/s, output: 664.18 toks/s]
Processed prompts:   1%|          | 366/32000 [01:02<1:45:23,  5.00it/s, est. speed input: 4658.78 toks/s, output: 664.22 toks/s]
Processed prompts:   1%|          | 369/32000 [01:03<1:26:36,  6.09it/s, est. speed input: 4672.45 toks/s, output: 666.94 toks/s]
Processed prompts:   1%|          | 371/32000 [01:03<1:34:57,  5.55it/s, est. speed input: 4670.75 toks/s, output: 666.56 toks/s]
Processed prompts:   1%|          | 373/32000 [01:03<1:23:07,  6.34it/s, est. speed input: 4676.09 toks/s, output: 668.89 toks/s]
Processed prompts:   1%|          | 375/32000 [01:05<2:30:28,  3.50it/s, est. speed input: 4610.82 toks/s, output: 660.27 toks/s]
Processed prompts:   1%|          | 377/32000 [01:05<2:07:39,  4.13it/s, est. speed input: 4614.74 toks/s, output: 661.54 toks/s]
Processed prompts:   1%|          | 378/32000 [01:05<2:22:24,  3.70it/s, est. speed input: 4602.92 toks/s, output: 659.37 toks/s]
Processed prompts:   1%|          | 380/32000 [01:06<2:06:29,  4.17it/s, est. speed input: 4628.80 toks/s, output: 661.57 toks/s]
Processed prompts:   1%|          | 383/32000 [01:06<1:48:53,  4.84it/s, est. speed input: 4623.90 toks/s, output: 664.32 toks/s]
Processed prompts:   1%|          | 386/32000 [01:07<1:47:33,  4.90it/s, est. speed input: 4619.24 toks/s, output: 663.51 toks/s]
Processed prompts:   1%|          | 387/32000 [01:07<2:08:52,  4.09it/s, est. speed input: 4601.37 toks/s, output: 662.65 toks/s]
Processed prompts:   1%|          | 389/32000 [01:07<1:47:46,  4.89it/s, est. speed input: 4607.15 toks/s, output: 664.01 toks/s]
Processed prompts:   1%|          | 393/32000 [01:08<1:35:01,  5.54it/s, est. speed input: 4644.38 toks/s, output: 664.33 toks/s]
Processed prompts:   1%|          | 396/32000 [01:09<1:35:41,  5.50it/s, est. speed input: 4624.51 toks/s, output: 663.85 toks/s]
Processed prompts:   1%|          | 398/32000 [01:10<2:17:31,  3.83it/s, est. speed input: 4566.81 toks/s, output: 657.09 toks/s]
Processed prompts:   1%|          | 399/32000 [01:10<2:18:41,  3.80it/s, est. speed input: 4558.87 toks/s, output: 656.14 toks/s]
Processed prompts:   1%|▏         | 401/32000 [01:10<1:56:33,  4.52it/s, est. speed input: 4561.02 toks/s, output: 657.15 toks/s]
Processed prompts:   1%|▏         | 404/32000 [01:11<1:37:23,  5.41it/s, est. speed input: 4563.30 toks/s, output: 657.29 toks/s]
Processed prompts:   1%|▏         | 405/32000 [01:11<2:11:44,  4.00it/s, est. speed input: 4532.96 toks/s, output: 654.32 toks/s]
Processed prompts:   1%|▏         | 408/32000 [01:12<2:35:31,  3.39it/s, est. speed input: 4512.25 toks/s, output: 651.08 toks/s]
Processed prompts:   1%|▏         | 412/32000 [01:13<2:30:39,  3.49it/s, est. speed input: 4476.94 toks/s, output: 647.53 toks/s]
Processed prompts:   1%|▏         | 416/32000 [01:14<2:20:13,  3.75it/s, est. speed input: 4447.06 toks/s, output: 644.64 toks/s]
Processed prompts:   1%|▏         | 417/32000 [01:15<2:24:26,  3.64it/s, est. speed input: 4440.32 toks/s, output: 643.49 toks/s]
Processed prompts:   1%|▏         | 421/32000 [01:15<1:31:19,  5.76it/s, est. speed input: 4470.23 toks/s, output: 648.87 toks/s]
Processed prompts:   1%|▏         | 424/32000 [01:16<1:47:00,  4.92it/s, est. speed input: 4480.12 toks/s, output: 648.69 toks/s]
Processed prompts:   1%|▏         | 427/32000 [01:17<2:21:19,  3.72it/s, est. speed input: 4442.52 toks/s, output: 643.65 toks/s]
Processed prompts:   1%|▏         | 430/32000 [01:17<2:01:44,  4.32it/s, est. speed input: 4434.37 toks/s, output: 645.07 toks/s]
Processed prompts:   1%|▏         | 434/32000 [01:17<1:27:24,  6.02it/s, est. speed input: 4469.32 toks/s, output: 648.38 toks/s]
Processed prompts:   1%|▏         | 436/32000 [01:18<2:02:02,  4.31it/s, est. speed input: 4432.89 toks/s, output: 643.75 toks/s]
Processed prompts:   1%|▏         | 438/32000 [01:19<1:59:59,  4.38it/s, est. speed input: 4429.09 toks/s, output: 643.69 toks/s]
Processed prompts:   1%|▏         | 440/32000 [01:20<2:56:03,  2.99it/s, est. speed input: 4381.75 toks/s, output: 635.79 toks/s]
Processed prompts:   1%|▏         | 442/32000 [01:21<3:14:04,  2.71it/s, est. speed input: 4362.78 toks/s, output: 631.84 toks/s]
Processed prompts:   1%|▏         | 444/32000 [01:21<2:28:42,  3.54it/s, est. speed input: 4367.98 toks/s, output: 633.26 toks/s]
Processed prompts:   1%|▏         | 445/32000 [01:22<3:09:01,  2.78it/s, est. speed input: 4330.88 toks/s, output: 628.35 toks/s]
Processed prompts:   1%|▏         | 446/32000 [01:22<2:57:22,  2.96it/s, est. speed input: 4351.17 toks/s, output: 629.17 toks/s]
Processed prompts:   1%|▏         | 448/32000 [01:23<2:26:23,  3.59it/s, est. speed input: 4349.26 toks/s, output: 630.28 toks/s]
Processed prompts:   1%|▏         | 451/32000 [01:23<1:47:49,  4.88it/s, est. speed input: 4366.88 toks/s, output: 632.43 toks/s]
Processed prompts:   1%|▏         | 455/32000 [01:24<1:57:28,  4.48it/s, est. speed input: 4345.32 toks/s, output: 631.12 toks/s]
Processed prompts:   1%|▏         | 456/32000 [01:24<2:15:15,  3.89it/s, est. speed input: 4328.91 toks/s, output: 628.86 toks/s]
Processed prompts:   1%|▏         | 458/32000 [01:25<2:04:53,  4.21it/s, est. speed input: 4349.23 toks/s, output: 627.85 toks/s]
Processed prompts:   1%|▏         | 459/32000 [01:25<2:05:04,  4.20it/s, est. speed input: 4350.43 toks/s, output: 627.35 toks/s]
Processed prompts:   1%|▏         | 460/32000 [01:26<2:45:04,  3.18it/s, est. speed input: 4320.43 toks/s, output: 623.76 toks/s]
Processed prompts:   1%|▏         | 461/32000 [01:26<2:36:27,  3.36it/s, est. speed input: 4318.93 toks/s, output: 623.92 toks/s]
Processed prompts:   1%|▏         | 465/32000 [01:27<2:12:47,  3.96it/s, est. speed input: 4317.34 toks/s, output: 622.95 toks/s]
Processed prompts:   1%|▏         | 467/32000 [01:27<1:53:04,  4.65it/s, est. speed input: 4320.77 toks/s, output: 624.13 toks/s]
Processed prompts:   1%|▏         | 469/32000 [01:28<2:08:31,  4.09it/s, est. speed input: 4298.38 toks/s, output: 623.09 toks/s]
Processed prompts:   1%|▏         | 471/32000 [01:28<1:56:21,  4.52it/s, est. speed input: 4292.28 toks/s, output: 624.33 toks/s]
Processed prompts:   1%|▏         | 473/32000 [01:28<1:46:18,  4.94it/s, est. speed input: 4282.44 toks/s, output: 625.14 toks/s]
Processed prompts:   1%|▏         | 477/32000 [01:29<1:30:29,  5.81it/s, est. speed input: 4281.23 toks/s, output: 627.10 toks/s]
Processed prompts:   2%|▏         | 481/32000 [01:30<1:44:07,  5.05it/s, est. speed input: 4273.63 toks/s, output: 626.69 toks/s]
Processed prompts:   2%|▏         | 482/32000 [01:30<1:45:14,  4.99it/s, est. speed input: 4274.94 toks/s, output: 627.94 toks/s]
Processed prompts:   2%|▏         | 485/32000 [01:30<1:30:32,  5.80it/s, est. speed input: 4281.94 toks/s, output: 629.70 toks/s]
Processed prompts:   2%|▏         | 486/32000 [01:31<2:20:10,  3.75it/s, est. speed input: 4247.51 toks/s, output: 625.77 toks/s]
Processed prompts:   2%|▏         | 489/32000 [01:32<2:10:37,  4.02it/s, est. speed input: 4235.67 toks/s, output: 624.62 toks/s]
Processed prompts:   2%|▏         | 493/32000 [01:32<1:31:01,  5.77it/s, est. speed input: 4261.49 toks/s, output: 626.48 toks/s]
Processed prompts:   2%|▏         | 494/32000 [01:33<2:26:51,  3.58it/s, est. speed input: 4251.87 toks/s, output: 620.28 toks/s]
Processed prompts:   2%|▏         | 495/32000 [01:33<2:43:41,  3.21it/s, est. speed input: 4238.03 toks/s, output: 619.72 toks/s]
Processed prompts:   2%|▏         | 498/32000 [01:34<1:46:16,  4.94it/s, est. speed input: 4263.68 toks/s, output: 621.98 toks/s]
Processed prompts:   2%|▏         | 500/32000 [01:34<2:14:10,  3.91it/s, est. speed input: 4243.92 toks/s, output: 620.00 toks/s]
Processed prompts:   2%|▏         | 501/32000 [01:34<2:01:36,  4.32it/s, est. speed input: 4245.83 toks/s, output: 621.14 toks/s]
Processed prompts:   2%|▏         | 504/32000 [01:35<2:23:13,  3.67it/s, est. speed input: 4215.99 toks/s, output: 618.36 toks/s]
Processed prompts:   2%|▏         | 508/32000 [01:36<1:34:06,  5.58it/s, est. speed input: 4230.34 toks/s, output: 622.36 toks/s]
Processed prompts:   2%|▏         | 511/32000 [01:36<1:35:13,  5.51it/s, est. speed input: 4236.12 toks/s, output: 622.45 toks/s]
Processed prompts:   2%|▏         | 513/32000 [01:37<2:16:49,  3.84it/s, est. speed input: 4207.53 toks/s, output: 618.00 toks/s]
Processed prompts:   2%|▏         | 515/32000 [01:38<2:27:52,  3.55it/s, est. speed input: 4196.88 toks/s, output: 616.71 toks/s]
Processed prompts:   2%|▏         | 519/32000 [01:38<1:38:16,  5.34it/s, est. speed input: 4216.47 toks/s, output: 621.43 toks/s]
Processed prompts:   2%|▏         | 522/32000 [01:39<2:03:39,  4.24it/s, est. speed input: 4227.31 toks/s, output: 618.90 toks/s]
Processed prompts:   2%|▏         | 524/32000 [01:40<2:04:16,  4.22it/s, est. speed input: 4223.43 toks/s, output: 618.38 toks/s]
Processed prompts:   2%|▏         | 526/32000 [01:40<1:46:12,  4.94it/s, est. speed input: 4223.07 toks/s, output: 619.59 toks/s]
Processed prompts:   2%|▏         | 530/32000 [01:41<1:47:44,  4.87it/s, est. speed input: 4216.00 toks/s, output: 618.69 toks/s]
Processed prompts:   2%|▏         | 533/32000 [01:41<1:44:31,  5.02it/s, est. speed input: 4211.88 toks/s, output: 619.15 toks/s]
Processed prompts:   2%|▏         | 535/32000 [01:42<1:46:55,  4.90it/s, est. speed input: 4210.31 toks/s, output: 620.33 toks/s]
Processed prompts:   2%|▏         | 537/32000 [01:42<1:55:53,  4.52it/s, est. speed input: 4195.18 toks/s, output: 618.76 toks/s]
Processed prompts:   2%|▏         | 538/32000 [01:43<2:04:03,  4.23it/s, est. speed input: 4186.26 toks/s, output: 617.72 toks/s]
Processed prompts:   2%|▏         | 539/32000 [01:43<2:12:01,  3.97it/s, est. speed input: 4178.88 toks/s, output: 617.00 toks/s]
Processed prompts:   2%|▏         | 541/32000 [01:44<2:45:32,  3.17it/s, est. speed input: 4160.83 toks/s, output: 615.41 toks/s]
Processed prompts:   2%|▏         | 542/32000 [01:44<2:38:00,  3.32it/s, est. speed input: 4158.81 toks/s, output: 616.44 toks/s]
Processed prompts:   2%|▏         | 545/32000 [01:44<1:46:37,  4.92it/s, est. speed input: 4162.19 toks/s, output: 618.89 toks/s]
Processed prompts:   2%|▏         | 552/32000 [01:46<2:13:25,  3.93it/s, est. speed input: 4164.87 toks/s, output: 614.36 toks/s]
Processed prompts:   2%|▏         | 553/32000 [01:47<2:12:42,  3.95it/s, est. speed input: 4188.39 toks/s, output: 615.36 toks/s]
Processed prompts:   2%|▏         | 555/32000 [01:47<1:56:19,  4.51it/s, est. speed input: 4187.08 toks/s, output: 616.71 toks/s]
Processed prompts:   2%|▏         | 557/32000 [01:48<2:25:29,  3.60it/s, est. speed input: 4181.88 toks/s, output: 614.79 toks/s]
Processed prompts:   2%|▏         | 559/32000 [01:48<2:11:34,  3.98it/s, est. speed input: 4175.91 toks/s, output: 615.03 toks/s]
Processed prompts:   2%|▏         | 562/32000 [01:49<1:50:32,  4.74it/s, est. speed input: 4183.96 toks/s, output: 617.17 toks/s]
Processed prompts:   2%|▏         | 563/32000 [01:50<3:04:47,  2.84it/s, est. speed input: 4150.99 toks/s, output: 611.35 toks/s]
Processed prompts:   2%|▏         | 565/32000 [01:50<3:01:11,  2.89it/s, est. speed input: 4155.80 toks/s, output: 609.37 toks/s]
Processed prompts:   2%|▏         | 567/32000 [01:51<3:10:04,  2.76it/s, est. speed input: 4142.48 toks/s, output: 607.79 toks/s]
Processed prompts:   2%|▏         | 569/32000 [01:51<2:28:39,  3.52it/s, est. speed input: 4146.79 toks/s, output: 609.00 toks/s]
Processed prompts:   2%|▏         | 570/32000 [01:52<2:38:00,  3.32it/s, est. speed input: 4139.33 toks/s, output: 608.44 toks/s]
Processed prompts:   2%|▏         | 571/32000 [01:52<3:22:33,  2.59it/s, est. speed input: 4115.37 toks/s, output: 605.43 toks/s]
Processed prompts:   2%|▏         | 573/32000 [01:53<2:25:51,  3.59it/s, est. speed input: 4120.56 toks/s, output: 606.74 toks/s]
Processed prompts:   2%|▏         | 575/32000 [01:53<2:14:50,  3.88it/s, est. speed input: 4139.72 toks/s, output: 606.79 toks/s]
Processed prompts:   2%|▏         | 578/32000 [01:54<2:05:06,  4.19it/s, est. speed input: 4125.82 toks/s, output: 605.68 toks/s]
Processed prompts:   2%|▏         | 579/32000 [01:54<2:08:45,  4.07it/s, est. speed input: 4123.93 toks/s, output: 605.45 toks/s]
Processed prompts:   2%|▏         | 582/32000 [01:54<1:43:02,  5.08it/s, est. speed input: 4127.58 toks/s, output: 606.52 toks/s]
Processed prompts:   2%|▏         | 583/32000 [01:55<2:09:23,  4.05it/s, est. speed input: 4124.50 toks/s, output: 604.62 toks/s]
Processed prompts:   2%|▏         | 585/32000 [01:55<1:50:44,  4.73it/s, est. speed input: 4125.16 toks/s, output: 604.89 toks/s]
Processed prompts:   2%|▏         | 588/32000 [01:56<1:48:56,  4.81it/s, est. speed input: 4127.50 toks/s, output: 605.88 toks/s]
Processed prompts:   2%|▏         | 593/32000 [01:56<1:20:04,  6.54it/s, est. speed input: 4151.95 toks/s, output: 608.12 toks/s]
Processed prompts:   2%|▏         | 594/32000 [01:57<2:03:16,  4.25it/s, est. speed input: 4124.93 toks/s, output: 604.94 toks/s]
Processed prompts:   2%|▏         | 595/32000 [01:57<2:09:55,  4.03it/s, est. speed input: 4117.46 toks/s, output: 605.47 toks/s]
Processed prompts:   2%|▏         | 598/32000 [01:58<1:35:13,  5.50it/s, est. speed input: 4125.70 toks/s, output: 608.21 toks/s]
Processed prompts:   2%|▏         | 599/32000 [01:59<2:49:33,  3.09it/s, est. speed input: 4091.54 toks/s, output: 603.64 toks/s]
Processed prompts:   2%|▏         | 602/32000 [01:59<2:01:20,  4.31it/s, est. speed input: 4097.76 toks/s, output: 605.03 toks/s]
Processed prompts:   2%|▏         | 603/32000 [02:00<2:28:06,  3.53it/s, est. speed input: 4082.94 toks/s, output: 603.79 toks/s]
Processed prompts:   2%|▏         | 604/32000 [02:00<2:29:56,  3.49it/s, est. speed input: 4076.62 toks/s, output: 603.55 toks/s]
Processed prompts:   2%|▏         | 605/32000 [02:00<2:25:49,  3.59it/s, est. speed input: 4074.60 toks/s, output: 603.51 toks/s]
Processed prompts:   2%|▏         | 608/32000 [02:01<2:49:58,  3.08it/s, est. speed input: 4054.30 toks/s, output: 600.89 toks/s]
Processed prompts:   2%|▏         | 612/32000 [02:01<1:35:49,  5.46it/s, est. speed input: 4095.44 toks/s, output: 604.13 toks/s]
Processed prompts:   2%|▏         | 614/32000 [02:02<2:18:31,  3.78it/s, est. speed input: 4072.04 toks/s, output: 600.85 toks/s]
Processed prompts:   2%|▏         | 616/32000 [02:03<2:36:52,  3.33it/s, est. speed input: 4056.70 toks/s, output: 600.05 toks/s]
Processed prompts:   2%|▏         | 618/32000 [02:04<2:20:42,  3.72it/s, est. speed input: 4055.27 toks/s, output: 599.54 toks/s]
Processed prompts:   2%|▏         | 619/32000 [02:04<2:23:52,  3.64it/s, est. speed input: 4053.08 toks/s, output: 598.97 toks/s]
Processed prompts:   2%|▏         | 620/32000 [02:04<2:34:49,  3.38it/s, est. speed input: 4045.93 toks/s, output: 597.94 toks/s]
Processed prompts:   2%|▏         | 621/32000 [02:04<2:24:27,  3.62it/s, est. speed input: 4042.83 toks/s, output: 598.00 toks/s]
Processed prompts:   2%|▏         | 622/32000 [02:05<2:29:30,  3.50it/s, est. speed input: 4040.95 toks/s, output: 597.73 toks/s]
Processed prompts:   2%|▏         | 626/32000 [02:05<1:24:40,  6.18it/s, est. speed input: 4047.98 toks/s, output: 600.10 toks/s]
Processed prompts:   2%|▏         | 627/32000 [02:06<2:03:12,  4.24it/s, est. speed input: 4031.45 toks/s, output: 597.96 toks/s]
Processed prompts:   2%|▏         | 628/32000 [02:06<2:10:15,  4.01it/s, est. speed input: 4029.55 toks/s, output: 597.56 toks/s]
Processed prompts:   2%|▏         | 630/32000 [02:06<1:33:07,  5.61it/s, est. speed input: 4041.28 toks/s, output: 599.00 toks/s]
Processed prompts:   2%|▏         | 632/32000 [02:07<2:09:03,  4.05it/s, est. speed input: 4024.67 toks/s, output: 597.37 toks/s]
Processed prompts:   2%|▏         | 633/32000 [02:07<1:54:14,  4.58it/s, est. speed input: 4032.61 toks/s, output: 598.90 toks/s]
Processed prompts:   2%|▏         | 636/32000 [02:07<1:43:39,  5.04it/s, est. speed input: 4031.07 toks/s, output: 599.48 toks/s]
Processed prompts:   2%|▏         | 637/32000 [02:08<2:31:55,  3.44it/s, est. speed input: 4010.71 toks/s, output: 596.74 toks/s]
Processed prompts:   2%|▏         | 638/32000 [02:08<2:31:35,  3.45it/s, est. speed input: 4009.63 toks/s, output: 596.19 toks/s]
Processed prompts:   2%|▏         | 640/32000 [02:09<1:59:30,  4.37it/s, est. speed input: 4015.06 toks/s, output: 596.24 toks/s]
Processed prompts:   2%|▏         | 642/32000 [02:10<2:37:15,  3.32it/s, est. speed input: 4016.91 toks/s, output: 594.60 toks/s]
Processed prompts:   2%|▏         | 647/32000 [02:10<1:58:44,  4.40it/s, est. speed input: 4022.39 toks/s, output: 596.27 toks/s]
Processed prompts:   2%|▏         | 650/32000 [02:11<1:35:23,  5.48it/s, est. speed input: 4039.45 toks/s, output: 597.61 toks/s]
Processed prompts:   2%|▏         | 651/32000 [02:11<1:53:01,  4.62it/s, est. speed input: 4029.90 toks/s, output: 596.36 toks/s]
Processed prompts:   2%|▏         | 652/32000 [02:11<1:59:40,  4.37it/s, est. speed input: 4022.78 toks/s, output: 596.35 toks/s]
Processed prompts:   2%|▏         | 654/32000 [02:12<1:47:55,  4.84it/s, est. speed input: 4019.18 toks/s, output: 596.44 toks/s]
Processed prompts:   2%|▏         | 655/32000 [02:12<2:22:58,  3.65it/s, est. speed input: 4009.27 toks/s, output: 594.58 toks/s]
Processed prompts:   2%|▏         | 656/32000 [02:13<2:23:22,  3.64it/s, est. speed input: 4007.89 toks/s, output: 594.38 toks/s]
Processed prompts:   2%|▏         | 659/32000 [02:14<2:38:18,  3.30it/s, est. speed input: 3995.17 toks/s, output: 592.66 toks/s]
Processed prompts:   2%|▏         | 662/32000 [02:14<2:24:41,  3.61it/s, est. speed input: 3999.58 toks/s, output: 591.59 toks/s]
Processed prompts:   2%|▏         | 664/32000 [02:15<2:04:28,  4.20it/s, est. speed input: 3998.94 toks/s, output: 591.92 toks/s]
Processed prompts:   2%|▏         | 666/32000 [02:15<2:23:12,  3.65it/s, est. speed input: 3985.25 toks/s, output: 590.85 toks/s]
Processed prompts:   2%|▏         | 668/32000 [02:16<2:49:13,  3.09it/s, est. speed input: 3980.73 toks/s, output: 588.41 toks/s]
Processed prompts:   2%|▏         | 670/32000 [02:16<2:12:48,  3.93it/s, est. speed input: 3995.21 toks/s, output: 589.31 toks/s]
Processed prompts:   2%|▏         | 671/32000 [02:17<2:31:29,  3.45it/s, est. speed input: 3992.08 toks/s, output: 588.75 toks/s]
Processed prompts:   2%|▏         | 673/32000 [02:17<2:03:51,  4.22it/s, est. speed input: 3992.99 toks/s, output: 589.42 toks/s]
Processed prompts:   2%|▏         | 677/32000 [02:17<1:22:01,  6.37it/s, est. speed input: 4010.70 toks/s, output: 592.00 toks/s]
Processed prompts:   2%|▏         | 679/32000 [02:18<1:59:07,  4.38it/s, est. speed input: 3998.49 toks/s, output: 589.85 toks/s]
Processed prompts:   2%|▏         | 683/32000 [02:19<1:27:27,  5.97it/s, est. speed input: 4010.33 toks/s, output: 590.94 toks/s]
Processed prompts:   2%|▏         | 685/32000 [02:20<2:13:16,  3.92it/s, est. speed input: 3996.08 toks/s, output: 587.36 toks/s]
Processed prompts:   2%|▏         | 687/32000 [02:21<2:40:39,  3.25it/s, est. speed input: 3985.80 toks/s, output: 585.69 toks/s]
Processed prompts:   2%|▏         | 692/32000 [02:21<1:34:36,  5.52it/s, est. speed input: 4013.42 toks/s, output: 590.59 toks/s]
Processed prompts:   2%|▏         | 694/32000 [02:22<2:23:11,  3.64it/s, est. speed input: 3992.18 toks/s, output: 587.82 toks/s]
Processed prompts:   2%|▏         | 695/32000 [02:22<2:24:00,  3.62it/s, est. speed input: 3988.75 toks/s, output: 587.53 toks/s]
Processed prompts:   2%|▏         | 699/32000 [02:23<2:28:19,  3.52it/s, est. speed input: 3974.93 toks/s, output: 585.55 toks/s]
Processed prompts:   2%|▏         | 700/32000 [02:24<2:20:59,  3.70it/s, est. speed input: 3971.69 toks/s, output: 585.67 toks/s]
Processed prompts:   2%|▏         | 702/32000 [02:24<2:20:55,  3.70it/s, est. speed input: 3964.52 toks/s, output: 585.24 toks/s]
Processed prompts:   2%|▏         | 705/32000 [02:25<1:51:40,  4.67it/s, est. speed input: 3970.55 toks/s, output: 586.22 toks/s]
Processed prompts:   2%|▏         | 707/32000 [02:25<1:56:11,  4.49it/s, est. speed input: 3979.07 toks/s, output: 586.33 toks/s]
Processed prompts:   2%|▏         | 710/32000 [02:25<1:43:22,  5.04it/s, est. speed input: 3985.91 toks/s, output: 587.44 toks/s]
Processed prompts:   2%|▏         | 713/32000 [02:26<1:35:38,  5.45it/s, est. speed input: 4002.47 toks/s, output: 587.47 toks/s]
Processed prompts:   2%|▏         | 717/32000 [02:26<1:20:40,  6.46it/s, est. speed input: 4021.74 toks/s, output: 588.53 toks/s]
Processed prompts:   2%|▏         | 719/32000 [02:27<1:38:30,  5.29it/s, est. speed input: 4014.54 toks/s, output: 587.14 toks/s]
Processed prompts:   2%|▏         | 721/32000 [02:27<1:36:53,  5.38it/s, est. speed input: 4020.39 toks/s, output: 587.94 toks/s]
Processed prompts:   2%|▏         | 724/32000 [02:28<2:08:27,  4.06it/s, est. speed input: 4001.60 toks/s, output: 586.38 toks/s]
Processed prompts:   2%|▏         | 728/32000 [02:29<1:30:04,  5.79it/s, est. speed input: 4025.00 toks/s, output: 588.52 toks/s]
Processed prompts:   2%|▏         | 731/32000 [02:30<1:49:01,  4.78it/s, est. speed input: 4022.12 toks/s, output: 587.51 toks/s]
Processed prompts:   2%|▏         | 732/32000 [02:30<1:54:02,  4.57it/s, est. speed input: 4016.05 toks/s, output: 586.96 toks/s]
Processed prompts:   2%|▏         | 733/32000 [02:30<2:14:24,  3.88it/s, est. speed input: 4006.61 toks/s, output: 585.88 toks/s]
Processed prompts:   2%|▏         | 735/32000 [02:31<2:07:16,  4.09it/s, est. speed input: 4004.81 toks/s, output: 585.56 toks/s]
Processed prompts:   2%|▏         | 736/32000 [02:31<2:11:08,  3.97it/s, est. speed input: 3999.68 toks/s, output: 586.11 toks/s]
Processed prompts:   2%|▏         | 740/32000 [02:32<1:37:25,  5.35it/s, est. speed input: 4024.36 toks/s, output: 587.23 toks/s]
Processed prompts:   2%|▏         | 741/32000 [02:32<2:22:12,  3.66it/s, est. speed input: 4006.61 toks/s, output: 585.43 toks/s]
Processed prompts:   2%|▏         | 743/32000 [02:33<2:47:12,  3.12it/s, est. speed input: 4000.01 toks/s, output: 583.58 toks/s]
Processed prompts:   2%|▏         | 744/32000 [02:33<2:39:02,  3.28it/s, est. speed input: 4001.83 toks/s, output: 583.67 toks/s]
Processed prompts:   2%|▏         | 747/32000 [02:34<2:13:58,  3.89it/s, est. speed input: 4002.17 toks/s, output: 584.14 toks/s]
Processed prompts:   2%|▏         | 748/32000 [02:34<2:19:32,  3.73it/s, est. speed input: 3997.56 toks/s, output: 584.11 toks/s]
Processed prompts:   2%|▏         | 750/32000 [02:35<1:54:08,  4.56it/s, est. speed input: 4003.88 toks/s, output: 585.33 toks/s]
Processed prompts:   2%|▏         | 752/32000 [02:35<1:52:58,  4.61it/s, est. speed input: 4015.47 toks/s, output: 585.16 toks/s]
Processed prompts:   2%|▏         | 753/32000 [02:36<2:32:33,  3.41it/s, est. speed input: 4005.98 toks/s, output: 584.01 toks/s]
Processed prompts:   2%|▏         | 755/32000 [02:36<2:39:32,  3.26it/s, est. speed input: 4005.14 toks/s, output: 583.20 toks/s]
Processed prompts:   2%|▏         | 756/32000 [02:36<2:30:28,  3.46it/s, est. speed input: 4010.95 toks/s, output: 584.02 toks/s]
Processed prompts:   2%|▏         | 763/32000 [02:37<1:02:29,  8.33it/s, est. speed input: 4043.96 toks/s, output: 588.12 toks/s]
Processed prompts:   2%|▏         | 767/32000 [02:38<1:33:47,  5.55it/s, est. speed input: 4051.40 toks/s, output: 586.88 toks/s]
Processed prompts:   2%|▏         | 769/32000 [02:39<2:08:38,  4.05it/s, est. speed input: 4037.25 toks/s, output: 584.62 toks/s]
Processed prompts:   2%|▏         | 770/32000 [02:39<2:19:27,  3.73it/s, est. speed input: 4032.03 toks/s, output: 584.10 toks/s]
Processed prompts:   2%|▏         | 771/32000 [02:40<2:13:52,  3.89it/s, est. speed input: 4029.91 toks/s, output: 584.37 toks/s]
Processed prompts:   2%|▏         | 774/32000 [02:40<2:08:20,  4.05it/s, est. speed input: 4035.44 toks/s, output: 584.14 toks/s]
Processed prompts:   2%|▏         | 776/32000 [02:40<1:52:56,  4.61it/s, est. speed input: 4040.12 toks/s, output: 584.61 toks/s]
Processed prompts:   2%|▏         | 779/32000 [02:41<1:40:52,  5.16it/s, est. speed input: 4039.87 toks/s, output: 585.62 toks/s]
Processed prompts:   2%|▏         | 781/32000 [02:41<1:45:49,  4.92it/s, est. speed input: 4040.37 toks/s, output: 585.98 toks/s]
Processed prompts:   2%|▏         | 785/32000 [02:42<1:38:33,  5.28it/s, est. speed input: 4054.15 toks/s, output: 586.53 toks/s]
Processed prompts:   2%|▏         | 787/32000 [02:43<1:53:32,  4.58it/s, est. speed input: 4060.42 toks/s, output: 585.30 toks/s]
Processed prompts:   2%|▏         | 789/32000 [02:43<1:44:52,  4.96it/s, est. speed input: 4057.66 toks/s, output: 585.28 toks/s]
Processed prompts:   2%|▏         | 791/32000 [02:43<1:46:44,  4.87it/s, est. speed input: 4055.81 toks/s, output: 584.95 toks/s]
Processed prompts:   2%|▏         | 792/32000 [02:44<2:17:24,  3.79it/s, est. speed input: 4049.03 toks/s, output: 584.45 toks/s]
Processed prompts:   2%|▏         | 796/32000 [02:45<2:13:01,  3.91it/s, est. speed input: 4043.17 toks/s, output: 583.28 toks/s]
Processed prompts:   2%|▏         | 797/32000 [02:45<2:11:57,  3.94it/s, est. speed input: 4039.30 toks/s, output: 583.15 toks/s]
Processed prompts:   2%|▎         | 800/32000 [02:46<2:06:41,  4.10it/s, est. speed input: 4032.21 toks/s, output: 582.66 toks/s]
Processed prompts:   3%|▎         | 802/32000 [02:46<1:47:57,  4.82it/s, est. speed input: 4031.52 toks/s, output: 582.86 toks/s]
Processed prompts:   3%|▎         | 803/32000 [02:46<1:48:06,  4.81it/s, est. speed input: 4029.31 toks/s, output: 583.31 toks/s]
Processed prompts:   3%|▎         | 806/32000 [02:47<1:50:13,  4.72it/s, est. speed input: 4027.51 toks/s, output: 583.10 toks/s]
Processed prompts:   3%|▎         | 807/32000 [02:47<1:50:41,  4.70it/s, est. speed input: 4025.37 toks/s, output: 583.07 toks/s]
Processed prompts:   3%|▎         | 808/32000 [02:48<1:57:59,  4.41it/s, est. speed input: 4021.13 toks/s, output: 582.71 toks/s]
Processed prompts:   3%|▎         | 809/32000 [02:48<2:10:31,  3.98it/s, est. speed input: 4014.52 toks/s, output: 582.31 toks/s]
Processed prompts:   3%|▎         | 810/32000 [02:48<2:03:38,  4.20it/s, est. speed input: 4021.25 toks/s, output: 582.67 toks/s]
Processed prompts:   3%|▎         | 812/32000 [02:48<1:44:03,  5.00it/s, est. speed input: 4025.46 toks/s, output: 583.28 toks/s]
Processed prompts:   3%|▎         | 814/32000 [02:49<2:43:23,  3.18it/s, est. speed input: 4013.19 toks/s, output: 581.26 toks/s]
Processed prompts:   3%|▎         | 819/32000 [02:49<1:17:16,  6.72it/s, est. speed input: 4030.46 toks/s, output: 583.88 toks/s]
Processed prompts:   3%|▎         | 821/32000 [02:50<1:42:16,  5.08it/s, est. speed input: 4019.53 toks/s, output: 582.68 toks/s]
Processed prompts:   3%|▎         | 823/32000 [02:50<1:33:23,  5.56it/s, est. speed input: 4028.63 toks/s, output: 583.08 toks/s]
Processed prompts:   3%|▎         | 827/32000 [02:51<1:08:52,  7.54it/s, est. speed input: 4039.39 toks/s, output: 584.37 toks/s]
Processed prompts:   3%|▎         | 829/32000 [02:52<2:09:54,  4.00it/s, est. speed input: 4023.20 toks/s, output: 582.43 toks/s]
Processed prompts:   3%|▎         | 832/32000 [02:52<1:38:24,  5.28it/s, est. speed input: 4030.15 toks/s, output: 583.05 toks/s]
Processed prompts:   3%|▎         | 834/32000 [02:53<2:27:12,  3.53it/s, est. speed input: 4008.72 toks/s, output: 580.06 toks/s]
Processed prompts:   3%|▎         | 835/32000 [02:53<2:13:37,  3.89it/s, est. speed input: 4011.85 toks/s, output: 581.18 toks/s]
Processed prompts:   3%|▎         | 837/32000 [02:54<2:36:46,  3.31it/s, est. speed input: 4001.92 toks/s, output: 580.13 toks/s]
Processed prompts:   3%|▎         | 839/32000 [02:55<2:37:04,  3.31it/s, est. speed input: 3997.24 toks/s, output: 579.52 toks/s]
Processed prompts:   3%|▎         | 842/32000 [02:55<1:43:10,  5.03it/s, est. speed input: 4007.59 toks/s, output: 580.91 toks/s]
Processed prompts:   3%|▎         | 844/32000 [02:55<1:47:50,  4.81it/s, est. speed input: 4003.46 toks/s, output: 581.47 toks/s]
Processed prompts:   3%|▎         | 845/32000 [02:56<2:11:58,  3.93it/s, est. speed input: 3998.07 toks/s, output: 580.49 toks/s]
Processed prompts:   3%|▎         | 849/32000 [02:57<2:28:26,  3.50it/s, est. speed input: 3992.80 toks/s, output: 579.35 toks/s]
Processed prompts:   3%|▎         | 850/32000 [02:58<2:46:40,  3.11it/s, est. speed input: 3986.94 toks/s, output: 578.21 toks/s]
Processed prompts:   3%|▎         | 851/32000 [02:58<3:01:55,  2.85it/s, est. speed input: 3978.94 toks/s, output: 577.22 toks/s]
Processed prompts:   3%|▎         | 852/32000 [02:59<2:55:53,  2.95it/s, est. speed input: 3981.03 toks/s, output: 577.11 toks/s]
Processed prompts:   3%|▎         | 857/32000 [02:59<1:27:42,  5.92it/s, est. speed input: 3987.76 toks/s, output: 579.01 toks/s]
Processed prompts:   3%|▎         | 858/32000 [03:00<2:19:35,  3.72it/s, est. speed input: 3973.74 toks/s, output: 576.86 toks/s]
Processed prompts:   3%|▎         | 861/32000 [03:00<1:45:49,  4.90it/s, est. speed input: 3973.19 toks/s, output: 577.53 toks/s]
Processed prompts:   3%|▎         | 862/32000 [03:00<2:03:47,  4.19it/s, est. speed input: 3977.06 toks/s, output: 576.94 toks/s]
Processed prompts:   3%|▎         | 865/32000 [03:01<1:46:51,  4.86it/s, est. speed input: 3976.64 toks/s, output: 577.52 toks/s]
Processed prompts:   3%|▎         | 867/32000 [03:02<2:12:18,  3.92it/s, est. speed input: 3972.17 toks/s, output: 576.66 toks/s]
Processed prompts:   3%|▎         | 873/32000 [03:03<1:53:54,  4.55it/s, est. speed input: 3974.04 toks/s, output: 577.94 toks/s]
Processed prompts:   3%|▎         | 875/32000 [03:03<1:47:06,  4.84it/s, est. speed input: 3973.15 toks/s, output: 578.05 toks/s]
Processed prompts:   3%|▎         | 877/32000 [03:04<1:50:23,  4.70it/s, est. speed input: 3970.41 toks/s, output: 577.93 toks/s]
Processed prompts:   3%|▎         | 878/32000 [03:04<2:35:45,  3.33it/s, est. speed input: 3956.52 toks/s, output: 575.88 toks/s]
Processed prompts:   3%|▎         | 880/32000 [03:05<2:07:31,  4.07it/s, est. speed input: 3959.04 toks/s, output: 576.35 toks/s]
Processed prompts:   3%|▎         | 882/32000 [03:05<2:01:51,  4.26it/s, est. speed input: 3962.30 toks/s, output: 576.20 toks/s]
Processed prompts:   3%|▎         | 884/32000 [03:05<1:49:56,  4.72it/s, est. speed input: 3965.60 toks/s, output: 576.24 toks/s]
Processed prompts:   3%|▎         | 886/32000 [03:06<1:39:24,  5.22it/s, est. speed input: 3965.35 toks/s, output: 576.72 toks/s]
Processed prompts:   3%|▎         | 888/32000 [03:06<1:32:23,  5.61it/s, est. speed input: 3964.79 toks/s, output: 576.85 toks/s]
Processed prompts:   3%|▎         | 890/32000 [03:06<1:26:05,  6.02it/s, est. speed input: 3963.61 toks/s, output: 576.95 toks/s]
Processed prompts:   3%|▎         | 891/32000 [03:07<2:25:01,  3.58it/s, est. speed input: 3951.09 toks/s, output: 574.93 toks/s]
Processed prompts:   3%|▎         | 892/32000 [03:07<2:25:22,  3.57it/s, est. speed input: 3947.29 toks/s, output: 575.42 toks/s]
Processed prompts:   3%|▎         | 894/32000 [03:07<1:53:00,  4.59it/s, est. speed input: 3949.84 toks/s, output: 575.67 toks/s]
Processed prompts:   3%|▎         | 895/32000 [03:08<2:09:22,  4.01it/s, est. speed input: 3946.12 toks/s, output: 575.24 toks/s]
Processed prompts:   3%|▎         | 898/32000 [03:08<1:40:24,  5.16it/s, est. speed input: 3954.84 toks/s, output: 575.98 toks/s]
Processed prompts:   3%|▎         | 900/32000 [03:09<1:34:47,  5.47it/s, est. speed input: 3959.27 toks/s, output: 576.65 toks/s]
Processed prompts:   3%|▎         | 904/32000 [03:09<1:25:33,  6.06it/s, est. speed input: 3972.95 toks/s, output: 577.57 toks/s]
Processed prompts:   3%|▎         | 906/32000 [03:10<1:33:54,  5.52it/s, est. speed input: 3969.23 toks/s, output: 577.85 toks/s]
Processed prompts:   3%|▎         | 911/32000 [03:11<1:41:34,  5.10it/s, est. speed input: 3962.05 toks/s, output: 577.63 toks/s]
Processed prompts:   3%|▎         | 914/32000 [03:11<1:29:51,  5.77it/s, est. speed input: 3991.28 toks/s, output: 578.29 toks/s]
Processed prompts:   3%|▎         | 915/32000 [03:12<1:53:48,  4.55it/s, est. speed input: 3983.45 toks/s, output: 577.27 toks/s]
Processed prompts:   3%|▎         | 916/32000 [03:12<2:05:33,  4.13it/s, est. speed input: 3979.11 toks/s, output: 576.81 toks/s]
Processed prompts:   3%|▎         | 918/32000 [03:13<2:53:58,  2.98it/s, est. speed input: 3964.79 toks/s, output: 574.92 toks/s]
Processed prompts:   3%|▎         | 919/32000 [03:13<2:32:59,  3.39it/s, est. speed input: 3965.39 toks/s, output: 575.44 toks/s]
Processed prompts:   3%|▎         | 921/32000 [03:14<2:12:55,  3.90it/s, est. speed input: 3961.01 toks/s, output: 575.32 toks/s]
Processed prompts:   3%|▎         | 923/32000 [03:14<2:24:01,  3.60it/s, est. speed input: 3954.89 toks/s, output: 574.38 toks/s]
Processed prompts:   3%|▎         | 924/32000 [03:14<2:19:19,  3.72it/s, est. speed input: 3951.44 toks/s, output: 574.11 toks/s]
Processed prompts:   3%|▎         | 925/32000 [03:15<2:29:20,  3.47it/s, est. speed input: 3949.28 toks/s, output: 574.12 toks/s]
Processed prompts:   3%|▎         | 926/32000 [03:15<2:53:03,  2.99it/s, est. speed input: 3943.33 toks/s, output: 573.14 toks/s]
Processed prompts:   3%|▎         | 928/32000 [03:15<2:07:15,  4.07it/s, est. speed input: 3948.94 toks/s, output: 574.23 toks/s]
Processed prompts:   3%|▎         | 933/32000 [03:17<2:06:55,  4.08it/s, est. speed input: 3947.31 toks/s, output: 573.73 toks/s]
Processed prompts:   3%|▎         | 935/32000 [03:17<1:50:10,  4.70it/s, est. speed input: 3947.34 toks/s, output: 574.36 toks/s]
Processed prompts:   3%|▎         | 937/32000 [03:17<1:49:07,  4.74it/s, est. speed input: 3950.58 toks/s, output: 574.36 toks/s]
Processed prompts:   3%|▎         | 939/32000 [03:18<2:31:32,  3.42it/s, est. speed input: 3936.97 toks/s, output: 572.67 toks/s]
Processed prompts:   3%|▎         | 941/32000 [03:19<2:03:53,  4.18it/s, est. speed input: 3938.78 toks/s, output: 573.99 toks/s]
Processed prompts:   3%|▎         | 944/32000 [03:19<1:44:10,  4.97it/s, est. speed input: 3950.17 toks/s, output: 574.89 toks/s]
Processed prompts:   3%|▎         | 948/32000 [03:20<1:32:20,  5.60it/s, est. speed input: 3957.74 toks/s, output: 575.14 toks/s]
Processed prompts:   3%|▎         | 949/32000 [03:20<1:35:45,  5.40it/s, est. speed input: 3958.36 toks/s, output: 575.06 toks/s]
Processed prompts:   3%|▎         | 952/32000 [03:20<1:26:20,  5.99it/s, est. speed input: 3960.97 toks/s, output: 575.83 toks/s]
Processed prompts:   3%|▎         | 953/32000 [03:21<2:15:31,  3.82it/s, est. speed input: 3957.11 toks/s, output: 574.33 toks/s]
Processed prompts:   3%|▎         | 955/32000 [03:21<1:56:46,  4.43it/s, est. speed input: 3955.23 toks/s, output: 574.93 toks/s]
Processed prompts:   3%|▎         | 957/32000 [03:22<2:26:05,  3.54it/s, est. speed input: 3950.92 toks/s, output: 573.74 toks/s]
Processed prompts:   3%|▎         | 958/32000 [03:23<2:34:12,  3.35it/s, est. speed input: 3946.22 toks/s, output: 573.25 toks/s]
Processed prompts:   3%|▎         | 960/32000 [03:23<2:02:42,  4.22it/s, est. speed input: 3947.91 toks/s, output: 573.97 toks/s]
Processed prompts:   3%|▎         | 962/32000 [03:23<2:16:10,  3.80it/s, est. speed input: 3941.27 toks/s, output: 573.20 toks/s]
Processed prompts:   3%|▎         | 964/32000 [03:24<1:54:51,  4.50it/s, est. speed input: 3941.23 toks/s, output: 573.61 toks/s]
Processed prompts:   3%|▎         | 965/32000 [03:24<2:00:28,  4.29it/s, est. speed input: 3936.82 toks/s, output: 573.50 toks/s]
Processed prompts:   3%|▎         | 969/32000 [03:24<1:23:29,  6.19it/s, est. speed input: 3940.56 toks/s, output: 574.08 toks/s]
Processed prompts:   3%|▎         | 970/32000 [03:25<1:47:47,  4.80it/s, est. speed input: 3934.01 toks/s, output: 573.36 toks/s]
Processed prompts:   3%|▎         | 971/32000 [03:25<1:48:30,  4.77it/s, est. speed input: 3935.07 toks/s, output: 573.60 toks/s]
Processed prompts:   3%|▎         | 975/32000 [03:26<1:53:20,  4.56it/s, est. speed input: 3945.29 toks/s, output: 573.36 toks/s]
Processed prompts:   3%|▎         | 976/32000 [03:26<1:49:52,  4.71it/s, est. speed input: 3943.25 toks/s, output: 574.11 toks/s]
Processed prompts:   3%|▎         | 977/32000 [03:26<2:01:14,  4.26it/s, est. speed input: 3943.33 toks/s, output: 574.42 toks/s]
Processed prompts:   3%|▎         | 981/32000 [03:28<2:12:15,  3.91it/s, est. speed input: 3937.51 toks/s, output: 573.90 toks/s]
Processed prompts:   3%|▎         | 987/32000 [03:28<1:12:01,  7.18it/s, est. speed input: 3955.62 toks/s, output: 576.18 toks/s]
Processed prompts:   3%|▎         | 990/32000 [03:29<1:45:08,  4.92it/s, est. speed input: 3942.43 toks/s, output: 574.42 toks/s]
Processed prompts:   3%|▎         | 993/32000 [03:30<2:00:29,  4.29it/s, est. speed input: 3933.73 toks/s, output: 573.67 toks/s]
Processed prompts:   3%|▎         | 996/32000 [03:30<1:36:43,  5.34it/s, est. speed input: 3938.42 toks/s, output: 574.79 toks/s]
Processed prompts:   3%|▎         | 997/32000 [03:31<2:03:31,  4.18it/s, est. speed input: 3929.09 toks/s, output: 574.26 toks/s]
Processed prompts:   3%|▎         | 998/32000 [03:31<2:11:20,  3.93it/s, est. speed input: 3924.22 toks/s, output: 574.30 toks/s]
Processed prompts:   3%|▎         | 1002/32000 [03:31<1:29:52,  5.75it/s, est. speed input: 3932.79 toks/s, output: 575.68 toks/s]
Processed prompts:   3%|▎         | 1003/32000 [03:32<1:50:07,  4.69it/s, est. speed input: 3928.07 toks/s, output: 575.21 toks/s]
Processed prompts:   3%|▎         | 1004/32000 [03:32<1:51:32,  4.63it/s, est. speed input: 3939.50 toks/s, output: 575.38 toks/s]
Processed prompts:   3%|▎         | 1006/32000 [03:32<1:40:19,  5.15it/s, est. speed input: 3939.96 toks/s, output: 575.74 toks/s]
Processed prompts:   3%|▎         | 1007/32000 [03:33<2:20:19,  3.68it/s, est. speed input: 3930.54 toks/s, output: 574.57 toks/s]
Processed prompts:   3%|▎         | 1009/32000 [03:33<2:01:14,  4.26it/s, est. speed input: 3928.04 toks/s, output: 574.83 toks/s]
Processed prompts:   3%|▎         | 1011/32000 [03:33<1:39:46,  5.18it/s, est. speed input: 3932.71 toks/s, output: 575.49 toks/s]
Processed prompts:   3%|▎         | 1012/32000 [03:34<2:28:20,  3.48it/s, est. speed input: 3922.79 toks/s, output: 574.36 toks/s]
Processed prompts:   3%|▎         | 1014/32000 [03:35<2:33:19,  3.37it/s, est. speed input: 3915.07 toks/s, output: 573.88 toks/s]
Processed prompts:   3%|▎         | 1016/32000 [03:35<2:01:54,  4.24it/s, est. speed input: 3917.38 toks/s, output: 574.48 toks/s]
Processed prompts:   3%|▎         | 1018/32000 [03:35<2:01:15,  4.26it/s, est. speed input: 3917.23 toks/s, output: 574.08 toks/s]
Processed prompts:   3%|▎         | 1019/32000 [03:36<2:07:39,  4.04it/s, est. speed input: 3915.35 toks/s, output: 573.81 toks/s]
Processed prompts:   3%|▎         | 1021/32000 [03:36<1:53:17,  4.56it/s, est. speed input: 3917.66 toks/s, output: 574.03 toks/s]
Processed prompts:   3%|▎         | 1023/32000 [03:36<1:34:08,  5.48it/s, est. speed input: 3918.12 toks/s, output: 574.50 toks/s]
Processed prompts:   3%|▎         | 1024/32000 [03:37<1:44:09,  4.96it/s, est. speed input: 3915.44 toks/s, output: 574.23 toks/s]
Processed prompts:   3%|▎         | 1025/32000 [03:37<2:14:25,  3.84it/s, est. speed input: 3908.74 toks/s, output: 573.63 toks/s]
Processed prompts:   3%|▎         | 1027/32000 [03:38<2:51:14,  3.01it/s, est. speed input: 3899.16 toks/s, output: 572.35 toks/s]
Processed prompts:   3%|▎         | 1028/32000 [03:39<3:54:32,  2.20it/s, est. speed input: 3885.26 toks/s, output: 570.35 toks/s]
Processed prompts:   3%|▎         | 1029/32000 [03:39<3:12:27,  2.68it/s, est. speed input: 3891.50 toks/s, output: 570.75 toks/s]
Processed prompts:   3%|▎         | 1031/32000 [03:39<2:38:32,  3.26it/s, est. speed input: 3889.76 toks/s, output: 571.00 toks/s]
Processed prompts:   3%|▎         | 1034/32000 [03:40<1:40:41,  5.13it/s, est. speed input: 3901.42 toks/s, output: 572.22 toks/s]
Processed prompts:   3%|▎         | 1036/32000 [03:40<1:55:06,  4.48it/s, est. speed input: 3898.35 toks/s, output: 571.60 toks/s]
Processed prompts:   3%|▎         | 1037/32000 [03:40<1:56:35,  4.43it/s, est. speed input: 3899.22 toks/s, output: 571.83 toks/s]
Processed prompts:   3%|▎         | 1038/32000 [03:41<3:31:52,  2.44it/s, est. speed input: 3880.35 toks/s, output: 569.33 toks/s]
Processed prompts:   3%|▎         | 1040/32000 [03:42<2:23:30,  3.60it/s, est. speed input: 3883.97 toks/s, output: 570.28 toks/s]
Processed prompts:   3%|▎         | 1043/32000 [03:42<1:50:36,  4.66it/s, est. speed input: 3889.98 toks/s, output: 571.17 toks/s]
Processed prompts:   3%|▎         | 1045/32000 [03:42<1:54:53,  4.49it/s, est. speed input: 3890.94 toks/s, output: 571.10 toks/s]
Processed prompts:   3%|▎         | 1046/32000 [03:43<2:13:03,  3.88it/s, est. speed input: 3884.57 toks/s, output: 570.84 toks/s]
Processed prompts:   3%|▎         | 1049/32000 [03:43<1:30:14,  5.72it/s, est. speed input: 3892.36 toks/s, output: 571.76 toks/s]
Processed prompts:   3%|▎         | 1051/32000 [03:44<1:46:27,  4.85it/s, est. speed input: 3887.48 toks/s, output: 571.37 toks/s]
Processed prompts:   3%|▎         | 1052/32000 [03:44<2:42:33,  3.17it/s, est. speed input: 3879.12 toks/s, output: 570.37 toks/s]
Processed prompts:   3%|▎         | 1055/32000 [03:45<1:55:42,  4.46it/s, est. speed input: 3881.12 toks/s, output: 571.04 toks/s]
Processed prompts:   3%|▎         | 1058/32000 [03:45<1:24:45,  6.08it/s, est. speed input: 3886.58 toks/s, output: 571.90 toks/s]
Processed prompts:   3%|▎         | 1062/32000 [03:46<1:19:05,  6.52it/s, est. speed input: 3888.65 toks/s, output: 572.74 toks/s]
Processed prompts:   3%|▎         | 1066/32000 [03:46<1:19:48,  6.46it/s, est. speed input: 3883.69 toks/s, output: 573.47 toks/s]
Processed prompts:   3%|▎         | 1070/32000 [03:47<1:13:35,  7.01it/s, est. speed input: 3890.53 toks/s, output: 574.45 toks/s]
Processed prompts:   3%|▎         | 1071/32000 [03:48<2:09:47,  3.97it/s, est. speed input: 3877.12 toks/s, output: 572.73 toks/s]
Processed prompts:   3%|▎         | 1072/32000 [03:48<2:12:30,  3.89it/s, est. speed input: 3875.51 toks/s, output: 572.75 toks/s]
Processed prompts:   3%|▎         | 1074/32000 [03:49<2:25:53,  3.53it/s, est. speed input: 3875.25 toks/s, output: 572.16 toks/s]
Processed prompts:   3%|▎         | 1077/32000 [03:50<2:25:17,  3.55it/s, est. speed input: 3871.62 toks/s, output: 571.89 toks/s]
Processed prompts:   3%|▎         | 1082/32000 [03:51<2:18:41,  3.72it/s, est. speed input: 3869.27 toks/s, output: 571.20 toks/s]
Processed prompts:   3%|▎         | 1085/32000 [03:51<1:53:08,  4.55it/s, est. speed input: 3871.29 toks/s, output: 571.84 toks/s]
Processed prompts:   3%|▎         | 1086/32000 [03:52<2:14:17,  3.84it/s, est. speed input: 3869.83 toks/s, output: 571.06 toks/s]
Processed prompts:   3%|▎         | 1089/32000 [03:52<1:49:48,  4.69it/s, est. speed input: 3881.10 toks/s, output: 571.70 toks/s]
Processed prompts:   3%|▎         | 1092/32000 [03:53<1:46:15,  4.85it/s, est. speed input: 3882.19 toks/s, output: 571.83 toks/s]
Processed prompts:   3%|▎         | 1095/32000 [03:54<2:11:23,  3.92it/s, est. speed input: 3875.08 toks/s, output: 570.92 toks/s]
Processed prompts:   3%|▎         | 1097/32000 [03:55<2:47:05,  3.08it/s, est. speed input: 3873.04 toks/s, output: 569.94 toks/s]
Processed prompts:   3%|▎         | 1102/32000 [03:55<1:42:12,  5.04it/s, est. speed input: 3884.88 toks/s, output: 572.27 toks/s]
Processed prompts:   3%|▎         | 1105/32000 [03:56<2:11:50,  3.91it/s, est. speed input: 3884.81 toks/s, output: 570.63 toks/s]
Processed prompts:   3%|▎         | 1107/32000 [03:57<2:13:25,  3.86it/s, est. speed input: 3883.82 toks/s, output: 570.01 toks/s]
Processed prompts:   3%|▎         | 1108/32000 [03:57<2:16:06,  3.78it/s, est. speed input: 3880.26 toks/s, output: 569.97 toks/s]
Processed prompts:   3%|▎         | 1110/32000 [03:57<1:58:58,  4.33it/s, est. speed input: 3882.50 toks/s, output: 570.45 toks/s]
Processed prompts:   3%|▎         | 1113/32000 [03:58<1:48:34,  4.74it/s, est. speed input: 3883.62 toks/s, output: 570.52 toks/s]
Processed prompts:   3%|▎         | 1116/32000 [03:58<1:23:37,  6.15it/s, est. speed input: 3887.85 toks/s, output: 571.47 toks/s]
Processed prompts:   3%|▎         | 1118/32000 [03:59<2:01:08,  4.25it/s, est. speed input: 3881.51 toks/s, output: 570.47 toks/s]
Processed prompts:   4%|▎         | 1120/32000 [03:59<1:42:40,  5.01it/s, est. speed input: 3885.69 toks/s, output: 571.29 toks/s]
Processed prompts:   4%|▎         | 1124/32000 [04:00<1:24:36,  6.08it/s, est. speed input: 3892.10 toks/s, output: 572.08 toks/s]
Processed prompts:   4%|▎         | 1126/32000 [04:01<2:26:15,  3.52it/s, est. speed input: 3873.20 toks/s, output: 570.15 toks/s]
Processed prompts:   4%|▎         | 1127/32000 [04:01<2:21:28,  3.64it/s, est. speed input: 3873.34 toks/s, output: 570.24 toks/s]
Processed prompts:   4%|▎         | 1128/32000 [04:02<3:16:51,  2.61it/s, est. speed input: 3860.50 toks/s, output: 569.16 toks/s]
Processed prompts:   4%|▎         | 1131/32000 [04:03<2:18:29,  3.71it/s, est. speed input: 3862.08 toks/s, output: 569.87 toks/s]
Processed prompts:   4%|▎         | 1134/32000 [04:03<1:43:57,  4.95it/s, est. speed input: 3865.26 toks/s, output: 570.70 toks/s]
Processed prompts:   4%|▎         | 1135/32000 [04:03<2:04:01,  4.15it/s, est. speed input: 3861.93 toks/s, output: 570.16 toks/s]
Processed prompts:   4%|▎         | 1137/32000 [04:04<1:51:15,  4.62it/s, est. speed input: 3862.30 toks/s, output: 570.49 toks/s]
Processed prompts:   4%|▎         | 1138/32000 [04:04<2:09:39,  3.97it/s, est. speed input: 3857.25 toks/s, output: 569.83 toks/s]
Processed prompts:   4%|▎         | 1139/32000 [04:04<2:15:01,  3.81it/s, est. speed input: 3856.60 toks/s, output: 569.87 toks/s]
Processed prompts:   4%|▎         | 1143/32000 [04:04<1:09:41,  7.38it/s, est. speed input: 3863.17 toks/s, output: 571.74 toks/s]
Processed prompts:   4%|▎         | 1145/32000 [04:05<1:38:28,  5.22it/s, est. speed input: 3861.57 toks/s, output: 572.11 toks/s]
Processed prompts:   4%|▎         | 1149/32000 [04:06<1:22:17,  6.25it/s, est. speed input: 3867.01 toks/s, output: 573.40 toks/s]
Processed prompts:   4%|▎         | 1151/32000 [04:07<2:16:10,  3.78it/s, est. speed input: 3854.12 toks/s, output: 571.78 toks/s]
Processed prompts:   4%|▎         | 1153/32000 [04:07<2:05:17,  4.10it/s, est. speed input: 3858.83 toks/s, output: 571.86 toks/s]
Processed prompts:   4%|▎         | 1155/32000 [04:07<1:42:39,  5.01it/s, est. speed input: 3861.71 toks/s, output: 572.59 toks/s]
Processed prompts:   4%|▎         | 1157/32000 [04:08<1:49:07,  4.71it/s, est. speed input: 3859.09 toks/s, output: 572.34 toks/s]
Processed prompts:   4%|▎         | 1158/32000 [04:09<2:56:22,  2.91it/s, est. speed input: 3848.31 toks/s, output: 570.88 toks/s]
Processed prompts:   4%|▎         | 1164/32000 [04:09<1:18:19,  6.56it/s, est. speed input: 3865.30 toks/s, output: 573.73 toks/s]
Processed prompts:   4%|▎         | 1167/32000 [04:10<1:30:36,  5.67it/s, est. speed input: 3863.07 toks/s, output: 573.44 toks/s]
Processed prompts:   4%|▎         | 1169/32000 [04:10<1:27:09,  5.90it/s, est. speed input: 3866.61 toks/s, output: 573.90 toks/s]
Processed prompts:   4%|▎         | 1171/32000 [04:11<2:13:34,  3.85it/s, est. speed input: 3855.50 toks/s, output: 572.60 toks/s]
Processed prompts:   4%|▎         | 1174/32000 [04:11<1:45:15,  4.88it/s, est. speed input: 3858.41 toks/s, output: 573.33 toks/s]
Processed prompts:   4%|▎         | 1177/32000 [04:12<1:40:58,  5.09it/s, est. speed input: 3868.44 toks/s, output: 573.61 toks/s]
Processed prompts:   4%|▎         | 1178/32000 [04:12<2:04:47,  4.12it/s, est. speed input: 3869.81 toks/s, output: 573.39 toks/s]
Processed prompts:   4%|▎         | 1181/32000 [04:13<1:30:07,  5.70it/s, est. speed input: 3874.99 toks/s, output: 574.61 toks/s]
Processed prompts:   4%|▎         | 1185/32000 [04:13<1:26:15,  5.95it/s, est. speed input: 3897.57 toks/s, output: 574.97 toks/s]
Processed prompts:   4%|▎         | 1186/32000 [04:14<2:26:29,  3.51it/s, est. speed input: 3883.09 toks/s, output: 573.06 toks/s]
Processed prompts:   4%|▎         | 1187/32000 [04:15<2:31:30,  3.39it/s, est. speed input: 3878.48 toks/s, output: 572.57 toks/s]
Processed prompts:   4%|▎         | 1189/32000 [04:15<2:20:16,  3.66it/s, est. speed input: 3873.90 toks/s, output: 572.36 toks/s]
Processed prompts:   4%|▎         | 1192/32000 [04:15<1:44:25,  4.92it/s, est. speed input: 3878.14 toks/s, output: 572.86 toks/s]
Processed prompts:   4%|▎         | 1193/32000 [04:16<2:05:07,  4.10it/s, est. speed input: 3875.33 toks/s, output: 572.34 toks/s]
Processed prompts:   4%|▎         | 1195/32000 [04:16<1:44:54,  4.89it/s, est. speed input: 3876.32 toks/s, output: 572.59 toks/s]
Processed prompts:   4%|▎         | 1196/32000 [04:16<1:59:31,  4.30it/s, est. speed input: 3873.19 toks/s, output: 572.76 toks/s]
Processed prompts:   4%|▍         | 1200/32000 [04:17<1:45:09,  4.88it/s, est. speed input: 3872.14 toks/s, output: 573.95 toks/s]
Processed prompts:   4%|▍         | 1202/32000 [04:17<1:36:00,  5.35it/s, est. speed input: 3872.54 toks/s, output: 574.58 toks/s]
Processed prompts:   4%|▍         | 1205/32000 [04:18<1:34:21,  5.44it/s, est. speed input: 3870.85 toks/s, output: 574.92 toks/s]
Processed prompts:   4%|▍         | 1208/32000 [04:18<1:19:52,  6.43it/s, est. speed input: 3874.86 toks/s, output: 575.27 toks/s]
Processed prompts:   4%|▍         | 1209/32000 [04:19<1:53:30,  4.52it/s, est. speed input: 3867.37 toks/s, output: 574.78 toks/s]
Processed prompts:   4%|▍         | 1211/32000 [04:19<1:48:08,  4.75it/s, est. speed input: 3867.47 toks/s, output: 575.05 toks/s]
Processed prompts:   4%|▍         | 1216/32000 [04:20<1:08:29,  7.49it/s, est. speed input: 3881.58 toks/s, output: 577.40 toks/s]
Processed prompts:   4%|▍         | 1218/32000 [04:21<1:53:56,  4.50it/s, est. speed input: 3869.44 toks/s, output: 576.36 toks/s]
Processed prompts:   4%|▍         | 1219/32000 [04:21<2:14:45,  3.81it/s, est. speed input: 3862.56 toks/s, output: 575.75 toks/s]
Processed prompts:   4%|▍         | 1221/32000 [04:22<2:07:14,  4.03it/s, est. speed input: 3860.09 toks/s, output: 575.78 toks/s]
Processed prompts:   4%|▍         | 1224/32000 [04:22<2:09:54,  3.95it/s, est. speed input: 3863.59 toks/s, output: 575.29 toks/s]
Processed prompts:   4%|▍         | 1225/32000 [04:22<2:05:18,  4.09it/s, est. speed input: 3865.46 toks/s, output: 575.25 toks/s]
Processed prompts:   4%|▍         | 1226/32000 [04:23<2:28:54,  3.44it/s, est. speed input: 3861.77 toks/s, output: 574.55 toks/s]
Processed prompts:   4%|▍         | 1228/32000 [04:23<2:03:47,  4.14it/s, est. speed input: 3862.57 toks/s, output: 574.79 toks/s]
Processed prompts:   4%|▍         | 1229/32000 [04:24<2:20:24,  3.65it/s, est. speed input: 3858.15 toks/s, output: 574.24 toks/s]
Processed prompts:   4%|▍         | 1231/32000 [04:25<2:57:30,  2.89it/s, est. speed input: 3856.88 toks/s, output: 572.98 toks/s]
Processed prompts:   4%|▍         | 1238/32000 [04:26<1:56:17,  4.41it/s, est. speed input: 3855.80 toks/s, output: 573.19 toks/s]
Processed prompts:   4%|▍         | 1241/32000 [04:26<1:39:39,  5.14it/s, est. speed input: 3867.49 toks/s, output: 573.63 toks/s]
Processed prompts:   4%|▍         | 1243/32000 [04:27<1:45:37,  4.85it/s, est. speed input: 3865.22 toks/s, output: 573.22 toks/s]
Processed prompts:   4%|▍         | 1245/32000 [04:27<1:43:02,  4.97it/s, est. speed input: 3866.08 toks/s, output: 573.37 toks/s]
Processed prompts:   4%|▍         | 1248/32000 [04:27<1:28:08,  5.81it/s, est. speed input: 3870.39 toks/s, output: 574.17 toks/s]
Processed prompts:   4%|▍         | 1253/32000 [04:28<1:21:32,  6.28it/s, est. speed input: 3875.37 toks/s, output: 574.73 toks/s]
Processed prompts:   4%|▍         | 1257/32000 [04:29<1:48:41,  4.71it/s, est. speed input: 3873.39 toks/s, output: 573.86 toks/s]
Processed prompts:   4%|▍         | 1259/32000 [04:30<1:39:17,  5.16it/s, est. speed input: 3874.86 toks/s, output: 574.14 toks/s]
Processed prompts:   4%|▍         | 1260/32000 [04:30<1:46:46,  4.80it/s, est. speed input: 3873.41 toks/s, output: 573.83 toks/s]
Processed prompts:   4%|▍         | 1262/32000 [04:30<1:33:48,  5.46it/s, est. speed input: 3874.07 toks/s, output: 574.35 toks/s]
Processed prompts:   4%|▍         | 1264/32000 [04:31<2:41:37,  3.17it/s, est. speed input: 3869.24 toks/s, output: 573.04 toks/s]
Processed prompts:   4%|▍         | 1268/32000 [04:33<2:47:06,  3.07it/s, est. speed input: 3866.35 toks/s, output: 571.99 toks/s]
Processed prompts:   4%|▍         | 1270/32000 [04:34<3:01:30,  2.82it/s, est. speed input: 3872.58 toks/s, output: 570.77 toks/s]
Processed prompts:   4%|▍         | 1272/32000 [04:34<2:48:49,  3.03it/s, est. speed input: 3868.08 toks/s, output: 570.43 toks/s]
Processed prompts:   4%|▍         | 1274/32000 [04:34<2:24:02,  3.56it/s, est. speed input: 3870.22 toks/s, output: 570.58 toks/s]
Processed prompts:   4%|▍         | 1277/32000 [04:35<1:49:47,  4.66it/s, est. speed input: 3871.41 toks/s, output: 571.00 toks/s]
Processed prompts:   4%|▍         | 1278/32000 [04:35<2:29:37,  3.42it/s, est. speed input: 3863.69 toks/s, output: 569.91 toks/s]
Processed prompts:   4%|▍         | 1279/32000 [04:36<2:22:58,  3.58it/s, est. speed input: 3864.44 toks/s, output: 569.89 toks/s]
Processed prompts:   4%|▍         | 1281/32000 [04:36<2:00:39,  4.24it/s, est. speed input: 3866.55 toks/s, output: 570.40 toks/s]
Processed prompts:   4%|▍         | 1286/32000 [04:36<1:12:49,  7.03it/s, est. speed input: 3877.81 toks/s, output: 571.78 toks/s]
Processed prompts:   4%|▍         | 1288/32000 [04:37<1:37:52,  5.23it/s, est. speed input: 3870.06 toks/s, output: 571.24 toks/s]
Processed prompts:   4%|▍         | 1289/32000 [04:37<1:43:37,  4.94it/s, est. speed input: 3869.04 toks/s, output: 571.22 toks/s]
Processed prompts:   4%|▍         | 1290/32000 [04:38<2:02:13,  4.19it/s, est. speed input: 3868.54 toks/s, output: 571.06 toks/s]
Processed prompts:   4%|▍         | 1293/32000 [04:38<1:24:36,  6.05it/s, est. speed input: 3869.81 toks/s, output: 571.60 toks/s]
Processed prompts:   4%|▍         | 1294/32000 [04:38<1:45:17,  4.86it/s, est. speed input: 3872.61 toks/s, output: 571.39 toks/s]
Processed prompts:   4%|▍         | 1297/32000 [04:39<1:35:06,  5.38it/s, est. speed input: 3875.47 toks/s, output: 572.03 toks/s]
Processed prompts:   4%|▍         | 1298/32000 [04:39<1:55:02,  4.45it/s, est. speed input: 3872.84 toks/s, output: 571.65 toks/s]
Processed prompts:   4%|▍         | 1300/32000 [04:39<1:32:10,  5.55it/s, est. speed input: 3873.34 toks/s, output: 571.91 toks/s]
Processed prompts:   4%|▍         | 1301/32000 [04:40<1:53:36,  4.50it/s, est. speed input: 3870.31 toks/s, output: 571.53 toks/s]
Processed prompts:   4%|▍         | 1302/32000 [04:40<1:53:08,  4.52it/s, est. speed input: 3868.66 toks/s, output: 571.46 toks/s]
Processed prompts:   4%|▍         | 1306/32000 [04:40<58:36,  8.73it/s, est. speed input: 3875.04 toks/s, output: 572.47 toks/s]  
Processed prompts:   4%|▍         | 1308/32000 [04:41<1:41:38,  5.03it/s, est. speed input: 3866.73 toks/s, output: 571.37 toks/s]
Processed prompts:   4%|▍         | 1310/32000 [04:41<1:37:17,  5.26it/s, est. speed input: 3866.03 toks/s, output: 571.44 toks/s]
Processed prompts:   4%|▍         | 1311/32000 [04:42<1:48:02,  4.73it/s, est. speed input: 3865.40 toks/s, output: 571.71 toks/s]
Processed prompts:   4%|▍         | 1313/32000 [04:42<1:29:25,  5.72it/s, est. speed input: 3868.16 toks/s, output: 572.58 toks/s]
Processed prompts:   4%|▍         | 1315/32000 [04:42<1:34:57,  5.39it/s, est. speed input: 3865.52 toks/s, output: 572.61 toks/s]
Processed prompts:   4%|▍         | 1318/32000 [04:42<1:16:16,  6.70it/s, est. speed input: 3868.07 toks/s, output: 573.14 toks/s]
Processed prompts:   4%|▍         | 1319/32000 [04:43<2:03:15,  4.15it/s, est. speed input: 3861.10 toks/s, output: 572.30 toks/s]
Processed prompts:   4%|▍         | 1321/32000 [04:44<2:39:22,  3.21it/s, est. speed input: 3852.41 toks/s, output: 571.51 toks/s]
Processed prompts:   4%|▍         | 1322/32000 [04:44<2:42:16,  3.15it/s, est. speed input: 3849.62 toks/s, output: 571.25 toks/s]
Processed prompts:   4%|▍         | 1324/32000 [04:45<2:11:16,  3.89it/s, est. speed input: 3855.28 toks/s, output: 571.53 toks/s]
Processed prompts:   4%|▍         | 1325/32000 [04:45<2:09:06,  3.96it/s, est. speed input: 3855.37 toks/s, output: 571.54 toks/s]
Processed prompts:   4%|▍         | 1326/32000 [04:46<2:44:55,  3.10it/s, est. speed input: 3849.21 toks/s, output: 570.58 toks/s]
Processed prompts:   4%|▍         | 1327/32000 [04:46<2:37:30,  3.25it/s, est. speed input: 3849.06 toks/s, output: 570.45 toks/s]
Processed prompts:   4%|▍         | 1328/32000 [04:46<2:23:18,  3.57it/s, est. speed input: 3848.65 toks/s, output: 570.38 toks/s]
Processed prompts:   4%|▍         | 1329/32000 [04:46<2:33:34,  3.33it/s, est. speed input: 3845.18 toks/s, output: 570.04 toks/s]
Processed prompts:   4%|▍         | 1332/32000 [04:47<1:37:28,  5.24it/s, est. speed input: 3848.87 toks/s, output: 570.64 toks/s]
Processed prompts:   4%|▍         | 1335/32000 [04:47<1:02:53,  8.13it/s, est. speed input: 3850.23 toks/s, output: 571.70 toks/s]
Processed prompts:   4%|▍         | 1337/32000 [04:47<1:19:20,  6.44it/s, est. speed input: 3852.24 toks/s, output: 571.66 toks/s]
Processed prompts:   4%|▍         | 1339/32000 [04:48<1:51:35,  4.58it/s, est. speed input: 3848.02 toks/s, output: 570.93 toks/s]
Processed prompts:   4%|▍         | 1343/32000 [04:49<2:04:51,  4.09it/s, est. speed input: 3839.59 toks/s, output: 569.98 toks/s]
Processed prompts:   4%|▍         | 1344/32000 [04:49<2:13:55,  3.81it/s, est. speed input: 3837.66 toks/s, output: 570.14 toks/s]
Processed prompts:   4%|▍         | 1347/32000 [04:50<1:53:44,  4.49it/s, est. speed input: 3837.16 toks/s, output: 570.54 toks/s]
Processed prompts:   4%|▍         | 1349/32000 [04:50<2:00:55,  4.22it/s, est. speed input: 3838.34 toks/s, output: 570.64 toks/s]
Processed prompts:   4%|▍         | 1352/32000 [04:51<1:39:16,  5.15it/s, est. speed input: 3838.17 toks/s, output: 571.13 toks/s]
Processed prompts:   4%|▍         | 1354/32000 [04:51<1:49:19,  4.67it/s, est. speed input: 3833.63 toks/s, output: 571.05 toks/s]
Processed prompts:   4%|▍         | 1357/32000 [04:52<1:27:35,  5.83it/s, est. speed input: 3836.65 toks/s, output: 572.02 toks/s]
Processed prompts:   4%|▍         | 1360/32000 [04:52<1:24:58,  6.01it/s, est. speed input: 3836.72 toks/s, output: 572.19 toks/s]
Processed prompts:   4%|▍         | 1362/32000 [04:53<1:41:51,  5.01it/s, est. speed input: 3833.97 toks/s, output: 571.72 toks/s]
Processed prompts:   4%|▍         | 1363/32000 [04:53<1:40:10,  5.10it/s, est. speed input: 3835.35 toks/s, output: 571.98 toks/s]
Processed prompts:   4%|▍         | 1365/32000 [04:53<1:25:54,  5.94it/s, est. speed input: 3837.88 toks/s, output: 572.57 toks/s]
Processed prompts:   4%|▍         | 1367/32000 [04:54<1:47:10,  4.76it/s, est. speed input: 3834.36 toks/s, output: 572.07 toks/s]
Processed prompts:   4%|▍         | 1370/32000 [04:54<1:19:46,  6.40it/s, est. speed input: 3846.20 toks/s, output: 572.43 toks/s]
Processed prompts:   4%|▍         | 1371/32000 [04:55<2:09:34,  3.94it/s, est. speed input: 3841.06 toks/s, output: 571.78 toks/s]
Processed prompts:   4%|▍         | 1373/32000 [04:55<1:51:27,  4.58it/s, est. speed input: 3858.63 toks/s, output: 572.03 toks/s]
Processed prompts:   4%|▍         | 1377/32000 [04:55<1:23:00,  6.15it/s, est. speed input: 3866.79 toks/s, output: 572.70 toks/s]
Processed prompts:   4%|▍         | 1379/32000 [04:56<2:02:08,  4.18it/s, est. speed input: 3859.21 toks/s, output: 571.55 toks/s]
Processed prompts:   4%|▍         | 1380/32000 [04:57<2:11:16,  3.89it/s, est. speed input: 3857.66 toks/s, output: 571.14 toks/s]
Processed prompts:   4%|▍         | 1381/32000 [04:57<2:48:59,  3.02it/s, est. speed input: 3850.71 toks/s, output: 570.27 toks/s]
Processed prompts:   4%|▍         | 1382/32000 [04:58<2:34:48,  3.30it/s, est. speed input: 3850.57 toks/s, output: 570.23 toks/s]
Processed prompts:   4%|▍         | 1383/32000 [04:58<2:47:17,  3.05it/s, est. speed input: 3848.36 toks/s, output: 569.66 toks/s]
Processed prompts:   4%|▍         | 1384/32000 [04:58<2:42:37,  3.14it/s, est. speed input: 3852.56 toks/s, output: 569.57 toks/s]
Processed prompts:   4%|▍         | 1386/32000 [04:58<1:54:14,  4.47it/s, est. speed input: 3852.59 toks/s, output: 570.27 toks/s]
Processed prompts:   4%|▍         | 1388/32000 [04:59<2:06:38,  4.03it/s, est. speed input: 3853.20 toks/s, output: 570.04 toks/s]
Processed prompts:   4%|▍         | 1391/32000 [05:00<2:34:49,  3.30it/s, est. speed input: 3848.75 toks/s, output: 569.28 toks/s]
Processed prompts:   4%|▍         | 1394/32000 [05:01<2:24:44,  3.52it/s, est. speed input: 3845.78 toks/s, output: 568.96 toks/s]
Processed prompts:   4%|▍         | 1396/32000 [05:01<1:57:57,  4.32it/s, est. speed input: 3855.13 toks/s, output: 569.89 toks/s]
Processed prompts:   4%|▍         | 1400/32000 [05:01<1:22:53,  6.15it/s, est. speed input: 3867.26 toks/s, output: 570.66 toks/s]
Processed prompts:   4%|▍         | 1401/32000 [05:02<2:20:55,  3.62it/s, est. speed input: 3855.20 toks/s, output: 569.14 toks/s]
Processed prompts:   4%|▍         | 1402/32000 [05:03<2:21:16,  3.61it/s, est. speed input: 3853.52 toks/s, output: 569.06 toks/s]
Processed prompts:   4%|▍         | 1405/32000 [05:03<1:36:54,  5.26it/s, est. speed input: 3856.13 toks/s, output: 570.01 toks/s]
Processed prompts:   4%|▍         | 1406/32000 [05:04<2:32:54,  3.33it/s, est. speed input: 3850.67 toks/s, output: 568.73 toks/s]
Processed prompts:   4%|▍         | 1408/32000 [05:04<2:41:36,  3.16it/s, est. speed input: 3845.40 toks/s, output: 568.02 toks/s]
Processed prompts:   4%|▍         | 1409/32000 [05:05<2:39:03,  3.21it/s, est. speed input: 3847.68 toks/s, output: 567.95 toks/s]
Processed prompts:   4%|▍         | 1410/32000 [05:05<2:41:13,  3.16it/s, est. speed input: 3847.38 toks/s, output: 568.15 toks/s]
Processed prompts:   4%|▍         | 1414/32000 [05:05<1:35:12,  5.35it/s, est. speed input: 3851.80 toks/s, output: 569.21 toks/s]
Processed prompts:   4%|▍         | 1416/32000 [05:06<1:56:28,  4.38it/s, est. speed input: 3846.03 toks/s, output: 568.62 toks/s]
Processed prompts:   4%|▍         | 1418/32000 [05:07<1:59:42,  4.26it/s, est. speed input: 3843.79 toks/s, output: 568.47 toks/s]
Processed prompts:   4%|▍         | 1419/32000 [05:07<1:56:14,  4.38it/s, est. speed input: 3846.44 toks/s, output: 568.80 toks/s]
Processed prompts:   4%|▍         | 1423/32000 [05:07<1:25:07,  5.99it/s, est. speed input: 3854.38 toks/s, output: 570.08 toks/s]
Processed prompts:   4%|▍         | 1425/32000 [05:08<1:42:33,  4.97it/s, est. speed input: 3854.58 toks/s, output: 569.75 toks/s]
Processed prompts:   4%|▍         | 1427/32000 [05:08<1:38:46,  5.16it/s, est. speed input: 3856.77 toks/s, output: 569.76 toks/s]
Processed prompts:   4%|▍         | 1429/32000 [05:09<1:52:56,  4.51it/s, est. speed input: 3854.44 toks/s, output: 569.47 toks/s]
Processed prompts:   4%|▍         | 1432/32000 [05:10<2:26:45,  3.47it/s, est. speed input: 3848.56 toks/s, output: 568.74 toks/s]
Processed prompts:   4%|▍         | 1435/32000 [05:10<1:52:46,  4.52it/s, est. speed input: 3852.62 toks/s, output: 569.58 toks/s]
Processed prompts:   4%|▍         | 1439/32000 [05:11<1:53:02,  4.51it/s, est. speed input: 3851.10 toks/s, output: 569.31 toks/s]
Processed prompts:   4%|▍         | 1440/32000 [05:11<1:53:46,  4.48it/s, est. speed input: 3850.76 toks/s, output: 569.37 toks/s]
Processed prompts:   5%|▍         | 1443/32000 [05:12<2:07:31,  3.99it/s, est. speed input: 3847.26 toks/s, output: 569.10 toks/s]
Processed prompts:   5%|▍         | 1448/32000 [05:12<1:19:53,  6.37it/s, est. speed input: 3857.35 toks/s, output: 570.41 toks/s]
Processed prompts:   5%|▍         | 1450/32000 [05:13<1:58:46,  4.29it/s, est. speed input: 3847.46 toks/s, output: 569.03 toks/s]
Processed prompts:   5%|▍         | 1451/32000 [05:14<2:11:08,  3.88it/s, est. speed input: 3845.48 toks/s, output: 568.73 toks/s]
Processed prompts:   5%|▍         | 1456/32000 [05:14<1:19:03,  6.44it/s, est. speed input: 3854.41 toks/s, output: 570.21 toks/s]
Processed prompts:   5%|▍         | 1458/32000 [05:15<1:57:31,  4.33it/s, est. speed input: 3848.85 toks/s, output: 569.27 toks/s]
Processed prompts:   5%|▍         | 1460/32000 [05:15<1:49:55,  4.63it/s, est. speed input: 3851.11 toks/s, output: 569.65 toks/s]
Processed prompts:   5%|▍         | 1461/32000 [05:16<1:41:29,  5.01it/s, est. speed input: 3852.29 toks/s, output: 570.20 toks/s]
Processed prompts:   5%|▍         | 1464/32000 [05:17<2:26:17,  3.48it/s, est. speed input: 3841.50 toks/s, output: 569.39 toks/s]
Processed prompts:   5%|▍         | 1468/32000 [05:18<2:03:05,  4.13it/s, est. speed input: 3837.15 toks/s, output: 569.59 toks/s]
Processed prompts:   5%|▍         | 1471/32000 [05:18<1:32:54,  5.48it/s, est. speed input: 3845.50 toks/s, output: 570.82 toks/s]
Processed prompts:   5%|▍         | 1474/32000 [05:19<2:03:37,  4.12it/s, est. speed input: 3844.22 toks/s, output: 570.73 toks/s]
Processed prompts:   5%|▍         | 1475/32000 [05:19<2:08:17,  3.97it/s, est. speed input: 3847.83 toks/s, output: 570.62 toks/s]
Processed prompts:   5%|▍         | 1478/32000 [05:19<1:36:02,  5.30it/s, est. speed input: 3855.77 toks/s, output: 571.50 toks/s]
Processed prompts:   5%|▍         | 1484/32000 [05:20<1:29:22,  5.69it/s, est. speed input: 3858.67 toks/s, output: 572.06 toks/s]
Processed prompts:   5%|▍         | 1487/32000 [05:21<1:31:13,  5.57it/s, est. speed input: 3864.35 toks/s, output: 571.94 toks/s]
Processed prompts:   5%|▍         | 1488/32000 [05:21<1:47:20,  4.74it/s, est. speed input: 3860.12 toks/s, output: 571.65 toks/s]
Processed prompts:   5%|▍         | 1491/32000 [05:22<1:38:30,  5.16it/s, est. speed input: 3861.40 toks/s, output: 571.68 toks/s]
Processed prompts:   5%|▍         | 1492/32000 [05:22<1:54:59,  4.42it/s, est. speed input: 3858.39 toks/s, output: 571.30 toks/s]
Processed prompts:   5%|▍         | 1494/32000 [05:23<1:38:41,  5.15it/s, est. speed input: 3859.33 toks/s, output: 571.66 toks/s]
Processed prompts:   5%|▍         | 1497/32000 [05:23<1:22:54,  6.13it/s, est. speed input: 3860.31 toks/s, output: 571.94 toks/s]
Processed prompts:   5%|▍         | 1498/32000 [05:23<1:54:41,  4.43it/s, est. speed input: 3855.77 toks/s, output: 571.45 toks/s]
Processed prompts:   5%|▍         | 1499/32000 [05:24<1:50:59,  4.58it/s, est. speed input: 3857.28 toks/s, output: 571.92 toks/s]
Processed prompts:   5%|▍         | 1502/32000 [05:24<1:26:02,  5.91it/s, est. speed input: 3866.06 toks/s, output: 573.03 toks/s]
Processed prompts:   5%|▍         | 1504/32000 [05:26<3:35:10,  2.36it/s, est. speed input: 3844.99 toks/s, output: 570.13 toks/s]
Processed prompts:   5%|▍         | 1509/32000 [05:26<1:54:32,  4.44it/s, est. speed input: 3853.78 toks/s, output: 571.79 toks/s]
Processed prompts:   5%|▍         | 1511/32000 [05:27<2:00:42,  4.21it/s, est. speed input: 3850.43 toks/s, output: 571.49 toks/s]
Processed prompts:   5%|▍         | 1514/32000 [05:28<2:05:18,  4.05it/s, est. speed input: 3845.64 toks/s, output: 571.38 toks/s]
Processed prompts:   5%|▍         | 1516/32000 [05:28<1:49:45,  4.63it/s, est. speed input: 3845.58 toks/s, output: 571.68 toks/s]
Processed prompts:   5%|▍         | 1518/32000 [05:28<2:02:51,  4.14it/s, est. speed input: 3844.76 toks/s, output: 571.20 toks/s]
Processed prompts:   5%|▍         | 1520/32000 [05:29<2:26:48,  3.46it/s, est. speed input: 3838.61 toks/s, output: 570.54 toks/s]
Processed prompts:   5%|▍         | 1522/32000 [05:30<2:52:15,  2.95it/s, est. speed input: 3830.49 toks/s, output: 569.74 toks/s]
Processed prompts:   5%|▍         | 1525/32000 [05:31<2:30:26,  3.38it/s, est. speed input: 3830.51 toks/s, output: 569.89 toks/s]
Processed prompts:   5%|▍         | 1528/32000 [05:31<1:51:11,  4.57it/s, est. speed input: 3838.08 toks/s, output: 571.17 toks/s]
Processed prompts:   5%|▍         | 1530/32000 [05:31<1:46:42,  4.76it/s, est. speed input: 3837.02 toks/s, output: 571.26 toks/s]
Processed prompts:   5%|▍         | 1533/32000 [05:32<1:37:03,  5.23it/s, est. speed input: 3836.96 toks/s, output: 571.60 toks/s]
Processed prompts:   5%|▍         | 1535/32000 [05:32<1:30:13,  5.63it/s, est. speed input: 3839.73 toks/s, output: 571.88 toks/s]
Processed prompts:   5%|▍         | 1537/32000 [05:33<1:42:34,  4.95it/s, est. speed input: 3836.75 toks/s, output: 571.46 toks/s]
Processed prompts:   5%|▍         | 1538/32000 [05:33<1:54:53,  4.42it/s, est. speed input: 3833.92 toks/s, output: 571.12 toks/s]
Processed prompts:   5%|▍         | 1539/32000 [05:34<2:58:18,  2.85it/s, est. speed input: 3828.95 toks/s, output: 570.12 toks/s]
Processed prompts:   5%|▍         | 1542/32000 [05:34<1:47:28,  4.72it/s, est. speed input: 3834.17 toks/s, output: 571.23 toks/s]
Processed prompts:   5%|▍         | 1544/32000 [05:35<2:16:04,  3.73it/s, est. speed input: 3829.37 toks/s, output: 570.94 toks/s]
Processed prompts:   5%|▍         | 1546/32000 [05:35<1:53:42,  4.46it/s, est. speed input: 3828.10 toks/s, output: 571.14 toks/s]
Processed prompts:   5%|▍         | 1548/32000 [05:36<1:50:47,  4.58it/s, est. speed input: 3829.92 toks/s, output: 571.34 toks/s]
Processed prompts:   5%|▍         | 1551/32000 [05:36<1:40:41,  5.04it/s, est. speed input: 3826.58 toks/s, output: 571.30 toks/s]
Processed prompts:   5%|▍         | 1552/32000 [05:36<1:46:55,  4.75it/s, est. speed input: 3824.54 toks/s, output: 571.51 toks/s]
Processed prompts:   5%|▍         | 1554/32000 [05:37<1:29:21,  5.68it/s, est. speed input: 3827.21 toks/s, output: 571.83 toks/s]
Processed prompts:   5%|▍         | 1555/32000 [05:37<2:15:44,  3.74it/s, est. speed input: 3822.67 toks/s, output: 571.06 toks/s]
Processed prompts:   5%|▍         | 1557/32000 [05:38<2:01:39,  4.17it/s, est. speed input: 3822.92 toks/s, output: 571.09 toks/s]
Processed prompts:   5%|▍         | 1560/32000 [05:38<1:32:57,  5.46it/s, est. speed input: 3823.80 toks/s, output: 571.51 toks/s]
Processed prompts:   5%|▍         | 1563/32000 [05:39<2:00:33,  4.21it/s, est. speed input: 3826.84 toks/s, output: 571.09 toks/s]
Processed prompts:   5%|▍         | 1566/32000 [05:40<2:12:37,  3.82it/s, est. speed input: 3826.31 toks/s, output: 570.54 toks/s]
Processed prompts:   5%|▍         | 1567/32000 [05:40<2:15:07,  3.75it/s, est. speed input: 3823.60 toks/s, output: 570.29 toks/s]
Processed prompts:   5%|▍         | 1569/32000 [05:41<2:18:19,  3.67it/s, est. speed input: 3822.21 toks/s, output: 569.99 toks/s]
Processed prompts:   5%|▍         | 1571/32000 [05:41<2:25:39,  3.48it/s, est. speed input: 3822.78 toks/s, output: 569.64 toks/s]
Processed prompts:   5%|▍         | 1573/32000 [05:42<2:02:55,  4.13it/s, est. speed input: 3823.35 toks/s, output: 569.83 toks/s]
Processed prompts:   5%|▍         | 1574/32000 [05:42<2:50:48,  2.97it/s, est. speed input: 3820.30 toks/s, output: 569.06 toks/s]
Processed prompts:   5%|▍         | 1575/32000 [05:43<3:08:31,  2.69it/s, est. speed input: 3817.30 toks/s, output: 568.51 toks/s]
Processed prompts:   5%|▍         | 1576/32000 [05:43<2:46:54,  3.04it/s, est. speed input: 3816.49 toks/s, output: 568.55 toks/s]
Processed prompts:   5%|▍         | 1577/32000 [05:43<2:58:11,  2.85it/s, est. speed input: 3812.72 toks/s, output: 568.31 toks/s]
Processed prompts:   5%|▍         | 1579/32000 [05:44<2:23:34,  3.53it/s, est. speed input: 3810.01 toks/s, output: 568.38 toks/s]
Processed prompts:   5%|▍         | 1581/32000 [05:44<1:47:05,  4.73it/s, est. speed input: 3810.69 toks/s, output: 568.66 toks/s]
Processed prompts:   5%|▍         | 1582/32000 [05:44<2:00:43,  4.20it/s, est. speed input: 3808.35 toks/s, output: 568.46 toks/s]
Processed prompts:   5%|▍         | 1586/32000 [05:45<1:16:16,  6.65it/s, est. speed input: 3814.26 toks/s, output: 569.11 toks/s]
Processed prompts:   5%|▍         | 1587/32000 [05:45<1:50:19,  4.59it/s, est. speed input: 3808.84 toks/s, output: 568.49 toks/s]
Processed prompts:   5%|▍         | 1588/32000 [05:45<1:48:20,  4.68it/s, est. speed input: 3807.41 toks/s, output: 568.58 toks/s]
Processed prompts:   5%|▍         | 1591/32000 [05:46<1:16:37,  6.61it/s, est. speed input: 3808.93 toks/s, output: 569.06 toks/s]
Processed prompts:   5%|▍         | 1592/32000 [05:47<2:32:55,  3.31it/s, est. speed input: 3798.76 toks/s, output: 567.95 toks/s]
Processed prompts:   5%|▍         | 1594/32000 [05:47<2:05:40,  4.03it/s, est. speed input: 3802.82 toks/s, output: 568.43 toks/s]
Processed prompts:   5%|▍         | 1596/32000 [05:47<1:52:55,  4.49it/s, est. speed input: 3805.77 toks/s, output: 568.54 toks/s]
Processed prompts:   5%|▍         | 1599/32000 [05:48<2:05:35,  4.03it/s, est. speed input: 3802.93 toks/s, output: 568.45 toks/s]
Processed prompts:   5%|▌         | 1602/32000 [05:48<1:37:20,  5.20it/s, est. speed input: 3808.25 toks/s, output: 568.95 toks/s]
Processed prompts:   5%|▌         | 1603/32000 [05:49<1:53:52,  4.45it/s, est. speed input: 3804.53 toks/s, output: 568.77 toks/s]
Processed prompts:   5%|▌         | 1606/32000 [05:49<1:46:49,  4.74it/s, est. speed input: 3804.55 toks/s, output: 568.90 toks/s]
Processed prompts:   5%|▌         | 1608/32000 [05:50<1:49:40,  4.62it/s, est. speed input: 3805.02 toks/s, output: 568.67 toks/s]
Processed prompts:   5%|▌         | 1609/32000 [05:50<2:12:54,  3.81it/s, est. speed input: 3801.47 toks/s, output: 568.08 toks/s]
Processed prompts:   5%|▌         | 1610/32000 [05:51<2:08:14,  3.95it/s, est. speed input: 3800.89 toks/s, output: 568.01 toks/s]
Processed prompts:   5%|▌         | 1614/32000 [05:51<1:22:01,  6.17it/s, est. speed input: 3804.50 toks/s, output: 568.63 toks/s]
Processed prompts:   5%|▌         | 1615/32000 [05:52<2:40:40,  3.15it/s, est. speed input: 3793.67 toks/s, output: 567.07 toks/s]
Processed prompts:   5%|▌         | 1616/32000 [05:52<2:39:25,  3.18it/s, est. speed input: 3792.42 toks/s, output: 567.01 toks/s]
Processed prompts:   5%|▌         | 1619/32000 [05:53<2:16:22,  3.71it/s, est. speed input: 3789.09 toks/s, output: 566.69 toks/s]
Processed prompts:   5%|▌         | 1620/32000 [05:54<3:04:35,  2.74it/s, est. speed input: 3781.81 toks/s, output: 565.70 toks/s]
Processed prompts:   5%|▌         | 1622/32000 [05:54<2:18:44,  3.65it/s, est. speed input: 3782.29 toks/s, output: 565.94 toks/s]
Processed prompts:   5%|▌         | 1624/32000 [05:55<2:31:30,  3.34it/s, est. speed input: 3777.79 toks/s, output: 565.34 toks/s]
Processed prompts:   5%|▌         | 1626/32000 [05:55<2:05:12,  4.04it/s, est. speed input: 3776.81 toks/s, output: 565.48 toks/s]
Processed prompts:   5%|▌         | 1628/32000 [05:55<1:50:51,  4.57it/s, est. speed input: 3776.21 toks/s, output: 565.94 toks/s]
Processed prompts:   5%|▌         | 1630/32000 [05:56<1:47:47,  4.70it/s, est. speed input: 3774.47 toks/s, output: 565.86 toks/s]
Processed prompts:   5%|▌         | 1633/32000 [05:57<2:37:02,  3.22it/s, est. speed input: 3766.61 toks/s, output: 564.65 toks/s]
Processed prompts:   5%|▌         | 1638/32000 [05:58<1:55:40,  4.37it/s, est. speed input: 3768.82 toks/s, output: 564.82 toks/s]
Processed prompts:   5%|▌         | 1639/32000 [05:58<2:00:28,  4.20it/s, est. speed input: 3766.41 toks/s, output: 564.61 toks/s]
Processed prompts:   5%|▌         | 1640/32000 [05:58<2:04:11,  4.07it/s, est. speed input: 3765.01 toks/s, output: 564.56 toks/s]
Processed prompts:   5%|▌         | 1642/32000 [05:59<1:50:52,  4.56it/s, est. speed input: 3769.59 toks/s, output: 565.07 toks/s]
Processed prompts:   5%|▌         | 1647/32000 [06:00<2:12:11,  3.83it/s, est. speed input: 3764.66 toks/s, output: 564.74 toks/s]
Processed prompts:   5%|▌         | 1649/32000 [06:00<1:55:35,  4.38it/s, est. speed input: 3766.79 toks/s, output: 565.13 toks/s]
Processed prompts:   5%|▌         | 1653/32000 [06:01<1:50:54,  4.56it/s, est. speed input: 3766.40 toks/s, output: 565.22 toks/s]
Processed prompts:   5%|▌         | 1657/32000 [06:02<1:40:12,  5.05it/s, est. speed input: 3768.58 toks/s, output: 565.38 toks/s]
Processed prompts:   5%|▌         | 1658/32000 [06:02<2:01:41,  4.16it/s, est. speed input: 3767.24 toks/s, output: 564.76 toks/s]
Processed prompts:   5%|▌         | 1661/32000 [06:03<1:45:58,  4.77it/s, est. speed input: 3775.20 toks/s, output: 564.85 toks/s]
Processed prompts:   5%|▌         | 1663/32000 [06:03<1:57:22,  4.31it/s, est. speed input: 3770.34 toks/s, output: 564.66 toks/s]
Processed prompts:   5%|▌         | 1666/32000 [06:04<1:41:18,  4.99it/s, est. speed input: 3770.53 toks/s, output: 565.27 toks/s]
Processed prompts:   5%|▌         | 1670/32000 [06:05<2:17:26,  3.68it/s, est. speed input: 3768.87 toks/s, output: 564.11 toks/s]
Processed prompts:   5%|▌         | 1673/32000 [06:06<1:46:33,  4.74it/s, est. speed input: 3775.20 toks/s, output: 564.93 toks/s]
Processed prompts:   5%|▌         | 1676/32000 [06:06<1:48:59,  4.64it/s, est. speed input: 3775.82 toks/s, output: 564.89 toks/s]
Processed prompts:   5%|▌         | 1678/32000 [06:08<2:37:43,  3.20it/s, est. speed input: 3770.80 toks/s, output: 563.39 toks/s]
Processed prompts:   5%|▌         | 1679/32000 [06:08<2:37:45,  3.20it/s, est. speed input: 3775.42 toks/s, output: 563.27 toks/s]
Processed prompts:   5%|▌         | 1682/32000 [06:08<1:53:20,  4.46it/s, est. speed input: 3777.69 toks/s, output: 563.81 toks/s]
Processed prompts:   5%|▌         | 1683/32000 [06:09<2:23:28,  3.52it/s, est. speed input: 3776.14 toks/s, output: 563.38 toks/s]
Processed prompts:   5%|▌         | 1685/32000 [06:09<2:00:50,  4.18it/s, est. speed input: 3778.95 toks/s, output: 564.04 toks/s]
Processed prompts:   5%|▌         | 1689/32000 [06:10<2:11:13,  3.85it/s, est. speed input: 3775.81 toks/s, output: 564.02 toks/s]
Processed prompts:   5%|▌         | 1690/32000 [06:10<2:08:30,  3.93it/s, est. speed input: 3774.97 toks/s, output: 563.96 toks/s]
Processed prompts:   5%|▌         | 1692/32000 [06:11<1:54:14,  4.42it/s, est. speed input: 3775.21 toks/s, output: 564.11 toks/s]
Processed prompts:   5%|▌         | 1694/32000 [06:11<2:06:56,  3.98it/s, est. speed input: 3772.58 toks/s, output: 563.81 toks/s]
Processed prompts:   5%|▌         | 1696/32000 [06:12<1:44:27,  4.83it/s, est. speed input: 3774.87 toks/s, output: 564.45 toks/s]
Processed prompts:   5%|▌         | 1697/32000 [06:12<1:44:06,  4.85it/s, est. speed input: 3776.30 toks/s, output: 564.35 toks/s]
Processed prompts:   5%|▌         | 1698/32000 [06:12<1:55:18,  4.38it/s, est. speed input: 3774.37 toks/s, output: 564.10 toks/s]
Processed prompts:   5%|▌         | 1699/32000 [06:13<3:14:41,  2.59it/s, est. speed input: 3767.47 toks/s, output: 563.08 toks/s]
Processed prompts:   5%|▌         | 1702/32000 [06:13<1:48:25,  4.66it/s, est. speed input: 3772.57 toks/s, output: 563.91 toks/s]
Processed prompts:   5%|▌         | 1704/32000 [06:14<2:04:36,  4.05it/s, est. speed input: 3777.25 toks/s, output: 563.75 toks/s]
Processed prompts:   5%|▌         | 1705/32000 [06:14<2:20:40,  3.59it/s, est. speed input: 3773.66 toks/s, output: 563.35 toks/s]
Processed prompts:   5%|▌         | 1706/32000 [06:15<2:41:22,  3.13it/s, est. speed input: 3773.57 toks/s, output: 563.06 toks/s]
Processed prompts:   5%|▌         | 1707/32000 [06:15<3:24:07,  2.47it/s, est. speed input: 3769.71 toks/s, output: 562.70 toks/s]
Processed prompts:   5%|▌         | 1709/32000 [06:15<2:21:12,  3.58it/s, est. speed input: 3771.56 toks/s, output: 563.10 toks/s]
Processed prompts:   5%|▌         | 1710/32000 [06:16<2:45:38,  3.05it/s, est. speed input: 3769.83 toks/s, output: 563.03 toks/s]
Processed prompts:   5%|▌         | 1712/32000 [06:17<3:07:02,  2.70it/s, est. speed input: 3766.50 toks/s, output: 562.68 toks/s]
Processed prompts:   5%|▌         | 1717/32000 [06:17<1:35:07,  5.31it/s, est. speed input: 3773.24 toks/s, output: 564.40 toks/s]
Processed prompts:   5%|▌         | 1718/32000 [06:18<1:52:45,  4.48it/s, est. speed input: 3771.68 toks/s, output: 563.84 toks/s]
Processed prompts:   5%|▌         | 1719/32000 [06:18<1:52:03,  4.50it/s, est. speed input: 3771.90 toks/s, output: 563.85 toks/s]
Processed prompts:   5%|▌         | 1722/32000 [06:18<1:33:23,  5.40it/s, est. speed input: 3772.76 toks/s, output: 564.20 toks/s]
Processed prompts:   5%|▌         | 1724/32000 [06:19<1:37:07,  5.19it/s, est. speed input: 3774.37 toks/s, output: 564.28 toks/s]
Processed prompts:   5%|▌         | 1725/32000 [06:19<1:58:10,  4.27it/s, est. speed input: 3774.88 toks/s, output: 564.30 toks/s]
Processed prompts:   5%|▌         | 1728/32000 [06:20<2:07:02,  3.97it/s, est. speed input: 3775.80 toks/s, output: 563.90 toks/s]
Processed prompts:   5%|▌         | 1730/32000 [06:20<2:01:54,  4.14it/s, est. speed input: 3774.55 toks/s, output: 563.90 toks/s]
Processed prompts:   5%|▌         | 1732/32000 [06:20<1:34:17,  5.35it/s, est. speed input: 3778.04 toks/s, output: 564.44 toks/s]
Processed prompts:   5%|▌         | 1734/32000 [06:21<1:49:11,  4.62it/s, est. speed input: 3780.03 toks/s, output: 564.51 toks/s]
Processed prompts:   5%|▌         | 1736/32000 [06:21<1:43:19,  4.88it/s, est. speed input: 3778.07 toks/s, output: 564.92 toks/s]
Processed prompts:   5%|▌         | 1739/32000 [06:22<1:40:13,  5.03it/s, est. speed input: 3778.48 toks/s, output: 565.18 toks/s]
Processed prompts:   5%|▌         | 1741/32000 [06:23<1:56:06,  4.34it/s, est. speed input: 3776.06 toks/s, output: 564.73 toks/s]
Processed prompts:   5%|▌         | 1743/32000 [06:23<1:40:22,  5.02it/s, est. speed input: 3777.05 toks/s, output: 564.82 toks/s]
Processed prompts:   5%|▌         | 1744/32000 [06:23<2:04:05,  4.06it/s, est. speed input: 3775.31 toks/s, output: 564.37 toks/s]
Processed prompts:   5%|▌         | 1745/32000 [06:24<2:37:09,  3.21it/s, est. speed input: 3772.98 toks/s, output: 564.02 toks/s]
Processed prompts:   5%|▌         | 1746/32000 [06:25<3:17:52,  2.55it/s, est. speed input: 3768.27 toks/s, output: 563.39 toks/s]
Processed prompts:   5%|▌         | 1747/32000 [06:25<2:54:20,  2.89it/s, est. speed input: 3769.97 toks/s, output: 563.73 toks/s]
Processed prompts:   5%|▌         | 1751/32000 [06:25<1:34:21,  5.34it/s, est. speed input: 3776.75 toks/s, output: 564.63 toks/s]
Processed prompts:   5%|▌         | 1753/32000 [06:26<2:05:49,  4.01it/s, est. speed input: 3774.76 toks/s, output: 564.42 toks/s]
Processed prompts:   5%|▌         | 1759/32000 [06:27<1:48:33,  4.64it/s, est. speed input: 3776.07 toks/s, output: 564.76 toks/s]
Processed prompts:   6%|▌         | 1761/32000 [06:27<1:40:15,  5.03it/s, est. speed input: 3778.07 toks/s, output: 565.30 toks/s]
Processed prompts:   6%|▌         | 1765/32000 [06:28<1:19:36,  6.33it/s, est. speed input: 3780.72 toks/s, output: 566.13 toks/s]
Processed prompts:   6%|▌         | 1769/32000 [06:28<1:22:48,  6.08it/s, est. speed input: 3788.15 toks/s, output: 566.17 toks/s]
Processed prompts:   6%|▌         | 1773/32000 [06:29<1:14:40,  6.75it/s, est. speed input: 3793.35 toks/s, output: 566.58 toks/s]
Processed prompts:   6%|▌         | 1774/32000 [06:30<1:59:16,  4.22it/s, est. speed input: 3784.74 toks/s, output: 565.67 toks/s]
Processed prompts:   6%|▌         | 1775/32000 [06:30<2:05:49,  4.00it/s, est. speed input: 3782.16 toks/s, output: 565.68 toks/s]
Processed prompts:   6%|▌         | 1777/32000 [06:31<2:33:09,  3.29it/s, est. speed input: 3777.21 toks/s, output: 564.91 toks/s]
Processed prompts:   6%|▌         | 1779/32000 [06:31<2:03:51,  4.07it/s, est. speed input: 3780.78 toks/s, output: 565.42 toks/s]
Processed prompts:   6%|▌         | 1782/32000 [06:32<1:46:37,  4.72it/s, est. speed input: 3784.23 toks/s, output: 565.57 toks/s]
Processed prompts:   6%|▌         | 1785/32000 [06:32<1:25:21,  5.90it/s, est. speed input: 3794.41 toks/s, output: 565.78 toks/s]
Processed prompts:   6%|▌         | 1786/32000 [06:32<1:46:04,  4.75it/s, est. speed input: 3793.40 toks/s, output: 565.46 toks/s]
Processed prompts:   6%|▌         | 1788/32000 [06:33<2:15:49,  3.71it/s, est. speed input: 3791.55 toks/s, output: 565.21 toks/s]
Processed prompts:   6%|▌         | 1791/32000 [06:34<1:46:26,  4.73it/s, est. speed input: 3792.97 toks/s, output: 565.59 toks/s]
Processed prompts:   6%|▌         | 1793/32000 [06:34<1:45:36,  4.77it/s, est. speed input: 3794.27 toks/s, output: 565.73 toks/s]
Processed prompts:   6%|▌         | 1795/32000 [06:34<1:44:21,  4.82it/s, est. speed input: 3796.65 toks/s, output: 565.69 toks/s]
Processed prompts:   6%|▌         | 1797/32000 [06:35<1:27:07,  5.78it/s, est. speed input: 3797.10 toks/s, output: 565.89 toks/s]
Processed prompts:   6%|▌         | 1799/32000 [06:35<1:44:31,  4.82it/s, est. speed input: 3796.31 toks/s, output: 565.58 toks/s]
Processed prompts:   6%|▌         | 1802/32000 [06:36<1:42:00,  4.93it/s, est. speed input: 3798.68 toks/s, output: 565.67 toks/s]
Processed prompts:   6%|▌         | 1804/32000 [06:37<2:20:04,  3.59it/s, est. speed input: 3791.57 toks/s, output: 564.89 toks/s]
Processed prompts:   6%|▌         | 1805/32000 [06:37<2:40:13,  3.14it/s, est. speed input: 3788.38 toks/s, output: 564.39 toks/s]
Processed prompts:   6%|▌         | 1806/32000 [06:37<2:20:02,  3.59it/s, est. speed input: 3791.41 toks/s, output: 564.54 toks/s]
Processed prompts:   6%|▌         | 1807/32000 [06:38<2:35:19,  3.24it/s, est. speed input: 3791.32 toks/s, output: 564.22 toks/s]
Processed prompts:   6%|▌         | 1809/32000 [06:38<2:03:28,  4.08it/s, est. speed input: 3792.13 toks/s, output: 564.46 toks/s]
Processed prompts:   6%|▌         | 1811/32000 [06:38<1:52:25,  4.48it/s, est. speed input: 3797.67 toks/s, output: 564.52 toks/s]
Processed prompts:   6%|▌         | 1813/32000 [06:39<1:41:06,  4.98it/s, est. speed input: 3799.10 toks/s, output: 564.94 toks/s]
Processed prompts:   6%|▌         | 1816/32000 [06:39<1:24:43,  5.94it/s, est. speed input: 3799.79 toks/s, output: 565.41 toks/s]
Processed prompts:   6%|▌         | 1817/32000 [06:40<2:05:43,  4.00it/s, est. speed input: 3797.75 toks/s, output: 564.85 toks/s]
Processed prompts:   6%|▌         | 1819/32000 [06:40<1:44:46,  4.80it/s, est. speed input: 3800.54 toks/s, output: 565.39 toks/s]
Processed prompts:   6%|▌         | 1822/32000 [06:41<2:28:59,  3.38it/s, est. speed input: 3801.76 toks/s, output: 564.65 toks/s]
Processed prompts:   6%|▌         | 1823/32000 [06:41<2:19:45,  3.60it/s, est. speed input: 3802.16 toks/s, output: 564.71 toks/s]
Processed prompts:   6%|▌         | 1824/32000 [06:42<3:07:03,  2.69it/s, est. speed input: 3799.71 toks/s, output: 563.94 toks/s]
Processed prompts:   6%|▌         | 1826/32000 [06:42<2:20:30,  3.58it/s, est. speed input: 3803.90 toks/s, output: 564.51 toks/s]
Processed prompts:   6%|▌         | 1828/32000 [06:43<1:58:08,  4.26it/s, est. speed input: 3805.40 toks/s, output: 564.72 toks/s]
Processed prompts:   6%|▌         | 1833/32000 [06:43<1:25:48,  5.86it/s, est. speed input: 3812.60 toks/s, output: 565.17 toks/s]
Processed prompts:   6%|▌         | 1834/32000 [06:44<1:53:23,  4.43it/s, est. speed input: 3809.27 toks/s, output: 564.61 toks/s]
Processed prompts:   6%|▌         | 1837/32000 [06:44<1:33:24,  5.38it/s, est. speed input: 3809.35 toks/s, output: 564.98 toks/s]
Processed prompts:   6%|▌         | 1839/32000 [06:45<1:49:11,  4.60it/s, est. speed input: 3815.13 toks/s, output: 565.00 toks/s]
Processed prompts:   6%|▌         | 1841/32000 [06:45<1:47:40,  4.67it/s, est. speed input: 3815.72 toks/s, output: 565.35 toks/s]
Processed prompts:   6%|▌         | 1842/32000 [06:46<3:02:41,  2.75it/s, est. speed input: 3805.59 toks/s, output: 563.95 toks/s]
Processed prompts:   6%|▌         | 1843/32000 [06:46<2:38:07,  3.18it/s, est. speed input: 3806.55 toks/s, output: 564.17 toks/s]
Processed prompts:   6%|▌         | 1847/32000 [06:47<1:37:15,  5.17it/s, est. speed input: 3816.35 toks/s, output: 564.97 toks/s]
Processed prompts:   6%|▌         | 1850/32000 [06:47<1:43:10,  4.87it/s, est. speed input: 3820.60 toks/s, output: 565.07 toks/s]
Processed prompts:   6%|▌         | 1852/32000 [06:48<1:45:56,  4.74it/s, est. speed input: 3820.46 toks/s, output: 564.91 toks/s]
Processed prompts:   6%|▌         | 1853/32000 [06:49<2:26:53,  3.42it/s, est. speed input: 3816.96 toks/s, output: 564.20 toks/s]
Processed prompts:   6%|▌         | 1856/32000 [06:49<1:35:30,  5.26it/s, est. speed input: 3824.83 toks/s, output: 564.90 toks/s]
Processed prompts:   6%|▌         | 1858/32000 [06:50<2:30:33,  3.34it/s, est. speed input: 3818.54 toks/s, output: 563.66 toks/s]
Processed prompts:   6%|▌         | 1859/32000 [06:50<2:29:21,  3.36it/s, est. speed input: 3819.18 toks/s, output: 563.57 toks/s]
Processed prompts:   6%|▌         | 1860/32000 [06:51<2:33:10,  3.28it/s, est. speed input: 3818.09 toks/s, output: 563.38 toks/s]
Processed prompts:   6%|▌         | 1862/32000 [06:51<2:03:22,  4.07it/s, est. speed input: 3819.56 toks/s, output: 563.53 toks/s]
Processed prompts:   6%|▌         | 1863/32000 [06:51<2:20:45,  3.57it/s, est. speed input: 3816.40 toks/s, output: 563.27 toks/s]
Processed prompts:   6%|▌         | 1865/32000 [06:52<1:54:45,  4.38it/s, est. speed input: 3821.54 toks/s, output: 563.65 toks/s]
Processed prompts:   6%|▌         | 1869/32000 [06:53<2:06:11,  3.98it/s, est. speed input: 3817.29 toks/s, output: 563.41 toks/s]
Processed prompts:   6%|▌         | 1871/32000 [06:53<2:04:00,  4.05it/s, est. speed input: 3818.69 toks/s, output: 563.64 toks/s]
Processed prompts:   6%|▌         | 1875/32000 [06:54<2:02:51,  4.09it/s, est. speed input: 3815.07 toks/s, output: 563.28 toks/s]
Processed prompts:   6%|▌         | 1876/32000 [06:54<2:04:42,  4.03it/s, est. speed input: 3813.83 toks/s, output: 563.19 toks/s]
Processed prompts:   6%|▌         | 1878/32000 [06:55<1:54:39,  4.38it/s, est. speed input: 3813.40 toks/s, output: 563.13 toks/s]
Processed prompts:   6%|▌         | 1879/32000 [06:55<2:32:21,  3.29it/s, est. speed input: 3807.60 toks/s, output: 562.54 toks/s]
Processed prompts:   6%|▌         | 1880/32000 [06:56<2:35:07,  3.24it/s, est. speed input: 3805.71 toks/s, output: 562.42 toks/s]
Processed prompts:   6%|▌         | 1883/32000 [06:56<1:43:08,  4.87it/s, est. speed input: 3809.15 toks/s, output: 562.79 toks/s]
Processed prompts:   6%|▌         | 1884/32000 [06:56<1:51:19,  4.51it/s, est. speed input: 3812.14 toks/s, output: 563.01 toks/s]
Processed prompts:   6%|▌         | 1885/32000 [06:57<2:59:46,  2.79it/s, est. speed input: 3808.65 toks/s, output: 562.06 toks/s]
Processed prompts:   6%|▌         | 1887/32000 [06:57<2:10:46,  3.84it/s, est. speed input: 3810.83 toks/s, output: 562.45 toks/s]
Processed prompts:   6%|▌         | 1888/32000 [06:58<2:13:51,  3.75it/s, est. speed input: 3809.44 toks/s, output: 562.29 toks/s]
Processed prompts:   6%|▌         | 1890/32000 [06:58<1:57:20,  4.28it/s, est. speed input: 3809.80 toks/s, output: 562.18 toks/s]
Processed prompts:   6%|▌         | 1892/32000 [06:58<1:49:16,  4.59it/s, est. speed input: 3809.00 toks/s, output: 562.16 toks/s]
Processed prompts:   6%|▌         | 1895/32000 [07:00<2:26:24,  3.43it/s, est. speed input: 3806.50 toks/s, output: 561.48 toks/s]
Processed prompts:   6%|▌         | 1896/32000 [07:00<2:42:44,  3.08it/s, est. speed input: 3802.79 toks/s, output: 561.06 toks/s]
Processed prompts:   6%|▌         | 1897/32000 [07:00<2:32:05,  3.30it/s, est. speed input: 3801.96 toks/s, output: 561.10 toks/s]
Processed prompts:   6%|▌         | 1900/32000 [07:01<1:48:41,  4.62it/s, est. speed input: 3803.55 toks/s, output: 561.56 toks/s]
Processed prompts:   6%|▌         | 1901/32000 [07:01<2:26:14,  3.43it/s, est. speed input: 3800.56 toks/s, output: 561.25 toks/s]
Processed prompts:   6%|▌         | 1902/32000 [07:02<2:30:51,  3.33it/s, est. speed input: 3798.31 toks/s, output: 561.40 toks/s]
Processed prompts:   6%|▌         | 1905/32000 [07:02<1:44:48,  4.79it/s, est. speed input: 3799.25 toks/s, output: 561.96 toks/s]
Processed prompts:   6%|▌         | 1907/32000 [07:02<1:30:52,  5.52it/s, est. speed input: 3798.75 toks/s, output: 562.51 toks/s]
Processed prompts:   6%|▌         | 1911/32000 [07:03<1:12:39,  6.90it/s, est. speed input: 3803.83 toks/s, output: 562.82 toks/s]
Processed prompts:   6%|▌         | 1912/32000 [07:04<2:30:32,  3.33it/s, est. speed input: 3795.28 toks/s, output: 561.51 toks/s]
Processed prompts:   6%|▌         | 1916/32000 [07:05<2:15:26,  3.70it/s, est. speed input: 3793.05 toks/s, output: 561.39 toks/s]
Processed prompts:   6%|▌         | 1918/32000 [07:05<1:48:42,  4.61it/s, est. speed input: 3794.30 toks/s, output: 561.71 toks/s]
Processed prompts:   6%|▌         | 1919/32000 [07:05<1:54:27,  4.38it/s, est. speed input: 3793.98 toks/s, output: 561.92 toks/s]
Processed prompts:   6%|▌         | 1922/32000 [07:05<1:28:57,  5.64it/s, est. speed input: 3795.78 toks/s, output: 562.43 toks/s]
Processed prompts:   6%|▌         | 1924/32000 [07:06<2:02:42,  4.09it/s, est. speed input: 3792.99 toks/s, output: 561.84 toks/s]
Processed prompts:   6%|▌         | 1927/32000 [07:07<1:46:25,  4.71it/s, est. speed input: 3792.99 toks/s, output: 562.00 toks/s]
Processed prompts:   6%|▌         | 1928/32000 [07:07<2:01:34,  4.12it/s, est. speed input: 3794.37 toks/s, output: 562.06 toks/s]
Processed prompts:   6%|▌         | 1929/32000 [07:07<2:06:09,  3.97it/s, est. speed input: 3793.40 toks/s, output: 562.05 toks/s]
Processed prompts:   6%|▌         | 1931/32000 [07:08<1:40:26,  4.99it/s, est. speed input: 3795.71 toks/s, output: 562.66 toks/s]
Processed prompts:   6%|▌         | 1934/32000 [07:09<2:10:34,  3.84it/s, est. speed input: 3792.74 toks/s, output: 562.41 toks/s]
Processed prompts:   6%|▌         | 1938/32000 [07:09<1:37:40,  5.13it/s, est. speed input: 3794.47 toks/s, output: 562.81 toks/s]
Processed prompts:   6%|▌         | 1939/32000 [07:09<1:40:04,  5.01it/s, est. speed input: 3795.16 toks/s, output: 562.79 toks/s]
Processed prompts:   6%|▌         | 1941/32000 [07:10<1:34:11,  5.32it/s, est. speed input: 3794.46 toks/s, output: 562.81 toks/s]
Processed prompts:   6%|▌         | 1942/32000 [07:10<2:20:32,  3.56it/s, est. speed input: 3789.10 toks/s, output: 562.13 toks/s]
Processed prompts:   6%|▌         | 1943/32000 [07:11<2:20:06,  3.58it/s, est. speed input: 3787.82 toks/s, output: 562.09 toks/s]
Processed prompts:   6%|▌         | 1944/32000 [07:11<2:59:26,  2.79it/s, est. speed input: 3782.91 toks/s, output: 561.64 toks/s]
Processed prompts:   6%|▌         | 1945/32000 [07:11<2:29:36,  3.35it/s, est. speed input: 3785.32 toks/s, output: 561.85 toks/s]
Processed prompts:   6%|▌         | 1947/32000 [07:12<2:00:49,  4.15it/s, est. speed input: 3787.04 toks/s, output: 562.26 toks/s]
Processed prompts:   6%|▌         | 1951/32000 [07:13<2:11:12,  3.82it/s, est. speed input: 3783.47 toks/s, output: 561.79 toks/s]
Processed prompts:   6%|▌         | 1953/32000 [07:13<1:54:18,  4.38it/s, est. speed input: 3785.21 toks/s, output: 561.93 toks/s]
Processed prompts:   6%|▌         | 1955/32000 [07:13<1:38:23,  5.09it/s, est. speed input: 3787.64 toks/s, output: 562.42 toks/s]
Processed prompts:   6%|▌         | 1959/32000 [07:14<1:29:31,  5.59it/s, est. speed input: 3788.65 toks/s, output: 562.67 toks/s]
Processed prompts:   6%|▌         | 1962/32000 [07:14<1:18:49,  6.35it/s, est. speed input: 3789.63 toks/s, output: 563.07 toks/s]
Processed prompts:   6%|▌         | 1965/32000 [07:15<1:10:02,  7.15it/s, est. speed input: 3791.01 toks/s, output: 563.47 toks/s]
Processed prompts:   6%|▌         | 1968/32000 [07:15<1:25:15,  5.87it/s, est. speed input: 3789.36 toks/s, output: 563.22 toks/s]
Processed prompts:   6%|▌         | 1971/32000 [07:16<1:19:39,  6.28it/s, est. speed input: 3791.46 toks/s, output: 563.36 toks/s]
Processed prompts:   6%|▌         | 1973/32000 [07:17<1:49:15,  4.58it/s, est. speed input: 3788.26 toks/s, output: 562.94 toks/s]
Processed prompts:   6%|▌         | 1975/32000 [07:17<1:48:29,  4.61it/s, est. speed input: 3788.17 toks/s, output: 562.96 toks/s]
Processed prompts:   6%|▌         | 1978/32000 [07:18<1:47:46,  4.64it/s, est. speed input: 3792.57 toks/s, output: 563.14 toks/s]
Processed prompts:   6%|▌         | 1979/32000 [07:18<2:34:15,  3.24it/s, est. speed input: 3785.98 toks/s, output: 562.32 toks/s]
Processed prompts:   6%|▌         | 1981/32000 [07:19<2:09:50,  3.85it/s, est. speed input: 3786.87 toks/s, output: 562.49 toks/s]
Processed prompts:   6%|▌         | 1983/32000 [07:19<2:01:09,  4.13it/s, est. speed input: 3788.14 toks/s, output: 562.62 toks/s]
Processed prompts:   6%|▌         | 1984/32000 [07:20<3:15:02,  2.56it/s, est. speed input: 3780.54 toks/s, output: 561.62 toks/s]
Processed prompts:   6%|▌         | 1989/32000 [07:20<1:38:11,  5.09it/s, est. speed input: 3782.95 toks/s, output: 562.67 toks/s]
Processed prompts:   6%|▌         | 1991/32000 [07:22<2:22:09,  3.52it/s, est. speed input: 3775.00 toks/s, output: 561.92 toks/s]
Processed prompts:   6%|▌         | 1994/32000 [07:23<2:33:59,  3.25it/s, est. speed input: 3769.96 toks/s, output: 561.45 toks/s]
Processed prompts:   6%|▌         | 1996/32000 [07:23<2:41:04,  3.10it/s, est. speed input: 3766.16 toks/s, output: 561.11 toks/s]
Processed prompts:   6%|▌         | 1997/32000 [07:24<2:30:20,  3.33it/s, est. speed input: 3765.58 toks/s, output: 561.14 toks/s]
Processed prompts:   6%|▌         | 1998/32000 [07:24<2:31:31,  3.30it/s, est. speed input: 3766.11 toks/s, output: 561.05 toks/s]
Processed prompts:   6%|▋         | 2001/32000 [07:25<2:24:27,  3.46it/s, est. speed input: 3765.57 toks/s, output: 560.95 toks/s]
Processed prompts:   6%|▋         | 2004/32000 [07:25<1:43:32,  4.83it/s, est. speed input: 3771.28 toks/s, output: 561.57 toks/s]
Processed prompts:   6%|▋         | 2005/32000 [07:25<1:59:39,  4.18it/s, est. speed input: 3771.88 toks/s, output: 561.62 toks/s]
Processed prompts:   6%|▋         | 2008/32000 [07:26<1:35:23,  5.24it/s, est. speed input: 3772.84 toks/s, output: 562.01 toks/s]
Processed prompts:   6%|▋         | 2010/32000 [07:26<1:56:18,  4.30it/s, est. speed input: 3771.82 toks/s, output: 561.82 toks/s]
Processed prompts:   6%|▋         | 2014/32000 [07:26<1:10:49,  7.06it/s, est. speed input: 3778.49 toks/s, output: 562.62 toks/s]
Processed prompts:   6%|▋         | 2016/32000 [07:28<2:11:58,  3.79it/s, est. speed input: 3769.83 toks/s, output: 561.45 toks/s]
Processed prompts:   6%|▋         | 2021/32000 [07:28<1:44:17,  4.79it/s, est. speed input: 3778.35 toks/s, output: 561.67 toks/s]
Processed prompts:   6%|▋         | 2023/32000 [07:29<1:40:45,  4.96it/s, est. speed input: 3777.17 toks/s, output: 561.77 toks/s]
Processed prompts:   6%|▋         | 2024/32000 [07:30<2:14:33,  3.71it/s, est. speed input: 3773.51 toks/s, output: 561.06 toks/s]
Processed prompts:   6%|▋         | 2025/32000 [07:30<2:11:53,  3.79it/s, est. speed input: 3772.93 toks/s, output: 561.03 toks/s]
Processed prompts:   6%|▋         | 2027/32000 [07:30<2:05:23,  3.98it/s, est. speed input: 3775.80 toks/s, output: 560.96 toks/s]
Processed prompts:   6%|▋         | 2030/32000 [07:31<1:48:00,  4.62it/s, est. speed input: 3775.07 toks/s, output: 561.04 toks/s]
Processed prompts:   6%|▋         | 2032/32000 [07:31<1:34:28,  5.29it/s, est. speed input: 3777.21 toks/s, output: 561.46 toks/s]
Processed prompts:   6%|▋         | 2038/32000 [07:33<1:59:46,  4.17it/s, est. speed input: 3778.41 toks/s, output: 560.71 toks/s]
Processed prompts:   6%|▋         | 2039/32000 [07:33<2:02:32,  4.07it/s, est. speed input: 3781.34 toks/s, output: 560.91 toks/s]
Processed prompts:   6%|▋         | 2040/32000 [07:34<3:22:48,  2.46it/s, est. speed input: 3772.17 toks/s, output: 559.52 toks/s]
Processed prompts:   6%|▋         | 2045/32000 [07:36<2:42:03,  3.08it/s, est. speed input: 3769.53 toks/s, output: 559.14 toks/s]
Processed prompts:   6%|▋         | 2048/32000 [07:36<2:07:19,  3.92it/s, est. speed input: 3771.52 toks/s, output: 559.63 toks/s]
Processed prompts:   6%|▋         | 2050/32000 [07:37<2:22:00,  3.52it/s, est. speed input: 3767.46 toks/s, output: 559.09 toks/s]
Processed prompts:   6%|▋         | 2053/32000 [07:37<2:10:19,  3.83it/s, est. speed input: 3769.12 toks/s, output: 559.47 toks/s]
Processed prompts:   6%|▋         | 2057/32000 [07:37<1:32:07,  5.42it/s, est. speed input: 3773.26 toks/s, output: 560.29 toks/s]
Processed prompts:   6%|▋         | 2058/32000 [07:38<1:58:06,  4.23it/s, est. speed input: 3770.48 toks/s, output: 559.82 toks/s]
Processed prompts:   6%|▋         | 2060/32000 [07:38<1:45:23,  4.73it/s, est. speed input: 3778.43 toks/s, output: 560.07 toks/s]
Processed prompts:   6%|▋         | 2061/32000 [07:39<1:54:56,  4.34it/s, est. speed input: 3778.58 toks/s, output: 560.22 toks/s]
Processed prompts:   6%|▋         | 2064/32000 [07:40<2:07:03,  3.93it/s, est. speed input: 3773.22 toks/s, output: 559.70 toks/s]
Processed prompts:   6%|▋         | 2065/32000 [07:40<2:02:02,  4.09it/s, est. speed input: 3774.16 toks/s, output: 559.98 toks/s]
Processed prompts:   6%|▋         | 2067/32000 [07:41<2:57:56,  2.80it/s, est. speed input: 3768.06 toks/s, output: 559.35 toks/s]
Processed prompts:   6%|▋         | 2069/32000 [07:42<2:47:19,  2.98it/s, est. speed input: 3766.37 toks/s, output: 559.09 toks/s]
Processed prompts:   6%|▋         | 2070/32000 [07:42<2:42:38,  3.07it/s, est. speed input: 3766.14 toks/s, output: 559.10 toks/s]
Processed prompts:   6%|▋         | 2071/32000 [07:42<2:19:42,  3.57it/s, est. speed input: 3765.99 toks/s, output: 559.13 toks/s]
Processed prompts:   6%|▋         | 2072/32000 [07:42<2:23:36,  3.47it/s, est. speed input: 3764.45 toks/s, output: 558.99 toks/s]
Processed prompts:   6%|▋         | 2073/32000 [07:42<2:12:06,  3.78it/s, est. speed input: 3766.02 toks/s, output: 559.20 toks/s]
Processed prompts:   6%|▋         | 2076/32000 [07:43<1:58:01,  4.23it/s, est. speed input: 3762.74 toks/s, output: 559.34 toks/s]
Processed prompts:   6%|▋         | 2077/32000 [07:44<2:14:51,  3.70it/s, est. speed input: 3760.22 toks/s, output: 559.10 toks/s]
Processed prompts:   7%|▋         | 2083/32000 [07:45<1:45:28,  4.73it/s, est. speed input: 3765.18 toks/s, output: 559.59 toks/s]
Processed prompts:   7%|▋         | 2085/32000 [07:45<1:39:06,  5.03it/s, est. speed input: 3766.38 toks/s, output: 559.78 toks/s]
Processed prompts:   7%|▋         | 2087/32000 [07:45<1:44:14,  4.78it/s, est. speed input: 3770.60 toks/s, output: 559.54 toks/s]
Processed prompts:   7%|▋         | 2088/32000 [07:47<3:10:32,  2.62it/s, est. speed input: 3762.56 toks/s, output: 558.30 toks/s]
Processed prompts:   7%|▋         | 2091/32000 [07:47<2:08:55,  3.87it/s, est. speed input: 3766.94 toks/s, output: 559.00 toks/s]
Processed prompts:   7%|▋         | 2092/32000 [07:47<2:23:03,  3.48it/s, est. speed input: 3770.01 toks/s, output: 558.67 toks/s]
Processed prompts:   7%|▋         | 2094/32000 [07:48<3:03:31,  2.72it/s, est. speed input: 3764.76 toks/s, output: 557.79 toks/s]
Processed prompts:   7%|▋         | 2096/32000 [07:49<2:39:04,  3.13it/s, est. speed input: 3770.00 toks/s, output: 557.78 toks/s]
Processed prompts:   7%|▋         | 2097/32000 [07:49<2:24:26,  3.45it/s, est. speed input: 3771.75 toks/s, output: 557.95 toks/s]
Processed prompts:   7%|▋         | 2098/32000 [07:49<2:15:10,  3.69it/s, est. speed input: 3770.94 toks/s, output: 558.16 toks/s]
Processed prompts:   7%|▋         | 2101/32000 [07:50<1:43:52,  4.80it/s, est. speed input: 3771.82 toks/s, output: 558.48 toks/s]
Processed prompts:   7%|▋         | 2103/32000 [07:50<1:38:12,  5.07it/s, est. speed input: 3773.50 toks/s, output: 558.71 toks/s]
Processed prompts:   7%|▋         | 2104/32000 [07:50<1:57:37,  4.24it/s, est. speed input: 3771.26 toks/s, output: 558.53 toks/s]
Processed prompts:   7%|▋         | 2106/32000 [07:51<1:47:19,  4.64it/s, est. speed input: 3775.07 toks/s, output: 558.82 toks/s]
Processed prompts:   7%|▋         | 2108/32000 [07:51<1:45:18,  4.73it/s, est. speed input: 3774.24 toks/s, output: 559.16 toks/s]
Processed prompts:   7%|▋         | 2109/32000 [07:52<2:06:54,  3.93it/s, est. speed input: 3771.27 toks/s, output: 558.95 toks/s]
Processed prompts:   7%|▋         | 2113/32000 [07:52<1:14:47,  6.66it/s, est. speed input: 3774.30 toks/s, output: 559.88 toks/s]
Processed prompts:   7%|▋         | 2114/32000 [07:52<1:27:19,  5.70it/s, est. speed input: 3772.49 toks/s, output: 559.89 toks/s]
Processed prompts:   7%|▋         | 2117/32000 [07:52<1:09:59,  7.12it/s, est. speed input: 3778.46 toks/s, output: 560.26 toks/s]
Processed prompts:   7%|▋         | 2119/32000 [07:53<1:25:02,  5.86it/s, est. speed input: 3776.73 toks/s, output: 560.31 toks/s]
Processed prompts:   7%|▋         | 2121/32000 [07:53<1:35:08,  5.23it/s, est. speed input: 3776.23 toks/s, output: 560.50 toks/s]
Processed prompts:   7%|▋         | 2127/32000 [07:54<1:07:27,  7.38it/s, est. speed input: 3785.43 toks/s, output: 561.56 toks/s]
Processed prompts:   7%|▋         | 2128/32000 [07:55<2:09:02,  3.86it/s, est. speed input: 3776.69 toks/s, output: 560.45 toks/s]
Processed prompts:   7%|▋         | 2129/32000 [07:56<2:19:54,  3.56it/s, est. speed input: 3774.30 toks/s, output: 560.18 toks/s]
Processed prompts:   7%|▋         | 2132/32000 [07:57<2:40:57,  3.09it/s, est. speed input: 3768.67 toks/s, output: 559.59 toks/s]
Processed prompts:   7%|▋         | 2133/32000 [07:57<3:03:44,  2.71it/s, est. speed input: 3768.47 toks/s, output: 559.40 toks/s]
Processed prompts:   7%|▋         | 2136/32000 [07:58<2:14:20,  3.71it/s, est. speed input: 3769.49 toks/s, output: 559.66 toks/s]
Processed prompts:   7%|▋         | 2138/32000 [07:58<2:05:27,  3.97it/s, est. speed input: 3774.88 toks/s, output: 559.90 toks/s]
Processed prompts:   7%|▋         | 2140/32000 [07:58<1:54:43,  4.34it/s, est. speed input: 3779.77 toks/s, output: 559.90 toks/s]
Processed prompts:   7%|▋         | 2142/32000 [07:59<1:55:23,  4.31it/s, est. speed input: 3780.51 toks/s, output: 560.03 toks/s]
Processed prompts:   7%|▋         | 2145/32000 [08:00<2:03:03,  4.04it/s, est. speed input: 3777.77 toks/s, output: 559.95 toks/s]
Processed prompts:   7%|▋         | 2147/32000 [08:00<1:45:51,  4.70it/s, est. speed input: 3780.93 toks/s, output: 560.21 toks/s]
Processed prompts:   7%|▋         | 2148/32000 [08:00<1:58:24,  4.20it/s, est. speed input: 3778.98 toks/s, output: 559.91 toks/s]
Processed prompts:   7%|▋         | 2149/32000 [08:01<2:28:23,  3.35it/s, est. speed input: 3775.55 toks/s, output: 559.50 toks/s]
Processed prompts:   7%|▋         | 2152/32000 [08:02<2:22:30,  3.49it/s, est. speed input: 3774.09 toks/s, output: 559.20 toks/s]
Processed prompts:   7%|▋         | 2154/32000 [08:02<1:53:24,  4.39it/s, est. speed input: 3775.01 toks/s, output: 559.62 toks/s]
Processed prompts:   7%|▋         | 2157/32000 [08:02<1:35:32,  5.21it/s, est. speed input: 3775.10 toks/s, output: 559.96 toks/s]
Processed prompts:   7%|▋         | 2158/32000 [08:03<2:38:59,  3.13it/s, est. speed input: 3769.53 toks/s, output: 559.11 toks/s]
Processed prompts:   7%|▋         | 2160/32000 [08:04<2:06:52,  3.92it/s, est. speed input: 3771.99 toks/s, output: 559.37 toks/s]
Processed prompts:   7%|▋         | 2164/32000 [08:04<1:33:08,  5.34it/s, est. speed input: 3777.45 toks/s, output: 559.69 toks/s]
Processed prompts:   7%|▋         | 2166/32000 [08:05<1:44:49,  4.74it/s, est. speed input: 3775.17 toks/s, output: 559.66 toks/s]
Processed prompts:   7%|▋         | 2168/32000 [08:05<1:36:00,  5.18it/s, est. speed input: 3778.29 toks/s, output: 559.80 toks/s]
Processed prompts:   7%|▋         | 2170/32000 [08:06<2:26:18,  3.40it/s, est. speed input: 3779.05 toks/s, output: 559.25 toks/s]
Processed prompts:   7%|▋         | 2174/32000 [08:06<1:32:43,  5.36it/s, est. speed input: 3786.74 toks/s, output: 559.97 toks/s]
Processed prompts:   7%|▋         | 2176/32000 [08:07<1:42:23,  4.85it/s, est. speed input: 3787.63 toks/s, output: 560.19 toks/s]
Processed prompts:   7%|▋         | 2181/32000 [08:07<1:07:29,  7.36it/s, est. speed input: 3795.57 toks/s, output: 561.07 toks/s]
Processed prompts:   7%|▋         | 2184/32000 [08:08<1:35:25,  5.21it/s, est. speed input: 3792.92 toks/s, output: 560.92 toks/s]
Processed prompts:   7%|▋         | 2188/32000 [08:08<1:20:43,  6.16it/s, est. speed input: 3795.77 toks/s, output: 561.56 toks/s]
Processed prompts:   7%|▋         | 2193/32000 [08:10<1:58:26,  4.19it/s, est. speed input: 3792.67 toks/s, output: 561.01 toks/s]
Processed prompts:   7%|▋         | 2195/32000 [08:11<2:23:33,  3.46it/s, est. speed input: 3787.75 toks/s, output: 560.54 toks/s]
Processed prompts:   7%|▋         | 2196/32000 [08:11<2:19:23,  3.56it/s, est. speed input: 3789.00 toks/s, output: 560.57 toks/s]
Processed prompts:   7%|▋         | 2198/32000 [08:12<1:56:54,  4.25it/s, est. speed input: 3789.10 toks/s, output: 560.79 toks/s]
Processed prompts:   7%|▋         | 2200/32000 [08:12<2:03:56,  4.01it/s, est. speed input: 3787.37 toks/s, output: 560.54 toks/s]
Processed prompts:   7%|▋         | 2202/32000 [08:12<1:49:13,  4.55it/s, est. speed input: 3787.01 toks/s, output: 560.74 toks/s]
Processed prompts:   7%|▋         | 2203/32000 [08:13<1:54:58,  4.32it/s, est. speed input: 3787.11 toks/s, output: 560.61 toks/s]
Processed prompts:   7%|▋         | 2204/32000 [08:14<2:41:25,  3.08it/s, est. speed input: 3784.17 toks/s, output: 559.97 toks/s]
Processed prompts:   7%|▋         | 2205/32000 [08:14<2:18:03,  3.60it/s, est. speed input: 3784.45 toks/s, output: 560.08 toks/s]
Processed prompts:   7%|▋         | 2206/32000 [08:14<2:39:45,  3.11it/s, est. speed input: 3784.50 toks/s, output: 560.00 toks/s]
Processed prompts:   7%|▋         | 2210/32000 [08:14<1:25:11,  5.83it/s, est. speed input: 3787.12 toks/s, output: 560.71 toks/s]
Processed prompts:   7%|▋         | 2211/32000 [08:15<2:42:01,  3.06it/s, est. speed input: 3780.40 toks/s, output: 559.73 toks/s]
Processed prompts:   7%|▋         | 2213/32000 [08:16<2:27:01,  3.38it/s, est. speed input: 3781.75 toks/s, output: 559.96 toks/s]
Processed prompts:   7%|▋         | 2216/32000 [08:16<1:39:53,  4.97it/s, est. speed input: 3783.88 toks/s, output: 560.61 toks/s]
Processed prompts:   7%|▋         | 2219/32000 [08:16<1:18:15,  6.34it/s, est. speed input: 3786.42 toks/s, output: 561.20 toks/s]
Processed prompts:   7%|▋         | 2221/32000 [08:17<1:49:31,  4.53it/s, est. speed input: 3781.85 toks/s, output: 560.63 toks/s]
Processed prompts:   7%|▋         | 2222/32000 [08:17<1:47:39,  4.61it/s, est. speed input: 3780.78 toks/s, output: 560.65 toks/s]
Processed prompts:   7%|▋         | 2225/32000 [08:18<1:20:08,  6.19it/s, est. speed input: 3784.18 toks/s, output: 560.93 toks/s]
Processed prompts:   7%|▋         | 2226/32000 [08:18<2:12:58,  3.73it/s, est. speed input: 3778.76 toks/s, output: 560.21 toks/s]
Processed prompts:   7%|▋         | 2227/32000 [08:19<2:07:49,  3.88it/s, est. speed input: 3778.35 toks/s, output: 560.19 toks/s]
Processed prompts:   7%|▋         | 2228/32000 [08:19<2:40:36,  3.09it/s, est. speed input: 3775.12 toks/s, output: 559.79 toks/s]
Xet Storage Details

Size:: 205 kB
Xet hash:: 69dd5c833caac8d9c9dee012c17a645bf0ef94ac711e482b7f3c56afd7e54866
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.