(APIServer pid=1126830) INFO 07-23 21:57:49 [api_utils.py:339] (APIServer pid=1126830) INFO 07-23 21:57:49 [api_utils.py:339] █ █ █▄ ▄█ (APIServer pid=1126830) INFO 07-23 21:57:49 [api_utils.py:339] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.25.0 (APIServer pid=1126830) INFO 07-23 21:57:49 [api_utils.py:339] █▄█▀ █ █ █ █ model outputs/qwen35_pruned_keep25 (APIServer pid=1126830) INFO 07-23 21:57:49 [api_utils.py:339] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀ (APIServer pid=1126830) INFO 07-23 21:57:49 [api_utils.py:339] (APIServer pid=1126830) INFO 07-23 21:57:49 [api_utils.py:273] non-default args: {'model_tag': 'outputs/qwen35_pruned_keep25', 'default_chat_template_kwargs': {'enable_thinking': False}, 'host': '127.0.0.1', 'port': 8399, 'model': 'outputs/qwen35_pruned_keep25', 'max_model_len': 2048, 'enforce_eager': True, 'served_model_name': ['student'], 'gpu_memory_utilization': 0.95} (APIServer pid=1126830) INFO 07-23 21:57:49 [model.py:619] Resolved architecture: PrunedQwen3_5MoeForCausalLM (APIServer pid=1126830) INFO 07-23 21:57:49 [model.py:1776] Using max model len 2048 (APIServer pid=1126830) INFO 07-23 21:57:49 [vllm.py:1042] Asynchronous scheduling is enabled. (APIServer pid=1126830) WARNING 07-23 21:57:49 [vllm.py:1096] Enforce eager set, disabling torch.compile and CUDAGraphs. This is equivalent to setting -cc.mode=none -cc.cudagraph_mode=none (APIServer pid=1126830) WARNING 07-23 21:57:49 [vllm.py:1144] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. (APIServer pid=1126830) INFO 07-23 21:57:49 [kernel.py:292] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native']) (APIServer pid=1126830) INFO 07-23 21:57:50 [vllm.py:1322] Cudagraph is disabled under eager mode (APIServer pid=1126830) INFO 07-23 21:57:50 [compilation.py:312] Enabled custom fusions: norm_quant, act_quant (EngineCore pid=1126963) INFO 07-23 21:58:01 [core.py:114] Initializing a V1 LLM engine (v0.25.0) with config: model='outputs/qwen35_pruned_keep25', speculative_config=None, tokenizer='outputs/qwen35_pruned_keep25', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=2048, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=student, enable_prefix_caching=False, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'ir_enable_torch_wrap': False, 'splitting_ops': [], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': True, 'fuse_act_quant': True, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native']), enable_flashinfer_autotune=True, enable_cutedsl_warmup=True, moe_backend='auto', linear_backend='auto') (EngineCore pid=1126963) INFO 07-23 21:58:03 [parallel_state.py:1607] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://192.168.0.15:44175 backend=nccl (EngineCore pid=1126963) INFO 07-23 21:58:03 [parallel_state.py:1942] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0, EPLB rank N/A (EngineCore pid=1126963) INFO 07-23 21:58:03 [topk_topp_sampler.py:55] Using FlashInfer for top-p & top-k sampling. (EngineCore pid=1126963) INFO 07-23 21:58:04 [gpu_model_runner.py:5209] Starting to load model outputs/qwen35_pruned_keep25... (EngineCore pid=1126963) INFO 07-23 21:58:04 [qwen_gdn_linear_attn.py:228] Using Triton/FLA GDN prefill kernel (requested=auto, head_k_dim=128). (EngineCore pid=1126963) /home/henry/.cache/glean/megablocks-variable-93a1479bc15b/megablocks/grouped_gemm_util.py:10: UserWarning: Grouped GEMM not available. (EngineCore pid=1126963) warnings.warn('Grouped GEMM not available.') (EngineCore pid=1126963) INFO 07-23 21:58:04 [cuda.py:476] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. (EngineCore pid=1126963) INFO 07-23 21:58:04 [flash_attn.py:718] Using FlashAttention version 2 (EngineCore pid=1126963) INFO 07-23 21:58:04 [weight_utils.py:849] Filesystem type for checkpoints: EXT4. Checkpoint size: 19.54 GiB. Available RAM: 108.90 GiB. (EngineCore pid=1126963) INFO 07-23 21:58:04 [weight_utils.py:872] Auto-prefetch is disabled because the filesystem (EXT4) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch. (EngineCore pid=1126963) Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:00= mamba page size. (EngineCore pid=1126963) INFO 07-23 21:58:18 [interface.py:914] Padding mamba page size by 1.49% to ensure that mamba page size and attention page size are exactly equal. (EngineCore pid=1126963) INFO 07-23 21:58:20 [gpu_worker.py:538] Available KV cache memory: 1.71 GiB (EngineCore pid=1126963) INFO 07-23 21:58:20 [kv_cache_utils.py:2146] GPU KV cache size: 48,274 tokens (EngineCore pid=1126963) INFO 07-23 21:58:20 [kv_cache_utils.py:2147] Maximum concurrency for 2,048 tokens per request: 23.57x (EngineCore pid=1126963) INFO 07-23 21:58:20 [cutedsl_warmup.py:97] Skipping CuTeDSL warmup because no compile units were requested. (EngineCore pid=1126963) INFO 07-23 21:58:20 [jit_monitor.py:73] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn. (EngineCore pid=1126963) INFO 07-23 21:58:20 [core.py:344] init engine (profile, create kv cache, warmup model) took 2.62 s (EngineCore pid=1126963) INFO 07-23 21:58:22 [vllm.py:1042] Asynchronous scheduling is enabled. (EngineCore pid=1126963) WARNING 07-23 21:58:22 [vllm.py:1096] Enforce eager set, disabling torch.compile and CUDAGraphs. This is equivalent to setting -cc.mode=none -cc.cudagraph_mode=none (EngineCore pid=1126963) WARNING 07-23 21:58:22 [vllm.py:1144] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. (EngineCore pid=1126963) INFO 07-23 21:58:22 [kernel.py:292] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native']) (EngineCore pid=1126963) INFO 07-23 21:58:22 [vllm.py:1322] Cudagraph is disabled under eager mode (EngineCore pid=1126963) INFO 07-23 21:58:22 [compilation.py:312] Enabled custom fusions: norm_quant, act_quant (APIServer pid=1126830) INFO 07-23 21:58:22 [api_server.py:612] Supported tasks: ['generate'] (APIServer pid=1126830) INFO 07-23 21:58:24 [hf.py:548] Detected the chat template content format to be 'openai'. You can set `--chat-template-content-format` to override this. (APIServer pid=1126830) INFO 07-23 21:58:24 [api_server.py:616] Starting vLLM server on http://127.0.0.1:8399 (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:37] Available routes are: (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /openapi.json, Methods: GET, HEAD (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /docs, Methods: GET, HEAD (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /docs/oauth2-redirect, Methods: GET, HEAD (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /redoc, Methods: GET, HEAD (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /load, Methods: GET (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /version, Methods: GET (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /health, Methods: GET (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /metrics, Methods: GET (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /tokenize, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /detokenize, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/models, Methods: GET (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /ping, Methods: GET (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /ping, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /invocations, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/chat/completions, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/chat/completions/batch, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/responses, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/responses/{response_id}, Methods: GET (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/responses/{response_id}/cancel, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/completions, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/messages, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/messages/count_tokens, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /generative_scoring, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /scale_elastic_ep, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /is_scaling_elastic_ep, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/chat/completions/render, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/completions/render, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/chat/completions/derender, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /v1/completions/derender, Methods: POST (APIServer pid=1126830) INFO 07-23 21:58:24 [launcher.py:46] Route: /inference/v1/generate, Methods: POST (APIServer pid=1126830) INFO: Started server process [1126830] (APIServer pid=1126830) INFO: Waiting for application startup. (APIServer pid=1126830) INFO: Application startup complete. (APIServer pid=1126830) INFO: 127.0.0.1:34316 - "GET /health HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:56878 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56882 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56884 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56890 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56906 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56914 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56922 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56938 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56948 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56950 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56956 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56968 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56980 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56994 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56998 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57006 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57018 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57022 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57030 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57046 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57050 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57064 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57076 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57084 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57094 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57104 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57116 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57132 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57134 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57138 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57150 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57160 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56878 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56882 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56884 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56890 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56906 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56914 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56922 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56938 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56948 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56950 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56956 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56968 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56980 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56994 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56998 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57006 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57018 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57022 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57030 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57046 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57050 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57064 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57076 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57084 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57094 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57104 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57116 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57132 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57134 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57138 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57150 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57160 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56878 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56882 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56884 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56890 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56906 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56914 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56922 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56938 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56948 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56950 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56956 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56968 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56980 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56994 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56998 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57006 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57018 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57022 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57030 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57046 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57050 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57064 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57076 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57084 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57094 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57104 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57116 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57132 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57134 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57138 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57150 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57160 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56878 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56882 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56884 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56890 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56906 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56914 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56922 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56938 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56948 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56950 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56956 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56968 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56980 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56994 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56998 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57006 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57018 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57022 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57030 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57046 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57050 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57064 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57076 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57084 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57094 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57104 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57116 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57132 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57134 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57138 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57150 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57160 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56878 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56882 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56884 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56890 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56906 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56914 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56922 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56938 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56948 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56950 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56956 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56968 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56980 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56994 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56998 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57006 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57018 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57022 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57030 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57046 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57050 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57064 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57076 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57084 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57094 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57104 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57116 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57132 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57134 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57138 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57150 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57160 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56878 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56882 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56884 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56890 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56906 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56914 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56922 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56938 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56948 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56950 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56956 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56968 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56980 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56994 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56998 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57006 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57018 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57022 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57030 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57046 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57050 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57064 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57076 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57084 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57094 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57104 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57116 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57132 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57134 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57138 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57150 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57160 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56878 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56882 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56884 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56890 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56906 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56914 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56922 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56938 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56948 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56950 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56956 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56968 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56980 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56994 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56998 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57006 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57018 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57022 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57030 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57046 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57050 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57064 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57076 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57084 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57094 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57104 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57116 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57132 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57134 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57138 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57150 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57160 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56878 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56882 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56884 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56890 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56906 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56914 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56922 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56938 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56948 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56950 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56956 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56968 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56980 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56994 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56998 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57006 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57018 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57022 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57030 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57046 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57050 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57064 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57076 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57084 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57094 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57104 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57116 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57132 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57134 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57138 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57150 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57160 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56878 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56882 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56884 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56890 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56906 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56914 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56922 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56938 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56948 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56950 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56956 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56968 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56980 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56994 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:56998 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57006 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57018 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57022 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57030 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57046 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57050 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57064 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57076 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57084 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:57094 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request (APIServer pid=1126830) INFO: 127.0.0.1:39342 - "GET /tokenizer_info HTTP/1.1" 404 Not Found (APIServer pid=1126830) INFO: 127.0.0.1:39342 - "GET /tokenizer_info HTTP/1.1" 404 Not Found (APIServer pid=1126830) INFO: 127.0.0.1:39342 - "GET /tokenizer_info HTTP/1.1" 404 Not Found (EngineCore pid=1126963) WARNING 07-23 21:59:17 [jit_monitor.py:129] Triton kernel JIT compilation during inference: _zero_kv_blocks_kernel. This causes a latency spike; consider extending warmup to cover this shape/config. (EngineCore pid=1126963) WARNING 07-23 21:59:17 [jit_monitor.py:129] Triton kernel JIT compilation during inference: _compute_slot_mapping_kernel. This causes a latency spike; consider extending warmup to cover this shape/config. (EngineCore pid=1126963) WARNING 07-23 21:59:17 [jit_monitor.py:129] Triton kernel JIT compilation during inference: _causal_conv1d_fwd_kernel. This causes a latency spike; consider extending warmup to cover this shape/config. (EngineCore pid=1126963) WARNING 07-23 21:59:17 [jit_monitor.py:129] Triton kernel JIT compilation during inference: _fused_post_conv_kernel. This causes a latency spike; consider extending warmup to cover this shape/config. (EngineCore pid=1126963) WARNING 07-23 21:59:17 [jit_monitor.py:129] Triton kernel JIT compilation during inference: _build_route_rows. This causes a latency spike; consider extending warmup to cover this shape/config. (EngineCore pid=1126963) WARNING 07-23 21:59:17 [jit_monitor.py:129] Triton kernel JIT compilation during inference: fused_sigmoid_gating_delta_rule_update_kernel. This causes a latency spike; consider extending warmup to cover this shape/config. (EngineCore pid=1126963) WARNING 07-23 21:59:17 [jit_monitor.py:129] Triton kernel JIT compilation during inference: _causal_conv1d_update_kernel. This causes a latency spike; consider extending warmup to cover this shape/config. (EngineCore pid=1126963) WARNING 07-23 21:59:17 [jit_monitor.py:129] Triton kernel JIT compilation during inference: fused_recurrent_gated_delta_rule_packed_decode_kernel. This causes a latency spike; consider extending warmup to cover this shape/config. (EngineCore pid=1126963) WARNING 07-23 21:59:17 [jit_monitor.py:129] Triton kernel JIT compilation during inference: layer_norm_fwd_kernel. This causes a latency spike; consider extending warmup to cover this shape/config. (APIServer pid=1126830) INFO 07-23 21:59:24 [loggers.py:273] Engine 000: Avg prompt throughput: 107.1 tokens/s, Avg generation throughput: 74.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.4%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO 07-23 21:59:34 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 112.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.4%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO 07-23 21:59:44 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 113.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.4%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO 07-23 21:59:54 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 112.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.4%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO 07-23 22:00:04 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 112.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.5%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO 07-23 22:00:14 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 112.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.5%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO 07-23 22:00:24 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 112.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.5%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO 07-23 22:00:34 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 113.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.5%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO 07-23 22:00:44 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 112.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.3%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO: 127.0.0.1:49340 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:49354 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:49364 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:49372 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:49382 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:49392 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:49398 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:49400 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:49402 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:49406 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:55546 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:55544 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:55564 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:55536 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO 07-23 22:00:54 [loggers.py:273] Engine 000: Avg prompt throughput: 692.4 tokens/s, Avg generation throughput: 92.8 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 18.3%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO: 127.0.0.1:55572 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:55576 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:55578 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:55594 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO: 127.0.0.1:55558 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO 07-23 22:01:04 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 14.3 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.0%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO: 127.0.0.1:55598 - "POST /v1/completions HTTP/1.1" 200 OK (APIServer pid=1126830) INFO 07-23 22:01:14 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 9.1 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% (APIServer pid=1126830) INFO 07-23 22:01:18 [launcher.py:100] [shutdown] API server: shutdown triggered (APIServer pid=1126830) INFO 07-23 22:01:18 [launcher.py:116] [shutdown] API server: stopping engine client mode=abort timeout=0s (APIServer pid=1126830) INFO 07-23 22:01:18 [core_client.py:655] [shutdown] MPClient: start timeout=0s (APIServer pid=1126830) INFO 07-23 22:01:18 [core_client.py:657] [shutdown] MPClient: stopping engine manager (APIServer pid=1126830) WARNING 07-23 22:01:18 [utils.py:626] [shutdown] Process manager: force killing remaining processes count=1 (EngineCore pid=1126963) INFO 07-23 22:01:18 [core.py:1214] [shutdown] EngineCore: trigger received signal=SIGTERM (EngineCore pid=1126963) INFO 07-23 22:01:18 [core.py:1333] [shutdown] EngineCore: start mode=abort timeout=0s (EngineCore pid=1126963) INFO 07-23 22:01:18 [core.py:1364] [shutdown] EngineCore: request processing complete; starting resource teardown (EngineCore pid=1126963) INFO 07-23 22:01:18 [core.py:1227] [shutdown] EngineCore: exiting busy loop (APIServer pid=1126830) INFO: Shutting down (APIServer pid=1126830) INFO 07-23 22:01:18 [core_client.py:659] [shutdown] MPClient: engine manager stopped (APIServer pid=1126830) INFO 07-23 22:01:18 [core_client.py:660] [shutdown] MPClient: cleaning up background resources (APIServer pid=1126830) INFO 07-23 22:01:18 [core_client.py:662] [shutdown] MPClient: complete (APIServer pid=1126830) INFO 07-23 22:01:18 [launcher.py:125] [shutdown] API server: engine client stopped (APIServer pid=1126830) INFO 07-23 22:01:18 [launcher.py:128] [shutdown] API server: signalling HTTP server shutdown (APIServer pid=1126830) INFO 07-23 22:01:18 [launcher.py:149] [shutdown] API server: shutting down FastAPI HTTP server (APIServer pid=1126830) INFO: Shutting down (APIServer pid=1126830) INFO: Waiting for application shutdown. (APIServer pid=1126830) INFO: Application shutdown complete. /home/henry/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/multiprocessing/resource_tracker.py:279: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '