INFO 09-18 17:24:35 [__init__.py:241] Automatically detected platform cuda. (APIServer pid=3603498) INFO 09-18 17:24:38 [api_server.py:1805] vLLM API server version 0.10.1.1 (APIServer pid=3603498) INFO 09-18 17:24:38 [utils.py:326] non-default args: {'model_tag': '/data/wyt/codes/DocDPO/sft/checkpoints_multilang/ted_base_balanced_en_zhdefr_320/dpo/merged/checkpoint-1000', 'host': '0.0.0.0', 'port': 8013, 'model': '/data/wyt/codes/DocDPO/sft/checkpoints_multilang/ted_base_balanced_en_zhdefr_320/dpo/merged/checkpoint-1000', 'served_model_name': ['qwen'], 'enable_prefix_caching': True} (APIServer pid=3603498) INFO 09-18 17:24:45 [__init__.py:711] Resolved architecture: Qwen2ForCausalLM (APIServer pid=3603498) `torch_dtype` is deprecated! Use `dtype` instead! (APIServer pid=3603498) INFO 09-18 17:24:45 [__init__.py:1750] Using max model len 32768 (APIServer pid=3603498) INFO 09-18 17:24:45 [scheduler.py:222] Chunked prefill is enabled with max_num_batched_tokens=8192. INFO 09-18 17:24:49 [__init__.py:241] Automatically detected platform cuda. (EngineCore_0 pid=3604733) INFO 09-18 17:24:51 [core.py:636] Waiting for init message from front-end. (EngineCore_0 pid=3604733) INFO 09-18 17:24:51 [core.py:74] Initializing a V1 LLM engine (v0.10.1.1) with config: model='/data/wyt/codes/DocDPO/sft/checkpoints_multilang/ted_base_balanced_en_zhdefr_320/dpo/merged/checkpoint-1000', speculative_config=None, tokenizer='/data/wyt/codes/DocDPO/sft/checkpoints_multilang/ted_base_balanced_en_zhdefr_320/dpo/merged/checkpoint-1000', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=qwen, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output","vllm.mamba_mixer2"],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"cudagraph_mode":1,"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"pass_config":{},"max_capture_size":512,"local_cache_dir":null} (EngineCore_0 pid=3604733) INFO 09-18 17:24:53 [parallel_state.py:1134] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0 (EngineCore_0 pid=3604733) WARNING 09-18 17:24:53 [topk_topp_sampler.py:61] FlashInfer is not available. Falling back to the PyTorch-native implementation of top-p & top-k sampling. For the best performance, please install FlashInfer. (EngineCore_0 pid=3604733) INFO 09-18 17:24:53 [gpu_model_runner.py:1953] Starting to load model /data/wyt/codes/DocDPO/sft/checkpoints_multilang/ted_base_balanced_en_zhdefr_320/dpo/merged/checkpoint-1000... (EngineCore_0 pid=3604733) INFO 09-18 17:24:53 [gpu_model_runner.py:1985] Loading model from scratch... (EngineCore_0 pid=3604733) INFO 09-18 17:24:53 [cuda.py:328] Using Flash Attention backend on V1 engine. (EngineCore_0 pid=3604733) Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00