Hugging Face's logo

Buckets:
CherithCutestory
/
media-analyzer-storage

Files
xet
CherithCutestory/media-analyzer-storage / logs /space.log
CherithCutestory's picture
CherithCutestory
download
raw
29.6 kB
boot 2026-07-14T22:45:25Z — logging to /model-storage/logs/space.log (previous boot in space.log.1)
LD_LIBRARY_PATH=/usr/local/lib/python3.12/dist-packages/nvidia/cu13/lib:/usr/local/lib/python3.12/dist-packages/nvidia/cudnn/lib:/usr/local/lib/python3.12/dist-packages/nvidia/cusparselt/lib:/usr/local/lib/python3.12/dist-packages/nvidia/nccl/lib:/usr/local/lib/python3.12/dist-packages/nvidia/nvshmem/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64
restoring HF cache from /model-storage/huggingface -> /app/.hf-cache
starting vLLM (fancyfeast/llama-joycaption-beta-one-hf-llava) on :8001
starting gateway on :7860
INFO: Started server process [1]
INFO: Waiting for application startup.
2026-07-14 22:47:59.330507176 [W:onnxruntime:Default, device_discovery.cc:283 GetGpuDevices] Failed to detect devices under "/sys/class/drm/card0": device_discovery.cc:93 ReadFileContents Failed to open file: "/sys/class/drm/card0/device/vendor"
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:7860 (Press CTRL+C to quit)
2026-07-14 22:47:59,576 gateway INFO onnxruntime available providers: ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'sdpa_kernel': '0', 'use_tf32': '1', 'fuse_conv_bias': '0', 'prefer_nhwc': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_enable': '0', 'use_ep_level_unified_stream': '0', 'device_id': '0', 'has_user_compute_stream': '0', 'gpu_external_empty_cache': '0', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_alloc': '0', 'gpu_external_free': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'enable_cuda_graph': '0', 'user_compute_stream': '0', 'cudnn_conv_use_max_workspace': '1'}}
model ignore: /opt/insightface/models/buffalo_l/1k3d68.onnx landmark_3d_68
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'sdpa_kernel': '0', 'use_tf32': '1', 'fuse_conv_bias': '0', 'prefer_nhwc': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_enable': '0', 'use_ep_level_unified_stream': '0', 'device_id': '0', 'has_user_compute_stream': '0', 'gpu_external_empty_cache': '0', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_alloc': '0', 'gpu_external_free': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'enable_cuda_graph': '0', 'user_compute_stream': '0', 'cudnn_conv_use_max_workspace': '1'}}
model ignore: /opt/insightface/models/buffalo_l/2d106det.onnx landmark_2d_106
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'sdpa_kernel': '0', 'use_tf32': '1', 'fuse_conv_bias': '0', 'prefer_nhwc': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_enable': '0', 'use_ep_level_unified_stream': '0', 'device_id': '0', 'has_user_compute_stream': '0', 'gpu_external_empty_cache': '0', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_alloc': '0', 'gpu_external_free': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'enable_cuda_graph': '0', 'user_compute_stream': '0', 'cudnn_conv_use_max_workspace': '1'}}
find model: /opt/insightface/models/buffalo_l/det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'sdpa_kernel': '0', 'use_tf32': '1', 'fuse_conv_bias': '0', 'prefer_nhwc': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_enable': '0', 'use_ep_level_unified_stream': '0', 'device_id': '0', 'has_user_compute_stream': '0', 'gpu_external_empty_cache': '0', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_alloc': '0', 'gpu_external_free': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'enable_cuda_graph': '0', 'user_compute_stream': '0', 'cudnn_conv_use_max_workspace': '1'}}
model ignore: /opt/insightface/models/buffalo_l/genderage.onnx genderage
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'sdpa_kernel': '0', 'use_tf32': '1', 'fuse_conv_bias': '0', 'prefer_nhwc': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_enable': '0', 'use_ep_level_unified_stream': '0', 'device_id': '0', 'has_user_compute_stream': '0', 'gpu_external_empty_cache': '0', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_alloc': '0', 'gpu_external_free': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'enable_cuda_graph': '0', 'user_compute_stream': '0', 'cudnn_conv_use_max_workspace': '1'}}
find model: /opt/insightface/models/buffalo_l/w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (640, 640)
2026-07-14 22:48:00,699 gateway INFO face model ready; onnxruntime providers=['CPUExecutionProvider', 'CUDAExecutionProvider']
(APIServer pid=63) INFO 07-14 22:48:16 [api_utils.py:339]
(APIServer pid=63) INFO 07-14 22:48:16 [api_utils.py:339] █ █ █▄ ▄█
(APIServer pid=63) INFO 07-14 22:48:16 [api_utils.py:339] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.25.1
(APIServer pid=63) INFO 07-14 22:48:16 [api_utils.py:339] █▄█▀ █ █ █ █ model fancyfeast/llama-joycaption-beta-one-hf-llava
(APIServer pid=63) INFO 07-14 22:48:16 [api_utils.py:339] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
(APIServer pid=63) INFO 07-14 22:48:16 [api_utils.py:339]
(APIServer pid=63) INFO 07-14 22:48:16 [api_utils.py:273] non-default args: {'model_tag': 'fancyfeast/llama-joycaption-beta-one-hf-llava', 'host': '127.0.0.1', 'port': 8001, 'model': 'fancyfeast/llama-joycaption-beta-one-hf-llava', 'max_model_len': 4096, 'served_model_name': ['joycaption-beta-one'], 'gpu_memory_utilization': 0.85, 'enable_prefix_caching': True}
(APIServer pid=63) WARNING 07-14 22:48:16 [envs.py:2041] Unknown vLLM environment variable detected: VLLM_BUILD_URL
(APIServer pid=63) WARNING 07-14 22:48:16 [envs.py:2041] Unknown vLLM environment variable detected: VLLM_ENFORCE_EAGER
(APIServer pid=63) WARNING 07-14 22:48:16 [envs.py:2041] Unknown vLLM environment variable detected: VLLM_IMAGE_TAG
(APIServer pid=63) WARNING 07-14 22:48:16 [envs.py:2041] Unknown vLLM environment variable detected: VLLM_BUILD_PIPELINE
(APIServer pid=63) WARNING 07-14 22:48:16 [envs.py:2041] Unknown vLLM environment variable detected: VLLM_BUILD_COMMIT
(APIServer pid=63) INFO 07-14 22:48:39 [model.py:619] Resolved architecture: LlavaForConditionalGeneration
(APIServer pid=63) INFO 07-14 22:48:39 [model.py:1776] Using max model len 4096
(APIServer pid=63) INFO 07-14 22:48:40 [vllm.py:1042] Asynchronous scheduling is enabled.
(APIServer pid=63) INFO 07-14 22:48:40 [kernel.py:292] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(APIServer pid=63) [transformers] You have used a torchvision backend image processor with LANCZOS resample which is not supported for torch.Tensor with torchvision < 0.27. BICUBIC resample will be used as an alternative. Please upgrade torchvision to 0.27+ or fall back to a pil backend image processor if you want full consistency with the original model.
(EngineCore pid=183) INFO 07-14 22:49:02 [core.py:114] Initializing a V1 LLM engine (v0.25.1) with config: model='fancyfeast/llama-joycaption-beta-one-hf-llava', speculative_config=None, tokenizer='fancyfeast/llama-joycaption-beta-one-hf-llava', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=joycaption-beta-one, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::qwen_gdn_attention_core', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::kda_attention', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'size_asserts': False, 'alignment_asserts': False, 'scalar_asserts': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']), enable_flashinfer_autotune=True, enable_cutedsl_warmup=True, moe_backend='auto', linear_backend='auto')
(EngineCore pid=183) INFO 07-14 22:49:02 [network_utils.py:205] Port 8001 is already in use, trying port 8002
(EngineCore pid=183) INFO 07-14 22:49:08 [parallel_state.py:1607] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.114.124.40:8002 backend=nccl
(EngineCore pid=183) INFO 07-14 22:49:08 [parallel_state.py:1942] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=183) INFO 07-14 22:49:08 [gpu_worker.py:360] Using V2 Model Runner
(EngineCore pid=183) INFO 07-14 22:49:09 [model_runner.py:281] Loading model from scratch...
(EngineCore pid=183) INFO 07-14 22:49:09 [interfaces.py:171] Contains out of vocabulary multimodal tokens? False
(EngineCore pid=183) INFO 07-14 22:49:09 [cuda.py:535] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(EngineCore pid=183) INFO 07-14 22:49:09 [mm_encoder_attention.py:373] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(EngineCore pid=183) INFO 07-14 22:49:09 [vllm.py:1042] Asynchronous scheduling is enabled.
(EngineCore pid=183) INFO 07-14 22:49:09 [kernel.py:292] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(EngineCore pid=183) INFO 07-14 22:49:09 [cuda.py:476] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=183) INFO 07-14 22:49:09 [flash_attn.py:718] Using FlashAttention version 2
(EngineCore pid=183) INFO 07-14 22:49:10 [weight_utils.py:849] Filesystem type for checkpoints: OVERLAY. Checkpoint size: 15.80 GiB. Available RAM: 54.77 GiB.
(EngineCore pid=183) INFO 07-14 22:49:10 [weight_utils.py:872] Auto-prefetch is disabled because the filesystem (OVERLAY) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.
(EngineCore pid=183) Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00<?, ?it/s]
(EngineCore pid=183) Loading safetensors checkpoint shards: 25% Completed | 1/4 [00:10<00:30, 10.10s/it]
(EngineCore pid=183) Loading safetensors checkpoint shards: 50% Completed | 2/4 [00:10<00:08, 4.42s/it]
(EngineCore pid=183) Loading safetensors checkpoint shards: 75% Completed | 3/4 [00:10<00:02, 2.61s/it]
(EngineCore pid=183) Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:11<00:00, 1.66s/it]
(EngineCore pid=183) Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:11<00:00, 2.80s/it]
(EngineCore pid=183)
(EngineCore pid=183) INFO 07-14 22:49:21 [default_loader.py:430] Loading weights took 11.21 seconds
(EngineCore pid=183) INFO 07-14 22:49:22 [model_runner.py:302] Model loading took 15.77 GiB and 13.028473 seconds
(EngineCore pid=183) INFO 07-14 22:49:22 [topk_topp_sampler.py:55] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=183) INFO 07-14 22:49:27 [backends.py:1089] Using cache directory: /home/user/.cache/vllm/torch_compile_cache/4235460569/rank_0_0/backbone for vLLM's torch.compile
(EngineCore pid=183) INFO 07-14 22:49:27 [backends.py:1148] Dynamo bytecode transform time: 5.11 s
(EngineCore pid=183) INFO 07-14 22:49:31 [backends.py:378] Cache the graph of compile range (1, 2048) for later use
(EngineCore pid=183) INFO 07-14 22:49:35 [backends.py:393] Compiling a graph for compile range (1, 2048) takes 8.22 s
(EngineCore pid=183) INFO 07-14 22:49:39 [decorators.py:708] saved AOT compiled function to /home/user/.cache/vllm/torch_compile_cache/torch_aot_compile/c7e6135c513b7703a8a35284f00ce46f2f1d83ffe52948094e850ea46bf2f9df/rank_0_0/model
(EngineCore pid=183) INFO 07-14 22:49:39 [monitor.py:53] torch.compile took 16.69 s in total
(EngineCore pid=183) INFO 07-14 22:49:39 [monitor.py:81] Initial profiling/warmup run took 0.79 s
(EngineCore pid=183) INFO 07-14 22:49:41 [gpu_worker.py:538] Available KV cache memory: 21.56 GiB
(EngineCore pid=183) INFO 07-14 22:49:41 [kv_cache_utils.py:2146] GPU KV cache size: 176,656 tokens
(EngineCore pid=183) INFO 07-14 22:49:41 [kv_cache_utils.py:2147] Maximum concurrency for 4,096 tokens per request: 43.13x
(EngineCore pid=183) INFO 07-14 22:49:41 [cutedsl_warmup.py:97] Skipping CuTeDSL warmup because no compile units were requested.
(EngineCore pid=183) Capturing CUDA graphs (PIECEWISE): 0%| | 0/51 [00:00<?, ?it/s] Capturing CUDA graphs (PIECEWISE): 4%|▍ | 2/51 [00:00<00:02, 16.42it/s] Capturing CUDA graphs (PIECEWISE): 8%|▊ | 4/51 [00:00<00:02, 16.98it/s] Capturing CUDA graphs (PIECEWISE): 12%|█▏ | 6/51 [00:00<00:02, 17.25it/s] Capturing CUDA graphs (PIECEWISE): 16%|█▌ | 8/51 [00:00<00:02, 17.46it/s] Capturing CUDA graphs (PIECEWISE): 20%|█▉ | 10/51 [00:00<00:02, 18.07it/s] Capturing CUDA graphs (PIECEWISE): 24%|██▎ | 12/51 [00:00<00:02, 18.52it/s] Capturing CUDA graphs (PIECEWISE): 27%|██▋ | 14/51 [00:00<00:01, 18.88it/s] Capturing CUDA graphs (PIECEWISE): 31%|███▏ | 16/51 [00:00<00:01, 19.18it/s] Capturing CUDA graphs (PIECEWISE): 35%|███▌ | 18/51 [00:00<00:01, 19.12it/s] Capturing CUDA graphs (PIECEWISE): 41%|████ | 21/51 [00:01<00:01, 19.86it/s] Capturing CUDA graphs (PIECEWISE): 47%|████▋ | 24/51 [00:01<00:01, 20.37it/s] Capturing CUDA graphs (PIECEWISE): 53%|█████▎ | 27/51 [00:01<00:01, 20.53it/s] Capturing CUDA graphs (PIECEWISE): 59%|█████▉ | 30/51 [00:01<00:01, 20.89it/s] Capturing CUDA graphs (PIECEWISE): 65%|██████▍ | 33/51 [00:01<00:00, 21.14it/s] Capturing CUDA graphs (PIECEWISE): 71%|███████ | 36/51 [00:01<00:00, 21.37it/s] Capturing CUDA graphs (PIECEWISE): 76%|███████▋ | 39/51 [00:01<00:00, 20.96it/s] Capturing CUDA graphs (PIECEWISE): 82%|████████▏ | 42/51 [00:02<00:00, 21.41it/s] Capturing CUDA graphs (PIECEWISE): 88%|████████▊ | 45/51 [00:02<00:00, 21.82it/s] Capturing CUDA graphs (PIECEWISE): 94%|█████████▍| 48/51 [00:02<00:00, 22.35it/s] Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:02<00:00, 22.21it/s] Capturing CUDA graphs (PIECEWISE): 100%|██████████| 51/51 [00:02<00:00, 20.43it/s]
(EngineCore pid=183) Capturing CUDA graphs (FULL): 0%| | 0/35 [00:00<?, ?it/s] Capturing CUDA graphs (FULL): 6%|▌ | 2/35 [00:00<00:01, 19.85it/s] Capturing CUDA graphs (FULL): 14%|█▍ | 5/35 [00:00<00:01, 21.21it/s] Capturing CUDA graphs (FULL): 23%|██▎ | 8/35 [00:00<00:01, 21.78it/s] Capturing CUDA graphs (FULL): 31%|███▏ | 11/35 [00:00<00:01, 22.07it/s] Capturing CUDA graphs (FULL): 40%|████ | 14/35 [00:00<00:00, 22.39it/s] Capturing CUDA graphs (FULL): 49%|████▊ | 17/35 [00:00<00:00, 22.60it/s] Capturing CUDA graphs (FULL): 57%|█████▋ | 20/35 [00:00<00:00, 22.90it/s] Capturing CUDA graphs (FULL): 66%|██████▌ | 23/35 [00:01<00:00, 23.19it/s] Capturing CUDA graphs (FULL): 74%|███████▍ | 26/35 [00:01<00:00, 23.61it/s] Capturing CUDA graphs (FULL): 83%|████████▎ | 29/35 [00:01<00:00, 24.08it/s] Capturing CUDA graphs (FULL): 91%|█████████▏| 32/35 [00:01<00:00, 24.69it/s] Capturing CUDA graphs (FULL): 100%|██████████| 35/35 [00:01<00:00, 25.11it/s] Capturing CUDA graphs (FULL): 100%|██████████| 35/35 [00:01<00:00, 23.49it/s]
(EngineCore pid=183) INFO 07-14 22:49:46 [model_runner.py:722] Graph capturing finished in 4 secs, took 0.59 GiB
(EngineCore pid=183) INFO 07-14 22:49:51 [jit_monitor.py:73] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
(EngineCore pid=183) INFO 07-14 22:49:51 [core.py:337] init engine (profile, create kv cache, warmup model) took 29.70 s (compilation: 16.69 s)
(EngineCore pid=183) [transformers] You have used a torchvision backend image processor with LANCZOS resample which is not supported for torch.Tensor with torchvision < 0.27. BICUBIC resample will be used as an alternative. Please upgrade torchvision to 0.27+ or fall back to a pil backend image processor if you want full consistency with the original model.
(EngineCore pid=183) INFO 07-14 22:49:56 [kernel.py:292] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(APIServer pid=63) INFO 07-14 22:49:56 [api_server.py:612] Supported tasks: ['generate']
(APIServer pid=63) WARNING 07-14 22:49:57 [model.py:1528] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_p': 0.9}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(APIServer pid=63) INFO 07-14 22:49:58 [hf.py:548] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
(APIServer pid=63) INFO 07-14 22:50:07 [base.py:236] Multi-modal warmup completed in 8.666s
(APIServer pid=63) INFO 07-14 22:50:07 [base.py:236] Readonly multi-modal warmup completed in 0.002s
(APIServer pid=63) INFO 07-14 22:50:07 [api_server.py:616] Starting vLLM server on http://127.0.0.1:8001
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:37] Available routes are:
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /openapi.json, Methods: HEAD, GET
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /docs, Methods: HEAD, GET
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /docs/oauth2-redirect, Methods: HEAD, GET
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /redoc, Methods: HEAD, GET
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /load, Methods: GET
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /version, Methods: GET
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /health, Methods: GET
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /metrics, Methods: GET
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /tokenize, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /detokenize, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/models, Methods: GET
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /ping, Methods: GET
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /ping, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /invocations, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/chat/completions, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/chat/completions/batch, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/responses, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/responses/{response_id}, Methods: GET
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/responses/{response_id}/cancel, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/completions, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/messages, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/messages/count_tokens, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /generative_scoring, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /scale_elastic_ep, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /is_scaling_elastic_ep, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/chat/completions/render, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/completions/render, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/chat/completions/derender, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /v1/completions/derender, Methods: POST
(APIServer pid=63) INFO 07-14 22:50:07 [launcher.py:46] Route: /inference/v1/generate, Methods: POST
(APIServer pid=63) INFO: Started server process [63]
(APIServer pid=63) INFO: Waiting for application startup.
(APIServer pid=63) INFO: Application startup complete.
(APIServer pid=63) INFO: 127.0.0.1:45062 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:50:09,481 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
2026-07-14 22:50:09,481 gateway INFO vLLM is ready at http://127.0.0.1:8001
(APIServer pid=63) INFO: 127.0.0.1:52688 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:50:39,484 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:49910 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:51:09,486 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:38014 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:51:39,489 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:48754 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:52:09,491 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:53060 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:52:39,493 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:36446 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:53:09,495 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:45242 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:53:39,497 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:40408 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:54:09,500 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:35978 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:54:39,503 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:40294 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:55:09,506 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:39552 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:55:39,508 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:51456 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:56:09,510 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:56050 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:56:39,513 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:32968 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:57:09,515 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:53080 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:57:39,518 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:48462 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:58:09,521 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:36630 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:58:39,523 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:48098 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:59:09,526 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:36372 - "GET /health HTTP/1.1" 200 OK
2026-07-14 22:59:39,527 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:44740 - "GET /health HTTP/1.1" 200 OK
2026-07-14 23:00:09,530 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:47376 - "GET /health HTTP/1.1" 200 OK
2026-07-14 23:00:39,533 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:38278 - "GET /health HTTP/1.1" 200 OK
2026-07-14 23:01:09,536 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:49090 - "GET /health HTTP/1.1" 200 OK
2026-07-14 23:01:39,539 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:48350 - "GET /health HTTP/1.1" 200 OK
2026-07-14 23:02:09,541 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
(APIServer pid=63) INFO: 127.0.0.1:41056 - "GET /health HTTP/1.1" 200 OK
2026-07-14 23:02:39,544 httpx INFO HTTP Request: GET http://127.0.0.1:8001/health "HTTP/1.1 200 OK"
INFO: Shutting down
INFO: Waiting for application shutdown.
INFO: Application shutdown complete.
INFO: Finished server process [1]

Xet Storage Details

Size:
29.6 kB
·
Xet hash:
b2bfe7dc595e79aeeb8a66ac572ad9698ad70a2a3d89a5177ec487be5949264e

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.