TypeError: Qwen2_5_VLProcessor.__init__() got multiple values for argument 'image_processor'
#13
by
rocky-vr-46
- opened
I tried to host the vllm model using
vllm serve reducto/RolmOCR
And its not able to run it , fails with this error :
ERROR 06-29 14:29:28 [core.py:515] EngineCore failed to start.
ERROR 06-29 14:29:28 [core.py:515] Traceback (most recent call last):
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 506, in run_engine_core
ERROR 06-29 14:29:28 [core.py:515] engine_core = EngineCoreProc(*args, **kwargs)
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 390, in __init__
ERROR 06-29 14:29:28 [core.py:515] super().__init__(vllm_config, executor_class, log_stats,
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 76, in __init__
ERROR 06-29 14:29:28 [core.py:515] self.model_executor = executor_class(vllm_config)
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 53, in __init__
ERROR 06-29 14:29:28 [core.py:515] self._init_executor()
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 47, in _init_executor
ERROR 06-29 14:29:28 [core.py:515] self.collective_rpc("init_device")
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
ERROR 06-29 14:29:28 [core.py:515] answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/utils.py", line 2671, in run_method
ERROR 06-29 14:29:28 [core.py:515] return func(*args, **kwargs)
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/worker/worker_base.py", line 606, in init_device
ERROR 06-29 14:29:28 [core.py:515] self.worker.init_device() # type: ignore
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/worker/gpu_worker.py", line 160, in init_device
ERROR 06-29 14:29:28 [core.py:515] self.model_runner: GPUModelRunner = GPUModelRunner(
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/worker/gpu_model_runner.py", line 129, in __init__
ERROR 06-29 14:29:28 [core.py:515] encoder_compute_budget, encoder_cache_size = compute_encoder_budget(
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/core/encoder_cache_manager.py", line 95, in compute_encoder_budget
ERROR 06-29 14:29:28 [core.py:515] ) = _compute_encoder_budget_multimodal(
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/core/encoder_cache_manager.py", line 125, in _compute_encoder_budget_multimodal
ERROR 06-29 14:29:28 [core.py:515] .get_max_tokens_per_item_by_nonzero_modality(model_config)
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/multimodal/registry.py", line 158, in get_max_tokens_per_item_by_nonzero_modality
ERROR 06-29 14:29:28 [core.py:515] self.get_max_tokens_per_item_by_modality(model_config).items()
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/multimodal/registry.py", line 132, in get_max_tokens_per_item_by_modality
ERROR 06-29 14:29:28 [core.py:515] return profiler.get_mm_max_tokens(
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 256, in get_mm_max_tokens
ERROR 06-29 14:29:28 [core.py:515] mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 166, in _get_dummy_mm_inputs
ERROR 06-29 14:29:28 [core.py:515] processor_inputs = factory.get_dummy_processor_inputs(
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 91, in get_dummy_processor_inputs
ERROR 06-29 14:29:28 [core.py:515] dummy_text = self.get_dummy_text(mm_counts)
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/model_executor/models/qwen2_vl.py", line 973, in get_dummy_text
ERROR 06-29 14:29:28 [core.py:515] hf_processor = self.info.get_hf_processor()
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/model_executor/models/qwen2_5_vl.py", line 795, in get_hf_processor
ERROR 06-29 14:29:28 [core.py:515] return self.ctx.get_hf_processor(
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/inputs/registry.py", line 131, in get_hf_processor
ERROR 06-29 14:29:28 [core.py:515] return super().get_hf_processor(
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/inputs/registry.py", line 94, in get_hf_processor
ERROR 06-29 14:29:28 [core.py:515] return cached_processor_from_config(
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/transformers_utils/processor.py", line 110, in cached_processor_from_config
ERROR 06-29 14:29:28 [core.py:515] return cached_get_processor(
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/transformers_utils/processor.py", line 72, in get_processor
ERROR 06-29 14:29:28 [core.py:515] processor = processor_factory.from_pretrained(
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/transformers/processing_utils.py", line 1304, in from_pretrained
ERROR 06-29 14:29:28 [core.py:515] return cls.from_args_and_dict(args, processor_dict, **kwargs)
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/transformers/processing_utils.py", line 1105, in from_args_and_dict
ERROR 06-29 14:29:28 [core.py:515] processor = cls(*args, **valid_kwargs)
ERROR 06-29 14:29:28 [core.py:515] ^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-29 14:29:28 [core.py:515] TypeError: Qwen2_5_VLProcessor.__init__() got multiple values for argument 'image_processor'
Process EngineCore_0:
Traceback (most recent call last):
File "/home/ml/anaconda3/envs/temp/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/home/ml/anaconda3/envs/temp/lib/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 519, in run_engine_core
raise e
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 506, in run_engine_core
engine_core = EngineCoreProc(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 390, in __init__
super().__init__(vllm_config, executor_class, log_stats,
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 76, in __init__
self.model_executor = executor_class(vllm_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 53, in __init__
self._init_executor()
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 47, in _init_executor
self.collective_rpc("init_device")
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
answer = run_method(self.driver_worker, method, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/utils.py", line 2671, in run_method
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/worker/worker_base.py", line 606, in init_device
self.worker.init_device() # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/worker/gpu_worker.py", line 160, in init_device
self.model_runner: GPUModelRunner = GPUModelRunner(
^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/worker/gpu_model_runner.py", line 129, in __init__
encoder_compute_budget, encoder_cache_size = compute_encoder_budget(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/core/encoder_cache_manager.py", line 95, in compute_encoder_budget
) = _compute_encoder_budget_multimodal(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/core/encoder_cache_manager.py", line 125, in _compute_encoder_budget_multimodal
.get_max_tokens_per_item_by_nonzero_modality(model_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/multimodal/registry.py", line 158, in get_max_tokens_per_item_by_nonzero_modality
self.get_max_tokens_per_item_by_modality(model_config).items()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/multimodal/registry.py", line 132, in get_max_tokens_per_item_by_modality
return profiler.get_mm_max_tokens(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 256, in get_mm_max_tokens
mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 166, in _get_dummy_mm_inputs
processor_inputs = factory.get_dummy_processor_inputs(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/multimodal/profiling.py", line 91, in get_dummy_processor_inputs
dummy_text = self.get_dummy_text(mm_counts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/model_executor/models/qwen2_vl.py", line 973, in get_dummy_text
hf_processor = self.info.get_hf_processor()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/model_executor/models/qwen2_5_vl.py", line 795, in get_hf_processor
return self.ctx.get_hf_processor(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/inputs/registry.py", line 131, in get_hf_processor
return super().get_hf_processor(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/inputs/registry.py", line 94, in get_hf_processor
return cached_processor_from_config(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/transformers_utils/processor.py", line 110, in cached_processor_from_config
return cached_get_processor(
^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/transformers_utils/processor.py", line 72, in get_processor
processor = processor_factory.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/transformers/processing_utils.py", line 1304, in from_pretrained
return cls.from_args_and_dict(args, processor_dict, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/transformers/processing_utils.py", line 1105, in from_args_and_dict
processor = cls(*args, **valid_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Qwen2_5_VLProcessor.__init__() got multiple values for argument 'image_processor'
[rank0]:[W629 14:29:28.543640345 ProcessGroupNCCL.cpp:1476] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Traceback (most recent call last):
File "/home/ml/anaconda3/envs/temp/bin/vllm", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/entrypoints/cli/main.py", line 59, in main
args.dispatch_function(args)
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/entrypoints/cli/serve.py", line 58, in cmd
uvloop.run(run_server(args))
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/uvloop/__init__.py", line 105, in run
return runner.run(wrapper())
^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/uvloop/__init__.py", line 61, in wrapper
return await main
^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 1323, in run_server
await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 1343, in run_server_worker
async with build_async_engine_client(args, client_config) as engine_client:
File "/home/ml/anaconda3/envs/temp/lib/python3.11/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 155, in build_async_engine_client
async with build_async_engine_client_from_engine_args(
File "/home/ml/anaconda3/envs/temp/lib/python3.11/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 191, in build_async_engine_client_from_engine_args
async_llm = AsyncLLM.from_vllm_config(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/async_llm.py", line 162, in from_vllm_config
return cls(
^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/async_llm.py", line 124, in __init__
self.engine_core = EngineCoreClient.make_async_mp_client(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 93, in make_async_mp_client
return AsyncMPClient(vllm_config, executor_class, log_stats,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 716, in __init__
super().__init__(
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 422, in __init__
self._init_engines_direct(vllm_config, local_only,
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 491, in _init_engines_direct
self._wait_for_engine_startup(handshake_socket, input_address,
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 511, in _wait_for_engine_startup
wait_for_engine_startup(
File "/home/ml/anaconda3/envs/temp/lib/python3.11/site-packages/vllm/v1/utils.py", line 494, in wait_for_engine_startup
raise RuntimeError("Engine core initialization failed. "
RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {
System :
OS : Ubuntu (Linux wsl)
GPU : RTX 4090
Cuda : 12.6
The new transformers lib version 4.53.0 was causing this error.
I downgraded it to version 4.52.4 and it worked.
rocky-vr-46
changed discussion status to
closed