Unable to start
#16
by jondurbin - opened
It seems after this commit the model can no longer start properly (e.g. with vllm). It seems having the auto_map still declared in the config.json causes it to try to load those (via trust_remote_code) from the repo, but these files no longer exist. I see they exist in the base transformers library now, but evidently it still does not work even after I remove the auto_map from the config manually and try to start without trust_remote_code:
(VllmWorkerProcess pid=3950820) ERROR 01-20 17:58:04 multiproc_worker_utils.py:236] File "/home/jdurbin/.local/lib/python3.10/site-packages/transformers/configuration_utils.py", line 211, in __getattribute__
(VllmWorkerProcess pid=3950820) ERROR 01-20 17:58:04 multiproc_worker_utils.py:236] return super().__getattribute__(key)
(VllmWorkerProcess pid=3950820) ERROR 01-20 17:58:04 multiproc_worker_utils.py:236] AttributeError: 'AriaTextConfig' object has no attribute 'moe_intermediate_size'
I can just add the revision flag to the model when starting it to use a commit prior to this one, but just thought you should know.
@jondurbin Thanks for pointing that out!
I noticed that vLLM has mad some related changes recently (https://github.com/vllm-project/vllm/pull/12203)