mlx-community

/

IQuest-Coder-V1-40B-Instruct-4bit

like 2

Follow

MLX Community 14.9k

Code snippet:

python
from mlx_lm import load, generate
model, tokenizer = load("mlx-community/IQuest-Coder-V1-40B-Instruct-4bit")
Issue Description: The model downloads successfully (21.4GB), but mlx_lm fails to load it because the model architecture type "iquestcoder" is not recognized. The library is looking for a module at mlx_lm.models.iquestcoder which doesn't exist.

Environment:

Python 3.12

pip show mlx-lm
Name: mlx-lm
Version: 0.30.0
Summary: LLMs with MLX and the Hugging Face Hub
Home-page: https://github.com/ml-explore/mlx-lm
Author: MLX Contributors
Author-email: mlx@group.apple.com
License: MIT
Location: /opt/homebrew/Caskroom/miniforge/base/lib/python3.12/site-packages
Requires: jinja2, mlx, numpy, protobuf, pyyaml, sentencepiece, transformers
Required-by:

Model: mlx-community/IQuest-Coder-V1-40B-Instruct-4bit
Question: Is this model architecture supported by the current version of mlx_lm? Do I need to upgrade the library or use a different approach to load this model?

Loading IQuest-Coder-V1-40B-Instruct-4bit model...
This may take a few minutes on first run as it downloads the model.

Fetching 17 files: 100%|███████████████████████| 17/17 [08:47<00:00, 31.06s/it]
Download complete: 100%|████████████████████| 21.4G/21.4G [08:48<00:00, 40.5MB/s]
ERROR:root:Model type iquestcoder not supported.
Traceback (most recent call last):
File "/path/to/site-packages/mlx_lm/utils.py", line 71, in _get_classes
arch = importlib.import_module(f"mlx_lm.models.{model_type}")
File "/path/to/importlib/init.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'mlx_lm.models.iquestcoder'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "run_iquest_coder.py", line 7, in
model, tokenizer = load("mlx-community/IQuest-Coder-V1-40B-Instruct-4bit")
File "/path/to/site-packages/mlx_lm/utils.py", line 322, in load
model, config = load_model(model_path, lazy, model_config=model_config)
File "/path/to/site-packages/mlx_lm/utils.py", line 201, in load_model
model_class, model_args_class = get_model_classes(config=config)
File "/path/to/site-packages/mlx_lm/utils.py", line 75, in _get_classes
raise ValueError(msg)
ValueError: Model type iquestcoder not supported.