Spaces:

---

Intel

/

low_bit_open_llm_leaderboard

like 219

Running

Pipeline Failure Report

Model: Qwen/Qwen3.5-35B-A3B
Quantization Scheme: INT4 (W4A16)
Failed Phase: evaluate
Run ID: Qwen3.5-35B-A3B-AutoRound-W4A16-ModelFree
Error Category: auto_round_unsupported_layer_backend

Full Error Log

Replacing modules:  20%|██        | 8/40 [00:00<00:03,  8.33it/s]
Replacing modules:  25%|██▌       | 10/40 [00:00<00:02, 10.20it/s]
Replacing modules:  30%|███       | 12/40 [00:01<00:02, 11.80it/s]
Replacing modules:  35%|███▌      | 14/40 [00:01<00:01, 13.16it/s]
Replacing modules:  40%|████      | 16/40 [00:01<00:01, 14.24it/s]
Replacing modules:  45%|████▌     | 18/40 [00:01<00:02,  7.90it/s]
Replacing modules:  50%|█████     | 20/40 [00:01<00:02,  9.46it/s]
Replacing modules:  55%|█████▌    | 22/40 [00:01<00:01, 10.97it/s]
Replacing modules:  60%|██████    | 24/40 [00:02<00:01, 12.33it/s]
Replacing modules:  65%|██████▌   | 26/40 [00:02<00:01, 13.52it/s]
Replacing modules:  70%|███████   | 28/40 [00:02<00:00, 14.50it/s]
Replacing modules:  75%|███████▌  | 30/40 [00:02<00:00, 15.20it/s]
Replacing modules:  80%|████████  | 32/40 [00:03<00:01,  7.00it/s]
Replacing modules:  85%|████████▌ | 34/40 [00:03<00:00,  8.52it/s]
Replacing modules:  90%|█████████ | 36/40 [00:03<00:00, 10.03it/s]
Replacing modules:  95%|█████████▌| 38/40 [00:03<00:00, 11.45it/s]
Replacing modules: 100%|██████████| 40/40 [00:03<00:00, 12.70it/s]
Replacing modules: 100%|██████████| 40/40 [00:03<00:00, 11.24it/s]
2026-07-10 04:49:01 INFO replace_modules.py L399: Replaced 40 modules
2026-07-10 04:49:01 INFO device.py L1484: [Memory Monitor] After applying custom replacements: 'peak_ram': 1.27GB
2026-07-10 04:49:02 WARNING modeling_utils.py L4682: `loss_type=None` was set in the config but it is unrecognized. Using the default loss: `ForCausalLMLoss`.
2026-07-10 04:49:04 INFO replace_modules.py L121: Experts (after replacement/skip) [model.layers.0.mlp.experts] (SequentialQwen3_5MoeExperts):
SequentialQwen3_5MoeExperts(
  (0-255): 256 x Qwen3_5MoeMLP(
    (gate_proj): Linear(in_features=2048, out_features=512, bias=False)
    (up_proj): Linear(in_features=2048, out_features=512, bias=False)
    (down_proj): Linear(in_features=512, out_features=2048, bias=False)
    (act_fn): SiLUActivation()
  )
)
Traceback (most recent call last):
  File "/root/.venv/bin/lm_eval", line 10, in <module>
    sys.exit(cli_evaluate())
             ^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/lm_eval/__main__.py", line 10, in cli_evaluate
    parser.execute(args)
  File "/root/.venv/lib/python3.12/site-packages/lm_eval/_cli/harness.py", line 60, in execute
    args.func(args)
  File "/root/.venv/lib/python3.12/site-packages/lm_eval/_cli/run.py", line 391, in _execute
    results = simple_evaluate(
              ^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/lm_eval/utils.py", line 575, in _wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/lm_eval/evaluator.py", line 242, in simple_evaluate
    lm = lm_eval.api.registry.get_model(model).create_from_arg_obj(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/lm_eval/api/model.py", line 169, in create_from_arg_obj
    return cls(**arg_dict, **additional_config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/lm_eval/models/huggingface.py", line 365, in __init__
    self._create_model(
  File "/root/.venv/lib/python3.12/site-packages/lm_eval/models/huggingface.py", line 814, in _create_model
    self._model = self.AUTO_MODEL_CLASS.from_pretrained(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 402, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4328, in from_pretrained
    hf_quantizer.preprocess_model(
  File "/root/.venv/lib/python3.12/site-packages/transformers/quantizers/base.py", line 171, in preprocess_model
    self._process_model_before_weight_loading(model, **kwargs)
  File "/root/.venv/lib/python3.12/site-packages/transformers/quantizers/quantizer_auto_round.py", line 54, in _process_model_before_weight_loading
    model, used_backends = convert_hf_model(model, target_device)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/auto_round/inference/convert_model.py", line 874, in convert_hf_model
    used_backends = _replace_by_quant_layers(model, layer_configs, backend, target_device, packing_format)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/auto_round/inference/convert_model.py", line 471, in _replace_by_quant_layers
    raise ValueError(f"No compatible backend found for layer {layer_name} with config {config}")
ValueError: No compatible backend found for layer model.layers.26.mlp.shared_expert_gate with config QuantizationScheme(bits=4, group_size=128, sym=True, data_type='int', act_bits=None, act_group_size=False, act_sym=None, act_data_type=None, act_dynamic=False, super_bits=None, super_group_size=None, rotation_config=None)

Auto-generated by error_analysis pipeline. cc @lvkaokao