Spaces:

---

Intel

/

low_bit_open_llm_leaderboard

like 219

Running

Pipeline Failure Report

Model: FlameF0X/LFM2.5-1.2B-Distilled-Claude
Quantization Scheme: INT4 (W4A16)
Failed Phase: quantize
Run ID: LFM2.5-1.2B-Distilled-Claude-AutoRound-W4A16-RTN
Error Category: quantization_index_error

Full Error Log

Filter:   0%|          | 0/10000 [00:00<?, ? examples/s]
Filter: 100%|██████████| 10000/10000 [00:06<00:00, 1657.46 examples/s]

Casting the dataset:   0%|          | 0/1245 [00:00<?, ? examples/s]
Casting the dataset: 100%|██████████| 1245/1245 [00:04<00:00, 275.55 examples/s]
11:10:07 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
11:10:07 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/NeelNanda/pile-10k/127bfedcd5047750df5ccf3a12979a47bfa0bafa/README.md "HTTP/1.1 200 OK"
11:10:07 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/pile-10k.py "HTTP/1.1 404 Not Found"
11:10:07 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/NeelNanda/pile-10k/NeelNanda/pile-10k.py "HTTP/1.1 404 Not Found"
11:10:07 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/revision/127bfedcd5047750df5ccf3a12979a47bfa0bafa "HTTP/1.1 200 OK"
11:10:08 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/.huggingface.yaml "HTTP/1.1 404 Not Found"
11:10:08 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=NeelNanda/pile-10k "HTTP/1.1 200 OK"
11:10:08 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/tree/127bfedcd5047750df5ccf3a12979a47bfa0bafa/data?recursive=true&expand=false "HTTP/1.1 200 OK"
11:10:08 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/tree/127bfedcd5047750df5ccf3a12979a47bfa0bafa?recursive=false&expand=false "HTTP/1.1 200 OK"
2026-06-26 11:10:08 INFO calib_dataset.py L977: Preprocessing calibration dataset in a subprocess to avoid memory leaks...
11:10:09 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
11:10:09 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/NeelNanda/pile-10k/127bfedcd5047750df5ccf3a12979a47bfa0bafa/README.md "HTTP/1.1 200 OK"
11:10:09 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/pile-10k.py "HTTP/1.1 404 Not Found"
11:10:09 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/NeelNanda/pile-10k/NeelNanda/pile-10k.py "HTTP/1.1 404 Not Found"
11:10:10 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/.huggingface.yaml "HTTP/1.1 404 Not Found"
11:10:10 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=NeelNanda/pile-10k "HTTP/1.1 200 OK"
11:10:10 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
11:10:10 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/NeelNanda/pile-10k/127bfedcd5047750df5ccf3a12979a47bfa0bafa/README.md "HTTP/1.1 200 OK"
11:10:10 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/pile-10k.py "HTTP/1.1 404 Not Found"
11:10:10 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/NeelNanda/pile-10k/NeelNanda/pile-10k.py "HTTP/1.1 404 Not Found"
11:10:11 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/.huggingface.yaml "HTTP/1.1 404 Not Found"
11:10:11 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=NeelNanda/pile-10k "HTTP/1.1 200 OK"

  0%|          | 0/16 [00:00<?, ?it/s]
Quantizing model.layers.0:   0%|          | 0/16 [00:00<?, ?it/s]11:10:11 [ERROR] Quantization failed: list index out of range
Traceback (most recent call last):
  File "/root/_work/1/s/auto_quant/phases/quantize.py", line 282, in <module>
    quantize(args)
  File "/root/_work/1/s/auto_quant/phases/quantize.py", line 183, in quantize
    autoround.quantize()
  File "/root/.venv/lib/python3.12/site-packages/auto_round/compressors/data_driven.py", line 1149, in quantize
    return self._quantize_impl()
           ^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/auto_round/compressors/data_driven.py", line 1175, in _quantize_impl
    self._quant_rtn_with_imatrix()
  File "/root/.venv/lib/python3.12/site-packages/auto_round/compressors/data_driven.py", line 1109, in _quant_rtn_with_imatrix
    self._quantize_via_rtn_blockwise()
  File "/root/.venv/lib/python3.12/site-packages/auto_round/compressors/data_driven.py", line 1027, in _quantize_via_rtn_blockwise
    input_ids = self.quantizer._get_block_outputs(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/auto_round/algorithms/quantization/base.py", line 444, in _get_block_outputs
    tmp_input_ids, tmp_input_others = self._sampling_inputs(
                                      ^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/auto_round/algorithms/quantization/base.py", line 603, in _sampling_inputs
    current_input_others[key] = [input_others[key][i] for i in indices]
                                 ~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Quantizing model.layers.0:   0%|          | 0/16 [00:00<?, ?it/s]

Auto-generated by error_analysis pipeline. cc @lvkaokao