Spaces:

Intel
/

low_bit_open_llm_leaderboard

Running

App Files Files Community
aquiffoo/neo-3-3B-A400M-Base-AutoRound-W4A16-RTN | INT4 (W4A16)

#10
by lvkaokao - opened 17 days ago
Discussion
lvkaokao
Intel org 17 days ago
Pipeline Failure Report

Model: aquiffoo/neo-3-3B-A400M-Base-AutoRound-W4A16-RTN
Quantization Scheme: INT4 (W4A16)
Failed Phase: quantize
Run ID: run_2026-06-11-16-47-36
Error Category: autoround_internal_error
Full Error Log

15:56:22 [INFO] HTTP Request: HEAD https://huggingface.co/aquiffoo/neo-3-3B-A400M-Base/resolve/main/model_index.json "HTTP/1.1 404 Not Found"
404 Client Error. (Request ID: Root=1-6a2adaa6-36cb01bf569b85723136ffb0;7eecce26-2f45-47b7-90a7-04860a9d2a9a)

Entry Not Found for url: https://huggingface.co/aquiffoo/neo-3-3B-A400M-Base/resolve/main/model_index.json.
15:56:22 [INFO] HTTP Request: HEAD https://huggingface.co/aquiffoo/neo-3-3B-A400M-Base/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
15:56:22 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/aquiffoo/neo-3-3B-A400M-Base/71efb8d31dcb942dac7db2925b2917237d7ee34c/config.json "HTTP/1.1 200 OK"
[33;1m2026-06-11 15:56:22 WARNING model.py L175: This MoE model has not been optimized by AutoRound yet, which may result in high RAM usage, Please consider submitting an issue to https://github.com/intel/auto-round/issues[0m
15:56:22 [INFO] HTTP Request: HEAD https://huggingface.co/aquiffoo/neo-3-3B-A400M-Base/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
15:56:24 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/aquiffoo/neo-3-3B-A400M-Base/71efb8d31dcb942dac7db2925b2917237d7ee34c/config.json "HTTP/1.1 200 OK"
15:56:25 [INFO] HTTP Request: HEAD https://huggingface.co/aquiffoo/neo-3-3B-A400M-Base/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
15:56:25 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/aquiffoo/neo-3-3B-A400M-Base/71efb8d31dcb942dac7db2925b2917237d7ee34c/config.json "HTTP/1.1 200 OK"
15:56:25 [INFO] HTTP Request: HEAD https://huggingface.co/aquiffoo/neo-3-3B-A400M-Base/resolve/main/tokenizer_config.json "HTTP/1.1 307 Temporary Redirect"
15:56:25 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/aquiffoo/neo-3-3B-A400M-Base/71efb8d31dcb942dac7db2925b2917237d7ee34c/tokenizer_config.json "HTTP/1.1 200 OK"
15:56:25 [INFO] HTTP Request: HEAD https://huggingface.co/aquiffoo/neo-3-3B-A400M-Base/resolve/main/tokenizer_config.json "HTTP/1.1 307 Temporary Redirect"
15:56:25 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/aquiffoo/neo-3-3B-A400M-Base/71efb8d31dcb942dac7db2925b2917237d7ee34c/tokenizer_config.json "HTTP/1.1 200 OK"
15:56:25 [INFO] HTTP Request: GET https://huggingface.co/api/models/aquiffoo/neo-3-3B-A400M-Base/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 404 Not Found"
15:56:25 [INFO] HTTP Request: GET https://huggingface.co/api/models/aquiffoo/neo-3-3B-A400M-Base/tree/main?recursive=true&expand=false "HTTP/1.1 200 OK"
15:56:26 [INFO] HTTP Request: HEAD https://huggingface.co/aquiffoo/neo-3-3B-A400M-Base/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
15:56:26 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/aquiffoo/neo-3-3B-A400M-Base/71efb8d31dcb942dac7db2925b2917237d7ee34c/config.json "HTTP/1.1 200 OK"
16:01:41 [INFO] HTTP Request: HEAD https://huggingface.co/aquiffoo/neo-3-3B-A400M-Base/resolve/main/generation_config.json "HTTP/1.1 307 Temporary Redirect"
16:01:41 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/aquiffoo/neo-3-3B-A400M-Base/71efb8d31dcb942dac7db2925b2917237d7ee34c/generation_config.json "HTTP/1.1 200 OK"
16:01:41 [INFO] HTTP Request: HEAD https://huggingface.co/aquiffoo/neo-3-3B-A400M-Base/resolve/main/custom_generate/generate.py "HTTP/1.1 404 Not Found"
16:01:42 [INFO] Starting quantization...
[38;20m2026-06-11 16:01:42 INFO replace_modules.py L120: Experts (before replacement) [model.layers.0.mlp.experts] (MixtralExperts):
MixtralExperts(
  (act_fn): SiLUActivation()
)[0m
[transformers] `loss_type=None` was set in the config but it is unrecognized. Using the default loss: `ForCausalLMLoss`.
[38;20m2026-06-11 16:01:42 INFO device.py L1838: Before applying custom replacements 'peak_ram': 12.06GB, 'peak_vram': 6.11GB[0m
[38;20m2026-06-11 16:01:56 INFO moe_experts_interface.py L655: [MoE Prep] Unfused 30 MOE experts modules[0m
[38;20m2026-06-11 16:01:56 INFO device.py L1838: After applying custom replacements 'peak_ram': 12.06GB, 'peak_vram': 6.11GB[0m
[38;20m2026-06-11 16:01:56 INFO replace_modules.py L93: Prepared 30 MOE modules for quantization[0m
[38;20m2026-06-11 16:01:56 INFO replace_modules.py L120: Experts (after replacement) [model.layers.0.mlp.experts] (MixtralExperts):
MixtralExperts(
  (act_fn): SiLUActivation()
  (0-37): 38 x _ExpertContainer(
    (down_proj): Linear(in_features=1536, out_features=576, bias=False)
    (gate_proj): Linear(in_features=576, out_features=1536, bias=False)
    (up_proj): Linear(in_features=576, out_features=1536, bias=False)
  )
)[0m
[38;20m2026-06-11 16:01:56 INFO base.py L572: Using predefined ignore_layers: model.layers.[0-29].mlp.gate[0m
[38;20m2026-06-11 16:01:56 INFO utils.py L1069: Ignored layers: lm_head, lm_head, model.layers.[0-29].mlp.gate[0m
[33;1m2026-06-11 16:01:56 WARNING logging.py L340: Layer name or regex 'model.layers.[0-29].mlp.gate' in layer_config does not match any supported layers. Please check for typos or update the regex pattern, ignore it for now[0m
[33;1m2026-06-11 16:01:56 WARNING utils.py L539: reset `quant_lm_head` to false as quantizing lm_head with tied weights has not been supported currently[0m
[38;20m2026-06-11 16:01:57 INFO base.py L662: 'enable_torch_compile' is set to `False` by default. Enabling it can reduce tuning cost by 20%, but it might throw an exception.[0m
[38;20m2026-06-11 16:01:57 INFO data_driven.py L1089: start to compute imatrix[0m
[38;20m2026-06-11 16:01:57 INFO calib_dataset.py L977: Preprocessing calibration dataset in a subprocess to avoid memory leaks...[0m
16:01:58 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
16:01:58 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/NeelNanda/pile-10k/127bfedcd5047750df5ccf3a12979a47bfa0bafa/README.md "HTTP/1.1 200 OK"
16:01:58 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/NeelNanda/pile-10k/127bfedcd5047750df5ccf3a12979a47bfa0bafa/README.md "HTTP/1.1 200 OK"
16:01:58 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/pile-10k.py "HTTP/1.1 404 Not Found"
16:01:58 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/NeelNanda/pile-10k/NeelNanda/pile-10k.py "HTTP/1.1 404 Not Found"
16:01:58 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/revision/127bfedcd5047750df5ccf3a12979a47bfa0bafa "HTTP/1.1 200 OK"
16:01:58 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/.huggingface.yaml "HTTP/1.1 404 Not Found"
16:01:59 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=NeelNanda/pile-10k "HTTP/1.1 200 OK"
16:01:59 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/tree/127bfedcd5047750df5ccf3a12979a47bfa0bafa/data?recursive=true&expand=false "HTTP/1.1 200 OK"
16:01:59 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/tree/127bfedcd5047750df5ccf3a12979a47bfa0bafa?recursive=false&expand=false "HTTP/1.1 200 OK"
16:01:59 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/dataset_infos.json "HTTP/1.1 307 Temporary Redirect"
16:01:59 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/NeelNanda/pile-10k/127bfedcd5047750df5ccf3a12979a47bfa0bafa/dataset_infos.json "HTTP/1.1 200 OK"
16:01:59 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/NeelNanda/pile-10k/127bfedcd5047750df5ccf3a12979a47bfa0bafa/dataset_infos.json "HTTP/1.1 200 OK"
16:01:59 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/data/train-00000-of-00001-4746b8785c874cc7.parquet "HTTP/1.1 302 Found"

Generating train split:   0%|          | 0/10000 [00:00<?, ? examples/s]
Generating train split: 100%|██████████| 10000/10000 [00:00<00:00, 14638.22 examples/s]

Map:   0%|          | 0/10000 [00:00<?, ? examples/s]
Map: 100%|██████████| 10000/10000 [00:10<00:00, 999.00 examples/s]

Filter:   0%|          | 0/10000 [00:00<?, ? examples/s]
Filter: 100%|██████████| 10000/10000 [00:04<00:00, 2270.22 examples/s]

Casting the dataset:   0%|          | 0/1307 [00:00<?, ? examples/s]
Casting the dataset: 100%|██████████| 1307/1307 [00:04<00:00, 295.70 examples/s]
16:02:24 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
16:02:24 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/NeelNanda/pile-10k/127bfedcd5047750df5ccf3a12979a47bfa0bafa/README.md "HTTP/1.1 200 OK"
16:02:24 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/pile-10k.py "HTTP/1.1 404 Not Found"
16:02:24 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/NeelNanda/pile-10k/NeelNanda/pile-10k.py "HTTP/1.1 404 Not Found"
16:02:24 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/revision/127bfedcd5047750df5ccf3a12979a47bfa0bafa "HTTP/1.1 200 OK"
16:02:24 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/.huggingface.yaml "HTTP/1.1 404 Not Found"
16:02:25 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=NeelNanda/pile-10k "HTTP/1.1 200 OK"
16:02:25 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/tree/127bfedcd5047750df5ccf3a12979a47bfa0bafa/data?recursive=true&expand=false "HTTP/1.1 200 OK"
16:02:25 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/tree/127bfedcd5047750df5ccf3a12979a47bfa0bafa?recursive=false&expand=false "HTTP/1.1 200 OK"
[38;20m2026-06-11 16:02:25 INFO calib_dataset.py L977: Preprocessing calibration dataset in a subprocess to avoid memory leaks...[0m
16:02:26 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
16:02:26 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/NeelNanda/pile-10k/127bfedcd5047750df5ccf3a12979a47bfa0bafa/README.md "HTTP/1.1 200 OK"
16:02:26 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/pile-10k.py "HTTP/1.1 404 Not Found"
16:02:26 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/NeelNanda/pile-10k/NeelNanda/pile-10k.py "HTTP/1.1 404 Not Found"
16:02:26 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/.huggingface.yaml "HTTP/1.1 404 Not Found"
16:02:27 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=NeelNanda/pile-10k "HTTP/1.1 200 OK"
16:02:27 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
16:02:27 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/NeelNanda/pile-10k/127bfedcd5047750df5ccf3a12979a47bfa0bafa/README.md "HTTP/1.1 200 OK"
16:02:27 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/pile-10k.py "HTTP/1.1 404 Not Found"
16:02:28 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/NeelNanda/pile-10k/NeelNanda/pile-10k.py "HTTP/1.1 404 Not Found"
16:02:28 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/.huggingface.yaml "HTTP/1.1 404 Not Found"
16:02:28 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=NeelNanda/pile-10k "HTTP/1.1 200 OK"

  0%|          | 0/30 [00:00<?, ?it/s]
Quantizing model.layers.0:   0%|          | 0/30 [00:00<?, ?it/s][33;1m2026-06-11 16:03:09 WARNING logging.py L340: MoE layer detected: optimized RTN is disabled for efficiency. Use `--enable_opt_rtn` to force-enable it for MoE layers.[0m
[38;20m2026-06-11 16:03:21 INFO offload.py L706: OffloadManager (compressor): tempdir = /root/_work/1/s/auto_quant/ar_work_space/offload/compressor_xzvjb1js[0m
[38;20m2026-06-11 16:03:21 INFO device.py L1840: 'peak_ram': 13.86GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.1:   3%|▎         | 1/30 [00:37<18:15, 37.76s/it][38;20m2026-06-11 16:03:57 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.2:   7%|▋         | 2/30 [01:12<17:01, 36.48s/it][38;20m2026-06-11 16:04:32 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.3:  10%|█         | 3/30 [01:48<16:17, 36.22s/it][38;20m2026-06-11 16:05:07 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.3:  13%|█▎        | 4/30 [02:23<15:30, 35.80s/it]
Quantizing model.layers.4:  13%|█▎        | 4/30 [02:23<15:30, 35.80s/it]
Quantizing model.layers.4:  13%|█▎        | 4/30 [02:35<15:30, 35.80s/it][38;20m2026-06-11 16:05:44 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.5:  17%|█▋        | 5/30 [02:59<14:54, 35.80s/it][38;20m2026-06-11 16:06:18 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.6:  20%|██        | 6/30 [03:34<14:19, 35.80s/it][38;20m2026-06-11 16:06:52 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.7:  23%|██▎       | 7/30 [04:08<13:43, 35.80s/it][38;20m2026-06-11 16:07:26 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.7:  27%|██▋       | 8/30 [04:42<12:55, 35.24s/it]
Quantizing model.layers.8:  27%|██▋       | 8/30 [04:42<12:55, 35.24s/it]
Quantizing model.layers.8:  27%|██▋       | 8/30 [04:55<12:55, 35.24s/it][38;20m2026-06-11 16:07:59 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.9:  30%|███       | 9/30 [05:15<12:19, 35.24s/it][38;20m2026-06-11 16:08:33 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.10:  33%|███▎      | 10/30 [05:49<11:44, 35.24s/it][38;20m2026-06-11 16:09:09 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.11:  37%|███▋      | 11/30 [06:25<11:09, 35.24s/it][38;20m2026-06-11 16:09:43 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.11:  40%|████      | 12/30 [06:58<10:24, 34.72s/it]
Quantizing model.layers.12:  40%|████      | 12/30 [06:58<10:24, 34.72s/it]
Quantizing model.layers.12:  40%|████      | 12/30 [07:10<10:24, 34.72s/it][38;20m2026-06-11 16:10:16 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.13:  43%|████▎     | 13/30 [07:32<09:50, 34.72s/it][38;20m2026-06-11 16:10:48 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.14:  47%|████▋     | 14/30 [08:04<09:15, 34.72s/it][38;20m2026-06-11 16:11:23 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.15:  50%|█████     | 15/30 [08:39<08:40, 34.72s/it][38;20m2026-06-11 16:11:59 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.15:  53%|█████▎    | 16/30 [09:15<08:02, 34.50s/it]
Quantizing model.layers.16:  53%|█████▎    | 16/30 [09:15<08:02, 34.50s/it]
Quantizing model.layers.16:  53%|█████▎    | 16/30 [09:25<08:02, 34.50s/it][38;20m2026-06-11 16:12:32 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.17:  57%|█████▋    | 17/30 [09:47<07:28, 34.50s/it][38;20m2026-06-11 16:13:06 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.18:  60%|██████    | 18/30 [10:22<06:53, 34.50s/it][38;20m2026-06-11 16:13:41 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.19:  63%|██████▎   | 19/30 [10:57<06:19, 34.50s/it][38;20m2026-06-11 16:14:16 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.19:  67%|██████▋   | 20/30 [11:32<05:43, 34.37s/it]
Quantizing model.layers.20:  67%|██████▋   | 20/30 [11:32<05:43, 34.37s/it]
Quantizing model.layers.20:  67%|██████▋   | 20/30 [11:42<05:43, 34.37s/it][38;20m2026-06-11 16:14:50 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.21:  70%|███████   | 21/30 [12:06<05:09, 34.37s/it][38;20m2026-06-11 16:15:25 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.22:  73%|███████▎  | 22/30 [12:41<04:34, 34.37s/it][38;20m2026-06-11 16:16:00 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.23:  77%|███████▋  | 23/30 [13:16<04:00, 34.37s/it][38;20m2026-06-11 16:16:34 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.23:  80%|████████  | 24/30 [13:50<03:26, 34.45s/it]
Quantizing model.layers.24:  80%|████████  | 24/30 [13:50<03:26, 34.45s/it]
Quantizing model.layers.24:  80%|████████  | 24/30 [14:01<03:26, 34.45s/it][38;20m2026-06-11 16:17:07 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.25:  83%|████████▎ | 25/30 [14:23<02:52, 34.45s/it][38;20m2026-06-11 16:17:45 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.26:  87%|████████▋ | 26/30 [15:00<02:17, 34.45s/it][38;20m2026-06-11 16:18:17 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.27:  90%|█████████ | 27/30 [15:33<01:43, 34.45s/it][38;20m2026-06-11 16:18:51 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.27:  93%|█████████▎| 28/30 [16:07<01:08, 34.34s/it]
Quantizing model.layers.28:  93%|█████████▎| 28/30 [16:07<01:08, 34.34s/it]
Quantizing model.layers.28:  93%|█████████▎| 28/30 [16:21<01:08, 34.34s/it][38;20m2026-06-11 16:19:26 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.29:  97%|█████████▋| 29/30 [16:42<00:34, 34.34s/it][38;20m2026-06-11 16:20:02 INFO device.py L1840: 'peak_ram': 14.28GB, 'peak_vram': 6.11GB[0m

Quantizing model.layers.29: 100%|██████████| 30/30 [17:18<00:00, 34.61s/it]
16:20:10 [INFO] Quantization completed in 1107.9s
16:20:10 [INFO] Saving quantized model (auto_round format)...
[33;1m2026-06-11 16:20:10 WARNING logging.py L340: some layers are skipped quantization (shape not divisible by 32): [0m

packing:   0%|          | 0/3541 [00:00<?, ?it/s]
packing:  14%|█▍        | 498/3541 [02:00<12:13,  4.15it/s]
packing:  14%|█▍        | 498/3541 [02:10<12:13,  4.15it/s]
packing:  28%|██▊       | 1008/3541 [04:00<10:02,  4.21it/s]
packing:  28%|██▊       | 1008/3541 [04:10<10:02,  4.21it/s]
packing:  42%|████▏     | 1495/3541 [06:00<08:14,  4.14it/s]
packing:  42%|████▏     | 1495/3541 [06:10<08:14,  4.14it/s]
packing:  57%|█████▋    | 2016/3541 [08:00<06:01,  4.21it/s]
packing:  57%|█████▋    | 2016/3541 [08:10<06:01,  4.21it/s]
packing:  71%|███████   | 2502/3541 [10:00<04:10,  4.16it/s]
packing:  71%|███████   | 2502/3541 [10:10<04:10,  4.16it/s]
packing:  85%|████████▍ | 2993/3541 [12:00<02:12,  4.13it/s]
packing:  85%|████████▍ | 2993/3541 [12:10<02:12,  4.13it/s]
packing:  99%|█████████▉| 3518/3541 [14:00<00:05,  4.21it/s]
packing: 100%|██████████| 3541/3541 [14:05<00:00,  4.19it/s]
[33;1m2026-06-11 16:34:16 WARNING export.py L344: /root/_work/1/s/auto_quant/output/runs/neo-3-3B-A400M-Base-AutoRound-W4A16-RTN/quantized_model already exists, this may cause model conflict[0m
16:34:20 [INFO] HTTP Request: HEAD https://huggingface.co/aquiffoo/neo-3-3B-A400M-Base/resolve/main/config.json "HTTP/1.1 307 Temporary Redirect"
16:34:20 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/models/aquiffoo/neo-3-3B-A400M-Base/71efb8d31dcb942dac7db2925b2917237d7ee34c/config.json "HTTP/1.1 200 OK"
16:34:20 [INFO] Summary written to /root/_work/1/s/auto_quant/output/runs/neo-3-3B-A400M-Base-AutoRound-W4A16-RTN/quant_summary.json
16:34:20 [INFO] === Phase 2: DONE ===
Agent Analysis

Root Cause:
Bug or limitation in AutoRound library code itself
Affected Component: upgrade
Severity: medium
Retryable: True
Suggested Fix:
pip install auto-round@git+https://github.com/intel/auto-round.git@main; Try scheme=W8A16 if W4A16 fails on specific layers
Workaround:
pip install auto-round@git+https://github.com/intel/auto-round.git@main
Auto-generated by error_analysis pipeline. cc @lvkaokao
lvkaokao changed discussion status to closed 17 days ago
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
· Sign up or log in to comment