| 2025-08-04 08:59:29,418 - INFO - Starting conversion for model: HuggingFaceTB/SmolLM2-360M-Instruct |
| 2025-08-04 08:59:29,418 - INFO - Model folder: model/HuggingFaceTB_SmolLM2-360M-Instruct |
| 2025-08-04 08:59:29,419 - INFO - Log file: model/HuggingFaceTB_SmolLM2-360M-Instruct/conversion_log.txt |
| 2025-08-04 08:59:29,419 - INFO - Model compatibility rules: {'supported_quantized_dtypes': ['w8a8'], 'supported_hybrid_rates': [0], 'notes': 'Only basic w8a8 quantization works, no grouped quantization or hybrid quantization'} |
| 2025-08-04 08:59:29,419 - INFO - Generated 96 parameter combinations |
| 2025-08-04 08:59:29,419 - INFO - Filtered to 12 compatible combinations |
| 2025-08-04 08:59:29,419 - INFO - Processing combination 1/12: w8a8-opt0-hybrid0-npu1-ctx4032-rk3588 |
| 2025-08-04 08:59:29,632 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:00:22,598 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:00:23,284 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:00:23,285 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:00:23,285 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:00:23,285 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:00:26,499 - INFO - Setting max_context_limit to 4032 |
| 2025-08-04 09:00:28,225 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu1-ctx4032-rk3588.rkllm! |
| 2025-08-04 09:00:28,225 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu1-ctx4032-rk3588.rkllm (Time: 58.81s) |
| 2025-08-04 09:00:28,226 - INFO - Processing combination 2/12: w8a8-opt0-hybrid0-npu2-ctx4032-rk3588 |
| 2025-08-04 09:00:28,226 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:01:19,391 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:01:20,077 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:01:20,077 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:01:20,078 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:01:20,078 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:01:23,067 - INFO - Setting max_context_limit to 4032 |
| 2025-08-04 09:01:26,023 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu2-ctx4032-rk3588.rkllm! |
| 2025-08-04 09:01:26,024 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu2-ctx4032-rk3588.rkllm (Time: 57.80s) |
| 2025-08-04 09:01:26,024 - INFO - Processing combination 3/12: w8a8-opt0-hybrid0-npu3-ctx4032-rk3588 |
| 2025-08-04 09:01:26,024 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:02:17,411 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:02:18,093 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:02:18,094 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:02:18,094 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:02:18,094 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:02:20,936 - INFO - Setting max_context_limit to 4032 |
| 2025-08-04 09:02:25,102 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu3-ctx4032-rk3588.rkllm! |
| 2025-08-04 09:02:25,102 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu3-ctx4032-rk3588.rkllm (Time: 59.08s) |
| 2025-08-04 09:02:25,102 - INFO - Processing combination 4/12: w8a8-opt0-hybrid0-npu1-ctx16384-rk3588 |
| 2025-08-04 09:02:25,102 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:03:16,418 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:03:17,109 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:03:17,109 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:03:17,110 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:03:17,110 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:03:19,862 - INFO - Setting max_context_limit to 16384 |
| 2025-08-04 09:03:24,225 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu1-ctx16384-rk3588.rkllm! |
| 2025-08-04 09:03:24,226 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu1-ctx16384-rk3588.rkllm (Time: 59.12s) |
| 2025-08-04 09:03:24,226 - INFO - Processing combination 5/12: w8a8-opt0-hybrid0-npu2-ctx16384-rk3588 |
| 2025-08-04 09:03:24,226 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:04:15,520 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:04:16,205 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:04:16,205 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:04:16,205 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:04:16,205 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:04:19,111 - INFO - Setting max_context_limit to 16384 |
| 2025-08-04 09:04:27,510 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu2-ctx16384-rk3588.rkllm! |
| 2025-08-04 09:04:27,510 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu2-ctx16384-rk3588.rkllm (Time: 63.28s) |
| 2025-08-04 09:04:27,510 - INFO - Processing combination 6/12: w8a8-opt0-hybrid0-npu3-ctx16384-rk3588 |
| 2025-08-04 09:04:27,510 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:05:18,689 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:05:19,373 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:05:19,373 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:05:19,373 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:05:19,373 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:05:22,298 - INFO - Setting max_context_limit to 16384 |
| 2025-08-04 09:05:34,862 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu3-ctx16384-rk3588.rkllm! |
| 2025-08-04 09:05:34,863 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu3-ctx16384-rk3588.rkllm (Time: 67.35s) |
| 2025-08-04 09:05:34,863 - INFO - Processing combination 7/12: w8a8-opt1-hybrid0-npu1-ctx4032-rk3588 |
| 2025-08-04 09:05:34,863 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:06:26,041 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:06:26,723 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:06:26,723 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:06:26,723 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:06:26,723 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:06:29,719 - INFO - Setting max_context_limit to 4032 |
| 2025-08-04 09:06:31,396 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu1-ctx4032-rk3588.rkllm! |
| 2025-08-04 09:06:31,396 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu1-ctx4032-rk3588.rkllm (Time: 56.53s) |
| 2025-08-04 09:06:31,396 - INFO - Processing combination 8/12: w8a8-opt1-hybrid0-npu2-ctx4032-rk3588 |
| 2025-08-04 09:06:31,396 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:07:22,751 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:07:23,428 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:07:23,428 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:07:23,428 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:07:23,428 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:07:26,177 - INFO - Setting max_context_limit to 4032 |
| 2025-08-04 09:07:29,065 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu2-ctx4032-rk3588.rkllm! |
| 2025-08-04 09:07:29,066 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu2-ctx4032-rk3588.rkllm (Time: 57.67s) |
| 2025-08-04 09:07:29,066 - INFO - Processing combination 9/12: w8a8-opt1-hybrid0-npu3-ctx4032-rk3588 |
| 2025-08-04 09:07:29,066 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:08:20,450 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:08:21,138 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:08:21,139 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:08:21,139 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:08:21,139 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:08:23,932 - INFO - Setting max_context_limit to 4032 |
| 2025-08-04 09:08:28,040 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu3-ctx4032-rk3588.rkllm! |
| 2025-08-04 09:08:28,040 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu3-ctx4032-rk3588.rkllm (Time: 58.97s) |
| 2025-08-04 09:08:28,040 - INFO - Processing combination 10/12: w8a8-opt1-hybrid0-npu1-ctx16384-rk3588 |
| 2025-08-04 09:08:28,040 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:09:19,521 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:09:20,198 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:09:20,198 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:09:20,198 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:09:20,198 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:09:22,937 - INFO - Setting max_context_limit to 16384 |
| 2025-08-04 09:09:27,265 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu1-ctx16384-rk3588.rkllm! |
| 2025-08-04 09:09:27,265 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu1-ctx16384-rk3588.rkllm (Time: 59.22s) |
| 2025-08-04 09:09:27,265 - INFO - Processing combination 11/12: w8a8-opt1-hybrid0-npu2-ctx16384-rk3588 |
| 2025-08-04 09:09:27,266 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:10:18,692 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:10:19,384 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:10:19,384 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:10:19,384 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:10:19,384 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:10:22,285 - INFO - Setting max_context_limit to 16384 |
| 2025-08-04 09:10:30,771 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu2-ctx16384-rk3588.rkllm! |
| 2025-08-04 09:10:30,771 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu2-ctx16384-rk3588.rkllm (Time: 63.51s) |
| 2025-08-04 09:10:30,771 - INFO - Processing combination 12/12: w8a8-opt1-hybrid0-npu3-ctx16384-rk3588 |
| 2025-08-04 09:10:30,771 - INFO - rkllm-toolkit version: 1.2.1b1 |
| 2025-08-04 09:11:22,003 - INFO - Setting chat_template to "<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n<|im_start|>user\n[content]<|im_end|>\n<|im_start|>assistant\n" |
| 2025-08-04 09:11:22,673 - INFO - Setting token_id of bos to 1 |
| 2025-08-04 09:11:22,673 - INFO - Setting token_id of eos to 2 |
| 2025-08-04 09:11:22,673 - INFO - Setting token_id of unk to 0 |
| 2025-08-04 09:11:22,673 - INFO - Setting token_id of pad to 2 |
| 2025-08-04 09:11:25,547 - INFO - Setting max_context_limit to 16384 |
| 2025-08-04 09:11:38,079 - INFO - Model has been saved to model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu3-ctx16384-rk3588.rkllm! |
| 2025-08-04 09:11:38,079 - INFO - Successfully exported: model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu3-ctx16384-rk3588.rkllm (Time: 67.31s) |
| 2025-08-04 09:11:38,080 - INFO - Conversion complete! |
| 2025-08-04 09:11:38,081 - INFO - Total time: 728.66s |
| 2025-08-04 09:11:38,081 - INFO - Successful conversions: 12 |
| 2025-08-04 09:11:38,081 - INFO - Failed conversions: 0 |
| 2025-08-04 09:11:38,081 - INFO - Success rate: 100.0% |
| 2025-08-04 09:11:38,081 - INFO - Total combinations processed: 12 |
| 2025-08-04 09:11:38,081 - INFO - Successfully generated models: |
| 2025-08-04 09:11:38,081 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu1-ctx4032-rk3588.rkllm (444.5 MB) |
| 2025-08-04 09:11:38,081 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu2-ctx4032-rk3588.rkllm (449.5 MB) |
| 2025-08-04 09:11:38,081 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu3-ctx4032-rk3588.rkllm (454.5 MB) |
| 2025-08-04 09:11:38,082 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu1-ctx16384-rk3588.rkllm (451.1 MB) |
| 2025-08-04 09:11:38,082 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu2-ctx16384-rk3588.rkllm (462.7 MB) |
| 2025-08-04 09:11:38,082 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt0-hybrid0-npu3-ctx16384-rk3588.rkllm (474.4 MB) |
| 2025-08-04 09:11:38,082 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu1-ctx4032-rk3588.rkllm (444.5 MB) |
| 2025-08-04 09:11:38,082 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu2-ctx4032-rk3588.rkllm (449.5 MB) |
| 2025-08-04 09:11:38,082 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu3-ctx4032-rk3588.rkllm (454.5 MB) |
| 2025-08-04 09:11:38,082 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu1-ctx16384-rk3588.rkllm (451.1 MB) |
| 2025-08-04 09:11:38,082 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu2-ctx16384-rk3588.rkllm (462.7 MB) |
| 2025-08-04 09:11:38,082 - INFO - - model/HuggingFaceTB_SmolLM2-360M-Instruct/HuggingFaceTB_SmolLM2-360M-Instruct-w8a8-opt1-hybrid0-npu3-ctx16384-rk3588.rkllm (474.4 MB) |
|
|
