Instructions to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Use Docker
docker model run hf.co/magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "magiccodingman/Qwen3.8-27B-MagicQuant-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "magiccodingman/Qwen3.8-27B-MagicQuant-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
- Ollama
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Ollama:
ollama run hf.co/magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
- Unsloth Studio
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for magiccodingman/Qwen3.8-27B-MagicQuant-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for magiccodingman/Qwen3.8-27B-MagicQuant-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for magiccodingman/Qwen3.8-27B-MagicQuant-GGUF to start chatting
- Pi
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Docker Model Runner:
docker model run hf.co/magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
- Lemonade
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Qwen3.8-27B-MagicQuant-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
| [ | |
| { | |
| "reasonCode": "STRICT_DOMINANCE", | |
| "reasonDescription": "The winner was no larger and had lower real KLD than the removed anchor.", | |
| "rawReason": "smart baseline-tuning strict dominance fallback: real benchmark validated lower KLD at same-or-smaller size", | |
| "removed": { | |
| "key": "118:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-Q6_K_XL.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-Q6_K_XL", | |
| "shortName": "Unsloth-UD-Q6_K_XL", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-Q6_K_XL", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.001057, | |
| "ppl": 5.611228, | |
| "pplDeltaPercent": 0.06648238965670498, | |
| "sizeBytes": 25963226816, | |
| "sizeGB": 25.963226816, | |
| "sizeGiB": 24.180139243602753 | |
| }, | |
| "winner": { | |
| "key": "118:119:119:119:117:119:119:119:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q6_K_1.gguf", | |
| "displayName": "MQ-Q6_K_1", | |
| "shortName": "MQ-Q6_K_1", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Q6_K", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.001052, | |
| "ppl": 5.611099, | |
| "pplDeltaPercent": 0.06418189924209278, | |
| "sizeBytes": 25941640896, | |
| "sizeGB": 25.941640896, | |
| "sizeGiB": 24.16003578901291 | |
| }, | |
| "deltas": { | |
| "kld": 5.000000000000013E-06, | |
| "sizeBytes": 21585920, | |
| "sizeGB": 0.021585919999999998, | |
| "sizeGiB": 0.02010345458984375, | |
| "removedPplDeltaPercent": 0.06648238965670498, | |
| "winnerPplDeltaPercent": 0.06418189924209278, | |
| "pplDeltaPercentImprovement": 0.002300490414612208 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "117:118:118:118:117:118:118:118:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q6_K.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-Q6_K", | |
| "shortName": "MQ-Unsloth-Q6_K", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-Q6_K", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.001723, | |
| "ppl": 5.609167, | |
| "pplDeltaPercent": 0.029728042799827145, | |
| "sizeBytes": 22905992896, | |
| "sizeGB": 22.905992896000004, | |
| "sizeGiB": 21.33286827802658 | |
| }, | |
| "winner": { | |
| "key": "117:118:1:118:117:118:118:118:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q6_K_2.gguf", | |
| "displayName": "MQ-Q6_K_2", | |
| "shortName": "MQ-Q6_K_2", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Q6_K", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.001518, | |
| "ppl": 5.613591, | |
| "pplDeltaPercent": 0.10862238074008938, | |
| "sizeBytes": 23213909696, | |
| "sizeGB": 23.213909695999998, | |
| "sizeGiB": 21.61963814496994 | |
| }, | |
| "deltas": { | |
| "kld": 0.0002049999999999999, | |
| "sizeBytes": -307916800, | |
| "sizeGB": -0.3079168, | |
| "sizeGiB": -0.2867698669433594, | |
| "removedPplDeltaPercent": 0.029728042799827145, | |
| "winnerPplDeltaPercent": 0.10862238074008938, | |
| "pplDeltaPercentImprovement": -0.07889433794026224 | |
| } | |
| }, | |
| { | |
| "reasonCode": "NEAR_BASELINE_PREMIUM", | |
| "reasonDescription": "The winner used only the configured near-baseline size premium and beat the real linear KLD trade line.", | |
| "rawReason": "smart baseline-tuning near-baseline fallback within \u002B1.5% size premium", | |
| "removed": { | |
| "key": "117:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-Q6_K.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-Q6_K", | |
| "shortName": "Unsloth-Q6_K", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-Q6_K", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.001835, | |
| "ppl": 5.608243, | |
| "pplDeltaPercent": 0.013250111457867824, | |
| "sizeBytes": 22884406976, | |
| "sizeGB": 22.884406975999998, | |
| "sizeGiB": 21.312764823436737 | |
| }, | |
| "winner": { | |
| "key": "117:118:1:118:117:118:118:118:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q6_K_2.gguf", | |
| "displayName": "MQ-Q6_K_2", | |
| "shortName": "MQ-Q6_K_2", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Q6_K", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.001518, | |
| "ppl": 5.613591, | |
| "pplDeltaPercent": 0.10862238074008938, | |
| "sizeBytes": 23213909696, | |
| "sizeGB": 23.213909695999998, | |
| "sizeGiB": 21.61963814496994 | |
| }, | |
| "deltas": { | |
| "kld": 0.000317, | |
| "sizeBytes": -329502720, | |
| "sizeGB": -0.32950271999999997, | |
| "sizeGiB": -0.3068733215332031, | |
| "removedPplDeltaPercent": 0.013250111457867824, | |
| "winnerPplDeltaPercent": 0.10862238074008938, | |
| "pplDeltaPercentImprovement": -0.09537226928222156 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "117:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-Q6_K.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-Q6_K", | |
| "shortName": "Unsloth-Q6_K", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-Q6_K", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.001835, | |
| "ppl": 5.608243, | |
| "pplDeltaPercent": 0.013250111457867824, | |
| "sizeBytes": 22884406976, | |
| "sizeGB": 22.884406975999998, | |
| "sizeGiB": 21.312764823436737 | |
| }, | |
| "winner": { | |
| "key": "117:118:118:118:117:118:118:118:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q6_K.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-Q6_K", | |
| "shortName": "MQ-Unsloth-Q6_K", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-Q6_K", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.001723, | |
| "ppl": 5.609167, | |
| "pplDeltaPercent": 0.029728042799827145, | |
| "sizeBytes": 22905992896, | |
| "sizeGB": 22.905992896000004, | |
| "sizeGiB": 21.33286827802658 | |
| }, | |
| "deltas": { | |
| "kld": 0.00011200000000000012, | |
| "sizeBytes": -21585920, | |
| "sizeGB": -0.021585919999999998, | |
| "sizeGiB": -0.02010345458984375, | |
| "removedPplDeltaPercent": 0.013250111457867824, | |
| "winnerPplDeltaPercent": 0.029728042799827145, | |
| "pplDeltaPercentImprovement": -0.016477931341959323 | |
| } | |
| }, | |
| { | |
| "reasonCode": "NEAR_BASELINE_PREMIUM", | |
| "reasonDescription": "The winner used only the configured near-baseline size premium and beat the real linear KLD trade line.", | |
| "rawReason": "smart baseline-tuning near-baseline fallback within \u002B1.5% size premium", | |
| "removed": { | |
| "key": "116:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-Q5_K_XL.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-Q5_K_XL", | |
| "shortName": "Unsloth-UD-Q5_K_XL", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-Q5_K_XL", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.004133, | |
| "ppl": 5.618565, | |
| "pplDeltaPercent": 0.19732501114579268, | |
| "sizeBytes": 20257253056, | |
| "sizeGB": 20.257253056, | |
| "sizeGiB": 18.86603707075119 | |
| }, | |
| "winner": { | |
| "key": "116:117:117:117:117:1:117:117:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q5_K_1.gguf", | |
| "displayName": "MQ-Q5_K_1", | |
| "shortName": "MQ-Q5_K_1", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Q5_K", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.003955, | |
| "ppl": 5.616773, | |
| "pplDeltaPercent": 0.1653678109674598, | |
| "sizeBytes": 20386768576, | |
| "sizeGB": 20.386768576, | |
| "sizeGiB": 18.986657798290253 | |
| }, | |
| "deltas": { | |
| "kld": 0.0001780000000000002, | |
| "sizeBytes": -129515520, | |
| "sizeGB": -0.12951552000000002, | |
| "sizeGiB": -0.1206207275390625, | |
| "removedPplDeltaPercent": 0.19732501114579268, | |
| "winnerPplDeltaPercent": 0.1653678109674598, | |
| "pplDeltaPercentImprovement": 0.03195720017833287 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "116:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-Q5_K_XL.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-Q5_K_XL", | |
| "shortName": "Unsloth-UD-Q5_K_XL", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-Q5_K_XL", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.004133, | |
| "ppl": 5.618565, | |
| "pplDeltaPercent": 0.19732501114579268, | |
| "sizeBytes": 20257253056, | |
| "sizeGB": 20.257253056, | |
| "sizeGiB": 18.86603707075119 | |
| }, | |
| "winner": { | |
| "key": "116:117:117:117:117:1:117:117:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q5_K_1.gguf", | |
| "displayName": "MQ-Q5_K_1", | |
| "shortName": "MQ-Q5_K_1", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Q5_K", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.003955, | |
| "ppl": 5.616773, | |
| "pplDeltaPercent": 0.1653678109674598, | |
| "sizeBytes": 20386768576, | |
| "sizeGB": 20.386768576, | |
| "sizeGiB": 18.986657798290253 | |
| }, | |
| "deltas": { | |
| "kld": 0.0001780000000000002, | |
| "sizeBytes": -129515520, | |
| "sizeGB": -0.12951552000000002, | |
| "sizeGiB": -0.1206207275390625, | |
| "removedPplDeltaPercent": 0.19732501114579268, | |
| "winnerPplDeltaPercent": 0.1653678109674598, | |
| "pplDeltaPercentImprovement": 0.03195720017833287 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "115:118:116:116:118:118:116:116:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q5_K.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-Q5_K", | |
| "shortName": "MQ-Unsloth-Q5_K_M", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-Q5_K_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.004366, | |
| "ppl": 5.614868, | |
| "pplDeltaPercent": 0.1313954525189565, | |
| "sizeBytes": 20091344576, | |
| "sizeGB": 20.091344576, | |
| "sizeGiB": 18.71152275800705 | |
| }, | |
| "winner": { | |
| "key": "116:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-Q5_K_XL.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-Q5_K_XL", | |
| "shortName": "Unsloth-UD-Q5_K_XL", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-Q5_K_XL", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.004133, | |
| "ppl": 5.618565, | |
| "pplDeltaPercent": 0.19732501114579268, | |
| "sizeBytes": 20257253056, | |
| "sizeGB": 20.257253056, | |
| "sizeGiB": 18.86603707075119 | |
| }, | |
| "deltas": { | |
| "kld": 0.0002329999999999997, | |
| "sizeBytes": -165908480, | |
| "sizeGB": -0.16590848, | |
| "sizeGiB": -0.15451431274414062, | |
| "removedPplDeltaPercent": 0.1313954525189565, | |
| "winnerPplDeltaPercent": 0.19732501114579268, | |
| "pplDeltaPercentImprovement": -0.06592955862683617 | |
| } | |
| }, | |
| { | |
| "reasonCode": "NEAR_BASELINE_PREMIUM", | |
| "reasonDescription": "The winner used only the configured near-baseline size premium and beat the real linear KLD trade line.", | |
| "rawReason": "smart baseline-tuning near-baseline fallback within \u002B1.5% size premium", | |
| "removed": { | |
| "key": "115:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-Q5_K_M.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-Q5_K_M", | |
| "shortName": "Unsloth-Q5_K_M", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-Q5_K_M", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.004814, | |
| "ppl": 5.619185, | |
| "pplDeltaPercent": 0.2083816317432001, | |
| "sizeBytes": 19834054336, | |
| "sizeGB": 19.834054335999998, | |
| "sizeGiB": 18.471902549266815 | |
| }, | |
| "winner": { | |
| "key": "115:118:116:116:118:118:116:116:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q5_K.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-Q5_K", | |
| "shortName": "MQ-Unsloth-Q5_K_M", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-Q5_K_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.004366, | |
| "ppl": 5.614868, | |
| "pplDeltaPercent": 0.1313954525189565, | |
| "sizeBytes": 20091344576, | |
| "sizeGB": 20.091344576, | |
| "sizeGiB": 18.71152275800705 | |
| }, | |
| "deltas": { | |
| "kld": 0.0004479999999999996, | |
| "sizeBytes": -257290240, | |
| "sizeGB": -0.25729024, | |
| "sizeGiB": -0.23962020874023438, | |
| "removedPplDeltaPercent": 0.2083816317432001, | |
| "winnerPplDeltaPercent": 0.1313954525189565, | |
| "pplDeltaPercentImprovement": 0.07698617922424358 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "115:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-Q5_K_M.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-Q5_K_M", | |
| "shortName": "Unsloth-Q5_K_M", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-Q5_K_M", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.004814, | |
| "ppl": 5.619185, | |
| "pplDeltaPercent": 0.2083816317432001, | |
| "sizeBytes": 19834054336, | |
| "sizeGB": 19.834054335999998, | |
| "sizeGiB": 18.471902549266815 | |
| }, | |
| "winner": { | |
| "key": "115:118:116:116:118:118:116:116:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q5_K.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-Q5_K", | |
| "shortName": "MQ-Unsloth-Q5_K_M", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-Q5_K_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.004366, | |
| "ppl": 5.614868, | |
| "pplDeltaPercent": 0.1313954525189565, | |
| "sizeBytes": 20091344576, | |
| "sizeGB": 20.091344576, | |
| "sizeGiB": 18.71152275800705 | |
| }, | |
| "deltas": { | |
| "kld": 0.0004479999999999996, | |
| "sizeBytes": -257290240, | |
| "sizeGB": -0.25729024, | |
| "sizeGiB": -0.23962020874023438, | |
| "removedPplDeltaPercent": 0.2083816317432001, | |
| "winnerPplDeltaPercent": 0.1313954525189565, | |
| "pplDeltaPercentImprovement": 0.07698617922424358 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "114:115:115:115:117:115:115:115:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q5_K_S.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-Q5_K_S", | |
| "shortName": "MQ-Unsloth-Q5_K_S", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-Q5_K_S", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.005255, | |
| "ppl": 5.62576, | |
| "pplDeltaPercent": 0.32563530985287065, | |
| "sizeBytes": 19315295936, | |
| "sizeGB": 19.315295936000002, | |
| "sizeGiB": 17.98877114057541 | |
| }, | |
| "winner": { | |
| "key": "115:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-Q5_K_M.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-Q5_K_M", | |
| "shortName": "Unsloth-Q5_K_M", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-Q5_K_M", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.004814, | |
| "ppl": 5.619185, | |
| "pplDeltaPercent": 0.2083816317432001, | |
| "sizeBytes": 19834054336, | |
| "sizeGB": 19.834054335999998, | |
| "sizeGiB": 18.471902549266815 | |
| }, | |
| "deltas": { | |
| "kld": 0.0004410000000000004, | |
| "sizeBytes": -518758400, | |
| "sizeGB": -0.5187584000000001, | |
| "sizeGiB": -0.48313140869140625, | |
| "removedPplDeltaPercent": 0.32563530985287065, | |
| "winnerPplDeltaPercent": 0.2083816317432001, | |
| "pplDeltaPercentImprovement": 0.11725367810967055 | |
| } | |
| }, | |
| { | |
| "reasonCode": "NEAR_BASELINE_PREMIUM", | |
| "reasonDescription": "The winner used only the configured near-baseline size premium and beat the real linear KLD trade line.", | |
| "rawReason": "smart baseline-tuning near-baseline fallback within \u002B1.5% size premium", | |
| "removed": { | |
| "key": "114:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-Q5_K_S.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-Q5_K_S", | |
| "shortName": "Unsloth-Q5_K_S", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-Q5_K_S", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.005358, | |
| "ppl": 5.624662, | |
| "pplDeltaPercent": 0.3060543914400339, | |
| "sizeBytes": 19270035136, | |
| "sizeGB": 19.270035135999997, | |
| "sizeGiB": 17.946618735790253 | |
| }, | |
| "winner": { | |
| "key": "114:115:115:115:117:115:115:115:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q5_K_S.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-Q5_K_S", | |
| "shortName": "MQ-Unsloth-Q5_K_S", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-Q5_K_S", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.005255, | |
| "ppl": 5.62576, | |
| "pplDeltaPercent": 0.32563530985287065, | |
| "sizeBytes": 19315295936, | |
| "sizeGB": 19.315295936000002, | |
| "sizeGiB": 17.98877114057541 | |
| }, | |
| "deltas": { | |
| "kld": 0.00010299999999999979, | |
| "sizeBytes": -45260800, | |
| "sizeGB": -0.045260800000000004, | |
| "sizeGiB": -0.04215240478515625, | |
| "removedPplDeltaPercent": 0.3060543914400339, | |
| "winnerPplDeltaPercent": 0.32563530985287065, | |
| "pplDeltaPercentImprovement": -0.019580918412836767 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "114:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-Q5_K_S.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-Q5_K_S", | |
| "shortName": "Unsloth-Q5_K_S", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-Q5_K_S", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.005358, | |
| "ppl": 5.624662, | |
| "pplDeltaPercent": 0.3060543914400339, | |
| "sizeBytes": 19270035136, | |
| "sizeGB": 19.270035135999997, | |
| "sizeGiB": 17.946618735790253 | |
| }, | |
| "winner": { | |
| "key": "114:115:115:115:117:115:115:115:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q5_K_S.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-Q5_K_S", | |
| "shortName": "MQ-Unsloth-Q5_K_S", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-Q5_K_S", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.005255, | |
| "ppl": 5.62576, | |
| "pplDeltaPercent": 0.32563530985287065, | |
| "sizeBytes": 19315295936, | |
| "sizeGB": 19.315295936000002, | |
| "sizeGiB": 17.98877114057541 | |
| }, | |
| "deltas": { | |
| "kld": 0.00010299999999999979, | |
| "sizeBytes": -45260800, | |
| "sizeGB": -0.045260800000000004, | |
| "sizeGiB": -0.04215240478515625, | |
| "removedPplDeltaPercent": 0.3060543914400339, | |
| "winnerPplDeltaPercent": 0.32563530985287065, | |
| "pplDeltaPercentImprovement": -0.019580918412836767 | |
| } | |
| }, | |
| { | |
| "reasonCode": "NEAR_BASELINE_PREMIUM", | |
| "reasonDescription": "The winner used only the configured near-baseline size premium and beat the real linear KLD trade line.", | |
| "rawReason": "smart baseline-tuning near-baseline fallback within \u002B1.5% size premium", | |
| "removed": { | |
| "key": "113:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-Q4_K_XL.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-Q4_K_XL", | |
| "shortName": "Unsloth-UD-Q4_K_XL", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-Q4_K_XL", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.009094, | |
| "ppl": 5.610511, | |
| "pplDeltaPercent": 0.05369594293356888, | |
| "sizeBytes": 17963943616, | |
| "sizeGB": 17.963943616, | |
| "sizeGiB": 16.73022621870041 | |
| }, | |
| "winner": { | |
| "key": "113:114:114:117:114:114:114:114:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q4_K_M_1.gguf", | |
| "displayName": "MQ-Q4_K_M_1", | |
| "shortName": "MQ-Q4_K_M_1", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Q4_K_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.008508, | |
| "ppl": 5.611953, | |
| "pplDeltaPercent": 0.07941150245206988, | |
| "sizeBytes": 18105992896, | |
| "sizeGB": 18.105992896000004, | |
| "sizeGiB": 16.862519919872284 | |
| }, | |
| "deltas": { | |
| "kld": 0.0005859999999999997, | |
| "sizeBytes": -142049280, | |
| "sizeGB": -0.14204928, | |
| "sizeGiB": -0.132293701171875, | |
| "removedPplDeltaPercent": 0.05369594293356888, | |
| "winnerPplDeltaPercent": 0.07941150245206988, | |
| "pplDeltaPercentImprovement": -0.025715559518500998 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "113:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-Q4_K_XL.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-Q4_K_XL", | |
| "shortName": "Unsloth-UD-Q4_K_XL", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-Q4_K_XL", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.009094, | |
| "ppl": 5.610511, | |
| "pplDeltaPercent": 0.05369594293356888, | |
| "sizeBytes": 17963943616, | |
| "sizeGB": 17.963943616, | |
| "sizeGiB": 16.73022621870041 | |
| }, | |
| "winner": { | |
| "key": "113:114:114:117:114:114:114:114:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-Q4_K_M_1.gguf", | |
| "displayName": "MQ-Q4_K_M_1", | |
| "shortName": "MQ-Q4_K_M_1", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Q4_K_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.008508, | |
| "ppl": 5.611953, | |
| "pplDeltaPercent": 0.07941150245206988, | |
| "sizeBytes": 18105992896, | |
| "sizeGB": 18.105992896000004, | |
| "sizeGiB": 16.862519919872284 | |
| }, | |
| "deltas": { | |
| "kld": 0.0005859999999999997, | |
| "sizeBytes": -142049280, | |
| "sizeGB": -0.14204928, | |
| "sizeGiB": -0.132293701171875, | |
| "removedPplDeltaPercent": 0.05369594293356888, | |
| "winnerPplDeltaPercent": 0.07941150245206988, | |
| "pplDeltaPercentImprovement": -0.025715559518500998 | |
| } | |
| }, | |
| { | |
| "reasonCode": "STRICT_DOMINANCE", | |
| "reasonDescription": "The winner was no larger and had lower real KLD than the removed anchor.", | |
| "rawReason": "smart baseline-tuning strict dominance fallback: real benchmark validated lower KLD at same-or-smaller size", | |
| "removed": { | |
| "key": "112:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-Q4_K_M.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-Q4_K_M", | |
| "shortName": "Unsloth-Q4_K_M", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-Q4_K_M", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.012959, | |
| "ppl": 5.607668, | |
| "pplDeltaPercent": 0.0029959875167256365, | |
| "sizeBytes": 17106773696, | |
| "sizeGB": 17.106773696, | |
| "sizeGiB": 15.931924521923065 | |
| }, | |
| "winner": { | |
| "key": "112:113:113:110:113:113:113:115:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ4_NL_1.gguf", | |
| "displayName": "MQ-IQ4_NL_1", | |
| "shortName": "MQ-IQ4_NL_1", | |
| "provider": "MagicQuant", | |
| "quantFamily": "IQ4_NL", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.012627, | |
| "ppl": 5.607022, | |
| "pplDeltaPercent": -0.008524297815429346, | |
| "sizeBytes": 17062823616, | |
| "sizeGB": 17.062823616000003, | |
| "sizeGiB": 15.890992820262909 | |
| }, | |
| "deltas": { | |
| "kld": 0.0003320000000000007, | |
| "sizeBytes": 43950080, | |
| "sizeGB": 0.04395008, | |
| "sizeGiB": 0.04093170166015625, | |
| "removedPplDeltaPercent": 0.0029959875167256365, | |
| "winnerPplDeltaPercent": -0.008524297815429346, | |
| "pplDeltaPercentImprovement": 0.011520285332154982 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "107:108:108:110:115:118:108:108:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ4_XS.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-IQ4_XS", | |
| "shortName": "MQ-Unsloth-IQ4_XS", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-IQ4_XS", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.014337, | |
| "ppl": 5.650981, | |
| "pplDeltaPercent": 0.7754079358002655, | |
| "sizeBytes": 15904679616, | |
| "sizeGB": 15.904679616000001, | |
| "sizeGiB": 14.81238716840744 | |
| }, | |
| "winner": { | |
| "key": "109:110:110:110:110:110:113:110:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ4_XS_1.gguf", | |
| "displayName": "MQ-IQ4_XS_1", | |
| "shortName": "MQ-IQ4_XS_1", | |
| "provider": "MagicQuant", | |
| "quantFamily": "IQ4_XS", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.013794, | |
| "ppl": 5.641085, | |
| "pplDeltaPercent": 0.5989300044583223, | |
| "sizeBytes": 16337626816, | |
| "sizeGB": 16.337626816, | |
| "sizeGiB": 15.215600669384003 | |
| }, | |
| "deltas": { | |
| "kld": 0.0005430000000000001, | |
| "sizeBytes": -432947200, | |
| "sizeGB": -0.43294720000000003, | |
| "sizeGiB": -0.4032135009765625, | |
| "removedPplDeltaPercent": 0.7754079358002655, | |
| "winnerPplDeltaPercent": 0.5989300044583223, | |
| "pplDeltaPercentImprovement": 0.17647793134194312 | |
| } | |
| }, | |
| { | |
| "reasonCode": "STRICT_DOMINANCE", | |
| "reasonDescription": "The winner was no larger and had lower real KLD than the removed anchor.", | |
| "rawReason": "smart baseline-tuning strict dominance fallback: real benchmark validated lower KLD at same-or-smaller size", | |
| "removed": { | |
| "key": "109:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-IQ4_NL.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-IQ4_NL", | |
| "shortName": "Unsloth-IQ4_NL", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-IQ4_NL", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.014992, | |
| "ppl": 5.654784, | |
| "pplDeltaPercent": 0.8432278198840897, | |
| "sizeBytes": 16337626816, | |
| "sizeGB": 16.337626816, | |
| "sizeGiB": 15.215600669384003 | |
| }, | |
| "winner": { | |
| "key": "109:110:110:110:110:110:113:110:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ4_XS_1.gguf", | |
| "displayName": "MQ-IQ4_XS_1", | |
| "shortName": "MQ-IQ4_XS_1", | |
| "provider": "MagicQuant", | |
| "quantFamily": "IQ4_XS", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.013794, | |
| "ppl": 5.641085, | |
| "pplDeltaPercent": 0.5989300044583223, | |
| "sizeBytes": 16337626816, | |
| "sizeGB": 16.337626816, | |
| "sizeGiB": 15.215600669384003 | |
| }, | |
| "deltas": { | |
| "kld": 0.0011979999999999994, | |
| "sizeBytes": 0, | |
| "sizeGB": 0, | |
| "sizeGiB": 0, | |
| "removedPplDeltaPercent": 0.8432278198840897, | |
| "winnerPplDeltaPercent": 0.5989300044583223, | |
| "pplDeltaPercentImprovement": 0.24429781542576734 | |
| } | |
| }, | |
| { | |
| "reasonCode": "NEAR_BASELINE_PREMIUM", | |
| "reasonDescription": "The winner used only the configured near-baseline size premium and beat the real linear KLD trade line.", | |
| "rawReason": "smart baseline-tuning near-baseline fallback within \u002B1.5% size premium", | |
| "removed": { | |
| "key": "107:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-IQ4_XS.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-IQ4_XS", | |
| "shortName": "Unsloth-IQ4_XS", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-IQ4_XS", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.015238, | |
| "ppl": 5.666636, | |
| "pplDeltaPercent": 1.054587605884969, | |
| "sizeBytes": 15705859776, | |
| "sizeGB": 15.705859776, | |
| "sizeGiB": 14.627221763134003 | |
| }, | |
| "winner": { | |
| "key": "107:108:108:110:115:118:108:108:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ4_XS.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-IQ4_XS", | |
| "shortName": "MQ-Unsloth-IQ4_XS", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-IQ4_XS", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.014337, | |
| "ppl": 5.650981, | |
| "pplDeltaPercent": 0.7754079358002655, | |
| "sizeBytes": 15904679616, | |
| "sizeGB": 15.904679616000001, | |
| "sizeGiB": 14.81238716840744 | |
| }, | |
| "deltas": { | |
| "kld": 0.000900999999999999, | |
| "sizeBytes": -198819840, | |
| "sizeGB": -0.19881984, | |
| "sizeGiB": -0.1851654052734375, | |
| "removedPplDeltaPercent": 1.054587605884969, | |
| "winnerPplDeltaPercent": 0.7754079358002655, | |
| "pplDeltaPercentImprovement": 0.27917967008470357 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "107:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-IQ4_XS.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-IQ4_XS", | |
| "shortName": "Unsloth-IQ4_XS", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-IQ4_XS", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.015238, | |
| "ppl": 5.666636, | |
| "pplDeltaPercent": 1.054587605884969, | |
| "sizeBytes": 15705859776, | |
| "sizeGB": 15.705859776, | |
| "sizeGiB": 14.627221763134003 | |
| }, | |
| "winner": { | |
| "key": "107:108:108:110:115:118:108:108:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ4_XS.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-IQ4_XS", | |
| "shortName": "MQ-Unsloth-IQ4_XS", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-IQ4_XS", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.014337, | |
| "ppl": 5.650981, | |
| "pplDeltaPercent": 0.7754079358002655, | |
| "sizeBytes": 15904679616, | |
| "sizeGB": 15.904679616000001, | |
| "sizeGiB": 14.81238716840744 | |
| }, | |
| "deltas": { | |
| "kld": 0.000900999999999999, | |
| "sizeBytes": -198819840, | |
| "sizeGB": -0.19881984, | |
| "sizeGiB": -0.1851654052734375, | |
| "removedPplDeltaPercent": 1.054587605884969, | |
| "winnerPplDeltaPercent": 0.7754079358002655, | |
| "pplDeltaPercentImprovement": 0.27917967008470357 | |
| } | |
| }, | |
| { | |
| "reasonCode": "NEAR_BASELINE_PREMIUM", | |
| "reasonDescription": "The winner used only the configured near-baseline size premium and beat the real linear KLD trade line.", | |
| "rawReason": "smart baseline-tuning near-baseline fallback within \u002B1.5% size premium", | |
| "removed": { | |
| "key": "103:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-IQ3_XXS.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-IQ3_XXS", | |
| "shortName": "Unsloth-UD-IQ3_XXS", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-IQ3_XXS", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.058766, | |
| "ppl": 5.876035, | |
| "pplDeltaPercent": 4.788854213107445, | |
| "sizeBytes": 11954476736, | |
| "sizeGB": 11.954476735999998, | |
| "sizeGiB": 11.133474051952362 | |
| }, | |
| "winner": { | |
| "key": "103:104:104:104:109:104:104:104:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ2_M_1.gguf", | |
| "displayName": "MQ-IQ2_M_1", | |
| "shortName": "MQ-IQ2_M_1", | |
| "provider": "MagicQuant", | |
| "quantFamily": "IQ2_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.057811, | |
| "ppl": 5.8607, | |
| "pplDeltaPercent": 4.5153811859117186, | |
| "sizeBytes": 11960702656, | |
| "sizeGB": 11.960702655999999, | |
| "sizeGiB": 11.139272391796112 | |
| }, | |
| "deltas": { | |
| "kld": 0.0009549999999999975, | |
| "sizeBytes": -6225920, | |
| "sizeGB": -0.00622592, | |
| "sizeGiB": -0.00579833984375, | |
| "removedPplDeltaPercent": 4.788854213107445, | |
| "winnerPplDeltaPercent": 4.5153811859117186, | |
| "pplDeltaPercentImprovement": 0.27347302719572664 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "103:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-IQ3_XXS.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-IQ3_XXS", | |
| "shortName": "Unsloth-UD-IQ3_XXS", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-IQ3_XXS", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.058766, | |
| "ppl": 5.876035, | |
| "pplDeltaPercent": 4.788854213107445, | |
| "sizeBytes": 11954476736, | |
| "sizeGB": 11.954476735999998, | |
| "sizeGiB": 11.133474051952362 | |
| }, | |
| "winner": { | |
| "key": "103:104:104:104:109:104:104:104:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ2_M_1.gguf", | |
| "displayName": "MQ-IQ2_M_1", | |
| "shortName": "MQ-IQ2_M_1", | |
| "provider": "MagicQuant", | |
| "quantFamily": "IQ2_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.057811, | |
| "ppl": 5.8607, | |
| "pplDeltaPercent": 4.5153811859117186, | |
| "sizeBytes": 11960702656, | |
| "sizeGB": 11.960702655999999, | |
| "sizeGiB": 11.139272391796112 | |
| }, | |
| "deltas": { | |
| "kld": 0.0009549999999999975, | |
| "sizeBytes": -6225920, | |
| "sizeGB": -0.00622592, | |
| "sizeGiB": -0.00579833984375, | |
| "removedPplDeltaPercent": 4.788854213107445, | |
| "winnerPplDeltaPercent": 4.5153811859117186, | |
| "pplDeltaPercentImprovement": 0.27347302719572664 | |
| } | |
| }, | |
| { | |
| "reasonCode": "NEAR_BASELINE_PREMIUM", | |
| "reasonDescription": "The winner used only the configured near-baseline size premium and beat the real linear KLD trade line.", | |
| "rawReason": "smart baseline-tuning near-baseline fallback within \u002B1.5% size premium", | |
| "removed": { | |
| "key": "102:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-Q2_K_XL.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-Q2_K_XL", | |
| "shortName": "Unsloth-UD-Q2_K_XL", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-Q2_K_XL", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.094036, | |
| "ppl": 6.019528, | |
| "pplDeltaPercent": 7.347802050824794, | |
| "sizeBytes": 10721027776, | |
| "sizeGB": 10.721027776000001, | |
| "sizeGiB": 9.984735190868378 | |
| }, | |
| "winner": { | |
| "key": "102:103:103:103:114:103:103:103:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ2_M_2.gguf", | |
| "displayName": "MQ-IQ2_M_2", | |
| "shortName": "MQ-IQ2_M_2", | |
| "provider": "MagicQuant", | |
| "quantFamily": "IQ2_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.092394, | |
| "ppl": 6.009092, | |
| "pplDeltaPercent": 7.161694159607668, | |
| "sizeBytes": 10769770176, | |
| "sizeGB": 10.769770176000002, | |
| "sizeGiB": 10.030130088329315 | |
| }, | |
| "deltas": { | |
| "kld": 0.0016419999999999907, | |
| "sizeBytes": -48742400, | |
| "sizeGB": -0.048742400000000005, | |
| "sizeGiB": -0.0453948974609375, | |
| "removedPplDeltaPercent": 7.347802050824794, | |
| "winnerPplDeltaPercent": 7.161694159607668, | |
| "pplDeltaPercentImprovement": 0.18610789121712568 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "102:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-Q2_K_XL.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-Q2_K_XL", | |
| "shortName": "Unsloth-UD-Q2_K_XL", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-Q2_K_XL", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.094036, | |
| "ppl": 6.019528, | |
| "pplDeltaPercent": 7.347802050824794, | |
| "sizeBytes": 10721027776, | |
| "sizeGB": 10.721027776000001, | |
| "sizeGiB": 9.984735190868378 | |
| }, | |
| "winner": { | |
| "key": "102:103:103:103:114:103:103:103:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ2_M_2.gguf", | |
| "displayName": "MQ-IQ2_M_2", | |
| "shortName": "MQ-IQ2_M_2", | |
| "provider": "MagicQuant", | |
| "quantFamily": "IQ2_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.092394, | |
| "ppl": 6.009092, | |
| "pplDeltaPercent": 7.161694159607668, | |
| "sizeBytes": 10769770176, | |
| "sizeGB": 10.769770176000002, | |
| "sizeGiB": 10.030130088329315 | |
| }, | |
| "deltas": { | |
| "kld": 0.0016419999999999907, | |
| "sizeBytes": -48742400, | |
| "sizeGB": -0.048742400000000005, | |
| "sizeGiB": -0.0453948974609375, | |
| "removedPplDeltaPercent": 7.347802050824794, | |
| "winnerPplDeltaPercent": 7.161694159607668, | |
| "pplDeltaPercentImprovement": 0.18610789121712568 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "101:102:102:102:113:102:102:102:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ2_M.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-IQ2_M", | |
| "shortName": "MQ-Unsloth-UD-IQ2_M", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-UD-IQ2_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.108218, | |
| "ppl": 6.097877, | |
| "pplDeltaPercent": 8.745020062416415, | |
| "sizeBytes": 10391299776, | |
| "sizeGB": 10.391299776, | |
| "sizeGiB": 9.677652060985565 | |
| }, | |
| "winner": { | |
| "key": "102:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-Q2_K_XL.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-Q2_K_XL", | |
| "shortName": "Unsloth-UD-Q2_K_XL", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-Q2_K_XL", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.094036, | |
| "ppl": 6.019528, | |
| "pplDeltaPercent": 7.347802050824794, | |
| "sizeBytes": 10721027776, | |
| "sizeGB": 10.721027776000001, | |
| "sizeGiB": 9.984735190868378 | |
| }, | |
| "deltas": { | |
| "kld": 0.014182, | |
| "sizeBytes": -329728000, | |
| "sizeGB": -0.329728, | |
| "sizeGiB": -0.3070831298828125, | |
| "removedPplDeltaPercent": 8.745020062416415, | |
| "winnerPplDeltaPercent": 7.347802050824794, | |
| "pplDeltaPercentImprovement": 1.3972180115916215 | |
| } | |
| }, | |
| { | |
| "reasonCode": "NEAR_BASELINE_PREMIUM", | |
| "reasonDescription": "The winner used only the configured near-baseline size premium and beat the real linear KLD trade line.", | |
| "rawReason": "smart baseline-tuning near-baseline fallback within \u002B1.5% size premium", | |
| "removed": { | |
| "key": "101:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-IQ2_M.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-IQ2_M", | |
| "shortName": "Unsloth-UD-IQ2_M", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-IQ2_M", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.109431, | |
| "ppl": 6.109936, | |
| "pplDeltaPercent": 8.960071333036119, | |
| "sizeBytes": 10364511936, | |
| "sizeGB": 10.364511936000001, | |
| "sizeGiB": 9.652703940868378 | |
| }, | |
| "winner": { | |
| "key": "101:102:102:102:113:102:102:102:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ2_M.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-IQ2_M", | |
| "shortName": "MQ-Unsloth-UD-IQ2_M", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-UD-IQ2_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.108218, | |
| "ppl": 6.097877, | |
| "pplDeltaPercent": 8.745020062416415, | |
| "sizeBytes": 10391299776, | |
| "sizeGB": 10.391299776, | |
| "sizeGiB": 9.677652060985565 | |
| }, | |
| "deltas": { | |
| "kld": 0.0012130000000000057, | |
| "sizeBytes": -26787840, | |
| "sizeGB": -0.02678784, | |
| "sizeGiB": -0.0249481201171875, | |
| "removedPplDeltaPercent": 8.960071333036119, | |
| "winnerPplDeltaPercent": 8.745020062416415, | |
| "pplDeltaPercentImprovement": 0.21505127061970342 | |
| } | |
| }, | |
| { | |
| "reasonCode": "SPACING_COLLAPSE", | |
| "reasonDescription": "Two candidates were too close in practical output space; the stronger one was kept.", | |
| "rawReason": "meaningful spacing collapse; size gap below 599,772,979 bytes", | |
| "removed": { | |
| "key": "101:0:0:0:0:0:0:0:0:0", | |
| "fileName": "Qwen3.8-27B-UD-Unsloth-UD-IQ2_M.gguf", | |
| "displayName": "Qwen3.8-27B-UD-Unsloth-UD-IQ2_M", | |
| "shortName": "Unsloth-UD-IQ2_M", | |
| "provider": "Unsloth", | |
| "quantFamily": "Unsloth-UD-IQ2_M", | |
| "isHybrid": false, | |
| "isExternalPureBaseline": true, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": false, | |
| "kld": 0.109431, | |
| "ppl": 6.109936, | |
| "pplDeltaPercent": 8.960071333036119, | |
| "sizeBytes": 10364511936, | |
| "sizeGB": 10.364511936000001, | |
| "sizeGiB": 9.652703940868378 | |
| }, | |
| "winner": { | |
| "key": "101:102:102:102:113:102:102:102:0:0", | |
| "fileName": "Qwen3.8-27B-MQ-IQ2_M.gguf", | |
| "displayName": "Qwen3.8-27B-MQ-IQ2_M", | |
| "shortName": "MQ-Unsloth-UD-IQ2_M", | |
| "provider": "MagicQuant", | |
| "quantFamily": "Unsloth-UD-IQ2_M", | |
| "isHybrid": true, | |
| "isExternalPureBaseline": false, | |
| "isExternalRebuiltBaseline": false, | |
| "isMaterializedTensorMapped": true, | |
| "kld": 0.108218, | |
| "ppl": 6.097877, | |
| "pplDeltaPercent": 8.745020062416415, | |
| "sizeBytes": 10391299776, | |
| "sizeGB": 10.391299776, | |
| "sizeGiB": 9.677652060985565 | |
| }, | |
| "deltas": { | |
| "kld": 0.0012130000000000057, | |
| "sizeBytes": -26787840, | |
| "sizeGB": -0.02678784, | |
| "sizeGiB": -0.0249481201171875, | |
| "removedPplDeltaPercent": 8.960071333036119, | |
| "winnerPplDeltaPercent": 8.745020062416415, | |
| "pplDeltaPercentImprovement": 0.21505127061970342 | |
| } | |
| } | |
| ] |