Instructions to use magiccodingman/Qwen3.6-27B-MagicQuant-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use magiccodingman/Qwen3.6-27B-MagicQuant-GGUF with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="magiccodingman/Qwen3.6-27B-MagicQuant-GGUF", filename="Qwen3.6-27B-LM-IQ2_M.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use magiccodingman/Qwen3.6-27B-MagicQuant-GGUF with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M # Run inference directly in the terminal: llama-cli -hf magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M # Run inference directly in the terminal: llama-cli -hf magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M # Run inference directly in the terminal: ./llama-cli -hf magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
Use Docker
docker model run hf.co/magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
- LM Studio
- Jan
- vLLM
How to use magiccodingman/Qwen3.6-27B-MagicQuant-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "magiccodingman/Qwen3.6-27B-MagicQuant-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "magiccodingman/Qwen3.6-27B-MagicQuant-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
- Ollama
How to use magiccodingman/Qwen3.6-27B-MagicQuant-GGUF with Ollama:
ollama run hf.co/magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
- Unsloth Studio new
How to use magiccodingman/Qwen3.6-27B-MagicQuant-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for magiccodingman/Qwen3.6-27B-MagicQuant-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for magiccodingman/Qwen3.6-27B-MagicQuant-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for magiccodingman/Qwen3.6-27B-MagicQuant-GGUF to start chatting
- Pi new
How to use magiccodingman/Qwen3.6-27B-MagicQuant-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use magiccodingman/Qwen3.6-27B-MagicQuant-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
Run Hermes
hermes
- Docker Model Runner
How to use magiccodingman/Qwen3.6-27B-MagicQuant-GGUF with Docker Model Runner:
docker model run hf.co/magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
- Lemonade
How to use magiccodingman/Qwen3.6-27B-MagicQuant-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull magiccodingman/Qwen3.6-27B-MagicQuant-GGUF:IQ2_M
Run and chat with the model
lemonade run user.Qwen3.6-27B-MagicQuant-GGUF-IQ2_M
List all available models
lemonade list
| [ | |
| { | |
| "key": "baseonly:0", | |
| "description": "Base-only isolation for Q8_0 with all active groups forced native.", | |
| "kind": "BaseOnlyIsolation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:202:0:0", | |
| "group": null, | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": null, | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": null, | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50989659840, | |
| "sizeGB": 50.98965984, | |
| "sizeGiB": 47.48782128095627, | |
| "kld": 0.001094, | |
| "ppl": 5.672405, | |
| "pplDeltaPercent": -0.2776801097007802, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001094, | |
| "Ppl": 5.672405, | |
| "PplError": 0.109034 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "carrier-baseonly:0", | |
| "description": "Carrier base-only isolation on Q8 with all active groups forced native.", | |
| "kind": "BaseOnlyIsolation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:202:0:0", | |
| "group": null, | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": null, | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": null, | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50989659840, | |
| "sizeGB": 50.98965984, | |
| "sizeGiB": 47.48782128095627, | |
| "kld": 0.001094, | |
| "ppl": 5.672405, | |
| "pplDeltaPercent": -0.2776801097007802, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001094, | |
| "Ppl": 5.672405, | |
| "PplError": 0.109034 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "probe:0:0:12", | |
| "description": "Smallest-first probe for group \u0027embeddings\u0027 using \u0027IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationProbe", | |
| "tensorConfigKey": "0:13:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xxs", | |
| "isSmallestProbe": true, | |
| "sizeBytes": 48864040640, | |
| "sizeGB": 48.86404064, | |
| "sizeGiB": 45.508184134960175, | |
| "kld": 0.011315, | |
| "ppl": 5.737331, | |
| "pplDeltaPercent": 0.8637354523399323, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.011315, | |
| "Ppl": 5.737331, | |
| "PplError": 0.111283 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 13, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "probe:0:1:12", | |
| "description": "Smallest-first probe for group \u0027lm_head\u0027 using \u0027IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationProbe", | |
| "tensorConfigKey": "0:202:13:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xxs", | |
| "isSmallestProbe": true, | |
| "sizeBytes": 49320949440, | |
| "sizeGB": 49.32094944, | |
| "sizeGiB": 45.93371361494064, | |
| "kld": 0.001913, | |
| "ppl": 5.672279, | |
| "pplDeltaPercent": -0.27989522168701025, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001913, | |
| "Ppl": 5.672279, | |
| "PplError": 0.10901 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 13, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "probe:0:2:12", | |
| "description": "Smallest-first probe for group \u0027attn_q\u0027 using \u0027IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationProbe", | |
| "tensorConfigKey": "0:202:202:13:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xxs", | |
| "isSmallestProbe": true, | |
| "sizeBytes": 45618329280, | |
| "sizeGB": 45.61832928, | |
| "sizeGiB": 42.48537987470627, | |
| "kld": 0.025593, | |
| "ppl": 5.747106, | |
| "pplDeltaPercent": 1.0355824338103348, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.025593, | |
| "Ppl": 5.747106, | |
| "PplError": 0.110843 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 13, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "probe:0:3:12", | |
| "description": "Smallest-first probe for group \u0027attn_kv\u0027 using \u0027IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationProbe", | |
| "tensorConfigKey": "0:202:202:202:13:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xxs", | |
| "isSmallestProbe": true, | |
| "sizeBytes": 50722928320, | |
| "sizeGB": 50.72292832, | |
| "sizeGiB": 47.23940819501877, | |
| "kld": 0.013368, | |
| "ppl": 5.695146, | |
| "pplDeltaPercent": 0.12211244330368339, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.013368, | |
| "Ppl": 5.695146, | |
| "PplError": 0.109352 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 13, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "probe:0:4:12", | |
| "description": "Smallest-first probe for group \u0027attn_output\u0027 using \u0027IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationProbe", | |
| "tensorConfigKey": "0:202:202:202:202:13:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xxs", | |
| "isSmallestProbe": true, | |
| "sizeBytes": 50112788160, | |
| "sizeGB": 50.112788159999994, | |
| "sizeGiB": 46.67117089033127, | |
| "kld": 0.025985, | |
| "ppl": 5.759142, | |
| "pplDeltaPercent": 1.2471783692556453, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.025985, | |
| "Ppl": 5.759142, | |
| "PplError": 0.111838 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 13, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "probe:0:5:12", | |
| "description": "Smallest-first probe for group \u0027ffn_up_gate\u0027 using \u0027IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationProbe", | |
| "tensorConfigKey": "0:202:202:202:202:202:13:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xxs", | |
| "isSmallestProbe": true, | |
| "sizeBytes": 31113901760, | |
| "sizeGB": 31.11390176, | |
| "sizeGiB": 28.97707909345627, | |
| "kld": 0.138941, | |
| "ppl": 6.152836, | |
| "pplDeltaPercent": 8.168418831967927, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.138941, | |
| "Ppl": 6.152836, | |
| "PplError": 0.118876 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 13, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "probe:0:6:12", | |
| "description": "Smallest-first probe for group \u0027ffn_down\u0027 using \u0027IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationProbe", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:13:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xxs", | |
| "isSmallestProbe": true, | |
| "sizeBytes": 41101915840, | |
| "sizeGB": 41.101915840000004, | |
| "sizeGiB": 38.27914208173752, | |
| "kld": 0.087734, | |
| "ppl": 5.877366, | |
| "pplDeltaPercent": 3.3255863014661964, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.087734, | |
| "Ppl": 5.877366, | |
| "PplError": 0.112173 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 13, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:0", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027Q8_0\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:1:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q8_0", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q8_0", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49797723840, | |
| "sizeGB": 49.79772384, | |
| "sizeGiB": 46.37774437665939, | |
| "kld": 0.001412, | |
| "ppl": 5.677291, | |
| "pplDeltaPercent": -0.1917829893463633, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001412, | |
| "Ppl": 5.677291, | |
| "PplError": 0.109214 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 1, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:1", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027Q6_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:2:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q6_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q6_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001442, | |
| "ppl": 5.680834, | |
| "pplDeltaPercent": -0.12949614992440853, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001442, | |
| "Ppl": 5.680834, | |
| "PplError": 0.109351 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 2, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:2", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027Q5_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:3:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49320949440, | |
| "sizeGB": 49.32094944, | |
| "sizeGiB": 45.93371361494064, | |
| "kld": 0.001848, | |
| "ppl": 5.678225, | |
| "pplDeltaPercent": -0.17536303224218286, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001848, | |
| "Ppl": 5.678225, | |
| "PplError": 0.109241 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 3, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:3", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027Q4_K_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:4:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49162024640, | |
| "sizeGB": 49.162024640000006, | |
| "sizeGiB": 45.78570336103439, | |
| "kld": 0.002805, | |
| "ppl": 5.688145, | |
| "pplDeltaPercent": -0.0009669139622478528, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002805, | |
| "Ppl": 5.688145, | |
| "PplError": 0.109668 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 4, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:5", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027IQ4_NL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:6:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_NL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_nl", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49162024640, | |
| "sizeGB": 49.162024640000006, | |
| "sizeGiB": 45.78570336103439, | |
| "kld": 0.001695, | |
| "ppl": 5.694757, | |
| "pplDeltaPercent": 0.1152737245525812, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001695, | |
| "Ppl": 5.694757, | |
| "PplError": 0.109851 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 6, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:6", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027IQ4_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:7:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49122293440, | |
| "sizeGB": 49.12229343999999, | |
| "sizeGiB": 45.74870079755783, | |
| "kld": 0.003366, | |
| "ppl": 5.712073, | |
| "pplDeltaPercent": 0.419693400372702, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.003366, | |
| "Ppl": 5.712073, | |
| "PplError": 0.11048 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 7, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:7", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027IQ3_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:8:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 48993167040, | |
| "sizeGB": 48.99316704, | |
| "sizeGiB": 45.628442466259, | |
| "kld": 0.008128, | |
| "ppl": 5.629306, | |
| "pplDeltaPercent": -1.0353714707640456, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.008128, | |
| "Ppl": 5.629306, | |
| "PplError": 0.107648 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 8, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:8", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027IQ3_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:9:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 48993167040, | |
| "sizeGB": 48.99316704, | |
| "sizeGiB": 45.628442466259, | |
| "kld": 0.008128, | |
| "ppl": 5.629306, | |
| "pplDeltaPercent": -1.0353714707640456, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.008128, | |
| "Ppl": 5.629306, | |
| "PplError": 0.107648 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 9, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:9", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:10:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xxs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 48993167040, | |
| "sizeGB": 48.99316704, | |
| "sizeGiB": 45.628442466259, | |
| "kld": 0.008128, | |
| "ppl": 5.629306, | |
| "pplDeltaPercent": -1.0353714707640456, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.008128, | |
| "Ppl": 5.629306, | |
| "PplError": 0.107648 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 10, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:10", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027IQ2_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:11:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 48993167040, | |
| "sizeGB": 48.99316704, | |
| "sizeGiB": 45.628442466259, | |
| "kld": 0.008128, | |
| "ppl": 5.629306, | |
| "pplDeltaPercent": -1.0353714707640456, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.008128, | |
| "Ppl": 5.629306, | |
| "PplError": 0.107648 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 11, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:11", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027IQ2_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:12:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 48864040640, | |
| "sizeGB": 48.86404064, | |
| "sizeGiB": 45.508184134960175, | |
| "kld": 0.011315, | |
| "ppl": 5.737331, | |
| "pplDeltaPercent": 0.8637354523399323, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.011315, | |
| "Ppl": 5.737331, | |
| "PplError": 0.111283 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 12, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:13", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027Q5_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:14:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49320949440, | |
| "sizeGB": 49.32094944, | |
| "sizeGiB": 45.93371361494064, | |
| "kld": 0.001848, | |
| "ppl": 5.678225, | |
| "pplDeltaPercent": -0.17536303224218286, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001848, | |
| "Ppl": 5.678225, | |
| "PplError": 0.109241 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 14, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:14", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027Q4_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:15:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49162024640, | |
| "sizeGB": 49.162024640000006, | |
| "sizeGiB": 45.78570336103439, | |
| "kld": 0.002805, | |
| "ppl": 5.688145, | |
| "pplDeltaPercent": -0.0009669139622478528, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002805, | |
| "Ppl": 5.688145, | |
| "PplError": 0.109668 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 15, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:16", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:17:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 48993167040, | |
| "sizeGB": 48.99316704, | |
| "sizeGiB": 45.628442466259, | |
| "kld": 0.008128, | |
| "ppl": 5.629306, | |
| "pplDeltaPercent": -1.0353714707640456, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.008128, | |
| "Ppl": 5.629306, | |
| "PplError": 0.107648 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 17, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:17", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027IQ3_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:18:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 48993167040, | |
| "sizeGB": 48.99316704, | |
| "sizeGiB": 45.628442466259, | |
| "kld": 0.008128, | |
| "ppl": 5.629306, | |
| "pplDeltaPercent": -1.0353714707640456, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.008128, | |
| "Ppl": 5.629306, | |
| "PplError": 0.107648 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 18, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:100", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027UD-IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:101:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_m.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 48993167040, | |
| "sizeGB": 48.99316704, | |
| "sizeGiB": 45.628442466259, | |
| "kld": 0.002693, | |
| "ppl": 5.698149, | |
| "pplDeltaPercent": 0.1749059456418509, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002693, | |
| "Ppl": 5.698149, | |
| "PplError": 0.109845 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 101, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:101", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027UD-IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:102:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 48993167040, | |
| "sizeGB": 48.99316704, | |
| "sizeGiB": 45.628442466259, | |
| "kld": 0.002693, | |
| "ppl": 5.698149, | |
| "pplDeltaPercent": 0.1749059456418509, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002693, | |
| "Ppl": 5.698149, | |
| "PplError": 0.109845 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 102, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:102", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027UD-IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:103:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq3_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 48993167040, | |
| "sizeGB": 48.99316704, | |
| "sizeGiB": 45.628442466259, | |
| "kld": 0.002693, | |
| "ppl": 5.698149, | |
| "pplDeltaPercent": 0.1749059456418509, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002693, | |
| "Ppl": 5.698149, | |
| "PplError": 0.109845 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 103, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:103", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027UD-Q2_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:104:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q2_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q2_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49162024640, | |
| "sizeGB": 49.162024640000006, | |
| "sizeGiB": 45.78570336103439, | |
| "kld": 0.002805, | |
| "ppl": 5.688145, | |
| "pplDeltaPercent": -0.0009669139622478528, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002805, | |
| "Ppl": 5.688145, | |
| "PplError": 0.109668 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 104, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:104", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027UD-Q3_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:105:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q3_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q3_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49162024640, | |
| "sizeGB": 49.162024640000006, | |
| "sizeGiB": 45.78570336103439, | |
| "kld": 0.002805, | |
| "ppl": 5.688145, | |
| "pplDeltaPercent": -0.0009669139622478528, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002805, | |
| "Ppl": 5.688145, | |
| "PplError": 0.109668 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 105, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:105", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027UD-Q4_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:106:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q4_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q4_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49162024640, | |
| "sizeGB": 49.162024640000006, | |
| "sizeGiB": 45.78570336103439, | |
| "kld": 0.002805, | |
| "ppl": 5.688145, | |
| "pplDeltaPercent": -0.0009669139622478528, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002805, | |
| "Ppl": 5.688145, | |
| "PplError": 0.109668 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 106, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:106", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027UD-Q5_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:107:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q5_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q5_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49320949440, | |
| "sizeGB": 49.32094944, | |
| "sizeGiB": 45.93371361494064, | |
| "kld": 0.001848, | |
| "ppl": 5.678225, | |
| "pplDeltaPercent": -0.17536303224218286, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001848, | |
| "Ppl": 5.678225, | |
| "PplError": 0.109241 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 107, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:0:107", | |
| "description": "Continuation isolation for group \u0027embeddings\u0027 using \u0027UD-Q6_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:108:202:202:202:202:202:202:0:0", | |
| "group": "embeddings", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q6_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q6_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49797723840, | |
| "sizeGB": 49.79772384, | |
| "sizeGiB": 46.37774437665939, | |
| "kld": 0.001412, | |
| "ppl": 5.677291, | |
| "pplDeltaPercent": -0.1917829893463633, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001412, | |
| "Ppl": 5.677291, | |
| "PplError": 0.109214 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 108, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:0", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027Q8_0\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:1:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q8_0", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q8_0", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49797723840, | |
| "sizeGB": 49.79772384, | |
| "sizeGiB": 46.37774437665939, | |
| "kld": 0.001031, | |
| "ppl": 5.670186, | |
| "pplDeltaPercent": -0.31669069301360664, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001031, | |
| "Ppl": 5.670186, | |
| "PplError": 0.108996 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 1, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:1", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027Q6_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:2:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q6_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q6_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 2, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:2", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027Q5_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:3:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 3, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:3", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027Q4_K_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:4:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 4, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:5", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027IQ4_NL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:6:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_NL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_nl", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 6, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:6", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027IQ4_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:7:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 7, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:7", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027IQ3_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:8:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 8, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:8", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027IQ3_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:9:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 9, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:9", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:10:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xxs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49320949440, | |
| "sizeGB": 49.32094944, | |
| "sizeGiB": 45.93371361494064, | |
| "kld": 0.001913, | |
| "ppl": 5.672279, | |
| "pplDeltaPercent": -0.27989522168701025, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001913, | |
| "Ppl": 5.672279, | |
| "PplError": 0.10901 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 10, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:10", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027IQ2_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:11:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49320949440, | |
| "sizeGB": 49.32094944, | |
| "sizeGiB": 45.93371361494064, | |
| "kld": 0.001913, | |
| "ppl": 5.672279, | |
| "pplDeltaPercent": -0.27989522168701025, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001913, | |
| "Ppl": 5.672279, | |
| "PplError": 0.10901 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 11, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:11", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027IQ2_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:12:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49320949440, | |
| "sizeGB": 49.32094944, | |
| "sizeGiB": 45.93371361494064, | |
| "kld": 0.001913, | |
| "ppl": 5.672279, | |
| "pplDeltaPercent": -0.27989522168701025, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001913, | |
| "Ppl": 5.672279, | |
| "PplError": 0.10901 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 12, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:13", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027Q5_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:14:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 14, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:14", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027Q4_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:15:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 15, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:16", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:17:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49320949440, | |
| "sizeGB": 49.32094944, | |
| "sizeGiB": 45.93371361494064, | |
| "kld": 0.001913, | |
| "ppl": 5.672279, | |
| "pplDeltaPercent": -0.27989522168701025, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001913, | |
| "Ppl": 5.672279, | |
| "PplError": 0.10901 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 17, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:17", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027IQ3_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:18:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 18, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:100", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027UD-IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:101:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_m.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49320949440, | |
| "sizeGB": 49.32094944, | |
| "sizeGiB": 45.93371361494064, | |
| "kld": 0.001913, | |
| "ppl": 5.672279, | |
| "pplDeltaPercent": -0.27989522168701025, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001913, | |
| "Ppl": 5.672279, | |
| "PplError": 0.10901 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 101, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:101", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027UD-IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:102:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49320949440, | |
| "sizeGB": 49.32094944, | |
| "sizeGiB": 45.93371361494064, | |
| "kld": 0.001913, | |
| "ppl": 5.672279, | |
| "pplDeltaPercent": -0.27989522168701025, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001913, | |
| "Ppl": 5.672279, | |
| "PplError": 0.10901 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 102, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:102", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027UD-IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:103:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq3_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49320949440, | |
| "sizeGB": 49.32094944, | |
| "sizeGiB": 45.93371361494064, | |
| "kld": 0.001913, | |
| "ppl": 5.672279, | |
| "pplDeltaPercent": -0.27989522168701025, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001913, | |
| "Ppl": 5.672279, | |
| "PplError": 0.10901 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 103, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:103", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027UD-Q2_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:104:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q2_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q2_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 104, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:104", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027UD-Q3_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:105:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q3_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q3_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 105, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:105", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027UD-Q4_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:106:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q4_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q4_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 106, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:106", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027UD-Q5_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:107:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q5_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q5_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49489807040, | |
| "sizeGB": 49.48980704, | |
| "sizeGiB": 46.090974509716034, | |
| "kld": 0.001249, | |
| "ppl": 5.671991, | |
| "pplDeltaPercent": -0.2849583347983531, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001249, | |
| "Ppl": 5.671991, | |
| "PplError": 0.109049 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 107, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:1:107", | |
| "description": "Continuation isolation for group \u0027lm_head\u0027 using \u0027UD-Q6_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:108:202:202:202:202:202:0:0", | |
| "group": "lm_head", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q6_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q6_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 49797723840, | |
| "sizeGB": 49.79772384, | |
| "sizeGiB": 46.37774437665939, | |
| "kld": 0.001031, | |
| "ppl": 5.670186, | |
| "pplDeltaPercent": -0.31669069301360664, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001031, | |
| "Ppl": 5.670186, | |
| "PplError": 0.108996 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 108, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:0", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027Q8_0\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:1:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q8_0", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q8_0", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 47686645440, | |
| "sizeGB": 47.68664544, | |
| "sizeGiB": 44.41164940595627, | |
| "kld": 0.002882, | |
| "ppl": 5.685283, | |
| "pplDeltaPercent": -0.051281600506312336, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002882, | |
| "Ppl": 5.685283, | |
| "PplError": 0.109495 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 1, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:1", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027Q6_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:2:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q6_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q6_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 46833366720, | |
| "sizeGB": 46.83336672, | |
| "sizeGiB": 43.61697167158127, | |
| "kld": 0.004061, | |
| "ppl": 5.688662, | |
| "pplDeltaPercent": 0.0081220772827914, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.004061, | |
| "Ppl": 5.688662, | |
| "PplError": 0.109592 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 2, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:2", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027Q5_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:3:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 46532556480, | |
| "sizeGB": 46.53255648, | |
| "sizeGiB": 43.33682030439377, | |
| "kld": 0.006678, | |
| "ppl": 5.688812, | |
| "pplDeltaPercent": 0.010759115361630727, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.006678, | |
| "Ppl": 5.688812, | |
| "PplError": 0.109708 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 3, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:3", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027Q4_K_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:4:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 46249440960, | |
| "sizeGB": 46.24944096, | |
| "sizeGiB": 43.07314842939377, | |
| "kld": 0.008189, | |
| "ppl": 5.699906, | |
| "pplDeltaPercent": 0.20579445167188595, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.008189, | |
| "Ppl": 5.699906, | |
| "PplError": 0.109963 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 4, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:5", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027IQ4_NL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:6:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_NL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_nl", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 46239610560, | |
| "sizeGB": 46.23961056, | |
| "sizeGiB": 43.06399315595627, | |
| "kld": 0.006558, | |
| "ppl": 5.680577, | |
| "pplDeltaPercent": -0.13401427516612835, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.006558, | |
| "Ppl": 5.680577, | |
| "PplError": 0.109252 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 6, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:6", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027IQ4_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:7:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 46208153280, | |
| "sizeGB": 46.20815328, | |
| "sizeGiB": 43.03469628095627, | |
| "kld": 0.004185, | |
| "ppl": 5.674899, | |
| "pplDeltaPercent": -0.233834956576777, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.004185, | |
| "Ppl": 5.674899, | |
| "PplError": 0.109076 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 7, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:7", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027IQ3_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:8:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45791344320, | |
| "sizeGB": 45.79134432, | |
| "sizeGiB": 42.64651268720627, | |
| "kld": 0.01176, | |
| "ppl": 5.71396, | |
| "pplDeltaPercent": 0.4528673394043811, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.01176, | |
| "Ppl": 5.71396, | |
| "PplError": 0.110101 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 8, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:8", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027IQ3_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:9:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45744158400, | |
| "sizeGB": 45.7441584, | |
| "sizeGiB": 42.60256737470627, | |
| "kld": 0.016045, | |
| "ppl": 5.752761, | |
| "pplDeltaPercent": 1.1349987693822197, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.016045, | |
| "Ppl": 5.752761, | |
| "PplError": 0.111474 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 9, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:9", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:10:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xxs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45681243840, | |
| "sizeGB": 45.68124384, | |
| "sizeGiB": 42.54397362470627, | |
| "kld": 0.020084, | |
| "ppl": 5.7699, | |
| "pplDeltaPercent": 1.4363067402693235, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.020084, | |
| "Ppl": 5.7699, | |
| "PplError": 0.111783 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 10, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:10", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027IQ2_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:11:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45649786560, | |
| "sizeGB": 45.64978656, | |
| "sizeGiB": 42.51467674970627, | |
| "kld": 0.02375, | |
| "ppl": 5.737253, | |
| "pplDeltaPercent": 0.8623641925389366, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.02375, | |
| "Ppl": 5.737253, | |
| "PplError": 0.110401 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 11, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:11", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027IQ2_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:12:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45649786560, | |
| "sizeGB": 45.64978656, | |
| "sizeGiB": 42.51467674970627, | |
| "kld": 0.02375, | |
| "ppl": 5.737253, | |
| "pplDeltaPercent": 0.8623641925389366, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.02375, | |
| "Ppl": 5.737253, | |
| "PplError": 0.110401 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 12, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:13", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027Q5_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:14:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 46365439680, | |
| "sizeGB": 46.36543968, | |
| "sizeGiB": 43.18118065595627, | |
| "kld": 0.006444, | |
| "ppl": 5.683781, | |
| "pplDeltaPercent": -0.07768714180233458, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.006444, | |
| "Ppl": 5.683781, | |
| "PplError": 0.109507 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 14, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:14", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027Q4_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:15:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45944698560, | |
| "sizeGB": 45.944698560000006, | |
| "sizeGiB": 42.78933495283127, | |
| "kld": 0.010534, | |
| "ppl": 5.695671, | |
| "pplDeltaPercent": 0.131342076579582, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.010534, | |
| "Ppl": 5.695671, | |
| "PplError": 0.10995 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 15, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:16", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:17:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45681243840, | |
| "sizeGB": 45.68124384, | |
| "sizeGiB": 42.54397362470627, | |
| "kld": 0.020084, | |
| "ppl": 5.7699, | |
| "pplDeltaPercent": 1.4363067402693235, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.020084, | |
| "Ppl": 5.7699, | |
| "PplError": 0.111783 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 17, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:17", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027IQ3_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:18:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45791344320, | |
| "sizeGB": 45.79134432, | |
| "sizeGiB": 42.64651268720627, | |
| "kld": 0.01176, | |
| "ppl": 5.71396, | |
| "pplDeltaPercent": 0.4528673394043811, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.01176, | |
| "Ppl": 5.71396, | |
| "PplError": 0.110101 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 18, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:100", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027UD-IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:101:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_m.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45693040320, | |
| "sizeGB": 45.69304032, | |
| "sizeGiB": 42.55495995283127, | |
| "kld": 0.01898, | |
| "ppl": 5.766296, | |
| "pplDeltaPercent": 1.3729475053619686, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.01898, | |
| "Ppl": 5.766296, | |
| "PplError": 0.111946 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 101, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:101", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027UD-IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:102:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45618329280, | |
| "sizeGB": 45.61832928, | |
| "sizeGiB": 42.48537987470627, | |
| "kld": 0.025593, | |
| "ppl": 5.747106, | |
| "pplDeltaPercent": 1.0355824338103348, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.025593, | |
| "Ppl": 5.747106, | |
| "PplError": 0.110843 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 102, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:102", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027UD-IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:103:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq3_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45691074240, | |
| "sizeGB": 45.69107424, | |
| "sizeGiB": 42.55312889814377, | |
| "kld": 0.020584, | |
| "ppl": 5.773623, | |
| "pplDeltaPercent": 1.5017580253858793, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.020584, | |
| "Ppl": 5.773623, | |
| "PplError": 0.112243 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 103, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:103", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027UD-Q2_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:104:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q2_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q2_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45696972480, | |
| "sizeGB": 45.69697248, | |
| "sizeGiB": 42.55862206220627, | |
| "kld": 0.017254, | |
| "ppl": 5.713894, | |
| "pplDeltaPercent": 0.4517070426496899, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.017254, | |
| "Ppl": 5.713894, | |
| "PplError": 0.110161 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 104, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:104", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027UD-Q3_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:105:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q3_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q3_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45820671680, | |
| "sizeGB": 45.82067168, | |
| "sizeGiB": 42.67382591962814, | |
| "kld": 0.010964, | |
| "ppl": 5.7155, | |
| "pplDeltaPercent": 0.4799409303470243, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.010964, | |
| "Ppl": 5.7155, | |
| "PplError": 0.110464 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 105, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:105", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027UD-Q4_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:106:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q4_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q4_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 46230517440, | |
| "sizeGB": 46.23051743999999, | |
| "sizeGiB": 43.05552452802658, | |
| "kld": 0.009716, | |
| "ppl": 5.680896, | |
| "pplDeltaPercent": -0.12840617418516262, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.009716, | |
| "Ppl": 5.680896, | |
| "PplError": 0.10941 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 106, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:106", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027UD-Q5_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:107:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q5_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q5_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 46528870080, | |
| "sizeGB": 46.52887008, | |
| "sizeGiB": 43.333387076854706, | |
| "kld": 0.008331, | |
| "ppl": 5.711135, | |
| "pplDeltaPercent": 0.4032031222530762, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.008331, | |
| "Ppl": 5.711135, | |
| "PplError": 0.110641 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 107, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:2:107", | |
| "description": "Continuation isolation for group \u0027attn_q\u0027 using \u0027UD-Q6_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:108:202:202:202:202:0:0", | |
| "group": "attn_q", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q6_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q6_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 47077160640, | |
| "sizeGB": 47.07716064, | |
| "sizeGiB": 43.84402245283127, | |
| "kld": 0.003793, | |
| "ppl": 5.69145, | |
| "pplDeltaPercent": 0.05713582504130535, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.003793, | |
| "Ppl": 5.69145, | |
| "PplError": 0.109698 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 108, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:0", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027Q8_0\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:1:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q8_0", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q8_0", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50832373440, | |
| "sizeGB": 50.83237344, | |
| "sizeGiB": 47.34133690595627, | |
| "kld": 0.002568, | |
| "ppl": 5.678903, | |
| "pplDeltaPercent": -0.16344362012587657, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002568, | |
| "Ppl": 5.678903, | |
| "PplError": 0.109292 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 1, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:1", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027Q6_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:2:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q6_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q6_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50791741120, | |
| "sizeGB": 50.79174112, | |
| "sizeGiB": 47.30349510908127, | |
| "kld": 0.003087, | |
| "ppl": 5.685338, | |
| "pplDeltaPercent": -0.0503146865440801, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.003087, | |
| "Ppl": 5.685338, | |
| "PplError": 0.109503 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 2, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:2", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027Q5_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:3:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50775029440, | |
| "sizeGB": 50.77502944, | |
| "sizeGiB": 47.28793114423752, | |
| "kld": 0.002603, | |
| "ppl": 5.703656, | |
| "pplDeltaPercent": 0.2717204036426193, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002603, | |
| "Ppl": 5.703656, | |
| "PplError": 0.11018 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 3, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:3", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027Q4_K_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:4:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50759300800, | |
| "sizeGB": 50.7593008, | |
| "sizeGiB": 47.27328270673752, | |
| "kld": 0.005335, | |
| "ppl": 5.677901, | |
| "pplDeltaPercent": -0.18105903449245517, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.005335, | |
| "Ppl": 5.677901, | |
| "PplError": 0.109315 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 4, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:5", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027IQ4_NL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:6:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_NL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_nl", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50758973120, | |
| "sizeGB": 50.75897311999999, | |
| "sizeGiB": 47.27297753095627, | |
| "kld": 0.004331, | |
| "ppl": 5.684584, | |
| "pplDeltaPercent": -0.06357019795365956, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.004331, | |
| "Ppl": 5.684584, | |
| "PplError": 0.109422 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 6, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:6", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027IQ4_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:7:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50756351680, | |
| "sizeGB": 50.75635168, | |
| "sizeGiB": 47.27053612470627, | |
| "kld": 0.003732, | |
| "ppl": 5.683055, | |
| "pplDeltaPercent": -0.09045040610385945, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.003732, | |
| "Ppl": 5.683055, | |
| "PplError": 0.109388 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 7, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:7", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027IQ3_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:8:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50737346240, | |
| "sizeGB": 50.73734624, | |
| "sizeGiB": 47.25283592939377, | |
| "kld": 0.005692, | |
| "ppl": 5.663189, | |
| "pplDeltaPercent": -0.4396997292640926, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.005692, | |
| "Ppl": 5.663189, | |
| "PplError": 0.108762 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 8, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:8", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027IQ3_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:9:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50733414080, | |
| "sizeGB": 50.733414079999996, | |
| "sizeGiB": 47.24917382001877, | |
| "kld": 0.006912, | |
| "ppl": 5.660766, | |
| "pplDeltaPercent": -0.48229668436412926, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.006912, | |
| "Ppl": 5.660766, | |
| "PplError": 0.108566 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 9, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:9", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:10:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xxs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50728171200, | |
| "sizeGB": 50.728171200000006, | |
| "sizeGiB": 47.24429100751877, | |
| "kld": 0.010221, | |
| "ppl": 5.667152, | |
| "pplDeltaPercent": -0.37002918322141276, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.010221, | |
| "Ppl": 5.667152, | |
| "PplError": 0.108803 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 10, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:10", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027IQ2_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:11:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50725549760, | |
| "sizeGB": 50.72554975999999, | |
| "sizeGiB": 47.24184960126877, | |
| "kld": 0.013903, | |
| "ppl": 5.653013, | |
| "pplDeltaPercent": -0.6185963925319173, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.013903, | |
| "Ppl": 5.653013, | |
| "PplError": 0.108196 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 11, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:11", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027IQ2_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:12:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50725549760, | |
| "sizeGB": 50.72554975999999, | |
| "sizeGiB": 47.24184960126877, | |
| "kld": 0.013903, | |
| "ppl": 5.653013, | |
| "pplDeltaPercent": -0.6185963925319173, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.013903, | |
| "Ppl": 5.653013, | |
| "PplError": 0.108196 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 12, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:13", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027Q5_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:14:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50769458880, | |
| "sizeGB": 50.76945888000001, | |
| "sizeGiB": 47.28274315595627, | |
| "kld": 0.002824, | |
| "ppl": 5.689095, | |
| "pplDeltaPercent": 0.015734327203682504, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002824, | |
| "Ppl": 5.689095, | |
| "PplError": 0.109665 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 14, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:14", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027Q4_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:15:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50749142720, | |
| "sizeGB": 50.749142719999995, | |
| "sizeGiB": 47.26382225751877, | |
| "kld": 0.006155, | |
| "ppl": 5.653874, | |
| "pplDeltaPercent": -0.6034597939594262, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.006155, | |
| "Ppl": 5.653874, | |
| "PplError": 0.10839 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 15, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:16", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:17:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50728171200, | |
| "sizeGB": 50.728171200000006, | |
| "sizeGiB": 47.24429100751877, | |
| "kld": 0.010221, | |
| "ppl": 5.667152, | |
| "pplDeltaPercent": -0.37002918322141276, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.010221, | |
| "Ppl": 5.667152, | |
| "PplError": 0.108803 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 17, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:17", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027IQ3_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:18:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50737346240, | |
| "sizeGB": 50.73734624, | |
| "sizeGiB": 47.25283592939377, | |
| "kld": 0.005692, | |
| "ppl": 5.663189, | |
| "pplDeltaPercent": -0.4396997292640926, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.005692, | |
| "Ppl": 5.663189, | |
| "PplError": 0.108762 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 18, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:100", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027UD-IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:101:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_m.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50717849280, | |
| "sizeGB": 50.71784928, | |
| "sizeGiB": 47.23467797040939, | |
| "kld": 0.015307, | |
| "ppl": 5.64212, | |
| "pplDeltaPercent": -0.8100980978165306, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.015307, | |
| "Ppl": 5.64212, | |
| "PplError": 0.108132 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 101, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:101", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027UD-IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:102:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50707855040, | |
| "sizeGB": 50.707855040000005, | |
| "sizeGiB": 47.22537010908127, | |
| "kld": 0.014015, | |
| "ppl": 5.711117, | |
| "pplDeltaPercent": 0.40288667768361924, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.014015, | |
| "Ppl": 5.711117, | |
| "PplError": 0.109999 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 102, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:102", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027UD-IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:103:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq3_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50714654400, | |
| "sizeGB": 50.7146544, | |
| "sizeGiB": 47.231702506542206, | |
| "kld": 0.011997, | |
| "ppl": 5.680866, | |
| "pplDeltaPercent": -0.12893358180092426, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.011997, | |
| "Ppl": 5.680866, | |
| "PplError": 0.109334 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 103, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:103", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027UD-Q2_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:104:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q2_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q2_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50720429760, | |
| "sizeGB": 50.72042976, | |
| "sizeGiB": 47.23708122968674, | |
| "kld": 0.025781, | |
| "ppl": 5.896809, | |
| "pplDeltaPercent": 3.6673991772441203, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.025781, | |
| "Ppl": 5.896809, | |
| "PplError": 0.11723 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 104, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:104", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027UD-Q3_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:105:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q3_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q3_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50750617280, | |
| "sizeGB": 50.75061728, | |
| "sizeGiB": 47.26519554853439, | |
| "kld": 0.005477, | |
| "ppl": 5.68738, | |
| "pplDeltaPercent": -0.014415808164270646, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.005477, | |
| "Ppl": 5.68738, | |
| "PplError": 0.109579 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 105, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:105", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027UD-Q4_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:106:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q4_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q4_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50771220160, | |
| "sizeGB": 50.77122016, | |
| "sizeGiB": 47.28438347578049, | |
| "kld": 0.004339, | |
| "ppl": 5.687013, | |
| "pplDeltaPercent": -0.02086776133047057, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.004339, | |
| "Ppl": 5.687013, | |
| "PplError": 0.109575 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 106, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:106", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027UD-Q5_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:107:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q5_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q5_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50781337280, | |
| "sizeGB": 50.78133728, | |
| "sizeGiB": 47.29380577802658, | |
| "kld": 0.00374, | |
| "ppl": 5.687394, | |
| "pplDeltaPercent": -0.014169684610243352, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.00374, | |
| "Ppl": 5.687394, | |
| "PplError": 0.109594 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 107, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:3:107", | |
| "description": "Continuation isolation for group \u0027attn_kv\u0027 using \u0027UD-Q6_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:108:202:202:202:0:0", | |
| "group": "attn_kv", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q6_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q6_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50832373440, | |
| "sizeGB": 50.83237344, | |
| "sizeGiB": 47.34133690595627, | |
| "kld": 0.002568, | |
| "ppl": 5.678903, | |
| "pplDeltaPercent": -0.16344362012587657, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002568, | |
| "Ppl": 5.678903, | |
| "PplError": 0.109292 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 108, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:0", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027Q8_0\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:1:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q8_0", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q8_0", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50517800640, | |
| "sizeGB": 50.517800640000004, | |
| "sizeGiB": 47.04836815595627, | |
| "kld": 0.001096, | |
| "ppl": 5.6775, | |
| "pplDeltaPercent": -0.18810871628986203, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001096, | |
| "Ppl": 5.6775, | |
| "PplError": 0.109248 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 1, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:1", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027Q6_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:2:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q6_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q6_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50395903680, | |
| "sizeGB": 50.39590368, | |
| "sizeGiB": 46.93484276533127, | |
| "kld": 0.002057, | |
| "ppl": 5.679887, | |
| "pplDeltaPercent": -0.14614465032875493, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002057, | |
| "Ppl": 5.679887, | |
| "PplError": 0.109279 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 2, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:2", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027Q5_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:3:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50329056960, | |
| "sizeGB": 50.32905696, | |
| "sizeGiB": 46.87258690595627, | |
| "kld": 0.002439, | |
| "ppl": 5.688978, | |
| "pplDeltaPercent": 0.013677437502188763, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002439, | |
| "Ppl": 5.688978, | |
| "PplError": 0.109603 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 3, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:3", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027Q4_K_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:4:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50266142400, | |
| "sizeGB": 50.2661424, | |
| "sizeGiB": 46.81399315595627, | |
| "kld": 0.004767, | |
| "ppl": 5.690939, | |
| "pplDeltaPercent": 0.048152315319434036, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.004767, | |
| "Ppl": 5.690939, | |
| "PplError": 0.109735 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 4, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:5", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027IQ4_NL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:6:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_NL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_nl", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50266142400, | |
| "sizeGB": 50.2661424, | |
| "sizeGiB": 46.81399315595627, | |
| "kld": 0.004957, | |
| "ppl": 5.669532, | |
| "pplDeltaPercent": -0.32818817903730363, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.004957, | |
| "Ppl": 5.669532, | |
| "PplError": 0.108839 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 6, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:6", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027IQ4_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:7:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50250413760, | |
| "sizeGB": 50.250413759999994, | |
| "sizeGiB": 46.79934471845627, | |
| "kld": 0.004994, | |
| "ppl": 5.687364, | |
| "pplDeltaPercent": -0.014697092226020584, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.004994, | |
| "Ppl": 5.687364, | |
| "PplError": 0.109533 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 7, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:7", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027IQ3_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:8:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50199295680, | |
| "sizeGB": 50.199295680000006, | |
| "sizeGiB": 46.75173729658127, | |
| "kld": 0.009037, | |
| "ppl": 5.68317, | |
| "pplDeltaPercent": -0.08842867691010396, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.009037, | |
| "Ppl": 5.68317, | |
| "PplError": 0.110028 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 8, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:8", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027IQ3_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:9:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50199295680, | |
| "sizeGB": 50.199295680000006, | |
| "sizeGiB": 46.75173729658127, | |
| "kld": 0.009037, | |
| "ppl": 5.68317, | |
| "pplDeltaPercent": -0.08842867691010396, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.009037, | |
| "Ppl": 5.68317, | |
| "PplError": 0.110028 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 9, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:9", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:10:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xxs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50199295680, | |
| "sizeGB": 50.199295680000006, | |
| "sizeGiB": 46.75173729658127, | |
| "kld": 0.009037, | |
| "ppl": 5.68317, | |
| "pplDeltaPercent": -0.08842867691010396, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.009037, | |
| "Ppl": 5.68317, | |
| "PplError": 0.110028 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 10, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:10", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027IQ2_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:11:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50199295680, | |
| "sizeGB": 50.199295680000006, | |
| "sizeGiB": 46.75173729658127, | |
| "kld": 0.009037, | |
| "ppl": 5.68317, | |
| "pplDeltaPercent": -0.08842867691010396, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.009037, | |
| "Ppl": 5.68317, | |
| "PplError": 0.110028 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 11, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:11", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027IQ2_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:12:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50128516800, | |
| "sizeGB": 50.1285168, | |
| "sizeGiB": 46.68581932783127, | |
| "kld": 0.02225, | |
| "ppl": 5.752587, | |
| "pplDeltaPercent": 1.1319398052107867, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.02225, | |
| "Ppl": 5.752587, | |
| "PplError": 0.111786 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 12, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:13", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027Q5_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:14:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50329056960, | |
| "sizeGB": 50.32905696, | |
| "sizeGiB": 46.87258690595627, | |
| "kld": 0.002439, | |
| "ppl": 5.688978, | |
| "pplDeltaPercent": 0.013677437502188763, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002439, | |
| "Ppl": 5.688978, | |
| "PplError": 0.109603 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 14, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:14", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027Q4_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:15:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50266142400, | |
| "sizeGB": 50.2661424, | |
| "sizeGiB": 46.81399315595627, | |
| "kld": 0.004767, | |
| "ppl": 5.690939, | |
| "pplDeltaPercent": 0.048152315319434036, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.004767, | |
| "Ppl": 5.690939, | |
| "PplError": 0.109735 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 15, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:16", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:17:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50199295680, | |
| "sizeGB": 50.199295680000006, | |
| "sizeGiB": 46.75173729658127, | |
| "kld": 0.009037, | |
| "ppl": 5.68317, | |
| "pplDeltaPercent": -0.08842867691010396, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.009037, | |
| "Ppl": 5.68317, | |
| "PplError": 0.110028 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 17, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:17", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027IQ3_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:18:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50266142400, | |
| "sizeGB": 50.2661424, | |
| "sizeGiB": 46.81399315595627, | |
| "kld": 0.004767, | |
| "ppl": 5.690939, | |
| "pplDeltaPercent": 0.048152315319434036, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.004767, | |
| "Ppl": 5.690939, | |
| "PplError": 0.109735 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 18, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:100", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027UD-IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:101:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_m.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50199295680, | |
| "sizeGB": 50.199295680000006, | |
| "sizeGiB": 46.75173729658127, | |
| "kld": 0.009037, | |
| "ppl": 5.68317, | |
| "pplDeltaPercent": -0.08842867691010396, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.009037, | |
| "Ppl": 5.68317, | |
| "PplError": 0.110028 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 101, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:101", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027UD-IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:102:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50112788160, | |
| "sizeGB": 50.112788159999994, | |
| "sizeGiB": 46.67117089033127, | |
| "kld": 0.025985, | |
| "ppl": 5.759142, | |
| "pplDeltaPercent": 1.2471783692556453, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.025985, | |
| "Ppl": 5.759142, | |
| "PplError": 0.111838 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 102, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:102", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027UD-IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:103:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq3_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50199295680, | |
| "sizeGB": 50.199295680000006, | |
| "sizeGiB": 46.75173729658127, | |
| "kld": 0.009037, | |
| "ppl": 5.68317, | |
| "pplDeltaPercent": -0.08842867691010396, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.009037, | |
| "Ppl": 5.68317, | |
| "PplError": 0.110028 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 103, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:103", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027UD-Q2_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:104:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q2_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q2_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50199295680, | |
| "sizeGB": 50.199295680000006, | |
| "sizeGiB": 46.75173729658127, | |
| "kld": 0.011034, | |
| "ppl": 5.671247, | |
| "pplDeltaPercent": -0.2980380436693505, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.011034, | |
| "Ppl": 5.671247, | |
| "PplError": 0.108783 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 104, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:104", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027UD-Q3_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:105:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q3_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q3_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50266142400, | |
| "sizeGB": 50.2661424, | |
| "sizeGiB": 46.81399315595627, | |
| "kld": 0.004767, | |
| "ppl": 5.690939, | |
| "pplDeltaPercent": 0.048152315319434036, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.004767, | |
| "Ppl": 5.690939, | |
| "PplError": 0.109735 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 105, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:105", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027UD-Q4_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:106:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q4_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q4_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50266142400, | |
| "sizeGB": 50.2661424, | |
| "sizeGiB": 46.81399315595627, | |
| "kld": 0.004767, | |
| "ppl": 5.690939, | |
| "pplDeltaPercent": 0.048152315319434036, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.004767, | |
| "Ppl": 5.690939, | |
| "PplError": 0.109735 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 106, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:106", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027UD-Q5_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:107:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q5_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q5_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50329056960, | |
| "sizeGB": 50.32905696, | |
| "sizeGiB": 46.87258690595627, | |
| "kld": 0.002439, | |
| "ppl": 5.688978, | |
| "pplDeltaPercent": 0.013677437502188763, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002439, | |
| "Ppl": 5.688978, | |
| "PplError": 0.109603 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 107, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:4:107", | |
| "description": "Continuation isolation for group \u0027attn_output\u0027 using \u0027UD-Q6_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:108:202:202:0:0", | |
| "group": "attn_output", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q6_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q6_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 50395903680, | |
| "sizeGB": 50.39590368, | |
| "sizeGiB": 46.93484276533127, | |
| "kld": 0.002057, | |
| "ppl": 5.679887, | |
| "pplDeltaPercent": -0.14614465032875493, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002057, | |
| "Ppl": 5.679887, | |
| "PplError": 0.109279 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 108, | |
| "FfnUpGate": 202, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:0", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027Q8_0\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:1:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q8_0", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q8_0", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 40294184640, | |
| "sizeGB": 40.29418464, | |
| "sizeGiB": 37.52688378095627, | |
| "kld": 0.00112, | |
| "ppl": 5.678049, | |
| "pplDeltaPercent": -0.1784571569213541, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.00112, | |
| "Ppl": 5.678049, | |
| "PplError": 0.10917 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 1, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:1", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027Q6_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:2:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q6_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q6_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 37531186880, | |
| "sizeGB": 37.53118688, | |
| "sizeGiB": 34.95364159345627, | |
| "kld": 0.002695, | |
| "ppl": 5.674973, | |
| "pplDeltaPercent": -0.23253401779122648, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002695, | |
| "Ppl": 5.674973, | |
| "PplError": 0.109155 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 2, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:2", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027Q5_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:3:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 36015994560, | |
| "sizeGB": 36.015994559999996, | |
| "sizeGiB": 33.54250878095627, | |
| "kld": 0.006427, | |
| "ppl": 5.663638, | |
| "pplDeltaPercent": -0.43180619528146713, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.006427, | |
| "Ppl": 5.663638, | |
| "PplError": 0.108632 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 3, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:3", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027Q4_K_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:4:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 34589931200, | |
| "sizeGB": 34.58993120000001, | |
| "sizeGiB": 32.21438378095627, | |
| "kld": 0.014683, | |
| "ppl": 5.728316, | |
| "pplDeltaPercent": 0.7052494638022619, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.014683, | |
| "Ppl": 5.728316, | |
| "PplError": 0.110862 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 4, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:5", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027IQ4_NL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:6:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_NL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_nl", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 34589931200, | |
| "sizeGB": 34.58993120000001, | |
| "sizeGiB": 32.21438378095627, | |
| "kld": 0.011949, | |
| "ppl": 5.717842, | |
| "pplDeltaPercent": 0.5211138848844968, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.011949, | |
| "Ppl": 5.717842, | |
| "PplError": 0.110679 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 6, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:6", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027IQ4_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:7:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 34233415360, | |
| "sizeGB": 34.23341536, | |
| "sizeGiB": 31.88235253095627, | |
| "kld": 0.012538, | |
| "ppl": 5.740632, | |
| "pplDeltaPercent": 0.9217678703280403, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.012538, | |
| "Ppl": 5.740632, | |
| "PplError": 0.111442 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 7, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:7", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027IQ3_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:8:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 33074738880, | |
| "sizeGB": 33.07473888, | |
| "sizeGiB": 30.80325096845627, | |
| "kld": 0.03803, | |
| "ppl": 5.902345, | |
| "pplDeltaPercent": 3.7647234626068045, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.03803, | |
| "Ppl": 5.902345, | |
| "PplError": 0.114549 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 8, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:8", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027IQ3_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:9:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 32673658560, | |
| "sizeGB": 32.67365856, | |
| "sizeGiB": 30.42971581220627, | |
| "kld": 0.041287, | |
| "ppl": 5.775586, | |
| "pplDeltaPercent": 1.5362680637108317, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.041287, | |
| "Ppl": 5.775586, | |
| "PplError": 0.109541 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 9, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:9", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:10:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xxs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 32539965120, | |
| "sizeGB": 32.53996512, | |
| "sizeGiB": 30.30520409345627, | |
| "kld": 0.045448, | |
| "ppl": 5.836802, | |
| "pplDeltaPercent": 2.6124608839351544, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.045448, | |
| "Ppl": 5.836802, | |
| "PplError": 0.112606 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 10, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:10", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027IQ2_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:11:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 31470417600, | |
| "sizeGB": 31.4704176, | |
| "sizeGiB": 29.30911034345627, | |
| "kld": 0.110912, | |
| "ppl": 6.028145, | |
| "pplDeltaPercent": 5.9763193980521105, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.110912, | |
| "Ppl": 6.028145, | |
| "PplError": 0.115702 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 11, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:11", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027IQ2_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:12:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 31470417600, | |
| "sizeGB": 31.4704176, | |
| "sizeGiB": 29.30911034345627, | |
| "kld": 0.110912, | |
| "ppl": 6.028145, | |
| "pplDeltaPercent": 5.9763193980521105, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.110912, | |
| "Ppl": 6.028145, | |
| "PplError": 0.115702 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 12, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:13", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027Q5_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:14:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 36015994560, | |
| "sizeGB": 36.015994559999996, | |
| "sizeGiB": 33.54250878095627, | |
| "kld": 0.006427, | |
| "ppl": 5.663638, | |
| "pplDeltaPercent": -0.43180619528146713, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.006427, | |
| "Ppl": 5.663638, | |
| "PplError": 0.108632 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 14, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:14", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027Q4_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:15:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 34589931200, | |
| "sizeGB": 34.58993120000001, | |
| "sizeGiB": 32.21438378095627, | |
| "kld": 0.014683, | |
| "ppl": 5.728316, | |
| "pplDeltaPercent": 0.7052494638022619, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.014683, | |
| "Ppl": 5.728316, | |
| "PplError": 0.110862 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 15, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:16", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:17:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 31826933440, | |
| "sizeGB": 31.82693344, | |
| "sizeGiB": 29.64114159345627, | |
| "kld": 0.080897, | |
| "ppl": 5.942669, | |
| "pplDeltaPercent": 4.473629619211706, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.080897, | |
| "Ppl": 5.942669, | |
| "PplError": 0.114555 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 17, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:17", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027IQ3_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:18:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 33074738880, | |
| "sizeGB": 33.07473888, | |
| "sizeGiB": 30.80325096845627, | |
| "kld": 0.03803, | |
| "ppl": 5.902345, | |
| "pplDeltaPercent": 3.7647234626068045, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.03803, | |
| "Ppl": 5.902345, | |
| "PplError": 0.114549 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 18, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:100", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027UD-IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:101:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_m.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 31882639040, | |
| "sizeGB": 31.882639039999997, | |
| "sizeGiB": 29.69302147626877, | |
| "kld": 0.073267, | |
| "ppl": 5.90988, | |
| "pplDeltaPercent": 3.8971906754333547, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.073267, | |
| "Ppl": 5.90988, | |
| "PplError": 0.114196 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 101, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:101", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027UD-IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:102:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 31113901760, | |
| "sizeGB": 31.11390176, | |
| "sizeGiB": 28.97707909345627, | |
| "kld": 0.138941, | |
| "ppl": 6.152836, | |
| "pplDeltaPercent": 8.168418831967927, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.138941, | |
| "Ppl": 6.152836, | |
| "PplError": 0.118876 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 102, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:102", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027UD-IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:103:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq3_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 32539965120, | |
| "sizeGB": 32.53996512, | |
| "sizeGiB": 30.30520409345627, | |
| "kld": 0.045448, | |
| "ppl": 5.836802, | |
| "pplDeltaPercent": 2.6124608839351544, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.045448, | |
| "Ppl": 5.836802, | |
| "PplError": 0.112606 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 103, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:103", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027UD-Q2_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:104:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q2_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q2_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 31999620800, | |
| "sizeGB": 31.9996208, | |
| "sizeGiB": 29.80196923017502, | |
| "kld": 0.085379, | |
| "ppl": 5.74554, | |
| "pplDeltaPercent": 1.0080517562673597, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.085379, | |
| "Ppl": 5.74554, | |
| "PplError": 0.107222 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 104, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:104", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027UD-Q3_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:105:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q3_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q3_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 33165260480, | |
| "sizeGB": 33.16526048, | |
| "sizeGiB": 30.88755577802658, | |
| "kld": 0.03526, | |
| "ppl": 5.698414, | |
| "pplDeltaPercent": 0.1795647129144457, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.03526, | |
| "Ppl": 5.698414, | |
| "PplError": 0.109728 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 105, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:105", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027UD-Q4_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:106:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q4_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q4_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 34673489600, | |
| "sizeGB": 34.6734896, | |
| "sizeGiB": 32.29220360517502, | |
| "kld": 0.010419, | |
| "ppl": 5.703639, | |
| "pplDeltaPercent": 0.2714215393270236, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.010419, | |
| "Ppl": 5.703639, | |
| "PplError": 0.109995 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 106, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:106", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027UD-Q5_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:107:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q5_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q5_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 36022957760, | |
| "sizeGB": 36.02295776, | |
| "sizeGiB": 33.54899376630783, | |
| "kld": 0.007368, | |
| "ppl": 5.660052, | |
| "pplDeltaPercent": -0.4948489856193495, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.007368, | |
| "Ppl": 5.660052, | |
| "PplError": 0.108567 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 107, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:5:107", | |
| "description": "Continuation isolation for group \u0027ffn_up_gate\u0027 using \u0027UD-Q6_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:108:202:0:0", | |
| "group": "ffn_up_gate", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q6_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q6_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 37747046080, | |
| "sizeGB": 37.74704608, | |
| "sizeGiB": 35.154676139354706, | |
| "kld": 0.002627, | |
| "ppl": 5.676606, | |
| "pplDeltaPercent": -0.20382546323969883, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002627, | |
| "Ppl": 5.676606, | |
| "PplError": 0.109137 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 108, | |
| "FfnDown": 202, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:0", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027Q8_0\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:1:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q8_0", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q8_0", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 45641922240, | |
| "sizeGB": 45.64192224, | |
| "sizeGiB": 42.50735253095627, | |
| "kld": 0.001056, | |
| "ppl": 5.685102, | |
| "pplDeltaPercent": -0.05446362645477461, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.001056, | |
| "Ppl": 5.685102, | |
| "PplError": 0.109477 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 1, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:1", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027Q6_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:2:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q6_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q6_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 44260423360, | |
| "sizeGB": 44.26042336, | |
| "sizeGiB": 41.22073143720627, | |
| "kld": 0.002455, | |
| "ppl": 5.679706, | |
| "pplDeltaPercent": -0.1493266762772016, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002455, | |
| "Ppl": 5.679706, | |
| "PplError": 0.109303 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 2, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:2", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027Q5_K\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:3:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 43881625280, | |
| "sizeGB": 43.88162528, | |
| "sizeGiB": 40.86794823408127, | |
| "kld": 0.002653, | |
| "ppl": 5.669557, | |
| "pplDeltaPercent": -0.327748672690833, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002653, | |
| "Ppl": 5.669557, | |
| "PplError": 0.109013 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 3, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:3", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027Q4_K_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:4:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 43525109440, | |
| "sizeGB": 43.52510944, | |
| "sizeGiB": 40.53591698408127, | |
| "kld": 0.005287, | |
| "ppl": 5.675623, | |
| "pplDeltaPercent": -0.22110685278295913, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.005287, | |
| "Ppl": 5.675623, | |
| "PplError": 0.108993 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 4, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:5", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027IQ4_NL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:6:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_NL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_nl", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 42789795520, | |
| "sizeGB": 42.78979552, | |
| "sizeGiB": 39.85110253095627, | |
| "kld": 0.009732, | |
| "ppl": 5.715562, | |
| "pplDeltaPercent": 0.4810309060862858, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.009732, | |
| "Ppl": 5.715562, | |
| "PplError": 0.110543 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 6, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:6", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027IQ4_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:7:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ4_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq4_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 42611537600, | |
| "sizeGB": 42.611537600000005, | |
| "sizeGiB": 39.68508690595627, | |
| "kld": 0.010185, | |
| "ppl": 5.70013, | |
| "pplDeltaPercent": 0.20973242853626017, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.010185, | |
| "Ppl": 5.70013, | |
| "PplError": 0.109995 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 7, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:7", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027IQ3_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:8:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 42032199360, | |
| "sizeGB": 42.03219936, | |
| "sizeGiB": 39.14553612470627, | |
| "kld": 0.020499, | |
| "ppl": 5.683589, | |
| "pplDeltaPercent": -0.08106255054324016, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.020499, | |
| "Ppl": 5.683589, | |
| "PplError": 0.10829 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 8, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:8", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027IQ3_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:9:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 42032199360, | |
| "sizeGB": 42.03219936, | |
| "sizeGiB": 39.14553612470627, | |
| "kld": 0.020499, | |
| "ppl": 5.683589, | |
| "pplDeltaPercent": -0.08106255054324016, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.020499, | |
| "Ppl": 5.683589, | |
| "PplError": 0.10829 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 9, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:9", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:10:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_xxs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 41764812480, | |
| "sizeGB": 41.764812479999996, | |
| "sizeGiB": 38.89651268720627, | |
| "kld": 0.030433, | |
| "ppl": 5.752347, | |
| "pplDeltaPercent": 1.1277205442846625, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.030433, | |
| "Ppl": 5.752347, | |
| "PplError": 0.110685 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 10, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:10", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027IQ2_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:11:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 41330308800, | |
| "sizeGB": 41.3303088, | |
| "sizeGiB": 38.49184960126877, | |
| "kld": 0.070285, | |
| "ppl": 5.927973, | |
| "pplDeltaPercent": 4.215270208501803, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.070285, | |
| "Ppl": 5.927973, | |
| "PplError": 0.115668 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 11, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:11", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027IQ2_XS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:12:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_XS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_xs", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 41257891520, | |
| "sizeGB": 41.25789152, | |
| "sizeGiB": 38.42440575361252, | |
| "kld": 0.070733, | |
| "ppl": 5.857743, | |
| "pplDeltaPercent": 2.9806089799936712, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.070733, | |
| "Ppl": 5.857743, | |
| "PplError": 0.113065 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 12, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:13", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027Q5_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:14:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q5_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q5_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 43502827200, | |
| "sizeGB": 43.5028272, | |
| "sizeGiB": 40.51516503095627, | |
| "kld": 0.003697, | |
| "ppl": 5.669461, | |
| "pplDeltaPercent": -0.32943637706128587, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.003697, | |
| "Ppl": 5.669461, | |
| "PplError": 0.109026 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 14, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:14", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027Q4_K_S\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:15:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "Q4_K_S", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:q4_k_s", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 42878924480, | |
| "sizeGB": 42.878924479999995, | |
| "sizeGiB": 39.93411034345627, | |
| "kld": 0.008234, | |
| "ppl": 5.681326, | |
| "pplDeltaPercent": -0.12084666502583988, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.008234, | |
| "Ppl": 5.681326, | |
| "PplError": 0.109077 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 15, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:16", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:17:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq2_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 41486284480, | |
| "sizeGB": 41.486284479999995, | |
| "sizeGiB": 38.63711327314377, | |
| "kld": 0.054731, | |
| "ppl": 5.906807, | |
| "pplDeltaPercent": 3.8431665553250514, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.054731, | |
| "Ppl": 5.906807, | |
| "PplError": 0.115651 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 17, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:17", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027IQ3_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:18:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "IQ3_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "standard:iq3_m", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 42126898880, | |
| "sizeGB": 42.12689888, | |
| "sizeGiB": 39.23373192548752, | |
| "kld": 0.019127, | |
| "ppl": 5.684905, | |
| "pplDeltaPercent": -0.0579269364649712, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.019127, | |
| "Ppl": 5.684905, | |
| "PplError": 0.108274 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 18, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:100", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027UD-IQ2_M\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:101:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_M", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_m.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 41535026880, | |
| "sizeGB": 41.535026880000004, | |
| "sizeGiB": 38.682508170604706, | |
| "kld": 0.048188, | |
| "ppl": 5.892098, | |
| "pplDeltaPercent": 3.5845786013149974, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.048188, | |
| "Ppl": 5.892098, | |
| "PplError": 0.115132 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 101, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:101", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027UD-IQ2_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:102:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ2_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq2_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 41206363840, | |
| "sizeGB": 41.20636384000001, | |
| "sizeGiB": 38.376416862010956, | |
| "kld": 0.074376, | |
| "ppl": 5.880349, | |
| "pplDeltaPercent": 3.3780281987271836, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.074376, | |
| "Ppl": 5.880349, | |
| "PplError": 0.112875 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 102, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:102", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027UD-IQ3_XXS\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:103:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-IQ3_XXS", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-iq3_xxs.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 41830962880, | |
| "sizeGB": 41.83096288, | |
| "sizeGiB": 38.95812004804611, | |
| "kld": 0.027733, | |
| "ppl": 5.739434, | |
| "pplDeltaPercent": 0.9007067262051265, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.027733, | |
| "Ppl": 5.739434, | |
| "PplError": 0.110842 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 103, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:103", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027UD-Q2_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:104:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q2_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q2_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 42077460160, | |
| "sizeGB": 42.077460159999994, | |
| "sizeGiB": 39.187688529491425, | |
| "kld": 0.021462, | |
| "ppl": 5.728484, | |
| "pplDeltaPercent": 0.7082029464505426, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.021462, | |
| "Ppl": 5.728484, | |
| "PplError": 0.110528 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 104, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:104", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027UD-Q3_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:105:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q3_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q3_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 42890065600, | |
| "sizeGB": 42.8900656, | |
| "sizeGiB": 39.94448632001877, | |
| "kld": 0.00936, | |
| "ppl": 5.686137, | |
| "pplDeltaPercent": -0.036268063710834776, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.00936, | |
| "Ppl": 5.686137, | |
| "PplError": 0.109345 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 105, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:105", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027UD-Q4_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:106:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q4_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q4_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 43488900800, | |
| "sizeGB": 43.488900799999996, | |
| "sizeGiB": 40.50219506025314, | |
| "kld": 0.006587, | |
| "ppl": 5.683135, | |
| "pplDeltaPercent": -0.08904398579515659, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.006587, | |
| "Ppl": 5.683135, | |
| "PplError": 0.109305 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 106, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:106", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027UD-Q5_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:107:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q5_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q5_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 43834275520, | |
| "sizeGB": 43.834275520000006, | |
| "sizeGiB": 40.82385033369064, | |
| "kld": 0.002832, | |
| "ppl": 5.689272, | |
| "pplDeltaPercent": 0.01884603213669948, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.002832, | |
| "Ppl": 5.689272, | |
| "PplError": 0.109766 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 107, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| }, | |
| { | |
| "key": "cont:0:6:107", | |
| "description": "Continuation isolation for group \u0027ffn_down\u0027 using \u0027UD-Q6_K_XL\u0027.", | |
| "kind": "GroupIsolationContinuation", | |
| "tensorConfigKey": "0:202:202:202:202:202:202:108:0:0", | |
| "group": "ffn_down", | |
| "testedBaseline": "Q8_0", | |
| "testedCandidate": "UD-Q6_K_XL", | |
| "testedBaselineCanonicalKey": "standard:q8_0", | |
| "testedCandidateCanonicalKey": "custom:qwen3.6-27b:unsloth/qwen3.6-27b-gguf:qwen3.6-27b-ud-q6_k_xl.gguf", | |
| "isSmallestProbe": false, | |
| "sizeBytes": 44476282560, | |
| "sizeGB": 44.47628256, | |
| "sizeGiB": 41.421765983104706, | |
| "kld": 0.00204, | |
| "ppl": 5.680807, | |
| "pplDeltaPercent": -0.12997081677860178, | |
| "foundBenchmark": true, | |
| "categories": [ | |
| { | |
| "CategoryId": 1, | |
| "Category": "General", | |
| "Kld": 0.00204, | |
| "Ppl": 5.680807, | |
| "PplError": 0.109376 | |
| } | |
| ], | |
| "config": { | |
| "BaseQuant": 0, | |
| "Embeddings": 202, | |
| "LmHead": 202, | |
| "AttnQ": 202, | |
| "AttnKV": 202, | |
| "AttnOutput": 202, | |
| "FfnUpGate": 202, | |
| "FfnDown": 108, | |
| "MoeExperts": 0, | |
| "MoeRouter": 0 | |
| } | |
| } | |
| ] |