Instructions to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Use Docker
docker model run hf.co/magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "magiccodingman/Qwen3.8-27B-MagicQuant-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "magiccodingman/Qwen3.8-27B-MagicQuant-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
- Ollama
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Ollama:
ollama run hf.co/magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
- Unsloth Studio
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for magiccodingman/Qwen3.8-27B-MagicQuant-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for magiccodingman/Qwen3.8-27B-MagicQuant-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for magiccodingman/Qwen3.8-27B-MagicQuant-GGUF to start chatting
- Pi
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Docker Model Runner:
docker model run hf.co/magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
- Lemonade
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Qwen3.8-27B-MagicQuant-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use magiccodingman/Qwen3.8-27B-MagicQuant-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "magiccodingman/Qwen3.8-27B-MagicQuant-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
| { | |
| "generatedUtc": "2026-08-15T16:34:01.6732222Z", | |
| "modelId": "mq-blake3:c6248ee1b24a8339716e4f620cad188eacc68755d2a1b88406a84b6249407c1d", | |
| "architectureFamily": "Qwen3.8-27B", | |
| "summary": { | |
| "badTradeEliminations": 53, | |
| "synergySecondChanceReinstatements": 2, | |
| "finalKldCleanupEliminations": 0, | |
| "disabledBaselines": 0, | |
| "structuredBadTradeRows": 53, | |
| "structuredSynergySecondChanceRows": 2 | |
| }, | |
| "thresholds": { | |
| "maxSizeDeltaPercent": 4, | |
| "kldMultiplier": 2.5, | |
| "pplMultiplier": 3.5, | |
| "floatingPointEpsilon": 1E-08 | |
| }, | |
| "badTrades": [ | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-Q4_K_M", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.32%) but disproportionate damage (KLD x1.46, |PPL| x6.07).", | |
| "RemovedSizeBytes": 49962325696, | |
| "RemovedSizeGB": 49.962325696, | |
| "RemovedSizeGiB": 46.5310418009758, | |
| "RemovedKld": 0.000401, | |
| "RemovedPplDeltaPercent": 0.007900133749438237, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.31708067621473895, | |
| "KldRatio": 1.4635036496350364, | |
| "PplAbsRatio": 6.0684931506454305 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-UD-Q4_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.32%) but disproportionate damage (KLD x1.46, |PPL| x6.07).", | |
| "RemovedSizeBytes": 49962325696, | |
| "RemovedSizeGB": 49.962325696, | |
| "RemovedSizeGiB": 46.5310418009758, | |
| "RemovedKld": 0.000401, | |
| "RemovedPplDeltaPercent": 0.007900133749438237, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.31708067621473895, | |
| "KldRatio": 1.4635036496350364, | |
| "PplAbsRatio": 6.0684931506454305 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-Q4_K_S", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.32%) but disproportionate damage (KLD x1.46, |PPL| x6.07).", | |
| "RemovedSizeBytes": 49962325696, | |
| "RemovedSizeGB": 49.962325696, | |
| "RemovedSizeGiB": 46.5310418009758, | |
| "RemovedKld": 0.000401, | |
| "RemovedPplDeltaPercent": 0.007900133749438237, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.31708067621473895, | |
| "KldRatio": 1.4635036496350364, | |
| "PplAbsRatio": 6.0684931506454305 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-IQ4_NL", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.32%) but disproportionate damage (KLD x1.46, |PPL| x6.07).", | |
| "RemovedSizeBytes": 49962325696, | |
| "RemovedSizeGB": 49.962325696, | |
| "RemovedSizeGiB": 46.5310418009758, | |
| "RemovedKld": 0.000401, | |
| "RemovedPplDeltaPercent": 0.007900133749438237, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.31708067621473895, | |
| "KldRatio": 1.4635036496350364, | |
| "PplAbsRatio": 6.0684931506454305 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-IQ4_XS", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.32%) but disproportionate damage (KLD x1.46, |PPL| x6.07).", | |
| "RemovedSizeBytes": 49962325696, | |
| "RemovedSizeGB": 49.962325696, | |
| "RemovedSizeGiB": 46.5310418009758, | |
| "RemovedKld": 0.000401, | |
| "RemovedPplDeltaPercent": 0.007900133749438237, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.31708067621473895, | |
| "KldRatio": 1.4635036496350364, | |
| "PplAbsRatio": 6.0684931506454305 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-Q3_K_M", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.65%) but disproportionate damage (KLD x3.19, |PPL| x39.63).", | |
| "RemovedSizeBytes": 49793468096, | |
| "RemovedSizeGB": 49.793468096000005, | |
| "RemovedSizeGiB": 46.37378090620041, | |
| "RemovedKld": 0.000873, | |
| "RemovedPplDeltaPercent": -0.05159161836826138, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.653978894692899, | |
| "KldRatio": 3.1861313868613137, | |
| "PplAbsRatio": 39.63013698606937 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-Q3_K_S", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.65%) but disproportionate damage (KLD x3.19, |PPL| x39.63).", | |
| "RemovedSizeBytes": 49793468096, | |
| "RemovedSizeGB": 49.793468096000005, | |
| "RemovedSizeGiB": 46.37378090620041, | |
| "RemovedKld": 0.000873, | |
| "RemovedPplDeltaPercent": -0.05159161836826138, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.653978894692899, | |
| "KldRatio": 3.1861313868613137, | |
| "PplAbsRatio": 39.63013698606937 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-UD-Q3_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.65%) but disproportionate damage (KLD x3.19, |PPL| x39.63).", | |
| "RemovedSizeBytes": 49793468096, | |
| "RemovedSizeGB": 49.793468096000005, | |
| "RemovedSizeGiB": 46.37378090620041, | |
| "RemovedKld": 0.000873, | |
| "RemovedPplDeltaPercent": -0.05159161836826138, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.653978894692899, | |
| "KldRatio": 3.1861313868613137, | |
| "PplAbsRatio": 39.63013698606937 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-UD-IQ3_XXS", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.91%) but disproportionate damage (KLD x7.11, |PPL| x1.04).", | |
| "RemovedSizeBytes": 49664341696, | |
| "RemovedSizeGB": 49.664341696, | |
| "RemovedSizeGiB": 46.25352257490158, | |
| "RemovedKld": 0.001949, | |
| "RemovedPplDeltaPercent": -0.0013553276861340167, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.9116069441173744, | |
| "KldRatio": 7.1131386861313874, | |
| "PplAbsRatio": 1.0410958904042922 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-UD-IQ2_M", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.91%) but disproportionate damage (KLD x7.11, |PPL| x1.04).", | |
| "RemovedSizeBytes": 49664341696, | |
| "RemovedSizeGB": 49.664341696, | |
| "RemovedSizeGiB": 46.25352257490158, | |
| "RemovedKld": 0.001949, | |
| "RemovedPplDeltaPercent": -0.0013553276861340167, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.9116069441173744, | |
| "KldRatio": 7.1131386861313874, | |
| "PplAbsRatio": 1.0410958904042922 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-UD-Q2_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.91%) but disproportionate damage (KLD x7.11, |PPL| x1.04).", | |
| "RemovedSizeBytes": 49664341696, | |
| "RemovedSizeGB": 49.664341696, | |
| "RemovedSizeGiB": 46.25352257490158, | |
| "RemovedKld": 0.001949, | |
| "RemovedPplDeltaPercent": -0.0013553276861340167, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.9116069441173744, | |
| "KldRatio": 7.1131386861313874, | |
| "PplAbsRatio": 1.0410958904042922 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "embeddings", | |
| "RemovedCandidate": "Unsloth-UD-IQ2_XXS", | |
| "AcceptedAnchor": "Unsloth-Q5_K_M", | |
| "Reason": "Reason: small size gain (0.91%) but disproportionate damage (KLD x7.11, |PPL| x1.04).", | |
| "RemovedSizeBytes": 49664341696, | |
| "RemovedSizeGB": 49.664341696, | |
| "RemovedSizeGiB": 46.25352257490158, | |
| "RemovedKld": 0.001949, | |
| "RemovedPplDeltaPercent": -0.0013553276861340167, | |
| "AnchorSizeBytes": 50121250496, | |
| "AnchorSizeGB": 50.121250496, | |
| "AnchorSizeGiB": 46.67905205488205, | |
| "AnchorKld": 0.000274, | |
| "AnchorPplDeltaPercent": 0.0013018279090581157, | |
| "SizeDeltaPercent": 0.9116069441173744, | |
| "KldRatio": 7.1131386861313874, | |
| "PplAbsRatio": 1.0410958904042922 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "lm_head", | |
| "RemovedCandidate": "Unsloth-UD-Q3_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.34%) but disproportionate damage (KLD x2.65, |PPL| x2.32).", | |
| "RemovedSizeBytes": 50121250496, | |
| "RemovedSizeGB": 50.121250496, | |
| "RemovedSizeGiB": 46.67905205488205, | |
| "RemovedKld": 0.00124, | |
| "RemovedPplDeltaPercent": 0.13134195274186475, | |
| "AnchorSizeBytes": 50290108096, | |
| "AnchorSizeGB": 50.290108096000004, | |
| "AnchorSizeGiB": 46.83631294965744, | |
| "AnchorKld": 0.000468, | |
| "AnchorPplDeltaPercent": -0.05670976370930926, | |
| "SizeDeltaPercent": 0.33576702535151376, | |
| "KldRatio": 2.6495726495726495, | |
| "PplAbsRatio": 2.316037735849429 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "lm_head", | |
| "RemovedCandidate": "Unsloth-UD-IQ3_XXS", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.34%) but disproportionate damage (KLD x2.65, |PPL| x2.32).", | |
| "RemovedSizeBytes": 50121250496, | |
| "RemovedSizeGB": 50.121250496, | |
| "RemovedSizeGiB": 46.67905205488205, | |
| "RemovedKld": 0.00124, | |
| "RemovedPplDeltaPercent": 0.13134195274186475, | |
| "AnchorSizeBytes": 50290108096, | |
| "AnchorSizeGB": 50.290108096000004, | |
| "AnchorSizeGiB": 46.83631294965744, | |
| "AnchorKld": 0.000468, | |
| "AnchorPplDeltaPercent": -0.05670976370930926, | |
| "SizeDeltaPercent": 0.33576702535151376, | |
| "KldRatio": 2.6495726495726495, | |
| "PplAbsRatio": 2.316037735849429 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "lm_head", | |
| "RemovedCandidate": "Unsloth-UD-IQ2_M", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.99%) but disproportionate damage (KLD x35.33, |PPL| x32.48).", | |
| "RemovedSizeBytes": 49793468096, | |
| "RemovedSizeGB": 49.793468096000005, | |
| "RemovedSizeGiB": 46.37378090620041, | |
| "RemovedKld": 0.016535, | |
| "RemovedPplDeltaPercent": 1.8420864913062889, | |
| "AnchorSizeBytes": 50290108096, | |
| "AnchorSizeGB": 50.290108096000004, | |
| "AnchorSizeGiB": 46.83631294965744, | |
| "AnchorKld": 0.000468, | |
| "AnchorPplDeltaPercent": -0.05670976370930926, | |
| "SizeDeltaPercent": 0.9875500745632757, | |
| "KldRatio": 35.331196581196586, | |
| "PplAbsRatio": 32.4827044025207 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "lm_head", | |
| "RemovedCandidate": "Unsloth-UD-Q2_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.99%) but disproportionate damage (KLD x35.33, |PPL| x32.48).", | |
| "RemovedSizeBytes": 49793468096, | |
| "RemovedSizeGB": 49.793468096000005, | |
| "RemovedSizeGiB": 46.37378090620041, | |
| "RemovedKld": 0.016535, | |
| "RemovedPplDeltaPercent": 1.8420864913062889, | |
| "AnchorSizeBytes": 50290108096, | |
| "AnchorSizeGB": 50.290108096000004, | |
| "AnchorSizeGiB": 46.83631294965744, | |
| "AnchorKld": 0.000468, | |
| "AnchorPplDeltaPercent": -0.05670976370930926, | |
| "SizeDeltaPercent": 0.9875500745632757, | |
| "KldRatio": 35.331196581196586, | |
| "PplAbsRatio": 32.4827044025207 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "lm_head", | |
| "RemovedCandidate": "Unsloth-UD-IQ2_XXS", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.99%) but disproportionate damage (KLD x35.33, |PPL| x32.48).", | |
| "RemovedSizeBytes": 49793468096, | |
| "RemovedSizeGB": 49.793468096000005, | |
| "RemovedSizeGiB": 46.37378090620041, | |
| "RemovedKld": 0.016535, | |
| "RemovedPplDeltaPercent": 1.8420864913062889, | |
| "AnchorSizeBytes": 50290108096, | |
| "AnchorSizeGB": 50.290108096000004, | |
| "AnchorSizeGiB": 46.83631294965744, | |
| "AnchorKld": 0.000468, | |
| "AnchorPplDeltaPercent": -0.05670976370930926, | |
| "SizeDeltaPercent": 0.9875500745632757, | |
| "KldRatio": 35.331196581196586, | |
| "PplAbsRatio": 32.4827044025207 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_q", | |
| "RemovedCandidate": "Unsloth-Q3_K_M", | |
| "AcceptedAnchor": "Unsloth-UD-Q3_K_XL", | |
| "Reason": "Reason: small size gain (0.04%) but disproportionate damage (KLD x1.58, |PPL| x25.66).", | |
| "RemovedSizeBytes": 46505957056, | |
| "RemovedSizeGB": 46.505957056, | |
| "RemovedSizeGiB": 43.3120476603508, | |
| "RemovedKld": 0.003666, | |
| "RemovedPplDeltaPercent": 0.45446277307178795, | |
| "AnchorSizeBytes": 46522832576, | |
| "AnchorSizeGB": 46.522832576, | |
| "AnchorSizeGiB": 43.327764213085175, | |
| "AnchorKld": 0.002327, | |
| "AnchorPplDeltaPercent": 0.017708426214895115, | |
| "SizeDeltaPercent": 0.03627362966868374, | |
| "KldRatio": 1.5754189944134078, | |
| "PplAbsRatio": 25.66364551862463 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_q", | |
| "RemovedCandidate": "Unsloth-UD-IQ3_XXS", | |
| "AcceptedAnchor": "Unsloth-UD-Q3_K_XL", | |
| "Reason": "Reason: small size gain (0.55%) but disproportionate damage (KLD x2.28, |PPL| x4.15).", | |
| "RemovedSizeBytes": 46267242176, | |
| "RemovedSizeGB": 46.267242175999996, | |
| "RemovedSizeGiB": 43.089727103710175, | |
| "RemovedKld": 0.005316, | |
| "RemovedPplDeltaPercent": 0.07347302719572618, | |
| "AnchorSizeBytes": 46522832576, | |
| "AnchorSizeGB": 46.522832576, | |
| "AnchorSizeGiB": 43.327764213085175, | |
| "AnchorKld": 0.002327, | |
| "AnchorPplDeltaPercent": 0.017708426214895115, | |
| "SizeDeltaPercent": 0.5493870124577342, | |
| "KldRatio": 2.284486463257413, | |
| "PplAbsRatio": 4.149043303121183 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_q", | |
| "RemovedCandidate": "Unsloth-UD-IQ2_M", | |
| "AcceptedAnchor": "Unsloth-UD-Q3_K_XL", | |
| "Reason": "Reason: small size gain (1.04%) but disproportionate damage (KLD x3.85, |PPL| x28.93).", | |
| "RemovedSizeBytes": 46040651456, | |
| "RemovedSizeGB": 46.040651456, | |
| "RemovedSizeGiB": 42.8786980509758, | |
| "RemovedKld": 0.008957, | |
| "RemovedPplDeltaPercent": 0.512349531876957, | |
| "AnchorSizeBytes": 46522832576, | |
| "AnchorSizeGB": 46.522832576, | |
| "AnchorSizeGiB": 43.327764213085175, | |
| "AnchorKld": 0.002327, | |
| "AnchorPplDeltaPercent": 0.017708426214895115, | |
| "SizeDeltaPercent": 1.0364397292712257, | |
| "KldRatio": 3.849162011173184, | |
| "PplAbsRatio": 28.93252769385027 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_q", | |
| "RemovedCandidate": "Unsloth-UD-Q2_K_XL", | |
| "AcceptedAnchor": "Unsloth-UD-Q3_K_XL", | |
| "Reason": "Reason: small size gain (1.04%) but disproportionate damage (KLD x3.85, |PPL| x28.93).", | |
| "RemovedSizeBytes": 46040651456, | |
| "RemovedSizeGB": 46.040651456, | |
| "RemovedSizeGiB": 42.8786980509758, | |
| "RemovedKld": 0.008957, | |
| "RemovedPplDeltaPercent": 0.512349531876957, | |
| "AnchorSizeBytes": 46522832576, | |
| "AnchorSizeGB": 46.522832576, | |
| "AnchorSizeGiB": 43.327764213085175, | |
| "AnchorKld": 0.002327, | |
| "AnchorPplDeltaPercent": 0.017708426214895115, | |
| "SizeDeltaPercent": 1.0364397292712257, | |
| "KldRatio": 3.849162011173184, | |
| "PplAbsRatio": 28.93252769385027 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_q", | |
| "RemovedCandidate": "Unsloth-UD-IQ2_XXS", | |
| "AcceptedAnchor": "Unsloth-UD-Q3_K_XL", | |
| "Reason": "Reason: small size gain (1.63%) but disproportionate damage (KLD x9.84, |PPL| x107.94).", | |
| "RemovedSizeBytes": 45763925696, | |
| "RemovedSizeGB": 45.76392569600001, | |
| "RemovedSizeGiB": 42.620977103710175, | |
| "RemovedKld": 0.022907, | |
| "RemovedPplDeltaPercent": 1.9115113687026275, | |
| "AnchorSizeBytes": 46522832576, | |
| "AnchorSizeGB": 46.522832576, | |
| "AnchorSizeGiB": 43.327764213085175, | |
| "AnchorKld": 0.002327, | |
| "AnchorPplDeltaPercent": 0.017708426214895115, | |
| "SizeDeltaPercent": 1.6312568216052727, | |
| "KldRatio": 9.844005156854319, | |
| "PplAbsRatio": 107.94360523662995 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_kv", | |
| "RemovedCandidate": "Unsloth-Q5_K_M", | |
| "AcceptedAnchor": "Unsloth-UD-Q4_K_XL", | |
| "Reason": "Reason: small size gain (0.01%) but disproportionate damage (KLD x0.96, |PPL| x4.79).", | |
| "RemovedSizeBytes": 51562264256, | |
| "RemovedSizeGB": 51.56226425599999, | |
| "RemovedSizeGiB": 48.02110069990158, | |
| "RemovedKld": 0.00043, | |
| "RemovedPplDeltaPercent": 0.035648684797140257, | |
| "AnchorSizeBytes": 51567834816, | |
| "AnchorSizeGB": 51.567834816, | |
| "AnchorSizeGiB": 48.02628868818283, | |
| "AnchorKld": 0.00045, | |
| "AnchorPplDeltaPercent": -0.007436469014706511, | |
| "SizeDeltaPercent": 0.010802392654018542, | |
| "KldRatio": 0.9555555555555556, | |
| "PplAbsRatio": 4.7937649880125495 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_kv", | |
| "RemovedCandidate": "Unsloth-UD-Q3_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q4_K_M", | |
| "Reason": "Reason: small size gain (0.01%) but disproportionate damage (KLD x0.91, |PPL| x18.69).", | |
| "RemovedSizeBytes": 51542070976, | |
| "RemovedSizeGB": 51.542070976000005, | |
| "RemovedSizeGiB": 48.00229424238205, | |
| "RemovedKld": 0.000657, | |
| "RemovedPplDeltaPercent": 0.07267053053946096, | |
| "AnchorSizeBytes": 51545880256, | |
| "AnchorSizeGB": 51.545880256, | |
| "AnchorSizeGiB": 48.00584191083908, | |
| "AnchorKld": 0.000722, | |
| "AnchorPplDeltaPercent": -0.003887650468127928, | |
| "SizeDeltaPercent": 0.007390076531977734, | |
| "KldRatio": 0.9099722991689752, | |
| "PplAbsRatio": 18.692660550436514 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_kv", | |
| "RemovedCandidate": "Unsloth-UD-IQ3_XXS", | |
| "AcceptedAnchor": "Unsloth-Q4_K_S", | |
| "Reason": "Reason: small size gain (0.01%) but disproportionate damage (KLD x1.49, |PPL| x19.97).", | |
| "RemovedSizeBytes": 51528144576, | |
| "RemovedSizeGB": 51.528144575999995, | |
| "RemovedSizeGiB": 47.989324271678925, | |
| "RemovedKld": 0.001308, | |
| "RemovedPplDeltaPercent": 0.1923138653588966, | |
| "AnchorSizeBytes": 51534370496, | |
| "AnchorSizeGB": 51.534370495999994, | |
| "AnchorSizeGiB": 47.995122611522675, | |
| "AnchorKld": 0.000877, | |
| "AnchorPplDeltaPercent": -0.009629959875166919, | |
| "SizeDeltaPercent": 0.012081102262582685, | |
| "KldRatio": 1.4914481185860888, | |
| "PplAbsRatio": 19.970370370371164 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_kv", | |
| "RemovedCandidate": "Unsloth-Q3_K_M", | |
| "AcceptedAnchor": "Unsloth-Q4_K_S", | |
| "Reason": "Reason: small size gain (0.02%) but disproportionate damage (KLD x1.70, |PPL| x7.98).", | |
| "RemovedSizeBytes": 51521877696, | |
| "RemovedSizeGB": 51.521877696000004, | |
| "RemovedSizeGiB": 47.98348778486252, | |
| "RemovedKld": 0.001491, | |
| "RemovedPplDeltaPercent": 0.07684351315203064, | |
| "AnchorSizeBytes": 51534370496, | |
| "AnchorSizeGB": 51.534370495999994, | |
| "AnchorSizeGiB": 47.995122611522675, | |
| "AnchorKld": 0.000877, | |
| "AnchorPplDeltaPercent": -0.009629959875166919, | |
| "SizeDeltaPercent": 0.024241685461103415, | |
| "KldRatio": 1.700114025085519, | |
| "PplAbsRatio": 7.979629629630071 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_kv", | |
| "RemovedCandidate": "Unsloth-UD-IQ2_M", | |
| "AcceptedAnchor": "Unsloth-Q4_K_S", | |
| "Reason": "Reason: small size gain (0.03%) but disproportionate damage (KLD x1.74, |PPL| x7.10).", | |
| "RemovedSizeBytes": 51519092416, | |
| "RemovedSizeGB": 51.519092416, | |
| "RemovedSizeGiB": 47.98089379072189, | |
| "RemovedKld": 0.001526, | |
| "RemovedPplDeltaPercent": 0.06835488185466246, | |
| "AnchorSizeBytes": 51534370496, | |
| "AnchorSizeGB": 51.534370495999994, | |
| "AnchorSizeGiB": 47.995122611522675, | |
| "AnchorKld": 0.000877, | |
| "AnchorPplDeltaPercent": -0.009629959875166919, | |
| "SizeDeltaPercent": 0.029646389104890404, | |
| "KldRatio": 1.7400228050171038, | |
| "PplAbsRatio": 7.098148148148711 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_kv", | |
| "RemovedCandidate": "Unsloth-UD-Q2_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q4_K_S", | |
| "Reason": "Reason: small size gain (0.03%) but disproportionate damage (KLD x1.74, |PPL| x7.10).", | |
| "RemovedSizeBytes": 51519092416, | |
| "RemovedSizeGB": 51.519092416, | |
| "RemovedSizeGiB": 47.98089379072189, | |
| "RemovedKld": 0.001526, | |
| "RemovedPplDeltaPercent": 0.06835488185466246, | |
| "AnchorSizeBytes": 51534370496, | |
| "AnchorSizeGB": 51.534370495999994, | |
| "AnchorSizeGiB": 47.995122611522675, | |
| "AnchorKld": 0.000877, | |
| "AnchorPplDeltaPercent": -0.009629959875166919, | |
| "SizeDeltaPercent": 0.029646389104890404, | |
| "KldRatio": 1.7400228050171038, | |
| "PplAbsRatio": 7.098148148148711 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_kv", | |
| "RemovedCandidate": "Unsloth-UD-IQ2_XXS", | |
| "AcceptedAnchor": "Unsloth-Q4_K_S", | |
| "Reason": "Reason: small size gain (0.04%) but disproportionate damage (KLD x2.43, |PPL| x14.11).", | |
| "RemovedSizeBytes": 51515692736, | |
| "RemovedSizeGB": 51.515692736000005, | |
| "RemovedSizeGiB": 47.977727591991425, | |
| "RemovedKld": 0.002135, | |
| "RemovedPplDeltaPercent": 0.1359072670530597, | |
| "AnchorSizeBytes": 51534370496, | |
| "AnchorSizeGB": 51.534370495999994, | |
| "AnchorSizeGiB": 47.995122611522675, | |
| "AnchorKld": 0.000877, | |
| "AnchorPplDeltaPercent": -0.009629959875166919, | |
| "SizeDeltaPercent": 0.03624330678774806, | |
| "KldRatio": 2.434435575826682, | |
| "PplAbsRatio": 14.112962962963953 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_kv", | |
| "RemovedCandidate": "Unsloth-Q3_K_S", | |
| "AcceptedAnchor": "Unsloth-Q4_K_S", | |
| "Reason": "Reason: small size gain (0.05%) but disproportionate damage (KLD x7.36, |PPL| x87.26).", | |
| "RemovedSizeBytes": 51510040256, | |
| "RemovedSizeGB": 51.510040256, | |
| "RemovedSizeGiB": 47.97246330976486, | |
| "RemovedKld": 0.006459, | |
| "RemovedPplDeltaPercent": 0.8403566651805623, | |
| "AnchorSizeBytes": 51534370496, | |
| "AnchorSizeGB": 51.534370495999994, | |
| "AnchorSizeGiB": 47.995122611522675, | |
| "AnchorKld": 0.000877, | |
| "AnchorPplDeltaPercent": -0.009629959875166919, | |
| "SizeDeltaPercent": 0.04721167594719813, | |
| "KldRatio": 7.364880273660206, | |
| "PplAbsRatio": 87.2648148148173 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_output", | |
| "RemovedCandidate": "Unsloth-Q5_K_M", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.14%) but disproportionate damage (KLD x1.78, |PPL| x5.88).", | |
| "RemovedSizeBytes": 51088070336, | |
| "RemovedSizeGB": 51.088070336, | |
| "RemovedSizeGiB": 47.579473197460175, | |
| "RemovedKld": 0.000445, | |
| "RemovedPplDeltaPercent": 0.045920641997328863, | |
| "AnchorSizeBytes": 51159094976, | |
| "AnchorSizeGB": 51.159094976, | |
| "AnchorSizeGiB": 47.64562004804611, | |
| "AnchorKld": 0.00025, | |
| "AnchorPplDeltaPercent": 0.007810967454301176, | |
| "SizeDeltaPercent": 0.1388309156628346, | |
| "KldRatio": 1.7799999999999998, | |
| "PplAbsRatio": 5.878995433791274 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_output", | |
| "RemovedCandidate": "Unsloth-Q5_K_S", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.14%) but disproportionate damage (KLD x1.78, |PPL| x5.88).", | |
| "RemovedSizeBytes": 51088070336, | |
| "RemovedSizeGB": 51.088070336, | |
| "RemovedSizeGiB": 47.579473197460175, | |
| "RemovedKld": 0.000445, | |
| "RemovedPplDeltaPercent": 0.045920641997328863, | |
| "AnchorSizeBytes": 51159094976, | |
| "AnchorSizeGB": 51.159094976, | |
| "AnchorSizeGiB": 47.64562004804611, | |
| "AnchorKld": 0.00025, | |
| "AnchorPplDeltaPercent": 0.007810967454301176, | |
| "SizeDeltaPercent": 0.1388309156628346, | |
| "KldRatio": 1.7799999999999998, | |
| "PplAbsRatio": 5.878995433791274 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_output", | |
| "RemovedCandidate": "Unsloth-UD-Q4_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.14%) but disproportionate damage (KLD x1.78, |PPL| x5.88).", | |
| "RemovedSizeBytes": 51088070336, | |
| "RemovedSizeGB": 51.088070336, | |
| "RemovedSizeGiB": 47.579473197460175, | |
| "RemovedKld": 0.000445, | |
| "RemovedPplDeltaPercent": 0.045920641997328863, | |
| "AnchorSizeBytes": 51159094976, | |
| "AnchorSizeGB": 51.159094976, | |
| "AnchorSizeGiB": 47.64562004804611, | |
| "AnchorKld": 0.00025, | |
| "AnchorPplDeltaPercent": 0.007810967454301176, | |
| "SizeDeltaPercent": 0.1388309156628346, | |
| "KldRatio": 1.7799999999999998, | |
| "PplAbsRatio": 5.878995433791274 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_output", | |
| "RemovedCandidate": "Unsloth-Q4_K_M", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.27%) but disproportionate damage (KLD x4.03, |PPL| x8.89).", | |
| "RemovedSizeBytes": 51021223616, | |
| "RemovedSizeGB": 51.02122361599999, | |
| "RemovedSizeGiB": 47.517217338085175, | |
| "RemovedKld": 0.001007, | |
| "RemovedPplDeltaPercent": -0.06947837717343061, | |
| "AnchorSizeBytes": 51159094976, | |
| "AnchorSizeGB": 51.159094976, | |
| "AnchorSizeGiB": 47.64562004804611, | |
| "AnchorKld": 0.00025, | |
| "AnchorPplDeltaPercent": 0.007810967454301176, | |
| "SizeDeltaPercent": 0.2694953068749142, | |
| "KldRatio": 4.0280000000000005, | |
| "PplAbsRatio": 8.8949771689513 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_output", | |
| "RemovedCandidate": "Unsloth-Q4_K_S", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.27%) but disproportionate damage (KLD x4.03, |PPL| x8.89).", | |
| "RemovedSizeBytes": 51021223616, | |
| "RemovedSizeGB": 51.02122361599999, | |
| "RemovedSizeGiB": 47.517217338085175, | |
| "RemovedKld": 0.001007, | |
| "RemovedPplDeltaPercent": -0.06947837717343061, | |
| "AnchorSizeBytes": 51159094976, | |
| "AnchorSizeGB": 51.159094976, | |
| "AnchorSizeGiB": 47.64562004804611, | |
| "AnchorKld": 0.00025, | |
| "AnchorPplDeltaPercent": 0.007810967454301176, | |
| "SizeDeltaPercent": 0.2694953068749142, | |
| "KldRatio": 4.0280000000000005, | |
| "PplAbsRatio": 8.8949771689513 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_output", | |
| "RemovedCandidate": "Unsloth-Q3_K_M", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.27%) but disproportionate damage (KLD x4.03, |PPL| x8.89).", | |
| "RemovedSizeBytes": 51021223616, | |
| "RemovedSizeGB": 51.02122361599999, | |
| "RemovedSizeGiB": 47.517217338085175, | |
| "RemovedKld": 0.001007, | |
| "RemovedPplDeltaPercent": -0.06947837717343061, | |
| "AnchorSizeBytes": 51159094976, | |
| "AnchorSizeGB": 51.159094976, | |
| "AnchorSizeGiB": 47.64562004804611, | |
| "AnchorKld": 0.00025, | |
| "AnchorPplDeltaPercent": 0.007810967454301176, | |
| "SizeDeltaPercent": 0.2694953068749142, | |
| "KldRatio": 4.0280000000000005, | |
| "PplAbsRatio": 8.8949771689513 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_output", | |
| "RemovedCandidate": "Unsloth-UD-Q3_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.30%) but disproportionate damage (KLD x4.44, |PPL| x21.45).", | |
| "RemovedSizeBytes": 51004511936, | |
| "RemovedSizeGB": 51.00451193599999, | |
| "RemovedSizeGiB": 47.501653373241425, | |
| "RemovedKld": 0.001109, | |
| "RemovedPplDeltaPercent": 0.16752563530985906, | |
| "AnchorSizeBytes": 51159094976, | |
| "AnchorSizeGB": 51.159094976, | |
| "AnchorSizeGiB": 47.64562004804611, | |
| "AnchorKld": 0.00025, | |
| "AnchorPplDeltaPercent": 0.007810967454301176, | |
| "SizeDeltaPercent": 0.30216140467793406, | |
| "KldRatio": 4.436, | |
| "PplAbsRatio": 21.44748858447869 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_output", | |
| "RemovedCandidate": "Unsloth-UD-IQ3_XXS", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.41%) but disproportionate damage (KLD x11.46, |PPL| x40.54).", | |
| "RemovedSizeBytes": 50950198976, | |
| "RemovedSizeGB": 50.95019897600001, | |
| "RemovedSizeGiB": 47.45107048749924, | |
| "RemovedKld": 0.002865, | |
| "RemovedPplDeltaPercent": 0.3166473473027254, | |
| "AnchorSizeBytes": 51159094976, | |
| "AnchorSizeGB": 51.159094976, | |
| "AnchorSizeGiB": 47.64562004804611, | |
| "AnchorKld": 0.00025, | |
| "AnchorPplDeltaPercent": 0.007810967454301176, | |
| "SizeDeltaPercent": 0.40832622253774875, | |
| "KldRatio": 11.459999999999999, | |
| "PplAbsRatio": 40.538812785394576 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_output", | |
| "RemovedCandidate": "Unsloth-UD-IQ2_M", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.41%) but disproportionate damage (KLD x11.46, |PPL| x40.54).", | |
| "RemovedSizeBytes": 50950198976, | |
| "RemovedSizeGB": 50.95019897600001, | |
| "RemovedSizeGiB": 47.45107048749924, | |
| "RemovedKld": 0.002865, | |
| "RemovedPplDeltaPercent": 0.3166473473027254, | |
| "AnchorSizeBytes": 51159094976, | |
| "AnchorSizeGB": 51.159094976, | |
| "AnchorSizeGiB": 47.64562004804611, | |
| "AnchorKld": 0.00025, | |
| "AnchorPplDeltaPercent": 0.007810967454301176, | |
| "SizeDeltaPercent": 0.40832622253774875, | |
| "KldRatio": 11.459999999999999, | |
| "PplAbsRatio": 40.538812785394576 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_output", | |
| "RemovedCandidate": "Unsloth-UD-Q2_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.41%) but disproportionate damage (KLD x11.46, |PPL| x40.54).", | |
| "RemovedSizeBytes": 50950198976, | |
| "RemovedSizeGB": 50.95019897600001, | |
| "RemovedSizeGiB": 47.45107048749924, | |
| "RemovedKld": 0.002865, | |
| "RemovedPplDeltaPercent": 0.3166473473027254, | |
| "AnchorSizeBytes": 51159094976, | |
| "AnchorSizeGB": 51.159094976, | |
| "AnchorSizeGiB": 47.64562004804611, | |
| "AnchorKld": 0.00025, | |
| "AnchorPplDeltaPercent": 0.007810967454301176, | |
| "SizeDeltaPercent": 0.40832622253774875, | |
| "KldRatio": 11.459999999999999, | |
| "PplAbsRatio": 40.538812785394576 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "attn_output", | |
| "RemovedCandidate": "Unsloth-UD-IQ2_XXS", | |
| "AcceptedAnchor": "Unsloth-Q6_K", | |
| "Reason": "Reason: small size gain (0.51%) but disproportionate damage (KLD x38.04, |PPL| x141.05).", | |
| "RemovedSizeBytes": 50898343616, | |
| "RemovedSizeGB": 50.898343616, | |
| "RemovedSizeGiB": 47.402776420116425, | |
| "RemovedKld": 0.00951, | |
| "RemovedPplDeltaPercent": 1.1017209094962188, | |
| "AnchorSizeBytes": 51159094976, | |
| "AnchorSizeGB": 51.159094976, | |
| "AnchorSizeGiB": 47.64562004804611, | |
| "AnchorKld": 0.00025, | |
| "AnchorPplDeltaPercent": 0.007810967454301176, | |
| "SizeDeltaPercent": 0.5096872024853546, | |
| "KldRatio": 38.04, | |
| "PplAbsRatio": 141.04794520550035 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_up_gate", | |
| "RemovedCandidate": "Unsloth-IQ4_XS", | |
| "AcceptedAnchor": "Unsloth-Q4_K_M", | |
| "Reason": "Reason: small size gain (1.01%) but disproportionate damage (KLD x1.06, |PPL| x4.12).", | |
| "RemovedSizeBytes": 34777470656, | |
| "RemovedSizeGB": 34.777470656000006, | |
| "RemovedSizeGiB": 32.389043509960175, | |
| "RemovedKld": 0.006919, | |
| "RemovedPplDeltaPercent": 0.45094962104325464, | |
| "AnchorSizeBytes": 35133986496, | |
| "AnchorSizeGB": 35.133986496, | |
| "AnchorSizeGiB": 32.721074759960175, | |
| "AnchorKld": 0.006524, | |
| "AnchorPplDeltaPercent": 0.10933571110120172, | |
| "SizeDeltaPercent": 1.014732102890144, | |
| "KldRatio": 1.0605456774984672, | |
| "PplAbsRatio": 4.124449518838848 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_down", | |
| "RemovedCandidate": "Unsloth-Q6_K", | |
| "AcceptedAnchor": "Q8_0", | |
| "Reason": "Reason: small size gain (3.03%) but disproportionate damage (KLD x1.82, |PPL| x38.22).", | |
| "RemovedSizeBytes": 44955580096, | |
| "RemovedSizeGB": 44.955580096, | |
| "RemovedSizeGiB": 41.86814659833908, | |
| "RemovedKld": 0.000572, | |
| "RemovedPplDeltaPercent": -0.040213999108335195, | |
| "AnchorSizeBytes": 46358664896, | |
| "AnchorSizeGB": 46.35866489599999, | |
| "AnchorSizeGiB": 43.174871146678925, | |
| "AnchorKld": 0.000314, | |
| "AnchorPplDeltaPercent": -0.0010521622826616726, | |
| "SizeDeltaPercent": 3.02658586727562, | |
| "KldRatio": 1.821656050955414, | |
| "PplAbsRatio": 38.220338982885 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_down", | |
| "RemovedCandidate": "Unsloth-UD-Q5_K_XL", | |
| "AcceptedAnchor": "Q8_0", | |
| "Reason": "Reason: small size gain (3.03%) but disproportionate damage (KLD x1.82, |PPL| x38.22).", | |
| "RemovedSizeBytes": 44955580096, | |
| "RemovedSizeGB": 44.955580096, | |
| "RemovedSizeGiB": 41.86814659833908, | |
| "RemovedKld": 0.000572, | |
| "RemovedPplDeltaPercent": -0.040213999108335195, | |
| "AnchorSizeBytes": 46358664896, | |
| "AnchorSizeGB": 46.35866489599999, | |
| "AnchorSizeGiB": 43.174871146678925, | |
| "AnchorKld": 0.000314, | |
| "AnchorPplDeltaPercent": -0.0010521622826616726, | |
| "SizeDeltaPercent": 3.02658586727562, | |
| "KldRatio": 1.821656050955414, | |
| "PplAbsRatio": 38.220338982885 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_down", | |
| "RemovedCandidate": "Unsloth-Q5_K_M", | |
| "AcceptedAnchor": "Q8_0", | |
| "Reason": "Reason: small size gain (3.84%) but disproportionate damage (KLD x3.08, |PPL| x40.51).", | |
| "RemovedSizeBytes": 44576782016, | |
| "RemovedSizeGB": 44.576782016, | |
| "RemovedSizeGiB": 41.51536339521408, | |
| "RemovedKld": 0.000966, | |
| "RemovedPplDeltaPercent": 0.042621489077130884, | |
| "AnchorSizeBytes": 46358664896, | |
| "AnchorSizeGB": 46.35866489599999, | |
| "AnchorSizeGiB": 43.174871146678925, | |
| "AnchorKld": 0.000314, | |
| "AnchorPplDeltaPercent": -0.0010521622826616726, | |
| "SizeDeltaPercent": 3.8436889500537523, | |
| "KldRatio": 3.0764331210191083, | |
| "PplAbsRatio": 40.50847457609922 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_down", | |
| "RemovedCandidate": "Unsloth-Q4_K_S", | |
| "AcceptedAnchor": "Unsloth-Q5_K_S", | |
| "Reason": "Reason: small size gain (1.44%) but disproportionate damage (KLD x2.61, |PPL| x1.55).", | |
| "RemovedSizeBytes": 43551102656, | |
| "RemovedSizeGB": 43.551102656000005, | |
| "RemovedSizeGiB": 40.560125052928925, | |
| "RemovedKld": 0.004065, | |
| "RemovedPplDeltaPercent": 0.10173874275523584, | |
| "AnchorSizeBytes": 44186146496, | |
| "AnchorSizeGB": 44.186146496, | |
| "AnchorSizeGiB": 41.151555716991425, | |
| "AnchorKld": 0.001558, | |
| "AnchorPplDeltaPercent": 0.06573339277753149, | |
| "SizeDeltaPercent": 1.4372012278950101, | |
| "KldRatio": 2.609114249037227, | |
| "PplAbsRatio": 1.547748236570735 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_down", | |
| "RemovedCandidate": "Unsloth-Q3_K_M", | |
| "AcceptedAnchor": "Unsloth-Q5_K_S", | |
| "Reason": "Reason: small size gain (1.54%) but disproportionate damage (KLD x2.67, |PPL| x3.14).", | |
| "RemovedSizeBytes": 43506538176, | |
| "RemovedSizeGB": 43.506538176, | |
| "RemovedSizeGiB": 40.518621146678925, | |
| "RemovedKld": 0.004153, | |
| "RemovedPplDeltaPercent": 0.20650913954525846, | |
| "AnchorSizeBytes": 44186146496, | |
| "AnchorSizeGB": 44.186146496, | |
| "AnchorSizeGiB": 41.151555716991425, | |
| "AnchorKld": 0.001558, | |
| "AnchorPplDeltaPercent": 0.06573339277753149, | |
| "SizeDeltaPercent": 1.5380574544139582, | |
| "KldRatio": 2.665596919127086, | |
| "PplAbsRatio": 3.1416169289202722 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_down", | |
| "RemovedCandidate": "Unsloth-IQ4_NL", | |
| "AcceptedAnchor": "Unsloth-Q5_K_S", | |
| "Reason": "Reason: small size gain (1.64%) but disproportionate damage (KLD x2.81, |PPL| x4.66).", | |
| "RemovedSizeBytes": 43461973696, | |
| "RemovedSizeGB": 43.461973696, | |
| "RemovedSizeGiB": 40.477117240428925, | |
| "RemovedKld": 0.004375, | |
| "RemovedPplDeltaPercent": 0.30619705751226267, | |
| "AnchorSizeBytes": 44186146496, | |
| "AnchorSizeGB": 44.186146496, | |
| "AnchorSizeGiB": 41.151555716991425, | |
| "AnchorKld": 0.001558, | |
| "AnchorPplDeltaPercent": 0.06573339277753149, | |
| "SizeDeltaPercent": 1.6389136809329063, | |
| "KldRatio": 2.80808729139923, | |
| "PplAbsRatio": 4.65816603364074 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_down", | |
| "RemovedCandidate": "Unsloth-UD-Q3_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q5_K_S", | |
| "Reason": "Reason: small size gain (2.05%) but disproportionate damage (KLD x2.93, |PPL| x6.73).", | |
| "RemovedSizeBytes": 43280930496, | |
| "RemovedSizeGB": 43.280930496, | |
| "RemovedSizeGiB": 40.3085076212883, | |
| "RemovedKld": 0.004566, | |
| "RemovedPplDeltaPercent": 0.4420864913062918, | |
| "AnchorSizeBytes": 44186146496, | |
| "AnchorSizeGB": 44.186146496, | |
| "AnchorSizeGiB": 41.151555716991425, | |
| "AnchorKld": 0.001558, | |
| "AnchorPplDeltaPercent": 0.06573339277753149, | |
| "SizeDeltaPercent": 2.048642101166133, | |
| "KldRatio": 2.9306803594351734, | |
| "PplAbsRatio": 6.725447639717793 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_down", | |
| "RemovedCandidate": "Unsloth-UD-IQ3_XXS", | |
| "AcceptedAnchor": "Unsloth-Q5_K_S", | |
| "Reason": "Reason: small size gain (3.38%) but disproportionate damage (KLD x8.28, |PPL| x16.29).", | |
| "RemovedSizeBytes": 42692540096, | |
| "RemovedSizeGB": 42.692540096, | |
| "RemovedSizeGiB": 39.76052635908127, | |
| "RemovedKld": 0.012893, | |
| "RemovedPplDeltaPercent": 1.0709228711546994, | |
| "AnchorSizeBytes": 44186146496, | |
| "AnchorSizeGB": 44.186146496, | |
| "AnchorSizeGiB": 41.151555716991425, | |
| "AnchorKld": 0.001558, | |
| "AnchorPplDeltaPercent": 0.06573339277753149, | |
| "SizeDeltaPercent": 3.3802594669241195, | |
| "KldRatio": 8.275353016688062, | |
| "PplAbsRatio": 16.291915355398398 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_down", | |
| "RemovedCandidate": "Unsloth-Q3_K_S", | |
| "AcceptedAnchor": "Unsloth-Q5_K_S", | |
| "Reason": "Reason: small size gain (3.38%) but disproportionate damage (KLD x11.23, |PPL| x10.28).", | |
| "RemovedSizeBytes": 42692540096, | |
| "RemovedSizeGB": 42.692540096, | |
| "RemovedSizeGiB": 39.76052635908127, | |
| "RemovedKld": 0.0175, | |
| "RemovedPplDeltaPercent": 0.6760410164957641, | |
| "AnchorSizeBytes": 44186146496, | |
| "AnchorSizeGB": 44.186146496, | |
| "AnchorSizeGiB": 41.151555716991425, | |
| "AnchorKld": 0.001558, | |
| "AnchorPplDeltaPercent": 0.06573339277753149, | |
| "SizeDeltaPercent": 3.3802594669241195, | |
| "KldRatio": 11.23234916559692, | |
| "PplAbsRatio": 10.284590341833741 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_down", | |
| "RemovedCandidate": "Unsloth-UD-IQ2_M", | |
| "AcceptedAnchor": "Unsloth-Q5_K_S", | |
| "Reason": "Reason: small size gain (3.96%) but disproportionate damage (KLD x14.85, |PPL| x13.57).", | |
| "RemovedSizeBytes": 42434205376, | |
| "RemovedSizeGB": 42.43420537600001, | |
| "RemovedSizeGiB": 39.5199334025383, | |
| "RemovedKld": 0.023144, | |
| "RemovedPplDeltaPercent": 0.8921444493981283, | |
| "AnchorSizeBytes": 44186146496, | |
| "AnchorSizeGB": 44.186146496, | |
| "AnchorSizeGiB": 41.151555716991425, | |
| "AnchorKld": 0.001558, | |
| "AnchorPplDeltaPercent": 0.06573339277753149, | |
| "SizeDeltaPercent": 3.9649104050261466, | |
| "KldRatio": 14.854942233632864, | |
| "PplAbsRatio": 13.572164948453333 | |
| }, | |
| { | |
| "Scope": "group", | |
| "GroupName": "ffn_down", | |
| "RemovedCandidate": "Unsloth-UD-Q2_K_XL", | |
| "AcceptedAnchor": "Unsloth-Q5_K_S", | |
| "Reason": "Reason: small size gain (3.96%) but disproportionate damage (KLD x14.85, |PPL| x13.57).", | |
| "RemovedSizeBytes": 42434205376, | |
| "RemovedSizeGB": 42.43420537600001, | |
| "RemovedSizeGiB": 39.5199334025383, | |
| "RemovedKld": 0.023144, | |
| "RemovedPplDeltaPercent": 0.8921444493981283, | |
| "AnchorSizeBytes": 44186146496, | |
| "AnchorSizeGB": 44.186146496, | |
| "AnchorSizeGiB": 41.151555716991425, | |
| "AnchorKld": 0.001558, | |
| "AnchorPplDeltaPercent": 0.06573339277753149, | |
| "SizeDeltaPercent": 3.9649104050261466, | |
| "KldRatio": 14.854942233632864, | |
| "PplAbsRatio": 13.572164948453333 | |
| } | |
| ], | |
| "synergySecondChances": [ | |
| { | |
| "SynergyName": "ffn_up_gate\u002Bffn_down", | |
| "GroupName": "ffn_down", | |
| "PeerGroupName": "ffn_up_gate", | |
| "RestoredCandidate": "Unsloth-UD-Q5_K_XL", | |
| "AcceptedAnchor": "Q8_0", | |
| "OriginalBadTradeReason": "Reason: small size gain (3.03%) but disproportionate damage (KLD x1.82, |PPL| x38.22).", | |
| "SecondChanceReason": "synergy \u0027ffn_up_gate\u002Bffn_down\u0027 second chance because \u0027Unsloth-UD-Q5_K_XL\u0027 survived in peer group \u0027ffn_up_gate\u0027 and the original bad-trade decision does not survive KLD-only review vs anchor \u0027Q8_0\u0027", | |
| "RestoredSizeBytes": 44955580096, | |
| "RestoredKld": 0.000572, | |
| "RestoredPplDeltaPercent": -0.040213999108335195, | |
| "AnchorSizeBytes": 46358664896, | |
| "AnchorKld": 0.000314, | |
| "AnchorPplDeltaPercent": -0.0010521622826616726 | |
| }, | |
| { | |
| "SynergyName": "ffn_up_gate\u002Bffn_down", | |
| "GroupName": "ffn_down", | |
| "PeerGroupName": "ffn_up_gate", | |
| "RestoredCandidate": "Unsloth-Q6_K", | |
| "AcceptedAnchor": "Q8_0", | |
| "OriginalBadTradeReason": "Reason: small size gain (3.03%) but disproportionate damage (KLD x1.82, |PPL| x38.22).", | |
| "SecondChanceReason": "synergy \u0027ffn_up_gate\u002Bffn_down\u0027 second chance because \u0027Unsloth-Q6_K\u0027 survived in peer group \u0027ffn_up_gate\u0027 and the original bad-trade decision does not survive KLD-only review vs anchor \u0027Q8_0\u0027", | |
| "RestoredSizeBytes": 44955580096, | |
| "RestoredKld": 0.000572, | |
| "RestoredPplDeltaPercent": -0.040213999108335195, | |
| "AnchorSizeBytes": 46358664896, | |
| "AnchorKld": 0.000314, | |
| "AnchorPplDeltaPercent": -0.0010521622826616726 | |
| } | |
| ], | |
| "notes": [ | |
| "Bad trade elimination: \u0027Unsloth-Q4_K_M\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.32%) but disproportionate damage (KLD x1.46, |PPL| x6.07).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q4_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.32%) but disproportionate damage (KLD x1.46, |PPL| x6.07).", | |
| "Bad trade elimination: \u0027Unsloth-Q4_K_S\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.32%) but disproportionate damage (KLD x1.46, |PPL| x6.07).", | |
| "Bad trade elimination: \u0027Unsloth-IQ4_NL\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.32%) but disproportionate damage (KLD x1.46, |PPL| x6.07).", | |
| "Bad trade elimination: \u0027Unsloth-IQ4_XS\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.32%) but disproportionate damage (KLD x1.46, |PPL| x6.07).", | |
| "Bad trade elimination: \u0027Unsloth-Q3_K_M\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.65%) but disproportionate damage (KLD x3.19, |PPL| x39.63).", | |
| "Bad trade elimination: \u0027Unsloth-Q3_K_S\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.65%) but disproportionate damage (KLD x3.19, |PPL| x39.63).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q3_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.65%) but disproportionate damage (KLD x3.19, |PPL| x39.63).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ3_XXS\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.91%) but disproportionate damage (KLD x7.11, |PPL| x1.04).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ2_M\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.91%) but disproportionate damage (KLD x7.11, |PPL| x1.04).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q2_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.91%) but disproportionate damage (KLD x7.11, |PPL| x1.04).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ2_XXS\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_M\u0027 for \u0027embeddings\u0027. Reason: small size gain (0.91%) but disproportionate damage (KLD x7.11, |PPL| x1.04).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q3_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027lm_head\u0027. Reason: small size gain (0.34%) but disproportionate damage (KLD x2.65, |PPL| x2.32).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ3_XXS\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027lm_head\u0027. Reason: small size gain (0.34%) but disproportionate damage (KLD x2.65, |PPL| x2.32).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ2_M\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027lm_head\u0027. Reason: small size gain (0.99%) but disproportionate damage (KLD x35.33, |PPL| x32.48).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q2_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027lm_head\u0027. Reason: small size gain (0.99%) but disproportionate damage (KLD x35.33, |PPL| x32.48).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ2_XXS\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027lm_head\u0027. Reason: small size gain (0.99%) but disproportionate damage (KLD x35.33, |PPL| x32.48).", | |
| "Bad trade elimination: \u0027Unsloth-Q3_K_M\u0027 removed vs accepted anchor \u0027Unsloth-UD-Q3_K_XL\u0027 for \u0027attn_q\u0027. Reason: small size gain (0.04%) but disproportionate damage (KLD x1.58, |PPL| x25.66).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ3_XXS\u0027 removed vs accepted anchor \u0027Unsloth-UD-Q3_K_XL\u0027 for \u0027attn_q\u0027. Reason: small size gain (0.55%) but disproportionate damage (KLD x2.28, |PPL| x4.15).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ2_M\u0027 removed vs accepted anchor \u0027Unsloth-UD-Q3_K_XL\u0027 for \u0027attn_q\u0027. Reason: small size gain (1.04%) but disproportionate damage (KLD x3.85, |PPL| x28.93).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q2_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-UD-Q3_K_XL\u0027 for \u0027attn_q\u0027. Reason: small size gain (1.04%) but disproportionate damage (KLD x3.85, |PPL| x28.93).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ2_XXS\u0027 removed vs accepted anchor \u0027Unsloth-UD-Q3_K_XL\u0027 for \u0027attn_q\u0027. Reason: small size gain (1.63%) but disproportionate damage (KLD x9.84, |PPL| x107.94).", | |
| "Bad trade elimination: \u0027Unsloth-Q5_K_M\u0027 removed vs accepted anchor \u0027Unsloth-UD-Q4_K_XL\u0027 for \u0027attn_kv\u0027. Reason: small size gain (0.01%) but disproportionate damage (KLD x0.96, |PPL| x4.79).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q3_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q4_K_M\u0027 for \u0027attn_kv\u0027. Reason: small size gain (0.01%) but disproportionate damage (KLD x0.91, |PPL| x18.69).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ3_XXS\u0027 removed vs accepted anchor \u0027Unsloth-Q4_K_S\u0027 for \u0027attn_kv\u0027. Reason: small size gain (0.01%) but disproportionate damage (KLD x1.49, |PPL| x19.97).", | |
| "Bad trade elimination: \u0027Unsloth-Q3_K_M\u0027 removed vs accepted anchor \u0027Unsloth-Q4_K_S\u0027 for \u0027attn_kv\u0027. Reason: small size gain (0.02%) but disproportionate damage (KLD x1.70, |PPL| x7.98).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ2_M\u0027 removed vs accepted anchor \u0027Unsloth-Q4_K_S\u0027 for \u0027attn_kv\u0027. Reason: small size gain (0.03%) but disproportionate damage (KLD x1.74, |PPL| x7.10).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q2_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q4_K_S\u0027 for \u0027attn_kv\u0027. Reason: small size gain (0.03%) but disproportionate damage (KLD x1.74, |PPL| x7.10).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ2_XXS\u0027 removed vs accepted anchor \u0027Unsloth-Q4_K_S\u0027 for \u0027attn_kv\u0027. Reason: small size gain (0.04%) but disproportionate damage (KLD x2.43, |PPL| x14.11).", | |
| "Bad trade elimination: \u0027Unsloth-Q3_K_S\u0027 removed vs accepted anchor \u0027Unsloth-Q4_K_S\u0027 for \u0027attn_kv\u0027. Reason: small size gain (0.05%) but disproportionate damage (KLD x7.36, |PPL| x87.26).", | |
| "Bad trade elimination: \u0027Unsloth-Q5_K_M\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027attn_output\u0027. Reason: small size gain (0.14%) but disproportionate damage (KLD x1.78, |PPL| x5.88).", | |
| "Bad trade elimination: \u0027Unsloth-Q5_K_S\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027attn_output\u0027. Reason: small size gain (0.14%) but disproportionate damage (KLD x1.78, |PPL| x5.88).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q4_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027attn_output\u0027. Reason: small size gain (0.14%) but disproportionate damage (KLD x1.78, |PPL| x5.88).", | |
| "Bad trade elimination: \u0027Unsloth-Q4_K_M\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027attn_output\u0027. Reason: small size gain (0.27%) but disproportionate damage (KLD x4.03, |PPL| x8.89).", | |
| "Bad trade elimination: \u0027Unsloth-Q4_K_S\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027attn_output\u0027. Reason: small size gain (0.27%) but disproportionate damage (KLD x4.03, |PPL| x8.89).", | |
| "Bad trade elimination: \u0027Unsloth-Q3_K_M\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027attn_output\u0027. Reason: small size gain (0.27%) but disproportionate damage (KLD x4.03, |PPL| x8.89).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q3_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027attn_output\u0027. Reason: small size gain (0.30%) but disproportionate damage (KLD x4.44, |PPL| x21.45).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ3_XXS\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027attn_output\u0027. Reason: small size gain (0.41%) but disproportionate damage (KLD x11.46, |PPL| x40.54).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ2_M\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027attn_output\u0027. Reason: small size gain (0.41%) but disproportionate damage (KLD x11.46, |PPL| x40.54).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q2_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027attn_output\u0027. Reason: small size gain (0.41%) but disproportionate damage (KLD x11.46, |PPL| x40.54).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ2_XXS\u0027 removed vs accepted anchor \u0027Unsloth-Q6_K\u0027 for \u0027attn_output\u0027. Reason: small size gain (0.51%) but disproportionate damage (KLD x38.04, |PPL| x141.05).", | |
| "Bad trade elimination: \u0027Unsloth-IQ4_XS\u0027 removed vs accepted anchor \u0027Unsloth-Q4_K_M\u0027 for \u0027ffn_up_gate\u0027. Reason: small size gain (1.01%) but disproportionate damage (KLD x1.06, |PPL| x4.12).", | |
| "Bad trade elimination: \u0027Unsloth-Q6_K\u0027 removed vs accepted anchor \u0027Q8_0\u0027 for \u0027ffn_down\u0027. Reason: small size gain (3.03%) but disproportionate damage (KLD x1.82, |PPL| x38.22).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q5_K_XL\u0027 removed vs accepted anchor \u0027Q8_0\u0027 for \u0027ffn_down\u0027. Reason: small size gain (3.03%) but disproportionate damage (KLD x1.82, |PPL| x38.22).", | |
| "Bad trade elimination: \u0027Unsloth-Q5_K_M\u0027 removed vs accepted anchor \u0027Q8_0\u0027 for \u0027ffn_down\u0027. Reason: small size gain (3.84%) but disproportionate damage (KLD x3.08, |PPL| x40.51).", | |
| "Bad trade elimination: \u0027Unsloth-Q4_K_S\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_S\u0027 for \u0027ffn_down\u0027. Reason: small size gain (1.44%) but disproportionate damage (KLD x2.61, |PPL| x1.55).", | |
| "Bad trade elimination: \u0027Unsloth-Q3_K_M\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_S\u0027 for \u0027ffn_down\u0027. Reason: small size gain (1.54%) but disproportionate damage (KLD x2.67, |PPL| x3.14).", | |
| "Bad trade elimination: \u0027Unsloth-IQ4_NL\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_S\u0027 for \u0027ffn_down\u0027. Reason: small size gain (1.64%) but disproportionate damage (KLD x2.81, |PPL| x4.66).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q3_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_S\u0027 for \u0027ffn_down\u0027. Reason: small size gain (2.05%) but disproportionate damage (KLD x2.93, |PPL| x6.73).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ3_XXS\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_S\u0027 for \u0027ffn_down\u0027. Reason: small size gain (3.38%) but disproportionate damage (KLD x8.28, |PPL| x16.29).", | |
| "Bad trade elimination: \u0027Unsloth-Q3_K_S\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_S\u0027 for \u0027ffn_down\u0027. Reason: small size gain (3.38%) but disproportionate damage (KLD x11.23, |PPL| x10.28).", | |
| "Bad trade elimination: \u0027Unsloth-UD-IQ2_M\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_S\u0027 for \u0027ffn_down\u0027. Reason: small size gain (3.96%) but disproportionate damage (KLD x14.85, |PPL| x13.57).", | |
| "Bad trade elimination: \u0027Unsloth-UD-Q2_K_XL\u0027 removed vs accepted anchor \u0027Unsloth-Q5_K_S\u0027 for \u0027ffn_down\u0027. Reason: small size gain (3.96%) but disproportionate damage (KLD x14.85, |PPL| x13.57).", | |
| "Synergy second-chance rejected: \u0027Unsloth-UD-IQ2_M\u0027 remained removed for \u0027ffn_down\u0027 even though it survived in \u0027ffn_up_gate\u0027 under synergy \u0027ffn_up_gate\u002Bffn_down\u0027. Reason: small size gain (3.96%) but disproportionate KLD damage after PPL was ignored for synergy review (KLD x14.85).", | |
| "Synergy second-chance rejected: \u0027Unsloth-UD-Q2_K_XL\u0027 remained removed for \u0027ffn_down\u0027 even though it survived in \u0027ffn_up_gate\u0027 under synergy \u0027ffn_up_gate\u002Bffn_down\u0027. Reason: small size gain (3.96%) but disproportionate KLD damage after PPL was ignored for synergy review (KLD x14.85).", | |
| "Synergy second-chance rejected: \u0027Unsloth-UD-IQ3_XXS\u0027 remained removed for \u0027ffn_down\u0027 even though it survived in \u0027ffn_up_gate\u0027 under synergy \u0027ffn_up_gate\u002Bffn_down\u0027. Reason: small size gain (3.38%) but disproportionate KLD damage after PPL was ignored for synergy review (KLD x8.28).", | |
| "Synergy second-chance rejected: \u0027Unsloth-UD-Q3_K_XL\u0027 remained removed for \u0027ffn_down\u0027 even though it survived in \u0027ffn_up_gate\u0027 under synergy \u0027ffn_up_gate\u002Bffn_down\u0027. Reason: small size gain (2.05%) but disproportionate KLD damage after PPL was ignored for synergy review (KLD x2.93).", | |
| "Synergy second-chance rejected: \u0027Unsloth-Q4_K_S\u0027 remained removed for \u0027ffn_down\u0027 even though it survived in \u0027ffn_up_gate\u0027 under synergy \u0027ffn_up_gate\u002Bffn_down\u0027. Reason: small size gain (1.44%) but disproportionate KLD damage after PPL was ignored for synergy review (KLD x2.61).", | |
| "Synergy second-chance rejected: \u0027Unsloth-Q5_K_M\u0027 remained removed for \u0027ffn_down\u0027 even though it survived in \u0027ffn_up_gate\u0027 under synergy \u0027ffn_up_gate\u002Bffn_down\u0027. Reason: small size gain (3.84%) but disproportionate KLD damage after PPL was ignored for synergy review (KLD x3.08).", | |
| "Synergy second-chance restored: \u0027Unsloth-UD-Q5_K_XL\u0027 restored for \u0027ffn_down\u0027 because it survived in peer group \u0027ffn_up_gate\u0027 under synergy \u0027ffn_up_gate\u002Bffn_down\u0027, and KLD-only bad-trade review did not eliminate it vs anchor \u0027Q8_0\u0027.", | |
| "Synergy second-chance restored: \u0027Unsloth-Q6_K\u0027 restored for \u0027ffn_down\u0027 because it survived in peer group \u0027ffn_up_gate\u0027 under synergy \u0027ffn_up_gate\u002Bffn_down\u0027, and KLD-only bad-trade review did not eliminate it vs anchor \u0027Q8_0\u0027." | |
| ] | |
| } |