Instructions to use saik0s/comfy_backup with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use saik0s/comfy_backup with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="saik0s/comfy_backup",
	filename="ComfyUI/models/text_encoders/gemma-3-12b-it-q2_k.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use saik0s/comfy_backup with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf saik0s/comfy_backup:Q4_K_S
# Run inference directly in the terminal:
llama cli -hf saik0s/comfy_backup:Q4_K_S

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf saik0s/comfy_backup:Q4_K_S
# Run inference directly in the terminal:
llama cli -hf saik0s/comfy_backup:Q4_K_S

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf saik0s/comfy_backup:Q4_K_S
# Run inference directly in the terminal:
./llama-cli -hf saik0s/comfy_backup:Q4_K_S

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf saik0s/comfy_backup:Q4_K_S
# Run inference directly in the terminal:
./build/bin/llama-cli -hf saik0s/comfy_backup:Q4_K_S

Use Docker

docker model run hf.co/saik0s/comfy_backup:Q4_K_S

LM Studio
Jan
Ollama
How to use saik0s/comfy_backup with Ollama:
```
ollama run hf.co/saik0s/comfy_backup:Q4_K_S
```

Unsloth Studio

How to use saik0s/comfy_backup with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for saik0s/comfy_backup to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for saik0s/comfy_backup to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for saik0s/comfy_backup to start chatting

How to use saik0s/comfy_backup with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf saik0s/comfy_backup:Q4_K_S

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "saik0s/comfy_backup:Q4_K_S"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use saik0s/comfy_backup with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf saik0s/comfy_backup:Q4_K_S

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default saik0s/comfy_backup:Q4_K_S

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use saik0s/comfy_backup with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf saik0s/comfy_backup:Q4_K_S

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "saik0s/comfy_backup:Q4_K_S" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use saik0s/comfy_backup with Docker Model Runner:
```
docker model run hf.co/saik0s/comfy_backup:Q4_K_S
```

Lemonade

How to use saik0s/comfy_backup with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull saik0s/comfy_backup:Q4_K_S

Run and chat with the model

lemonade run user.comfy_backup-Q4_K_S

List all available models

lemonade list

comfy_backup / custom_nodes /ComfyUI-INT8-Fast /Metrics.md

saik0s

Add files using upload-large-folder tool

8d5e143 verified 24 days ago

preview code

Raw

History Blame Contribute Delete

22.5 kB

This document serves as a quick overview of the quality of various quantization methods, with a focus on INT8. We capture the latents per step, and measure how much they diverge from the BF16 baseline.

General Model Quality

The general takeaway is that in terms of tested quantization methods the ranking is:

GGUF Q8 > INT8 ConvRot > MXFP8 > FP8 >= INT8 Row > INT8 Tensorwise

Every INT8 ConvRot and INT8 Row checkpoint was created from BF16 via on the fly quantization, unless stated otherwise. INT8 ConvRot is row-wise INT8 with parameters and activations rotated before quantization via ConvRot. INT8 Row is just regular row wise INT8.

Anima

100 samples per column.

Metric	INT8 ConvRot	INT8 Row	INT8 Row Bedovyy	INT8 Tensor Silver	FP8	GGUF_Q8	INT8 QuaRot
MSE ↓	`0.00746 ±0.00103` ★	`0.01467 ±0.00167`	`0.01438 ±0.00213`	`0.04069 ±0.00331`	`0.01756 ±0.00175`	`0.01364 ±0.00155`	`0.00952 ±0.00164`
MAE ↓	`0.03456 ±0.00256` ★	`0.05743 ±0.00350`	`0.05508 ±0.00437`	`0.10956 ±0.00503`	`0.06204 ±0.00323`	`0.04882 ±0.00332`	`0.03733 ±0.00304`
Max err ↓	`1.03295 ±0.04107` ★	`1.27721 ±0.03917`	`1.24410 ±0.04644`	`1.72119 ±0.04671`	`1.33251 ±0.03424`	`1.21710 ±0.03939`	`1.09261 ±0.04083`
Rel-RMSE ↓	`0.09032 ±0.00626` ★	`0.13396 ±0.00720`	`0.13084 ±0.00920`	`0.23802 ±0.01011`	`0.14523 ±0.00679`	`0.12124 ±0.00714`	`0.09632 ±0.00664`
SNR dB ↑	`24.05 ±0.53` ★	`19.68 ±0.39`	`20.24 ±0.52`	`14.48 ±0.36`	`19.66 ±0.35`	`21.98 ±0.46`	`23.98 ±0.45`
Cos-sim ↑	`0.992165 ±0.001113` ★	`0.984617 ±0.001780`	`0.984765 ±0.002368`	`0.957751 ±0.003461`	`0.981587 ±0.001878`	`0.985553 ±0.001704`	`0.990093 ±0.001646`
Var ratio →1	`1.00173 ±0.00081`	`1.00005 ±0.00145` ★	`0.99824 ±0.00155`	`0.99655 ±0.00221`	`0.99871 ±0.00107`	`1.00965 ±0.00085`	`1.00392 ±0.00087`
Outlier% ↓	`0.00008 ±0.00002` ★	`0.00022 ±0.00004`	`0.00022 ±0.00005`	`0.00103 ±0.00015`	`0.00027 ±0.00006`	`0.00019 ±0.00003`	`0.00019 ±0.00008`
Ch-MSE max ↓	`0.01443 ±0.00204` ★	`0.02867 ±0.00359`	`0.02837 ±0.00452`	`0.08719 ±0.00820`	`0.03471 ±0.00401`	`0.02732 ±0.00333`	`0.01964 ±0.00419`
Ch-MSE std ↓	`0.00346 ±0.00048` ★	`0.00684 ±0.00083`	`0.00670 ±0.00102`	`0.02075 ±0.00198`	`0.00816 ±0.00091`	`0.00648 ±0.00077`	`0.00468 ±0.00098`
ΔMSE/step ↓	`0.000757 ±0.000111` ★	`0.001306 ±0.000160`	`0.001300 ±0.000199`	`0.003241 ±0.000283`	`0.001577 ±0.000154`	`0.001251 ±0.000142`	`0.000903 ±0.000139`
ΔCos/step ↑	`-0.0007054 ±0.0001211` ★	`-0.0010668 ±0.0001787`	`-0.0012205 ±0.0002166`	`-0.0028863 ±0.0003109`	`-0.0015127 ±0.0001643`	`-0.0012179 ±0.0001540`	`-0.0008402 ±0.0001416`

Z Image Turbo

64 samples per column

(Different prompt, seeds and resolution from the other Z Image Turbo test. This one is just here to get more samples against MXFP8 checkpoints, which are hard to find in the wild.)

Metric	GGUF_Q8	I8ConvRot	I8Row	MXFP8
MSE ↓	`0.03616 ±0.00313` ★	`0.04834 ±0.00355`	`0.14951 ±0.00857`	`0.11037 ±0.00473`
MAE ↓	`0.08745 ±0.00362` ★	`0.10531 ±0.00384`	`0.21607 ±0.00612`	`0.17953 ±0.00387`
Max err ↓	`3.34811 ±0.09199` ★	`3.61884 ±0.09058`	`4.52879 ±0.07883`	`4.33244 ±0.07534`
Rel-RMSE ↓	`0.16740 ±0.00628` ★	`0.19634 ±0.00660`	`0.35659 ±0.00968`	`0.30729 ±0.00645`
SNR dB ↑	`16.42 ±0.29` ★	`14.86 ±0.26`	`9.27 ±0.23`	`10.59 ±0.18`
Cos-sim ↑	`0.978215 ±0.001696` ★	`0.971225 ±0.001920`	`0.916394 ±0.004070`	`0.935860 ±0.002428`
Var ratio →1	`1.00006 ±0.00050` ★	`1.00338 ±0.00045`	`0.97402 ±0.00155`	`0.99629 ±0.00101`
Outlier% ↓	`0.00054 ±0.00011` ★	`0.00084 ±0.00013`	`0.00418 ±0.00049`	`0.00251 ±0.00024`
Ch-MSE max ↓	`0.06081 ±0.00660` ★	`0.08189 ±0.00736`	`0.28000 ±0.02181`	`0.19093 ±0.01077`
Ch-MSE std ↓	`0.01206 ±0.00163` ★	`0.01657 ±0.00178`	`0.06623 ±0.00638`	`0.04204 ±0.00300`
ΔMSE/step ↓	`0.006970 ±0.000753` ★	`0.008928 ±0.000936`	`0.016678 ±0.001880`	`0.015876 ±0.001259`
ΔCos/step ↑	`-0.0038860 ±0.0004616` ★	`-0.0049137 ±0.0005605`	`-0.0090652 ±0.0010369`	`-0.0086360 ±0.0007459`

Flux2 Klein 9B Base

32 samples per column

Metric	INT8 ConvRot	INT8 Row	INT8 Row ModelOpt	GGUF Q8 0	FP8_Official
MSE ↓	`0.02204 ±0.00475` ★	`0.06017 ±0.01167`	`0.04246 ±0.00481`	`0.03024 ±0.00811`	`0.04142 ±0.00540`
MAE ↓	`0.05193 ±0.00639` ★	`0.10543 ±0.01075`	`0.08763 ±0.00509`	`0.05821 ±0.01005`	`0.08112 ±0.00684`
Max err ↓	`1.57007 ±0.12075` ★	`2.13746 ±0.09461`	`1.94028 ±0.08472`	`1.62537 ±0.17338`	`1.99069 ±0.11125`
Rel-RMSE ↓	`0.11172 ±0.01310` ★	`0.21613 ±0.01848`	`0.18317 ±0.01103`	`0.12411 ±0.01924`	`0.16800 ±0.01327`
SNR dB ↑	`23.10 ±0.81`	`15.79 ±0.59`	`17.03 ±0.50`	`23.52 ±1.11` ★	`19.12 ±0.69`
Cos-sim ↑	`0.987098 ±0.002851` ★	`0.961752 ±0.007710`	`0.973305 ±0.003089`	`0.981972 ±0.005046`	`0.975436 ±0.003389`
Var ratio →1	`1.00008 ±0.00121` ★	`1.00311 ±0.00263`	`1.00190 ±0.00224`	`1.00229 ±0.00136`	`1.00226 ±0.00181`
Outlier% ↓	`0.00055 ±0.00017` ★	`0.00220 ±0.00071`	`0.00121 ±0.00027`	`0.00104 ±0.00036`	`0.00118 ±0.00021`
Ch-MSE max ↓	`0.04205 ±0.00912` ★	`0.10400 ±0.01834`	`0.07711 ±0.00852`	`0.05674 ±0.01541`	`0.07300 ±0.00954`
Ch-MSE std ↓	`0.00632 ±0.00136` ★	`0.01511 ±0.00252`	`0.01146 ±0.00125`	`0.00832 ±0.00209`	`0.01073 ±0.00139`
ΔMSE/step ↓	`0.003857 ±0.000759` ★	`0.008846 ±0.001371`	`0.006936 ±0.000717`	`0.004814 ±0.001172`	`0.006657 ±0.000822`
ΔCos/step ↑	`-0.0017513 ±0.0004183` ★	`-0.0034872 ±0.0008234`	`-0.0026765 ±0.0005044`	`-0.0022502 ±0.0006240`	`-0.0030834 ±0.0005014`

Z Image Turbo

32 samples per column

Metric	INT8 ConvRot	INT8 Row	GGUF Q8	FP8
MSE ↓	`0.04326 ±0.00622`	`0.10273 ±0.00973`	`0.02627 ±0.00370` ★	`0.09472 ±0.00712`
MAE ↓	`0.09578 ±0.00821`	`0.17967 ±0.00961`	`0.07105 ±0.00531` ★	`0.18951 ±0.00700`
Max err ↓	`2.24582 ±0.10280`	`2.99726 ±0.08630`	`2.05538 ±0.09594` ★	`2.87146 ±0.06808`
Rel-RMSE ↓	`0.16181 ±0.01205`	`0.27953 ±0.01342`	`0.12857 ±0.00877` ★	`0.28443 ±0.01002`
SNR dB ↑	`17.97 ±0.63`	`11.91 ±0.41`	`19.79 ±0.57` ★	`11.17 ±0.29`
Cos-sim ↑	`0.972530 ±0.003675`	`0.935604 ±0.005455`	`0.982917 ±0.002238` ★	`0.933439 ±0.004578`
Var ratio →1	`1.00117 ±0.00131` ★	`0.99202 ±0.00301`	`1.00208 ±0.00098`	`0.94623 ±0.00320`
Outlier% ↓	`0.00053 ±0.00012`	`0.00172 ±0.00033`	`0.00032 ±0.00010` ★	`0.00083 ±0.00014`
Ch-MSE max ↓	`0.07861 ±0.01218`	`0.19714 ±0.02157`	`0.04706 ±0.00714` ★	`0.17641 ±0.01822`
Ch-MSE std ↓	`0.01621 ±0.00276`	`0.04345 ±0.00546`	`0.00956 ±0.00172` ★	`0.03825 ±0.00444`
ΔMSE/step ↓	`0.008738 ±0.001514`	`0.015233 ±0.002232`	`0.005719 ±0.001069` ★	`0.012003 ±0.001665`
ΔCos/step ↑	`-0.0050629 ±0.0009302`	`-0.0086229 ±0.0012933`	`-0.0033847 ±0.0006569` ★	`-0.0071850 ±0.0011542`

Chroma

32 samples per column

Metric	INT8 ConvRot	INT8 Row	GGUF Q8	FP8 Mixed
MSE ↓	`0.01021 ±0.00360`	`0.02799 ±0.00564`	`0.00555 ±0.00138` ★	`0.02030 ±0.00274`
MAE ↓	`0.03999 ±0.00420`	`0.07773 ±0.00677`	`0.02807 ±0.00324` ★	`0.06772 ±0.00447`
Max err ↓	`1.46539 ±0.20187`	`2.22444 ±0.21730`	`1.35671 ±0.22226` ★	`2.04296 ±0.19891`
Rel-RMSE ↓	`0.09169 ±0.01286`	`0.16790 ±0.01750`	`0.06770 ±0.00925` ★	`0.14417 ±0.01110`
SNR dB ↑	`23.54 ±0.99`	`17.33 ±0.89`	`26.31 ±1.11` ★	`18.79 ±0.76`
Cos-sim ↑	`0.990995 ±0.002884`	`0.976308 ±0.004694`	`0.995231 ±0.001197` ★	`0.982911 ±0.002356`
Var ratio →1	`0.99150 ±0.00407`	`1.03855 ±0.00951`	`1.00278 ±0.00227` ★	`1.00961 ±0.00768`
Outlier% ↓	`0.00041 ±0.00029`	`0.00129 ±0.00049`	`0.00015 ±0.00007` ★	`0.00059 ±0.00018`
Ch-MSE max ↓	`0.02014 ±0.00781`	`0.05817 ±0.01315`	`0.01074 ±0.00273` ★	`0.03952 ±0.00520`
Ch-MSE std ↓	`0.00431 ±0.00173`	`0.01372 ±0.00325`	`0.00243 ±0.00070` ★	`0.00934 ±0.00135`
ΔMSE/step ↓	`0.000707 ±0.000291`	`0.001820 ±0.000582`	`0.000410 ±0.000138` ★	`0.001372 ±0.000381`
ΔCos/step ↑	`-0.0005174 ±0.0003418`	`-0.0010664 ±0.0007771`	`-0.0002943 ±0.0001454` ★	`-0.0009653 ±0.0004437`

Qwen Image 2512

16 samples per column.

Metric	FP8	GGUF Q4 K M	GGUF Q8	I8 Conv	I8 Row	Nunchaku BestQuality
MSE ↓	`0.01643 ±0.00334`	`0.02188 ±0.00320`	`0.01062 ±0.00377`	`0.00894 ±0.00256` ★	`0.01305 ±0.00436`	`0.02146 ±0.00354`
MAE ↓	`0.07556 ±0.00707`	`0.07740 ±0.00661`	`0.04068 ±0.00892`	`0.04043 ±0.00619` ★	`0.05007 ±0.00917`	`0.08532 ±0.00742`
Max err ↓	`0.93735 ±0.06070`	`1.05423 ±0.05437`	`0.65768 ±0.09201` ★	`0.73333 ±0.08073`	`0.75177 ±0.07628`	`0.96512 ±0.04607`
Rel-RMSE ↓	`0.22316 ±0.02186`	`0.25253 ±0.02143`	`0.13382 ±0.02853` ★	`0.13795 ±0.02225`	`0.16354 ±0.02883`	`0.24947 ±0.02144`
SNR dB ↑	`14.08 ±0.75`	`13.78 ±0.84`	`22.44 ±1.67` ★	`20.34 ±1.31`	`18.70 ±1.27`	`13.54 ±0.72`
Cos-sim ↑	`0.943337 ±0.010885`	`0.929011 ±0.010479`	`0.967114 ±0.011496`	`0.972459 ±0.007414` ★	`0.957911 ±0.013642`	`0.927933 ±0.011458`
Var ratio →1	`1.00262 ±0.00459`	`0.99685 ±0.00597`	`0.99789 ±0.00268` ★	`0.98840 ±0.00348`	`1.00248 ±0.00378`	`0.94775 ±0.00588`
Outlier% ↓	`0.00076 ±0.00029`	`0.00162 ±0.00044`	`0.00079 ±0.00040`	`0.00064 ±0.00029` ★	`0.00116 ±0.00051`	`0.00093 ±0.00046`
Ch-MSE max ↓	`0.02873 ±0.00637`	`0.04307 ±0.00754`	`0.02095 ±0.00768`	`0.01662 ±0.00475` ★	`0.02675 ±0.00918`	`0.03918 ±0.00712`
Ch-MSE std ↓	`0.00681 ±0.00167`	`0.01152 ±0.00214`	`0.00555 ±0.00216`	`0.00420 ±0.00124` ★	`0.00735 ±0.00270`	`0.01014 ±0.00205`
ΔMSE/step ↓	`0.001429 ±0.000380`	`0.002038 ±0.000518`	`0.001062 ±0.000426`	`0.000907 ±0.000322` ★	`0.001278 ±0.000448`	`0.002092 ±0.000484`
ΔCos/step ↑	`-0.0023754 ±0.0015020` ★	`-0.0055836 ±0.0019522`	`-0.0029254 ±0.0013186`	`-0.0024572 ±0.0010212`	`-0.0034970 ±0.0014186`	`-0.0057739 ±0.0019517`

HiDream O1

16 Samples per column.

FP8 Naive refers to using a BF16 checkpoint with the dtype set to FP8, which naively casts most weights to FP8.

Metric	FP8_Naive	FP8 Scaled	INT8 ConvRot	INT8 Row	MXFP8
MSE ↓	`0.02261 ±0.00697`	`0.00324 ±0.00098`	`0.00199 ±0.00058` ★	`0.05192 ±0.01084`	`0.00354 ±0.00070`
MAE ↓	`0.06901 ±0.01116`	`0.02499 ±0.00291`	`0.01877 ±0.00202` ★	`0.13052 ±0.01274`	`0.02768 ±0.00254`
Max err ↓	`0.86595 ±0.05077`	`0.53393 ±0.04962`	`0.45624 ±0.03571` ★	`1.15126 ±0.04459`	`0.56008 ±0.03832`
Rel-RMSE ↓	`0.23140 ±0.03353`	`0.08793 ±0.01196`	`0.06738 ±0.00849` ★	`0.40533 ±0.03865`	`0.09269 ±0.00912`
SNR dB ↑	`14.86 ±1.00`	`22.98 ±0.91`	`25.65 ±0.85` ★	`8.77 ±0.76`	`22.65 ±0.79`
Cos-sim ↑	`0.957479 ±0.013819`	`0.993943 ±0.001945`	`0.996338 ±0.001124` ★	`0.901425 ±0.020387`	`0.993764 ±0.001271`
Var ratio →1	`0.96638 ±0.00868`	`0.96287 ±0.00445`	`0.99691 ±0.00313`	`1.00254 ±0.02455` ★	`1.01115 ±0.00402`
Outlier% ↓	`0.00499 ±0.00257`	`0.00028 ±0.00015`	`0.00010 ±0.00008` ★	`0.01168 ±0.00462`	`0.00022 ±0.00008`
Ch-MSE max ↓	`0.02596 ±0.00826`	`0.00362 ±0.00109`	`0.00237 ±0.00068` ★	`0.05824 ±0.01247`	`0.00399 ±0.00077`
Ch-MSE std ↓	`0.00305 ±0.00116`	`0.00034 ±0.00011`	`0.00034 ±0.00012` ★	`0.00590 ±0.00154`	`0.00044 ±0.00009`
ΔMSE/step ↓	`0.002661 ±0.000879`	`0.000514 ±0.000244`	`0.000299 ±0.000185` ★	`0.005193 ±0.001433`	`0.000592 ±0.000248`
ΔCos/step ↑	`-0.0044397 ±0.0016674`	`-0.0008584 ±0.0004631`	`-0.0005061 ±0.0003415` ★	`-0.0064275 ±0.0033903`	`-0.0009670 ±0.0004783`

Anima on a 5060

16 samples per column.

Metric	INT8ConvRot	MXFP8
MSE ↓	`0.00576 ±0.00109` ★	`0.01461 ±0.00217`
MAE ↓	`0.03466 ±0.00317` ★	`0.06382 ±0.00463`
Max err ↓	`0.66684 ±0.06254` ★	`0.92180 ±0.05310`
Rel-RMSE ↓	`0.08546 ±0.00846` ★	`0.14716 ±0.01107`
SNR dB ↑	`24.22 ±0.73` ★	`18.90 ±0.58`
Cos-sim ↑	`0.991708 ±0.001573` ★	`0.979025 ±0.003469`
Var ratio →1	`1.00804 ±0.00188` ★	`1.01619 ±0.00334`
Outlier% ↓	`0.00003 ±0.00001` ★	`0.00015 ±0.00005`
Ch-MSE max ↓	`0.01101 ±0.00206` ★	`0.02518 ±0.00361`
Ch-MSE std ↓	`0.00259 ±0.00050` ★	`0.00598 ±0.00085`
ΔMSE/step ↓	`0.000954 ±0.000213` ★	`0.002096 ±0.000369`
ΔCos/step ↑	`-0.0010690 ±0.0003465` ★	`-0.0023829 ±0.0006197`

Lora

In this table, we compare the quality of our various lora approaches, against a standard bf16 lora loader baseline. The TLDR is that Pre-Lora is within marging of error of Dynamic Lora. Post-Lora is slightly worse. GGUF Q8 dequantizes to bf16 during inference to apply the lora math which is both slow and cheating. Nunchaku lora appears to be a little broken.

Interesting observation: These consistently score higher than their non-lora counterparts. I suspect it could be that there is a QAT like effect for applying loras trained with quantization to quantized models. Alternatively, maybe there is a reverse QAT like effect when using QLora on a BF16 model, lowering the quality, bringing it closer to quantized models.

Anima:

32 Samples per column.

Metric	INT8 ConvRot Pre-Lora	INT8 ConvRot Dynamic Lora	INT8 ConvRot Post-Lora	GGUF Q8_0 Lora	FP8 Lora
MSE ↓	`0.00073 ±0.00021` ★	`0.00090 ±0.00028`	`0.00186 ±0.00043`	`0.00158 ±0.00048`	`0.00641 ±0.00092`
MAE ↓	`0.01302 ±0.00091` ★	`0.01327 ±0.00113`	`0.01990 ±0.00175`	`0.01694 ±0.00196`	`0.04095 ±0.00311`
Max err ↓	`0.30456 ±0.04180`	`0.30054 ±0.04417` ★	`0.42361 ±0.04273`	`0.37600 ±0.04041`	`0.63406 ±0.04540`
Rel-RMSE ↓	`0.04963 ±0.00505` ★	`0.05100 ±0.00601`	`0.07606 ±0.00772`	`0.06606 ±0.00872`	`0.14956 ±0.01150`
SNR dB ↑	`27.39 ±0.59`	`27.52 ±0.65` ★	`24.32 ±0.66`	`26.01 ±0.83`	`18.25 ±0.64`
Cos-sim ↑	`0.997709 ±0.000687` ★	`0.997376 ±0.000778`	`0.994509 ±0.001267`	`0.995339 ±0.001409`	`0.981066 ±0.002814`
Var ratio →1	`1.00645 ±0.00144`	`0.99760 ±0.00114`	`1.00029 ±0.00176`	`1.00010 ±0.00113` ★	`0.98184 ±0.00368`
Outlier% ↓	`0.00002 ±0.00002`	`0.00001 ±0.00001` ★	`0.00005 ±0.00003`	`0.00004 ±0.00003`	`0.00023 ±0.00008`
Ch-MSE max ↓	`0.00141 ±0.00044` ★	`0.00194 ±0.00067`	`0.00400 ±0.00099`	`0.00314 ±0.00099`	`0.01286 ±0.00201`
Ch-MSE std ↓	`0.00035 ±0.00011` ★	`0.00047 ±0.00016`	`0.00098 ±0.00025`	`0.00079 ±0.00026`	`0.00326 ±0.00052`
ΔMSE/step ↓	`0.000077 ±0.000031` ★	`0.000099 ±0.000039`	`0.000194 ±0.000061`	`0.000167 ±0.000062`	`0.000599 ±0.000122`
ΔCos/step ↑	`-0.0001550 ±0.0001023` ★	`-0.0002021 ±0.0001127`	`-0.0004421 ±0.0001852`	`-0.0004007 ±0.0001874`	`-0.0014563 ±0.0004473`

Qwen Image 2512

16 Samples per column.

Metric	FP8	GGUF Q4 K M	GGUF Q8	INT8 ConvRot Post-Lora	INT8 ConvRot Pre-Lora	Nunchaku_BestQuality
MSE ↓	`0.01139 ±0.00146`	`0.00874 ±0.00147`	`0.00135 ±0.00058`	`0.00185 ±0.00050`	`0.00111 ±0.00032` ★	`0.04326 ±0.00328`
MAE ↓	`0.06940 ±0.00369`	`0.05205 ±0.00418`	`0.01490 ±0.00233` ★	`0.02129 ±0.00215`	`0.01637 ±0.00156`	`0.14596 ±0.00556`
Max err ↓	`0.83818 ±0.05885`	`0.68868 ±0.04720`	`0.37840 ±0.05948` ★	`0.45491 ±0.05199`	`0.38492 ±0.03914`	`1.08649 ±0.03813`
Rel-RMSE ↓	`0.18603 ±0.01147`	`0.14543 ±0.01242`	`0.04687 ±0.00796` ★	`0.06366 ±0.00756`	`0.05016 ±0.00546`	`0.36876 ±0.01457`
SNR dB ↑	`15.19 ±0.48`	`18.56 ±0.65`	`29.23 ±0.95` ★	`25.81 ±0.80`	`27.56 ±0.70`	`9.33 ±0.35`
Cos-sim ↑	`0.957885 ±0.005072`	`0.971353 ±0.004980`	`0.995827 ±0.001845`	`0.993908 ±0.001672`	`0.996241 ±0.001149` ★	`0.874391 ±0.010770`
Var ratio →1	`1.03367 ±0.00407`	`0.98059 ±0.00510`	`0.99394 ±0.00142`	`0.99124 ±0.00185`	`0.99651 ±0.00217` ★	`1.17955 ±0.01708`
Outlier% ↓	`0.00016 ±0.00005`	`0.00008 ±0.00003`	`0.00002 ±0.00001`	`0.00002 ±0.00001`	`0.00001 ±0.00000` ★	`0.00097 ±0.00018`
Ch-MSE max ↓	`0.02053 ±0.00283`	`0.01603 ±0.00271`	`0.00269 ±0.00108`	`0.00388 ±0.00111`	`0.00204 ±0.00053` ★	`0.08783 ±0.00643`
Ch-MSE std ↓	`0.00464 ±0.00075`	`0.00399 ±0.00073`	`0.00067 ±0.00029`	`0.00089 ±0.00025`	`0.00048 ±0.00013` ★	`0.02458 ±0.00197`
ΔMSE/step ↓	`0.000958 ±0.000252`	`0.001059 ±0.000261`	`0.000183 ±0.000087`	`0.000237 ±0.000087`	`0.000152 ±0.000071` ★	`0.003098 ±0.000772`
ΔCos/step ↑	`-0.0007560 ±0.0012897`	`-0.0025382 ±0.0009487`	`-0.0004476 ±0.0002795`	`-0.0005603 ±0.0003156`	`-0.0003486 ±0.0002816` ★	`-0.0054101 ±0.0030137`

Some loras require stochastic lora to work

Anima, 16 samples:

Metric	I8Dynamic	I8None	I8Pre	I8Stoch
MSE ↓	`0.01128 ±0.00309` ★	`0.23273 ±0.01757`	`0.01204 ±0.00277`	`0.01488 ±0.00385`
MAE ↓	`0.04268 ±0.00710` ★	`0.35066 ±0.01357`	`0.04434 ±0.00612`	`0.05195 ±0.00784`
Max err ↓	`0.77778 ±0.07449` ★	`1.83527 ±0.05833`	`0.83019 ±0.06939`	`0.90528 ±0.06125`
Rel-RMSE ↓	`0.11532 ±0.01719` ★	`0.79263 ±0.03690`	`0.12302 ±0.01520`	`0.13954 ±0.01769`
SNR dB ↑	`23.07 ±1.13` ★	`2.63 ±0.40`	`22.50 ±1.05`	`21.04 ±0.92`
Cos-sim ↑	`0.983092 ±0.004472` ★	`0.594366 ±0.035211`	`0.982433 ±0.003686`	`0.977338 ±0.005859`
Var ratio →1	`0.99949 ±0.00296` ★	`0.69981 ±0.03352`	`0.99129 ±0.00373`	`0.99126 ±0.00392`
Outlier% ↓	`0.00027 ±0.00009` ★	`0.02900 ±0.00621`	`0.00029 ±0.00009`	`0.00032 ±0.00009`
Ch-MSE max ↓	`0.01921 ±0.00513` ★	`0.50483 ±0.04664`	`0.02048 ±0.00468`	`0.02571 ±0.00644`
Ch-MSE std ↓	`0.00436 ±0.00115` ★	`0.13416 ±0.01344`	`0.00491 ±0.00116`	`0.00583 ±0.00143`
ΔMSE/step ↓	`0.001330 ±0.000360` ★	`0.015669 ±0.001669`	`0.001472 ±0.000345`	`0.001712 ±0.000422`
ΔCos/step ↑	`-0.0017781 ±0.0005309` ★	`-0.0205223 ±0.0046041`	`-0.0019693 ±0.0004810`	`-0.0023263 ±0.0006257`

Collecting your own measurements:

Use this custom node: https://github.com/BobJohnson24/ComfyUI-EvalSampler