Instructions to use AesSedai/Step-3.7-Flash-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AesSedai/Step-3.7-Flash-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="AesSedai/Step-3.7-Flash-GGUF",
	filename="IQ2_S/Step-3.7-Flash-IQ2_S-00001-of-00003.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use AesSedai/Step-3.7-Flash-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use AesSedai/Step-3.7-Flash-GGUF with Ollama:
```
ollama run hf.co/AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
```

Unsloth Studio new

How to use AesSedai/Step-3.7-Flash-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/Step-3.7-Flash-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/Step-3.7-Flash-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AesSedai/Step-3.7-Flash-GGUF to start chatting

Pi new

How to use AesSedai/Step-3.7-Flash-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AesSedai/Step-3.7-Flash-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use AesSedai/Step-3.7-Flash-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use AesSedai/Step-3.7-Flash-GGUF with Docker Model Runner:
```
docker model run hf.co/AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
```

Lemonade

How to use AesSedai/Step-3.7-Flash-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Step-3.7-Flash-GGUF-Q4_K_M

List all available models

lemonade list

Step-3.7-Flash-GGUF

File size: 23,296 Bytes

b882a04

model_name,file_size_gb,bpw,Mean KLD_mean,0.04.309.160 I common_memory_breakdown_print,0.04.332.892 I common_params_fit_impl,0.04.332.897 I common_fit_params,0.04.353.444 I llama_model_loader,0.04.353.458 I llama_model_loader,0.04.353.459 I llama_model_loader,0.04.353.460 I llama_model_loader,0.04.353.461 I llama_model_loader,0.04.353.469 I llama_model_loader,0.04.353.476 I llama_model_loader,0.04.353.477 I llama_model_loader,0.04.353.478 I llama_model_loader,0.04.353.479 I llama_model_loader,0.04.353.480 I llama_model_loader,0.04.371.106 I llama_model_loader,0.04.371.107 I llama_model_loader,0.04.371.108 I llama_model_loader,0.04.371.112 I llama_model_loader,0.04.371.113 I llama_model_loader,0.04.371.115 I print_info,0.04.410.155 I load,0.04.418.991 I load,0.04.418.992 I load,0.04.419.065 I load,0.04.441.301 I load,0.04.441.321 I print_info,0.04.441.322 I print_info,0.04.441.323 I print_info,0.04.441.335 I print_info,0.04.441.336 I print_info,0.04.441.337 I print_info,0.04.441.339 I print_info,0.04.441.341 I print_info,0.04.441.342 I print_info,0.04.441.344 I print_info,0.04.441.345 I print_info,0.04.441.346 I print_info,0.04.441.348 I print_info,0.04.441.349 I print_info,0.04.441.350 I print_info,0.04.441.351 I print_info,0.04.441.352 I print_info,0.04.441.354 I print_info,0.04.441.355 I print_info,0.04.441.357 I print_info,0.04.441.358 I print_info,0.04.441.359 I print_info,0.04.441.360 I print_info,0.04.441.361 I print_info,0.05.004.750 I common_memory_breakdown_print,0.05.026.757 I common_params_fit_impl,0.05.026.762 I common_fit_params,0.05.047.101 I llama_model_loader,0.05.047.102 I llama_model_loader,0.05.047.116 I llama_model_loader,0.05.047.118 I llama_model_loader,0.05.047.119 I llama_model_loader,0.05.047.120 I llama_model_loader,0.05.047.132 I llama_model_loader,0.05.047.137 I llama_model_loader,0.05.047.138 I llama_model_loader,0.05.047.139 I llama_model_loader,0.05.047.140 I llama_model_loader,0.05.064.358 I llama_model_loader,0.05.064.359 I llama_model_loader,0.05.064.360 I llama_model_loader,0.05.064.363 I llama_model_loader,0.05.064.364 I llama_model_loader,0.05.064.365 I print_info,0.05.104.699 I load,0.05.113.282 I load,0.05.113.283 I load,0.05.113.347 I load,0.05.135.141 I load,0.05.135.155 I print_info,0.05.135.156 I print_info,0.05.135.157 I print_info,0.05.135.165 I print_info,0.05.135.166 I print_info,0.05.135.167 I print_info,0.05.135.168 I print_info,0.05.135.169 I print_info,0.05.135.171 I print_info,0.05.135.172 I print_info,0.05.135.173 I print_info,0.05.135.174 I print_info,0.05.135.175 I print_info,0.05.135.176 I print_info,0.05.135.177 I print_info,0.05.135.178 I print_info,0.05.135.179 I print_info,0.05.135.180 I print_info,0.05.135.181 I print_info,0.05.135.184 I print_info,0.05.135.185 I print_info,0.05.135.186 I print_info,0.05.135.187 I print_info,0.05.135.188 I print_info,0.05.135.189 I print_info,0.05.135.190 I print_info,0.05.135.191 I print_info,0.05.135.192 I print_info,0.05.135.194 I print_info,0.05.135.195 I print_info,0.05.135.196 I print_info,0.05.135.197 I print_info,0.05.135.198 I print_info,0.05.135.199 I print_info,0.05.135.200 I print_info,0.05.213.008 I common_memory_breakdown_print,0.05.237.650 I common_params_fit_impl,0.05.237.654 I common_fit_params,0.05.269.999 I llama_model_loader,0.05.270.000 I llama_model_loader,0.05.270.015 I llama_model_loader,0.05.270.017 I llama_model_loader,0.05.270.018 I llama_model_loader,0.05.270.026 I llama_model_loader,0.05.270.032 I llama_model_loader,0.05.270.033 I llama_model_loader,0.05.270.034 I llama_model_loader,0.05.270.035 I llama_model_loader,0.05.270.036 I llama_model_loader,0.05.270.037 I llama_model_loader,0.05.285.872 I llama_model_loader,0.05.285.873 I llama_model_loader,0.05.285.876 I llama_model_loader,0.05.285.877 I print_info,0.05.285.878 I print_info,0.05.324.812 I load,0.05.333.439 I load,0.05.333.440 I load,0.05.333.511 I load,0.05.355.590 I load,0.05.355.610 I print_info,0.05.355.611 I print_info,0.05.355.612 I print_info,0.05.355.623 I print_info,0.05.355.638 I print_info,0.05.355.639 I print_info,0.05.355.640 I print_info,0.05.355.642 I print_info,0.05.355.644 I print_info,0.05.355.645 I print_info,0.05.355.646 I print_info,0.05.355.647 I print_info,0.05.355.648 I print_info,0.05.355.649 I print_info,0.05.355.650 I print_info,0.05.355.651 I print_info,0.05.355.652 I print_info,0.05.355.653 I print_info,0.05.355.654 I print_info,0.05.355.657 I print_info,0.05.355.658 I print_info,0.05.355.659 I print_info,0.05.355.660 I print_info,0.05.528.691 I common_memory_breakdown_print,0.05.552.657 I common_params_fit_impl,0.05.552.661 I common_fit_params,0.05.573.363 I llama_model_loader,0.05.573.364 I llama_model_loader,0.05.573.384 I llama_model_loader,0.05.573.386 I llama_model_loader,0.05.573.387 I llama_model_loader,0.05.573.388 I llama_model_loader,0.05.573.396 I llama_model_loader,0.05.573.402 I llama_model_loader,0.05.573.403 I llama_model_loader,0.05.573.404 I llama_model_loader,0.05.573.405 I llama_model_loader,0.05.589.804 I llama_model_loader,0.05.589.805 I llama_model_loader,0.05.589.806 I llama_model_loader,0.05.589.808 I llama_model_loader,0.05.589.809 I llama_model_loader,0.05.589.810 I print_info,0.05.589.811 I print_info,0.05.630.168 I load,0.05.640.054 I load,0.05.640.055 I load,0.05.640.140 I load,0.05.664.413 I load,0.05.664.430 I print_info,0.05.664.431 I print_info,0.05.664.432 I print_info,0.05.664.441 I print_info,0.05.664.443 I print_info,0.05.664.444 I print_info,0.05.664.446 I print_info,0.05.664.447 I print_info,0.05.664.448 I print_info,0.05.664.449 I print_info,0.05.664.450 I print_info,0.05.664.451 I print_info,0.05.664.453 I print_info,0.05.664.454 I print_info,0.05.664.455 I print_info,0.05.664.456 I print_info,0.05.664.457 I print_info,0.05.664.458 I print_info,0.05.664.459 I print_info,0.05.664.460 I print_info,0.05.664.461 I print_info,0.05.664.462 I print_info,0.05.664.463 I print_info,0.05.664.465 I print_info,0.05.664.469 I print_info,0.05.664.489 I print_info,0.05.664.491 I print_info,0.05.664.492 I print_info,0.05.664.493 I print_info,0.05.664.494 I print_info,0.05.664.495 I print_info,0.05.664.496 I print_info,0.05.664.497 I print_info,0.06.704.467 I common_memory_breakdown_print,0.06.728.088 I common_params_fit_impl,0.06.728.093 I common_fit_params,0.06.748.712 I llama_model_loader,0.06.748.713 I llama_model_loader,0.06.748.733 I llama_model_loader,0.06.748.735 I llama_model_loader,0.06.748.736 I llama_model_loader,0.06.748.745 I llama_model_loader,0.06.748.752 I llama_model_loader,0.06.748.753 I llama_model_loader,0.06.748.754 I llama_model_loader,0.06.748.755 I llama_model_loader,0.06.765.036 I llama_model_loader,0.06.765.037 I llama_model_loader,0.06.765.038 I llama_model_loader,0.06.765.040 I llama_model_loader,0.06.765.041 I llama_model_loader,0.06.765.042 I print_info,0.06.765.043 I print_info,0.06.804.648 I load,0.06.813.975 I load,0.06.814.041 I load,0.06.835.935 I load,0.06.835.951 I print_info,0.06.835.952 I print_info,0.06.835.953 I print_info,0.06.835.962 I print_info,0.06.835.964 I print_info,0.06.835.965 I print_info,0.06.835.967 I print_info,0.06.835.968 I print_info,0.06.835.969 I print_info,0.06.835.970 I print_info,0.06.835.973 I print_info,0.06.835.974 I print_info,0.06.835.975 I print_info,0.06.835.976 I print_info,0.06.835.977 I print_info,0.06.835.978 I print_info,0.06.835.979 I print_info,0.06.835.980 I print_info,0.06.835.981 I print_info,0.06.835.982 I print_info,0.06.835.983 I print_info,0.06.835.984 I print_info,0.06.835.985 I print_info,0.06.835.986 I print_info,0.1% KLD,0.1% Δp,0.33.267.030 I load_tensors,0.33.267.037 I load_tensors,0.33.267.038 I load_tensors,0.33.267.039 I load_tensors,0.33.267.040 I load_tensors,0.33.267.041 I load_tensors,0.34.251.172 I load_tensors,0.34.251.173 I load_tensors,0.34.251.176 I load_tensors,0.34.251.177 I load_tensors,0.34.251.178 I load_tensors,0.34.251.179 I load_tensors,0.35.869.309 I load_tensors,0.35.869.315 I load_tensors,0.35.869.316 I load_tensors,0.35.869.317 I load_tensors,0.35.869.318 I load_tensors,0.36.595.829 I llama_context,0.36.595.830 I llama_context,0.36.595.834 I llama_context,0.36.595.835 I llama_context,0.36.599.156 I llama_context,0.36.599.167 I llama_kv_cache_iswa,0.36.599.470 I llama_kv_cache,0.36.599.726 I llama_kv_cache,0.36.599.927 I llama_kv_cache,0.36.600.133 I llama_kv_cache,0.36.600.335 I llama_kv_cache,0.36.600.537 I llama_kv_cache,0.36.600.768 I llama_kv_cache,0.36.600.974 I llama_kv_cache,0.36.601.015 I llama_kv_cache,0.36.601.016 I llama_kv_cache,0.36.601.017 I llama_kv_cache_iswa,0.36.601.302 I llama_kv_cache,0.36.601.533 I llama_kv_cache,0.36.601.799 I llama_kv_cache,0.36.602.034 I llama_kv_cache,0.36.602.278 I llama_kv_cache,0.36.602.520 I llama_kv_cache,0.36.606.391 I llama_kv_cache,0.36.606.650 I llama_kv_cache,0.36.606.717 I llama_kv_cache,0.36.606.718 I llama_kv_cache,0.36.713.418 I sched_reserve,0.36.713.432 I sched_reserve,0.36.713.433 I sched_reserve,0.36.713.434 I sched_reserve,0.36.713.435 I sched_reserve,0.36.713.436 I sched_reserve,0.36.713.437 I sched_reserve,0.36.713.438 I sched_reserve,0.36.889.960 I system_info,0.36.905.368 I kl_divergence,0.38.722.455 I llama_context,0.38.722.456 I llama_context,0.38.722.461 I llama_context,0.38.722.462 I llama_context,0.38.725.707 I llama_context,0.38.725.717 I llama_kv_cache_iswa,0.38.726.044 I llama_kv_cache,0.38.726.270 I llama_kv_cache,0.38.726.481 I llama_kv_cache,0.38.726.705 I llama_kv_cache,0.38.726.900 I llama_kv_cache,0.38.727.102 I llama_kv_cache,0.38.727.311 I llama_kv_cache,0.38.727.511 I llama_kv_cache,0.38.727.575 I llama_kv_cache,0.38.727.576 I llama_kv_cache,0.38.727.578 I llama_kv_cache_iswa,0.38.727.851 I llama_kv_cache,0.38.728.096 I llama_kv_cache,0.38.728.364 I llama_kv_cache,0.38.728.632 I llama_kv_cache,0.38.728.859 I llama_kv_cache,0.38.729.102 I llama_kv_cache,0.38.729.369 I llama_kv_cache,0.38.733.545 I llama_kv_cache,0.38.733.628 I llama_kv_cache,0.38.733.629 I llama_kv_cache,0.38.833.657 I sched_reserve,0.38.833.670 I sched_reserve,0.38.833.671 I sched_reserve,0.38.833.672 I sched_reserve,0.38.833.673 I sched_reserve,0.38.833.674 I sched_reserve,0.38.833.675 I sched_reserve,0.39.023.225 I system_info,0.39.308.636 I llama_context,0.39.308.637 I llama_context,0.39.308.638 I llama_context,0.39.308.642 I llama_context,0.39.312.202 I llama_context,0.39.312.210 I llama_kv_cache_iswa,0.39.312.515 I llama_kv_cache,0.39.312.757 I llama_kv_cache,0.39.312.957 I llama_kv_cache,0.39.313.195 I llama_kv_cache,0.39.313.389 I llama_kv_cache,0.39.313.593 I llama_kv_cache,0.39.313.801 I llama_kv_cache,0.39.313.992 I llama_kv_cache,0.39.314.035 I llama_kv_cache,0.39.314.036 I llama_kv_cache,0.39.314.037 I llama_kv_cache_iswa,0.39.314.345 I llama_kv_cache,0.39.314.592 I llama_kv_cache,0.39.314.860 I llama_kv_cache,0.39.315.135 I llama_kv_cache,0.39.315.378 I llama_kv_cache,0.39.315.633 I llama_kv_cache,0.39.315.893 I llama_kv_cache,0.39.316.175 I llama_kv_cache,0.39.316.242 I llama_kv_cache,0.39.316.243 I llama_kv_cache,0.39.428.668 I sched_reserve,0.39.428.680 I sched_reserve,0.39.428.681 I sched_reserve,0.39.428.682 I sched_reserve,0.39.428.683 I sched_reserve,0.39.428.684 I sched_reserve,0.39.428.685 I sched_reserve,0.39.614.422 I system_info,0.40.415.977 I kl_divergence,0.40.962.299 I kl_divergence,0.41.514.599 I load_tensors,0.41.514.600 I load_tensors,0.41.514.605 I load_tensors,0.41.514.606 I load_tensors,0.41.514.607 I load_tensors,0.41.514.608 I load_tensors,0.47.036.789 I llama_context,0.47.036.790 I llama_context,0.47.036.791 I llama_context,0.47.036.796 I llama_context,0.47.036.797 I llama_context,0.47.040.030 I llama_context,0.47.040.041 I llama_kv_cache_iswa,0.47.040.412 I llama_kv_cache,0.47.040.661 I llama_kv_cache,0.47.040.875 I llama_kv_cache,0.47.041.125 I llama_kv_cache,0.47.041.327 I llama_kv_cache,0.47.041.537 I llama_kv_cache,0.47.041.739 I llama_kv_cache,0.47.041.953 I llama_kv_cache,0.47.041.997 I llama_kv_cache,0.47.041.999 I llama_kv_cache_iswa,0.47.042.321 I llama_kv_cache,0.47.042.564 I llama_kv_cache,0.47.042.816 I llama_kv_cache,0.47.043.062 I llama_kv_cache,0.47.043.322 I llama_kv_cache,0.47.043.558 I llama_kv_cache,0.47.043.808 I llama_kv_cache,0.47.044.052 I llama_kv_cache,0.47.044.123 I llama_kv_cache,0.47.154.155 I sched_reserve,0.47.154.169 I sched_reserve,0.47.154.170 I sched_reserve,0.47.154.171 I sched_reserve,0.47.154.172 I sched_reserve,0.47.154.173 I sched_reserve,0.47.154.174 I sched_reserve,0.47.154.175 I sched_reserve,0.47.154.176 I sched_reserve,0.47.359.453 I system_info,0.48.750.107 I kl_divergence,0.49.906.258 I load_tensors,0.49.906.264 I load_tensors,0.49.906.265 I load_tensors,0.49.906.266 I load_tensors,0.49.906.267 I load_tensors,0.56.770.944 I llama_context,0.56.770.945 I llama_context,0.56.770.949 I llama_context,0.56.770.950 I llama_context,0.56.774.102 I llama_context,0.56.774.113 I llama_kv_cache_iswa,0.56.774.418 I llama_kv_cache,0.56.774.672 I llama_kv_cache,0.56.774.864 I llama_kv_cache,0.56.775.066 I llama_kv_cache,0.56.775.256 I llama_kv_cache,0.56.775.470 I llama_kv_cache,0.56.775.681 I llama_kv_cache,0.56.775.884 I llama_kv_cache,0.56.775.928 I llama_kv_cache,0.56.775.929 I llama_kv_cache,0.56.775.930 I llama_kv_cache_iswa,0.56.776.193 I llama_kv_cache,0.56.776.443 I llama_kv_cache,0.56.776.706 I llama_kv_cache,0.56.776.938 I llama_kv_cache,0.56.777.160 I llama_kv_cache,0.56.777.416 I llama_kv_cache,0.56.779.045 I llama_kv_cache,0.56.779.277 I llama_kv_cache,0.56.779.357 I llama_kv_cache,0.56.887.586 I sched_reserve,0.56.887.599 I sched_reserve,0.56.887.600 I sched_reserve,0.56.887.601 I sched_reserve,0.56.887.602 I sched_reserve,0.56.887.603 I sched_reserve,0.56.887.604 I sched_reserve,0.57.122.427 I system_info,0.58.666.989 I kl_divergence,1.0% KLD,1.0% Δp,10.0% KLD,10.0% Δp,2.00.156.208 I llama_perf_context_print,2.00.156.211 I llama_perf_context_print,2.00.156.213 I llama_perf_context_print,2.00.156.443 I common_memory_breakdown_print,2.08.859.306 I llama_perf_context_print,2.08.859.310 I llama_perf_context_print,2.08.859.311 I llama_perf_context_print,2.08.859.526 I common_memory_breakdown_print,2.18.249.549 I llama_perf_context_print,2.18.249.554 I llama_perf_context_print,2.18.249.777 I common_memory_breakdown_print,2.19.402.996 I llama_perf_context_print,2.19.403.000 I llama_perf_context_print,2.19.403.186 I common_memory_breakdown_print,2.21.704.462 I llama_perf_context_print,2.21.704.466 I llama_perf_context_print,2.21.704.706 I common_memory_breakdown_print,25.0% Δp,5.0% KLD,5.0% Δp,75.0% Δp,90.0% KLD,90.0% Δp,95.0% KLD,95.0% Δp,99.0% KLD,99.0% Δp,99.9% KLD,99.9% Δp,"Cor(ln(PPL(Q)), ln(PPL(base)))",Maximum KLD,Maximum Δp,Mean KLD_std,Mean PPL(Q)-PPL(base)_mean,Mean PPL(Q)-PPL(base)_std,Mean PPL(Q)/PPL(base)_mean,Mean PPL(Q)/PPL(base)_std,Mean PPL(Q)_mean,Mean PPL(Q)_std,Mean PPL(base)_mean,Mean PPL(base)_std,Mean ln(PPL(Q)/PPL(base))_mean,Mean ln(PPL(Q)/PPL(base))_std,Mean Δp_mean,Mean Δp_std,Median KLD,Median Δp,Minimum KLD,Minimum Δp,RMS Δp_mean,RMS Δp_std,Same top p_mean,Same top p_std,file_path,file_size_gib,is_self_ref,mixture
Step-3.7-Flash-IQ2_S (aes_sedai),66.59346792448001,2.71,0.623919,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-7984130385.0,94475.773503,4.22,-0.35,-1.0,-6.1,-935.324096,-1035.3211264,-1435.322,-2035.32288,-2135.328,-2235.32128,-2335.32128,-2535.0,-2735.321,-46.2,-48.6,-342.0,3.0,6.0,0.0,-1.0,-128007.0,818.0,0.822,35.0,262144.0,45.0,6.496969664969696e+89,8.0,1.0,128.0,8.121212812121281e+77,1024.0,1024.0,0.0,0.0,0.0,1.0,2.0,1.0,262144.0,196.11,196.96,0.0,201.0,1.0,256.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.0,-99.399,4646.0,4987.72,18890.9,37404.69,68864.55,76331.52,,,,,,,,,,,,512.0,1.0,5000000.0,0.0,7.87,-512.0,64.0,132.0,264.0,332.0,464.0,532.0,664.0,732.0,128.0,128.0,512.0,128.0,1160.0,2128.0,3128.0,4128.0,5160.0,6128.0,796.0,128.0,128.0,2924.12,13073.12,23073.12,43073.12,74989.25,385.38,9.0,106.624,4.848561200112841e+50,561512819216.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1.2e-05,-95.82,0.000786,-64.886,,,,,,,,,,,,31603.38,34.0,-7984130385.0,,,,-28.059,0.000176,-81.793,-0.058,1.876508,1.718,2.828169,10.191,5.058762,39.567,7.964636,74.823,70.37,12.580204,96.632,0.00281,1.550603,0.013523,1.819489,0.006613,3.442762,0.017581,1.892159,0.007192,0.598556,0.003634,-16.844,0.076,0.168992,-3.968,-0.000102,-99.999,33.237,0.086,74.215,0.116,kld/Step-3.7-Flash/wiki-test-raw/aes_sedai/Step-3.7-Flash-IQ2_S.md,62.02,False,Q6_K / IQ2_XS / IQ2_XS / IQ3_XXS
Step-3.7-Flash-IQ3_S (aes_sedai),73.52984010752,2.99,0.460066,-7984130385.0,101090.773503,3.89,-1.0,-6.1,-735.3245,-935.324096,-1035.3211264,-1435.322,-2035.32288,-2135.328,-2335.32128,-2535.0,-2735.321,-45.2,-47.3,-48.6,-32266.0,-284.0,6.0,0.0,-1.0,-128007.0,818.0,0.822,35.0,0.0,45.0,6.496969664969696e+89,512.0,128.0,8.121212812121281e+77,1024.0,0.0,0.0,288.0,2.0,1.0,262144.0,0.0,196.11,196.96,128896.0,127741.0,128007.0,1.0,128800.0,128007.0,256.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-0.0,-98.6,,,,,,,,,,,,,4646.0,413.03,19835.9,38192.19,76961.52,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,16.0,8192.0,1.0,0.0,7.87,-512.0,64.0,132.0,264.0,332.0,464.0,532.0,664.0,732.0,128.0,128.0,512.0,128.0,1160.0,2128.0,3128.0,4128.0,5160.0,6128.0,796.0,128.0,128.0,2924.12,13073.12,43073.12,74989.25,385.38,9.0,112.354,4.848561200112841e+50,,561512819216.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,7e-06,-92.45,0.000368,-53.017,,,,,,,,,35242.67,34.0,-7984130385.0,,,,,,,-19.21,8.5e-05,-72.722,-0.01,1.402419,2.673,2.191751,11.189,4.06242,39.652,6.674603,74.318,76.52,12.281671,98.967,0.002237,1.025134,0.009553,1.54178,0.004722,2.917292,0.013858,1.892159,0.007192,0.432938,0.003063,-12.807,0.068,0.100463,-1.919,-0.000338,-99.984,28.638,0.084,78.441,0.109,kld/Step-3.7-Flash/wiki-test-raw/aes_sedai/Step-3.7-Flash-IQ3_S.md,68.48,False,Q6_K / IQ2_S / IQ2_S / IQ3_S
Step-3.7-Flash-IQ4_XS (aes_sedai),95.45564815360001,3.88,0.159934,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-9205340385.0,121877.773503,4.57,-0.35,-1.0,-6.1,-735.3245,-935.324096,-1035.3211264,-1435.322,-2035.32288,-2335.32128,-2635.323,-2735.321,-43.322,-46.3,-48.8,-384.0,-442.0,80.0,0.0,-1.0,-128007.0,818.0,0.822,35.0,262144.0,45.0,6.496969664969696e+89,8.0,512.0,1.0,128.0,8.121212812121281e+77,1024.0,1024.0,0.0,0.0,0.0,11264.0,8.0,0.0,1.0,2.0,5000000.0,1.0,10000.0,128.0,128.0,262144.0,0.0,196.11,196.96,128896.0,0.0,128007.0,1.0,128801.0,128007.0,256.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-3e-06,-88.877,,,,,,,44.0,4646.0,534.97,112769.09,412734.95,79036.33,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,512.0,1.0,1.0,0.0,7.87,-512.0,64.0,132.0,264.0,332.0,464.0,532.0,664.0,732.0,128.0,128.0,512.0,128.0,1160.0,2128.0,3128.0,4128.0,5160.0,6128.0,796.0,128.0,128.0,2924.12,13073.12,23073.12,53073.12,74989.25,9.0,99.954,4.848561200112841e+50,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,561512819216.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1e-06,-67.647,7e-05,-22.437,33958.23,81133.22287233,34.0,-9205340385.0,,,,,,,,,,,,,,-5.258,1.7e-05,-38.017,0.017,0.465278,4.038,0.786417,11.589,1.731296,34.626,3.459361,65.419,91.05,17.015841,99.788,0.000946,0.295544,0.00388,1.156194,0.001976,2.187702,0.00904,1.892159,0.007192,0.145133,0.001709,-4.692,0.042,0.022801,-0.181,-0.000337,-99.385,16.698,0.068,88.144,0.085,kld/Step-3.7-Flash/wiki-test-raw/aes_sedai/Step-3.7-Flash-IQ4_XS.md,88.9,False,Q8_0 / IQ3_S / IQ3_S / IQ4_XS
Step-3.7-Flash-Q4_K_M (aes_sedai),122.21329440768,4.96,0.047855,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-9205340385.0,147392.773503,5.1,-0.35,-1.0,-6.1,-735.3245,-935.324096,-1035.3211264,-1435.322,-2135.328,-2335.32128,-2535.0,-2735.321,-44.327,-47.5,-48.8,-32266.0,-542.0,3.0,80.0,0.0,-1.0,-128007.0,818.0,0.822,35.0,0.0,45.0,6.496969664969696e+89,1.0,128.0,8.121212812121281e+77,1024.0,1024.0,0.0,0.0,0.0,0.0,8.0,0.0,-1.0,2.0,5000000.0,10000.0,128.0,128.0,262144.0,0.0,196.96,127741.0,0.0,128007.0,1.0,128801.0,128800.0,128802.0,1.0,256.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-4e-06,-57.904,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,44.0,4646.0,534.97,116414.09,416379.95,711466.33,16.0,512.0,1.0,5000000.0,0.0,7.87,-512.0,64.0,132.0,264.0,332.0,464.0,532.0,664.0,732.0,128.0,512.0,128.0,1160.0,2128.0,3128.0,4128.0,5160.0,6128.0,796.0,128.0,2924.12,13073.12,23073.12,33073.12,63073.12,74989.25,385.38,9.0,109.954,4.848561200112841e+50,561512819216.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-0.0,-35.418,1.6e-05,-8.168,,,,,41768.97,81500.09287233,34.0,-9205340385.0,,,,,,,,,,-1.21,4e-06,-15.912,0.202,0.135596,3.996,0.236381,9.449,0.560039,25.557,1.230636,47.067,97.23,3.140392,80.872,0.00031,0.065501,0.001794,1.034617,0.000941,1.957659,0.007616,1.892159,0.007192,0.034031,0.00091,-1.104,0.024,0.005462,-0.006,-0.000456,-89.721,8.961,0.046,93.716,0.064,kld/Step-3.7-Flash/wiki-test-raw/aes_sedai/Step-3.7-Flash-Q4_K_M.md,113.82,False,Q8_0 / Q4_K / Q4_K / Q5_K
Step-3.7-Flash-Q5_K_M (aes_sedai),146.49059704832,5.95,0.017019,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-9205340385.0,170545.773503,6.28,-0.35,-1.0,-6.1,-835.32262144,-1035.3211264,-1435.322,-2135.328,-2335.32128,-2635.323,-2735.321,-43.322,-46.5,-48.8,-32266.0,-642.0,3.0,80.0,0.0,-128007.0,818.0,0.822,35.0,262144.0,45.0,6.496969664969696e+89,1.0,128.0,8.121212812121281e+77,1024.0,1024.0,0.0,0.0,8.0,2.0,5000000.0,1.0,0.0,196.11,196.96,127741.0,128007.0,128801.0,128802.0,1.0,256.0,-6e-06,-36.256,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,4646.0,534.97,219687.45,519721.59,713671.33,8192.0,1.0,5000000.0,0.0,7.87,-512.0,64.0,132.0,264.0,332.0,464.0,532.0,664.0,732.0,128.0,128.0,512.0,128.0,1160.0,2128.0,3128.0,4128.0,5160.0,6128.0,796.0,128.0,2924.12,23073.12,43073.12,74989.25,385.38,9.0,108.164,4.848561200112841e+50,561512819216.0,-1e-06,-19.411,6e-06,-4.097,,,,,,,,,,,,,,,50356.68,34.0,-9205340385.0,-0.503,1e-06,-8.288,0.205,0.046934,2.888,0.081756,6.395,0.200418,16.39,0.487906,31.67,99.02,2.369091,64.64,0.00012,0.019841,0.001026,1.010486,0.000541,1.912,0.00733,1.892159,0.007192,0.010431,0.000536,-0.326,0.014,0.001884,-0.001,-0.000512,-66.874,5.283,0.03,96.302,0.05,kld/Step-3.7-Flash/wiki-test-raw/aes_sedai/Step-3.7-Flash-Q5_K_M.md,136.43,False,Q8_0 / Q5_K / Q5_K / Q6_K