skt/A.X-K2-GGUF

A.X K2 GGUF

🤗 Original model | 🤗 Collection | 🖥️ Github | 📄 Technical Report

This repository contains quantized GGUF builds of skt/A.X-K2 for llama.cpp.

A.X K2 is a Mixture-of-Experts model with 688B parameters, 33B of them active per token, released in block-scaled FP8. The file here is converted from that checkpoint and quantized to the format in Model files.

The original model card covers the architecture, training data, benchmarks, intended use and limitations. This card only adds what is specific to the GGUF build.

Requirements

Official llama.cpp does not support A.X K2 yet, so build from the A.X-K2 fork. It is upstream b10236 plus A.X K2 support, and nothing else:

git clone -b axk2-b10236 https://github.com/cys4/llama.cpp.git
cd llama.cpp

cmake -B build                    # CPU
cmake -B build -DGGML_CUDA=ON     # CUDA
cmake --build build -j

Model files

File	Size	Bits-per-weight
A.X-K2-IQ4_XS.gguf	345 GiB	4.30

hf download skt/A.X-K2-GGUF --include "A.X-K2-IQ4_XS.gguf" --local-dir .

Tensors left at full precision include output.weight, token_embd.weight, the Gated Norm projections (*norm_gate_a/b.weight), the sparse-attention indexer projection (*indexer.proj.weight), and the MoE router (*ffn_gate_inp.weight).

Quickstart

For anything the examples do not cover, see the llama.cpp documentation - it all applies here, as long as you build the fork above.

MODEL=A.X-K2-IQ4_XS.gguf

CLI

./build/bin/llama-cli -m "$MODEL" --temp 0.6 --top-p 0.95 -st -p "대한민국의 수도는?"

--temp and --top-p control the sampling: lower temperature is more deterministic, and top-p caps the cumulative probability of the token pool.

-st runs a single turn: llama-cli answers the prompt and exits. Without it, the CLI stays open for interactive chat.

Server

./build/bin/llama-server -m "$MODEL" --temp 0.6 --top-p 0.95 --host 0.0.0.0 --port 8080

The server exposes an OpenAI-compatible API at http://localhost:8080/v1. Without those flags it listens on 127.0.0.1:8080. Use --host 0.0.0.0 to let other machines connect.

Think / Non-Think mode

A.X K2 is a hybrid reasoning model. Thinking is on by default, so the commands above emit a <think> trace before the answer. Both llama-cli and llama-server let you set the mode explicitly:

--reasoning on       # emit the <think>...</think> trace before the answer (think mode, default)
--reasoning off      # concise direct answers (non-think mode)

A client can override the server's mode per request with chat_template_kwargs:

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "대한민국의 수도는?"}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

Contact

For questions about A.X K2 — including model behavior, deployment, and licensing — contact the A.X team at a.x@sk.com. Please send reports of vulnerabilities, harmful outputs, suspected misuse, or copyright infringement claims to the same address.

Citation

If you use A.X K2 in your research, please cite the technical report:

@techreport{axk2-2026,
      title={A.X K2 Technical Report},
      author={SK Telecom},
      year={2026},
      institution={SK Telecom},
      url={https://github.com/SKT-AI/A.X-K2/blob/main/A_X_K2_Tech_Report.pdf},
}