CAT-UT

The model files at the repository root are a native BF16 export of the dense int4 baseline. They can be loaded with standard Hugging Face tooling and SGLang. The source is openbmb/MiniCPM5-2B at revision 12a3808a956f869c767195e9266b59c4d21d92e2; the export records its source QPR checksum and each safetensors checksum in qprsi-export.json.

The accompanying QPR files preserve compact, dense int4 checkpoints. The quantizer uses group size 128 and GPTQ calibration without 2:4 pruning. QPR requires the separate qprsi loader; the BF16 export is about 5 GB, while each QPR file is about 1.30 GB. BF16 export changes the storage format, not the underlying quantized weights.

The refined candidate is dense-int4-distill2000-v2-best.qpr. The minicpm5-2b-dense-int4-g128.qpr file is the matching dense-int4 baseline. Each QPR file is about 1.30 GB and is stored with Git LFS. The candidate has not passed the current production acceptance gate. The native files at the repository root therefore contain the baseline, not the candidate.

Evaluation

The candidate was selected from a 2,000-update supervised refinement run starting at the dense-int4 baseline. It was selected at update 775 and independently reloaded before scoring.

Artifact	Wide code NLL	Wide chat NLL	Wide tool NLL
Dense int4 baseline	1.0654	5.1728	2.5966
Refined candidate	1.0323	5.0707	2.6200

In the current answer-only, 512-token bilingual chat check, an external pairwise judge gave 47 baseline wins, 49 candidate wins, and 4 ties across 100 prompts. The Chinese subset was 26/22/2 and the English subset 21/27/2 (baseline/candidate/tie). The candidate failed the Chinese non-regression gate. Candidate repetition was 0.0295, language mixing 0.0007, and truncation 0.2725; these automatic metrics passed their separate gates.

On 20 executable Python tasks, the baseline passed 14 and the candidate passed 15. Chinese results were 7/10 for both; English results were 7/10 and 8/10 respectively. A different, earlier chat protocol produced 39 baseline wins, 58 candidate wins, and 3 ties, but it does not determine release status under the current answer-only protocol.

The native BF16 baseline scored 51.92% MMLU (14,042 test questions) and 52.45% C-Eval (1,346 public validation questions) with five-shot continuation scoring in lm-eval 0.4.13. These are baseline measurements, not evidence of a refinement gain. The knowledge evaluation record contains the protocol and subject-level results. The update-775 candidate scored 51.81% MMLU and 51.86% C-Eval on the same complete tasks. Paired question comparison found 638 baseline-only versus 623 candidate-only correct MMLU answers, and 69 versus 61 on C-Eval. These differences are small and favor the baseline. Choice scoring uses no sampling temperature or deployed chat template, so the chat and code checks remain separate. GPQA Diamond has not yet been measured because its official dataset requires authenticated access.

Direct SGLang loading

On a BF16-capable NVIDIA GPU with a CUDA 13-compatible driver, install SGLang in its own environment and serve the model ID directly:

python -m pip install "sglang[srt]==0.5.20"
sglang serve \
  --model-path AvrovaDonz/CAT-UT \
  --served-model-name CAT-UT \
  --dtype bfloat16 --context-length 8192 \
  --mem-fraction-static 0.7 \
  --host 127.0.0.1 --port 30000

For answer-only chat, send "chat_template_kwargs": {"enable_thinking": false} in the chat-completions request. The deployment guide specifies the system prompts used in evaluation. The root BF16 files also load with transformers.AutoModelForCausalLM.from_pretrained. An RTX 4080 SUPER smoke test with SGLang 0.5.20 returned Chinese, English, and Python responses; the serving record lists the request settings and observed outputs.

Compact QPR loading

QPR is a custom artifact format and is not loaded directly by transformers.from_pretrained. Use the qprsi source at commit 8701707. The deployment README pins the base model to revision 12a3808a956f869c767195e9266b59c4d21d92e2 and provides an inference CLI, dependency lock, and SHA-256 verification.

After installing the base model and qprsi package, run:

qprsi-infer \
  --model /path/to/MiniCPM5-2B \
  --model-revision 12a3808a956f869c767195e9266b59c4d21d92e2 \
  --artifact /path/to/minicpm5-2b-dense-int4-g128.qpr \
  --expected-sha256 7b0ad4afe9eaf88ff6502cd7041487b8e1d833c00c40ec0eec2ac1eebfc35c02 \
  --language zh --prompt '请用两句话解释为什么雨后会有彩虹。'

For the candidate, replace the artifact and SHA-256 with dense-int4-distill2000-v2-best.qpr and 306d6475e1b0d07dc28a1b260186458a0494157a43a3c1028ba4fcc6af5b3615. Reproduce the current executable code check with:

python -m qprsi.code_eval \
  --model /path/to/MiniCPM5-2B \
  --artifacts /path/to/minicpm5-2b-dense-int4-g128.qpr /path/to/dense-int4-distill2000-v2-best.qpr \
  --answer-only --max-new-tokens 256 \
  --report code-eval.json

The QPR loader must use the same model architecture and tokenizer revision as the base model. QPR itself is not a drop-in Transformers, SGLang, or vLLM checkpoint; the native BF16 files at the repository root are for direct use.

Training details

Supervised refinement used cached reference responses under permission. The run used batch size 16, learning rate 3e-6, KL beta 0.05, last-block training, 1,075 executed updates, and wide-probe selection with a 1% code/tool regression gate. The selected update was 775. No API credentials are included in the artifacts or reports.

Limitations

• The chat comparison used one 100-prompt set and one external judge; the result is not a benchmark guarantee.
• Generation was capped at 512 new tokens. The answer-only prefill avoids spending that budget on the model's visible reasoning section.
• The 20-task code check and 100-prompt chat check are small samples. They do not establish a general quality gain.
• A separate 200-update rich-task pilot matched the baseline at 3/10 on that code check and lost its 100-prompt chat comparison (41 pilot wins, 51 baseline wins, 8 ties), so that experimental QPR is not included here.
• The QPR artifacts require the matching base model for loading. The native BF16 model files at the repository root are self-contained.

Licensing

The upstream openbmb/MiniCPM5-2B weights are released under Apache-2.0. The qprsi conversion and evaluation source is released under AGPL-3.0-or-later. The repository metadata currently declares MPL-2.0 for this CAT-UT repository; downstream users should review all applicable terms before redistribution.