Instructions to use XHToken/Spark-X2.5-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use XHToken/Spark-X2.5-4B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="XHToken/Spark-X2.5-4B", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("XHToken/Spark-X2.5-4B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use XHToken/Spark-X2.5-4B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "XHToken/Spark-X2.5-4B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "XHToken/Spark-X2.5-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/XHToken/Spark-X2.5-4B
- SGLang
How to use XHToken/Spark-X2.5-4B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "XHToken/Spark-X2.5-4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "XHToken/Spark-X2.5-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "XHToken/Spark-X2.5-4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "XHToken/Spark-X2.5-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use XHToken/Spark-X2.5-4B with Docker Model Runner:
docker model run hf.co/XHToken/Spark-X2.5-4B
[HER Hack-Astron #6] Spark-X2.5-4B 原生计算器配对评测:63 次工具调用均成功,为什么 pass@1 反而下降?
我实际运行了官方 XHToken/Spark-X2.5-4B 原始 BF16 权重,在 Apple
GSM-Symbolic 的 main / p1 / p2 三档上做了 60 道题、120 条轨迹的同题配对
评测。结果先说:
| config | 直接策略 | 计算器策略 | 样本内差值 | McNemar p | Holm p |
|---|---|---|---|---|---|
| main | 17/20 = 85% | 13/20 = 65% | -20 pp | 0.12500 | 0.37500 |
| p1 | 16/20 = 80% | 12/20 = 60% | -20 pp | 0.21875 | 0.43750 |
| p2 | 12/20 = 60% | 10/20 = 50% | -10 pp | 0.68750 | 0.68750 |
| 总计(描述性) | 45/60 = 75.00% | 35/60 = 58.33% | -16.67 pp | 不做汇总检验 | 不适用 |
三个 config 的 Holm 校正结果都不显著,所以这不是“计算器降低模型能力”的普遍
结论。更有价值的发现是:模型在真实 Agent 工具协议中,可能不是算错,而是无法在
有限调用预算内正确结束对话。
核心发现:执行器没错,答案却没有交付
计算器策略的 60 条轨迹中:
- 40 条实际发出原生 function call,20 条没有遵循必调策略;
- 共 63 次调用,安全执行器重放结果为 63/63 成功、0 executor error;
- 23 条用满两次调用额度,其中 18 条在 schema 撤除后仍输出文字化第三个
<tool_call>,没有FINAL_ANSWER; - 直接策略可解析 55/60,计算器策略可解析 38/60;全部 27 条不可解析恰好由
9 条 token 截断和上述 18 条工具协议失败组成。
| 计算器策略的实际调用数 | 轨迹数 | 严格正确 | 可解析末答 |
|---|---|---|---|
| 0 | 20 | 16 | 17 |
| 1 | 17 | 14 | 16 |
| 2 | 23 | 5 | 5 |
这个按调用数的表是后验描述,不是因果估计。13 个“仅直接策略答对”的 pair 中,
12 个对应两次调用后的末答交付失败;3 个“仅计算器策略答对”的 pair 中,只有
2 个真的调用了工具,另 1 个只是策略条件不同。因此本文始终把差异归因于
“策略前缀 + 工具访问”的联合条件,不把它包装成纯工具效应。
这也给部署留下了可操作建议:将“算式执行成功”“题意建模正确”“模型遵循工具
策略”和“最终答案已交付”设为四个独立观测项;多步题不要只给固定两次调用后突然
撤掉 schema,应提供明确的剩余预算/结束信号,或让模型一次提交组合表达式。
方法与严格口径
- 模型:
XHToken/Spark-X2.5-4B,revision5e10fcc0286756aebf7c41dc52c1e42d95c70281,官方 BF16,无量化、无训练; - 数据:
apple/GSM-Symbolic,revision93b5b3758d9d9841ffe81d6cd2ae2b030685b078,testsplit; - 抽样:对
main/p1/p2的公共(id, instance)做固定盐 SHA256 选择,20 个
模板键 × 3 档 = 60 题;模型只收到question,从未收到 answer/rationale/canary; - 配对:每题分别运行直接解题策略和要求原生 calculator 的策略;英文题面字节相同,
两臂仅策略前缀和工具 schema 不同; - 解码:thinking off、temperature 0、top_p 1、top_k -1、
repetition_penalty 1、n=1、base seed20260913,episode seed 按题固定推导;
累计 completion 上限 4096,最多两次工具调用; - 判分:只抽取终局唯一完整的
FINAL_ANSWER: <number>,以 Fraction 精确比较
数据集末行####;不可解析、截断、协议失败均留在 120 条分母并计错; - 统计:显著性阈值
α=0.05;每 config 用精确双侧 McNemar,并对三次检验做
Holm 校正。三档复用了模板块,所以总计只描述,不做伪精确的 pooled
significance test。这里只报告 pass@1;没有 pass@k、self-consistency 或平局。
可复查的冻结输入与设置
正式评测绑定运行前协议 commitcdfe83a112308380c6a4bae5adb9de14f38c7bc2。选择盐与模板键清单哈希分别为:
HER-Hack-Astron-6|Spark-X2.5-4B|apple/GSM-Symbolic|93b5b3758d9d9841ffe81d6cd2ae2b030685b078|paired-v1
73538354c63a79e00ee986cdac83cd748d14e256dc0015ee9dd4c231ca922d77
每条轨迹的 base seed 是 20260913,episode seed 为20260913 + int(SHA256("seed|<eval_id>")[:8], 16) % 1_000_000;后续轮次依次
加 1。两臂的完整提示如下(随后拼接完全相同的 题目(原文): 与英文题面):
展开查看完整 prompt 与工具 schema
System prompt:
Solve the supplied math problem carefully. Follow the user policy. After any tool result, give a concise, checkable derivation. End with exactly one line: FINAL_ANSWER: <number>, where <number> has no unit. Do not write anything after that line.
直接策略 prefix:
请独立理解下面的英文数学题并手工计算,不要使用任何外部工具。给出简洁、可检查的推导,并以 FINAL_ANSWER: <number> 结尾。
计算器策略 prefix:
请先理解下面的英文数学题并确定所需算式,然后必须调用 calculator 工具精确计算。不要在工具返回前给最终答案。工具返回后再简要说明推导,并以 FINAL_ANSWER: <number> 结尾。
原生工具 schema:
{
"type": "function",
"function": {
"name": "calculator",
"description": "Exactly evaluate one arithmetic expression. When this tool is available, it must be called at least once. It does not interpret or solve the word problem.",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "Arithmetic using numeric literals, parentheses, and + - * / // % ** only."
}
},
"required": ["expression"],
"additionalProperties": false
}
}
}
正式运行命令(合成 pilot 通过后执行):
uv run python scripts/run_evaluation.py \
--base-url http://127.0.0.1:50269/v1 \
--model spark-x2.5-4b \
--manifest data/selection_manifest.jsonl \
--excerpt third_party/gsm_symbolic_selected.jsonl \
--runtime-json evidence/environment.json \
--protocol-commit cdfe83a112308380c6a4bae5adb9de14f38c7bc2 \
--output-dir runs/live --log runs/run.log --workers 4
运行栈为 Python 3.12.7、OpenAI SDK 2.21.0、vLLM 0.23.0 镜像vllm/vllm-openai:v0.23.0(repo digestsha256:6d8429e38e3747723ca07ee1b17972e09bb9c51c4032b266f24fb1cc3b22ed8f),
Spark plugin commit ddef50f6f1eda417b6716b27e0aa35d4a6cd0cf3(wheel SHA256a7728f6482fffd734ddbedd230a6e066be6263132c2b59c648f39bb28977c09b)。
脱敏日志片段:
Loading safetensors checkpoint shards: 100% Completed | 5/5
INFO: Application startup complete.
2026-09-13T08:14:05.220653+00:00 start jobs=120 workers=4
2026-09-13T08:16:56.733824+00:00 finish counts={"completed": 120}
正式生成前先用不重叠的合成题校准 vLLM spark25 工具接口;所有失败 pilot 都
保留。正式生成开始后没有改样本、prompt、工具、参数、评分器或分母。
推理完整性:不是只看末答
我按冻结规则审阅了 75/120 条轨迹,覆盖全部 40 条错误/不可解析、全部 40 条真实
工具调用、16 组 A/B 分歧的两臂,以及每个 config 的预选对照。主标签为:coherent
32、tool protocol 18、truncation 9、semantic/planning 7、relevant-clause
omission 4、dataset ambiguity 4、arithmetic 1。
代表案例包括:
p1-id44-i39:直接策略把“男孩为女孩的 1/4”方向写反;计算器策略正确建模并
通过两次原生调用得到 80,属于真实调用伴随的修复;main-id10-i44:两次调用分别正确得到 11 和 3,但模型还想发第三次调用,最终
没有提交 14,是典型的答案交付失败;p1-id24-i08:模型把运动消耗写成错误符号,计算器只是精确执行了错误方程,
说明 executor success 不等于 semantic success;p2-id26-i27:末答 1680 正确,但可见推导仍把连续减少 20% 和 50% 错称为
总计减少 70%,是 1 条严格 RWRA;p2-id33-i27:两臂按“每周减少 5 个”的字面含义得到 14,而金标 10 依赖
“降到每周 5 个”的读法,作为 dataset ambiguity 保留而不偷偷改分。
下面内嵌三条完整轨迹:模型内容未删改,仅规范化行尾排版空格,结构化字段仅做
排版;字节级记录可用 raw_record_sha256 对照仓库中的 runs/raw.jsonl。
1. 工具调用成功并交付正确答案:p1-id44-i39 / calculator
- 严格金标:
80 - 严格预测:
80,计为正确 - episode seed:
20318171;后续轮次依次加 1 raw_record_sha256:3e92c08c1a76c981e0303831fed5b126e596a9cfdd4d3ad938c884a0399a72cc
系统提示:
Solve the supplied math problem carefully. Follow the user policy. After any tool result, give a concise, checkable derivation. End with exactly one line: FINAL_ANSWER: <number>, where <number> has no unit. Do not write anything after that line.
用户消息:
请先理解下面的英文数学题并确定所需算式,然后必须调用 calculator 工具精确计算。不要在工具返回前给最终答案。工具返回后再简要说明推导,并以 FINAL_ANSWER: <number> 结尾。
题目(原文):
There are 8 girls in the garden. If there are 1/4 the number of boys in the garden, what percentage of the kids in the garden are girls?
第 1 轮 assistant(finish_reason=tool_calls):
Let me solve this step by step.
**Understanding the problem:**
- There are 8 girls in the garden.
- The number of boys is 1/4 the number of girls.
- We need to find what percentage of the total kids are girls.
**Step 1: Find the number of boys**
Boys = 1/4 × Girls = 1/4 × 8 = 2
**Step 2: Find the total number of kids**
Total = Girls + Boys = 8 + 2 = 10
**Step 3: Find the percentage of girls**
Percentage = (Girls / Total) × 100 = (8 / 10) × 100 = 80%
Let me verify with the calculator.
第 1 次原生函数调用与执行器返回:
{
"call": {
"function": {
"arguments": "{\"expression\":\"(1/4) * 8\"}",
"name": "calculator"
},
"id": "chatcmpl-tool-ac2ccbf6cd37f5eb",
"type": "function"
},
"executor_error": null,
"executor_result": {"decimal": "2", "exact": "2", "ok": true}
}
第 2 轮 assistant(finish_reason=tool_calls):原始 content 为空字符串。
第 2 次原生函数调用与执行器返回:
{
"call": {
"function": {
"arguments": "{\"expression\":\"(8 / (8 + (1/4) * 8)) * 100\"}",
"name": "calculator"
},
"id": "chatcmpl-tool-9ef0f5a7798f9a5a",
"type": "function"
},
"executor_error": null,
"executor_result": {"decimal": "80", "exact": "80", "ok": true}
}
第 3 轮 assistant(finish_reason=stop,tools_available=false):
**Derivation:**
- Number of boys = 1/4 × 8 = 2
- Total kids = 8 + 2 = 10
- Percentage of girls = (8 / 10) × 100 = 80%
FINAL_ANSWER: 80
这条轨迹在计算器联合条件中完成了题意建模、两次原生调用和末答交付;单条案例本身
不用于声称纯工具因果。
2. 两次调用均成功,但协议交付失败:main-id10-i44 / calculator
- 严格金标:
14 - 严格预测:不可解析,计为错误
- 解析错误:
missing_or_multiple_final_answer_markers - episode seed:
20610456;后续轮次依次加 1 raw_record_sha256:89d135e0b32ab1b5b5df980481898297eeb875fc1660535e63643cea61f68a4d
系统提示:
Solve the supplied math problem carefully. Follow the user policy. After any tool result, give a concise, checkable derivation. End with exactly one line: FINAL_ANSWER: <number>, where <number> has no unit. Do not write anything after that line.
用户消息:
请先理解下面的英文数学题并确定所需算式,然后必须调用 calculator 工具精确计算。不要在工具返回前给最终答案。工具返回后再简要说明推导,并以 FINAL_ANSWER: <number> 结尾。
题目(原文):
To make a call from a phone booth, you must pay €0.5 for each minute of your call. After 22 minutes, that price drops to €0.1 per minute. How much would a 52-minute call cost?
第 1 轮 assistant(finish_reason=tool_calls):
Let me break down this problem step by step.
The pricing structure is:
- First 22 minutes: €0.5 per minute
- After 22 minutes (remaining minutes): €0.1 per minute
For a 52-minute call:
- First 22 minutes cost: 22 × €0.5
- Remaining minutes: 52 - 22 = 30 minutes, costing: 30 × €0.1
Let me calculate each part.
第 1 次原生函数调用与执行器返回:
{
"call": {
"function": {"arguments": "{\"expression\":\"22 * 0.5\"}", "name": "calculator"},
"id": "chatcmpl-tool-963101f8df8888d8",
"type": "function"
},
"executor_error": null,
"executor_result": {"decimal": "11", "exact": "11", "ok": true}
}
第 2 轮 assistant(finish_reason=tool_calls):原始 content 为空字符串。
第 2 次原生函数调用与执行器返回:
{
"call": {
"function": {
"arguments": "{\"expression\":\"(52 - 22) * 0.1\"}",
"name": "calculator"
},
"id": "chatcmpl-tool-baab61188a06015a",
"type": "function"
},
"executor_error": null,
"executor_result": {"decimal": "3", "exact": "3", "ok": true}
}
第 3 轮 assistant(finish_reason=stop,两次额度已用完,tools_available=false):
<tool_call>calculator<arg_key>expression</arg_key><arg_value>11 + 3</arg_value></tool_call>
两次执行结果 11 和 3 都正确,但最后一轮没有提交要求的 FINAL_ANSWER: 14,
而是生成了不可执行的文字化第三次调用。因此该轨迹保留在分母并计错,主标签为tool_protocol。
3. 末答正确但推理保留矛盾:p2-id26-i27 / no_tool
- 严格金标:
1680 - 严格预测:
1680,末答计为正确 - 推理审阅:
right_answer_wrong_reasoning - episode seed:
20789663 raw_record_sha256:927c4bd2d8ac5f6210b81a66b772906a50e41e220fae3f0ee56520c34ffb27ca
系统提示:
Solve the supplied math problem carefully. Follow the user policy. After any tool result, give a concise, checkable derivation. End with exactly one line: FINAL_ANSWER: <number>, where <number> has no unit. Do not write anything after that line.
用户消息:
请独立理解下面的英文数学题并手工计算,不要使用任何外部工具。给出简洁、可检查的推导,并以 FINAL_ANSWER: <number> 结尾。
题目(原文):
A grandfather eats 5 hamburgers per day, his partner eats 15 hamburgers per day, and their son eats three-fourths of the combined number of hamburgers they eat per day. After one month, the family decides to go on a diet in the second month, reducing their food consumption by 20% for the first half of the month, and then by an additional 50% for the second half. How many hamburgers does this family eat in two months?
第 1 轮 assistant(finish_reason=stop):
Step 1: Compute daily hamburger consumption.
Grandfather: 5
Partner: 15
Combined: 5 + 15 = 20
Step 2: Son eats three-fourths of the combined amount.
3/4 × 20 = 15
Step 3: Total daily consumption in first month (normal):
5 + 15 + 15 = 35 hamburgers per day
Step 4: First month total (assume 30 days):
35 × 30 = 1050
Step 5: Second month diet plan:
- First half (15 days): reduce by 20% → multiply by (1 – 0.20) = 0.80
Daily consumption = 35 × 0.80 = 28
Total for first half = 28 × 15 = 420
- Second half (15 days): reduce by an additional 50% from the already reduced level
So total reduction = 20% + 50% = 70% reduction → multiply by (1 – 0.70) = 0.30
Alternatively: 28 × (1 – 0.50) = 28 × 0.50 = 14
Daily consumption = 14
Total for second half = 14 × 15 = 210
Step 6: Second month total = 420 + 210 = 630
Step 7: Total for two months = First month + Second month = 1050 + 630 = 1680
FINAL_ANSWER: 1680
末答采用正确的连续折减路径 35×0.8×0.5=14,因此得到 1680;但同一段推导又
声称“20% + 50% = 总计减少 70%”,对应保留比例 0.3。两种说法相互矛盾,所以
这是严格的“答案正确、推理错误”,而不是普通的已纠正 false start。
全部 75 条审阅与具体证据:
https://github.com/MittaPei/spark-x2-5-gsm-symbolic-tool-study/blob/f91d795c1b054b71142bdc32a6af22c8a3e31497/reasoning/review.jsonl
与现有 Issue / Discussion 投稿相比的互补优势
截至提交时,公开案例已经覆盖标准数学集、thinking 开关、多语言与数值扰动、答案
交付等方向,Issue 中也出现了受限计算器对照。本提交的互补价值在于把
Spark-X2.5-4B 原始 BF16、GSM-Symbolic main/p1/p2 三复杂度模板配平、完全
同题的直接策略/原生函数调用策略 A/B,以及工具采纳—执行器结果—语义建模—调用
预算—末答交付的分层分析放进同一个运行前冻结、可离线复算的证据闭环。120 条
原始轨迹逐轮保留函数参数与返回,并结合逐 config 的 McNemar/Holm 校正和 75 条
推理审阅;差异化价值不在宣称更高单一分数,而在于把 Agent 工具链的实际失效位置
转化为可验证、可改进的部署证据。
完整证据与复现
关键入口:
- 完整报告与命令:
README.md - 冻结协议与完整 prompt:
PROTOCOL.md、src/spark_math_eval/runner.py - 120 条原始响应和工具转录:
runs/raw.jsonl - 逐题评分与汇总:
results/per_item.jsonl、results/summary.json - 环境、权重 LFS SHA256、数据源 SHA256、镜像 digest:
evidence/environment.json - 正式运行和脱敏 vLLM 日志:
runs/run.log、evidence/runtime/ - 失败及通过的合成 pilot:
evidence/pilot* - 证据完整性:
SHA256SUMS - CI:https://github.com/MittaPei/spark-x2-5-gsm-symbolic-tool-study/actions/runs/34748779510
离线复算不需要 GPU:
uv sync --all-extras --locked
uv run python scripts/summarize.py
uv run python scripts/audit_reasoning.py
uv run python scripts/render_cases.py
uv run python scripts/verify_artifacts.py
uv run ruff check .
uv run ruff format --check .
uv run pytest -q
verify_artifacts.py 会重新评分、重放 63 次白名单算术调用、重建推理审阅、核验
SHA256,并阻止 secret、个人路径和模型权重进入公开材料。
运行环境为共享 A800-SXM4-80GB、vLLM 0.23.0、TP=1、4 workers;正式窗口
171.509 秒。直接/计算器策略的总 API token 分别为 52,444 / 107,596。本次复用
现有服务器和本地权重,增量费用为 0,整机历史成本未知。固定 seed 和 greedy
decoding 用于减少方差,但不承诺跨硬件 bit-identical generation。20 个模板只是
审计样本,数值替换也不能完全排除模板污染。
原创代码为 Apache-2.0;60 条必要且未修改的数据集摘录仍按 CC BY-NC-ND 4.0,
不受代码许可证覆盖。没有上传任何模型权重,也没有公开 token、个人 cache 路径、
内网地址或私有数据。
个人参赛。GitHub @MittaPei 与发布本 Discussion 的 Hugging Face@Forrest20231206 是同一位参赛者。Codex 多代理辅助了实验设计、代码、真实执行、
逐条审阅和文稿;参赛者已审阅并理解结果、局限与材料,并对本提交负责。没有把 AI
角色列为真人参与者。