Instructions to use thunquant/qwen3.5-4B-super-coder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use thunquant/qwen3.5-4B-super-coder with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf thunquant/qwen3.5-4B-super-coder:BF16
# Run inference directly in the terminal:
llama cli -hf thunquant/qwen3.5-4B-super-coder:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf thunquant/qwen3.5-4B-super-coder:BF16
# Run inference directly in the terminal:
llama cli -hf thunquant/qwen3.5-4B-super-coder:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf thunquant/qwen3.5-4B-super-coder:BF16
# Run inference directly in the terminal:
./llama-cli -hf thunquant/qwen3.5-4B-super-coder:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf thunquant/qwen3.5-4B-super-coder:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf thunquant/qwen3.5-4B-super-coder:BF16

Use Docker

docker model run hf.co/thunquant/qwen3.5-4B-super-coder:BF16

LM Studio
Jan

vLLM

How to use thunquant/qwen3.5-4B-super-coder with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "thunquant/qwen3.5-4B-super-coder"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "thunquant/qwen3.5-4B-super-coder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/thunquant/qwen3.5-4B-super-coder:BF16

Ollama
How to use thunquant/qwen3.5-4B-super-coder with Ollama:
```
ollama run hf.co/thunquant/qwen3.5-4B-super-coder:BF16
```

Unsloth Studio

How to use thunquant/qwen3.5-4B-super-coder with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for thunquant/qwen3.5-4B-super-coder to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for thunquant/qwen3.5-4B-super-coder to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for thunquant/qwen3.5-4B-super-coder to start chatting

How to use thunquant/qwen3.5-4B-super-coder with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf thunquant/qwen3.5-4B-super-coder:BF16

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "thunquant/qwen3.5-4B-super-coder:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use thunquant/qwen3.5-4B-super-coder with Docker Model Runner:
```
docker model run hf.co/thunquant/qwen3.5-4B-super-coder:BF16
```

Lemonade

How to use thunquant/qwen3.5-4B-super-coder with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull thunquant/qwen3.5-4B-super-coder:BF16

Run and chat with the model

lemonade run user.qwen3.5-4B-super-coder-BF16

List all available models

lemonade list

Hermes Agent

How to use thunquant/qwen3.5-4B-super-coder with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf thunquant/qwen3.5-4B-super-coder:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default thunquant/qwen3.5-4B-super-coder:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use thunquant/qwen3.5-4B-super-coder with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf thunquant/qwen3.5-4B-super-coder:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "thunquant/qwen3.5-4B-super-coder:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

thunquant

jica98 commited on 6 days ago

Commit

cc99df6

0 Parent(s):

Duplicate from jica98/qwen3.5-4B-super-coder

Browse files

Co-authored-by: Jishnu Raj <jica98@users.noreply.huggingface.co>

Files changed (14) hide show

.gitattributes +38 -0
README.md +126 -0
adapter_config.json +52 -0
adapter_model.safetensors +3 -0
benchmark/lmstudio_q4_benchmark/benchmark_report.md +101 -0
benchmark/lmstudio_q4_benchmark/benchmark_results.json +1201 -0
chat_template.jinja +158 -0
config.json +111 -0
processor_config.json +63 -0
qwen3.5-4B-super-coder.BF16-mmproj.gguf +3 -0
qwen3.5-4B-super-coder.Q4_0.gguf +3 -0
tokenizer.json +3 -0
tokenizer_config.json +299 -0
training_args.bin +3 -0

.gitattributes ADDED Viewed

	@@ -0,0 +1,38 @@

+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+qwen3.5-4B-super-coder.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3.5-4B-super-coder.BF16-mmproj.gguf filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,126 @@

+---
+base_model: Qwen/Qwen3.5-4B
+tags:
+- coding
+- tool-calling
+- gguf
+- unsloth
+- llama-cpp
+- thinking-mode
+- Gated-DeltaNet
+- mobile-fit
+licence: apache-2.0
+pipeline_tag: text-generation
+---
+# qwen3.5-4B-super-coder (Q_4.0 GGUF)
+`qwen3.5-4B-super-coder` is a 4-bit quantized GGUF model optimized for fast, reliable coding, structured tool calling, and active reasoning (thinking mode) on consumer/mobile hardware. It is distilled from Claude Sonnet 4.6 & Opus 4.6, and merged/quantized using Unsloth.
+## Model Summary & Architecture
+- **Base Model**: `Qwen/Qwen3.5-4B`
+- **Format**: GGUF (Q_4.0 Quantization)
+- **Size**: ~2.6 GB
+- **Context Window**: 32K (optimized for mobile RAM budgets, natively supports up to 262K/1M context via YaRN)
+- **Key Architectural Advantage**: The base `Qwen3.5-4B` model uses a hybrid architecture combining **Gated DeltaNet** (3 layers) and **Full Attention** (1 layer) repeating. Since only 8 of the 32 layers store a full KV cache, the KV cache footprint is incredibly small (~0.4GB for 32K context), making it exceptionally well-suited for high-context coding on mobile devices (e.g., iPhone 15 Pro+, flagship Android, iPad Pro).
+## Distillation & Training Procedure
+This model was trained using a staged Supervised Fine-Tuning (SFT) pipeline to systematically inject reasoning capability, coding specialization, and tool-calling precision:
+```
+                  ┌──────────────────────────────────────────┐
+                  │                 Phase A:                 │
+                  │   General Distillation (Claude Style)   │
+                  │   Dataset: Claude-Distills (140K)        │
+                  └────────────────────┬─────────────────────┘
+                                       │
+                                       ▼
+                  ┌──────────────────────────────────────────┐
+                  │                 Phase B:                 │
+                  │   Specialization (Coding & Tool Calling) │
+                  │   Dataset: Curated Replay Mix (77K)      │
+                  └────────────────────┬─────────────────────┘
+                                       │
+                                       ▼
+                  ┌──────────────────────────────────────────┐
+                  │                 Phase C:                 │
+                  │   Tool Precision & Schema Conformance    │
+                  │   Dataset: Tool-focused Mix (~20K)       │
+                  └──────────────────────────────────────────┘
+```
+1. **Phase 1: Distillation (Claude Behavior)**
+   - **Dataset**: `clzoro/Claude-Distills` (140K samples; Sonnet 4.6 + Opus 4.6).
+   - **Objective**: Transfer general instruction-following, Claude-like formatting/tone, and reasoning capabilities. The Opus subset (21K samples) provided the crucial `<think>` block traces to establish thinking capabilities.
+2. **Phase 2: Specialization (Coding & Tools)**
+   - **Dataset**: Curated 77K sample mix (55K coding instructions, 13K tool calling, and 9K general anti-forgetting replay samples).
+   - **Objective**: Specialize the model on coding accuracy across Python, JS, Shell, etc., and introduce structured tool-calling.
+3. **Phase 3: Tool Precision**
+   - **Dataset**: Focused tool-calling dataset (~20K samples) with schema variations, neg/no-tool examples, and strict JSON format targets.
+   - **Objective**: Ensure precise JSON schema conformance and reduce tool false-positives.
+4. **Phase 4: Coding/Tool Specialization Continuation**
+   - **Starting point**: `jica98/qwen3.5-4b-claude-distill-lora` Phase 3 LoRA.
+   - **Output adapter**: `qwen3.5-4b-phase4-specialize-lora`.
+   - **Training mix**: local filtered coding/tool data from `filtered_dataset/train.jsonl`, Claude distillation replay from `data/claude_distill.jsonl`, and an Opus replay slice to retain visible reasoning behavior.
+   - **Objective**: Continue the distilled LoRA into a stronger coding/tool-specialized adapter while preserving anti-forgetting replay.
+   - **Default recipe**: 1024 max sequence length, batch size 1, gradient accumulation 8, learning rate `1e-4`, 1 epoch, checkpointing every 200 steps.
+## Phase 5 Fable Reasoning Fine-Tune
+The latest adapter was further fine-tuned for Fable reasoning and agentic coding traces after the Phase 4 specialization pass.
+Phase 5 training data:
+- `kelexine/fable-5-sft-traces` for cleaned Fable reasoning/SFT traces.
+- `armand0e/claude-fable-5-claude-code` for raw Claude/Fable-5 agent traces.
+- `victor/fable-5-boeing-747-trace` for the Boeing 747 Claude Code/Fable-5 trace.
+Training summary:
+- Starting point: `qwen3.5-4b-phase4-specialize-lora`.
+- Output adapter: `qwen3.5-4b-phase5-fable-lora`.
+- After dedupe/sample in the recorded run: 4,721 examples.
+- After max-length filtering at 4096 tokens: 4,267 examples.
+- Default recipe: batch size 1, gradient accumulation 8, learning rate `5e-5`, 1 epoch, BF16, `adamw_8bit`.
+The Phase 5 data loader normalizes traces into Qwen chat-template text, groups raw Claude event logs into session conversations, deduplicates samples, filters by token length, and skips checkpoint artifacts during Hub upload by default.
+## Strengths & What It Is Good At
+- 💻 **Conversational Programming**: Excel at writing clean, efficient, and well-commented code in Python, C++, Rust, JavaScript, Shell, and more.
+- 🧠 **Visible Reasoning (Thinking Mode)**: When faced with complex reasoning or coding tasks, the model engages a `<think>...</think>` block to outline its plan before writing code.
+- 🛠️ **Reliable Tool Calling**: Specially tuned to parse and output valid JSON tool parameters conforming to provided function schemas.
+- 📱 **Mobile & Edge Execution**: With a weight footprint of ~2.6GB and extremely low KV cache overhead, it fits comfortably on 8GB+ RAM edge devices.
+## Recommended Inference Settings
+For the best balance of reasoning depth and formatting precision, use the following generation parameters:
+- **Temperature**: `0.6`
+- **Top-P**: `0.95`
+- **Top-K**: `20`
+- **Min-P**: `0.0`
+- **Flash Attention**: Enable `-fa` in llama.cpp/llama-cli for optimal speeds.
+- **System Prompt**: Set system prompt to guide the assistant (e.g. `You are a helpful coding assistant.`).
+## Benchmark Results (Q4_0 GGUF via LM Studio)
+Benchmark run against GGUF Q4_0 quant served through LM Studio on consumer AMD ROCm hardware. Results file: [`benchmark/lmstudio_q4_benchmark/benchmark_report.md`](https://huggingface.co/jica98/qwen3.5-4B-super-coder/blob/main/benchmark/lmstudio_q4_benchmark/benchmark_report.md)
+| Benchmark | Score | Status |
+|---|---|---|
+| HumanEval+ Pass@1 | 0.00 | ok |
+| MBPP+ Pass@1 | 0.00 | ok |
+| BigCodeBench-Hard | — | needs_review |
+| LiveCodeBench v6 | — | not_run |
+| BFCL v4 | — | needs_review |
+| IFEval | — | needs_review |
+| MMLU-Pro | — | needs_review |
+| JSON validity | 40.00% | ok |
+| No-tool accuracy | 87.50% | ok |
+**Notes:**
+- Several benchmarks require environment setup that wasn't completed (IFEval, MMLU-Pro, BFCL, BigCodeBench-Hard).
+- HumanEval+ and MBPP+ scored 0.00 — the Q4_0 quant may degrade code generation significantly; evaluation with the BF16 base is needed for comparison.
+- JSON validity and No-tool accuracy are custom deterministic diagnostics.

adapter_config.json ADDED Viewed

	@@ -0,0 +1,52 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Qwen3_5ForConditionalGeneration",
+    "parent_library": "transformers.models.qwen3_5.modeling_qwen3_5",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/Qwen3.5-4B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "gate_proj",
+    "q_proj",
+    "v_proj",
+    "up_proj",
+    "k_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:85fe4607a6ed26349f8bde53023b41804d4788dc0fb0f15a3febc907d4dc7a2a
+size 84972248

benchmark/lmstudio_q4_benchmark/benchmark_report.md ADDED Viewed

	@@ -0,0 +1,101 @@

+# Phase-5 Q4_0 LM Studio Benchmark Report
+- Generated: `2026-06-27T15:00:25+05:30`
+- Endpoint: `http://127.0.0.1:1234/v1`
+- Served model identifier: `qwen3.5-4b-super-coder`
+- Decoding: greedy / temperature 0 unless an official framework overrides it
+| Benchmark | Official base | Parent base | Phase-5 BF16 | Phase-5 Q4_0 | Delta vs parent | Status |
+|---|---:|---:|---:|---:|---:|---|
+| HumanEval+ Pass@1 | — | — | — | 0.00 | — | ok |
+| MBPP+ Pass@1 | — | — | — | 0.00 | — | ok |
+| BigCodeBench-Hard | — | — | — | — | — | needs_review |
+| LiveCodeBench v6 | — | — | — | — | — | not_run |
+| BFCL v4 | — | — | — | — | — | needs_review |
+| IFEval | — | — | — | — | — | needs_review |
+| MMLU-Pro | — | — | — | — | — | needs_review |
+| JSON validity | — | — | — | 40.00 | — | ok |
+| No-tool accuracy | — | — | — | 87.50 | — | ok |
+## Metric provenance
+HumanEval+, MBPP+, BigCodeBench-Hard, BFCL, IFEval and MMLU-Pro are run through their official frameworks. LiveCodeBench uses its official release-v6 dataset and evaluator, with a generic OpenAI-chat generation prompt because the custom LM Studio model is not registered in LiveCodeBench's model-style table. JSON validity and No-tool accuracy are custom deterministic diagnostics.
+### HumanEval+ Pass@1
+- Score: **0.00**
+- Metric: Pass@1
+- Source: Official EvalPlus
+- Status: `ok`
+- Details: `{"parsed_from": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/evalplus/humaneval/evalplus_results/humaneval/qwen3.5-4b-super-coder_openai_temp_0.0_eval_results.json", "json_key": "eval.HumanEval/22.0.plus_fail_tests.0.0.3", "raw_value": 0, "pattern_matches": 1, "command": {"name": "evalplus_humaneval", "command": ["/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/evalplus/bin/evalplus.evaluate", "--model", "qwen3.5-4b-super-coder", "--dataset", "humaneval", "--backend", "openai", "--base-url", "http://127.0.0.1:1234/v1", "--greedy"], "returncode": 0, "elapsed_seconds": 1646.0040039900014, "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/evalplus_humaneval.log", "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/evalplus/humaneval"}}`
+### MBPP+ Pass@1
+- Score: **0.00**
+- Metric: Pass@1
+- Source: Official EvalPlus
+- Status: `ok`
+- Details: `{"parsed_from": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/evalplus/mbpp/evalplus_results/mbpp/qwen3.5-4b-super-coder_openai_temp_0.0_eval_results.json", "json_key": "eval.Mbpp/6.0.plus_fail_tests.0.0", "raw_value": 0, "pattern_matches": 1, "command": {"name": "evalplus_mbpp", "command": ["/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/evalplus/bin/evalplus.evaluate", "--model", "qwen3.5-4b-super-coder", "--dataset", "mbpp", "--backend", "openai", "--base-url", "http://127.0.0.1:1234/v1", "--greedy"], "returncode": 0, "elapsed_seconds": 3226.43256972, "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/evalplus_mbpp.log", "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/evalplus/mbpp"}}`
+### BigCodeBench-Hard
+- Score: **—**
+- Metric: Calibrated Pass@1
+- Source: Official BigCodeBench
+- Status: `needs_review`
+- Details: `{"command": {"name": "bigcodebench_hard", "command": ["/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/bigcodebench/bin/bigcodebench.evaluate", "--model", "qwen3.5-4b-super-coder", "--execution", "gradio", "--split", "instruct", "--subset", "hard", "--backend", "openai", "--base_url", "http://127.0.0.1:1234/v1", "--bs", "1", "--n_samples", "1", "--temperature", "0.0", "--greedy", "--pass_k", "1", "--root", "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bigcodebench/bcb_results"], "returncode": 1, "elapsed_seconds": 1760.3364485260026, "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/bigcodebench_hard.log", "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bigcodebench"}, "execution_backend": "gradio"}`
+### BFCL v4
+- Score: **—**
+- Metric: Overall accuracy for selected categories
+- Source: Official BFCL
+- Status: `needs_review`
+- Details: `{"categories": "all_scoring", "generation_command": {"name": "bfcl_v4_generate", "command": ["/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/bfcl/bin/bfcl", "generate", "--model", "Qwen/Qwen3-4B-Instruct-2507-FC", "--test-category", "all_scoring", "--skip-server-setup", "--num-threads", "1", "--result-dir", "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl/project/result"], "returncode": 1, "elapsed_seconds": 2.7761374679976143, "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/bfcl_v4_generate.log", "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl"}, "evaluation_command": {"name": "bfcl_v4_evaluate", "command": ["/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/bfcl/bin/bfcl", "evaluate", "--model", "Qwen/Qwen3-4B-Instruct-2507-FC", "--test-category", "all_scoring", "--result-dir", "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl/project/result", "--score-dir", "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl/project/score"], "returncode": 1, "elapsed_seconds": 2.4078060280007776, "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/bfcl_v4_evaluate.log", "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl"}, "served_alias": "Qwen/Qwen3-4B-Instruct-2507-FC"}`
+### BFCL no-tool/relevance
+- Score: **—**
+- Metric: BFCL irrelevance/relevance detection
+- Source: Official BFCL
+- Status: `not_available`
+- Details: `{}`
+### IFEval
+- Score: **—**
+- Metric: Prompt-level strict accuracy
+- Source: LM Evaluation Harness
+- Status: `needs_review`
+- Details: `{"command": {"name": "lm_eval_ifeval", "command": ["/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/lm_eval/bin/lm_eval", "--model", "local-chat-completions", "--model_args", "model=qwen3.5-4b-super-coder,base_url=http://127.0.0.1:1234/v1/chat/completions,num_concurrent=1,max_retries=4,tokenized_requests=False,batch_size=1", "--tasks", "ifeval", "--batch_size", "1", "--log_samples", "--output_path", "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/ifeval/results"], "returncode": 1, "elapsed_seconds": 5.503946738001105, "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/lm_eval_ifeval.log", "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/ifeval"}}`
+### MMLU-Pro
+- Score: **—**
+- Metric: Accuracy
+- Source: LM Evaluation Harness
+- Status: `needs_review`
+- Details: `{"command": {"name": "lm_eval_mmlu_pro", "command": ["/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/lm_eval/bin/lm_eval", "--model", "local-chat-completions", "--model_args", "model=qwen3.5-4b-super-coder,base_url=http://127.0.0.1:1234/v1/chat/completions,num_concurrent=1,max_retries=4,tokenized_requests=False,batch_size=1", "--tasks", "mmlu_pro", "--batch_size", "1", "--log_samples", "--output_path", "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/mmlu_pro/results"], "returncode": 1, "elapsed_seconds": 4.813302385999123, "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/lm_eval_mmlu_pro.log", "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/mmlu_pro"}}`
+### JSON validity
+- Score: **40.00**
+- Metric: Schema-valid JSON rate
+- Source: Custom deterministic endpoint diagnostic
+- Status: `ok`
+- Details: `{"total": 30, "json_parse_rate_percent": 96.66666666666667, "schema_valid_rate_percent": 40.0, "responses": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/custom/json_validity/responses.json"}`
+### No-tool accuracy
+- Score: **87.50**
+- Metric: Correctly abstained from tool use
+- Source: Custom deterministic native-tool diagnostic
+- Status: `ok`
+- Details: `{"correct": 35, "total": 40, "responses": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/custom/no_tool/responses.json"}`
+## Important interpretation notes
+- Only the Q4_0 model served by LM Studio was evaluated. The other comparison columns intentionally remain blank.
+- `Delta vs parent` cannot be calculated without a parent-model score.
+- BFCL's score is for the categories passed through `--bfcl-categories`; check the details before comparing it to the public full-suite leaderboard.
+- Generated code is untrusted. Use an isolated machine or container.

benchmark/lmstudio_q4_benchmark/benchmark_results.json ADDED Viewed

	@@ -0,0 +1,1201 @@

+{
+  "metadata": {
+    "generated_at": "2026-06-27T15:00:25+05:30",
+    "selected_benchmarks": [
+      "bfcl",
+      "bigcodebench",
+      "humaneval",
+      "ifeval",
+      "json_validity",
+      "livecodebench",
+      "mbpp",
+      "mmlu_pro",
+      "no_tool"
+    ],
+    "errors": [
+      {
+        "benchmark": "LiveCodeBench",
+        "error": "CalledProcessError: Command '['/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/livecodebench/bin/python', '-c', \"\\nimport json\\nfrom datasets import load_dataset\\nds = load_dataset(\\n    'livecodebench/code_generation_lite',\\n    split='test',\\n    version_tag='release_v6',\\n    trust_remote_code=True,\\n)\\nwith open('/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/livecodebench/livecodebench_dataset.json', 'w', encoding='utf-8') as f:\\n    json.dump([dict(x) for x in ds], f, ensure_ascii=False)\\nprint(len(ds))\\n\"]' returned non-zero exit status 1.",
+        "traceback": "Traceback (most recent call last):\n  File \"/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_full_benchmark.py\", line 1526, in guarded\n    value = fn()\n            ^^^^\n  File \"/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_full_benchmark.py\", line 1594, in <lambda>\n    lambda: run_livecodebench(\n            ^^^^^^^^^^^^^^^^^^\n  File \"/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_full_benchmark.py\", line 730, in run_livecodebench\n    generation_details = generate_livecodebench_outputs(\n                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_full_benchmark.py\", line 662, in generate_livecodebench_outputs\n    subprocess.run([str(lcb_python), \"-c\", loader_code], check=True)\n  File \"/usr/lib/python3.11/subprocess.py\", line 571, in run\n    raise CalledProcessError(retcode, process.args,\nsubprocess.CalledProcessError: Command '['/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/livecodebench/bin/python', '-c', \"\\nimport json\\nfrom datasets import load_dataset\\nds = load_dataset(\\n    'livecodebench/code_generation_lite',\\n    split='test',\\n    version_tag='release_v6',\\n    trust_remote_code=True,\\n)\\nwith open('/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/livecodebench/livecodebench_dataset.json', 'w', encoding='utf-8') as f:\\n    json.dump([dict(x) for x in ds], f, ensure_ascii=False)\\nprint(len(ds))\\n\"]' returned non-zero exit status 1.\n"
+      }
+    ],
+    "environment_versions": {
+      "script_version": "1.0.1",
+      "python": "3.11.14 (main, Nov 20 2025, 15:58:12) [GCC 15.2.1 20251112]",
+      "platform": "Linux-7.1.1-2-cachyos-x86_64-with-glibc2.43",
+      "bfcl": [
+        "aiohappyeyeballs==2.6.2",
+        "aiohttp==3.14.1",
+        "aiosignal==1.4.0",
+        "annotated-doc==0.0.4",
+        "annotated-types==0.7.0",
+        "anthropic==0.112.0",
+        "anyio==4.14.1",
+        "argcomplete==3.6.3",
+        "attrs==26.1.0",
+        "beautifulsoup4==4.15.0",
+        "bfcl_eval==2026.3.23",
+        "black==26.5.1",
+        "boto3==1.43.36",
+        "botocore==1.43.36",
+        "certifi==2026.6.17",
+        "cffi==2.0.0",
+        "charset-normalizer==3.4.7",
+        "click==8.4.2",
+        "cohere==5.18.0",
+        "cryptography==49.0.0",
+        "cuda-bindings==13.3.1",
+        "cuda-pathfinder==1.5.5",
+        "cuda-toolkit==13.0.2",
+        "dashscope==1.26.0",
+        "datamodel-code-generator==0.25.7",
+        "distro==1.9.0",
+        "dnspython==2.8.0",
+        "docstring_parser==0.18.0",
+        "dotenv==0.9.9",
+        "email-validator==2.3.0",
+        "eval_type_backport==0.4.0",
+        "faiss-cpu==1.11.0",
+        "fastavro==1.12.2",
+        "filelock==3.20.0",
+        "frozenlist==1.8.0",
+        "fsspec==2026.6.0",
+        "genson==1.3.0",
+        "google-auth==2.55.1",
+        "google-genai==2.10.0",
+        "google_search_results==2.4.2",
+        "h11==0.16.0",
+        "hf-xet==1.5.1",
+        "html2text==2025.4.15",
+        "httpcore==1.0.9",
+        "httpx==0.28.1",
+        "httpx-sse==0.4.0",
+        "huggingface_hub==1.21.0",
+        "idna==3.18",
+        "inflect==5.6.2",
+        "isort==5.13.2",
+        "Jinja2==3.1.6",
+        "jiter==0.15.0",
+        "jmespath==1.1.0",
+        "joblib==1.5.3",
+        "json5==0.15.0",
+        "jsonlines==4.0.0",
+        "jsonschema==4.26.0",
+        "jsonschema-specifications==2025.9.1",
+        "markdown-it-py==4.2.0",
+        "MarkupSafe==3.0.3",
+        "mdurl==0.1.2",
+        "mistralai==1.7.0",
+        "mpmath==1.3.0",
+        "multidict==6.7.1",
+        "mypy_extensions==1.1.0",
+        "narwhals==2.22.1",
+        "networkx==3.3",
+        "numpy==1.26.4",
+        "nvidia-cublas==13.1.1.3",
+        "nvidia-cuda-cupti==13.0.85",
+        "nvidia-cuda-nvrtc==13.0.88",
+        "nvidia-cuda-runtime==13.0.96",
+        "nvidia-cudnn-cu13==9.20.0.48",
+        "nvidia-cufft==12.0.0.61",
+        "nvidia-cufile==1.15.1.6",
+        "nvidia-curand==10.4.0.35",
+        "nvidia-cusolver==12.0.4.66",
+        "nvidia-cusparse==12.6.3.3",
+        "nvidia-cusparselt-cu13==0.8.1",
+        "nvidia-nccl-cu13==2.29.7",
+        "nvidia-nvjitlink==13.0.88",
+        "nvidia-nvshmem-cu13==3.4.5",
+        "nvidia-nvtx==13.0.85",
+        "openai==2.44.0",
+        "overrides==7.7.0",
+        "packaging==26.2",
+        "pandas==3.0.3",
+        "pathspec==1.1.1",
+        "pillow==12.2.0",
+        "platformdirs==4.10.0",
+        "propcache==0.5.2",
+        "pyasn1==0.6.3",
+        "pyasn1_modules==0.4.2",
+        "pycparser==3.0",
+        "pydantic==2.13.4",
+        "pydantic_core==2.46.4",
+        "Pygments==2.20.0",
+        "python-dateutil==2.9.0.post0",
+        "python-dotenv==1.2.2",
+        "pytokens==0.4.1",
+        "PyYAML==6.0.3",
+        "qwen-agent==0.0.34",
+        "rank-bm25==0.2.2",
+        "referencing==0.37.0",
+        "regex==2026.5.9",
+        "requests==2.34.2",
+        "rich==15.0.0",
+        "rpds-py==2026.5.1",
+        "s3transfer==0.19.0",
+        "safetensors==0.8.0",
+        "scikit-learn==1.9.0",
+        "scipy==1.17.1",
+        "sentence-transformers==5.6.0",
+        "shellingham==1.5.4",
+        "six==1.17.0",
+        "sniffio==1.3.1",
+        "soupsieve==2.8.4",
+        "sympy==1.14.0",
+        "tabulate==0.10.0",
+        "tenacity==9.1.4",
+        "threadpoolctl==3.6.0",
+        "tiktoken==0.13.0",
+        "tokenizers==0.22.2",
+        "torch==2.12.1",
+        "tqdm==4.68.3",
+        "transformers==5.12.1",
+        "tree-sitter==0.21.3",
+        "tree-sitter-java==0.21.0",
+        "tree-sitter-javascript==0.21.4",
+        "triton==3.7.1",
+        "typer==0.25.1",
+        "types-requests==2.33.0.20260518",
+        "typing-inspection==0.4.2",
+        "typing_extensions==4.15.0",
+        "urllib3==2.7.0",
+        "websocket-client==1.9.0",
+        "websockets==16.0",
+        "writer-sdk==3.0.0",
+        "yarl==1.24.2"
+      ],
+      "bigcodebench": [
+        "accelerate==1.14.0",
+        "aiohappyeyeballs==2.6.2",
+        "aiohttp==3.14.1",
+        "aiosignal==1.4.0",
+        "annotated-doc==0.0.4",
+        "annotated-types==0.7.0",
+        "anthropic==0.112.0",
+        "anyio==4.14.1",
+        "apache-tvm-ffi==0.1.9",
+        "appdirs==1.4.4",
+        "astor==0.8.1",
+        "attrs==26.1.0",
+        "bigcodebench==0.2.5",
+        "blake3==1.0.9",
+        "bounded-pool-executor==0.0.3",
+        "bracex==2.6",
+        "cachetools==7.1.4",
+        "cbor2==6.1.2",
+        "certifi==2026.6.17",
+        "cffi==2.0.0",
+        "charset-normalizer==3.4.7",
+        "click==8.4.2",
+        "cloudpickle==3.1.2",
+        "compressed-tensors==0.17.0",
+        "cryptography==49.0.0",
+        "cuda-bindings==13.3.1",
+        "cuda-core==1.0.1",
+        "cuda-pathfinder==1.5.5",
+        "cuda-python==13.3.1",
+        "cuda-tile==1.3.0",
+        "cuda-toolkit==13.0.2",
+        "datasets==5.0.0",
+        "depyf==0.20.0",
+        "detect-installer==0.1.0",
+        "dill==0.4.1",
+        "diskcache==5.6.3",
+        "distro==1.9.0",
+        "dnspython==2.8.0",
+        "dockerfile-parse==2.0.1",
+        "docstring_parser==0.18.0",
+        "e2b==2.30.0",
+        "einops==0.8.2",
+        "email-validator==2.3.0",
+        "fastapi==0.136.3",
+        "fastapi-cli==0.0.27",
+        "fastapi-cloud-cli==0.21.0",
+        "fastar==0.11.0",
+        "fastsafetensors==0.3.2",
+        "filelock==3.29.4",
+        "fire==0.7.1",
+        "flashinfer-cubin==0.6.12",
+        "flashinfer-python==0.6.12",
+        "frozenlist==1.8.0",
+        "fsspec==2026.4.0",
+        "gguf==0.19.0",
+        "google-auth==2.55.1",
+        "google-genai==2.10.0",
+        "googleapis-common-protos==1.75.0",
+        "gradio_client==2.5.0",
+        "grpcio==1.81.1",
+        "h11==0.16.0",
+        "h2==4.3.0",
+        "hf-xet==1.5.1",
+        "hpack==4.2.0",
+        "httpcore==1.0.9",
+        "httptools==0.8.0",
+        "httpx==0.28.1",
+        "httpx-sse==0.4.3",
+        "huggingface_hub==1.21.0",
+        "humming-kernels==0.1.4",
+        "hyperframe==6.1.0",
+        "idna==3.18",
+        "ijson==3.5.0",
+        "interegular==0.3.3",
+        "Jinja2==3.1.6",
+        "jiter==0.15.0",
+        "jmespath==1.1.0",
+        "jsonschema==4.26.0",
+        "jsonschema-specifications==2025.9.1",
+        "lark==1.2.2",
+        "llguidance==1.7.6",
+        "llvmlite==0.47.0",
+        "lm-format-enforcer==0.11.3",
+        "loguru==0.7.3",
+        "markdown-it-py==4.2.0",
+        "MarkupSafe==3.0.3",
+        "mcp==1.28.1",
+        "mdurl==0.1.2",
+        "mistral_common==1.11.5",
+        "mistralai==0.4.2",
+        "ml_dtypes==0.5.4",
+        "model-hosting-container-standards==0.1.16",
+        "mpmath==1.3.0",
+        "msgspec==0.21.1",
+        "multidict==6.7.1",
+        "multipledispatch==1.0.0",
+        "multiprocess==0.70.19",
+        "networkx==3.6.1",
+        "ninja==1.13.0",
+        "numba==0.65.0",
+        "numpy==2.3.5",
+        "nvidia-cublas==13.1.0.3",
+        "nvidia-cuda-cccl==13.3.3.3.1",
+        "nvidia-cuda-crt==13.3.33",
+        "nvidia-cuda-cupti==13.0.85",
+        "nvidia-cuda-nvcc==13.2.78",
+        "nvidia-cuda-nvrtc==13.0.88",
+        "nvidia-cuda-runtime==13.0.96",
+        "nvidia-cuda-tileiras==13.2.78",
+        "nvidia-cudnn-cu13==9.19.0.56",
+        "nvidia-cudnn-frontend==1.25.0",
+        "nvidia-cufft==12.0.0.61",
+        "nvidia-cufile==1.15.1.6",
+        "nvidia-curand==10.4.0.35",
+        "nvidia-cusolver==12.0.4.66",
+        "nvidia-cusparse==12.6.3.3",
+        "nvidia-cusparselt-cu13==0.8.0",
+        "nvidia-cutlass-dsl==4.5.2",
+        "nvidia-cutlass-dsl-libs-base==4.5.2",
+        "nvidia-cutlass-dsl-libs-cu13==4.5.2",
+        "nvidia-ml-py==13.610.43",
+        "nvidia-nccl-cu13==2.28.9",
+        "nvidia-nvjitlink==13.0.88",
+        "nvidia-nvshmem-cu13==3.4.5",
+        "nvidia-nvtx==13.0.85",
+        "nvidia-nvvm==13.2.78",
+        "openai==2.44.0",
+        "openai-harmony==0.0.8",
+        "opencv-python-headless==4.13.0.92",
+        "opentelemetry-api==1.43.0",
+        "opentelemetry-exporter-otlp==1.43.0",
+        "opentelemetry-exporter-otlp-proto-common==1.43.0",
+        "opentelemetry-exporter-otlp-proto-grpc==1.43.0",
+        "opentelemetry-exporter-otlp-proto-http==1.43.0",
+        "opentelemetry-proto==1.43.0",
+        "opentelemetry-sdk==1.43.0",
+        "opentelemetry-semantic-conventions==0.64b0",
+        "opentelemetry-semantic-conventions-ai==0.5.1",
+        "orjson==3.10.18",
+        "outlines_core==0.2.14",
+        "packaging==26.2",
+        "pandas==3.0.3",
+        "partial-json-parser==0.2.1.1.post7",
+        "pillow==12.2.0",
+        "pqdm==0.2.0",
+        "prometheus-fastapi-instrumentator==8.0.2",
+        "prometheus_client==0.25.0",
+        "propcache==0.5.2",
+        "protobuf==7.35.1",
+        "psutil==7.2.2",
+        "py-cpuinfo==9.0.0",
+        "pyarrow==24.0.0",
+        "pyasn1==0.6.3",
+        "pyasn1_modules==0.4.2",
+        "pybase64==1.4.3",
+        "pycountry==26.2.16",
+        "pycparser==3.0",
+        "pydantic==2.13.4",
+        "pydantic-extra-types==2.11.1",
+        "pydantic-settings==2.14.2",
+        "pydantic_core==2.46.4",
+        "pyelftools==0.33",
+        "Pygments==2.20.0",
+        "PyJWT==2.13.0",
+        "python-dateutil==2.9.0.post0",
+        "python-dotenv==1.2.2",
+        "python-json-logger==4.1.0",
+        "python-multipart==0.0.32",
+        "PyYAML==6.0.3",
+        "pyzmq==27.1.0",
+        "quack-kernels==0.5.0",
+        "referencing==0.37.0",
+        "regex==2026.5.9",
+        "requests==2.34.2",
+        "rich==15.0.0",
+        "rich-toolkit==0.20.1",
+        "rignore==0.7.6",
+        "rpds-py==2026.5.1",
+        "safetensors==0.8.0",
+        "sentencepiece==0.2.1",
+        "sentry-sdk==2.63.0",
+        "setproctitle==1.3.7",
+        "shellingham==1.5.4",
+        "six==1.17.0",
+        "sniffio==1.3.1",
+        "sse-starlette==3.4.5",
+        "starlette==1.3.1",
+        "supervisor==4.3.0",
+        "sympy==1.14.0",
+        "tabulate==0.10.0",
+        "tempdir==0.7.1",
+        "tenacity==9.1.4",
+        "termcolor==3.3.0",
+        "tiktoken==0.13.0",
+        "tilelang==0.1.9",
+        "tokenizers==0.22.2",
+        "tokenspeed-mla==0.1.2",
+        "tokenspeed-triton==3.7.10.post20260531",
+        "torch==2.11.0",
+        "torch_c_dlpack_ext==0.1.5",
+        "torchaudio==2.11.0",
+        "torchvision==0.26.0",
+        "tqdm==4.68.3",
+        "transformers==5.12.1",
+        "tree-sitter==0.25.2",
+        "tree-sitter-python==0.25.0",
+        "triton==3.6.0",
+        "typer==0.25.1",
+        "typing-inspection==0.4.2",
+        "typing_extensions==4.15.0",
+        "urllib3==2.7.0",
+        "uvicorn==0.49.0",
+        "uvloop==0.22.1",
+        "vllm==0.23.0",
+        "watchfiles==1.2.0",
+        "wcmatch==10.1",
+        "websockets==16.0",
+        "wget==3.2",
+        "xgrammar==0.2.3",
+        "xxhash==3.7.1",
+        "yarl==1.24.2",
+        "z3-solver==4.15.4.0"
+      ],
+      "evalplus": [
+        "aiohappyeyeballs==2.6.2",
+        "aiohttp==3.14.1",
+        "aiosignal==1.4.0",
+        "annotated-doc==0.0.4",
+        "annotated-types==0.7.0",
+        "anthropic==0.112.0",
+        "anyio==4.14.1",
+        "appdirs==1.4.4",
+        "attrs==26.1.0",
+        "certifi==2026.6.17",
+        "cffi==2.0.0",
+        "charset-normalizer==3.4.7",
+        "click==8.4.2",
+        "cryptography==49.0.0",
+        "datasets==5.0.0",
+        "dill==0.4.1",
+        "distro==1.9.0",
+        "docstring_parser==0.18.0",
+        "evalplus==0.3.1",
+        "filelock==3.29.4",
+        "fire==0.7.1",
+        "frozenlist==1.8.0",
+        "fsspec==2026.4.0",
+        "google-ai-generativelanguage==0.6.15",
+        "google-api-core==2.31.0",
+        "google-api-python-client==2.198.0",
+        "google-auth==2.55.1",
+        "google-auth-httplib2==0.4.0",
+        "google-generativeai==0.8.6",
+        "googleapis-common-protos==1.75.0",
+        "grpcio==1.81.1",
+        "grpcio-status==1.71.2",
+        "h11==0.16.0",
+        "hf-xet==1.5.1",
+        "httpcore==1.0.9",
+        "httplib2==0.32.0",
+        "httpx==0.28.1",
+        "huggingface_hub==1.21.0",
+        "idna==3.18",
+        "jiter==0.15.0",
+        "markdown-it-py==4.2.0",
+        "mdurl==0.1.2",
+        "multidict==6.7.1",
+        "multipledispatch==1.0.0",
+        "multiprocess==0.70.19",
+        "numpy==2.4.6",
+        "openai==2.44.0",
+        "packaging==26.2",
+        "pandas==3.0.3",
+        "propcache==0.5.2",
+        "proto-plus==1.28.0",
+        "protobuf==5.29.6",
+        "psutil==7.2.2",
+        "pyarrow==24.0.0",
+        "pyasn1==0.6.3",
+        "pyasn1_modules==0.4.2",
+        "pycparser==3.0",
+        "pydantic==2.13.4",
+        "pydantic_core==2.46.4",
+        "Pygments==2.20.0",
+        "pyparsing==3.3.2",
+        "python-dateutil==2.9.0.post0",
+        "PyYAML==6.0.3",
+        "regex==2026.5.9",
+        "requests==2.34.2",
+        "rich==15.0.0",
+        "safetensors==0.8.0",
+        "shellingham==1.5.4",
+        "six==1.17.0",
+        "sniffio==1.3.1",
+        "stop-sequencer==1.2.3",
+        "tempdir==0.7.1",
+        "termcolor==3.3.0",
+        "tokenizers==0.22.2",
+        "tqdm==4.68.3",
+        "transformers==5.12.1",
+        "tree-sitter==0.25.2",
+        "tree-sitter-python==0.25.0",
+        "typer==0.25.1",
+        "typing-inspection==0.4.2",
+        "typing_extensions==4.15.0",
+        "uritemplate==4.2.0",
+        "urllib3==2.7.0",
+        "wget==3.2",
+        "xxhash==3.7.1",
+        "yarl==1.24.2"
+      ],
+      "livecodebench": [
+        "aiohappyeyeballs==2.6.2",
+        "aiohttp==3.14.1",
+        "aiosignal==1.4.0",
+        "annotated-doc==0.0.4",
+        "annotated-types==0.7.0",
+        "anthropic==0.112.0",
+        "anyio==4.14.1",
+        "apache-tvm-ffi==0.1.9",
+        "astor==0.8.1",
+        "attrs==26.1.0",
+        "blake3==1.0.9",
+        "cachetools==7.1.4",
+        "cbor2==6.1.2",
+        "certifi==2026.6.17",
+        "cffi==2.0.0",
+        "charset-normalizer==3.4.7",
+        "click==8.4.2",
+        "cloudpickle==3.1.2",
+        "cohere==7.0.4",
+        "compressed-tensors==0.17.0",
+        "cryptography==49.0.0",
+        "cuda-bindings==13.3.1",
+        "cuda-core==1.0.1",
+        "cuda-pathfinder==1.5.5",
+        "cuda-python==13.3.1",
+        "cuda-tile==1.3.0",
+        "cuda-toolkit==13.0.2",
+        "cyclopts==4.19.0",
+        "datasets==5.0.0",
+        "depyf==0.20.0",
+        "detect-installer==0.1.0",
+        "detect_agent==0.3.0",
+        "dill==0.4.1",
+        "diskcache==5.6.3",
+        "distro==1.9.0",
+        "dnspython==2.8.0",
+        "docstring_parser==0.18.0",
+        "einops==0.8.2",
+        "email-validator==2.3.0",
+        "fastapi==0.136.3",
+        "fastapi-cli==0.0.27",
+        "fastapi-cloud-cli==0.21.0",
+        "fastar==0.11.0",
+        "fastavro==1.12.2",
+        "fastsafetensors==0.3.2",
+        "filelock==3.29.4",
+        "flashinfer-cubin==0.6.12",
+        "flashinfer-python==0.6.12",
+        "frozenlist==1.8.0",
+        "fsspec==2026.4.0",
+        "gguf==0.19.0",
+        "google-auth==2.55.1",
+        "google-genai==2.10.0",
+        "googleapis-common-protos==1.75.0",
+        "grpcio==1.81.1",
+        "h11==0.16.0",
+        "hf-xet==1.5.1",
+        "httpcore==1.0.9",
+        "httptools==0.8.0",
+        "httpx==0.28.1",
+        "httpx-sse==0.4.3",
+        "huggingface_hub==1.21.0",
+        "humming-kernels==0.1.4",
+        "idna==3.18",
+        "ijson==3.5.0",
+        "interegular==0.3.3",
+        "Jinja2==3.1.6",
+        "jiter==0.15.0",
+        "jmespath==1.1.0",
+        "jsonschema==4.26.0",
+        "jsonschema-specifications==2025.9.1",
+        "lark==1.2.2",
+        "livecodebench @ git+https://github.com/LiveCodeBench/LiveCodeBench.git@28fef95ea8c9f7a547c8329f2cd3d32b92c1fa24",
+        "llguidance==1.7.6",
+        "llvmlite==0.47.0",
+        "lm-format-enforcer==0.11.3",
+        "loguru==0.7.3",
+        "markdown-it-py==4.2.0",
+        "MarkupSafe==3.0.3",
+        "mcp==1.28.1",
+        "mdurl==0.1.2",
+        "mistral_common==1.11.5",
+        "mistralai==0.4.2",
+        "ml_dtypes==0.5.4",
+        "model-hosting-container-standards==0.1.16",
+        "mpmath==1.3.0",
+        "msgspec==0.21.1",
+        "multidict==6.7.1",
+        "multiprocess==0.70.19",
+        "networkx==3.6.1",
+        "ninja==1.13.0",
+        "numba==0.65.0",
+        "numpy==2.3.5",
+        "nvidia-cublas==13.1.0.3",
+        "nvidia-cuda-cccl==13.3.3.3.1",
+        "nvidia-cuda-crt==13.3.33",
+        "nvidia-cuda-cupti==13.0.85",
+        "nvidia-cuda-nvcc==13.2.78",
+        "nvidia-cuda-nvrtc==13.0.88",
+        "nvidia-cuda-runtime==13.0.96",
+        "nvidia-cuda-tileiras==13.2.78",
+        "nvidia-cudnn-cu13==9.19.0.56",
+        "nvidia-cudnn-frontend==1.25.0",
+        "nvidia-cufft==12.0.0.61",
+        "nvidia-cufile==1.15.1.6",
+        "nvidia-curand==10.4.0.35",
+        "nvidia-cusolver==12.0.4.66",
+        "nvidia-cusparse==12.6.3.3",
+        "nvidia-cusparselt-cu13==0.8.0",
+        "nvidia-cutlass-dsl==4.5.2",
+        "nvidia-cutlass-dsl-libs-base==4.5.2",
+        "nvidia-cutlass-dsl-libs-cu13==4.5.2",
+        "nvidia-ml-py==13.610.43",
+        "nvidia-nccl-cu13==2.28.9",
+        "nvidia-nvjitlink==13.0.88",
+        "nvidia-nvshmem-cu13==3.4.5",
+        "nvidia-nvtx==13.0.85",
+        "nvidia-nvvm==13.2.78",
+        "openai==2.44.0",
+        "openai-harmony==0.0.8",
+        "opencv-python-headless==4.13.0.92",
+        "opentelemetry-api==1.43.0",
+        "opentelemetry-exporter-otlp==1.43.0",
+        "opentelemetry-exporter-otlp-proto-common==1.43.0",
+        "opentelemetry-exporter-otlp-proto-grpc==1.43.0",
+        "opentelemetry-exporter-otlp-proto-http==1.43.0",
+        "opentelemetry-proto==1.43.0",
+        "opentelemetry-sdk==1.43.0",
+        "opentelemetry-semantic-conventions==0.64b0",
+        "opentelemetry-semantic-conventions-ai==0.5.1",
+        "orjson==3.10.18",
+        "outlines_core==0.2.14",
+        "packaging==26.2",
+        "pandas==3.0.3",
+        "partial-json-parser==0.2.1.1.post7",
+        "Pebble==5.2.0",
+        "pillow==12.2.0",
+        "prometheus-fastapi-instrumentator==8.0.2",
+        "prometheus_client==0.25.0",
+        "propcache==0.5.2",
+        "protobuf==7.35.1",
+        "psutil==7.2.2",
+        "py-cpuinfo==9.0.0",
+        "pyarrow==24.0.0",
+        "pyasn1==0.6.3",
+        "pyasn1_modules==0.4.2",
+        "pybase64==1.4.3",
+        "pycountry==26.2.16",
+        "pycparser==3.0",
+        "pydantic==2.13.4",
+        "pydantic-extra-types==2.11.1",
+        "pydantic-settings==2.14.2",
+        "pydantic_core==2.46.4",
+        "pyelftools==0.33",
+        "Pygments==2.20.0",
+        "PyJWT==2.13.0",
+        "python-dateutil==2.9.0.post0",
+        "python-dotenv==1.2.2",
+        "python-json-logger==4.1.0",
+        "python-multipart==0.0.32",
+        "PyYAML==6.0.3",
+        "pyzmq==27.1.0",
+        "quack-kernels==0.5.0",
+        "referencing==0.37.0",
+        "regex==2026.5.9",
+        "requests==2.34.2",
+        "rich==15.0.0",
+        "rich-rst==2.0.2",
+        "rich-toolkit==0.20.1",
+        "rignore==0.7.6",
+        "rpds-py==2026.5.1",
+        "safetensors==0.8.0",
+        "sentencepiece==0.2.1",
+        "sentry-sdk==2.63.0",
+        "setproctitle==1.3.7",
+        "shellingham==1.5.4",
+        "six==1.17.0",
+        "sniffio==1.3.1",
+        "sse-starlette==3.4.5",
+        "starlette==1.3.1",
+        "supervisor==4.3.0",
+        "sympy==1.14.0",
+        "tabulate==0.10.0",
+        "tenacity==9.1.4",
+        "tiktoken==0.13.0",
+        "tilelang==0.1.9",
+        "together==2.19.0",
+        "tokenizers==0.22.2",
+        "tokenspeed-mla==0.1.2",
+        "tokenspeed-triton==3.7.10.post20260531",
+        "torch==2.11.0",
+        "torch_c_dlpack_ext==0.1.5",
+        "torchaudio==2.11.0",
+        "torchvision==0.26.0",
+        "tqdm==4.68.3",
+        "transformers==5.12.1",
+        "triton==3.6.0",
+        "typer==0.25.1",
+        "types-PyYAML==6.0.12.20260518",
+        "types-requests==2.33.0.20260518",
+        "types-tabulate==0.10.0.20260508",
+        "types-tqdm==4.68.0.20260608",
+        "typing-inspection==0.4.2",
+        "typing_extensions==4.15.0",
+        "urllib3==2.7.0",
+        "uvicorn==0.49.0",
+        "uvloop==0.22.1",
+        "vllm==0.23.0",
+        "watchfiles==1.2.0",
+        "websockets==16.0",
+        "xgrammar==0.2.3",
+        "xxhash==3.7.1",
+        "yarl==1.24.2",
+        "z3-solver==4.15.4.0"
+      ],
+      "lm_eval": [
+        "absl-py==2.4.0",
+        "aiohappyeyeballs==2.6.2",
+        "aiohttp==3.14.1",
+        "aiosignal==1.4.0",
+        "annotated-doc==0.0.4",
+        "anyio==4.14.1",
+        "attrs==26.1.0",
+        "certifi==2026.6.17",
+        "chardet==6.0.0.post1",
+        "charset-normalizer==3.4.7",
+        "click==8.4.2",
+        "colorama==0.4.6",
+        "DataProperty==1.1.1",
+        "datasets==5.0.0",
+        "dill==0.4.1",
+        "evaluate==0.4.6",
+        "filelock==3.29.4",
+        "frozenlist==1.8.0",
+        "fsspec==2026.4.0",
+        "h11==0.16.0",
+        "hf-xet==1.5.1",
+        "httpcore==1.0.9",
+        "httpx==0.28.1",
+        "huggingface_hub==1.21.0",
+        "idna==3.18",
+        "immutabledict==4.3.1",
+        "Jinja2==3.1.6",
+        "joblib==1.5.3",
+        "langdetect==1.0.9",
+        "lm_eval==0.4.12",
+        "lxml==6.1.1",
+        "markdown-it-py==4.2.0",
+        "MarkupSafe==3.0.3",
+        "mbstrdecoder==1.1.5",
+        "mdurl==0.1.2",
+        "more-itertools==11.1.0",
+        "multidict==6.7.1",
+        "multiprocess==0.70.19",
+        "narwhals==2.22.1",
+        "nltk==3.9.4",
+        "numpy==2.4.6",
+        "packaging==26.2",
+        "pandas==3.0.3",
+        "pathvalidate==3.3.1",
+        "portalocker==3.2.0",
+        "propcache==0.5.2",
+        "pyarrow==24.0.0",
+        "Pygments==2.20.0",
+        "pytablewriter==1.2.1",
+        "python-dateutil==2.9.0.post0",
+        "pytz==2026.2",
+        "PyYAML==6.0.3",
+        "regex==2026.5.9",
+        "requests==2.34.2",
+        "rich==15.0.0",
+        "rouge_score==0.1.2",
+        "sacrebleu==2.6.0",
+        "scikit-learn==1.9.0",
+        "scipy==1.17.1",
+        "shellingham==1.5.4",
+        "six==1.17.0",
+        "sqlitedict==2.1.0",
+        "tabledata==1.3.5",
+        "tabulate==0.10.0",
+        "tcolorpy==0.1.7",
+        "tenacity==9.1.4",
+        "threadpoolctl==3.6.0",
+        "tiktoken==0.13.0",
+        "tqdm==4.68.3",
+        "typepy==1.3.5",
+        "typer==0.25.1",
+        "typing_extensions==4.15.0",
+        "urllib3==2.7.0",
+        "word2number==1.1",
+        "xxhash==3.7.1",
+        "yarl==1.24.2"
+      ]
+    },
+    "arguments": {
+      "setup": false,
+      "run": true,
+      "benchmarks": "all",
+      "base_url": "http://127.0.0.1:1234/v1",
+      "api_key": "lm-studio",
+      "model": "auto",
+      "output": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark",
+      "env_root": "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs",
+      "resume": true,
+      "bigcode_execution": "gradio",
+      "lcb_release": "release_v6",
+      "max_code_tokens": 4096,
+      "bfcl_model_alias": "Qwen/Qwen3-4B-Instruct-2507-FC",
+      "bfcl_categories": "all_scoring",
+      "bfcl_tokenizer_path": null,
+      "continue_on_error": true
+    }
+  },
+  "model": "qwen3.5-4b-super-coder",
+  "base_url": "http://127.0.0.1:1234/v1",
+  "results": [
+    {
+      "benchmark": "HumanEval+ Pass@1",
+      "score_percent": 0.0,
+      "status": "ok",
+      "metric": "Pass@1",
+      "source": "Official EvalPlus",
+      "details": {
+        "parsed_from": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/evalplus/humaneval/evalplus_results/humaneval/qwen3.5-4b-super-coder_openai_temp_0.0_eval_results.json",
+        "json_key": "eval.HumanEval/22.0.plus_fail_tests.0.0.3",
+        "raw_value": 0,
+        "pattern_matches": 1,
+        "command": {
+          "name": "evalplus_humaneval",
+          "command": [
+            "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/evalplus/bin/evalplus.evaluate",
+            "--model",
+            "qwen3.5-4b-super-coder",
+            "--dataset",
+            "humaneval",
+            "--backend",
+            "openai",
+            "--base-url",
+            "http://127.0.0.1:1234/v1",
+            "--greedy"
+          ],
+          "returncode": 0,
+          "elapsed_seconds": 1646.0040039900014,
+          "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/evalplus_humaneval.log",
+          "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/evalplus/humaneval"
+        }
+      }
+    },
+    {
+      "benchmark": "MBPP+ Pass@1",
+      "score_percent": 0.0,
+      "status": "ok",
+      "metric": "Pass@1",
+      "source": "Official EvalPlus",
+      "details": {
+        "parsed_from": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/evalplus/mbpp/evalplus_results/mbpp/qwen3.5-4b-super-coder_openai_temp_0.0_eval_results.json",
+        "json_key": "eval.Mbpp/6.0.plus_fail_tests.0.0",
+        "raw_value": 0,
+        "pattern_matches": 1,
+        "command": {
+          "name": "evalplus_mbpp",
+          "command": [
+            "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/evalplus/bin/evalplus.evaluate",
+            "--model",
+            "qwen3.5-4b-super-coder",
+            "--dataset",
+            "mbpp",
+            "--backend",
+            "openai",
+            "--base-url",
+            "http://127.0.0.1:1234/v1",
+            "--greedy"
+          ],
+          "returncode": 0,
+          "elapsed_seconds": 3226.43256972,
+          "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/evalplus_mbpp.log",
+          "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/evalplus/mbpp"
+        }
+      }
+    },
+    {
+      "benchmark": "BigCodeBench-Hard",
+      "score_percent": null,
+      "status": "needs_review",
+      "metric": "Calibrated Pass@1",
+      "source": "Official BigCodeBench",
+      "details": {
+        "command": {
+          "name": "bigcodebench_hard",
+          "command": [
+            "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/bigcodebench/bin/bigcodebench.evaluate",
+            "--model",
+            "qwen3.5-4b-super-coder",
+            "--execution",
+            "gradio",
+            "--split",
+            "instruct",
+            "--subset",
+            "hard",
+            "--backend",
+            "openai",
+            "--base_url",
+            "http://127.0.0.1:1234/v1",
+            "--bs",
+            "1",
+            "--n_samples",
+            "1",
+            "--temperature",
+            "0.0",
+            "--greedy",
+            "--pass_k",
+            "1",
+            "--root",
+            "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bigcodebench/bcb_results"
+          ],
+          "returncode": 1,
+          "elapsed_seconds": 1760.3364485260026,
+          "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/bigcodebench_hard.log",
+          "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bigcodebench"
+        },
+        "execution_backend": "gradio"
+      }
+    },
+    {
+      "benchmark": "BFCL v4",
+      "score_percent": null,
+      "status": "needs_review",
+      "metric": "Overall accuracy for selected categories",
+      "source": "Official BFCL",
+      "details": {
+        "categories": "all_scoring",
+        "generation_command": {
+          "name": "bfcl_v4_generate",
+          "command": [
+            "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/bfcl/bin/bfcl",
+            "generate",
+            "--model",
+            "Qwen/Qwen3-4B-Instruct-2507-FC",
+            "--test-category",
+            "all_scoring",
+            "--skip-server-setup",
+            "--num-threads",
+            "1",
+            "--result-dir",
+            "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl/project/result"
+          ],
+          "returncode": 1,
+          "elapsed_seconds": 2.7761374679976143,
+          "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/bfcl_v4_generate.log",
+          "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl"
+        },
+        "evaluation_command": {
+          "name": "bfcl_v4_evaluate",
+          "command": [
+            "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/bfcl/bin/bfcl",
+            "evaluate",
+            "--model",
+            "Qwen/Qwen3-4B-Instruct-2507-FC",
+            "--test-category",
+            "all_scoring",
+            "--result-dir",
+            "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl/project/result",
+            "--score-dir",
+            "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl/project/score"
+          ],
+          "returncode": 1,
+          "elapsed_seconds": 2.4078060280007776,
+          "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/bfcl_v4_evaluate.log",
+          "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl"
+        },
+        "served_alias": "Qwen/Qwen3-4B-Instruct-2507-FC"
+      }
+    },
+    {
+      "benchmark": "BFCL no-tool/relevance",
+      "score_percent": null,
+      "status": "not_available",
+      "metric": "BFCL irrelevance/relevance detection",
+      "source": "Official BFCL",
+      "details": {}
+    },
+    {
+      "benchmark": "IFEval",
+      "score_percent": null,
+      "status": "needs_review",
+      "metric": "Prompt-level strict accuracy",
+      "source": "LM Evaluation Harness",
+      "details": {
+        "command": {
+          "name": "lm_eval_ifeval",
+          "command": [
+            "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/lm_eval/bin/lm_eval",
+            "--model",
+            "local-chat-completions",
+            "--model_args",
+            "model=qwen3.5-4b-super-coder,base_url=http://127.0.0.1:1234/v1/chat/completions,num_concurrent=1,max_retries=4,tokenized_requests=False,batch_size=1",
+            "--tasks",
+            "ifeval",
+            "--batch_size",
+            "1",
+            "--log_samples",
+            "--output_path",
+            "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/ifeval/results"
+          ],
+          "returncode": 1,
+          "elapsed_seconds": 5.503946738001105,
+          "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/lm_eval_ifeval.log",
+          "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/ifeval"
+        }
+      }
+    },
+    {
+      "benchmark": "MMLU-Pro",
+      "score_percent": null,
+      "status": "needs_review",
+      "metric": "Accuracy",
+      "source": "LM Evaluation Harness",
+      "details": {
+        "command": {
+          "name": "lm_eval_mmlu_pro",
+          "command": [
+            "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/lm_eval/bin/lm_eval",
+            "--model",
+            "local-chat-completions",
+            "--model_args",
+            "model=qwen3.5-4b-super-coder,base_url=http://127.0.0.1:1234/v1/chat/completions,num_concurrent=1,max_retries=4,tokenized_requests=False,batch_size=1",
+            "--tasks",
+            "mmlu_pro",
+            "--batch_size",
+            "1",
+            "--log_samples",
+            "--output_path",
+            "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/mmlu_pro/results"
+          ],
+          "returncode": 1,
+          "elapsed_seconds": 4.813302385999123,
+          "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/lm_eval_mmlu_pro.log",
+          "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/mmlu_pro"
+        }
+      }
+    },
+    {
+      "benchmark": "JSON validity",
+      "score_percent": 40.0,
+      "status": "ok",
+      "metric": "Schema-valid JSON rate",
+      "source": "Custom deterministic endpoint diagnostic",
+      "details": {
+        "total": 30,
+        "json_parse_rate_percent": 96.66666666666667,
+        "schema_valid_rate_percent": 40.0,
+        "responses": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/custom/json_validity/responses.json"
+      }
+    },
+    {
+      "benchmark": "No-tool accuracy",
+      "score_percent": 87.5,
+      "status": "ok",
+      "metric": "Correctly abstained from tool use",
+      "source": "Custom deterministic native-tool diagnostic",
+      "details": {
+        "correct": 35,
+        "total": 40,
+        "responses": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/custom/no_tool/responses.json"
+      }
+    }
+  ],
+  "commands": [
+    {
+      "name": "evalplus_humaneval",
+      "command": [
+        "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/evalplus/bin/evalplus.evaluate",
+        "--model",
+        "qwen3.5-4b-super-coder",
+        "--dataset",
+        "humaneval",
+        "--backend",
+        "openai",
+        "--base-url",
+        "http://127.0.0.1:1234/v1",
+        "--greedy"
+      ],
+      "returncode": 0,
+      "elapsed_seconds": 1646.0040039900014,
+      "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/evalplus_humaneval.log",
+      "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/evalplus/humaneval"
+    },
+    {
+      "name": "evalplus_mbpp",
+      "command": [
+        "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/evalplus/bin/evalplus.evaluate",
+        "--model",
+        "qwen3.5-4b-super-coder",
+        "--dataset",
+        "mbpp",
+        "--backend",
+        "openai",
+        "--base-url",
+        "http://127.0.0.1:1234/v1",
+        "--greedy"
+      ],
+      "returncode": 0,
+      "elapsed_seconds": 3226.43256972,
+      "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/evalplus_mbpp.log",
+      "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/evalplus/mbpp"
+    },
+    {
+      "name": "bigcodebench_hard",
+      "command": [
+        "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/bigcodebench/bin/bigcodebench.evaluate",
+        "--model",
+        "qwen3.5-4b-super-coder",
+        "--execution",
+        "gradio",
+        "--split",
+        "instruct",
+        "--subset",
+        "hard",
+        "--backend",
+        "openai",
+        "--base_url",
+        "http://127.0.0.1:1234/v1",
+        "--bs",
+        "1",
+        "--n_samples",
+        "1",
+        "--temperature",
+        "0.0",
+        "--greedy",
+        "--pass_k",
+        "1",
+        "--root",
+        "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bigcodebench/bcb_results"
+      ],
+      "returncode": 1,
+      "elapsed_seconds": 1760.3364485260026,
+      "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/bigcodebench_hard.log",
+      "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bigcodebench"
+    },
+    {
+      "name": "bfcl_v4_generate",
+      "command": [
+        "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/bfcl/bin/bfcl",
+        "generate",
+        "--model",
+        "Qwen/Qwen3-4B-Instruct-2507-FC",
+        "--test-category",
+        "all_scoring",
+        "--skip-server-setup",
+        "--num-threads",
+        "1",
+        "--result-dir",
+        "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl/project/result"
+      ],
+      "returncode": 1,
+      "elapsed_seconds": 2.7761374679976143,
+      "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/bfcl_v4_generate.log",
+      "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl"
+    },
+    {
+      "name": "bfcl_v4_evaluate",
+      "command": [
+        "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/bfcl/bin/bfcl",
+        "evaluate",
+        "--model",
+        "Qwen/Qwen3-4B-Instruct-2507-FC",
+        "--test-category",
+        "all_scoring",
+        "--result-dir",
+        "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl/project/result",
+        "--score-dir",
+        "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl/project/score"
+      ],
+      "returncode": 1,
+      "elapsed_seconds": 2.4078060280007776,
+      "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/bfcl_v4_evaluate.log",
+      "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/bfcl"
+    },
+    {
+      "name": "lm_eval_ifeval",
+      "command": [
+        "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/lm_eval/bin/lm_eval",
+        "--model",
+        "local-chat-completions",
+        "--model_args",
+        "model=qwen3.5-4b-super-coder,base_url=http://127.0.0.1:1234/v1/chat/completions,num_concurrent=1,max_retries=4,tokenized_requests=False,batch_size=1",
+        "--tasks",
+        "ifeval",
+        "--batch_size",
+        "1",
+        "--log_samples",
+        "--output_path",
+        "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/ifeval/results"
+      ],
+      "returncode": 1,
+      "elapsed_seconds": 5.503946738001105,
+      "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/lm_eval_ifeval.log",
+      "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/ifeval"
+    },
+    {
+      "name": "lm_eval_mmlu_pro",
+      "command": [
+        "/home/jica/repo/rocm_unsloth/benchmark/.benchmark_envs/lm_eval/bin/lm_eval",
+        "--model",
+        "local-chat-completions",
+        "--model_args",
+        "model=qwen3.5-4b-super-coder,base_url=http://127.0.0.1:1234/v1/chat/completions,num_concurrent=1,max_retries=4,tokenized_requests=False,batch_size=1",
+        "--tasks",
+        "mmlu_pro",
+        "--batch_size",
+        "1",
+        "--log_samples",
+        "--output_path",
+        "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/mmlu_pro/results"
+      ],
+      "returncode": 1,
+      "elapsed_seconds": 4.813302385999123,
+      "log_path": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/logs/lm_eval_mmlu_pro.log",
+      "cwd": "/home/jica/repo/rocm_unsloth/benchmark/lmstudio_q4_benchmark/lm_eval/mmlu_pro"
+    }
+  ]
+}

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,158 @@

+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- set num_sys = 0 %}
+{%- set merged_system = '' %}
+{%- if messages[0].role == 'system' or messages[0].role == 'developer' %}
+    {%- set first = render_content(messages[0].content, false, true)|trim %}
+    {%- if messages|length > 1 and (messages[1].role == 'system' or messages[1].role == 'developer') %}
+        {%- set second = render_content(messages[1].content, false, true)|trim %}
+        {%- set merged_system = first + '\n' + second %}
+        {%- set num_sys = 2 %}
+    {%- else %}
+        {%- set merged_system = first %}
+        {%- set num_sys = 1 %}
+    {%- endif %}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if merged_system %}
+        {{- '\n\n' + merged_system }}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if merged_system %}
+        {{- '<|im_start|>system\n' + merged_system + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if loop.index0 >= num_sys and message.role != "system" and message.role != "developer" %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is mapping %}
+                    {%- for args_name in tool_call.arguments %}
+                        {%- set args_value = tool_call.arguments[args_name] %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}
+{#- Unsloth fixes - developer role, tool calling #}

config.json ADDED Viewed

	@@ -0,0 +1,111 @@

+{
+    "architectures": [
+        "Qwen3_5ForConditionalGeneration"
+    ],
+    "torch_dtype": "bfloat16",
+    "image_token_id": 248056,
+    "model_name": "Qwen/Qwen3.5-4B",
+    "model_type": "qwen3_5",
+    "pad_token_id": 248044,
+    "text_config": {
+        "attention_bias": false,
+        "attention_dropout": 0.0,
+        "attn_output_gate": true,
+        "bos_token_id": null,
+        "torch_dtype": "bfloat16",
+        "eos_token_id": 248044,
+        "full_attention_interval": 4,
+        "head_dim": 256,
+        "hidden_act": "silu",
+        "hidden_size": 2560,
+        "initializer_range": 0.02,
+        "intermediate_size": 9216,
+        "layer_types": [
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention"
+        ],
+        "linear_conv_kernel_dim": 4,
+        "linear_key_head_dim": 128,
+        "linear_num_key_heads": 16,
+        "linear_num_value_heads": 32,
+        "linear_value_head_dim": 128,
+        "mamba_ssm_dtype": "float32",
+        "max_position_embeddings": 262144,
+        "mlp_only_layers": [],
+        "model_type": "qwen3_5_text",
+        "mtp_num_hidden_layers": 1,
+        "mtp_use_dedicated_embeddings": false,
+        "num_attention_heads": 16,
+        "num_hidden_layers": 32,
+        "num_key_value_heads": 4,
+        "pad_token_id": null,
+        "partial_rotary_factor": 0.25,
+        "rms_norm_eps": 1e-06,
+        "rope_parameters": {
+            "mrope_interleaved": true,
+            "mrope_section": [
+                11,
+                11,
+                10
+            ],
+            "partial_rotary_factor": 0.25,
+            "rope_theta": 10000000,
+            "rope_type": "default"
+        },
+        "tie_word_embeddings": true,
+        "use_cache": false,
+        "vocab_size": 248320
+    },
+    "tie_word_embeddings": true,
+    "unsloth_version": "2026.6.3",
+    "video_token_id": 248057,
+    "vision_config": {
+        "deepstack_visual_indexes": [],
+        "depth": 24,
+        "torch_dtype": "bfloat16",
+        "hidden_act": "gelu_pytorch_tanh",
+        "hidden_size": 1024,
+        "in_channels": 3,
+        "initializer_range": 0.02,
+        "intermediate_size": 4096,
+        "model_type": "qwen3_5",
+        "num_heads": 16,
+        "num_position_embeddings": 2304,
+        "out_hidden_size": 2560,
+        "patch_size": 16,
+        "spatial_merge_size": 2,
+        "temporal_patch_size": 2
+    },
+    "vision_end_token_id": 248054,
+    "vision_start_token_id": 248053
+}

processor_config.json ADDED Viewed

	@@ -0,0 +1,63 @@

+{
+  "image_processor": {
+    "data_format": "channels_first",
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "merge_size": 2,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 16777216,
+      "shortest_edge": 65536
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen3VLProcessor",
+  "video_processor": {
+    "data_format": "channels_first",
+    "default_to_square": true,
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "fps": 2,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_frames": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 25165824,
+      "shortest_edge": 4096
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen3VLVideoProcessor"
+  }
+}

qwen3.5-4B-super-coder.BF16-mmproj.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:020a4a0203074c2e3925b676f19784ae6ceed79448abc7b02a4389a6ab1a4687
+size 675568864

qwen3.5-4B-super-coder.Q4_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cb304049c80e71b5fddc967f41e0a95a362b8094b955f10a72a9eecb91155578
+size 2611783040

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:87a7830d63fcf43bf241c3c5242e96e62dd3fdc29224ca26fed8ea333db72de4
+size 19989343

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,299 @@

+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "right",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "processor_class": "Qwen3VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>",
+  "added_tokens_decoder": {
+    "248044": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248045": {
+      "content": "<|im_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248046": {
+      "content": "<|im_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248047": {
+      "content": "<|object_ref_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248048": {
+      "content": "<|object_ref_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248049": {
+      "content": "<|box_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248050": {
+      "content": "<|box_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248051": {
+      "content": "<|quad_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248052": {
+      "content": "<|quad_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248053": {
+      "content": "<|vision_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248054": {
+      "content": "<|vision_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248055": {
+      "content": "<|vision_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248056": {
+      "content": "<|image_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248057": {
+      "content": "<|video_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248058": {
+      "content": "<tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248059": {
+      "content": "</tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248060": {
+      "content": "<|fim_prefix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248061": {
+      "content": "<|fim_middle|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248062": {
+      "content": "<|fim_suffix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248063": {
+      "content": "<|fim_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248064": {
+      "content": "<|repo_name|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248065": {
+      "content": "<|file_sep|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248066": {
+      "content": "<tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248067": {
+      "content": "</tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248068": {
+      "content": "<think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248069": {
+      "content": "</think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248070": {
+      "content": "<|audio_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248071": {
+      "content": "<|audio_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248072": {
+      "content": "<tts_pad>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248073": {
+      "content": "<tts_text_bos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248074": {
+      "content": "<tts_text_eod>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248075": {
+      "content": "<tts_text_bos_single>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248076": {
+      "content": "<|audio_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1da83ab52b01141c6bf8eb27d4a4bfbc8573cd761a0485c3db5145825151b2c0
+size 5713