| | --- |
| | library_name: transformers |
| | license: mit |
| | base_model: |
| | - LocoreMind/LocoOperator-4B |
| | tags: |
| | - code |
| | - agent |
| | - tool-calling |
| | - distillation |
| | - qwen3 |
| | - gguf |
| | - llama-cpp |
| | language: |
| | - en |
| | pipeline_tag: text-generation |
| | --- |
| | # This is a static quantization of [LocoreMind/LocoOperator-4B](https://huggingface.co/LocoreMind/LocoOperator-4B), made by [SimplySara](https://huggingface.co/SimplySara) |
| |
|
| | | Model | Size_GB | BPW | PPL_Q | KLD_Mean | KLD_Max | Top_P_Match | |
| | |:----------------------------------|----------:|------:|----------:|-----------:|----------:|:--------------| |
| | | LocoOperator-4B-BF16.gguf | 7.498 | 16.01 | 9.24309 | -1.2e-05 | 4e-06 | 100.000% | |
| | | LocoOperator-4B-MXFP4_MOE.gguf | 3.986 | 8.51 | 9.24606 | 0.001835 | 2.98238 | 97.518% | |
| | | LocoOperator-4B-i1-MXFP4_MOE.gguf | 3.986 | 8.51 | 9.24606 | 0.001835 | 2.98238 | 97.518% | |
| | | LocoOperator-4B-Q8_0.gguf | 3.986 | 8.51 | 9.24606 | 0.001835 | 2.98238 | 97.518% | |
| | | LocoOperator-4B-i1-Q8_0.gguf | 3.986 | 8.51 | 9.24606 | 0.001835 | 2.98238 | 97.518% | |
| | | LocoOperator-4B-Q6_K.gguf | 3.079 | 6.58 | 9.27926 | 0.0068 | 10.5686 | 95.526% | |
| | | LocoOperator-4B-i1-Q6_K.gguf | 3.079 | 6.58 | 9.295 | 0.006075 | 15.9945 | 95.857% | |
| | | LocoOperator-4B-i1-Q5_1.gguf | 2.841 | 6.07 | 9.28859 | 0.01364 | 2.98838 | 94.135% | |
| | | LocoOperator-4B-Q5_1.gguf | 2.841 | 6.07 | 9.43222 | 0.022675 | 16.3454 | 93.161% | |
| | | LocoOperator-4B-Q5_K_M.gguf | 2.691 | 5.75 | 9.35457 | 0.017023 | 12.3947 | 93.635% | |
| | | LocoOperator-4B-i1-Q5_K_M.gguf | 2.691 | 5.75 | 9.2965 | 0.013153 | 7.78613 | 94.257% | |
| | | LocoOperator-4B-i1-Q5_0.gguf | 2.636 | 5.63 | 9.42255 | 0.019663 | 17.94 | 93.208% | |
| | | LocoOperator-4B-Q5_0.gguf | 2.63 | 5.62 | 9.41521 | 0.023403 | 31.4019 | 92.839% | |
| | | LocoOperator-4B-Q5_K_S.gguf | 2.63 | 5.62 | 9.44087 | 0.022119 | 13.6483 | 92.800% | |
| | | LocoOperator-4B-i1-Q5_K_S.gguf | 2.63 | 5.62 | 9.28767 | 0.014865 | 7.65169 | 93.702% | |
| | | LocoOperator-4B-Q4_1.gguf | 2.418 | 5.16 | 9.66722 | 0.074718 | 15.0861 | 87.757% | |
| | | LocoOperator-4B-i1-Q4_1.gguf | 2.418 | 5.16 | 9.45293 | 0.038707 | 13.8444 | 90.574% | |
| | | LocoOperator-4B-Q4_K_M.gguf | 2.326 | 4.97 | 9.48239 | 0.048236 | 15.3105 | 90.300% | |
| | | LocoOperator-4B-i1-Q4_K_M.gguf | 2.326 | 4.97 | 9.48582 | 0.03368 | 13.551 | 91.233% | |
| | | LocoOperator-4B-IQ4_NL.gguf | 2.229 | 4.76 | 9.60891 | 0.050173 | 11.4324 | 89.708% | |
| | | LocoOperator-4B-i1-Q4_K_S.gguf | 2.22 | 4.74 | 9.47603 | 0.039843 | 10.0551 | 90.557% | |
| | | LocoOperator-4B-Q4_K_S.gguf | 2.22 | 4.74 | 9.80236 | 0.068821 | 15.209 | 88.513% | |
| | | LocoOperator-4B-i1-IQ4_NL.gguf | 2.218 | 4.74 | 9.50223 | 0.039414 | 8.18964 | 90.573% | |
| | | LocoOperator-4B-i1-Q4_0.gguf | 2.213 | 4.73 | 9.79026 | 0.063915 | 12.6928 | 88.737% | |
| | | LocoOperator-4B-Q4_0.gguf | 2.207 | 4.71 | 9.86629 | 0.074527 | 13.2501 | 87.758% | |
| | | LocoOperator-4B-IQ4_XS.gguf | 2.129 | 4.55 | 9.62193 | 0.051911 | 11.0682 | 89.705% | |
| | | LocoOperator-4B-i1-IQ4_XS.gguf | 2.115 | 4.52 | 9.49687 | 0.040098 | 7.03875 | 90.402% | |
| | | LocoOperator-4B-Q3_K_L.gguf | 2.086 | 4.45 | 10.2476 | 0.121944 | 27.0257 | 84.146% | |
| | | LocoOperator-4B-i1-Q3_K_L.gguf | 2.086 | 4.45 | 9.90811 | 0.090874 | 15.8122 | 86.154% | |
| | | LocoOperator-4B-Q3_K_M.gguf | 1.933 | 4.13 | 10.7021 | 0.15788 | 20.2044 | 82.662% | |
| | | LocoOperator-4B-i1-Q3_K_M.gguf | 1.933 | 4.13 | 9.98057 | 0.102708 | 16.8243 | 85.354% | |
| | | LocoOperator-4B-i1-IQ3_M.gguf | 1.828 | 3.9 | 10.1634 | 0.137347 | 14.6883 | 83.180% | |
| | | LocoOperator-4B-IQ3_M.gguf | 1.828 | 3.9 | 14.2539 | 0.557713 | 19.4397 | 67.631% | |
| | | LocoOperator-4B-IQ3_S.gguf | 1.769 | 3.78 | 15.0624 | 0.619131 | 20.122 | 65.931% | |
| | | LocoOperator-4B-i1-IQ3_S.gguf | 1.769 | 3.78 | 10.1755 | 0.142066 | 17.0028 | 83.139% | |
| | | LocoOperator-4B-i1-Q3_K_S.gguf | 1.757 | 3.75 | 10.8886 | 0.171224 | 28.3373 | 82.133% | |
| | | LocoOperator-4B-Q3_K_S.gguf | 1.757 | 3.75 | 11.5475 | 0.237895 | 30.6868 | 79.412% | |
| | | LocoOperator-4B-i1-IQ3_XS.gguf | 1.69 | 3.61 | 10.3629 | 0.168783 | 14.3358 | 81.928% | |
| | | LocoOperator-4B-i1-Q2_K.gguf | 1.555 | 3.32 | 12.1574 | 0.328652 | 18.6622 | 75.570% | |
| | | LocoOperator-4B-i1-IQ3_XXS.gguf | 1.555 | 3.32 | 11.2795 | 0.263448 | 25.251 | 77.569% | |
| | | LocoOperator-4B-Q2_K.gguf | 1.555 | 3.32 | 17.153 | 0.713596 | 16.3946 | 64.880% | |
| | | LocoOperator-4B-i1-Q2_K_S.gguf | 1.456 | 3.11 | 13.1709 | 0.450125 | 18.3826 | 71.231% | |
| | | LocoOperator-4B-i1-IQ2_M.gguf | 1.409 | 3.01 | 14.0857 | 0.544764 | 18.5618 | 67.933% | |
| | | LocoOperator-4B-i1-IQ2_S.gguf | 1.32 | 2.82 | 15.0717 | 0.621189 | 24.0981 | 65.722% | |
| | | LocoOperator-4B-i1-IQ2_XS.gguf | 1.261 | 2.69 | 16.8277 | 0.750336 | 19.2128 | 63.162% | |
| | | LocoOperator-4B-i1-IQ2_XXS.gguf | 1.161 | 2.48 | 27.5988 | 1.32144 | 14.6807 | 52.522% | |
| | | LocoOperator-4B-i1-IQ1_M.gguf | 1.05 | 2.24 | 49.0978 | 1.9323 | 16.5947 | 44.067% | |
| | | LocoOperator-4B-i1-IQ1_S.gguf | 0.983 | 2.1 | 139.951 | 3.03274 | 16.0947 | 28.387% | |
| |
|
| |
|
| | ------ |
| |
|
| | <div align="center"> |
| | <img src="assets/loco_operator.png" width="55%" alt="LocoOperator" /> |
| | </div> |
| |
|
| | <br> |
| |
|
| | <div align="center"> |
| |
|
| | [](https://huggingface.co/LocoreMind/LocoOperator-4B) |
| | [](https://huggingface.co/LocoreMind/LocoOperator-4B-GGUF) |
| | [](https://locoremind.com/blog/loco-operator) |
| | [](https://github.com/LocoreMind/LocoOperator) |
| | [](https://colab.research.google.com/github/LocoreMind/LocoOperator/blob/main/LocoOperator_4B.ipynb) |
| |
|
| | </div> |
| |
|
| | ## Introduction |
| |
|
| | **LocoOperator-4B** is a 4B-parameter tool-calling agent model trained via knowledge distillation from **Qwen3-Coder-Next** inference traces. It specializes in multi-turn codebase exploration — reading files, searching code, and navigating project structures within a Claude Code-style agent loop. Designed as a local sub agent, it runs via llama.cpp at zero API cost. |
| |
|
| | | | LocoOperator-4B | |
| | |:--|:--| |
| | | **Base Model** | [Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) | |
| | | **Teacher Model** | Qwen3-Coder-Next | |
| | | **Training Method** | Full-parameter SFT (distillation) | |
| | | **Training Data** | 170,356 multi-turn conversation samples | |
| | | **Max Sequence Length** | 16,384 tokens | |
| | | **Training Hardware** | 4x NVIDIA H200 141GB SXM5 | |
| | | **Training Time** | ~25 hours | |
| | | **Framework** | MS-SWIFT | |
| |
|
| | ## Key Features |
| |
|
| | - **Tool-Calling Agent**: Generates structured `<tool_call>` JSON for Read, Grep, Glob, Bash, Write, Edit, and Task (subagent delegation) |
| | - **100% JSON Validity**: Every tool call is valid JSON with all required arguments — outperforming the teacher model (87.6%) |
| | - **Local Deployment**: GGUF quantized, runs on Mac Studio via llama.cpp at zero API cost |
| | - **Lightweight Explorer**: 4B parameters, optimized for fast codebase search and navigation |
| | - **Multi-Turn**: Handles conversation depths of 3–33 messages with consistent tool-calling behavior |
| |
|
| | ## Performance |
| |
|
| | Evaluated on 65 multi-turn conversation samples from diverse open-source projects (scipy, fastapi, arrow, attrs, gevent, gunicorn, etc.), with labels generated by Qwen3-Coder-Next. |
| |
|
| | ### Core Metrics |
| |
|
| | | Metric | Score | |
| | |:-------|:-----:| |
| | | **Tool Call Presence Alignment** | **100%** (65/65) | |
| | | **First Tool Type Match** | **65.6%** (40/61) | |
| | | **JSON Validity** | **100%** (76/76) | |
| | | **Argument Syntax Correctness** | **100%** (76/76) | |
| |
|
| | The model perfectly learned *when* to use tools vs. when to respond with text (100% presence alignment). Tool type mismatches are between semantically similar tools (e.g. Grep vs Read) — different but often valid strategies. |
| |
|
| | ### Tool Distribution Comparison |
| |
|
| | <div align="center"> |
| | <img src="assets/tool_distribution.png" width="80%" alt="Tool Distribution Comparison" /> |
| | </div> |
| |
|
| | ### JSON & Argument Syntax Correctness |
| |
|
| | | Model | JSON Valid | Argument Syntax Valid | |
| | |:------|:---------:|:--------------------:| |
| | | **LocoOperator-4B** | 76/76 (100%) | 76/76 (100%) | |
| | | Qwen3-Coder-Next (teacher) | 89/89 (100%) | 78/89 (87.6%) | |
| |
|
| | > LocoOperator-4B achieves perfect structured output. The teacher model has 11 tool calls with missing required arguments (empty `arguments: {}`). |
| |
|
| | ## Quick Start |
| |
|
| | ```python |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | |
| | model_name = "LocoreMind/LocoOperator-4B" |
| | |
| | # load the tokenizer and the model |
| | tokenizer = AutoTokenizer.from_pretrained(model_name) |
| | model = AutoModelForCausalLM.from_pretrained( |
| | model_name, |
| | torch_dtype="auto", |
| | device_map="auto" |
| | ) |
| | |
| | # prepare the messages |
| | messages = [ |
| | { |
| | "role": "system", |
| | "content": "You are a read-only codebase search specialist.\n\nCRITICAL CONSTRAINTS:\n1. STRICTLY READ-ONLY: You cannot create, edit, delete, move files, or run any state-changing commands. Use tools/bash ONLY for reading (e.g., ls, find, cat, grep).\n2. EFFICIENCY: Spawn multiple parallel tool calls for faster searching.\n3. OUTPUT RULES: \n - ALWAYS use absolute file paths.\n - STRICTLY NO EMOJIS in your response.\n - Output your final report directly. Do not use colons before tool calls.\n\nENV: Working directory is /Users/developer/workspace/code-analyzer (macOS, zsh)." |
| | }, |
| | { |
| | "role": "user", |
| | "content": "Analyze the Black codebase at `/Users/developer/workspace/code-analyzer/projects/black`.\nFind and explain:\n1. How Black discovers config files.\n2. The exact search order for config files.\n3. Supported config file formats.\n4. Where this configuration discovery logic lives in the codebase.\n\nReturn a comprehensive answer with relevant code snippets and absolute file paths." |
| | } |
| | ] |
| | |
| | # prepare the model input |
| | text = tokenizer.apply_chat_template( |
| | messages, |
| | tokenize=False, |
| | add_generation_prompt=True, |
| | ) |
| | model_inputs = tokenizer([text], return_tensors="pt").to(model.device) |
| | |
| | # conduct text completion |
| | generated_ids = model.generate( |
| | **model_inputs, |
| | max_new_tokens=512, |
| | ) |
| | output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() |
| | |
| | content = tokenizer.decode(output_ids, skip_special_tokens=True) |
| | print(content) |
| | ``` |
| |
|
| | ## Local Deployment |
| |
|
| | For GGUF quantized deployment with llama.cpp, hybrid proxy routing, and batch analysis pipelines, refer to our [GitHub repository](https://github.com/LocoreMind/LocoOperator). |
| |
|
| | ## Training Details |
| |
|
| | | Parameter | Value | |
| | |:----------|:------| |
| | | Base model | Qwen3-4B-Instruct-2507 | |
| | | Teacher model | Qwen3-Coder-Next | |
| | | Method | Full-parameter SFT | |
| | | Training data | 170,356 samples | |
| | | Hardware | 4x NVIDIA H200 141GB SXM5 | |
| | | Parallelism | DDP (no DeepSpeed) | |
| | | Precision | BF16 | |
| | | Epochs | 1 | |
| | | Batch size | 2/GPU, gradient accumulation 4 (effective batch 32) | |
| | | Learning rate | 2e-5, warmup ratio 0.03 | |
| | | Max sequence length | 16,384 tokens | |
| | | Template | qwen3_nothinking | |
| | | Framework | MS-SWIFT | |
| | | Training time | ~25 hours | |
| | | Checkpoint | Step 2524 | |
| | |
| | ## Known Limitations |
| | |
| | - First-tool-type match is 65.6% — the model sometimes picks a different (but not necessarily wrong) tool than the teacher |
| | - Tends to under-generate parallel tool calls compared to the teacher (76 vs 89 total calls across 65 samples) |
| | - Preference for Bash over Read may indicate the model defaults to shell commands where file reads would be more appropriate |
| | - Evaluated on 65 samples only; larger-scale evaluation needed |
| | |
| | ## License |
| | |
| | MIT |
| | |
| | ## Acknowledgments |
| | |
| | - [Qwen Team](https://huggingface.co/Qwen) for the Qwen3-4B-Instruct-2507 base model |
| | - [MS-SWIFT](https://github.com/modelscope/ms-swift) for the training framework |
| | - [llama.cpp](https://github.com/ggerganov/llama.cpp) for efficient local inference |
| | - [Anthropic](https://www.anthropic.com/) for the Claude Code agent loop design that inspired this work |
