---
license: mit
base_model: LocoreMind/LocoOperator-4B
tags:
- code
- agent
- tool-calling
- gguf
- llama-cpp
- qwen
---

# LocoOperator-4B-GGUF

This repository contains the **official GGUF quantized versions** of [LocoOperator-4B](https://huggingface.co/LocoreMind/LocoOperator-4B).

**LocoOperator-4B** is a 4B-parameter code exploration agent distilled from **Qwen3-Coder-Next**. It is specifically optimized for local agent loops (like Claude Code style), providing high-speed codebase navigation with **100% JSON tool-calling validity**.

## 🚀 Which file should I choose?

We provide several quantization levels to balance performance and memory usage:

| File Name | Size | Recommendation |
|-----------|------|----------------|
| **LocoOperator-4B.Q8_0.gguf** | 4.28 GB | **Best Accuracy.** Recommended for local agent loops to ensure perfect JSON output. |
| **LocoOperator-4B.Q6_K.gguf** | 3.31 GB | **Great Balance.** Near-lossless logic with a smaller footprint. |
| **LocoOperator-4B.Q4_K_M.gguf**| 2.50 GB | **Standard.** Compatible with almost all local LLM runners (LM Studio, Ollama, etc.). |
| **LocoOperator-4B.IQ4_XS.gguf**| 2.29 GB | **Advanced.** Uses Importance Quantization for better performance at smaller sizes. |

## 🛠 Usage (llama.cpp)

To run this model using `llama-cli` or `llama-server`, we recommend a **context size of at least 50K** to handle multi-turn codebase exploration:

### Simple CLI Chat:
```bash
./llama-cli \
    -m LocoOperator-4B.Q8_0.gguf \
    -c 51200 \
    -p "You are a helpful codebase explorer. Use tools to help the user."
```

### Serve as an OpenAI-compatible API:
```bash
./llama-server \
    -m LocoOperator-4B.Q8_0.gguf \
    --ctx-size 51200 \
    --port 8080
```

## 📋 Model Details
- **Base Model:** Qwen3-4B-Instruct-2507
- **Teacher Model:** Qwen3-Coder-Next
- **Training Method:** Full-parameter SFT (Knowledge Distillation)
- **Primary Use Case:** Codebase exploration (Read, Grep, Glob, Bash, Task)

## 🔗 Links
- **Main Repository:** [LocoreMind/LocoOperator-4B](https://huggingface.co/LocoreMind/LocoOperator-4B)
- **GitHub:** [LocoreMind/LocoOperator](https://github.com/LocoreMind/LocoOperator)
- **Blog:** [locoremind.com/blog/loco-operator](https://locoremind.com/blog/loco-operator)

## 🙏 Acknowledgments
Special thanks to `mradermacher` for the initial quantization work and the `llama.cpp` community.