Instructions to use RefinedNeuro/RefinedToolCallV5-3b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RefinedNeuro/RefinedToolCallV5-3b with HERMES:

# No code snippets available yet for this library.

# To use this model, check the repository files and the library's documentation.

# Want to help? PRs adding snippets are welcome at:
# https://github.com/huggingface/huggingface.js

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use RefinedNeuro/RefinedToolCallV5-3b with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf RefinedNeuro/RefinedToolCallV5-3b:Q6_K
# Run inference directly in the terminal:
llama cli -hf RefinedNeuro/RefinedToolCallV5-3b:Q6_K

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf RefinedNeuro/RefinedToolCallV5-3b:Q6_K
# Run inference directly in the terminal:
llama cli -hf RefinedNeuro/RefinedToolCallV5-3b:Q6_K

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf RefinedNeuro/RefinedToolCallV5-3b:Q6_K
# Run inference directly in the terminal:
./llama-cli -hf RefinedNeuro/RefinedToolCallV5-3b:Q6_K

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf RefinedNeuro/RefinedToolCallV5-3b:Q6_K
# Run inference directly in the terminal:
./build/bin/llama-cli -hf RefinedNeuro/RefinedToolCallV5-3b:Q6_K

Use Docker

docker model run hf.co/RefinedNeuro/RefinedToolCallV5-3b:Q6_K

LM Studio
Jan

vLLM

How to use RefinedNeuro/RefinedToolCallV5-3b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RefinedNeuro/RefinedToolCallV5-3b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RefinedNeuro/RefinedToolCallV5-3b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/RefinedNeuro/RefinedToolCallV5-3b:Q6_K

Ollama
How to use RefinedNeuro/RefinedToolCallV5-3b with Ollama:
```
ollama run hf.co/RefinedNeuro/RefinedToolCallV5-3b:Q6_K
```

Unsloth Studio

How to use RefinedNeuro/RefinedToolCallV5-3b with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for RefinedNeuro/RefinedToolCallV5-3b to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for RefinedNeuro/RefinedToolCallV5-3b to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for RefinedNeuro/RefinedToolCallV5-3b to start chatting

How to use RefinedNeuro/RefinedToolCallV5-3b with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf RefinedNeuro/RefinedToolCallV5-3b:Q6_K

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "RefinedNeuro/RefinedToolCallV5-3b:Q6_K"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use RefinedNeuro/RefinedToolCallV5-3b with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf RefinedNeuro/RefinedToolCallV5-3b:Q6_K

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default RefinedNeuro/RefinedToolCallV5-3b:Q6_K

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use RefinedNeuro/RefinedToolCallV5-3b with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf RefinedNeuro/RefinedToolCallV5-3b:Q6_K

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "RefinedNeuro/RefinedToolCallV5-3b:Q6_K" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use RefinedNeuro/RefinedToolCallV5-3b with Docker Model Runner:
```
docker model run hf.co/RefinedNeuro/RefinedToolCallV5-3b:Q6_K
```

Lemonade

How to use RefinedNeuro/RefinedToolCallV5-3b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull RefinedNeuro/RefinedToolCallV5-3b:Q6_K

Run and chat with the model

lemonade run user.RefinedToolCallV5-3b-Q6_K

List all available models

lemonade list

RefinedToolCallV5-3b / README.md

RefinedNeuro

RefinedToolCall-V5-3B model card

08058b1 verified about 1 month ago

preview code

Raw

History Blame Contribute Delete

3.88 kB

	---
	license: apache-2.0
	base_model: WeiboAI/VibeThinker-3B
	datasets:
	- lambda/hermes-agent-reasoning-traces
	language:
	- en
	pipeline_tag: text-generation
	tags:
	- tool-calling
	- function-calling
	- multi-turn
	- agentic
	- hermes
	- reasoning
	- qwen2
	---

	# 🛠️🧠 RefinedToolCall-V5-3B

	### A 3B model that reasons and calls tools — and actually holds a multi-turn conversation.

	Math-grade reasoning · real function calling · multi-turn agentic · 2.5 GB · runs on your laptop.

	> `ollama run refinedneuro/refinedtoolcallv5-3b`

	---

	## Why it's different

	Most 3B tool-callers nail a single function call and then fall apart the moment the task spans
	several turns. RefinedToolCall-V5 was built specifically to fix that — and the numbers moved on
	every axis at once, not just the one we were targeting.

	- 🔁 Multi-turn agentic that actually works — ~3.7× better at stateful, multi-step
	tool-use (Berkeley Function-Calling Leaderboard `multi_turn`) than where we started.
	- 🛠️ Sharper single-turn calling — 70.7% on BFCL single-turn (held-out), our best ever.
	- 💪 Recovers from tool errors — 0.896 recovery rate; it diagnoses failures instead of
	looping on them.
	- 🧮 Reasoning fully intact — AIME-2024 pass@8 0.933, unchanged by all the tool training.
	- ⚡ Tiny & local — 3B params, 2.5 GB Q6_K, one command on Ollama, no GPU required.
	- 🆓 Apache-2.0 — use it, ship it, fine-tune it.

	---

	## The receipts (all held-out, canary-gated)

	\| capability \| this model \|
	\|---\|---\|
	\| 🔁 Multi-turn agentic (BFCL `multi_turn`, k=3) \| 0.220 avg / 0.298 pass@3 \|
	\| 🛠️ Single-turn function calling (BFCL, held-out) \| 0.707 \|
	\| 🩹 Recovery from tool errors (n=250) \| 0.896 \|
	\| 🧮 Reasoning (AIME-2024 pass@8) \| 0.933 \|

	Every number is the best across five fine-tuning rounds — multi-turn, single-turn, recovery,
	and reasoning all peaked together.

	---

	## How we got here (and why it generalizes)

	We didn't just throw data at it. Five disciplined rounds, each one gated so it could never
	regress reasoning or recovery:

	1. Grounding — stop inventing shell commands; call the actual functions.
	2. Plan + finish — think before calling, and know when the turn is done.
	3. Scale + long context — harder tasks, up to 24k tokens.
	4. On-policy self-improvement (the breakthrough) — the model learns from its own successful
	multi-turn solutions (expert iteration), which broke past the imitation ceiling and sharpened
	single-turn calling and error-recovery as a bonus.

	---

	## Quick start

	Ollama
	```bash
	ollama run refinedneuro/refinedtoolcallv5-3b # latest = Q6_K, 2.5 GB
	```
	> 💡 Use Q6_K or higher for tool-calling — lower quants corrupt the call tokens.

	Format: ChatML + Hermes tools. Each turn the model emits a `<think>` plan → one or more
	`<tool_call>` blocks → a final reply. Recommended: temp 0.6, top_p 0.95, repeat_penalty 1.1.

	---

	## Great for
	✅ Local/offline agentic tool-use prototypes ✅ Multi-step function-calling assistants
	✅ Math & STEM reasoning ✅ Learning how small agentic models are actually built.

	## Be honest with me (research preview)
	⚠️ It's a 3B research preview. Multi-turn is dramatically improved (~3.7×) but not solved —
	very long, open-ended autonomous loops can still write buggy code or mis-plan. A brilliant,
	tiny building block; not yet a drop-in autonomous engineer.

	---

	*Built on [WeiboAI/VibeThinker-3B](https://huggingface.co/WeiboAI/VibeThinker-3B) +
	[lambda/hermes-agent-reasoning-traces](https://huggingface.co/datasets/lambda/hermes-agent-reasoning-traces).
	Trained with distribution-matched RFT + on-policy expert iteration, every checkpoint gated against
	reasoning/recovery canaries. Apache-2.0.*