Instructions to use testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M # Run inference directly in the terminal: llama cli -hf testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M # Run inference directly in the terminal: llama cli -hf testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
Use Docker
docker model run hf.co/testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
- Ollama
How to use testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF with Ollama:
ollama run hf.co/testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
- Unsloth Desktop
- Pi
How to use testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF with Docker Model Runner:
docker model run hf.co/testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
- Lemonade
How to use testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
Run and chat with the model
lemonade run user.Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF-Q5_K_M
List all available models
lemonade list
- Hermes Agent
How to use testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Tool calling broken in Ollama / GGUF: raw JSON returned in content without <tool_call> tags
Hi! Thanks for sharing this C# fine-tune of Qwen2.5-Coder.
I tested Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M in Ollama for agentic workflows in Visual Studio (Copilot / IDE extensions) and noticed that tool calling (tool_calls) does not work properly, even though ollama show reports tools under Capabilities.
When sending a standard OpenAI/Ollama tool-calling payload (e.g., asking to call a read_file tool), the model outputs a raw JSON object inside the plain text "content" field instead of wrapping it in Qwen's expected ... tags:
{
"model": "hf.co/testingusera1/Qwen2.5-Coder-7B-CSharp-QLoRA-GGUF:Q5_K_M",
"message": {
"role": "assistant",
"content": "{\"name\": \"read_file\", \"arguments\": {\"path\": \"ValidationJob.cs\"}}"
},
"done": true
}
Because the tags are missing, Ollama's parser fails to populate the "tool_calls" array, so IDE clients treat the response as plain text and cannot execute the tool.
Hi Alexey,
Thanks for the detailed report — and sorry for the trouble. This is a known limitation of the current weights, here is what's going on:
Root cause. This is a QLoRA adapter fine-tune (not a full fine-tune), built mainly for Russian-language C# coding assistance. Agentic data was only 8.5% of the training mix (7.9k examples from NousResearch/hermes-function-calling-v1). More importantly, the training pipeline used a flat "instruction → response" format without a system role: the ...<-(here should be tools block, but HF consumed it...) block was embedded inside the user turn. Qwen's native tool-calling template (which Ollama uses) expects the tool schemas in the system message instead. So the adapter never really reinforced Qwen's native protocol — it saw tool-call-like data only in a different context, and at this mixture weight it partially washed out what the base model already knew.
The Capabilities: tools flag from ollama show comes from the chat template embedded in the GGUF, not from the model's behavior — so the flag is there, but the model itself doesn't reliably emit the tags.
What I'd suggest for IDE/agentic use:
For tool calling in Ollama / VS extensions, the safest option is the official Qwen/Qwen2.5-Coder-7B-Instruct GGUF. My own benchmarks actually showed this fine-tune is at parity with the base model on MultiPL-E (humaneval-cs / mbpp-cs), so you lose essentially nothing there and get working tool_calls.
This repo's strength is Russian-language C# Q&A and code writing — on a set of 5 Russian coding tasks (bug fixes, writing an app, LINQ) the fine-tune produced compiling answers 5/5 vs 3/5 for the base model, and noticeably more compact answers. If you don't need tool calling, it works fine as a chat model.
p.s.: sorry if i made any mistakes in answer - I don't realy good in English, so used AI to translate your answer and my answer, so chapter of senses may be lost
Спасибо за ответ, можно и по-русски :) Я модельку как раз и скачал для C# и русского языка. Просто первый раз пробую агента настроить в VS, и с этой моделью не получилось, хотя там писалось что туллинг поддерживается. Теперь понял откуда там туллинг в поддержке написан, спасибо за разъяснения.
Вообще, я пытался сохранить умение использовать инструменты, но, похоже, это умение модели "поехало", т.к. сместился акцент модели на написание C# кода и понимание русского языка :/
(ну и, как и написано в карточке - выборка tool use была <10% от всего обучающего корпуса, это умение деградировало на фоне целевых навыков)
Я как-нибудь займусь этим вопросом, но у меня и это обучение заняло ~неделю(не мог нормально использовать ПК, т.к. вся видеопамять и ОЗУ были забиты), поэтому ничего не могу обещать
Ясно, все равно спасибо :)