Instructions to use interfaceinde/qwen35-4b-interface-classifier-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use interfaceinde/qwen35-4b-interface-classifier-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
Use Docker
docker model run hf.co/interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use interfaceinde/qwen35-4b-interface-classifier-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "interfaceinde/qwen35-4b-interface-classifier-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "interfaceinde/qwen35-4b-interface-classifier-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
- Ollama
How to use interfaceinde/qwen35-4b-interface-classifier-GGUF with Ollama:
ollama run hf.co/interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use interfaceinde/qwen35-4b-interface-classifier-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use interfaceinde/qwen35-4b-interface-classifier-GGUF with Docker Model Runner:
docker model run hf.co/interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
- Lemonade
How to use interfaceinde/qwen35-4b-interface-classifier-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.qwen35-4b-interface-classifier-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use interfaceinde/qwen35-4b-interface-classifier-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use interfaceinde/qwen35-4b-interface-classifier-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M# Run inference directly in the terminal:
llama cli -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_MBuild from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M# Run inference directly in the terminal:
./build/bin/llama cli -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_MUse Docker
docker model run hf.co/interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_MINTERFACE Qwen3.5 4B ๋ฉ์ผ ๋ถ๋ฅ ๋ชจ๋ธ
INTERFACE ํ๋ก์ ํธ์ ํ๊ตญ์ด ๋ฉ์ผ 20ํญ๋ชฉ JSON ๋ถ๋ฅ์ฉ ๋ชจ๋ธ์ด๋ค. ์ ์ฒด ํ์ต ์คํ์์ ๊ฐ๋ฐ ๊ฒ์ฆ ํญ๋ชฉ ์ผ์น์จ์ด ๊ฐ์ฅ ๋์๋ checkpoint 500๋ง ๋ฐฐํฌํ๋ค. ์ต์ข ์ ์ฅ checkpoint 650๊ณผ ๋ค๋ฅธ ํ์ต ์ฒดํฌํฌ์ธํธ๋ ํฌํจํ์ง ์๋๋ค.
์ด ๋ชจ๋ธ์ ์ผ๋ฐ ๋ํยท๋ฉ์ผ ์ด์ ์์ฑยทOCR ๋ชจ๋ธ์ ๋์ฒด๋ณธ์ด ์๋๋ค. ํ ์คํธ ์ ์ฉ GGUF์ด๋ฉฐ ์ด๋ฏธ์ง projector๋ฅผ ํฌํจํ์ง ์๋๋ค.
Ollama ์ฌ์ฉ
ollama run hf.co/interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M
ํ์ผ ๋ค์ด๋ก๋ ํ ๋ก์ปฌ ์ด๋ฆ์ผ๋ก ๋ฑ๋กํ ์๋ ์๋ค.
ollama create interface-qwen35-classifier:step500-q4_k_m -f Modelfile
์
๋ ฅ์๋ classifier_instruction.txt์ ์ ์ฒด ์ง์๋ฌธ์ ๋ถ์ด๊ณ ๊ทธ ๋ค์ ๋ฉ์ผ์ input_text๋ฅผ ์ ๊ณตํ๋ค.
์๋ต ํ์์ response_schema.json์ ์ ์ํ๋ค. API ํธ์ถ์์๋ think: false, stream: false,
temperature: 0, ์ถ๋ ฅ ํ๋ 1024๋ฅผ ์ฌ์ฉํ๋ค. 500ํต ๊ฐ๋ฐ ๊ฒ์ฆ๊ณผ ๋์ผํ ์
๋ ฅ ํฌ๋งท์ ๋ณ๋๋ก ์ ์งํด์ผ ํ๋ค.
๋ฐฐํฌ ๋ฒ์ ๊ณผ ์ ๋ฐ์ดํธ
BEST: ํ์ฌ ๋ฐฐํฌํ ๋จ์ผ GGUF ํ์ผ๋ช .manifest.json: ์๋ณธ ๋ชจ๋ธยท์ฒดํฌํฌ์ธํธยท๋ณํ ๋๊ตฌยทGGUF SHA256.- ์ ์
๋ก๋ ํ ๊ธฐ์กด Ollama ๋ชจ๋ธ์ด ์๋ ๊ต์ฒด๋์ง๋ ์๋๋ค. ๋ค์
ollama pullํ๊ฑฐ๋ ๋ณ๋์ ๊ฐฑ์ ์ ์ฐจ๊ฐ ํ์ํ๋ค. - INTERFACE ์ฑ์ ๊ธฐ์กด
OLLAMA_MODEL๊ณผ ์ด์ ์์ฑ ๋ชจ๋ธ์ ์ด๋ฒ ๊ณต๊ฐ ์ ๋ก๋๋ก ๋ณ๊ฒฝํ์ง ์์๋ค.
ํ์ต ์ถ์ฒ
- ์๋ณธ: Qwen/Qwen3.5-4B
- ๊ณ ์ ์๋ณธ ๋ฆฌ๋น์ :
851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a - QLoRA NF4, BF16 ๊ณ์ฐ, LoRA rank 4 / alpha 8, ํ์ต ํ๋ผ๋ฏธํฐ 458,752๊ฐ.
- 1 epoch ๊ณํ, microbatch 1, gradient accumulation 4, AdamW, ํ์ต๋ฅ 0.0001, seed 42.
- ํฉ์ฑ ๋ฉ์ผ train ๋ถํ 4,000ํต ์ค ๊ฒํ ํ๋ณด๋ฅผ ์ ์ธํ 2,868ํต ์ฌ์ฉ.
- ์คํ์ 650๋จ๊ณ์์ 3ํ ์ฐ์ ๋ฏธ๊ฐ์ ์ผ๋ก ์ข ๋ฃ. ๋ฐฐํฌ checkpoint 500์ ๋์ ํ์ต 2,000ํต ์ง์ ์ด๋ค.
- ์๋ณธ BF16 ๊ฐ์ค์น์ checkpoint 500 LoRA๋ฅผ ๋ณํฉํ๊ณ llama.cpp
b10969๋ณํ๊ธฐ๋ก ํ ์คํธ GGUF๋ฅผ ๋ง๋ค์๋ค. - ์ค์น๋ Ollama 0.34.3์ด ๊ณ ์ ํ llama.cpp ๋ฒ์ ๊ณผ ๋ง์ถ ๋ณํ๊ธฐ์ด๋ฉฐ, Q4_K_M ์์ํ ๋๊ตฌ๋
b12556์ด๋ค.
ํ์ต ๋ฐ์ดํฐยท๊ฒ์ฆ ๋ฉ์ผ ์๋ฌธยท์ ๋ต ํ์ผยทoptimizer ์ํยท๋ก์ปฌ ๊ฒฝ๋กยท์ธ์ฆ ์ ๋ณด๋ ์ด ์ ์ฅ์์ ์ ๋ก๋ํ์ง ์๋๋ค.
์ถ์ฒ ์ฒดํฌํฌ์ธํธ์ ๊ฐ๋ฐ ๊ฒ์ฆ ๊ฒฐ๊ณผ
๋ค์ ์์น๋ GGUF ๋ณํ ์ ์ native NF4 QLoRA checkpoint 500์์ ์ธก์ ํ์๋ค. ๋ณํยท์ฌ์์ํ ํ ๋์ผ ์ฑ๋ฅ์ ๋ณด์ฅํ์ง ์์ผ๋ฉฐ, GGUF ์ ์ฒด 500ํต ์ฌํ๊ฐ์ ๋ ๋ฆฝ ํ ์คํธ๋ ๋ฏธ์คํ์ด๋ค.
Ollama 0.34.3์์ ์ด GGUF์ ์ค์ ๋ก๋์ ๊ธฐ์กด ๊ฐ๋ฐ ๊ฒ์ฆ 1๊ฑด ์คํ์ ํ์ธํ์๋ค. ํด๋น 1๊ฑด์ JSON ํ์ฑยท์คํค๋ง ๊ฒ์ฌ๋ฅผ ํต๊ณผํ๊ณ 20ํญ๋ชฉ ์ค 19ํญ๋ชฉ์ด ์ผ์นํ์๋ค. ์ด๋ ๋ก์ปฌ ์คํยท์ถ๋ ฅ ๋์ ํ์ธ์ด๋ฉฐ 500ํต ํ์ง ํ๊ฐ๋ฅผ ๋์ฒดํ์ง ์๋๋ค.
| ์งํ | ๊ฒฐ๊ณผ |
|---|---|
| ๊ฐ๋ฐ ๊ฒ์ฆ ๊ท๋ชจ | 500ํต ร 20ํญ๋ชฉ |
| ํญ๋ชฉ ์๊ฒฉ ์ผ์น | 9,288 / 10,000 ยท 92.88% |
| JSON ํ์ฑ ์ฑ๊ณต | 500 / 500 ยท 100% |
| ์คํค๋ง ํต๊ณผ | 498 / 500 ยท 99.6% |
| ์ ์ฒด 20ํญ๋ชฉ ์ผ์น ๋ฐ ์คํค๋ง ํต๊ณผ | 31 / 500 ยท 6.2% |
ํญ๋ชฉ์ ๋์ผ ๋น์ค์ผ๋ก ์๊ฒฉ ๋น๊ตํ๋ค. action_items๋ 37/500(7.4%), routing์ 408/500(81.6%)์ด๋ค.
๋ฌธ์ฅ ํํยท๋ฐฐ์ด ์์ ์ฐจ์ด๋ ๋ถ์ผ์น๊ฐ ๋ ์ ์๋ค. ๊ฐ์ ๊ฐ๋ฐ ๊ฒ์ฆ ์ธํธ๋ฅผ ๋ฐ๋ณต ์ฌ์ฉํ์ผ๋ฉฐ,
์ ๋ต์ ๋
๋ฆฝ์ ์ธ ์๋ฏธ ๊ฒ์์ ์ค์ ์
๋ฌด ์ฑ๊ณต๋ฅ ์ ์ด๋ฒ ๊ฒฐ๊ณผ๋ก ํ์ธํ์ง ์์๋ค.
์ถ๋ ฅ ํ์ ์ฑ๊ณต์ ๋ด์ฉ ์ ํ๋๋ ํ์
์๋ ์ฒ๋ฆฌ ๊ฐ๋ฅ์ฑ์ผ๋ก ํด์ํ์ง ์๋๋ค.
ํ์ต ๋ด์ญ๊ณผ ์ค๋ช ํ์ผ
- learning_history.md: ๋ฐ์ดํฐ ๊ตฌ์ฑ, ํ์ต ์ค์ , ์์ยท์ข ๋ฃ, ์กฐ๊ธฐ ์ข ๋ฃ์ ์ต๊ณ ์ฒดํฌํฌ์ธํธ ์ ์ , ์ ์ฒด 13ํ ๊ฒ์ฆ ๊ฒฐ๊ณผ.
- training_history.csv: ๋ฉ์ผ ์๋ฌธ ์์ด 650๊ฐ optimizer ๋จ๊ณ์ ์์คยท๊ธฐ์ธ๊ธฐยท์ฒ๋ฆฌ๋ยท์๊ฐ ๊ธฐ๋ก.
- validation_history.json: 50~650๋จ๊ณ์ ๊ฒ์ฆ ์ง๊ณ.
- classifier_instruction.txt: ํ์ต ์ ์ฌ์ฉํ ๊ณตํต ๋ถ๋ฅ ์ง์๋ฌธ.
- response_schema.json: ์ถ๋ ฅ 20ํญ๋ชฉ์ ๊ตฌ์กฐ์ ํ์ฉ๊ฐ.
- manifest.json: ๋ฐฐํฌ ๊ฐ์ค์น ์ถ์ฒยท๋๊ตฌยทSHA256ยท๊ฒ์ฆ ๋ฒ์.
๋ผ์ด์ ์ค
์๋ณธ Qwen3.5-4B์ Apache-2.0 ๋ผ์ด์ ์ค์ ๊ณ ์ง ์กฐ๊ฑด์ ๋ฐ๋ฅธ๋ค. ์๋ณธ LICENSE๋ฅผ ๋๋ดํ์๋ค.
- Downloads last month
- 70
4-bit
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M# Run inference directly in the terminal: llama cli -hf interfaceinde/qwen35-4b-interface-classifier-GGUF:Q4_K_M