Instructions to use ggml-org-dev/tiny-lev-for-testing-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use ggml-org-dev/tiny-lev-for-testing-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0 # Run inference directly in the terminal: llama cli -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0 # Run inference directly in the terminal: llama cli -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Use Docker
docker model run hf.co/ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
- Llama new
- LM Studio
- Jan
- Ollama
How to use ggml-org-dev/tiny-lev-for-testing-GGUF with Ollama:
ollama run hf.co/ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
- Unsloth Desktop
- Pi
How to use ggml-org-dev/tiny-lev-for-testing-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use ggml-org-dev/tiny-lev-for-testing-GGUF with Docker Model Runner:
docker model run hf.co/ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
- Lemonade
How to use ggml-org-dev/tiny-lev-for-testing-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Run and chat with the model
lemonade run user.tiny-lev-for-testing-GGUF-Q4_0
List all available models
lemonade list
- Hermes Agent
How to use ggml-org-dev/tiny-lev-for-testing-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use ggml-org-dev/tiny-lev-for-testing-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download README.md from ggml-org-dev/tiny-lev-for-testing-GGUF: direct link, hf CLI and curl.
- Browser
- Download file 1.37 kB
-
https://huggingface.co/ggml-org-dev/tiny-lev-for-testing-GGUF/resolve/main/README.md
- Command line
-
hf download hf://ggml-org-dev/tiny-lev-for-testing-GGUF/README.md
-
curl -L -o README.md https://huggingface.co/ggml-org-dev/tiny-lev-for-testing-GGUF/resolve/main/README.md
license: apache-2.0
base_model: ggml-org-dev/tiny-lev-for-testing
tags:
- gguf
- testing
tiny-lev-for-testing-GGUF
This is a modified, damaged copy of interfaze-ai/lev, made only to test software. Its output is gibberish. Do not use it for anything else.
GGUF of ggml-org-dev/tiny-lev-for-testing merged into its base model ggml-org-dev/tiny-Qwen3.5-4B-for-testing, converted with convert_hf_to_gguf.py of llama.cpp (F16, then quantized to Q4_0 with llama-quantize, token embedding included). Used by the llama.cpp server tests of the lev decision type.
The vocabulary of the GGUF is cut from 248320 to 3348 tokens (the HF slice keeps the full tokenizer): the special tokens, all single characters, the BPE path of the decision label words (A..Z, AA..ZZ, digits, yes/no) and the first 2000 merges are kept, the token ids are renumbered and the rows of the token embedding are sliced to match. Text is still fully encodable, but into more tokens than with the original tokenizer.
llama-server -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
License
Derived from lev by Interfaze AI, under the Apache License 2.0 (as is the base model Qwen3.5-4B).