How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
# Run inference directly in the terminal:
llama cli -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
# Run inference directly in the terminal:
llama cli -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
# Run inference directly in the terminal:
./build/bin/llama cli -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Use Docker
docker model run hf.co/ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0
Quick Links

tiny-lev-for-testing-GGUF

This is a modified, damaged copy of interfaze-ai/lev, made only to test software. Its output is gibberish. Do not use it for anything else.

GGUF of ggml-org-dev/tiny-lev-for-testing merged into its base model ggml-org-dev/tiny-Qwen3.5-4B-for-testing, converted with convert_hf_to_gguf.py of llama.cpp (F16, then quantized to Q4_0 with llama-quantize, token embedding included). Used by the llama.cpp server tests of the lev decision type.

The vocabulary of the GGUF is cut from 248320 to 3348 tokens (the HF slice keeps the full tokenizer): the special tokens, all single characters, the BPE path of the decision label words (A..Z, AA..ZZ, digits, yes/no) and the first 2000 merges are kept, the token ids are renumbered and the rows of the token embedding are sliced to match. Text is still fully encodable, but into more tokens than with the original tokenizer.

llama-server -hf ggml-org-dev/tiny-lev-for-testing-GGUF:Q4_0

License

Derived from lev by Interfaze AI, under the Apache License 2.0 (as is the base model Qwen3.5-4B).

Downloads last month
-
GGUF
Model size
510k params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ggml-org-dev/tiny-lev-for-testing-GGUF