How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jun-ai/Rusa-1-Small:TQ1_0
# Run inference directly in the terminal:
llama cli -hf jun-ai/Rusa-1-Small:TQ1_0
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jun-ai/Rusa-1-Small:TQ1_0
# Run inference directly in the terminal:
llama cli -hf jun-ai/Rusa-1-Small:TQ1_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf jun-ai/Rusa-1-Small:TQ1_0
# Run inference directly in the terminal:
./build/bin/llama cli -hf jun-ai/Rusa-1-Small:TQ1_0
Use Docker
docker model run hf.co/jun-ai/Rusa-1-Small:TQ1_0
Quick Links

Rusa-1-Small. Better Human.

Rusa-1-Small

Experimental CPU GGUFs for rewriting with a 350M-parameter model under 100 MB. These variants use ternary weights in most projections and Q2_K in selected tensors. The standard model is Rusa-1.

Build Size Download
Q6_K-distilled mixed ternary with learned block scales 99.03 MB TQ1_0-QAT
Original fitted ternary 94.12 MB TQ1_0-Fit-Down

Evaluation

The 99.03 MB build was selected after two consecutive rounds failed to improve the combined automated development score by two percentage points.

Automated rewrite check results for the early ternary, selected ternary, and Q6_K builds

On eight additional drafts, a blinded manual review gave the selected build credit for 10 of 24 factual relationships, compared with 19 of 24 for Q6_K. The selected build produced zero fully correct rewrites on those drafts. The automated checks missed some factual changes. For reliable rewriting, use the standard Rusa-1 GGUFs.

Inference

Use a CPU llama.cpp build with LFM2 and TQ1_0 support. Rusa uses a plain-text prompt. Put the draft after the rewrite instructions and end the prompt with two newlines, ### Rewritten:, and two more newlines. See the Rusa-1 prompt format for the full instructions.

llama-cli -m gguf/Rusa-1-Small-TQ1_0-QAT.gguf -ngl 0 --temp 0 -f prompt.txt
Downloads last month
105
GGUF
Model size
0.4B params
Architecture
lfm2
Hardware compatibility
Log In to add your hardware

1-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for jun-ai/Rusa-1-Small

Quantized
jun-ai/Rusa-1
Quantized
(1)
this model