Instructions to use jun-ai/Rusa-1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use jun-ai/Rusa-1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="jun-ai/Rusa-1")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("jun-ai/Rusa-1") model = AutoModelForCausalLM.from_pretrained("jun-ai/Rusa-1", device_map="auto") - Transformers.js
How to use jun-ai/Rusa-1 with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('text-generation', 'jun-ai/Rusa-1'); - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use jun-ai/Rusa-1 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf jun-ai/Rusa-1:Q4_K_M # Run inference directly in the terminal: llama cli -hf jun-ai/Rusa-1:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf jun-ai/Rusa-1:Q4_K_M # Run inference directly in the terminal: llama cli -hf jun-ai/Rusa-1:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf jun-ai/Rusa-1:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf jun-ai/Rusa-1:Q4_K_M
Use Docker
docker model run hf.co/jun-ai/Rusa-1:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use jun-ai/Rusa-1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "jun-ai/Rusa-1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jun-ai/Rusa-1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/jun-ai/Rusa-1:Q4_K_M
- SGLang
How to use jun-ai/Rusa-1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "jun-ai/Rusa-1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jun-ai/Rusa-1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "jun-ai/Rusa-1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jun-ai/Rusa-1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Ollama
How to use jun-ai/Rusa-1 with Ollama:
ollama run hf.co/jun-ai/Rusa-1:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use jun-ai/Rusa-1 with Docker Model Runner:
docker model run hf.co/jun-ai/Rusa-1:Q4_K_M
- Lemonade
How to use jun-ai/Rusa-1 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull jun-ai/Rusa-1:Q4_K_M
Run and chat with the model
lemonade run user.Rusa-1-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Rusa-1
Rusa-1 is a 350M-parameter model that rewrites AI-sounding text so it reads like a person wrote it, while keeping every fact, number, name and quote intact.
It is small enough to run on a CPU, on a phone, or fully in the browser.
- Base model: LiquidAI/LFM2-350M
- Training: distilled from a 12B rewriting model, then preference-tuned on its own samples, with a 12B teacher ranking them and a fact checker filtering them
- Language: English
V1.1 files
The standard V1.1 GGUFs are in gguf/. The experimental ternary builds are available separately. Rusa-1-Small
| Format | Path | Size | Use with |
|---|---|---|---|
| Safetensors (BF16) | model.safetensors |
~710 MB | π€ Transformers, vLLM |
| GGUF | gguf/Rusa-{F16,Q8_0,Q6_K,Q5_K_M,Q4_K_M}.gguf |
Q4_K_M 229.31 MB, Q5_K_M 260.37 MB, Q6_K 293.38 MB, Q8_0 379.21 MB, F16 711.48 MB | llama.cpp, LM Studio, Ollama |
| ONNX | onnx/model{,_fp16,_quantized,_q4,_q4f16}.onnx |
q4f16 248 MB, q4 289 MB, q8 496 MB, fp16 692 MB, fp32 1.35 GB | Transformers.js v4 (browser), ONNX Runtime |
Q8_0 is the recommended GGUF. At 350M parameters, 4-bit quantization costs noticeably more quality than it does on large models.
Prompt format
Rusa uses a plain-text prompt, not a chat template. Put the draft between the instruction and the ### Rewritten: marker. The model stops by itself with <|im_end|>.
Rewrite the text below so it reads like a person wrote it, not a language model.
Reorganize it as you see fit. Vary sentence length on purpose. Cut hedging,
throat-clearing, and any sentence that only announces what comes next.
Prefer the concrete word over the abstract one. It is fine to sound uneven.
Every fact, number, unit, date, name and quotation must survive unchanged.
{your text}
### Rewritten:
Recommended sampling: temperature=1.0, top_p=0.95, top_k=0 (disabled), no repetition penalty. Sampling a few rewrites and keeping the best one works well. With 8 samples, at least one keeps every fact on 89% of the test drafts.
Usage
Transformers
from transformers import AutoTokenizer, AutoModelForCausalLM
INSTR = """Rewrite the text below so it reads like a person wrote it, not a language model.
Reorganize it as you see fit. Vary sentence length on purpose. Cut hedging,
throat-clearing, and any sentence that only announces what comes next.
Prefer the concrete word over the abstract one. It is fine to sound uneven.
Every fact, number, unit, date, name and quotation must survive unchanged."""
repo = "jun-ai/Rusa-1"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto")
draft = "In today's fast-paced world, effective time management is more important than ever."
inputs = tok(f"{INSTR}\n\n{draft}\n\n### Rewritten:\n\n", return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=1024, do_sample=True, temperature=1.0, top_p=0.95, top_k=0)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
llama.cpp
llama-cli -hf jun-ai/Rusa-1:Q8_0 --temp 1.0 --top-p 0.95 --top-k 0 -f prompt.txt
prompt.txt holds the full prompt shown above.
Transformers.js (browser)
import { pipeline } from "@huggingface/transformers";
const rewrite = await pipeline("text-generation", "jun-ai/Rusa-1", { dtype: "q4f16", device: "webgpu" });
const prompt = `${INSTR}\n\n${draft}\n\n### Rewritten:\n\n`;
const [out] = await rewrite(prompt, {
max_new_tokens: 1024, do_sample: true, temperature: 1.0, top_p: 0.95, top_k: 0, return_full_text: false,
});
console.log(out.generated_text);
Use dtype: "q8" or "fp16" for higher quality, or device: "wasm" where WebGPU is not available.
Evaluation
The test set holds 209 held-out AI-written drafts across essays, emails, reviews, forum posts and how-tos. Each draft got one sampled rewrite. AI likelihood comes from the desklib/ai-text-detector-v1.01 classifier (lower is better). A rewrite counts as flagged when its score is above 0.5. A rewrite "keeps the facts" when every number, date, name and quote from the draft survives.
| Model | Size | Mean AI score β | Flagged as AI β | Keeps all facts β | Keeps all facts, best of 8 β |
|---|---|---|---|---|---|
| 12B teacher | 12B | 0.190 | 30 / 209 | 67 / 209 | 146 / 209 |
| Rusa | 350M | 0.236 | 46 / 209 | 83 / 209 | 185 / 209 |
Rusa is 35Γ smaller than its teacher and keeps facts more reliably. Its detector score is slightly higher than the teacher's.
Limitations
- English only.
- Fact keeping is good, not perfect. Check numbers, names and quotes before you publish a rewrite.
- Long inputs (more than ~450 words) are outside the training range. Split long documents into paragraphs.
- Do not use Rusa to misrepresent authorship where that is prohibited, such as in academic work.
License
Rusa inherits the LFM Open License v1.0 from LFM2-350M.
- Downloads last month
- 129
