Instructions to use AJ-Gonzalez/rooster-coder-rwkv7-1.5b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use AJ-Gonzalez/rooster-coder-rwkv7-1.5b with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("RWKV/RWKV7-G1k-1.5B-20260930") model = PeftModel.from_pretrained(base_model, "AJ-Gonzalez/rooster-coder-rwkv7-1.5b") - RWKV
How to use AJ-Gonzalez/rooster-coder-rwkv7-1.5b with RWKV:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- MLX
How to use AJ-Gonzalez/rooster-coder-rwkv7-1.5b with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download AJ-Gonzalez/rooster-coder-rwkv7-1.5b --local-dir rooster-coder-rwkv7-1.5b
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AJ-Gonzalez/rooster-coder-rwkv7-1.5b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf AJ-Gonzalez/rooster-coder-rwkv7-1.5b:Q4_K_M # Run inference directly in the terminal: llama cli -hf AJ-Gonzalez/rooster-coder-rwkv7-1.5b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf AJ-Gonzalez/rooster-coder-rwkv7-1.5b:Q4_K_M # Run inference directly in the terminal: llama cli -hf AJ-Gonzalez/rooster-coder-rwkv7-1.5b:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf AJ-Gonzalez/rooster-coder-rwkv7-1.5b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf AJ-Gonzalez/rooster-coder-rwkv7-1.5b:Q4_K_M
Use Docker
docker model run hf.co/AJ-Gonzalez/rooster-coder-rwkv7-1.5b:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use AJ-Gonzalez/rooster-coder-rwkv7-1.5b with Ollama:
ollama run hf.co/AJ-Gonzalez/rooster-coder-rwkv7-1.5b:Q4_K_M
- Unsloth Desktop
- Pi
How to use AJ-Gonzalez/rooster-coder-rwkv7-1.5b with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "AJ-Gonzalez/rooster-coder-rwkv7-1.5b"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "AJ-Gonzalez/rooster-coder-rwkv7-1.5b" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use AJ-Gonzalez/rooster-coder-rwkv7-1.5b with Docker Model Runner:
docker model run hf.co/AJ-Gonzalez/rooster-coder-rwkv7-1.5b:Q4_K_M
- Lemonade
How to use AJ-Gonzalez/rooster-coder-rwkv7-1.5b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AJ-Gonzalez/rooster-coder-rwkv7-1.5b:Q4_K_M
Run and chat with the model
lemonade run user.rooster-coder-rwkv7-1.5b-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use AJ-Gonzalez/rooster-coder-rwkv7-1.5b with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "AJ-Gonzalez/rooster-coder-rwkv7-1.5b"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default AJ-Gonzalez/rooster-coder-rwkv7-1.5b
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use AJ-Gonzalez/rooster-coder-rwkv7-1.5b with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "AJ-Gonzalez/rooster-coder-rwkv7-1.5b"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "AJ-Gonzalez/rooster-coder-rwkv7-1.5b" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
rooster-coder-rwkv7-1.5b
LoRA adapter for RWKV/RWKV7-G1k-1.5B-20260930
(Apache-2.0), SFT on the coder profile of the Qwen4 Max reasoning corpus.
This repo ships the adapter and both merged deployment formats:
| path | what | size |
|---|---|---|
adapter_model.safetensors + adapter_config.json |
PEFT LoRA adapter (needs the base) | 56.7 MB |
gguf/rooster-coder-rwkv7-1.5b-Q6_K.gguf |
merged, llama.cpp (rwkv7 arch) |
1.34 GB |
gguf/rooster-coder-rwkv7-1.5b-Q4_K_M.gguf |
merged, llama.cpp (rwkv7 arch) |
990 MB |
mlx-4bit/ |
merged, MLX 4-bit (4.972 bpw) | 907 MB |
All four are the same weights: the adapter applied to the Apache-2.0 base, not a new pretrain.
Data
70 rows (1_code 32, 2_cs 29, 4_stats 9) from
AJ-Gonzalez/qwen4-max-reasoning-250x,
traces generated by qwen/qwen4-max-20261115 via OpenRouter. Each assistant reasoning is stored
as <think + reasoning + </think + content, which is the checkpoint's own template form
(verified against chat_template.jinja).
Rendered with the RWKV template, no generation prompt, <|endoftext|> appended per conversation,
chunked at 2048 tokens with 256 overlap:
| split | rows | chunks | tokens |
|---|---|---|---|
| train | 63 | 356 | 667,970 |
| valid | 7 | 46 | 88,854 |
Held-out corpus ids: 10, 19, 74, 76, 137, 169, 210. Dataset:
aliciajgonzalez/rooster-coder-rwkv7-sft-data.
Training
Plain LoRA (not QLoRA): r=16, alpha=32, dropout 0.05, task_type=CAUSAL_LM, targets
receptance, key, value, output. That name match also hits the FFN key/value
projections, so the adapter is 14,155,776 parameters (0.9183%). 3 epochs, batch 1 x grad-accum 8,
lr 1e-4, warmup 10, AdamW, gradient checkpointing, max sequence 2048.
fp32, deliberately. The checkpoint is bf16-native and fp16 training produced NaN logits and
grad_norm: nan from the first step on a T4; T4 has no bf16, so the run is fp32. Peak 11.43 GiB,
2.46 h for 135 optimizer steps on one Tesla T4 (Kaggle).
Benchmarks
Same harness for both rows: one T4, fp32, RWKV template with thinking=True, greedy, 768 new
tokens, evalplus 0.3.1 sanitize + evaluate. NLL is token-level cross-entropy; the held-out chunks
are the 46 validation chunks (88,677 tokens) and SWE20 is the same 20 SWE-bench statements used in
training (15,427 tokens).
| model | HumanEval | HumanEval+ | held-out NLL | SWE20 NLL |
|---|---|---|---|---|
base RWKV7-G1k-1.5B |
0.341 | 0.329 | 2.2417 | 1.6097 |
| + this adapter | 0.201 | 0.195 | 2.1179 | 1.7227 |
The adapter improves in-distribution NLL (2.2417 -> 2.1179) and degrades HumanEval+ by 13.4 points (0.329 -> 0.195); SWE20 NLL is also worse. 70 rows at lr 1e-4 for 3 epochs overfit the trace style and cost general code ability. Published as a measured negative result, not as a recommended coder model.
Training diagnostics: train loss 1.6735, eval loss 1.9399 (46 chunks), 135 optimizer steps.
Usage
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "AJ-Gonzalez/rooster-coder-rwkv7-1.5b"
base = AutoModelForCausalLM.from_pretrained(
"RWKV/RWKV7-G1k-1.5B-20260930", trust_remote_code=True, dtype=torch.float32)
model = PeftModel.from_pretrained(base, repo, trust_remote_code=True)
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
text = tok.apply_chat_template(
[{"role": "user", "content": "Write a Python LRU cache."}],
tokenize=False, add_generation_prompt=True, thinking=True)
ids = torch.tensor([tok.encode(text)])
out = model.generate(ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:]))
The tokenizer, chat template and remote-code files (configuration_rwkv7.py,
modeling_rwkv7.py, tokenization_rwkv7.py) are shipped beside the adapter so it loads without
the base repository; the weights still come from the base id in adapter_config.json.
Caveats
- Base checkpoint, not instruction-tuned; the adapter teaches the coder trace style, not general chat.
- One corpus, 70 rows. Expect the style and not much else.
- fp16 inference of this architecture is unreliable (that is why training is fp32); use fp32 in
PyTorch. The existing
RWKV7-G1k-1.5BGGUFs are the base model and do not contain this adapter. - The reasoning corpus prompts derive from TeichAI/MiMo; the traces are Qwen4 Max. Upstream terms apply.
Merged artefacts
The adapter is also shipped fused into the base, for inference without PEFT (files listed at the top).
llama-cli -m gguf/rooster-coder-rwkv7-1.5b-Q6_K.gguf \
-p 'User: Write a Python LRU cache.\n\nAssistant: <think' -n 256 --temp 0
from mlx_lm import load, generate
model, tok = load("AJ-Gonzalez/rooster-coder-rwkv7-1.5b/mlx-4bit")
print(generate(model, tok, prompt="User: Write a Python LRU cache.\n\nAssistant: <think",
max_tokens=256, temp=0.0))
Measured on the M1 Air (Metal, greedy):
| artefact | prompt | generation | peak |
|---|---|---|---|
| Q6_K GGUF | 173.6 t/s | 32.0 t/s | โ |
| Q4_K_M GGUF | 173.8 t/s | 36.1 t/s | โ |
| MLX 4-bit | 19.6 t/s | 46.0 t/s | 1.076 GB |
Q6_K and Q4_K_M load and answer coherently; MLX was checked on the same prompt. These are smoke results, not benchmarks.
How the merge was done: a streaming LoRA merge (scale 2.0) into the bf16 base, then
convert_hf_to_gguf.py from llama.cpp master. The GGUF has the same 678 tensors and shapes as the
published base GGUF: att.ln1 -> attn_norm, att.ln2 -> attn_norm_2 (llama.cpp uses
attn_norm_2 as the FFN norm), att.ln_x -> time_mix_ln, the six att.x_* vectors stack into
time_mix_lerp_fused, and the vocabulary is the same 65,536-token list. MLX maps att.ln_x to
g_norm after reshaping to (heads, head_dim) and transposes the low-rank gate matrices, since
the HF form applies them as x @ W.
- Downloads last month
- 91
Model tree for AJ-Gonzalez/rooster-coder-rwkv7-1.5b
Base model
RWKV/RWKV7-G1k-1.5B-20260930