Instructions to use theatechcorporation/amaara-code-20b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use theatechcorporation/amaara-code-20b-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf theatechcorporation/amaara-code-20b-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf theatechcorporation/amaara-code-20b-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf theatechcorporation/amaara-code-20b-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf theatechcorporation/amaara-code-20b-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf theatechcorporation/amaara-code-20b-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf theatechcorporation/amaara-code-20b-GGUF:Q4_K_M
Use Docker
docker model run hf.co/theatechcorporation/amaara-code-20b-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use theatechcorporation/amaara-code-20b-GGUF with Ollama:
ollama run hf.co/theatechcorporation/amaara-code-20b-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use theatechcorporation/amaara-code-20b-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf theatechcorporation/amaara-code-20b-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "theatechcorporation/amaara-code-20b-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use theatechcorporation/amaara-code-20b-GGUF with Docker Model Runner:
docker model run hf.co/theatechcorporation/amaara-code-20b-GGUF:Q4_K_M
- Lemonade
How to use theatechcorporation/amaara-code-20b-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull theatechcorporation/amaara-code-20b-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.amaara-code-20b-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use theatechcorporation/amaara-code-20b-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf theatechcorporation/amaara-code-20b-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default theatechcorporation/amaara-code-20b-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use theatechcorporation/amaara-code-20b-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf theatechcorporation/amaara-code-20b-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "theatechcorporation/amaara-code-20b-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Building your own AI? Join the AI Owner's Club on Skool β a community that owns its models instead of renting them.
Amaara Code 20B (GGUF) β round 1, provisional
Status: NOT YET EVALUATED for agentic capability. This repository holds a working GGUF export of the round-1 SFT adapter. It is published so the artefacts can be inspected and run. It carries no measured coding or agentic benchmark number, and should not be described as having one. The card will be updated with measured results, or the claim withdrawn, once the evaluation runs.
What it is
A LoRA SFT of openai/gpt-oss-20b (21B total parameters, 3.6B active β a mixture-of-experts
model, native MXFP4 weights) on 98,391 harmonised agent traces: 176,689 messages and 96,204 tool
calls from 2,083 real agent heartbeat runs, plus instruction and reasoning data.
The training corpus is 82% bash tool calls, which reflects the environment the traces came
from. It is an agentic coding model in the sense that it emits harmonised tool calls, not that a
benchmark has confirmed it is a good one.
Round-1 training facts (the only measurements that exist)
| steps | 1,200 |
| sequence length | 8,192 |
| LR | 2e-4, cosine |
| LoRA | r=16, Ξ±=16, 402 target modules |
| final train loss | 4.626 |
| loss trajectory | 6.80 β 5.40 β 4.95 β 4.55 β 4.46 |
| wall clock | 2 h 42 m on an RTX 4090 |
| adapter | 740,115,376 bytes, 3,264 tensors |
Loss is a training objective, not a capability measure. An absolute loss of ~4.6 is expected for this tokenizer and does not indicate a defect versus models that report ~0.9.
Quantisation β read this before choosing a file
llama-quantize leaves 289 of 459 tensors at F32 on gpt-oss (norms, router, small tensors),
so these files are larger than the quant name suggests, and Q6_K is the same size as Q8_0 β
that is a property of the format, not corruption.
| file | composition | size | fits 16 GB card? |
|---|---|---|---|
amaara-code-20b-r1-MXFP4_MOE.gguf |
72ΓMXFP4 (experts) + 98ΓQ8_0 + 289ΓF32 | 11.3 GB | yes, ~5 GB headroom |
amaara-code-20b-r1-Q4_K_M.gguf |
121ΓQ5_0 + 25ΓQ8_0 + 24ΓQ4_K + 289ΓF32 | 14.7 GB | marginal, ~1.6 GB |
Prefer the MXFP4_MOE file. It restores the experts to the format they were trained in, which is both more faithful and ~3.4 GB smaller. Measure your own quality delta before assuming it is lossless.
The quantisation target is spelled MXFP4_MOE in llama-quantize (type 38) β plain MXFP4
is rejected with invalid ftype. And convert_hf_to_gguf.py cannot emit it at all (its outtypes
are f32/f16/bf16/q8_0/tq1_0/tq2_0/auto), so the only path is F16 GGUF -> llama-quantize MXFP4_MOE, which needs ~2x the model size in free disk and is why round 1 shipped without it
initially.
Usage
llama-server -m amaara-code-20b-r1-Q4_K_M.gguf -ngl 99 -c 32768 --jinja \
--host 0.0.0.0 -ub 512 -b 1024
gpt-oss uses the harmony format with three channels β analysis, commentary, final. Tool
calls are emitted on commentary as <|start|>assistant to=functions.NAME<|channel|>commentary json<|message|>{args}<|call|>, and the template permits one tool call per message. Decoding
with skip_special_tokens=True alone yields mangled output like finalREADY; parse the channel
markers.
Serving on a CPU-only box is possible but slow: 6.0 tok/s for MXFP4_MOE and 3.2 tok/s for Q4_K_M, measured on 40-48 threads.
Known limitation observed in a smoke test
Asked to list a directory, the model reaches for a list_dir tool. The evaluation harness it will
be measured in exposes only bash. Round 2 targets this, along with the related gap that
only 3.4% of the training corpus demonstrates verification after a file edit.
Provenance and privacy
- Traces were generalised before training: 6 client brands, 5 people, 12 project slugs and local paths were replaced, and 25 banned patterns verified absent.
- No credentials, customer names, or filesystem paths from the source traces are present.
Apache-2.0. Published for inspection; treat every capability claim about it as unverified until this card reports a measurement.
Join the AI Owner's Club
We build these models in the open with a community of people who want to own their AI instead of renting it. If you're training your own models, running them on your own hardware, and refusing to accept that you have to pay per token forever, you'll fit right in.
- Downloads last month
- 453
4-bit
8-bit
Model tree for theatechcorporation/amaara-code-20b-GGUF
Base model
openai/gpt-oss-20b