Instructions to use nootxlm/Llamatron with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use nootxlm/Llamatron with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf nootxlm/Llamatron # Run inference directly in the terminal: llama cli -hf nootxlm/Llamatron
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf nootxlm/Llamatron # Run inference directly in the terminal: llama cli -hf nootxlm/Llamatron
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf nootxlm/Llamatron # Run inference directly in the terminal: ./build/bin/llama cli -hf nootxlm/Llamatron
Use Docker
docker model run hf.co/nootxlm/Llamatron
- Llama new
- LM Studio
- Jan
- vLLM
How to use nootxlm/Llamatron with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "nootxlm/Llamatron" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nootxlm/Llamatron", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/nootxlm/Llamatron
- Ollama
How to use nootxlm/Llamatron with Ollama:
ollama run hf.co/nootxlm/Llamatron
- Unsloth Desktop
- Pi
How to use nootxlm/Llamatron with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nootxlm/Llamatron
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "nootxlm/Llamatron" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use nootxlm/Llamatron with Docker Model Runner:
docker model run hf.co/nootxlm/Llamatron
- Lemonade
How to use nootxlm/Llamatron with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull nootxlm/Llamatron
Run and chat with the model
lemonade run user.Llamatron-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use nootxlm/Llamatron with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nootxlm/Llamatron
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default nootxlm/Llamatron
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use nootxlm/Llamatron with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nootxlm/Llamatron
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "nootxlm/Llamatron" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Llamatron
Llamatron is a quantized copy of NVIDIA's Nemotron-3-Nano-30B-A3B, built for ordinary machines.
Quickstart โ running it in a few minutes
Llamatron is a decision agent: it builds. Hand it a job whose tool is missing and it works the tool out from the shape of the thing itself, then builds it โ that is the behavior the head-to-head further down this page measured, and the worked example below shows it on a small job. It answers questions too; building is the character.
You need three things:
- The file. One file,
nemotron-split3b-imatrix.ggufโ 18,725,978,368 bytes, the only download. Its check value is in "The check that could prove this wrong" below, so you can confirm the bytes you received are the bytes that were measured. - The tool. llama.cpp, recent enough to carry the numeric
--reasoning-budgetsetting โ that arrived in March 2026 (llama.cpp PR #20297); earlier builds take the setting as on or off only. Check yours withllama-cli --help | grep reasoning-budget. On an older build the command below stops at once witherror: invalid argument: --reasoning-budgetโ that message means the tool is older than the setting, and a current llama.cpp is the fix. - A machine like the one on this page. Every number here was measured on a GTX 1650 Max-Q graphics card with 4 GB of its own memory and 16 GB of system memory. The command splits the work:
-ngl 10puts ten of the model's 52 layers on the graphics card, and--cpu-moekeeps the expert part of the model on the processor, which is how the whole thing fits a 4 GB card. With a bigger card, raise-ngland more of the model runs there; with less memory, lower it. The measured numbers below are all at-ngl 10.
Run it from the folder holding the file (or give -m the file's full path):
llama-cli -m nemotron-split3b-imatrix.gguf -ngl 10 --cpu-moe -c 2048 --reasoning-budget 256
That opens a conversation; type, and it answers. The --reasoning-budget 256 setting is the seatbelt: it caps the thinking at 256 tokens so the answer always has room to land. Keep it on.
A worked example โ the tool that was not there
Here is a real run, made on the home machine with the pinned, repeatable settings from "Running it" below. The job is the kind this model is for: the tool for it does not exist, so the model builds it.
The prompt:
A data logger writes its readings in a format no tool on this machine can read: one string like
t=0;v=21.5|t=60;v=21.9|t=120;v=22.4, where t is the second and v is the reading. Build the missing tool: a short Python program that takes such a string as its one command-line argument and prints the largest v and the average of the v values, separated by a space. Give the program only.
What came back โ the thinking first, complete:
We need to output only the program code, short Python program that takes a string argument and prints largest v and average of v values separated by a space. Must be only the program. No extra text. Provide code. Should handle parsing: split by '|', then each part like 't=0;v=21.5', extract v after 'v='. Compute max and average. Print as "max avg". Probably format with default float representation. Provide shebang? Probably just script. Provide code only. Ensure it reads sys.argv[1]. Use split. Provide final print. Let's craft concise code.
Example: import sys; s=sys.argv[1]; values=[float(p.split('v=')[1]) for p in s.split('|')]; print(max(values), sum(values)/len(values))
But need to ensure correct parsing: each part may have 't=...;v=...' maybe order not guaranteed? But given format t=...;v=... So we can split by ';v='? Simpler: split by '|', then for each segment, find 'v=' and take after it until maybe end or ';'. Use split('v=')[1].split(';')[0
You can see the fence at work: the thinking stops at its limit, mid-thought, and the answer lands complete in the room that leaves:
import sys
s = sys.argv[1]
v = [float(p.split('v=')[1].split(';')[0]) for p in s.split('|')]
print(max(v), sum(v) / len(v))
And the built thing checks out. We ran the program: on the string from the prompt it prints 22.4 21.933333333333334 โ the largest reading, then the average. On a second string, t=5;v=10.0|t=9;v=14.0, it prints 14.0 12.0, also right. The tool works.
Everything below is the full record behind those few minutes: what was measured, how it was built, the quality gate, the honest limits, and the checks that could prove any of it wrong.
It already carries the cache repair โ the fix is here: https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/discussions/65
It was built by quantization โ the importance-guided kind, from llama.cpp: first measure which weights the model leans on hardest, then round the rest down carefully, spending the error where the model leans least.
The parts that carry the model's attention and memory stay at 8 bits.
The rest rounds harder.
It is about 3.4 times smaller than the Nemotron Nano 30B.
Measured on our home machine, at the same shape, it answers at 5.7 tokens per second against the Nemotron Nano 30B's 0.6 โ a ratio of 9.5 between the two measured rates.
It finishes its thinking inside the standard 128-token window and answers, opening with the original's own first words.
Before: the repair was proven on the full-size model โ all 63 GB of it, with the cache fix in. The tokens came out identical, and generation took 18.6% less time.
Then: a plain copy was built from that fixed model. 18.7 GB, the straight rounding recipe. It runs โ but inside the standard 128-token window, it is still mid-thought when the window closes.
Finally: Llamatron. The same fixed model, rounded the careful way, with the importance map guiding where the error goes.
On identical held-out text, its quality score matched the plain copy almost to the digit โ and it finishes its thinking inside the window, and answers.
Llamatron leans towards engineering: handed a job whose tool was missing, it worked the tool out from the shape of the thing itself, and built it right.
The cache-fixed Nemotron made the shape up.
Two honest bounds.
It has loaded at the model's full 1,048,576-token context, but a load is not a benchmarked million-token run.
And it runs on the machine, not inside its 4 GiB graphics card โ ten layers ride the card, the expert part stays on the processor.
It is not the smallest this model can be.
A plainer 3-bit copy comes in at 14.7 GB.
Ours trades a little size for thinking that finishes.
And one scope line: this copy is text-only. VLM inference is the vision sibling's ground โ none was run or tested here.
This work began on November 7, 2026.
The whole effort โ the repair, the builds, and the checks โ ran on a team of one, a GTX 1650 Max-Q, and two almost-4 GB laptops.
"Llamatron" is our nickname for the copy โ Llama for the tooling that built it, Nemotron for the model it is a copy of. The file itself is named nemotron-split3b-imatrix.gguf.
What was measured
| Measure | Llamatron | The other side of the measure |
|---|---|---|
| File size | 18.7 GB | The Nemotron Nano 30B in full precision: 63 GB โ this copy is about 3.4 times smaller (the ratio of the two sizes) |
| Answer rate, home machine, same shape | 5.7 tokens per second | Nemotron Nano 30B on the same machine: 0.6 tokens per second โ a ratio of 9.5 between the two measured rates, each taken at the same shape on the same machine |
| Thinking inside a 128-token window | Finishes its thinking and answers, opening with the original's own first words | The cache-fixed Nemotron built first is still mid-thought when the window closes, and gives no answer |
| Answer quality (perplexity, identical held-out text, 16,384 tokens each, same harness) | 4.4819 ยฑ 0.11663 | The plain copy of the same fixed model, also 18.7 GB: 4.4778 ยฑ 0.11706 โ a ratio of 1.0009, inside the measurement's own uncertainty |
| Full context | Loads at the model's full 1,048,576-token context | Bounds on that load are stated under Honest limits |
Llamatron also went head to head with the cache-fixed Nemotron, under a charter written before the runs.
The measured rows tied โ same size, same gate, speeds too close to separate from the session's noise.
The win came on the charter's tiebreak. Handed a tool that did not exist and could not be installed, Llamatron designed it from the file format itself. The cache-fixed Nemotron drifted into an invented layout instead.
That is a written-response probe. It measures what a copy knows and chooses, not executed tool use โ and it is reported as such.
How it was built
First, the base checkpoint was patched โ the cache repair linked above โ and converted to a full-precision GGUF.
Then an importance matrix was measured over a 71,157-byte calibration text of plain English prose, in 38 chunks. That gives 185 importance entries, and they are recorded in this file's own metadata (
quantize.imatrix.entries_count,quantize.imatrix.chunks_count).The rounding recipe is the cache-fixed Nemotron's exact recipe โ the parts that carry the model's attention and memory held at 8 bits, the rest rounded harder โ with the importance matrix as the only added input.
The tensor census of the two files is identical: 72 tensors at q8_0, 46 at q5_0, 46 at iq4_nl. Importance guidance is the only variable between the copies.
The quality gate
The one unmeasured way this copy could have been quietly bad was damage from the rounding itself. So that was measured before this page went up.
Both copies were scored on identical held-out text โ 16 chunks of 1,024 tokens, the same harness flags for each.
Llamatron scored 4.4819. The cache-fixed Nemotron scored 4.4778. The ratio is 1.0009.
The lines were registered before either number existed: clear at 1.05 or under, and the page dies above 1.10.
On this corpus, the importance-guided build carries no measurable rounding damage.
The check that could prove this wrong
- Run
llama-perplexityon both this file and the cache-fixed Nemotron over the same held-out text โ 16 chunks of 1,024 tokens, the same flags. If the ratio of the two perplexities comes out above 1.05, the no-damage claim above fails. - Check the file itself: it must be exactly 18,725,978,368 bytes, with sha256
99d564ee23b82f3fc1cc15bb3a450f5c5694df2f3e39814dfb999835dcd4b8b4. Any other bytes are not this copy.
Honest limits
- It is not the smallest copy of this model โ a plainer 3-bit copy, at 14.7 GB, is smaller. This one trades a little size for thinking that finishes inside the standard window.
- Opening the full 1,048,576-token context is a load that landed, not a benchmarked million-token generation.
- The measured run does not hold the copy inside the graphics card's 4 GiB of memory. The harness offloads ten layers to the card and keeps the expert part on the processor. The copy runs on the machine, not inside its card.
- The rates above are one home machine's โ a GTX 1650 Max-Q laptop. In a loaded session both copies read far lower, with no separation between them. Absolute rates travel with the machine and its other work, not with the file.
Running it
With llama.cpp:
llama-cli -m nemotron-split3b-imatrix.gguf -ngl 10 --cpu-moe -c 2048 --reasoning-budget 256
The --reasoning-budget 256 flag is the demo's seatbelt: it fences the model's thinking at 256 tokens, so a final answer always has room to land inside the window. For the pinned 512-token demo form used in our published before/after test, run:
llama-cli -m nemotron-split3b-imatrix.gguf -ngl 10 --cpu-moe -c 2048 -n 512 --temp 0 --seed 1234 --reasoning-budget 256
Raise -c to 1048576 for the full context the model carries.
The file's own metadata records the architecture (nemotron_h_moe), its 52 blocks, 128 experts with 6 used per token, and the 1,048,576-token context.
License and attribution
Llamatron is a derivative work of NVIDIA's Nemotron-3-Nano-30B-A3B, ยฉ NVIDIA Corporation. It is distributed under the NVIDIA Nemotron Open Model License โ the same license as the base model. Licensed by NVIDIA Corporation under the NVIDIA Nemotron Model License.
The license, in brief: it permits creating and distributing derivative works, commercially. It asks three things of a redistribution โ give recipients a copy of the license (the full text is appended below), keep the source's attribution notices, and use NVIDIA's names only to describe where the work comes from.
The license text: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/
The work behind this copy โ the patch, the comparison, and the measurements โ is written up with the fix: https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/discussions/65
Appendix โ NVIDIA Nemotron Open Model License (full text)
NVIDIA Nemotron Open Model License โ Terms and Conditions for Use, Reproduction, and Distribution (v. December 15, 2025). Canonical text: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/
NVIDIA Works released under this License are intended to be used permissively and enable the further development of AI technologies. Subject to the terms of this License, NVIDIA confirms that: Works are commercially usable; you are free to create and distribute Derivative Works; NVIDIA does not claim ownership to any outputs generated using the Works or Derivative Works. By using, reproducing, modifying, distributing, performing or displaying any portion or element of the Works or Derivative Works, or otherwise accepting the terms of this License, you agree to be bound by this License.
Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 10 of this document. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Work" shall mean the work of authorship, including machine learning model, software, checkpoints, learnt weights, algorithms, parameters, configuration files and documentation, made available under the License, as indicated by a copyright notice that is included in or attached to the work. "Derivative Works" shall mean any work, whether in source or object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.
Grant of License. Subject to the terms and conditions of this License, NVIDIA hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in source or object form. If You institute patent or copyright litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or an output from the Work constitutes direct or contributory patent or copyright infringement, then any licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.
Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in source or object form, provided that You meet the following conditions: a. You must give any other recipients of the Work a copy of this License; and b. You must retain, in the source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and c. If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the following attribution notice within a "Notice" text file with such copies and the following statement: "Licensed by NVIDIA Corporation under the NVIDIA Nemotron Model License." You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of NVIDIA, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.
Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, NVIDIA provides the Work on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.
Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall NVIDIA be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work, a Derivative Work or an output from the Work or Derivative Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if NVIDIA has been advised of the possibility of such damages.
Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of NVIDIA, and only if You agree to indemnify, defend, and hold NVIDIA harmless for any liability incurred by, or claims asserted against, NVIDIA by reason of your accepting any such warranty or additional liability. You will indemnify and hold harmless NVIDIA from and against any claim by any third party arising out of or related to your use or distribution of the Works, Derivative Works thereof, or output from the Works or Derivative Works.
Feedback. NVIDIA appreciates your feedback, and You agree that NVIDIA may use it without restriction or compensation to You.
Governing Law. This Agreement will be governed in all respects by the laws of the United States and the laws of the State of Delaware, without regard to conflict of laws principles or the United Nations Convention on Contracts for the International Sale of Goods. The state and federal courts residing in Santa Clara County, California will have exclusive jurisdiction over any dispute or claim arising out of or related to this Agreement, and the parties irrevocably consent to personal jurisdiction and venue in those courts; except that, either party may apply for injunctive remedies or an equivalent type of urgent legal relief in any jurisdiction.
Trade and Compliance. You agree to comply with all applicable export, import, trade and economic sanctions laws and regulations, as amended, including without limitation U.S. Export Administration Regulations and Office of Foreign Assets Control regulations. These laws include restrictions on destinations, end-users and end-use.
END OF TERMS AND CONDITIONS
- Downloads last month
- 301
We're not able to determine the quantization variants.
Model tree for nootxlm/Llamatron
Base model
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
