README.md

---
license: other
library_name: gguf
base_model: nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
pipeline_tag: text-generation
model_creator: Orionfold LLC
language:
  - en
tags:
  - gguf
  - llama-cpp
  - spark-tested
  - orionfold
  - nvidia
  - nemotron
  - rag
  - grounded-citation
  - advisor
  - "base_model:nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16"
  - trained-with-nemo
license_name: nvidia-nemotron-open-model-license
license_link: "https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/"
---
# Orionfold Advisor GGUF

The Orionfold Advisor model lane: `NVIDIA-Nemotron-3-Nano-4B` fine-tuned for grounded citation discipline, refusal behavior, and workflow routing over a governed retrieval corpus — quantized to `Q4_K_M` (default) and `Q8_0` GGUF and verified end-to-end on the NVIDIA DGX Spark (GB10, 128 GB unified memory). The 2.6 GB `Q4_K_M` lane reproduces the `Q8_0` bench behavior byte-for-byte (same 18/21, refusals 9/9, same three misses) at ~70 tok/s, so it is the recommended pick.

## What this model does

**A governed local AI advisor lane for your enterprise corpus — answers cite exact source ids, refuses when the source isn't there.**

Generic local chat models fail the two behaviors an enterprise corpus assistant actually needs: citing the exact source document an answer came from, and refusing cleanly when the corpus does not contain the answer — instead they paraphrase citations, answer from pretraining memory, or fabricate private-looking state under adversarial pretexts. This model is the serving lane of Orionfold Advisor, a governed local advisor appliance: it was fine-tuned on a teacher-verified corpus to hold citation discipline (exact `source_id` values from the retrieved set, never aliases), a refusal floor that survived novel adversarial pretexts (urgency, roleplay, authority claims, false premises, instructed mis-citation), and `Route:` workflow handoffs — measured behind a frozen, pre-registered out-of-distribution gate before promotion. On that frozen OOD bench the prompt-engineered 30B baseline it replaced scored 8/21 with 3 fabricated private-state rows; this 4B lane scored 18/21 with refusals 9/9 and zero private-state risk.

Use cases:

- grounded Q&A over a retrieval corpus with exact source-id citations
- clean refusals on missing-source and private-state questions
- workflow routing (`Route:`) handoffs inside an advisor harness
- local-first serving with governed frontier escalation

**Who this is for:** Operators running a local advisor over a governed corpus on DGX Spark-class hardware (or any llama.cpp host with ~12 GB to spare), and builders evaluating small fine-tuned lanes against prompt-engineered larger baselines.

## Spark-tested

Every Orionfold quant ships with a measurement quad on the NVIDIA DGX Spark (GB10, 128 GB unified memory): perplexity, sustained `tok/s`, thermal envelope, and **advisor curveball-v0.2, frozen OOD bench (n=21, scored==strict; refusals 9/9, 0 private-state risk)** accuracy. The numbers below are the actual run, not a wishlist.

| Variant | Size | Perplexity (wikitext-2) | tok/s on Spark | advisor curveball-v0.2, frozen OOD bench (n=21, scored==strict; refusals 9/9, 0 private-state risk) |
|---|---|---|---|---|
| Q4_K_M | 2.6 GB | — | 70.0 | 85.7% |
| Q8_0 | 4.0 GB | — | 42.0 | 85.7% |

## Variants

| Variant | Recommended use |
|---|---|
| Q4_K_M | The promoted default serving lane — 2.6 GB, ~70 tok/s on the Spark, and byte-identical bench behavior to Q8_0 (18/21 scored==strict, refusals 9/9, same three safe-direction misses). Recommended pick. |
| Q8_0 | Effectively lossless — ~12 GB resident with an 8K context on the Spark, warm start ~2 s. Reach for it when you want maximum fidelity over throughput; the curveball numbers match Q4_K_M. |

## Choosing this lane

Pick this lane to serve Orionfold Advisor behavior locally: it expects retrieval packets (`Source N:` labelled excerpts plus the Advisor system contract) and answers with `Citations: [source_id]` lines. Trained with NVIDIA NeMo (LoRA r16 on `NVIDIA-Nemotron-3-Nano-4B`, merged and exported), quantized with llama.cpp. Run with reasoning off (`chat_template_kwargs: {"enable_thinking": false}`) to reproduce the measured behavior; the 30B teacher (`nemotron-3-nano-30b-a3b`) stays a prompt-only comparison lane, not a published artifact.

## How to run

Pull a variant (`model-Q4_K_M.gguf` is the default; swap in `model-Q8_0.gguf` for the lossless lane):

```bash
huggingface-cli download Orionfold/Advisor-GGUF model-Q4_K_M.gguf \
  --local-dir ./models/advisor
```

Serve it via `llama-server` (OpenAI-compatible API):

```bash
llama-server -m ./models/advisor/model-Q4_K_M.gguf \
  -c 8192 -ngl 99 --jinja \
  --host 0.0.0.0 --port 8080
```

`--jinja` applies the embedded Nemotron-3 chat template. To reproduce the
measured Advisor behavior, keep reasoning off per request:

```bash
curl -s http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "Question: Which gates must pass before an Orionfold artifact is published?"}],
  "temperature": 0,
  "chat_template_kwargs": {"enable_thinking": false}
}'
```

Or run in-process via `llama-cpp-python`:

```python
from llama_cpp import Llama
llm = Llama(
    model_path="./models/advisor/model-Q4_K_M.gguf",
    n_ctx=4096, n_gpu_layers=99,
)
out = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Question: Which gates must pass before an Orionfold artifact is published?\nAnswer with citations to the supplied sources."}],
    temperature=0.0,
)
print(out["choices"][0]["message"]["content"])
```

LM Studio and Ollama (via a Modelfile) load the GGUF directly with no additional setup.

## Known drift

Bounded limitations observed during Spark-side measurement. Each item below names the artifact and the scope of the drift; the balance of the bench measures clean — see [Methods](#methods) for the full breakdown.

- **`Route:` workflow-prefix discipline on "which doc defines X" phrasings without an evaluator hint** — 2/5 route rows on curveball-v0.1 rerun; all misses were citation-correct, only the prefix was absent
- **one over-refusal class out-of-distribution (safe direction)** — within the 3/21 misses on frozen curveball-v0.2
- **the 28/28 frozen held-out shares template machinery with the SFT corpus (in-distribution); treat the frozen OOD curveball as the honest floor** — OOD floor 18/21 scored==strict on curveball-v0.2
- **behavior is contract-shaped: outside Advisor-style packets (system contract + `Source N:` excerpts) citation/refusal discipline is unmeasured** — all published receipts use the packet contract

## Other Orionfold variants

Sibling repos from the same release:

| Variant | Lane | Format |
|---|---|---|
| [`Orionfold/Kepler-GGUF`](https://huggingface.co/Orionfold/Kepler-GGUF) | astrodynamics vertical curator (Qwen3-8B SFT) | gguf |

## Methods

Full methodology, gate definitions, and the publish decision:
[Orionfold Advisor — product launch](https://ainative.business/products/orionfold-advisor/).

Every number above is backed by a tracked receipt in the public monorepo:
[`evidence/orionfold-advisor/`](https://github.com/manavsehgal/ainative-business.github.io/tree/main/evidence/orionfold-advisor)
— including the frozen OOD bench (`advisor-curveball-v0.2.jsonl`, sha12
`4b6cac85e41f`, frozen **before** training), the 28-row frozen held-out
receipts (28/28 scored==strict on hinted and hint-free packets), the
three-lane curveball comparison (`advisor-curveball2-compare-v0.1.json`),
and the §14 publish receipt (`advisor-publish-receipt-v0.1.json`, verdict
PROMOTED, 9/9 gates).

---

Published by **Orionfold LLC** · [orionfold.com](https://orionfold.com) · Methods documented at [ainative.business/field-notes](https://ainative.business/field-notes/).