NGARiAI/ngari-ft-distilled

NGARi FT Distilled — Sovereign QA 1.5B

1.5B distilled QA model fine-tuned on NGARi's sovereign-agent domain data. Base: Qwen2.5-1.5B-Instruct (Apache 2.0), LoRA r=32, synthetic teacher qwen3:8b. Production-tested as the NGARi guardrail content-safety judge and QA model on aarch64 edge with 8GB RAM.

Provenance (verified Aug 3, 2026)

Attribute	Value
Base model	Qwen/Qwen2.5-1.5B-Instruct (Apache 2.0) — pinned in adapter_config.json
LoRA	rank 32, alpha 64, dropout 0.05, all linear projections
Synthetic data teacher	qwen3:8b
License	Apache 2.0 (NGARi-authored artifacts)
Hardware validated	aarch64 / NVIDIA Jetson AGX Orin, 8GB RAM, air-gap verified

Note: Google Gemma models were served only on NGARi hardware and were never used in NGARi training. All training used the Apache-2.0 Qwen2.5 lineage.

Evaluation

ngari-ft-distilled_chat_eval.json

{
  "model": "ngari-ft-distilled",
  "num_examples": 200,
  "total_examples": 200,
  "avg_score": 0.3766,
  "avg_latency_sec": 2.81,
  "tokens_per_sec": 39.78,
  "total_time_sec": 561.98
}

ngari-ft-distilled-stable_tool_eval.json

{
  "model": "ngari-ft-distilled:stable",
  "num_examples": 20,
  "total_examples": 20,
  "tool_detection_rate": 0.6,
  "tool_name_accuracy": 0.55,
  "params_validity_rate": 0.6,
  "tool_detected": 12,
  "name_correct": 11,
  "params_valid": 12,
  "avg_latency_sec": 2.38,
  "total_time_sec": 47.6
}

Files

File	Purpose
model-*.safetensors (+ config)	Merged full model — use with Transformers
adapter_model.safetensors	PEFT LoRA adapter — apply on the base
*-q4_K_M.gguf / *-f16.gguf	GGUF — use with Ollama / llama.cpp

Usage

# Ollama (GGUF)
ollama create ngariai/ngari-ft-distilled:q4_K_M -f Modelfile

# Transformers (merged)
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("ngariai/ngari-ft-distilled")

# PEFT adapter (apply on base)
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
adapter = PeftModel.from_pretrained(base, "ngariai/ngari-ft-distilled")

Sovereign AI

Trained and verified on user-owned edge hardware with zero cloud dependency. Part of the NGARi Sovereign Business Operating System (NS-BOS) — see https://github.com/ngariai/ns-bos-kernel for the Apache 2.0 kernel.