Text Classification
GGUF
ollaya
ollama
trainaya
decision-model
system-1
llm-logits-v1
calibrated
ministral
conversational
push-3b / README.md
solarpush's picture
Upload folder using huggingface_hub
435ea01 verified
|
Raw History Blame Contribute Delete
6.66 kB
metadata
license: apache-2.0
library_name: gguf
pipeline_tag: text-classification
base_model: mistralai/Ministral-3-3B-Instruct-2512
base-model: mistralai/Ministral-3-3B-Instruct-2512
language:
  - en
  - de
  - fr
  - es
  - multilingual
tags:
  - ollaya
  - ollama
  - trainaya
  - decision-model
  - system-1
  - llm-logits-v1
  - gguf
  - calibrated
  - ministral
datasets:
  - ZefanCai/Open-Jev-v1.1
  - SargeDev/jev-distill-corpus-v3
  - tasksource/tasksource-jev-typed-decisions

push:3b — Ultra-Fast Calibrated System-1 Decision Model (3B GGUF)

push:3b (solarpush/push-3b) is an ultra-low-latency (86 ms p50, 2.31 GiB on disk, ~4.2 GB VRAM), calibrated System-1 Decision Model (llm-logits-v1) designed for single-forward-pass structured decision-making across three universal primitives:

  • choice — Multi-way categorical selection (A..Z, 2 to 26 options),
  • noul — Binary epistemic / policy / safety / entailment verification (Yes vs No),
  • score — Ordinal rubric & severity evaluation (0..9, 3- to 5-level calibrated scales).

Built on a dense 3B parameter GGUF backbone (Q4_K_M, 2.31 GiB) with an untied F16 (ttype=1) subspace-orthogonal decision head ([131072, 3072]) and cardinality-conditioned temperature scaling (temperature_by_options), push:3b achieves 3.1% Pooled Public ECE, 85.7% accuracy on civil_comments, 84.0% on summeval-consistency, 74.6% on squad2, and 73.2% on pubmedqa.


1. Official Ollaya Decision Benchmark (bench.ollaya.dev — 14 Tasks, 5,880 Questions)

Evaluated strictly zero-shot / out-of-domain across all 14 official bench.ollaya.dev tasks (3,880 Public benchmark questions across 13 datasets + 2,000 typed-decisions workflow questions across 400 multi-question JSON states = 5,880 total questions, zero benchmark train/dev/test overlap):

Model Overall Skill [95% CI] Public Skill (13 Tasks) Public Macro Acc Typed-Decisions Acc (Skill) Public Pooled ECE (lower is better) Latency (p50) VRAM License
push:3b (Adapted 3B v12+) 46.0 [43.9, 48.2] 47.1 64.8% 53.9% (32.5) 3.1% (0.031) 86 ms 4.2 GB Apache-2.0
push_ministral:3b (Baseline Adapted 3B) 44.1 [42.0, 46.2] 45.3 63.7% 50.6% (27.6) 6.5% (0.065) 86 ms 4.2 GB Apache-2.0
push:26b (26B-A4B MoE) 62.7 [60.8, 64.7] 63.1 75.4% 71.4% (58.1) 2.8% (0.028) 223 ms 15.0 GB Apache-2.0

2. Per-Task Breakdown (14 Tasks — 5,880 Questions)

Benchmark Task Primitive Questions push:3b (v12+) Accuracy push:3b Skill (0–100) push_ministral:3b Baseline (Skill) Delta vs Baseline push:3b ECE (10-bin) Median Latency
civil_comments (Toxicity Detection) noul 300 85.7% 71.3 83.7% (67.3) +2.0% (+4.0 Skill) 0.043 51 ms
summeval-consistency (Factual Consistency) score 144 84.0% 80.0 83.3% (79.2) +0.7% (+0.8 Skill) 0.151 102 ms
massive-en-US (18-Way Intent) choice 350 76.0% 74.6 75.7% (74.3) +0.3% (+0.3 Skill) 0.064 65 ms
squad2 (Unanswerable QA) noul 299 74.6% 49.2 75.6% (51.2) -1.0% 0.056 67 ms
boolq (Reading Verification) noul 300 74.3% 48.7 75.7% (51.3) -1.4% 0.039 59 ms
pubmedqa (Biomedical 3-Way QA) choice 250 73.2% 59.8 74.4% (61.6) -1.2% 0.069 90 ms
massive-de-DE (18-Way Intent DE) choice 350 72.3% 70.7 72.3% (70.7) 0.0% 0.037 67 ms
aegis2 (AI Safety Guardrail) noul 250 64.8% 29.6 68.4% (36.8) -3.6% 0.125 54 ms
multinli (3-Way NLI) choice 299 62.2% 43.3 55.5% (33.3) +6.7% (+10.0 Skill) 0.072 46 ms
paws (Adversarial Paraphrase) noul 250 60.0% 20.0 61.2% (22.4) -1.2% 0.105 50 ms
vitaminc-dev (3-Way Fact Verification) choice 599 58.1% 37.1 52.9% (29.4) +5.2% (+7.7 Skill) 0.048 55 ms
typed-decisions (400 JSON Workflows) mixed 2000 53.9% 32.5 50.6% (27.6) +3.3% (+4.9 Skill) 0.153 110 ms
helpsteer2 (5-Level Helpfulness) score 249 33.7% 17.2 35.3% (19.2) -1.6% 0.127 109 ms
summeval-relevance (5-Level Relevance) score 240 24.2% 5.2 13.8% (-7.8) +10.4% (+13.0 Skill) 0.175 124 ms
OVERALL (14 Tasks) All 5880 64.1% (64.8% Public) 46.0 62.8% (44.1) +1.3% (+1.9 Skill) 0.090 (0.031 Pooled) 86 ms

typed-decisions Enterprise Workflow Breakdown (2,000 Questions / 400 JSON States)

  • security_incidents: 64.0%
  • agent_trace_observability: 53.0% (+6.4% vs baseline)
  • customer_service: 52.8% (+4.8% vs baseline)
  • invoice_processing: 45.8% (+2.4% vs baseline)

3. Repository Contents

File Size Description
model.gguf 2.31 GiB 3B GGUF (Q4_K_M backbone + untied F16 calibrated output.weight decision head)
decision.json 2.5 KB Ollaya llm-logits-v1 decision specification (labels, template, temperature_by_options, llama runtime config)
calibration.json 751 B Post-hoc temperature scaling parameters and out-of-fold / post-quantization accuracy metrics
ollaya_config.json 302 B Ollaya runtime manifest configuration
Modelfile 559 B Deterministic single-token Ollama Modelfile (temperature 0, top_k 1, num_predict 1)

4. Quickstart (Ollaya & Ollama)

Option A: Register in both Ollaya and Ollama via trainaya

git clone https://huggingface.co/solarpush/push-3b
cd push-3b
trainaya register --model . --name push:3b --runtime both

Option B: Upload to Hugging Face Hub from Trainaya Repo

make login
make push:3b