Imperius/meta-granite-3.3-8b-instruct

meta-granite-8b — Doubter wrapper for Granite-3.3-8B-Instruct

A trained meta-attention "Doubter" wrapper for ibm-granite/granite-3.3-8b-instruct. It is not a full model — it is a thin wrapper (~2% of the base) that reads the frozen base's own activations and injects cognitive tokens through gated cross-attention, so the model learns when to trust itself: answer confidently or refuse honestly. The base weights are never modified.

See the meta-spider framework for how to load and run it.

What's in here

File	What it is
doubter_checkpoint.pt	the trained wrapper weights (encoder + cross-attention + gates), ~353 MB
doubter_sidecar.gguf	the same wrapper exported for llama.cpp (CPU / Metal / edge), ~352 MB
run.json	the training manifest (base model, layers, encoder type, quantization, dataset)

Results (honest metrics)

Evaluated on MMLU (held-out, n=300). The base answers everything; the Doubter abstains on questions it would likely get wrong.

Metric	Base	+ Doubter
Selective accuracy (of answered, % correct)	0.63	0.77
Coverage (answered / total)	100% (300/300)	55% (164/300)
Refusal rate	0%	45%
Refusal precision (vs oracle*)	—	0.57
Over-refusal rate	—	0.43
Total recovery rate	—	0.69

*Refusal precision is scored against an oracle (would the base have been wrong if it answered?), not a naive text match.

How to read this. Selective accuracy rises 0.63 → 0.77 (+13.8 pp) — on the questions it chooses to answer, it is right much more often. This is statistically significant (McNemar p ≈ 0, 52 confident-wrong answers turned into refusals vs 4 lost). Refusal precision 0.57 is honest (moderately targeted, better than chance ~0.4); it over-refuses ~43% (the known cost of caution). The usefulness criterion is selective accuracy, and it moves solidly on a real-size test set (n=300).

Over-refusal is a known cost, not a failure. See the framework's honesty notes on metrics.

Training configuration (from run.json)

• Base: ibm-granite/granite-3.3-8b-instruct (frozen), nf4 quantized, float16
• Encoder: selective (1 cognitive token per layer, scalar tanh gate)
• Layers (read + inject): [26..39] (the late third, 14 layers)
• Data: MMLU, MCQ-direct (enable_thinking=False, thinking=False, answer-only suffix — required so the instruct model produces a letter on Pass 1)
• train / val / test = 1500 / 150 / 300

Usage

from meta_core import MetaSpiderConfig, MetaSpiderPipeline, Doubter

cfg = MetaSpiderConfig(
    model_name="ibm-granite/granite-3.3-8b-instruct",
    device="auto", dtype="float16", quantization="nf4",
    target_layers=list(range(26, 40)),
    cross_attn_layers=list(range(26, 40)),
)
pipe = MetaSpiderPipeline.from_pretrained(cfg)
pipe.attach(Doubter.from_checkpoint("doubter_checkpoint.pt"))

print(pipe.generate("What is the capital of France?"))
# → answers confidently
print(pipe.generate("<an obscure question the base would get wrong>"))
# → "I'm not confident enough to answer this question accurately."

Needs pip install meta-core transformers accelerate bitsandbytes.

Framework

This wrapper is produced and consumed by the meta-spider framework (codeberg.org/imperius/meta-spider) — meta-core / meta-loom / meta-agent / meta-deploy). The included GGUF sidecar (doubter_sidecar.gguf, produced by metadeploy export) runs the same wrapper on CPU inside llama.cpp — load it as a meta-adapter with llama-meta-generate (two-pass inference), validated end-to-end on this model. Pair it with a quantized Granite-3.3-8B GGUF (e.g. Q8_0); the calibrated refusal holds down to Q4_K_M.

Caveats

• This wrapper is model-specific — it is calibrated to the activation distribution of granite-3.3-8b-instruct. It will not transfer cleanly to a different model or even a different fine-tune of it (it would push hidden states out of distribution).
• It does not add knowledge or make the model smarter — it surfaces an existing internal uncertainty signal and turns "answer at random" into "answer when confident".