---
license: mit
base_model: ibm-granite/granite-3.3-8b-instruct
tags:
  - meta-spider
  - meta-attention
  - selective-prediction
  - calibration
  - doubter
library_name: meta-core
pipeline_tag: text-generation
---

# meta-granite-8b — Doubter wrapper for Granite-3.3-8B-Instruct

A trained **meta-attention "Doubter"** wrapper for `ibm-granite/granite-3.3-8b-instruct`. It is
**not** a full model — it is a thin wrapper (~2% of the base) that reads the frozen base's own
activations and injects **cognitive tokens** through gated cross-attention, so the model learns
**when to trust itself**: answer confidently or refuse honestly. The base weights are never modified.

See the [meta-spider framework](#framework) for how to load and run it.

## What's in here

| File | What it is |
|---|---|
| `doubter_checkpoint.pt` | the trained wrapper weights (encoder + cross-attention + gates), ~353 MB |
| `doubter_sidecar.gguf` | the same wrapper exported for llama.cpp (CPU / Metal / edge), ~352 MB |
| `run.json` | the training manifest (base model, layers, encoder type, quantization, dataset) |

## Results (honest metrics)

Evaluated on MMLU (held-out, n=300). The base answers everything; the Doubter abstains on questions
it would likely get wrong.

| Metric | Base | + Doubter |
|---|---|---|
| **Selective accuracy** (of answered, % correct) | 0.63 | **0.77** |
| Coverage (answered / total) | 100% (300/300) | 55% (164/300) |
| Refusal rate | 0% | 45% |
| Refusal precision (vs oracle*) | — | 0.57 |
| Over-refusal rate | — | 0.43 |
| Total recovery rate | — | 0.69 |

*Refusal precision is scored against an **oracle** (would the base have been wrong if it answered?),
not a naive text match.

**How to read this.** Selective accuracy rises **0.63 → 0.77 (+13.8 pp)** — on the questions it
chooses to answer, it is right much more often. This is **statistically significant** (McNemar
p ≈ 0, 52 confident-wrong answers turned into refusals vs 4 lost). Refusal precision 0.57 is honest
(moderately targeted, better than chance ~0.4); it over-refuses ~43% (the known cost of caution). The
usefulness criterion is **selective accuracy**, and it moves solidly on a real-size test set (n=300).

> Over-refusal is a **known cost**, not a failure. See the framework's honesty notes on metrics.

## Training configuration (from `run.json`)

- **Base:** `ibm-granite/granite-3.3-8b-instruct` (frozen), nf4 quantized, float16
- **Encoder:** `selective` (1 cognitive token per layer, scalar tanh gate)
- **Layers (read + inject):** `[26..39]` (the late third, 14 layers)
- **Data:** MMLU, MCQ-direct (`enable_thinking=False`, `thinking=False`, answer-only suffix —
  required so the instruct model produces a letter on Pass 1)
- train / val / test = 1500 / 150 / 300

## Usage

```python
from meta_core import MetaSpiderConfig, MetaSpiderPipeline, Doubter

cfg = MetaSpiderConfig(
    model_name="ibm-granite/granite-3.3-8b-instruct",
    device="auto", dtype="float16", quantization="nf4",
    target_layers=list(range(26, 40)),
    cross_attn_layers=list(range(26, 40)),
)
pipe = MetaSpiderPipeline.from_pretrained(cfg)
pipe.attach(Doubter.from_checkpoint("doubter_checkpoint.pt"))

print(pipe.generate("What is the capital of France?"))
# → answers confidently
print(pipe.generate("<an obscure question the base would get wrong>"))
# → "I'm not confident enough to answer this question accurately."
```

Needs `pip install meta-core transformers accelerate bitsandbytes`.

## Framework

This wrapper is produced and consumed by the **meta-spider** framework
([codeberg.org/imperius/meta-spider](https://codeberg.org/imperius/meta-spider)) — meta-core / meta-loom /
meta-agent / meta-deploy). The included **GGUF sidecar** (`doubter_sidecar.gguf`, produced by
`metadeploy export`) runs the same wrapper on CPU inside llama.cpp — load it as a meta-adapter with
`llama-meta-generate` (two-pass inference), validated end-to-end on this model. Pair it with a
quantized Granite-3.3-8B GGUF (e.g. Q8_0); the calibrated refusal holds down to Q4_K_M.

## Caveats

- This wrapper is **model-specific** — it is calibrated to the activation distribution of
  `granite-3.3-8b-instruct`. It will **not** transfer cleanly to a different model or even a
  different fine-tune of it (it would push hidden states out of distribution).
- It does **not** add knowledge or make the model smarter — it surfaces an existing internal
  uncertainty signal and turns "answer at random" into "answer when confident".