| --- |
| license: mit |
| base_model: ibm-granite/granite-3.3-8b-instruct |
| tags: |
| - meta-spider |
| - meta-attention |
| - selective-prediction |
| - calibration |
| - doubter |
| library_name: meta-core |
| pipeline_tag: text-generation |
| --- |
| |
| # meta-granite-8b — Doubter wrapper for Granite-3.3-8B-Instruct |
|
|
| A trained **meta-attention "Doubter"** wrapper for `ibm-granite/granite-3.3-8b-instruct`. It is |
| **not** a full model — it is a thin wrapper (~2% of the base) that reads the frozen base's own |
| activations and injects **cognitive tokens** through gated cross-attention, so the model learns |
| **when to trust itself**: answer confidently or refuse honestly. The base weights are never modified. |
|
|
| See the [meta-spider framework](#framework) for how to load and run it. |
|
|
| ## What's in here |
|
|
| | File | What it is | |
| |---|---| |
| | `doubter_checkpoint.pt` | the trained wrapper weights (encoder + cross-attention + gates), ~353 MB | |
| | `doubter_sidecar.gguf` | the same wrapper exported for llama.cpp (CPU / Metal / edge), ~352 MB | |
| | `run.json` | the training manifest (base model, layers, encoder type, quantization, dataset) | |
|
|
| ## Results (honest metrics) |
|
|
| Evaluated on MMLU (held-out, n=300). The base answers everything; the Doubter abstains on questions |
| it would likely get wrong. |
|
|
| | Metric | Base | + Doubter | |
| |---|---|---| |
| | **Selective accuracy** (of answered, % correct) | 0.63 | **0.77** | |
| | Coverage (answered / total) | 100% (300/300) | 55% (164/300) | |
| | Refusal rate | 0% | 45% | |
| | Refusal precision (vs oracle*) | — | 0.57 | |
| | Over-refusal rate | — | 0.43 | |
| | Total recovery rate | — | 0.69 | |
| |
| *Refusal precision is scored against an **oracle** (would the base have been wrong if it answered?), |
| not a naive text match. |
|
|
| **How to read this.** Selective accuracy rises **0.63 → 0.77 (+13.8 pp)** — on the questions it |
| chooses to answer, it is right much more often. This is **statistically significant** (McNemar |
| p ≈ 0, 52 confident-wrong answers turned into refusals vs 4 lost). Refusal precision 0.57 is honest |
| (moderately targeted, better than chance ~0.4); it over-refuses ~43% (the known cost of caution). The |
| usefulness criterion is **selective accuracy**, and it moves solidly on a real-size test set (n=300). |
|
|
| > Over-refusal is a **known cost**, not a failure. See the framework's honesty notes on metrics. |
|
|
| ## Training configuration (from `run.json`) |
|
|
| - **Base:** `ibm-granite/granite-3.3-8b-instruct` (frozen), nf4 quantized, float16 |
| - **Encoder:** `selective` (1 cognitive token per layer, scalar tanh gate) |
| - **Layers (read + inject):** `[26..39]` (the late third, 14 layers) |
| - **Data:** MMLU, MCQ-direct (`enable_thinking=False`, `thinking=False`, answer-only suffix — |
| required so the instruct model produces a letter on Pass 1) |
| - train / val / test = 1500 / 150 / 300 |
|
|
| ## Usage |
|
|
| ```python |
| from meta_core import MetaSpiderConfig, MetaSpiderPipeline, Doubter |
| |
| cfg = MetaSpiderConfig( |
| model_name="ibm-granite/granite-3.3-8b-instruct", |
| device="auto", dtype="float16", quantization="nf4", |
| target_layers=list(range(26, 40)), |
| cross_attn_layers=list(range(26, 40)), |
| ) |
| pipe = MetaSpiderPipeline.from_pretrained(cfg) |
| pipe.attach(Doubter.from_checkpoint("doubter_checkpoint.pt")) |
| |
| print(pipe.generate("What is the capital of France?")) |
| # → answers confidently |
| print(pipe.generate("<an obscure question the base would get wrong>")) |
| # → "I'm not confident enough to answer this question accurately." |
| ``` |
|
|
| Needs `pip install meta-core transformers accelerate bitsandbytes`. |
|
|
| ## Framework |
|
|
| This wrapper is produced and consumed by the **meta-spider** framework |
| ([codeberg.org/imperius/meta-spider](https://codeberg.org/imperius/meta-spider)) — meta-core / meta-loom / |
| meta-agent / meta-deploy). The included **GGUF sidecar** (`doubter_sidecar.gguf`, produced by |
| `metadeploy export`) runs the same wrapper on CPU inside llama.cpp — load it as a meta-adapter with |
| `llama-meta-generate` (two-pass inference), validated end-to-end on this model. Pair it with a |
| quantized Granite-3.3-8B GGUF (e.g. Q8_0); the calibrated refusal holds down to Q4_K_M. |
| |
| ## Caveats |
| |
| - This wrapper is **model-specific** — it is calibrated to the activation distribution of |
| `granite-3.3-8b-instruct`. It will **not** transfer cleanly to a different model or even a |
| different fine-tune of it (it would push hidden states out of distribution). |
| - It does **not** add knowledge or make the model smarter — it surfaces an existing internal |
| uncertainty signal and turns "answer at random" into "answer when confident". |
| |
