File size: 3,512 Bytes
d7e542a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 | ---
license: gemma
base_model: google/gemma-4-e4b-it
library_name: transformers
pipeline_tag: image-text-to-text
tags:
- vision-language
- visual-question-answering
- knowledge-distillation
- lora
- merged
- research
datasets:
- lmms-lab/DocVQA
- lmms-lab/GQA
- lmms-lab/ChartQA
language:
- en
---
# CEED B4 — gemma-4-e4b-it, distilled, with visual-advantage reweighting
A LoRA fine-tune of [`google/gemma-4-e4b-it`](https://huggingface.co/google/gemma-4-e4b-it)
trained with B2's objective plus **VA-OPD's visual-advantage reweighting** (arXiv:2605.21924).
The teacher is the sparse mixture-of-experts [`google/gemma-4-26b-a4b-it`](https://huggingface.co/google/gemma-4-26b-a4b-it).
The adapter has been folded into the base weights, so this is a standalone
checkpoint: load it exactly like the base model, with no PEFT and no CEED code.
This is **Group B4** of the CEED study (Causal Expert–Evidence Distillation),
a research artifact published for reproducibility. It is not a product.
## Usage
```python
from transformers import AutoModelForImageTextToText, AutoProcessor
model = AutoModelForImageTextToText.from_pretrained("gnitoahc/ceed-b4", dtype="float16")
processor = AutoProcessor.from_pretrained("gnitoahc/ceed-b4")
```
The model was trained and scored with a short-answer instruction in the prompt.
Without it an instruction-tuned model answers `"The total written in the image is
**28**."` against gold `"28"` and scores zero on every metric here.
## Training
| | |
| --- | --- |
| Corpus | chartqa 2,500, docvqa 5,349, gqa 10,000 (17,849 examples, 80/10/10 split by example id) |
| Passes over the training split | 2.69 |
| Adapter | LoRA rank 4 |
| Final cross-entropy | 1.0979 |
| Final KD term | 2.1309 |
| Seed | 0 |
| Run identity | `6a4544ff1af675be8b364825a3f3f5c7bd825801471267ebd5f4956ff3febaa8` |
## Evaluation
| Dataset | Metric | Score | n |
| --- | --- | --- | --- |
| docvqa | ANLS | 0.8538 | 565 |
| gqa | exact match | 0.6102 | 1016 |
| chartqa | relaxed accuracy | 0.6185 | 249 |
Scored by CEED's own harness (`harness_version: ceed-direct-1`)
with greedy decoding, on CEED's own 10% validation split.
**These numbers are not comparable to published DocVQA / GQA / ChartQA leaderboard
results.** Different splits, different prompt, different decoding. They are
meaningful only against the other CEED Groups, which were scored identically.
## Limitations
- **This is a LoRA result.** Merging folds the adapter into the weights; it does
not turn a rank-4 adapter into a full
fine-tune. CEED's own ADR-0005 bars LoRA numbers from the study's headline
table, because a null result under a small adapter cannot be attributed between
"the signal does not transfer" and "the adapter lacked the capacity to hold it".
Read any comparison involving this checkpoint with that in mind.
- **The distillation gain is not established.** The no-teacher control (CEED B1), trained identically but with `kd_weight: 0`, scored **above this checkpoint on every dataset** (docvqa 0.8538 vs 0.8798; gqa 0.6102 vs 0.6959; chartqa 0.6185 vs 0.7871). Whatever this checkpoint's objective contributes, it is not visible as an advantage over supervised fine-tuning here.
- Trained on document, natural-image and chart VQA in English only. Behaviour
outside that is untested.
- Inherits the base model's limitations and the Gemma licence.
`ceed_provenance.json` beside the weights carries the source run's identity,
parameter-efficiency mode, and metrics.
|