README.md

---
language:
- en
license: apache-2.0
tags:
- distillation
- knowledge-distillation
- json-extraction
- structured-output
- information-extraction
- gemma-4
- minicpm5
- edge-ai
- schemaforge
pipeline_tag: text-generation
base_model: openbmb/MiniCPM5-1B
library_name: transformers
metrics:
- accuracy
- f1
- throughput
---

# SchemaForge-1B — JSON Extractor

**A 1.08B-parameter edge SLM distilled from Gemma-4 for zero-shot enterprise JSON extraction.**

SchemaForge-1B converts unstructured business documents — invoices, bills of lading, requisitions, receipts — into strongly-typed, schema-conformant JSON. It was distilled from **`google/gemma-4-31B`** and **`google/gemma-4-E4B-it`** into **`openbmb/MiniCPM5-1B`** using a multi-task objective combining hard cross-entropy with temperature-scaled, log-space soft-logit KL divergence ($\alpha = 0.5$, $\tau = 2.0$), trained on an NVIDIA RTX PRO 6000 Blackwell Edition (96 GB).

| | 31B Teacher | **SchemaForge-1B** |
|---|---|---|
| In-domain JSON syntax error rate *(n = 5 docs)* | 0.0 % | **0.0 %** |
| In-domain extraction F1 *(n = 5 docs)* | 1.000 | **1.000** |
| Zero-shot validity (`suneeldk/text-json`) | — | **70.0 %** |
| Throughput | 12.40 tok/s | **61.91 – 76.27 tok/s** |
| Peak VRAM | ≈38.5 GB | **≈2.4 GB** |
| Workers per 96 GB GPU | 2 | **36** |

**16.0× smaller · 5.0× faster · ~110× aggregate system throughput**

---

## ⚠️ Read This First: The Prompt Template Is Not Optional

This model was distilled on **one exact prompt template**. Because it is a 1.08B student trained on a narrow task, it binds its behavior to the **literal surface form** of that prefix. In our experiments, changing only the instruction header dropped zero-shot validity from **70.0 % to 0.0 %** — worse than the *untrained* base model.

Use this string, byte for byte:

```python
TEMPLATE = "Extract structured JSON from the text:\n{doc}\nJSON Output:"
```

Do not wrap it in chat tokens. Do not prepend a system persona. Do not add a trailing newline. Treat it as a versioned API contract.

---

## 📊 Benchmark Evidence

### 1. Throughput and VRAM

![Figure 1: Inference throughput vs VRAM footprint](https://huggingface.co/arrochi112/SchemaForge-1B-JSON-Extractor/resolve/main/graphs/throughput_vs_vram.png)

*Figure 1: **5.0× throughput speedup** (61.91 vs. 12.40 tok/s, matched harness) and **16.0× VRAM reduction** (≈2.4 GB vs. ≈38.5 GB), measured on identical hardware.*

### 2. Zero-shot accuracy across distillation iterations

![Figure 2: Zero-shot JSON accuracy across iterations](https://huggingface.co/arrochi112/SchemaForge-1B-JSON-Extractor/resolve/main/graphs/accuracy_across_iterations.png)

*Figure 2: Validity on `suneeldk/text-json`. Iterations 1 and 3 differ from the winning Iteration 2 **only in prompt header** — and both collapse to 0.0 %.*

### 3. Training convergence

![Figure 3: Training loss convergence](https://huggingface.co/arrochi112/SchemaForge-1B-JSON-Extractor/resolve/main/graphs/loss_convergence.png)

*Figure 3: Loss over 3 epochs, Gemma-4-31B teacher. Iteration 2 (released): 9,132.9 → 6,962.3 → 6,612.7 (−27.6 %). Summed losses — comparable within a run, not across runs.*

---

## Quickstart

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "arrochi112/SchemaForge-1B-JSON-Extractor"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16 if torch.cuda.is_available() else torch.float32,
).to("cuda" if torch.cuda.is_available() else "cpu")
# No trust_remote_code needed — MiniCPM5-1B is a stock LlamaForCausalLM.

# CANONICAL TEMPLATE — do not modify
prompt = (
    "Extract structured JSON from the text:\n"
    "INVOICE #INV-1001. Vendor: Acme Supply Co. Date: 2026-04-10. "
    "Subtotal: $480.00. Tax (8%): $38.40. Total: $518.40.\n"
    "JSON Output:"
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)

print(tokenizer.decode(outputs[0][inputs["input_ids"].size(1):],
                       skip_special_tokens=True))
```

Expected:

```json
{
  "invoice_number": "INV-1001",
  "vendor_name": "Acme Supply Co",
  "invoice_date": "2026-04-10",
  "subtotal": 480.00,
  "tax": 38.40,
  "grand_total": 518.40
}
```

---

## Production Serving (vLLM)

```python
from vllm import LLM, SamplingParams

llm = LLM(
    model="arrochi112/SchemaForge-1B-JSON-Extractor",
    dtype="bfloat16",
    gpu_memory_utilization=0.90,
    max_model_len=2048,
    max_num_seqs=36,               # 36 workers fit in 96 GB at 2.4 GB each
)

sampling_params = SamplingParams(temperature=0.0, max_tokens=256)

TEMPLATE = "Extract structured JSON from the text:\n{doc}\nJSON Output:"
docs = [
    "Invoice #INV-881, Vendor: Globex Corp, Date: 2026-08-03, Total: $450.00",
    "Invoice #INV-882, Vendor: Initech LLC, Date: 2026-08-04, Total: $1200.00",
]

for out in llm.generate([TEMPLATE.format(doc=d) for d in docs], sampling_params):
    print(out.outputs[0].text)
```

### Recommended: layer schema-constrained decoding

Distillation supplies *semantics*; FSM-guided decoding guarantees *syntax*. Run both.

```python
from pydantic import BaseModel
from vllm.sampling_params import GuidedDecodingParams

class Invoice(BaseModel):
    invoice_number: str
    vendor_name: str
    invoice_date: str
    subtotal: float
    tax: float
    grand_total: float

sampling_params = SamplingParams(
    temperature=0.0,
    max_tokens=256,
    guided_decoding=GuidedDecodingParams(json=Invoice.model_json_schema()),
)
```

---

## Evaluation

### Five-domain enterprise suite (in-domain, n = 5 documents)

| Domain | Document type | Base MiniCPM5-1B | **SchemaForge-1B** | F1 | Throughput |
|---|---|---|---|---|---|
| BMK-01 Finance | Tax invoices | 65.8 % | **100.0 %** | 1.000 | 61.91 tok/s |
| BMK-02 Supply chain | Bills of lading | 67.1 % | **100.0 %** | 1.000 | 62.40 tok/s |
| BMK-03 IT hardware | Procurement bills | 64.2 % | **100.0 %** | 1.000 | 61.80 tok/s |
| BMK-04 Biomedical | Lab requisitions | 66.5 % | **100.0 %** | 1.000 | 62.15 tok/s |
| BMK-05 Cloud ops | Billing records | 65.4 % | **100.0 %** | 1.000 | 62.05 tok/s |

### Model comparison

| Variant | Teacher | JSON error rate | F1 | Throughput | VRAM |
|---|---|---|---|---|---|
| Base MiniCPM5-1B | none | 34.2 % | 0.612 | 62.00 tok/s | ≈2.4 GB |
| **SchemaForge-1B** | `gemma-4-E4B-it` | **0.0 %** | **1.000** | **61.91 tok/s** | **≈2.4 GB** |
| **SchemaForge-1B** | `gemma-4-31B` | **0.0 %** | **1.000** | 56.12 tok/s | **≈2.4 GB** |
| Gemma-4-31B | reference | 0.0 % | 1.000 | 12.40 tok/s | ≈38.5 GB |

Teacher scale conferred **no measurable quality advantage** on this task — the 4B teacher is the cost-effective choice.

### Out-of-domain (`suneeldk/text-json`)

| Iteration | Prompt template | Validity | Throughput |
|---|---|---|---|
| iter1 | chat tokens (`<start_of_turn>`) | 0.0 % | 76.94 tok/s |
| **iter2 (this model)** | **canonical** | **70.0 %** | **76.27 tok/s** |
| iter3 | system persona header | 0.0 % | 74.12 tok/s |
| base | canonical | 34.2 % | 62.00 tok/s |

---

## Training Details

| | |
|---|---|
| Architecture | `LlamaForCausalLM` — 24 layers, hidden 1536, GQA 16/2 heads, vocab 130,560 |
| Parameters | 1,080,632,832 total (679,552,512 non-embedding) |
| Objective | $\mathcal{L}_{KD} = \alpha\mathcal{L}_{CE} + (1-\alpha)\tau^2\mathcal{L}_{KL}$ |
| $\alpha$ / $\tau$ | 0.5 / 2.0 |
| Vocabulary projection | 256,000 → 130,560 (shared-subspace truncation) |
| Optimizer | AdamW, lr 2e-5, cosine, warmup 0.05 |
| Epochs | 3 (early-stopped on val loss) |
| Runtime | bfloat16, single-GPU PyTorch, eager attention (no ZeRO-3 / FlashAttention-2) |
| Max sequence length | 2,048 |
| Hardware | 1 × NVIDIA RTX PRO 6000 Blackwell Edition (96 GB), Nebius AI Cloud |
| Software | Python 3.12 · PyTorch 2.5 · transformers 5.x |

Full methodology, mathematics, compatibility patches, and ablations: **[`SCHEMAFORGE_WHITEPAPER.md`](./SCHEMAFORGE_WHITEPAPER.md)**.

---

## Limitations

Please read these before deploying.

- **Evaluation scale is small.** The in-domain suite is **n = 5 documents** (one per domain). The 100 % validity / 1.000 F1 figures are exact-match results on a small curated set, not population estimates — the Wilson 95 % CI on 5/5 is **[56.6 %, 100.0 %]**.
- **Training scale is small.** This checkpoint was distilled on **n = 5 samples**. An SFT control ($\alpha = 1.0$, no teacher logits) was **not run**, so we cannot presently separate the contribution of knowledge distillation from that of prompt-format conditioning.
- **Single seed.** No variance estimates or error bars. Sub-2B models vary substantially run-to-run on small datasets.
- **Prompt-template brittleness.** The headline failure mode. Deviating from the canonical template drops accuracy to ~0, not to a degraded-but-usable level.
- **Out-of-domain ceiling ≈ 70 %.** Roughly 30 % of unseen real-world documents produce unparseable output. Use constrained decoding in production.
- **Synthetic in-domain documents.** Clean ASCII, consistent labeling, no OCR noise, English-only. Real scanned documents will be harder.
- **Teacher outputs as targets.** Where the teacher was wrong, the student learned the error. No human-annotated gold standard exists for this checkpoint.
- **Not evaluated against alternatives.** No comparison to Qwen2.5-1.5B, Phi-3-mini, rule-based extractors, or commercial document-AI APIs.

**Intended use:** structured extraction from short English business documents, behind a schema-validation layer.
**Out of scope:** open-domain chat, reasoning, code, multilingual input, medical/legal decision-making, or any use where an unvalidated extraction reaches a system of record.

### Planned v2 run

This is a **v1 release**, and the accuracy numbers above should be read as provisional. A second training and evaluation campaign is planned to address the limitations listed here directly:

- **Real-world evaluation corpus** replacing the synthetic 5-document suite — $n \geq 500$ held-out documents per domain, including OCR-noisy scans, multi-column layouts, and non-English fields, with a **human-annotated gold subset** so accuracy is no longer measured against teacher output.
- **The SFT control** ($\alpha = 1.0$, no teacher logits) to determine whether the distillation objective contributes anything beyond prompt-format conditioning.
- **Competitive baselines** — Qwen2.5-1.5B, Phi-3-mini, prompt-engineered base MiniCPM5-1B with constrained decoding, and a rule-based extractor — under one unified harness.
- **Multi-seed runs** (≥3) with reported variance and confidence intervals on every metric.
- **Expanded metrics** beyond validity/F1/throughput/VRAM: per-field accuracy, schema-conformance rate, hallucinated-key rate, time-to-first-token, p50/p95 latency under concurrency, and cost per thousand documents.

Results will be published as a v2 card revision with the v1 numbers retained for comparison rather than quietly replaced.

---

## Citation

```bibtex
@techreport{ty2026schemaforge,
  title  = {SchemaForge: Distilling Ultra-Large Foundation Models into Edge SLMs
            for Real-Time Enterprise JSON Extraction --
            A Comparative Study of Gemma-4 Teachers and MiniCPM5-1B},
  author = {Ty, Arjhine A.},
  year   = {2026},
  note   = {Model: SchemaForge-1B (schemaforge-1b-iter2)},
  url    = {https://huggingface.co/arrochi112/SchemaForge-1B-JSON-Extractor}
}
```

## Acknowledgements

Teachers: `google/gemma-4-31B`, `google/gemma-4-E4B-it`. Student architecture: `openbmb/MiniCPM5-1B`. Compute: Nebius AI Cloud. Serving: vLLM. Constrained decoding: Outlines.

**License:** Apache 2.0 — subject to the upstream licenses of the base and teacher models.