Model card: remove internal evaluation details from the selection notes
8f052a8 verified 1 day ago
25.1 kB
---
license: apache-2.0
base_model:
  - google/gemma-4-31B-it
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-classification
language:
  - en
  - zh
tags:
  - decision-model
  - system-one
  - decision-index
  - lora-merged
---
# KnowLine-31B-Gen1

**The first release of PelaAI's System One decision model on a dense Gemma 4 base: 31B parameters, trained in the same
agent-driven data loop as the KnowLine-26B-A4B, KnowLine-35B-A3B and KnowLine-4B lines, and served with read twice.**

[Inference guide](INFERENCE.md) · [中文说明](README.zh.md) · weights Apache-2.0 · siblings:
[KnowLine-26B-A4B-Gen2.5](https://huggingface.co/PelaAI/KnowLine-26B-A4B-Gen2.5),
[KnowLine-35B-A3B-Gen1.5](https://huggingface.co/PelaAI/KnowLine-35B-A3B-Gen1.5),
[KnowLine-4B-Gen5.5](https://huggingface.co/PelaAI/KnowLine-4B-Gen5.5)

Highlights:

- **Decision Index 0.3 public suite: 70.13** (self-run, fresh full run, read twice). KnowLine-35B-A3B-Gen1.5 scores 70.72
  and KnowLine-26B-A4B-Gen2.5 68.21 on the same suite with the same serving; see [Comparison](#comparison).
- **Decision tasks from new domains:** on our larger set of decision tasks from domains not in training (34 tasks) it
  scores 74.9 (26B-A4B-Gen2.5: 66.4; 35B-A3B-Gen1.5: 56.2). On our older 10-task set it scores 57.7
  (26B-A4B-Gen2.5: 56.6; 35B-A3B-Gen1.5: 56.6).
- **Our other evaluations:** Chinese in-house sets 75.9 / 75.9 (26B-A4B-Gen2.5: 73.7 / 71.0), C-Eval 82.5 (77.9),
  Open-Jev 1.1 test / OOD 91.3 / 94.8 (92.7 / 92.7), JevBench / JevBench-hard 90.5 / 80.2 (90.5 / 81.1), English
  in-house set 70.2 (70.5); see [Evaluation](#evaluation).
- **Planted instructions:** 0.4% of answers changed because of an instruction planted in the state, and 5.3% with
  injection templates not seen in training, more than KnowLine-26B-A4B-Gen2.5 (0.1% and 1.7%); see
  [Limitations](#limitations).
- **Calibrated probabilities:** a temperature is folded into the weights. Computed with the board's method on the public
  suite, ECE is about 0.012-0.021 and the Brier score about 0.23-0.26 (KnowLine-26B-A4B-Gen2.5: 0.013-0.018 and
  0.24-0.27); on our own held-out sets calibration is weaker, see [Calibration](#calibration).
- **Images without any image training:** trained on text only, it keeps the base model's vision tower. On our
  reconstruction of the Decision Index 0.3.1 vision public suite it scores 69.9 (26B-A4B-Gen2: 64.7; 35B-A3B-Gen1:
  71.2), and 69.3 on the official vision public suite (self-run); see [Images](#images-zero-shot).
- **Boxes without box training:** asked for a named target in an image, it returns a bounding box zero-shot:
  ScreenSpot centre-hit 81.3, RefCOCO IoU≥0.5 77.3 (documents weaker, FUNSD 35.3); see
  [Boxes](#boxes-zero-shot-untrained).
- **KOF '98 harness: 16-2-0 against StartLux-Decision-35B-A3B and 4-14-0 against KnowLine-35B-A3B-Gen1**
  (single-bout character-mirror matches); the Jev pairing was not run. See [Game harness](#game-harness-kof-98).
- **An AI agent ran the whole loop end to end.** In each round it:
  - identified where the model was weak;
  - proposed a targeted data group;
  - built and decontaminated the data;
  - trained and evaluated the model on it;
  - kept or rejected the change based on the evidence.

  For this model, see [How Gen1 was made](#how-gen1-was-made).

KnowLine is an independent model. It is not affiliated with or endorsed by TypeSafe, Jev or Google, and it is not
derived from TypeSafe or Jev.

## What it does

You send a state (text or a chat) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model
answers every question with a probability distribution over its options:

- one forward pass per question, with no text generation;
- the output is the probability of each option's label token;
- existing Jev clients only need to change the base URL, and users of other KnowLine models only need to change the model
  name, because the interface is the same.

| | |
|---|---|
| Base model | [google/gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it) (Apache-2.0): dense, about 31B parameters |
| Training | LoRA SFT (rank 32, alpha 64, language model only) for one epoch, merged into bf16 weights; the result was then averaged with the base model, tensor by tensor: 0.75 × fine-tuned + 0.25 × gemma-4-31B-it (see [How Gen1 was made](#how-gen1-was-made)) |
| Release format | bf16 weights (about 63 GB), with a readout temperature T = 1.44 folded into the final norm weight and the final logit soft-cap (see [Calibration](#calibration)); the front end reads score questions at an effective temperature of 1.0 (`temperatures.json`). The vision tower is unchanged from the base model; the tokenizer is Gemma 4's, not the one of the Qwen-based KnowLine models. |
| Serving | `serve_knowline.sh`: SGLang in bf16, as for the 26B-A4B (we did not test FP8 at load for this model); `knowline_server.py` reads short requests twice, as in KnowLine-26B-A4B-Gen2.5 and KnowLine-35B-A3B-Gen1.5 (on by default; `--no-read-twice` reads each request once). SGLang reserves 85% of the GPU's memory by default (MEM); the weights take about 63 GB. We ran every evaluation with one server on one 96 GB H20. |
| Languages | English, Simplified Chinese, Traditional Chinese |

## Quickstart

```bash
pip install "sglang==0.5.21" "transformers==5.12.1" requests
bash serve_knowline.sh PelaAI/KnowLine-31B-Gen1 0 8080          # SGLang (bf16) on :9080 + /v1/systemone on :8080
curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "m",
  "state": "Customer: my order arrived broken, I want my money back.",
  "questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"},
                "tone": {"type": "choice", "instructions": "Customer tone?",
                         "criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}'
```

- **Front end:** `knowline_server.py` with `knowline_layout.py` next to it; together they depend only on transformers and
  requests. Text requests are rendered exactly as by the KnowLine-26B-A4B-Gen2.5 front end, prompt for prompt.
- **Full settings:** the exact settings used for our Decision Index run are in [INFERENCE.md](INFERENCE.md).

## How Gen1 was made

1. **Why a dense base?** Our MoE models are strong on the public suite and on private tests of the same skills, but on
   decision tasks from new domains they trail dense models of about 30B on the board. We trained the data of
   KnowLine-26B-A4B-Gen1 on this base to see how much of the new-domain gap comes from the base model.
2. **Data.** The data of KnowLine-26B-A4B-Gen1 in a single pass: the latest data round of the 35B line, rendered with
   Gemma's chat template, together with the injection-invariance items (instructions planted in the state that must not
   change the answer). All data went through the same decontamination check as the earlier releases; see
   [Disclosures](#disclosures).
3. **Training.** One epoch with a LoRA adapter (rank 32), merged into the weights.
4. **Averaging with the base model.** The fine-tuned weights were then averaged with those of the base model,
   [google/gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it) (Apache-2.0), over every tensor: 0.75 ×
   fine-tuned + 0.25 × base. Tensors that the LoRA adapter did not train, such as the embeddings and the vision tower,
   were already equal to the base model's and stay so.
5. **Serving and temperature.** It is served with read twice, as KnowLine-26B-A4B-Gen2.5 and KnowLine-35B-A3B-Gen1.5. A
   single temperature T = 1.44 was fitted on our in-training evaluation sets with this serving and folded into the
   weights. It changes the size of the probabilities, but not which option ranks first.
6. **Selection.** We compared the fine-tuned model and two averages (0.75 and 0.5 fine-tuned) on our release checks; the
   0.75 average is the release.

## Comparison

Scores on the Decision Index. Since Decision Index 0.4, the board's full score is 60% private tests of the same skills
and 40% private tasks from new domains; the public suite is shown but no longer counts. This model has no board entry
yet.

| model | size | DI 0.3 public | DI 0.4.1 full | source |
|---|---|---|---|---|
| **KnowLine-31B-Gen1** | 31B | **70.13** | not yet scored | self-run, official kit |
| KnowLine-35B-A3B-Gen1.5 | 35B-A3B | 70.72 | 65.77 | board |
| KnowLine-26B-A4B-Gen2.5 | 26B-A4B | 68.21 | 64.57 | board |
| Fastino GLiDE V1.2 | – | 71.99 | 71.52 | board |
| Jeffan-31b-1710 | 31B | 73.13 | 69.75 | board |
| mammoth-1 | 27B | 75.34 | 68.44 | board |
| Perplexity Decider v1.2 | 27B | 67.74 | 67.98 | board |
| StartLux-Decision v1.1 27B | 27B | 72.88 | 67.92 | board |
| FlexDM · Gemma-4-31B | 31B | 63.63 | 67.56 | board |
| Jev 1.14 | (API) | 60.70 | 60.24 | board |

Board rows from the board's data of 2026-10-22 (20:10 UTC, Decision Index 0.4.1).

Public-suite skill by area (DI 0.3, all three served read twice):

| area | KnowLine-31B-Gen1 | KnowLine-26B-A4B-Gen2.5 | KnowLine-35B-A3B-Gen1.5 |
|---|---|---|---|
| Knowledge & reasoning | 59.4 | 55.1 | 55.0 |
| Language | 72.0 | 71.2 | 72.9 |
| Retrieval & routing | 73.0 | 70.9 | 75.4 |
| Tools & agents | 84.7 | 83.3 | 88.2 |
| Arts & taste | 60.5 | 61.4 | 64.2 |

## Releases

The model is trained in a self-evolving loop, and new versions will follow.

| model | date | DI 0.3 public | golden held-out (en / zh-Hans / zh-Hant) | notes |
|---|---|---|---|---|
| KnowLine-31B-Gen1 (this model) | 2026-10-24 | 70.13 | 70.2 / 75.9 / 75.9 | first release on a dense Gemma 4 base; read-twice serving |
| [KnowLine-26B-A4B-Gen2.5](https://huggingface.co/PelaAI/KnowLine-26B-A4B-Gen2.5) | 2026-10-22 | 68.21 | 70.5 / 73.7 / 71.0 | Gen2 weights, read-twice serving |
| [KnowLine-35B-A3B-Gen1.5](https://huggingface.co/PelaAI/KnowLine-35B-A3B-Gen1.5) | 2026-10-22 | 70.72 | 70.8 / 74.6 / 73.2 | 35B-A3B Gen1 weights, read-twice serving |
| [KnowLine-4B-Gen5.5](https://huggingface.co/PelaAI/KnowLine-4B-Gen5.5) | 2026-10-22 | 66.75 | 70.3 / 72.1 / 67.5 | Gen5 weights, read-twice serving |
| [KnowLine-4B-Gen5](https://huggingface.co/PelaAI/KnowLine-4B-Gen5) | 2026-10-21 | 65.67 | 69.6 / 71.7 / 67.1 | 4B, fifth release |
| [KnowLine-26B-A4B-Gen2](https://huggingface.co/PelaAI/KnowLine-26B-A4B-Gen2) | 2026-10-19 | 66.94 | 70.6 / 72.6 / 69.8 | second 26B-A4B release |
| [KnowLine-26B-A4B-Gen1](https://huggingface.co/PelaAI/KnowLine-26B-A4B-Gen1) | 2026-10-19 | 65.76 | 70.6 / 72.2 / 69.2 | first release on a Gemma 4 base |
| [KnowLine-35B-A3B-Gen1](https://huggingface.co/PelaAI/KnowLine-35B-A3B-Gen1) | 2026-10-12 | 69.49 | 70.3 / 73.8 / 72.1 | first 35B-A3B release |

## Evaluation

All results are self-run and have not been verified by a third party. Every number for this model was measured through
its release front end (read twice on). From this release on, the rows for our new-domain task sets carry a footnote on
the tasks that resemble our training data. The 26B-A4B-Gen2.5 and 35B-A3B-Gen1.5 numbers are from their release
evaluations with the same front end.

### Held-out and Chinese evaluations

| suite | 31B-Gen1 | 26B-A4B-Gen2.5 | 35B-A3B-Gen1.5 |
|---|---|---|---|
| In-house evaluation set, English | 70.2 | 70.5 | 70.8 |
| In-house evaluation set, Simplified Chinese | 75.9 | 73.7 | 74.6 |
| In-house evaluation set, Traditional Chinese | 75.9 | 71.0 | 73.2 |
| C-Eval (4 categories, macro) | 82.5 | 77.9 | 88.1 |
| Open-Jev 1.1 test / OOD | 91.3 / 94.8 | 92.7 / 92.7 | 93.4 / 93.6 |
| JevBench / JevBench-hard | 90.5 / 80.2 | 90.5 / 81.1 | 87.0 / 73.0 |
| Decision tasks from new domains (our own, 34 tasks)¹ | 74.9 | 66.4 | 56.2 |
| Decision tasks from new domains (our own, 10 tasks)¹ | 57.7 | 56.6 | 56.6 |
| Prompt injection: answers changed (lower is better) | 0.4% | 0.1% | 0.0% |
| Prompt injection with templates not seen in training (lower is better) | 5.3% | 1.7% | 1.0% |

- In a paired sign test against KnowLine-26B-A4B-Gen2.5, it is higher on the Simplified and Traditional Chinese in-house
  sets, C-Eval and Open-Jev 1.1 OOD (p < 0.001 each) and lower on Open-Jev 1.1 test (p = 0.04); the English in-house set
  (p = 0.61), JevBench and JevBench-hard (p = 1.0) do not differ. With unseen injection templates, 32 of 600 answers
  moved to the planted option against 10 of 600 (p < 0.001).
- Against KnowLine-35B-A3B-Gen1.5, it is higher on the Simplified (p = 0.03) and Traditional (p = 0.004) Chinese in-house
  sets and lower on C-Eval (p < 0.001) and Open-Jev 1.1 test (p = 0.002); the English in-house set (p = 0.42),
  Open-Jev 1.1 OOD (p = 0.07), JevBench and JevBench-hard (p = 0.06) do not differ at p < 0.05. With unseen injection
  templates, 32 of 600 against 6 of 600 (p < 0.001).
- ¹ Our sets of decision tasks from domains not in training, run for all three models under the same serving (read
  twice through our evaluation front end, which uses the release layout). 11 of the 34 tasks (rules, policies, state
  tracking) resemble procedural synthetic tasks in our training data; on the other 23 it scores 73.3 (26B-A4B-Gen2.5:
  66.1; 35B-A3B-Gen1.5: 57.1).

### Web operation (Mind2Web official test splits, evaluation only)

| split | 31B-Gen1 element selection | 31B-Gen1 operation (balanced) | 26B-A4B-Gen2.5 element selection | 26B-A4B-Gen2.5 operation (balanced) |
|---|---|---|---|---|
| test_task (websites seen in training, new tasks) | 91.5 | 92.8 | 89.8 | 90.3 |
| test_website (new websites) | 91.5 | 95.9 | 89.2 | 94.0 |
| test_domain (new domains) | 90.4 | 96.6 | 89.6 | 92.9 |
- Element selection is higher than with KnowLine-26B-A4B-Gen2.5 on all three splits (paired test p = 0.05, 0.01 and
  0.11); operation accuracy is within noise (p ≥ 0.09). No request failed.
- The task is to pick the target element from a candidate set: the target plus up to 5 other elements sampled from the
  page. This is easier than the original Mind2Web protocol, so do not compare these results with the Mind2Web
  leaderboard.
### Images (zero-shot)
- **No image training:** the vision tower is the base model's, unchanged, and no images were seen in training.
  `knowline_server.py` accepts OpenAI-style `image` / `image_url` content parts in a chat state and passes them to
  SGLang; each question is still one prefill. Requests with images are read once (read twice is text-only).
- **How we measured it:** the same reconstruction of the Decision Index 0.3.1 vision public suite as for the earlier
  releases (11 benchmarks, chance-corrected skill and the board's weights), run through the release server, 0 failed
  requests. We also ran the official vision public suite (12,210 rows, complete, 0 failed).
| benchmark | 31B-Gen1 (our reconstruction) | 31B-Gen1 (official vision suite) | 26B-A4B-Gen2 (our reconstruction) | 35B-A3B-Gen1 (our reconstruction) |
|---|---|---|---|---|
| CV-Bench | 73.9 | 74.1 | 70.5 | 75.8 |
| BLINK | 65.7 | 58.0 | 57.4 | 59.8 |
| RealWorldQA | 57.2 | 55.9 | 54.2 | 71.9 |
| CharXiv | 65.8 | 74.7 | 61.9 | 66.9 |
| InfographicVQA | 86.9 | 91.3 | 81.4 | 90.7 |
| Mind2Web | 77.3 | 73.4 | 70.8 | 86.4 |
| Winoground | 85.5 | 86.0 | 78.5 | 80.0 |
| KIE (CORD+FUNSD) | 94.3 | 93.7 | 95.1 | 96.2 |
| Moderation (Hateful Memes) | 49.7 | 44.4 | 41.1 | 41.1 |
| R-Bench-M | 30.3 | 30.3 | 26.2 | 29.1 |
| MMMU-Pro vision | 38.7 | 36.2 | 28.0 | 45.1 |
| **weighted public index** | **69.9** | **69.3** | 64.7 | 71.2 |
- **Between the 26B-A4B and the 35B-A3B:** 5.2 points above KnowLine-26B-A4B-Gen2 and 1.3 points below
  KnowLine-35B-A3B-Gen1 on our reconstruction, with the largest gains over the 26B-A4B on MMMU-Pro vision (+10.7),
  Hateful Memes (+8.6) and BLINK (+8.3); KIE is 0.8 lower.
- **Indicative only:** the reconstruction is ours, not the board's rows; the official-suite column is a self-run of the
  official kit.
### Boxes (zero-shot, untrained)
- **What it does:** a box question names a target in an image; the server returns its bounding box, or a refusal when
  the model gives no box. There was no box training: this is the base model's own ability, read out through the
  release front end (`--box-prompt gemma --box-coords yxyx`).
- **How we measured it:** one request per item through the release server, 280 image soft tokens, images at most
  1,568 px per side, at most 64 new tokens. Every item counts and a refusal counts as a miss; 0 requests failed.
| set | n | measure | 31B-Gen1 | refusals |
|---|---|---|---|---|
| ScreenSpot (6 splits × 50) | 300 | centre inside the target | 81.3 | 5.3% |
| RefCOCO val | 300 | IoU ≥ 0.5 (mean IoU) | 77.3 (72.7) | 5.0% |
| FUNSD | 150 | IoU ≥ 0.5 (centre inside) | 35.3 (58.7) | 4.0% |
| Business UI screenshots, zh/en (our own)² | 1,500 | centre inside the target | 60.5 (en 67.4, zh 56.2) | 5.8% |
| Invoices, zh/en (our own)² | 800 | IoU ≥ 0.5 | 44.6 (clean 74.8, photo 43.4, scan 34.6) | 0.1% |
- **Weak on documents:** form and document fields are weak at the shipped 280 soft tokens (FUNSD 35.3). Higher image
  resolution helps documents; it is not offered in this release.
- **Text targets are easier than icons:** on ScreenSpot, icons 68.0 / 70.0 / 74.0 against text 92.0 / 94.0 / 90.0
  (desktop / mobile / web).
- **Confidence:** the box confidence is not a calibrated probability (ECE 0.238 on ScreenSpot, 0.153 on RefCOCO and
  0.247 on FUNSD). We ship no box calibration file.
- ² Synthetic held-out sets we built ourselves, for reference only; they are not public benchmarks.
### Game harness (KOF '98)
- **Setup:** single-bout character-mirror matches, 18 games per pair, argmax actions; the same settings as for the other
  KnowLine releases.
- **Result:** 16-2-0 against StartLux-Decision-35B-A3B and 4-14-0 against KnowLine-35B-A3B-Gen1; 20-16-0 overall
  (26B-A4B-Gen2: 17-1-0 and 6-6-6).
- **Jev:** not run: the Jev API was not available to us during this evaluation.
- **How the games ended:** against StartLux-Decision-35B-A3B, 6 of the 16 wins were by K.O. and 10 at time-out;
  against KnowLine-35B-A3B-Gen1, 3 of the 4 wins were by K.O., and 9 of the 14 losses were by K.O. and 5 at time-out.
- **Play style:** heavy kick 57%, special_2 25%, heavy punch 8%, special_3 5%, special_1 5%.
- **Caveats:** 18 games per pair give wide intervals; this is a measured result in this harness, not general
  fighting-game skill.

### Calibration

Computed on the 0.3 public suite using the method described by the Decision Index board: each field is scored right or
wrong, confidence is the probability of the chosen option, benchmarks are weighted equally, and 10 equal-width bins are
used.

| metric | 31B-Gen1 | 26B-A4B-Gen2.5 | 35B-A3B-Gen1.5 |
|---|---|---|---|
| ECE (lower is better) | 0.012-0.021 | 0.013-0.018 | 0.03-0.04 |
| Brier score (lower is better) | 0.23-0.26 | 0.24-0.27 | 0.24-0.26 |
| Confidence ≥95% but wrong | 0.9-1.0% | 0.9% | 1.5% |
| Mean confidence / accuracy | 0.83-0.84 / 0.81-0.83 | 0.82-0.83 / 0.80-0.82 | 0.85-0.86 / 0.81-0.83 |

- **Temperature in the weights:** the final norm weight and the final logit soft-cap include a readout temperature
  T = 1.44, fitted on our in-training evaluation sets with the read-twice serving, so every final logit is divided by 1.44
  before the softmax over the option labels. The top-ranked option does not change. On our held-out sets, ECE is 0.044
  with every question read at this temperature and 0.102 without it (T = 1).
- **Score questions at temperature 1.0 (new in this release):** `serve_knowline.sh` passes `temperatures.json`
  (`{"score": 0.6944}`), so the front end divides the logits of score (rubric) questions by a further 0.6944 and reads
  them at an effective temperature of 1.44 × 0.6944 = 1.0; yes/no and choice questions stay at 1.44. Score questions
  are often read as an expected value over the scale, which a higher temperature pulls toward the middle. This is a
  serving setting, not a weight change, and the earlier KnowLine releases did not use it; starting
  `knowline_server.py` without `--temperatures` (or `knowline_engine.py` with `--option temperatures=none`) reads every
  question at 1.44. The cost is calibration: with the release serving, ECE on our held-out sets is 0.064 (0.044 with
  score questions also at 1.44; KnowLine-26B-A4B-Gen2.5: 0.027; KnowLine-35B-A3B-Gen1.5: 0.023).
- **Why ranges:** the board does not publish which benchmarks it uses for calibration, so we report ranges over the
  plausible benchmark sets, as for the earlier releases.
- **Recommendation:** the temperature was fitted on our data. If your use depends on probability thresholds, check
  calibration on your own data.

## Disclosures

- **Decision Index format training data:** the training data is that of KnowLine-26B-A4B-Gen1 (906,122 rows in one
  pass). About 33% of the rows target Decision Index benchmarks in their request format or a reworded version of it:
  train splits of public datasets rewritten in that format (for example WinoGrande XL, ACOS and GSM8K) and synthetic
  items written in the same format. Another 0.5% are copies of the same items rendered in other formats, and 2.6%
  (23,722 rows) are injection-invariance items.
- **Decontamination:** all training data was checked with the same tool as the earlier releases against every Decision
  Index 0.3 row, all of our evaluation sets and the other public benchmarks we report (Intern-Decision, S1MB, JevBench,
  MMDM). 491 rows that nearly duplicated an evaluation item, repeated its question or claim, or contained its whole state
  were removed when this data was built for KnowLine-26B-A4B-Gen1.
- **Averaging with the base model:** every tensor is 0.75 × the fine-tuned tensor + 0.25 × the tensor of
  google/gemma-4-31B-it (Apache-2.0, the same licence as this release), embeddings and vision tower included (tensors
  the LoRA adapter did not train already equal the base model's). The averaging weight was chosen on our evaluation sets
  (see the next point).
- **Selection on evaluations:** we compared the fine-tuned model and two averages with the base model on our release
  checks, served read twice; the release is the 0.75 average. It is still below KnowLine-35B-A3B-Gen1.5 on C-Eval and
  Open-Jev 1.1 test, and easier to steer with planted instructions than both siblings (see [Evaluation](#evaluation)).
- **No model outputs as labels:** no output of Jev or any other decision model was used as a training label.
- **Teacher-labelled synthetic data:** LLM teachers wrote and labelled our synthetic tasks. The labels were filtered, but
  not all of them were checked by a human.

## Limitations

- **Size and speed:** about 63 GB of bf16 weights and dense, so every token uses all 31B parameters; we have only run it
  on 96 GB GPUs. One request at a time on one H20, requests take 340 ms median and 910 ms mean (3,977 ms at the 95th
  percentile), against 66.3 ms and 151 ms for KnowLine-26B-A4B-Gen2.5; see [INFERENCE.md](INFERENCE.md).
- **Read twice:** text-only requests up to 3,000 tokens are written twice, about 1.54 times the prompt tokens on the
  Decision Index suite. Use `--no-read-twice` if latency matters more; we did not evaluate this model without read twice
  for this card.
- **Planted instructions:** easier to steer than our other models: with injection templates not seen in training, 5.3%
  of answers moved to the planted option, against 1.7% for KnowLine-26B-A4B-Gen2.5 and 1.0% for KnowLine-35B-A3B-Gen1.5
  (0.4% with the seen templates). Keep untrusted text clearly delimited.
- **C-Eval and Open-Jev 1.1 test:** below KnowLine-35B-A3B-Gen1.5 on C-Eval (82.5 against 88.1) and Open-Jev 1.1 test
  (91.3 against 93.4).
- **Calibration on our sets:** with score questions read at an effective temperature of 1.0, ECE on our held-out sets
  is 0.064, higher than for KnowLine-26B-A4B-Gen2.5 (0.027) and KnowLine-35B-A3B-Gen1.5 (0.023).
- **Images:** there was no image training, so check image use on your own data.
- **Boxes:** zero-shot (no box training); weak on form and document fields, and the box confidence is not calibrated;
  see [Boxes](#boxes-zero-shot-untrained).
- **KOF '98:** 4-14-0 against KnowLine-35B-A3B-Gen1 in our harness (18 games, wide interval); see
  [Game harness](#game-harness-kof-98).
- **Other attacks:** other prompt-injection attacks may still work.
- **Private tests:** part of the public-suite score comes from benchmarks whose tasks are in our training data, so the
  private tests may show a lower score.

## Citation

```bibtex
@misc{knowline31bgen1,
  title  = {KnowLine-31B-Gen1: a 31B decision model},
  author = {PelaAI},
  year   = {2026},
  url    = {https://huggingface.co/PelaAI/KnowLine-31B-Gen1}
}
```

## License

- **Weights:** Apache-2.0, the same as the base model.
- **Base model licence:** google/gemma-4-31B-it is released under Apache-2.0; see Google's licence page,
  [ai.google.dev/gemma/docs/gemma_4_license](https://ai.google.dev/gemma/docs/gemma_4_license).
- **Code:** `knowline_server.py`, `knowline_layout.py` and `knowline_engine.py` are MIT (see the file headers).