Slayer1B 1.08: update cards and documentation

a64b838 verified about 6 hours ago

15.5 kB

---
language:
- pl
- en
library_name: pytorch
pipeline_tag: text-generation
license: apache-2.0
tags:
- polish
- causal-lm
- continued-pretraining
- experimental
---
# Slayer1B 1.08

**Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**

**Slayer1B 1.08 is an experimental base language model continued from Slayer1B 1.07.** Training completed on 9 November 2026 after 1,709 updates and 1,792,016,384 processed token positions, using Polish analytical text, English technical text and replay. It is not instruction-tuned and is distinct from the earlier 1.08b ablation.

[Public training trace](https://track.fabryka.ai/run/77126379-f09d-57b8-a0e6-9d9e4e5f1c7a).

## Run objective

Train on as many approved token positions as the fixed compute budget permits, recording actual progress and expense at the spot prices observed during the run. Synthetic speed, real-data training and downstream evaluation are separate measurements. The expense report covers the billed session, including preparation and evaluation; steady-state cost per billion tokens is an extrapolation, not that bill.

## Model and starting point

The run initializes from [SlayerLab/Slayer1B-1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07/blob/a090b54e658fb57dc5166c1fcc90cc6bf5c80106/README.md), revision `a090b54e658fb57dc5166c1fcc90cc6bf5c80106`. Its published card describes a Polish-dominant decoder-only model with **1,073,912,851 parameters**, a **32,768-token v6 vocabulary**, and final checkpoint **22,327**. These are base-model properties.

The continuation preserves that architecture and tokenizer. Its task is causal text continuation, not instruction tuning. The objective is greater exposure to Polish analytical and English technical text while retaining general-language coverage.

## Frozen launch recipe

| Setting | Selected configuration |
|---|---|
| Initialization | Public 1.07 weights; fresh optimizer, rather than optimizer-state resume |
| Trainer and optimizer | Existing r24.2; Muon with AdamW |
| Batch and context | 256 sequences × 4,096 token positions |
| Parallel layout | Eight GPU ranks; microbatch 8 × accumulation 4; activation checkpointing disabled |
| Schedule | 300 warmup updates, then WSD with decay over the final 25% to a nonzero floor of 0.00006 |
| Checkpoints | Every 250 updates and at the end; FINAL fixed in advance for evaluation |

## Training execution and cost

| Observation | Value and scope |
|---|---|
| Hardware | Eight NVIDIA B200 GPUs, Nebius spot; USD 0.99 per GPU-hour |
| Completed updates | 1,709 |
| Processed positions | 1,792,016,384 at the stated global batch; approximately 0.64 passes through the prepared input |
| Training window | 58.6 minutes, including compilation and checkpoints; not the full billed session |
| Steady interval throughput | Approximately 572,000 positions/s; not positions divided by the whole training window |
| Training loss | Approximately 2.20 at update 10 to 1.99 at completion; not an evaluation score |
| Training GPU cost | Approximately USD 7.73, or USD 4.32 per billion processed positions, calculated from the observed window and spot rate |
| Cost through evaluation completion | Approximately USD 21.5; preparation ≈11.2, training 7.73, evaluation/export ≈2.6. Provider-counter attribution, not a separate settled invoice |
| Complete allocation cost | Approximately USD 24.1; the allocation was preempted shortly afterwards and released, rather than reused for 1.09 |

Input capacity, processed positions, steady interval speed and the complete billed session remain separate quantities.

## Training inputs

| Input | Pinned revision |
|---|---|
| [Slayer PL Logika 2B](https://huggingface.co/datasets/SlayerLab/slayer-pl-logika-2b/tree/e24c8b71526ca1dc56a280100f88c2f09a79ecdd) | `e24c8b71526ca1dc56a280100f88c2f09a79ecdd` |
| [English Technical 1B](https://huggingface.co/datasets/SlayerLab/english-technical-1b/tree/3b833d36bd3ba9aeecf46f39605c9ac849bb96a3) | `3b833d36bd3ba9aeecf46f39605c9ac849bb96a3` |
| [Slayer 1.07 replay](https://huggingface.co/datasets/SlayerLab/slayer-1.07-pretrain/tree/6a337bbfba518d88f14eb54f7d0a9b9f297df0d1) | `6a337bbfba518d88f14eb54f7d0a9b9f297df0d1` |

Replay uses `pl_web`, `en_fineweb_edu` and `en_mix`: previously seen material, not fresh web. Reference v2 contains **2,800,806,882 native token positions including EOS**, after source selection and filtering; this is input capacity, not observed training exposure. The prepared TRAIN digest is `b73a69eaf399bb594300e7d1914a967a38b80b5bc35505fbc6459a6a27be496f`. GPL/GFDL and mixed GPL/LGPL sources are excluded. English validation/test remain outside TRAIN, and Polish held-out sections are grouped by source book.

## Comparison limits

A replay-only continued-pretraining control is not budgeted for the priority run. A difference from 1.07 therefore combines data, fresh optimizer initialization and **learning-rate decay: 1.07 was released from a constant-LR checkpoint without decay**. It does not isolate the new datasets' effect. Inherited exposure in the base model is an additional limitation.

Evaluation records identify checkpoints, evaluator, split, sample count, prompting, normalization and test-read history.

## Standard benchmark measurements

We thank **BenchTeam / Fabryka AI** for its benchmark implementations, protocol work and evaluation proposals. The team's independent [full benchmark report](https://github.com/slayerlabs/benchmarks-fabryka-ai/blob/8b547007526971bd2fc35dfe56ff2c7d9b802568/research/results/full-run-2026-11/README.md) is a separate measurement source. **The table below reports SlayerLab evaluations**, also published in the [1.07 training record](https://track.fabryka.ai/run/4973c068-8dd5-59d8-b6c4-671ca3beb925) and the [1.08 training record](https://track.fabryka.ai/run/77126379-f09d-57b8-a0e6-9d9e4e5f1c7a); it is not a copy of that independent run.

All nine tasks use their full evaluated split, zero-shot likelihood scoring and float32. Scores are percentages with **pointwise 95% confidence intervals**. `acc` is accuracy from summed continuation likelihood; `acc_norm` is the evaluator's length-normalized accuracy. The metric is named per task rather than combined into a score. WinoGrande uses partial-sentence likelihood; MMLU-PL is micro-averaged over 14,042 questions. Polish HellaSwag and MMLU use the OpenGPTX task definitions.

For comparison, the continuation columns are **1.09**, not additional 1.08 measurements: updates 5,000, 10,000, 15,000 and 20,000 are distinct intermediate working checkpoints of the ongoing [1.09 run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a), not inference-ready weight releases or its eventual budget-limited endpoint. Evaluator SHA, dataset pins, file digests, task protocols and sample counts match across these rows.

| Task | Metric | n | 1.07 | 1.08 | 1.09, update 5,000 | 1.09, update 10,000 | 1.09, update 15,000 | 1.09, update 20,000 |
|---|---|---:|---:|---:|---:|---:|---:|---:|
| ARC-Challenge | acc_norm | 1,172 | 27.30 [24.74, 29.86] | 30.03 [27.39, 32.68] | 29.35 [26.79, 32.00] | 30.46 [27.82, 33.11] | 32.51 [29.86, 35.24] | 33.11 [30.46, 35.84] |
| HellaSwag | acc_norm | 10,042 | 43.38 [42.43, 44.34] | 43.42 [42.43, 44.39] | 46.53 [45.55, 47.52] | 48.93 [47.97, 49.91] | 49.77 [48.80, 50.76] | 50.38 [49.39, 51.35] |
| PIQA | acc_norm | 1,838 | 66.97 [64.80, 69.15] | 66.21 [64.15, 68.34] | 68.28 [66.21, 70.40] | 68.99 [66.92, 71.11] | 68.99 [66.87, 71.06] | 70.78 [68.72, 72.85] |
| WinoGrande | acc | 1,267 | 53.59 [50.83, 56.35] | 53.67 [50.91, 56.43] | 54.93 [52.17, 57.70] | 54.93 [52.17, 57.62] | 56.75 [54.06, 59.43] | 55.01 [52.25, 57.77] |
| BoolQ | acc | 3,270 | 58.29 [56.61, 59.97] | 59.97 [58.26, 61.59] | 61.80 [60.18, 63.49] | 61.13 [59.45, 62.78] | 59.88 [58.20, 61.53] | 51.41 [49.69, 53.12] |
| OpenBookQA | acc_norm | 500 | 33.60 [29.40, 37.80] | 34.80 [30.60, 39.00] | 35.60 [31.40, 40.00] | 34.40 [30.20, 38.60] | 36.00 [31.80, 40.20] | 35.20 [31.00, 39.40] |
| Belebele, Polish | acc | 900 | 26.00 [23.11, 28.89] | 24.00 [21.22, 26.89] | 24.11 [21.33, 26.89] | 24.11 [21.33, 27.00] | 22.56 [19.89, 25.33] | 24.67 [21.89, 27.56] |
| HellaSwag-PL | acc_norm | 10,042 | 43.83 [42.84, 44.79] | 43.90 [42.90, 44.87] | 44.73 [43.76, 45.71] | 45.51 [44.51, 46.48] | 44.18 [43.21, 45.16] | 44.24 [43.26, 45.22] |
| MMLU-PL | acc | 14,042 | 24.01 [23.30, 24.71] | 23.04 [22.34, 23.74] | 24.10 [23.39, 24.82] | 23.32 [22.62, 24.03] | 23.50 [22.80, 24.21] | 23.86 [23.17, 24.58] |
Intervals use the percentile bootstrap with 10,000 replicates and seed 20261110. The updated approved aggregate does not supply the random-stream derivation or CI-generator SHA; the previous aggregate documented model-specific streams. **Items are treated as independent; subject, template and source clusters are not modeled.** These are intervals for individual results, not paired intervals for model differences or estimates of training-seed variance. They do not support a difference claim based on interval overlap.
[Machine-readable aggregate record](EVALUATION.json): exact scores, intervals, counts, dataset revisions and file digests, checkpoint and evaluator identities, and metric selection. It contains no prompts, answers or per-item predictions. ARC-Challenge, OpenBookQA and MMLU-PL use test splits; HellaSwag, PIQA, WinoGrande, BoolQ and HellaSwag-PL use validation splits; Belebele uses the full Polish evaluation set.
The approved aggregate SHA-256 is `fc6f67a20bdc6ada687349d35de26f6811e27766dfe1aa197082000c03535796`. The earlier 1.07 and 1.08 scores, intervals and dataset pins are unchanged. The continuation comparison does not extend or replace the separate GSM8K/SST-2 protocol below.
**Descriptive results; contamination scan of the training lineage against these tasks is pending.** Belebele and MMLU-PL results are near their four-choice 25% random baseline. No overall benchmark average, ranking or model-release threshold is inferred from this table. Results from other evaluators can differ with prompting, BOS handling, normalization or library version; they should not be silently substituted.
Earlier internal diagnostics and their measurements remain available in the [versioned evaluation record](https://huggingface.co/SlayerLab/Slayer1B-1.08/blob/5f854a0cc7160594d54d1788d37a44e8ebabe16c/README.md). They are historical measurements, not this card's release criteria.
## Additional GSM8K and SST-2 measurements
SlayerLab measured this two-task subset on 11 November 2026 using the [pinned BenchTeam code](https://github.com/slayerlabs/benchmarks-fabryka-ai/tree/8b547007526971bd2fc35dfe56ff2c7d9b802568), lm-eval 0.4.9.1, the `slayer` adapter, float32, batch 1 and context limit 4,096. These are full evaluated splits, not a full 50-task run or a copy of the community's independent report. The evaluated 1.07 and 1.08 weight identities match the completed checkpoints above.
| Task | Metric / prompting | n | 1.07, update 22,327 | 1.08, update 1,709 |
|---|---|---:|---:|---:|
| GSM8K | strict exact match, 5-shot | 1,319 | 1.52 [0.91, 2.20] | 0.91 [0.45, 1.44] |
| SST-2 | accuracy, 0-shot | 872 | 51.03 [47.71, 54.36] | 50.46 [47.13, 53.67] |
Scores are percentages with reported pointwise 95% bootstrap intervals (10,000 draws, seed 20261028). These intervals describe individual results, not model differences or training-seed variance. This protocol and its intervals are recorded separately under `additional_measurements` in [EVALUATION.json](EVALUATION.json); the preceding nine-task protocol is unchanged.

**Descriptive results; training-lineage contamination scan pending.** GSM8K strict-match scores are near zero. SST-2 zero-shot accuracy is calibration-sensitive; these scores alone do not establish a general reasoning or sentiment-understanding gain. No paired-difference or internal calibration analysis is published here.

## Licensing and limitations

Weights are released under **Apache-2.0**, as 1.07; source corpora keep their own obligations, with attributions in the [data card](DATA_CARD.md). FinePDFs compilation terms and its TDM rationale do not settle rights to every PDF. Book, translation and technical-source notices must be preserved where required.

A base language model can generate inaccurate, biased, offensive or personal information. Translated reasoning and technical text can contain errors; their presence does not demonstrate reasoning, technical reliability or instruction following. Privacy filtering is heuristic, and technical or tax material is not professional advice.


## Acknowledgements

We thank the entire **BenchTeam / Fabryka AI** community for benchmark implementations, original-protocol documentation, evaluation runs and independent checks. The team is coordinated by **[@LoIlita](https://github.com/LoIlita) and [@Elwenor](https://github.com/Elwenor)**, as documented in the repository's [contribution guide](https://github.com/slayerlabs/benchmarks-fabryka-ai/blob/8b547007526971bd2fc35dfe56ff2c7d9b802568/CONTRIBUTING.md).

The following accounts contributed to [benchmarks-fabryka-ai](https://github.com/slayerlabs/benchmarks-fabryka-ai). This is a summary of documented contributions, not a ranking or a complete membership list.

| GitHub account | Documented contribution |
|---|---|
| [@LoIlita](https://github.com/LoIlita) | Team coordination and onboarding; evaluation workflow, methodology and benchmark reviews ([PR #92](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/92), [PR #185](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/185)). |
| [@Elwenor](https://github.com/Elwenor) | Team coordination; EWoK implementation, evaluation runner, multi-model benchmark run and reports ([PR #93](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/93), [PR #179](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/179)). |
| [@adgr3](https://github.com/adgr3) | Original-benchmark cards, numerical-verdict validation and dataset-licence review ([PR #94](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/94), [PR #144](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/144), [PR #175](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/175)). |
| [@gzawadzki](https://github.com/gzawadzki) | Slayer integration with lm-eval and WinoGrande evaluation ([PR #64](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/64), [PR #100](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/100)). |
| [@KateMajzel](https://github.com/KateMajzel) | MathQA and PubMedQA evaluation work ([PR #158](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/158), [PR #156](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/156)). |
| [@jakublvndsky](https://github.com/jakublvndsky) | HellaSwag reproduction and Slayer evaluation ([PR #149](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/149)). |
| [@janbanot](https://github.com/janbanot) | ARC-Easy original-benchmark card ([PR #70](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/70)). |

We invite BenchTeam, DatasetTeam and other contributors to propose **datasets for subsequent Slayer training runs**. The collaboration aims to assemble a DatasetTeam corpus of **one trillion (1T) tokens**. Source proposals should include provenance, version, language and domain coverage, applicable rights and known privacy or evaluation-overlap risks. This is a collection objective, not a claim that the corpus has already been assembled or used in Slayer1B 1.08.

	---
	language:
	- pl
	- en
	library_name: pytorch
	pipeline_tag: text-generation
	license: apache-2.0
	tags:
	- polish
	- causal-lm
	- continued-pretraining
	- experimental
	---

	# Slayer1B 1.08

	Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)

	Slayer1B 1.08 is an experimental base language model continued from Slayer1B 1.07. Training completed on 9 November 2026 after 1,709 updates and 1,792,016,384 processed token positions, using Polish analytical text, English technical text and replay. It is not instruction-tuned and is distinct from the earlier 1.08b ablation.

	[Public training trace](https://track.fabryka.ai/run/77126379-f09d-57b8-a0e6-9d9e4e5f1c7a).

	## Run objective

	Train on as many approved token positions as the fixed compute budget permits, recording actual progress and expense at the spot prices observed during the run. Synthetic speed, real-data training and downstream evaluation are separate measurements. The expense report covers the billed session, including preparation and evaluation; steady-state cost per billion tokens is an extrapolation, not that bill.

	## Model and starting point

	The run initializes from [SlayerLab/Slayer1B-1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07/blob/a090b54e658fb57dc5166c1fcc90cc6bf5c80106/README.md), revision `a090b54e658fb57dc5166c1fcc90cc6bf5c80106`. Its published card describes a Polish-dominant decoder-only model with 1,073,912,851 parameters, a 32,768-token v6 vocabulary, and final checkpoint 22,327. These are base-model properties.

	The continuation preserves that architecture and tokenizer. Its task is causal text continuation, not instruction tuning. The objective is greater exposure to Polish analytical and English technical text while retaining general-language coverage.

	## Frozen launch recipe

	\| Setting \| Selected configuration \|
	\|---\|---\|
	\| Initialization \| Public 1.07 weights; fresh optimizer, rather than optimizer-state resume \|
	\| Trainer and optimizer \| Existing r24.2; Muon with AdamW \|
	\| Batch and context \| 256 sequences × 4,096 token positions \|
	\| Parallel layout \| Eight GPU ranks; microbatch 8 × accumulation 4; activation checkpointing disabled \|
	\| Schedule \| 300 warmup updates, then WSD with decay over the final 25% to a nonzero floor of 0.00006 \|
	\| Checkpoints \| Every 250 updates and at the end; FINAL fixed in advance for evaluation \|

	## Training execution and cost

	\| Observation \| Value and scope \|
	\|---\|---\|
	\| Hardware \| Eight NVIDIA B200 GPUs, Nebius spot; USD 0.99 per GPU-hour \|
	\| Completed updates \| 1,709 \|
	\| Processed positions \| 1,792,016,384 at the stated global batch; approximately 0.64 passes through the prepared input \|
	\| Training window \| 58.6 minutes, including compilation and checkpoints; not the full billed session \|
	\| Steady interval throughput \| Approximately 572,000 positions/s; not positions divided by the whole training window \|
	\| Training loss \| Approximately 2.20 at update 10 to 1.99 at completion; not an evaluation score \|
	\| Training GPU cost \| Approximately USD 7.73, or USD 4.32 per billion processed positions, calculated from the observed window and spot rate \|
	\| Cost through evaluation completion \| Approximately USD 21.5; preparation ≈11.2, training 7.73, evaluation/export ≈2.6. Provider-counter attribution, not a separate settled invoice \|
	\| Complete allocation cost \| Approximately USD 24.1; the allocation was preempted shortly afterwards and released, rather than reused for 1.09 \|

	Input capacity, processed positions, steady interval speed and the complete billed session remain separate quantities.

	## Training inputs

	\| Input \| Pinned revision \|
	\|---\|---\|
	\| [Slayer PL Logika 2B](https://huggingface.co/datasets/SlayerLab/slayer-pl-logika-2b/tree/e24c8b71526ca1dc56a280100f88c2f09a79ecdd) \| `e24c8b71526ca1dc56a280100f88c2f09a79ecdd` \|
	\| [English Technical 1B](https://huggingface.co/datasets/SlayerLab/english-technical-1b/tree/3b833d36bd3ba9aeecf46f39605c9ac849bb96a3) \| `3b833d36bd3ba9aeecf46f39605c9ac849bb96a3` \|
	\| [Slayer 1.07 replay](https://huggingface.co/datasets/SlayerLab/slayer-1.07-pretrain/tree/6a337bbfba518d88f14eb54f7d0a9b9f297df0d1) \| `6a337bbfba518d88f14eb54f7d0a9b9f297df0d1` \|

	Replay uses `pl_web`, `en_fineweb_edu` and `en_mix`: previously seen material, not fresh web. Reference v2 contains 2,800,806,882 native token positions including EOS, after source selection and filtering; this is input capacity, not observed training exposure. The prepared TRAIN digest is `b73a69eaf399bb594300e7d1914a967a38b80b5bc35505fbc6459a6a27be496f`. GPL/GFDL and mixed GPL/LGPL sources are excluded. English validation/test remain outside TRAIN, and Polish held-out sections are grouped by source book.

	## Comparison limits

	A replay-only continued-pretraining control is not budgeted for the priority run. A difference from 1.07 therefore combines data, fresh optimizer initialization and learning-rate decay: 1.07 was released from a constant-LR checkpoint without decay. It does not isolate the new datasets' effect. Inherited exposure in the base model is an additional limitation.

	Evaluation records identify checkpoints, evaluator, split, sample count, prompting, normalization and test-read history.

	## Standard benchmark measurements

	We thank BenchTeam / Fabryka AI for its benchmark implementations, protocol work and evaluation proposals. The team's independent [full benchmark report](https://github.com/slayerlabs/benchmarks-fabryka-ai/blob/8b547007526971bd2fc35dfe56ff2c7d9b802568/research/results/full-run-2026-11/README.md) is a separate measurement source. The table below reports SlayerLab evaluations, also published in the [1.07 training record](https://track.fabryka.ai/run/4973c068-8dd5-59d8-b6c4-671ca3beb925) and the [1.08 training record](https://track.fabryka.ai/run/77126379-f09d-57b8-a0e6-9d9e4e5f1c7a); it is not a copy of that independent run.

	All nine tasks use their full evaluated split, zero-shot likelihood scoring and float32. Scores are percentages with pointwise 95% confidence intervals. `acc` is accuracy from summed continuation likelihood; `acc_norm` is the evaluator's length-normalized accuracy. The metric is named per task rather than combined into a score. WinoGrande uses partial-sentence likelihood; MMLU-PL is micro-averaged over 14,042 questions. Polish HellaSwag and MMLU use the OpenGPTX task definitions.

	For comparison, the continuation columns are 1.09, not additional 1.08 measurements: updates 5,000, 10,000, 15,000 and 20,000 are distinct intermediate working checkpoints of the ongoing [1.09 run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a), not inference-ready weight releases or its eventual budget-limited endpoint. Evaluator SHA, dataset pins, file digests, task protocols and sample counts match across these rows.

	\| Task \| Metric \| n \| 1.07 \| 1.08 \| 1.09, update 5,000 \| 1.09, update 10,000 \| 1.09, update 15,000 \| 1.09, update 20,000 \|
	\|---\|---\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|
	\| ARC-Challenge \| acc_norm \| 1,172 \| 27.30 [24.74, 29.86] \| 30.03 [27.39, 32.68] \| 29.35 [26.79, 32.00] \| 30.46 [27.82, 33.11] \| 32.51 [29.86, 35.24] \| 33.11 [30.46, 35.84] \|
	\| HellaSwag \| acc_norm \| 10,042 \| 43.38 [42.43, 44.34] \| 43.42 [42.43, 44.39] \| 46.53 [45.55, 47.52] \| 48.93 [47.97, 49.91] \| 49.77 [48.80, 50.76] \| 50.38 [49.39, 51.35] \|
	\| PIQA \| acc_norm \| 1,838 \| 66.97 [64.80, 69.15] \| 66.21 [64.15, 68.34] \| 68.28 [66.21, 70.40] \| 68.99 [66.92, 71.11] \| 68.99 [66.87, 71.06] \| 70.78 [68.72, 72.85] \|
	\| WinoGrande \| acc \| 1,267 \| 53.59 [50.83, 56.35] \| 53.67 [50.91, 56.43] \| 54.93 [52.17, 57.70] \| 54.93 [52.17, 57.62] \| 56.75 [54.06, 59.43] \| 55.01 [52.25, 57.77] \|
	\| BoolQ \| acc \| 3,270 \| 58.29 [56.61, 59.97] \| 59.97 [58.26, 61.59] \| 61.80 [60.18, 63.49] \| 61.13 [59.45, 62.78] \| 59.88 [58.20, 61.53] \| 51.41 [49.69, 53.12] \|
	\| OpenBookQA \| acc_norm \| 500 \| 33.60 [29.40, 37.80] \| 34.80 [30.60, 39.00] \| 35.60 [31.40, 40.00] \| 34.40 [30.20, 38.60] \| 36.00 [31.80, 40.20] \| 35.20 [31.00, 39.40] \|
	\| Belebele, Polish \| acc \| 900 \| 26.00 [23.11, 28.89] \| 24.00 [21.22, 26.89] \| 24.11 [21.33, 26.89] \| 24.11 [21.33, 27.00] \| 22.56 [19.89, 25.33] \| 24.67 [21.89, 27.56] \|
	\| HellaSwag-PL \| acc_norm \| 10,042 \| 43.83 [42.84, 44.79] \| 43.90 [42.90, 44.87] \| 44.73 [43.76, 45.71] \| 45.51 [44.51, 46.48] \| 44.18 [43.21, 45.16] \| 44.24 [43.26, 45.22] \|
	\| MMLU-PL \| acc \| 14,042 \| 24.01 [23.30, 24.71] \| 23.04 [22.34, 23.74] \| 24.10 [23.39, 24.82] \| 23.32 [22.62, 24.03] \| 23.50 [22.80, 24.21] \| 23.86 [23.17, 24.58] \|

	Intervals use the percentile bootstrap with 10,000 replicates and seed 20261110. The updated approved aggregate does not supply the random-stream derivation or CI-generator SHA; the previous aggregate documented model-specific streams. Items are treated as independent; subject, template and source clusters are not modeled. These are intervals for individual results, not paired intervals for model differences or estimates of training-seed variance. They do not support a difference claim based on interval overlap.

	[Machine-readable aggregate record](EVALUATION.json): exact scores, intervals, counts, dataset revisions and file digests, checkpoint and evaluator identities, and metric selection. It contains no prompts, answers or per-item predictions. ARC-Challenge, OpenBookQA and MMLU-PL use test splits; HellaSwag, PIQA, WinoGrande, BoolQ and HellaSwag-PL use validation splits; Belebele uses the full Polish evaluation set.

	The approved aggregate SHA-256 is `fc6f67a20bdc6ada687349d35de26f6811e27766dfe1aa197082000c03535796`. The earlier 1.07 and 1.08 scores, intervals and dataset pins are unchanged. The continuation comparison does not extend or replace the separate GSM8K/SST-2 protocol below.

	Descriptive results; contamination scan of the training lineage against these tasks is pending. Belebele and MMLU-PL results are near their four-choice 25% random baseline. No overall benchmark average, ranking or model-release threshold is inferred from this table. Results from other evaluators can differ with prompting, BOS handling, normalization or library version; they should not be silently substituted.

	Earlier internal diagnostics and their measurements remain available in the [versioned evaluation record](https://huggingface.co/SlayerLab/Slayer1B-1.08/blob/5f854a0cc7160594d54d1788d37a44e8ebabe16c/README.md). They are historical measurements, not this card's release criteria.

	## Additional GSM8K and SST-2 measurements

	SlayerLab measured this two-task subset on 11 November 2026 using the [pinned BenchTeam code](https://github.com/slayerlabs/benchmarks-fabryka-ai/tree/8b547007526971bd2fc35dfe56ff2c7d9b802568), lm-eval 0.4.9.1, the `slayer` adapter, float32, batch 1 and context limit 4,096. These are full evaluated splits, not a full 50-task run or a copy of the community's independent report. The evaluated 1.07 and 1.08 weight identities match the completed checkpoints above.

	\| Task \| Metric / prompting \| n \| 1.07, update 22,327 \| 1.08, update 1,709 \|
	\|---\|---\|---:\|---:\|---:\|
	\| GSM8K \| strict exact match, 5-shot \| 1,319 \| 1.52 [0.91, 2.20] \| 0.91 [0.45, 1.44] \|
	\| SST-2 \| accuracy, 0-shot \| 872 \| 51.03 [47.71, 54.36] \| 50.46 [47.13, 53.67] \|

	Scores are percentages with reported pointwise 95% bootstrap intervals (10,000 draws, seed 20261028). These intervals describe individual results, not model differences or training-seed variance. This protocol and its intervals are recorded separately under `additional_measurements` in [EVALUATION.json](EVALUATION.json); the preceding nine-task protocol is unchanged.

	Descriptive results; training-lineage contamination scan pending. GSM8K strict-match scores are near zero. SST-2 zero-shot accuracy is calibration-sensitive; these scores alone do not establish a general reasoning or sentiment-understanding gain. No paired-difference or internal calibration analysis is published here.

	## Licensing and limitations

	Weights are released under Apache-2.0, as 1.07; source corpora keep their own obligations, with attributions in the [data card](DATA_CARD.md). FinePDFs compilation terms and its TDM rationale do not settle rights to every PDF. Book, translation and technical-source notices must be preserved where required.

	A base language model can generate inaccurate, biased, offensive or personal information. Translated reasoning and technical text can contain errors; their presence does not demonstrate reasoning, technical reliability or instruction following. Privacy filtering is heuristic, and technical or tax material is not professional advice.


	## Acknowledgements

	We thank the entire BenchTeam / Fabryka AI community for benchmark implementations, original-protocol documentation, evaluation runs and independent checks. The team is coordinated by [@LoIlita](https://github.com/LoIlita) and [@Elwenor](https://github.com/Elwenor), as documented in the repository's [contribution guide](https://github.com/slayerlabs/benchmarks-fabryka-ai/blob/8b547007526971bd2fc35dfe56ff2c7d9b802568/CONTRIBUTING.md).

	The following accounts contributed to [benchmarks-fabryka-ai](https://github.com/slayerlabs/benchmarks-fabryka-ai). This is a summary of documented contributions, not a ranking or a complete membership list.

	\| GitHub account \| Documented contribution \|
	\|---\|---\|
	\| [@LoIlita](https://github.com/LoIlita) \| Team coordination and onboarding; evaluation workflow, methodology and benchmark reviews ([PR #92](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/92), [PR #185](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/185)). \|
	\| [@Elwenor](https://github.com/Elwenor) \| Team coordination; EWoK implementation, evaluation runner, multi-model benchmark run and reports ([PR #93](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/93), [PR #179](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/179)). \|
	\| [@adgr3](https://github.com/adgr3) \| Original-benchmark cards, numerical-verdict validation and dataset-licence review ([PR #94](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/94), [PR #144](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/144), [PR #175](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/175)). \|
	\| [@gzawadzki](https://github.com/gzawadzki) \| Slayer integration with lm-eval and WinoGrande evaluation ([PR #64](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/64), [PR #100](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/100)). \|
	\| [@KateMajzel](https://github.com/KateMajzel) \| MathQA and PubMedQA evaluation work ([PR #158](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/158), [PR #156](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/156)). \|
	\| [@jakublvndsky](https://github.com/jakublvndsky) \| HellaSwag reproduction and Slayer evaluation ([PR #149](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/149)). \|
	\| [@janbanot](https://github.com/janbanot) \| ARC-Easy original-benchmark card ([PR #70](https://github.com/slayerlabs/benchmarks-fabryka-ai/pull/70)). \|

	We invite BenchTeam, DatasetTeam and other contributors to propose datasets for subsequent Slayer training runs. The collaboration aims to assemble a DatasetTeam corpus of one trillion (1T) tokens. Source proposals should include provenance, version, language and domain coverage, applicable rights and known privacy or evaluation-overlap risks. This is a collection objective, not a claim that the corpus has already been assembled or used in Slayer1B 1.08.