---
pretty_name: Slayer1B 1.08 training-data composition, reference v2
language:
- pl
- en
task_categories:
- text-generation
tags:
- continued-pretraining
- prepared-reference
---

# Slayer1B 1.08 — training-data composition

**Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**

**Prepared reference v2, 9 November 2026.** Target reconstruction must match the approved digests before training. This documents the new 1.08 composition, not a separately hosted mixed-corpus release; the earlier 1.08b experiment is distinct.

## Prepared training reference

The composition combines analytical/reasoning-oriented text, English technical material and general-language replay. Retained counts come from the reference assembly manifest; they describe available input positions, not tokens consumed by a training run. Component names identify sources, not an audited per-document language distribution.

| Retained component | Native token positions including EOS |
|---|---:|
| PL Logika | 1,631,862,622 |
| English technical | 293,943,541 |
| Replay `pl_web` | 435,000,183 |
| Replay `en_fineweb_edu` | 220,000,324 |
| Replay `en_mix` | 220,000,212 |
| **TRAIN total** | **2,800,806,882** |

Reference manifest SHA256: `c0f1e7771104f60b92ec38d3576fcfe34ed125a072da529d8b56fe2da0f87e7b`. TRAIN SHA256: `b73a69eaf399bb594300e7d1914a967a38b80b5bc35505fbc6459a6a27be496f`.

| Separate partition | Documents | Token positions including EOS |
|---|---:|---:|
| Polish-logic held-out | 3,158 | 18,294,311 |
| Trainer validation, a subset of that held-out | 360 | 2,002,010 |
| Eligible English VALID held-out | 5,218 | 17,988,388 |

These are partition totals, not domain-evaluation sample sizes.

## Published inputs before filtering

The following source-published counts use their stated split and EOS units.

| Source / revision | Split | Documents | Native Slayer tokens |
|---|---|---:|---:|
| [PL Logika](https://huggingface.co/datasets/SlayerLab/slayer-pl-logika-2b/tree/e24c8b71526ca1dc56a280100f88c2f09a79ecdd), `e24c8b71526ca1dc56a280100f88c2f09a79ecdd` | published train | 327,219 | 1,911,918,958, including EOS |
| [English Technical](https://huggingface.co/datasets/SlayerLab/english-technical-1b/tree/3b833d36bd3ba9aeecf46f39605c9ac849bb96a3), `3b833d36bd3ba9aeecf46f39605c9ac849bb96a3` | train | 188,904 | 899,001,323, before pack-added EOS |
| Same English release | valid | 10,370 | 49,886,904, before pack-added EOS |
| Same English release | test | 10,665 | 51,226,632, before pack-added EOS |

The English all-split total, 1,000,114,859 tokens, is not training size. Its pack manifest adds one EOS per document: source TRAIN becomes **899,190,227 token positions**. Proxy cl100k counts are a different unit. Source counts, retained reference positions and observed training exposure are distinct quantities.

Replay comes from [Slayer 1.07 pretraining data](https://huggingface.co/datasets/SlayerLab/slayer-1.07-pretrain/tree/6a337bbfba518d88f14eb54f7d0a9b9f297df0d1), revision `6a337bbfba518d88f14eb54f7d0a9b9f297df0d1`: `pl_web`, `en_fineweb_edu` and `en_mix`. These texts were seen by 1.07; retained quotas are shown above.

## Composition and rights

Licenses differ by source; compilation terms do not grant rights to every underlying text. Reference selection excludes GPL/GFDL sources and mixed GPL/LGPL English sources. Retained components still carry their individual terms, not a blanket permissive-license grant.

### Polish corpus


Required conditions remain source-specific:

- **FinePDFs:** retain the notice “Contains information from FinePDFs, which is made available under the ODC Attribution License,” applicable Common Crawl terms and annotation attribution. PDF-content rights remain unresolved; upstream relies on a TDM rationale and did not recheck domain-level opt-outs.
- **Dolci:** preserve original Dolci, translation and component notices, including NVIDIA Nemotron CC BY 4.0. The translation license does not replace underlying obligations. Attribution identifies Gemma translation and mixed upstream reasoning teachers; embedded conversation-history provenance remains uncertain.
- **DOAB:** preserve each book's creator/editor, title, publisher, license, URL and modification notices. Selected material includes English books, so the corpus label is not a per-document language guarantee.
- **Tax material:** preserve Romuald Członkowski and Eureka MF/KIS attribution. Upstream did not locate the portal terms; absence of known restrictions is not confirmed permission.

The pinned [attribution inventory](https://huggingface.co/datasets/SlayerLab/slayer-pl-logika-2b/blob/e24c8b71526ca1dc56a280100f88c2f09a79ecdd/ATRYBUCJA.md) carries the full notices; this summary does not replace them.

### English technical corpus and replay

English sources span systems, machine learning, compilers, CNC/CAM, simulation and graphics. Row licenses and matching notices govern selection: excluded GPL sources include Linux, Blender, OpenFOAM and some CNC material; retained entries use MIT, Apache-2.0, Apache-2.0 WITH LLVM-exception, BSD-3-Clause, dual Apache/MIT or the FreeCAD repository's CC0 terms. The [source-license inventory](https://huggingface.co/datasets/SlayerLab/english-technical-1b/blob/3b833d36bd3ba9aeecf46f39605c9ac849bb96a3/licenses/SOURCES.md) records the notices. The retained technical count above reflects all filters, not just license exclusion.

FreeCAD documentation: **FreeCAD wiki contributors**, via `FreeCAD/FreeCAD-documentation` at `0499378a`, repository license **CC0-1.0**. This identifies the conversion repository's license; the underlying wiki attribution is retained.

Replay includes ODC-By FineWeb-Edu/FineMath/open-web-math, CC BY DCLM and HPLT. HPLT's compilation is CC0, not its underlying web texts; the replay publisher relies on Directive (EU) 2019/790 Article 4. Redistribution requires separate rights review. Required notices follow the pinned replay card.

## Preparation and split isolation

Preparation uses the same Slayer v6 tokenizer and little-endian uint16 token format, with EOS 0 after each document.

Reference TRAIN excludes whole documents flagged by the contact/identifier filter or containing literal special-token IDs in their text. Assembly metadata records removal counts and overlapping filters.

English valid/test remain outside TRAIN; eligible VALID uses the TRAIN license exclusions and removes documents also present in TRAIN. Polish held-out sections are grouped by source-book identity rather than splitting chapters across train and evaluation.

Target reconstruction uses the pinned preparation recipe, exclusion lists and the same input revisions. TRAIN, validation and held-out digests must match the approved reference artifacts before use.

## Quality, privacy and limitations

The reference uses recorded per-row identifier/overlap exclusions for new sources and inherits the replay release's previous checks. Legacy replay did not detect near-duplicates, omitted EWoK from its scan and retained some short overlaps. These limitations and exposure inherited through the base model remain relevant to evaluation.

Whole-document removal reduces FinePDFs coverage, including academic text with author contact details. Residual names, addresses, translation artifacts, PDF-layout damage and tax-text fragmentation can remain. Polish metadata reports no toxicity classifier and masking exceptions in programming/math contexts. Technical text can contain unsafe instructions; reasoning traces can be wrong. Domain labels and source counts establish neither model capability nor safe professional advice.

## Source metadata

Further pinned evidence: Polish [aggregate manifest](https://huggingface.co/datasets/SlayerLab/slayer-pl-logika-2b/blob/e24c8b71526ca1dc56a280100f88c2f09a79ecdd/manifest.json); English [aggregate manifest](https://huggingface.co/datasets/SlayerLab/english-technical-1b/blob/3b833d36bd3ba9aeecf46f39605c9ac849bb96a3/manifest.json) and [EOS-inclusive pack manifest](https://huggingface.co/datasets/SlayerLab/english-technical-1b/blob/3b833d36bd3ba9aeecf46f39605c9ac849bb96a3/slayer_v6/manifest.json). This documentation was prepared from public cards and aggregate/license metadata, not raw records or token binaries.