---
license: apache-2.0
language:
- grc
library_name: transformers
tags:
- ancient-greek
- classical-philology
- character-level
- masked-diffusion
- pretrained
pipeline_tag: fill-mask
---

# Stoicheia -- documentary-clean backbone

**Stoicheia** is a 405M-parameter character-level masked-diffusion encoder for Ancient Greek
(`d_model` 1024, depth 32, banded attention: three of every four blocks attend within a
256-character window, the fourth globally). Its input is factored into five aligned planes --
letters, word/sentence boundaries, diacritics, capitalization, punctuation -- each of which can
be masked independently to an explicit *unknown* state at inference. That is what lets one model
read an edited text, *scriptio continua*, and a lacuna of unknown length without changing
anything but its input.

Anonymous release accompanying a paper under review.

The **documentary-clean** backbone: zero exposure to inscriptions or papyri of any kind. Every
downstream model in this release is fine-tuned from it, so no epigraphic or papyrological result
can be contaminated by pretraining. It is the flagship checkpoint -- use this one unless you
specifically need a literary fold.

## What this checkpoint has not read

Ten literary folds rotate an 80/10/10 split built by a 13-stage pipeline that clusters records
into *editions of the same work* -- by canonical identifier where one exists, by duplicate-aware
content matching where it does not -- and then excises from each fold's training data every
sentence colliding with its held-out zones (exact and reordered duplicates, word 5-grams,
document-level near-duplicates). The guarantee is exact at the fold level: **for any passage of
the open corpus, at least one released checkpoint has provably never seen it**, which is what
makes it possible to interrogate a transmitted text with a model that cannot merely be
recalling it.

`Stoicheia-doc_clean` extends the same discipline to documentary text: every inscription and
papyrus is excluded, along with anything a contamination screen flags as quoting one.

## Usage

```python
import torch
from transformers import AutoModel
from huggingface_hub import hf_hub_download

REPO = "Ericu950/Stoicheia-doc_clean"
model = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()

# `trust_remote_code` loads the model classes, but the processor is a separate helper:
# fetch it into the working directory before importing it.
hf_hub_download(repo_id=REPO, filename="processing_char_bert.py", local_dir=".")
from processing_char_bert import CharBertProcessor

proc = CharBertProcessor()

# a gap of uncertain width in unaccented scriptio continua: "[N±M]" scores every width
# in N-M..N+M by the model's own confidence, and restores accents and word division too
text = "εναρχηηνο[5±3]καιολογοςηνπροστονθεον"
best, width, candidates = proc.restore_elastic(model, text, mask_dia_boundary=True)
print(best)     # ἐν ἀρχῇ ἦν ὁ λόγος, καὶ ὁ λόγος ἦν πρὸς τὸν θεόν.
print(width)    # 5 -- the width the model judged most likely
```