README.md

---
license: mit
tags:
  - dna
  - genomics
  - tokenization
  - sequence-classification
---

# Baseline BPE-200K — SFT checkpoints (all 56 tasks)

Fine-tuned classification checkpoints from the **baseline BPE** pretrained model at
step 200,000 (vocab 5,120), for use as the control against
[`EvoLenTokenizer/evolen-200k-sft-checkpoints`](https://huggingface.co/EvoLenTokenizer/evolen-200k-sft-checkpoints).

Base model: [`EvoLenTokenizer/base-200k`](https://huggingface.co/EvoLenTokenizer/base-200k).
Tokenizer: standard BPE, vocab 5,120 — **not** the EvoLen tokenizer.

## Complete and hyperparameter-matched

All 56 tasks are published. Every checkpoint uses the **same learning rate, weight decay,
warmup ratio, epochs, seed, `model_max_length`, precision and checkpoint-selection rule**
as its EvoLen-200k counterpart, so the only differences are the tokenizer and the
pretrained model.

37 of these came from the original baseline runs. The remaining 19 — all of `mouse/*`,
all of `tf/*`, seven `EMP/*`, `splice/reconstructed` and `allchr_csv` — were re-run
specifically to match EvoLen's configuration; several of those had no baseline run at
all, and `allchr_csv` had previously been run at `model_max_length=200` where EvoLen used
512.

Nine GBM tasks were originally trained at `per_device_batch_size=64` against EvoLen's 128;
`coverage.csv` records the config used for every task.

## A second asymmetry to be aware of

Even for the 28 matched tasks, the *selection procedure* differs. EvoLen-200k reports the
maximum test MCC over a sweep of 18–119 configurations per task; the baseline is a single
run at one configuration. Comparing a max-over-many against a single draw favours EvoLen
independently of tokenizer quality. Treat the deltas accordingly.

## Layout

One folder per task, `<SUITE>/<task>/`, each with `model.safetensors`, `config.json`,
`tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`,
`trainer_state.json` and `training_args.bin`. Optimizer and scheduler state are stripped.

```python
from transformers import AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained(
    "EvoLenTokenizer/base-200k-sft-checkpoints",
    subfolder="NT/H3K27ac", trust_remote_code=True)
```

`coverage.csv` lists all 56 tasks with their status (published / batch-size mismatch /
hyperparameter mismatch / no run) and both configs side by side.

## Reproducing

Evaluation only. fp16 autocast, eval batch size from `training_args.bin`, per-task
`model_max_length` from `training_args.bin`, and `sorted(set(...))` label ordering for
multiclass tasks. Tokenize with `padding="longest"`, `truncation=True`; attention mask is
`input_ids.ne(pad_token_id)`; metric is `sklearn.metrics.matthews_corrcoef` over `argmax`
predictions.

Environment: Python 3.9.18, transformers 4.35.2, scikit-learn 1.6.1, torch 2.8.0,
numpy 2.0.2, tokenizers 0.15.2, accelerate 0.25.0.