File size: 3,525 Bytes
89dec6b 9801f83 89dec6b 9801f83 89dec6b 9801f83 89dec6b 9801f83 89dec6b 9801f83 89dec6b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | # Supplementary: tokenizer characterization benchmark
Code, data, and results behind Supplementary Figure 2:
- **(a)** motif preservation — perfect match rate and motif fragmentation
- **(b)** Jensen–Shannon distance between functional-region token distributions and a matched random genomic background
Tokenizers compared: DNABERT-2 BPE, Grover BPE, cCRE-guided BPE, motif-guided BPE, DNAMotifTokenizer.
## Layout
### Shared
- `tokenizers/` — all five tokenizers used in both panels
- `DNAbert2_tokenizer.json`, `Grover_tokenizer.json` — original DNABERT-2 / Grover tokenizers
- `cCRE_region_BPE_tokenizer.json`, `motif_region_BPE_tokenizer.json`
- `DNAMotifTokenizer/` — trie, motif list, vocab (loaded via `code/dna_motif_tokenizer.py`, seed = 42)
- `code/dna_motif_tokenizer.py` — wrapper exposing DNAMotifTokenizer through a `tokenizers`-like interface
### Panel a — motif preservation
- `data/motifs.txt` — 904 unique motifs; wildcard (`*`) motifs are expanded to all A/C/G/T variants (≤5 wildcards), giving 3,737 test sequences of length 5–12
- `code/motif_coverage_eval.py` — tokenizes each variant (no special tokens) and computes per-tokenizer metrics
- `results/summary_by_tokenizer.csv` — per-tokenizer metrics plotted in the figure
- `results/motif_variant_results.csv`, `results/consistency_by_motif.csv` — per-variant and per-motif detail
- `code/plot_supp_fig2a_motif.py` → `figures/supp_fig2a_motif.{pdf,png}`
Metrics:
- **Perfect match rate** — % of motif variants encoded as exactly one token
- **Avg. tokens per motif** — mean number of tokens per motif variant (lower = less fragmentation)
### Panel b — region vs random-background JS distance
- `data/regions/` — promoter (2 kb), enhancer, and exon interval sets (merged BED)
- `data/background/` — `chrom.sizes` (canonical hg38 chromosomes), `region_union.bed` (union of the three region sets, used as exclusion), `random.bed` (exon intervals shuffled into the non-region genome with `bedtools shuffle -seed 42`, so total bp matches exon)
- `data/fastas/` — hg38 sequences for promoter, enhancer, exon, and random (`bedtools getfasta`)
- `code/js_vs_random_heatmap.py` — builds the background, tokenizes each FASTA, and computes the Jensen–Shannon distance (base 2) between each region's token-frequency distribution and the random background
- `results/js_vs_random_matrix.csv` — 5 tokenizers × 3 regions matrix plotted in the figure
- `results/js_vs_random.log` — run log with token counts per region and tokenizer
- `figures/js_vs_random_heatmap.{pdf,png}` — heatmap from `js_vs_random_heatmap.py`
- `code/plot_combined_panel_a_b.py` → `figures/combined_panel_a_b.{pdf,png}` — combined draft of panels a and b
## Reproduce
```
# Panel a
python code/motif_coverage_eval.py --motif-file data/motifs.txt --output-dir results
python code/plot_supp_fig2a_motif.py
# Panel b (requires bedtools and hg38.fa from UCSC)
python code/js_vs_random_heatmap.py # full run
python code/js_vs_random_heatmap.py --plot-only # redraw from results/js_vs_random_matrix.csv
```
`motif_coverage_eval.py`, `js_vs_random_heatmap.py`, and `plot_combined_panel_a_b.py` are uploaded as run and resolve paths relative to the original project layout (`DEFAULT_TOKENIZER_PATHS`, `ROOT`, `REGION_BEDS`, etc.); point them at `tokenizers/` and `data/` before running. The hg38 reference genome is not included (public, ~3 GB); the extracted FASTAs in `data/fastas/` are what the analysis tokenized.
|