provenance/SOURCE_LOG.md

# Source Provenance Log — Bertose Nature Methods Paper

> Every fact, number, and claim in the paper is logged here with its source file/path.
> This is a living document — updated as new information is added to the draft.

---

## Pre-training Data

| Claim | Value | Source | Verified |
|---|---|---|---|
| Pre-training train split | 203,785 | `nova:/work/ratul1/supantha/glycan-SD-VS/bert_training_v3/v3.1_cluster_training/data/train_val_split.json` → `train_indices: 203785 entries` | ✅ cluster read |
| Pre-training val split | 50,947 | Same file → `val_indices: 50947 entries` | ✅ cluster read |
| Total pre-training (train+val) | 254,732 | Computed: 203,785 + 50,947 | ✅ |
| **Total BPE corpus** | **260,703** | `data/sequences_bpe.pkl` → `len(pickle.load(...))` | ✅ cluster python |
| **Excluded BPE corpus (v5b)** | **259,405** | `bert_v5b_excluded/data/sequences_bpe_excluded.pkl` → len | ✅ cluster python |
| Corpus pipeline | 260,703 → 254,732 (quality filter) → 259,405 (leakage exclusion) | Computed from above | ✅ |
| MS coverage | ~1% of glycans | `multimodal_config_v5b.yaml` → `ms_coverage: 1.0` | ✅ cluster read |
| 3D structure coverage used in the live manuscript | 9,922 glycans (3.8% of corpus) | `writing/supplementary/table_s3_multimodal_coverage.csv` | ✅ local submission table |
| BPE vocabulary size | ~2,200 tokens | `multimodal_config_v5b.yaml` → `vocab_size: 2200` | ✅ cluster read |
| Benchmark glycans excluded from pre-training | 1,298 | `bench/BENCHMARK_SUMMARY.md` line ~12 + `writing/NAMING_CONVENTION.md` | ✅ local file |
| IPA tokens resolved | 206,433 | `multimodal_config_v5_bpe_topo.yaml` → multimodal_note: "0.8 threshold, 206K tokens resolved" | ✅ cluster read |

## Architecture

| Claim | Value | Source | Verified |
|---|---|---|---|
| Sequence encoder: 12L/768d/12H | 12 layers, 768 hidden, 12 heads | `bert_v5b_excluded/multimodal_config_v5b.yaml` → `sequence:` block | ✅ cluster read |
| Intermediate size | 3,072 | Same config | ✅ |
| Max sequence length | 256 tokens | Same config → `max_length: 256` | ✅ |
| MS encoder: 6L/384d/6H | 6 layers, 384 hidden, 6 heads | Same config → `mass_spectrometry:` block | ✅ |
| 3D encoder: 8L/512d/8H | 8 layers, 512 hidden, 8 heads | Same config → `structure_3d:` block | ✅ |
| Cross-attention heads | 8 | Same config → `cross_attention: num_heads: 8` | ✅ |
| Fusion strategy | 2-layer MLP, concatenation | Same config → `fusion:` block | ✅ |
| **Total parameters** | **136,274,305 (~136M)** | `checkpoints_v5b_excluded/best_v5b_excluded_model.pt` → torch.load param count | ✅ cluster checkpoint |
| seq_layers | 85,054,464 (62.4%) | Same checkpoint → `state_dict` prefix grouping | ✅ |
| struct_layers | 25,219,072 (18.5%) | Same | ✅ |
| ms_layers | 10,646,784 (7.8%) | Same | ✅ |
| seq_embeddings | 5,443,072 (4.0%) | Same | ✅ |
| fusion_layer | 2,362,368 (1.7%) | Same | ✅ |
| cross_attention | 1,970,688 (1.4%) | Same | ✅ |
| seq_mlm_head + ms_mlm_head + struct_mlm_head | 3,157,282 (2.3%) | Same | ✅ |
| projections + dist/distance heads | 796,417 (0.6%) | Same | ✅ |
| CNN frontend | kernel size 3 | Same config → `cnn_kernel_size: 3` | ✅ |

## Training Hyperparameters (V5b — Production Model)

| Claim | Value | Source | Verified |
|---|---|---|---|
| Batch size | 128 | `multimodal_config_v5b.yaml` → `batch_size: 128` | ✅ cluster read |
| Learning rate | 5×10⁻⁵ | Same → `learning_rate: 5.0e-5` | ✅ |
| Optimizer | AdamW (β₁=0.9, β₂=0.999, ε=1e-8) | Same → `optimizer: "AdamW"` + β params | ✅ |
| Weight decay | 0.01 | Same → `weight_decay: 0.01` | ✅ |
| LR scheduler | Linear warmup + cosine decay | Same → `lr_scheduler: "linear_warmup_cosine_decay"` | ✅ |
| Warmup steps | 5,000 | Same → `warmup_steps: 5000` | ✅ |
| Max epochs | 100 | Same → `max_epochs: 100` | ✅ |
| Early stopping patience | 15 | Same → `early_stopping_patience: 15` | ✅ |
| Gradient clipping | max norm 1.0 | Same → `max_grad_norm: 1.0` | ✅ |
| Mixed precision (AMP) | Enabled | Same → `use_amp: true` | ✅ |
| **Stopping epoch** | **77** (early stopped) | `checkpoints_v5b_excluded/best_v5b_excluded_model.pt` → `ckpt['epoch']` = 77 | ✅ cluster checkpoint |
| Global step at stop | 126,516 | Same checkpoint → `ckpt['global_step']` | ✅ |
| Best validation loss | 19.66 | Same checkpoint → `ckpt['best_val_loss']` | ✅ |
| Matching leakage-proof training log | `logs_v5b_excluded/training_20260201_213617.log` | cluster SCP to local `writing/cluster_logs/` | ✅ log recovered |
| Best logged validation epoch | 63 | Parsed from `training_20260201_213617.log` | ✅ local log parse |
| Best logged validation loss | 19.657871730221903 | Same log; exactly matches checkpoint `best_val_loss` | ✅ log/checkpoint match |
| Last logged epoch | 78 | Same log | ✅ |
| Patience-consistent stop | Best epoch 63 + 15 epochs without improvement -> checkpoint stores `epoch=77`, `epochs_without_improvement=15` | log + checkpoint comparison | ✅ |
| Training duration | ~28 hours | Inferred from checkpoint timestamps (Feb 1 23:31 → Feb 3 03:27) | ✅ cluster ls -l |
| Masking probability | 0.15 | Same → `mask_prob: 0.15` | ✅ |

## Loss Weights (V5b)

| Component | Weight | Source | Verified |
|---|---|---|---|
| Sequence MLM | 0.50 | `multimodal_config_v5b.yaml` → `loss_weights: sequence: 0.50` | ✅ |
| Topology distance active coefficient | 0.25 | Executable path / corrected runnable release config -> `dist_loss_weight: 0.25` | ✅ |
| Original checkpoint raw topology field | 0.45 | Stored checkpoint config and historical cluster config copy -> `dist_loss_weight: 0.45` | ✅ provenance only |
| 3D structure | 2.5 | Same → `structure_3d: 2.5` | ✅ |
| MS | 13.5 | Same → `ms: 13.5` | ✅ |
| Best-epoch val seq loss (leakage-proof log) | 0.23753865399017124 | `training_20260201_213617.log` epoch 63 | ✅ local log parse |
| Best-epoch val MS loss (leakage-proof log) | 1.2365027339991383 | Same | ✅ |
| Best-epoch val 3D loss (leakage-proof log) | 1.13805541550291 | Same | ✅ |
| Best-epoch val topology loss (leakage-proof log) | 0.0047078563529525524 | Same | ✅ |

> 2026-05-06 execution-path caveat: the original checkpoint metadata and historical cluster config preserve `dist_loss_weight: 0.45`, but the training run used the model-code topology default `0.25` because the trainer did not pass the YAML field. Recomputing the best logged validation loss with topology `0.25` reproduces the checkpoint/log total within numerical precision, while `0.45` does not. The runnable code release now sets and passes `dist_loss_weight: 0.25` explicitly so the released config matches the active topology coefficient. Do not report `40%`, `25%`, `20%`, `15%` as verified effective contributions.

## Benchmark Results (Table 1)

| Claim | Value | Source | Verified |
|---|---|---|---|
| #1 overall ranking | Avg rank 4.09 across 11 tasks | `writing/sections/06_tables_figures_extended.md` Table 1 / Extended Data Table 1 | ✅ live manuscript |
| 24 models benchmarked | 24 | `bench/BENCHMARK_SUMMARY.md` line 4 | ✅ |
| Bertose Domain MCC | .863 | `bench/BENCHMARK_SUMMARY.md` line 100 | ✅ |
| Bertose Kingdom MCC | .901 | Same line | ✅ |
| Bertose Immunogenicity MCC | .898 | Same line | ✅ |
| Bertose Linkage MCC | .948 | Same line | ✅ |
| Bertose Interaction ρ | .296 | `writing/sections/06_tables_figures_extended.md` Table 1 / Extended Data Table 1 | ✅ live manuscript |
| CompGCN Avg Rank | 5.09 | `writing/sections/06_tables_figures_extended.md` Table 1 / Extended Data Table 1 | ✅ live manuscript |
| RGCN Avg Rank | 4.45 | Same file line 73 | ✅ |
| GlycanAA Avg Rank | 5.18 | Same file line ~76 | ✅ |
| GearNet-Edge Avg Rank | 7.09 | Same file line ~78 | ✅ |
| GIFFLAR Avg Rank | 8.91 | Same file line ~82 | ✅ |

## Benchmark Data Splits

| Claim | Value | Source | Verified |
|---|---|---|---|
| Classification train | 9,969 | Local grep of `v3.1_cluster_training` MD files (task/data split docs) | ⚠️ from local MD |
| Classification val | 1,199 | Same | ⚠️ from local MD |
| Classification test | 843 | Same | ⚠️ from local MD |
| Interaction train | 429,569 | Same | ⚠️ from local MD |
| Interaction val | 57,139 | Same | ⚠️ from local MD |
| Interaction test | 61,486 | Same | ⚠️ from local MD |

## Archived analysis references

| Claim | Value | Source | Verified |
|---|---|---|---|
| Probe result archive | local historical analyses retained outside the live manuscript | `bert_v6_contrastive/analysis/probe_results_v6/probe_results.md` | ✅ local file |
| Use in current paper | not cited in the live three-result manuscript | `writing/sections/03_results.md`, `writing/sections/04_discussion.md` | ✅ local file |

## Naming / Model Identity

| Claim | Value | Source | Verified |
|---|---|---|---|
| Internal v5b = paper "Bertose" | naming convention | `writing/NAMING_CONVENTION.md` | ✅ user-approved |
| BERT_v5b_MultiSeed in CSV | row key for our model | `bench/CONSOLIDATED_ALL_22_MODELS_v4.csv` line 255+ | ✅ local file |

## Figure 1 provenance notes

| Claim | Value | Source | Verified |
|---|---|---|---|
| Current safest Figure 1 base | `writing/figures/figure1/restored_previous_base/figure1_restored_previous_base.png` | restored from the preferred preserved edit chain on April 5, 2026 | ✅ local file |
| `Figure 1b` motif panel status | illustrative / hybrid-only, not final tokenizer evidence | `writing/FIGURE_1B_EVIDENCE_LOCK.md` | ✅ local file |
| Red fucose triangle orientation in `Figure 1b` | no evidence found that left/right triangle lean changes SNFG meaning; symbol identity is the important rule | NCBI SNFG 2.0 + SNFG notes + GlycoGlyph / GlycoDraw references | ✅ web + local review |
| `Figure 1c` manuscript-facing terms | `depth`, `ordering`, `branch index` | `writing/sections/05_methods.md` line 19 | ✅ local file |
| Currently exposed local token metadata arrays | `branch_depths`, `linkage_types` | `update_sequences_with_tree_info.py`, `bert_training_v4/downstream_tasks/utils/wurcs_bpe_tokenizer.py`, `model/multimodal_glycan_bert_v3.py` | ✅ local file |
| `Figure 1c` audited example payload | `writing/figures/figure1/c_truth_backed/figure1c_example_payload.json` | generated from `writing/figures/_build/derive_figure1c_example.py` | ✅ local derivation |
| `Figure 1c` standalone SNFG-colored replacement | `writing/figures/figure1/c_standalone_snfg/panel_c_standalone_snfg_1180x666.png` | generated from `writing/figures/_build/render_figure1c_standalone_snfg.py` using one real branched WURCS example plus a manuscript-faithful `depth / ordering / branch index` derivation | ✅ local derivation |

---

## V5-A Config (IPA Stage, Before Leakage Exclusion)

| Claim | Value | Source | Verified |
|---|---|---|---|
| V5-A config location | `bert_v5_bpe_topo/multimodal_config_v5_bpe_topo.yaml` | cluster `find` | ✅ |
| V5-A version label | v5_bpe_topo | Same config → `version: "v5_bpe_topo"` | ✅ |
| V5-A data | `sequences_bpe_expanded.pkl` (all glycans, IPA-expanded) | Same config → `data: sequences:` | ✅ |
| V5-A vs V5b difference | V5-A includes all glycans; V5b excludes 1,298 benchmark glycans | Comparison of config data paths | ✅ |
| Architecture same as V5b | Yes | Config comparison (identical arch blocks) | ✅ |
| IPA distillation script | `bert_v5_bpe_topo/ipa_bpe_distillation.py` | cluster `ls` | ✅ |
| **V5-A IPA checkpoint location** | `checkpoints_v5_bpe_topo/` (top-level) | cluster `ls` — note: `bert_v5_bpe_topo/checkpoints_v5_bpe_topo/` is EMPTY | ✅ |
| V5-A last epoch saved | epoch 95 | `checkpoints_v5_bpe_topo/checkpoint_epoch_95.pt` (Jan 26 13:11) | ✅ |
| Matching full-corpus training log | `logs_v5_bpe_topo/training_20260125_003751.log` | cluster SCP to local `writing/cluster_logs/` | ✅ log recovered |
| V5-A checkpoint metadata | `epoch=94`, `global_step=154850`, `best_val_loss=20.35809424475712`, `epochs_without_improvement=1` | cluster `torch.load(checkpoint_epoch_95.pt)` | ✅ |
| Best logged validation epoch | 94 | Parsed from `training_20260125_003751.log` | ✅ local log parse |
| Best logged validation loss | 20.35809424475712 | Same log; exactly matches checkpoint `best_val_loss` | ✅ log/checkpoint match |
| Last logged epoch | 100 | Same log | ✅ |
| Best-epoch val seq loss (full-corpus log) | 0.21999171783453694 | `training_20260125_003751.log` epoch 94 | ✅ local log parse |
| Best-epoch val MS loss (full-corpus log) | 1.3026634216600774 | Same | ✅ |
| Best-epoch val 3D loss (full-corpus log) | 1.0644906900238758 | Same | ✅ |
| Best-epoch val topology loss (full-corpus log) | 0.003661931324613226 | Same | ✅ |
| V5-A training duration | ~35 hours | Checkpoint timestamps (Jan 25 02:32 → Jan 26 13:11) | ✅ |

## V5.1 Contrastive Checkpoint

| Claim | Value | Source | Verified |
|---|---|---|---|
| V5.1 contrastive checkpoint | `bert_v5.1_contrastive/checkpoints_V51_FIXED_V4/best_v51_contrastive_model.pt` | cluster `find` | ✅ |
| V5.1 contrastive (FIXED V4) total params | 138,529,921 (~138.5M) | torch.load param count | ✅ cluster checkpoint |
| Extra params vs v5b | +2,255,616 (contrastive projection head) | checkpoint `proj_head_state_dict` key + larger `distance_head` | ✅ |
| V5.1 contrastive (FIXED V4) epoch | 2 | `ckpt['epoch']` | ✅ |
| V5.1 iterations | 5+ bugfix rounds (V2–V5, FINAL) | cluster `find -type d -name checkpoints_V51*` | ✅ |
| V5.1 MCNS (negative selection) | Monte Carlo Negative Selection with biology-grounded rules | internal contrastive-training notes and release scripts | ✅ |
| V5.1 old (non-FIXED) checkpoints | `checkpoints/` dir has epochs 35–50 + best | cluster `find` | ✅ |

## V6 Contrastive Checkpoint

| Claim | Value | Source | Verified |
|---|---|---|---|
| V6 contrastive checkpoint | `checkpoints_v6/phase_3_hard_checkpoint.pt` (top-level) | cluster `ls` | ✅ |
| V6 total params | 138,529,921 (~138.5M, same as V5.1) | torch.load `model_state_dict` | ✅ cluster checkpoint |
| V6 epoch | 30 | `ckpt['epoch']` | ✅ |
| V6 phase | phase_idx present (curriculum training, 3 phases) | `ckpt.keys()` | ✅ |
| V6 trainer | `contrastive_trainer_v6_curriculum.py` | `bert_v6_contrastive/training/` dir | ✅ |
| V6 checkpoint size | 1.3 GB | `ls -lah checkpoints_v6/` | ✅ |
| V6 checkpoint date | Feb 8 14:05 | file timestamp | ✅ |

## ⚠️ Items Needing Further Verification

1. **Benchmark classification splits** — sourced from local MD grep, not directly from data files. Should cross-check with actual data loaders or cluster split files.
2. **V5.1 contrastive config** — config directory on cluster was empty (`bert_v5.1_contrastive/config/`). V5.1/V6 contrastive training uses `contrastive_trainer_v6_curriculum.py` inline parameters rather than YAML configs.
3. **V5-A checkpoint loading** — V5-A epoch/param info was inferred from file listing rather than direct checkpoint loading. This does not affect the released BERTose v5b encoder, BERTose IAR resolver, or AFFINose checkpoints.