provenance/SOURCE_LOG.md

Source Provenance Log — Bertose Nature Methods Paper

Every fact, number, and claim in the paper is logged here with its source file/path. This is a living document — updated as new information is added to the draft.

---

Pre-training Data

Claim	Value	Source	Verified
Pre-training train split	203,785	nova:/work/ratul1/supantha/glycan-SD-VS/bert_training_v3/v3.1_cluster_training/data/train_val_split.json → train_indices: 203785 entries	✅ cluster read
Pre-training val split	50,947	Same file → val_indices: 50947 entries	✅ cluster read
Total pre-training (train+val)	254,732	Computed: 203,785 + 50,947	✅
Total BPE corpus	260,703	data/sequences_bpe.pkl → len(pickle.load(...))	✅ cluster python
Excluded BPE corpus (v5b)	259,405	bert_v5b_excluded/data/sequences_bpe_excluded.pkl → len	✅ cluster python
Corpus pipeline	260,703 → 254,732 (quality filter) → 259,405 (leakage exclusion)	Computed from above	✅
MS coverage	~1% of glycans	multimodal_config_v5b.yaml → ms_coverage: 1.0	✅ cluster read
3D structure coverage used in the live manuscript	9,922 glycans (3.8% of corpus)	writing/supplementary/table_s3_multimodal_coverage.csv	✅ local submission table
BPE vocabulary size	~2,200 tokens	multimodal_config_v5b.yaml → vocab_size: 2200	✅ cluster read
Benchmark glycans excluded from pre-training	1,298	bench/BENCHMARK_SUMMARY.md line ~12 + writing/NAMING_CONVENTION.md	✅ local file
IPA tokens resolved	206,433	multimodal_config_v5_bpe_topo.yaml → multimodal_note: "0.8 threshold, 206K tokens resolved"	✅ cluster read

Architecture

Claim	Value	Source	Verified
Sequence encoder: 12L/768d/12H	12 layers, 768 hidden, 12 heads	bert_v5b_excluded/multimodal_config_v5b.yaml → sequence: block	✅ cluster read
Intermediate size	3,072	Same config	✅
Max sequence length	256 tokens	Same config → max_length: 256	✅
MS encoder: 6L/384d/6H	6 layers, 384 hidden, 6 heads	Same config → mass_spectrometry: block	✅
3D encoder: 8L/512d/8H	8 layers, 512 hidden, 8 heads	Same config → structure_3d: block	✅
Cross-attention heads	8	Same config → cross_attention: num_heads: 8	✅
Fusion strategy	2-layer MLP, concatenation	Same config → fusion: block	✅
Total parameters	136,274,305 (~136M)	checkpoints_v5b_excluded/best_v5b_excluded_model.pt → torch.load param count	✅ cluster checkpoint
seq_layers	85,054,464 (62.4%)	Same checkpoint → state_dict prefix grouping	✅
struct_layers	25,219,072 (18.5%)	Same	✅
ms_layers	10,646,784 (7.8%)	Same	✅
seq_embeddings	5,443,072 (4.0%)	Same	✅
fusion_layer	2,362,368 (1.7%)	Same	✅
cross_attention	1,970,688 (1.4%)	Same	✅
seq_mlm_head + ms_mlm_head + struct_mlm_head	3,157,282 (2.3%)	Same	✅
projections + dist/distance heads	796,417 (0.6%)	Same	✅
CNN frontend	kernel size 3	Same config → cnn_kernel_size: 3	✅

Training Hyperparameters (V5b — Production Model)

Claim	Value	Source	Verified
Batch size	128	multimodal_config_v5b.yaml → batch_size: 128	✅ cluster read
Learning rate	5×10⁻⁵	Same → learning_rate: 5.0e-5	✅
Optimizer	AdamW (β₁=0.9, β₂=0.999, ε=1e-8)	Same → optimizer: "AdamW" + β params	✅
Weight decay	0.01	Same → weight_decay: 0.01	✅
LR scheduler	Linear warmup + cosine decay	Same → lr_scheduler: "linear_warmup_cosine_decay"	✅
Warmup steps	5,000	Same → warmup_steps: 5000	✅
Max epochs	100	Same → max_epochs: 100	✅
Early stopping patience	15	Same → early_stopping_patience: 15	✅
Gradient clipping	max norm 1.0	Same → max_grad_norm: 1.0	✅
Mixed precision (AMP)	Enabled	Same → use_amp: true	✅
Stopping epoch	77 (early stopped)	checkpoints_v5b_excluded/best_v5b_excluded_model.pt → ckpt['epoch'] = 77	✅ cluster checkpoint
Global step at stop	126,516	Same checkpoint → ckpt['global_step']	✅
Best validation loss	19.66	Same checkpoint → ckpt['best_val_loss']	✅
Matching leakage-proof training log	logs_v5b_excluded/training_20260201_213617.log	cluster SCP to local writing/cluster_logs/	✅ log recovered
Best logged validation epoch	63	Parsed from training_20260201_213617.log	✅ local log parse
Best logged validation loss	19.657871730221903	Same log; exactly matches checkpoint best_val_loss	✅ log/checkpoint match
Last logged epoch	78	Same log	✅
Patience-consistent stop	Best epoch 63 + 15 epochs without improvement -> checkpoint stores epoch=77, epochs_without_improvement=15	log + checkpoint comparison	✅
Training duration	~28 hours	Inferred from checkpoint timestamps (Feb 1 23:31 → Feb 3 03:27)	✅ cluster ls -l
Masking probability	0.15	Same → mask_prob: 0.15	✅

Loss Weights (V5b)

Component	Weight	Source	Verified
Sequence MLM	0.50	multimodal_config_v5b.yaml → loss_weights: sequence: 0.50	✅
Topology distance active coefficient	0.25	Executable path / corrected runnable release config -> dist_loss_weight: 0.25	✅
Original checkpoint raw topology field	0.45	Stored checkpoint config and historical cluster config copy -> dist_loss_weight: 0.45	✅ provenance only
3D structure	2.5	Same → structure_3d: 2.5	✅
MS	13.5	Same → ms: 13.5	✅
Best-epoch val seq loss (leakage-proof log)	0.23753865399017124	training_20260201_213617.log epoch 63	✅ local log parse
Best-epoch val MS loss (leakage-proof log)	1.2365027339991383	Same	✅
Best-epoch val 3D loss (leakage-proof log)	1.13805541550291	Same	✅
Best-epoch val topology loss (leakage-proof log)	0.0047078563529525524	Same	✅

2026-05-06 execution-path caveat: the original checkpoint metadata and historical cluster config preserve dist_loss_weight: 0.45, but the training run used the model-code topology default 0.25 because the trainer did not pass the YAML field. Recomputing the best logged validation loss with topology 0.25 reproduces the checkpoint/log total within numerical precision, while 0.45 does not. The runnable code release now sets and passes dist_loss_weight: 0.25 explicitly so the released config matches the active topology coefficient. Do not report 40%, 25%, 20%, 15% as verified effective contributions.

Benchmark Results (Table 1)

Claim	Value	Source	Verified
#1 overall ranking	Avg rank 4.09 across 11 tasks	writing/sections/06_tables_figures_extended.md Table 1 / Extended Data Table 1	✅ live manuscript
24 models benchmarked	24	bench/BENCHMARK_SUMMARY.md line 4	✅
Bertose Domain MCC	.863	bench/BENCHMARK_SUMMARY.md line 100	✅
Bertose Kingdom MCC	.901	Same line	✅
Bertose Immunogenicity MCC	.898	Same line	✅
Bertose Linkage MCC	.948	Same line	✅
Bertose Interaction ρ	.296	writing/sections/06_tables_figures_extended.md Table 1 / Extended Data Table 1	✅ live manuscript
CompGCN Avg Rank	5.09	writing/sections/06_tables_figures_extended.md Table 1 / Extended Data Table 1	✅ live manuscript
RGCN Avg Rank	4.45	Same file line 73	✅
GlycanAA Avg Rank	5.18	Same file line ~76	✅
GearNet-Edge Avg Rank	7.09	Same file line ~78	✅
GIFFLAR Avg Rank	8.91	Same file line ~82	✅

Benchmark Data Splits

Claim	Value	Source	Verified
Classification train	9,969	Local grep of v3.1_cluster_training MD files (task/data split docs)	⚠️ from local MD
Classification val	1,199	Same	⚠️ from local MD
Classification test	843	Same	⚠️ from local MD
Interaction train	429,569	Same	⚠️ from local MD
Interaction val	57,139	Same	⚠️ from local MD
Interaction test	61,486	Same	⚠️ from local MD

Archived analysis references

Claim	Value	Source	Verified
Probe result archive	local historical analyses retained outside the live manuscript	bert_v6_contrastive/analysis/probe_results_v6/probe_results.md	✅ local file
Use in current paper	not cited in the live three-result manuscript	writing/sections/03_results.md, writing/sections/04_discussion.md	✅ local file

Naming / Model Identity

Claim	Value	Source	Verified
Internal v5b = paper "Bertose"	naming convention	writing/NAMING_CONVENTION.md	✅ user-approved
BERT_v5b_MultiSeed in CSV	row key for our model	bench/CONSOLIDATED_ALL_22_MODELS_v4.csv line 255+	✅ local file

Figure 1 provenance notes

Claim	Value	Source	Verified
Current safest Figure 1 base	writing/figures/figure1/restored_previous_base/figure1_restored_previous_base.png	restored from the preferred preserved edit chain on April 5, 2026	✅ local file
Figure 1b motif panel status	illustrative / hybrid-only, not final tokenizer evidence	writing/FIGURE_1B_EVIDENCE_LOCK.md	✅ local file
Red fucose triangle orientation in Figure 1b	no evidence found that left/right triangle lean changes SNFG meaning; symbol identity is the important rule	NCBI SNFG 2.0 + SNFG notes + GlycoGlyph / GlycoDraw references	✅ web + local review
Figure 1c manuscript-facing terms	depth, ordering, branch index	writing/sections/05_methods.md line 19	✅ local file
Currently exposed local token metadata arrays	branch_depths, linkage_types	update_sequences_with_tree_info.py, bert_training_v4/downstream_tasks/utils/wurcs_bpe_tokenizer.py, model/multimodal_glycan_bert_v3.py	✅ local file
Figure 1c audited example payload	writing/figures/figure1/c_truth_backed/figure1c_example_payload.json	generated from writing/figures/_build/derive_figure1c_example.py	✅ local derivation
Figure 1c standalone SNFG-colored replacement	writing/figures/figure1/c_standalone_snfg/panel_c_standalone_snfg_1180x666.png	generated from writing/figures/_build/render_figure1c_standalone_snfg.py using one real branched WURCS example plus a manuscript-faithful depth / ordering / branch index derivation	✅ local derivation

---

V5-A Config (IPA Stage, Before Leakage Exclusion)

Claim	Value	Source	Verified
V5-A config location	bert_v5_bpe_topo/multimodal_config_v5_bpe_topo.yaml	cluster find	✅
V5-A version label	v5_bpe_topo	Same config → version: "v5_bpe_topo"	✅
V5-A data	sequences_bpe_expanded.pkl (all glycans, IPA-expanded)	Same config → data: sequences:	✅
V5-A vs V5b difference	V5-A includes all glycans; V5b excludes 1,298 benchmark glycans	Comparison of config data paths	✅
Architecture same as V5b	Yes	Config comparison (identical arch blocks)	✅
IPA distillation script	bert_v5_bpe_topo/ipa_bpe_distillation.py	cluster ls	✅
V5-A IPA checkpoint location	checkpoints_v5_bpe_topo/ (top-level)	cluster ls — note: bert_v5_bpe_topo/checkpoints_v5_bpe_topo/ is EMPTY	✅
V5-A last epoch saved	epoch 95	checkpoints_v5_bpe_topo/checkpoint_epoch_95.pt (Jan 26 13:11)	✅
Matching full-corpus training log	logs_v5_bpe_topo/training_20260125_003751.log	cluster SCP to local writing/cluster_logs/	✅ log recovered
V5-A checkpoint metadata	epoch=94, global_step=154850, best_val_loss=20.35809424475712, epochs_without_improvement=1	cluster torch.load(checkpoint_epoch_95.pt)	✅
Best logged validation epoch	94	Parsed from training_20260125_003751.log	✅ local log parse
Best logged validation loss	20.35809424475712	Same log; exactly matches checkpoint best_val_loss	✅ log/checkpoint match
Last logged epoch	100	Same log	✅
Best-epoch val seq loss (full-corpus log)	0.21999171783453694	training_20260125_003751.log epoch 94	✅ local log parse
Best-epoch val MS loss (full-corpus log)	1.3026634216600774	Same	✅
Best-epoch val 3D loss (full-corpus log)	1.0644906900238758	Same	✅
Best-epoch val topology loss (full-corpus log)	0.003661931324613226	Same	✅
V5-A training duration	~35 hours	Checkpoint timestamps (Jan 25 02:32 → Jan 26 13:11)	✅

V5.1 Contrastive Checkpoint

Claim	Value	Source	Verified
V5.1 contrastive checkpoint	bert_v5.1_contrastive/checkpoints_V51_FIXED_V4/best_v51_contrastive_model.pt	cluster find	✅
V5.1 contrastive (FIXED V4) total params	138,529,921 (~138.5M)	torch.load param count	✅ cluster checkpoint
Extra params vs v5b	+2,255,616 (contrastive projection head)	checkpoint proj_head_state_dict key + larger distance_head	✅
V5.1 contrastive (FIXED V4) epoch	2	ckpt['epoch']	✅
V5.1 iterations	5+ bugfix rounds (V2–V5, FINAL)	cluster find -type d -name checkpoints_V51*	✅
V5.1 MCNS (negative selection)	Monte Carlo Negative Selection with biology-grounded rules	internal contrastive-training notes and release scripts	✅
V5.1 old (non-FIXED) checkpoints	checkpoints/ dir has epochs 35–50 + best	cluster find	✅

V6 Contrastive Checkpoint

Claim	Value	Source	Verified
V6 contrastive checkpoint	checkpoints_v6/phase_3_hard_checkpoint.pt (top-level)	cluster ls	✅
V6 total params	138,529,921 (~138.5M, same as V5.1)	torch.load model_state_dict	✅ cluster checkpoint
V6 epoch	30	ckpt['epoch']	✅
V6 phase	phase_idx present (curriculum training, 3 phases)	ckpt.keys()	✅
V6 trainer	contrastive_trainer_v6_curriculum.py	bert_v6_contrastive/training/ dir	✅
V6 checkpoint size	1.3 GB	ls -lah checkpoints_v6/	✅
V6 checkpoint date	Feb 8 14:05	file timestamp	✅

⚠️ Items Needing Further Verification

1. Benchmark classification splits — sourced from local MD grep, not directly from data files. Should cross-check with actual data loaders or cluster split files.
2. V5.1 contrastive config — config directory on cluster was empty (bert_v5.1_contrastive/config/). V5.1/V6 contrastive training uses contrastive_trainer_v6_curriculum.py inline parameters rather than YAML configs.
3. V5-A checkpoint loading — V5-A epoch/param info was inferred from file listing rather than direct checkpoint loading. This does not affect the released BERTose v5b encoder, BERTose IAR resolver, or AFFINose checkpoints.