File size: 2,009 Bytes
8635acd
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
{
 "version": "run3-v1",
 "config": "SPAN",
 "stage": "SPAN",
 "max_len": 2048,
 "base": "answerdotai/ModernBERT-large",
 "input": ":\n  C0aL    C0L (deberta-v3-large, lr 1e-5, micro-batch 4 x 4, gradient checkpointing) + C0a's augmentations:\n          A3 citation-deleted copies of F's citing sentences (tagged non-citing, nearest OTHER citation as context)\n          + paraphrase swap p=0.3.\n  C0Lctx  C0L with the neighbouring sentences in the input: \"... Candidate: <s_i> Before: <s_i-1> After: <s_i+1>\";\n          max_len 448.\n  SPAN    span tagger: answerdotai/ModernBERT-large reads \"[CLS] Figure F. <caption> [SEP]\" followed by up to 2,048\n          tokens of the paper, one [MASK] marker before every sentence; a linear head on each marker gives that\n          sentence's logit for F. Sentences carry inline structure: \"\u00a7 <section>\" when the section changes, \"\u00b6\" at\n          each paragraph start, \"[cites this figure]\" / \"[cites Figure g]\" flags. Windows of sentences overlap by\n          a third and tile the WHOLE paper (every sentence scored; logits averaged over windows). Training keeps\n          every window that holds a citation or a gold sentence of F, and 30% of the others; A3 (a citing\n          sentence of F replaced by its citation-deleted copy, flag removed, p=0.3) and paraphrase swap p=0.3.\n          Loss = BCE over markers, pos_weight = neg/pos capped at 8; 5 epochs; batch 8 (1 window x 8 accumulation);\n          lr 2e-5; fp16 autocast, gradient checkpointing.\nRobustness: a failed job is logged (\"FAILED\"), marked in failed/, its GPU memory released, and the worker carries\non; re-running launch retries failed jobs.\nEnv: R1_SEEDS R1_EPOCHS R1_OUT; R1_SMOKE=1 = quick check (2 folds, 1 seed, 1 epoch, 40 sentences per paper, tiny\n     models); add R1_SMOKE_REAL=1 to use the real large models and 2,048-token windows (GPU memory / fp16 check).",
 "decision": "per-article z-score of the logit >= threshold (threshold from phase8.py)"
}