Samuka007/reclor-qa-from-scratch-10m

ReClor QA: decomposable-attention encoder + R-Drop (from-scratch, 10M params)

Course project (CITS4012 A2). Transformer-encoder model trained from scratch (random init, BPE tokenizer trained on the train split only) for 4-way multiple-choice logical reasoning on ReClor.

• Architecture: decomposable cross-attention (attend/compare/aggregate) + R-Drop consistency.
• Params: 10,020,353. Seeds: 42 / 0 / 1.
• Test accuracy (official ReClor validation split, n=500): 0.366 +/- 0.021 over 3 seeds (random = 0.250; BERT-base baseline = 0.498).

Files

• tokenizer.json — byte-level BPE (16k), trained on the train-portion only.
• best_seed{42,0,1}.pt — best-dev checkpoints (state_dict).

Verify

import hashlib, pathlib
for f, h in {
    "tokenizer.json": "41648afb488a9c2dbf916ed3ed0c14a5b27feb5864e61d5ccedb058d2b06c202",
    "best_seed42.pt": "ac36423e6638c8d23ddf0924616115eb0e2cdf230fbe7af90921bde47b3a7493",
}.items():
    assert hashlib.sha256(pathlib.Path(f).read_bytes()).hexdigest() == h