utTapVWtc7 / evidence /claim5 /limitations.md
DineshAI's picture
Add cumulative five-claim audit and standalone verifiers
f791e67 verified
|
Raw
History Blame Contribute Delete
1.05 kB

Claim 5 limitations and deviations

  • No Table 5 model was trained. The exact head implementations, row split, schedule, seeds, and checkpoint-selection rule are not public enough for a faithful comparison.
  • No Table 6 model was trained. The exact smaller training subset, evaluation row IDs, schedule, seeds, and RLM checkpoints are unidentified.
  • The cited Qin et al. implementation establishes a defensible interpretation of normalized regression but is not substituted for the paper experiment.
  • Public T5Gemma base metadata verifies rounded architecture sizes only; it cannot verify or falsify a correlation improvement.
  • Gated base access is recorded as an access constraint, not as scientific evidence.
  • The released 181.5M RLM discrepancy is not treated as a counterexample because that release is not identified as either exact Table 6 checkpoint.
  • Public non-discovery cannot prove that private artifacts do not exist.
  • The final scientific verdict is BLOCKED with LOW confidence and earns no forecast points.