results/determinism_check.json

{
  "what_this_is": "Greedy-determinism probe (HF Job 6a1a215c..., h200, 2026-05-29): the SAME baseline spec_rl reward eval (12-problem HumanEval slice, greedy, thinking off) run THREE times on ONE server instance. Purpose: show that greedy MoE decoding is not bit-reproducible run-to-run, which explains the DFlash A/B reward wobble.",
  "repeats": 3,
  "per_run_mean_reward": [0.85, 1.0, 1.0],
  "per_run_reward": [
    [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.2, 1.0],
    [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],
    [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
  ],
  "run_vs_run1_parity": [
    {"compared": 12, "mismatches": 3, "lossless": false},
    {"compared": 12, "mismatches": 3, "lossless": false}
  ],
  "greedy_bit_reproducible": false,
  "finding": "Three identical-config greedy runs of the BASELINE (no DFlash) gave 0.85, 1.0, 1.0 — the baseline itself swings on the same 2-3 hard, long HumanEval problems. So the DFlash A/B's 1.0-vs-0.85 reward gap is run-to-run greedy MoE nondeterminism (floating-point non-associativity in expert reductions), NOT a DFlash quality change. Reward-invariance holds BY CONSTRUCTION (lossless decode, proven byte-identical in the decode A/B, => identical reward); we decline to quote a 'DFlash reward' as a quality signal, the same discipline applied to acceptance length tau."
}