Monotonic hidden weight artifact in merged v2 parameters
#2
by rmems - opened
Problem
The 256 hidden weights in dataset/merged_v2/parameters_weights.mem show a suspicious monotonic pattern: each neuron's 16 weights increase by exactly Q8.8 step 0x0001 (0.0039).
Neuron 0: 00C0, 00C1, 00C2, 00C3, ... 00CF (+1 each)
Neuron 1: 00C4, 00C5, 00C6, 00C7, ... 00D3 (+1 each)
All 16 neurons follow this linear ramp with only the base offset changing. A trained SNN should show weights going up AND down across a neuron's 16 inputs.
Root Cause Hypotheses
- Training bug: Weight update applies same delta to all connections
- Export bug: GPU float to CPU Q8.8 hex pipeline rounding
- Telemetry bug: Corrupted/uniform training data
- FPGA supervisor artifact: Online learning used simplified update rule
Evidence
- Output weights (48 values) show signed, varied patterns -- NOT monotonic
- Decay rates show graduated 0.80-0.95 -- NOT monotonic
- Thresholds show graduated 1.125-1.594 -- NOT monotonic
- The issue is specific to the 256 hidden weights
Investigation Plan
- Save raw float weights before Q8.8 export and compare
- If floats diverse but Q8.8 monotonic then export bug
- If both monotonic then training bug
- Add snapshot logging at each pipeline stage
Mimo Code agent: MiMo-V2.5-Pro