# Robustness Exp 4: Implementation Stability

Status:
- Complete

Goal:
- Check whether the method is robust to implementation path.
- In the current study, the concrete axis is:
  - non-streaming whitening / recovery
  - streaming compression / aligned recovery

Source experiment:
- [ablation/exp_4_streaming_vs_nonstreaming](../../ablation/exp_4_streaming_vs_nonstreaming)

Why this is complete:
- Logs exist for all three ratios:
  - `r0.8`
  - `r0.6`
  - `r0.4`
- And for all four variants:
  - `nonstream/whitening_only`
  - `nonstream/fullseq_actlora`
  - `streaming/compression_only`
  - `streaming/recovery_aligned`

Key `r0.8` results:

| path | variant | raw avg | mixed7 | wt2 | ptb | c4 |
|---|---|---:|---:|---:|---:|---:|
| nonstream | whitening_only | 48.56 | 52.83 | 13.98 | 14.31 | 27.23 |
| nonstream | fullseq_actlora | 50.72 | 54.92 | 9.08 | 12.98 | 19.84 |
| streaming | compression_only | 29.40 | 32.06 | 2298.77 | 883.79 | 2725.46 |
| streaming | recovery_aligned | 38.14 | 40.72 | 15.48 | 35.38 | 51.52 |

Sources:
- [nonstream r0p8 whitening_only mcq](../../ablation/exp_4_streaming_vs_nonstreaming/logs/nonstream/r0p8/whitening_only/mcq.log)
- [nonstream r0p8 whitening_only ppl](../../ablation/exp_4_streaming_vs_nonstreaming/logs/nonstream/r0p8/whitening_only/ppl.log)
- [nonstream r0p8 fullseq_actlora mcq](../../ablation/exp_4_streaming_vs_nonstreaming/logs/nonstream/r0p8/fullseq_actlora/mcq.log)
- [nonstream r0p8 fullseq_actlora ppl](../../ablation/exp_4_streaming_vs_nonstreaming/logs/nonstream/r0p8/fullseq_actlora/ppl.log)
- [streaming r0p8 compression_only mcq](../../ablation/exp_4_streaming_vs_nonstreaming/logs/streaming/r0p8/compression_only/mcq.log)
- [streaming r0p8 compression_only ppl](../../ablation/exp_4_streaming_vs_nonstreaming/logs/streaming/r0p8/compression_only/ppl.log)
- [streaming r0p8 recovery_aligned mcq](../../ablation/exp_4_streaming_vs_nonstreaming/logs/streaming/r0p8/recovery_aligned/mcq.log)
- [streaming r0p8 recovery_aligned ppl](../../ablation/exp_4_streaming_vs_nonstreaming/logs/streaming/r0p8/recovery_aligned/ppl.log)

Conclusion:
- The current method is not robust to replacing the non-streaming path with the current streaming path.
- Non-streaming clearly dominates at `r0.8`, both before and after recovery.
- This robustness axis is already answered strongly enough; no immediate rerun is needed unless the streaming implementation changes.

Recommended use in writeup:
- Put this in the robustness section, not the core ablation section.
- The main claim is implementation sensitivity:
  - current results depend strongly on the non-streaming path.