Post
61
I published an article about training a network to write from reward alone. Code on GitHub with it.
Then someone read the code.
Dipankar Sarkar commented four times in a day. Each time he had run something first. He rebuilt my statistics in numpy because he had no torch installed.
He found a bound I had missed. A policy that never learned the determiner to noun dependency has a product support, so at full validity it cannot exceed the largest fully valid product in the sublanguage it entered. That is 12 on one side and 24 on the other, computable before any training. Over 70 seeds it is never crossed, and the most common outcome is the bound itself. I had published one of those numbers as an interesting coincidence.
Then four of my published numbers came apart. Three were a single seed. The fourth was twenty seeds, and I had produced it while fixing the other three.
And the test I built to validate his bound tested nothing. I had swapped two conditions so cleanly that the two grammars were isomorphic. Seventy seeds would have returned the mirror image by construction.
The real lesson:
A relabelling can permute, but it cannot change a ratio. A perfectly symmetric control is often a perfectly empty one. None of my errors were in the reasoning. They were in the plumbing, and nothing in my own process caught a single one.
Code, figures, and the notebook with eight dated refutations 👇
🔗 https://huggingface.co/blog/RDTvlokip/i-published-my-rl-experiments
💻 https://github.com/RDTvlokip/RDTRL
📦 https://doi.org/10.5281/zenodo.21726216
Then someone read the code.
Dipankar Sarkar commented four times in a day. Each time he had run something first. He rebuilt my statistics in numpy because he had no torch installed.
He found a bound I had missed. A policy that never learned the determiner to noun dependency has a product support, so at full validity it cannot exceed the largest fully valid product in the sublanguage it entered. That is 12 on one side and 24 on the other, computable before any training. Over 70 seeds it is never crossed, and the most common outcome is the bound itself. I had published one of those numbers as an interesting coincidence.
Then four of my published numbers came apart. Three were a single seed. The fourth was twenty seeds, and I had produced it while fixing the other three.
And the test I built to validate his bound tested nothing. I had swapped two conditions so cleanly that the two grammars were isomorphic. Seventy seeds would have returned the mirror image by construction.
The real lesson:
A relabelling can permute, but it cannot change a ratio. A perfectly symmetric control is often a perfectly empty one. None of my errors were in the reasoning. They were in the plumbing, and nothing in my own process caught a single one.
Code, figures, and the notebook with eight dated refutations 👇
🔗 https://huggingface.co/blog/RDTvlokip/i-published-my-rl-experiments
💻 https://github.com/RDTvlokip/RDTRL
📦 https://doi.org/10.5281/zenodo.21726216