Thanks !
#1
by
cdtmc
- opened
This is very cool to open-source your full run - thank you!
Seems like you have +4.3 over the arcprize release after the same amount of steps. Looks potentially like single node training is better than multi-node.
I had some immediate thoughts for your consideration:
- Have you considered training run only using "concept" and "train" split and then try post-train techniques on "evaluation" split?
- Have you considered training run replacing "concept" with "arc-gen-100k"
- If you do manage to crack a post-train technique I think it would be interesting seeing a progressive train "arc-gen-100k", then "train" "evaluation" from arc2025 and finally submission on "test"
Any more open sourcing would be greatly appreciated
Ah ok, just seeing this now, sorry for that. Check https://trelis.com/arc-agi-2/ for a full report!
- Sounds reasonable. I didn't specifically try that but you can see some similar ablations in the report.
- I didn't and I'm not familiar with that dataset
- I'm not quite sure what you mean by "progressive train"