Update README.md
Browse files
README.md
CHANGED
|
@@ -32,6 +32,13 @@ datasets:
|
|
| 32 |
---
|
| 33 |
This is the second model in the ensemble for the MindsAI @ Tufa Labs team for the ARC Prize 2025 competition. It was originally based on the CodeT5 model from Salesforce. It was modified to have 16 layers in the decoder from the original 24 layers. Testing demonstrated that removing layers was more harmful to performance when removed from the encoder, but was able to fully recover when removing decoder layers.
|
| 34 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
## ARC Data Formatting
|
| 36 |
|
| 37 |
- ARC tasks ship as JSON where each `task_id` contains `train` pairs and `test` inputs; every grid is a rectangular list of lists with integers `0-9`. Dimensions follow the original 1×1–30×30 spec, though the evaluator accepts up to 50×50.
|
|
|
|
| 32 |
---
|
| 33 |
This is the second model in the ensemble for the MindsAI @ Tufa Labs team for the ARC Prize 2025 competition. It was originally based on the CodeT5 model from Salesforce. It was modified to have 16 layers in the decoder from the original 24 layers. Testing demonstrated that removing layers was more harmful to performance when removed from the encoder, but was able to fully recover when removing decoder layers.
|
| 34 |
|
| 35 |
+
- **Span-Corruption Refinement (SCR)**: The model was trained with an additional
|
| 36 |
+
- pretraining objective I call SCR (chosen because of the model's deep history of
|
| 37 |
+
- training with the span corruption objective). The answer is noised with heavy
|
| 38 |
+
- span corruption and data augmentation (to mimic incorrect answers) and passed
|
| 39 |
+
- along with the prompt for refinement (model outputs the full corrected grid).
|
| 40 |
+
- Note: This did not result in better performance when used during inference (only used during TTT).
|
| 41 |
+
|
| 42 |
## ARC Data Formatting
|
| 43 |
|
| 44 |
- ARC tasks ship as JSON where each `task_id` contains `train` pairs and `test` inputs; every grid is a rectangular list of lists with integers `0-9`. Dimensions follow the original 1×1–30×30 spec, though the evaluator accepts up to 50×50.
|