Upload best LoRA adapter from arc_so_iclqa_v2_r32_K6_long_0606c

ec8ac3d verified 18 days ago

1.02 kB

base_model: Qwen/Qwen3-4B-Instruct-2507
library_name: peft
tags:
  - lora
  - meta-learning
  - arc-agi
  - test-time-training

meta_ttt_arc_v1 — second-order MAML LoRA on ARC-AGI

LoRA adapter for Qwen/Qwen3-4B-Instruct-2507 trained via second-order inner-checkpoint MAML on the ARC-AGI ICL-QA task. Best checkpoint by training-time val/post/f1.

Experiment

name: arc_so_iclqa_v2_r32_K6_long_0606c
meta step: 900
val/post/f1 (train-time monitoring): 0.2631578947368421
LoRA rank: 32, alpha: 64, target_modules: all-linear
Inner: K=6 Adam steps, ilr=2e-4, kl_lambda=0.1
Outer: 1500 meta steps, cosine 1e-4, warmup 50, bs=64 outer-QAs
Trained on HerrHruby/arc_agi_mini_docs (v2 mini-docs)

Test-time use

The intended use is meta-test-time training: load the adapter, run a few Adam steps on the task's inner_docs for the test example, then generate the outer answer. See the codebase for inner_loop_batched_adam_seqgrad_nograph.

License

Apache-2.0 (inherits from base model).