so101 FlowerVLA Stage-2 checkpoint
Vanilla FlowerVLAPolicy-compatible. Load with
from src.flower.policy import FlowerVLAPolicy
policy = FlowerVLAPolicy.from_pretrained("<this dir>", device="cuda")
How it was trained
- Stage 1: Florence-2-base + text-encoder LoRA (rank 8 alpha 4 dropout 0.2) trained to predict the target bowl's pixel coordinates given (image, prompt). Within-condition val split. slot_acc ~0.96.
- Stage 2: load Stage 1 ckpt, attach DiT action head, joint train with total = action_loss + 0.1 * pixel_loss (Stage 1 head kept as regularizer). Florence base frozen, LoRA adapters fine-tuned, DiT trained from scratch.
- Package: text LoRA merged into Florence base weights so this checkpoint loads into vanilla FlowerVLAPolicy without LoRA support.