File size: 7,779 Bytes
319eb16 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 | # Fine-tuning Robometer on Your Own Data
Preprocess a dataset, LoRA fine-tune from Robometer-4B, upload to the Hub, run inference. Example: [MINT-SJTU/RoboFAC-dataset](https://huggingface.co/datasets/MINT-SJTU/RoboFAC-dataset).
---
## 1. Preprocessing (RoboFAC)
Training needs a **preprocessed** cache from a HuggingFace dataset in RBM format. RoboFAC is folder-based; convert first, then preprocess.
1. **Download** (into `ROBOMETER_DATASET_PATH`):
```bash
export ROBOMETER_DATASET_PATH=/path/to/your/robometer_dataset
huggingface-cli download MINT-SJTU/RoboFAC-dataset --local-dir $ROBOMETER_DATASET_PATH/RoboFAC-dataset
```
2. **Convert and push to Hub** (required for recommended flow):
```bash
export HF_TOKEN=your_token_here
uv run python -m dataset_upload.generate_hf_dataset \
--config_path dataset_upload/configs/data_gen_configs/robofac.yaml \
--dataset.dataset_path=$ROBOMETER_DATASET_PATH/RoboFAC-dataset \
--hub.push_to_hub=true \
--hub.hub_repo_id=robofac_rbm
```
3. **If you pushed:** download converted repo and set preprocess config:
```bash
huggingface-cli download aliangdw/robofac_rbm --local-dir $ROBOMETER_DATASET_PATH/robofac_rbm
```
In `preprocess_finetune.yaml`: `train_datasets: ["aliangdw/robofac_rbm"]`, `train_subsets: [["robofac"]]`.
4. **Preprocess:**
```bash
export ROBOMETER_PROCESSED_DATASETS_PATH=/path/to/save/processed_datasets
uv run python -m robometer.data.scripts.preprocess_datasets \
--config robometer/configs/preprocess_finetune.yaml \
--cache_dir=$ROBOMETER_PROCESSED_DATASETS_PATH
```
Use the same path for training.
**Other datasets:** RBM-style Hub datasets: set `train_datasets`/`train_subsets` and `ROBOMETER_DATASET_PATH` in the preprocess config, then run step 4. Raw data: add a loader (see [CustomDataset.md](dataset_upload/dataset_guides/CustomDataset.md)).
---
## 2. LoRA fine-tuning
Use PEFT (LoRA) and `load_from_checkpoint` from a Qwen3-4B–based RBM checkpoint.
```bash
export ROBOMETER_PROCESSED_DATASETS_PATH=/path/to/your/processed_datasets
uv run python train.py \
model.base_model_id=Qwen/Qwen3-VL-4B-Instruct \
model.use_peft=true \
model.train_progress_head=true \
model.train_preference_head=true \
data.train_datasets=[aliangdw_robofac_rbm_robofac] \
data.eval_datasets=[mw] \
training.load_from_checkpoint=aliangdw/Robometer-4B \
training.per_device_train_batch_size=8 \
training.learning_rate=2e-5 \
training.warmup_ratio=0.1 \
training.weight_decay=0.01 \
training.max_steps=1000 \
training.output_dir=./logs \
training.exp_name=robometer4b_lora_robofac_2 \
logging.log_to=[wandb] \
custom_eval.eval_types=[reward_alignment,policy_ranking] \
custom_eval.reward_alignment=[aliangdw_robofac_rbm_robofac] \
custom_eval.policy_ranking=[aliangdw_robofac_rbm_robofac] \
logging.save_best.metric_names=[eval_rew_align/pearson_robofac,eval_p_rank/kendall_last_robofac] \
logging.save_best.greater_is_better=[true,true] \
training.overwrite_output_dir=True \
training.eval_steps=50 \
training.custom_eval_steps=50
```
The short name `robofac` is defined in `name_mapping.py` for `aliangdw_robofac_rbm_robofac`.
**Tunable LoRA / training:** Override `training.learning_rate`, `training.warmup_ratio`, `training.weight_decay`, `training.gradient_accumulation_steps`, or `training.max_steps` as needed. Defaults above match `robometer/configs/config.yaml`. Multi-GPU: `uv run accelerate launch --config_file robometer/configs/distributed/fsdp.yaml train.py ...` (same overrides).
### Full fine-tuning (no PEFT)
Load the same checkpoint but train the full model (no LoRA). Uses more memory; lower `per_device_train_batch_size` or gradient accumulation if needed.
```bash
export ROBOMETER_PROCESSED_DATASETS_PATH=/path/to/your/processed_datasets
uv run python train.py \
model.base_model_id=Qwen/Qwen3-VL-4B-Instruct \
model.use_peft=false \
model.train_progress_head=true \
model.train_preference_head=true \
data.train_datasets=[aliangdw_robofac_rbm_robofac] \
data.eval_datasets=[aliangdw_robofac_rbm_robofac] \
training.load_from_checkpoint=aliangdw/Robometer-4B \
training.per_device_train_batch_size=8 \
training.learning_rate=2e-5 \
training.warmup_ratio=0.1 \
training.weight_decay=0.01 \
training.gradient_accumulation_steps=1 \
training.max_steps=500 \
training.output_dir=./logs \
training.exp_name=robometer4b_full_robofac \
logging.log_to=[wandb] \
custom_eval.reward_alignment=[aliangdw_robofac_rbm_robofac] \
custom_eval.policy_ranking=[aliangdw_robofac_rbm_robofac] \
logging.save_best.metric_names=[eval_rew_align/pearson_robofac,eval_p_rank/kendall_last_robofac] \
logging.save_best.greater_is_better=[true,true] \
training.eval_steps=50 \
training.custom_eval_steps=50
```
---
## 3. Upload to Hub
```bash
uv run python robometer/utils/upload_to_hub.py \
--model_dir ./logs/robometer4b_lora_robofac/checkpoint-500 \
--hub_model_id aliangdw/robometer-4b-lora-robofac \
--base_model "Qwen/Qwen3-VL-4B-Instruct" \
--commit_message "LoRA fine-tune on RoboFAC"
```
Or enable `logging.save_best.upload_to_hub: true` in config for upload during training.
---
## 4. Inference
```bash
uv run python scripts/example_inference_local.py \
--model-path aliangdw/robometer-4b-lora-robofac \
--video /path/to/video.mp4 \
--task "Insert the cylinder"
```
Server: `uv run python robometer/evals/eval_server.py ... model_path=aliangdw/robometer-4b-lora-robofac`. Eval: `run_baseline_eval.py` with `reward_model=rbm`, `model_path=...` (see [README](README.md)).
---
## 5. Baseline: Fine-tune from base Qwen-VL (no Robometer checkpoint)
For comparison, run the same `train.py` on the same data but **without** loading a Robometer checkpoint. Training starts from the base Qwen-VL plus randomly initialized progress/preference heads.
```bash
export ROBOMETER_PROCESSED_DATASETS_PATH=/path/to/your/processed_datasets
uv run python train.py \
model.base_model_id=Qwen/Qwen3-VL-4B-Instruct \
model.use_peft=true \
model.train_progress_head=true \
model.train_preference_head=true \
data.train_datasets=[aliangdw_robofac_rbm_robofac] \
data.eval_datasets=[aliangdw_robofac_rbm_robofac] \
training.per_device_train_batch_size=8 \
training.learning_rate=2e-5 \
training.warmup_ratio=0.1 \
training.weight_decay=0.01 \
training.max_steps=1000 \
training.output_dir=./logs \
training.exp_name=qwen3vl_lora_robofac_baseline \
logging.log_to=[wandb] \
custom_eval.reward_alignment=[aliangdw_robofac_rbm_robofac] \
custom_eval.policy_ranking=[aliangdw_robofac_rbm_robofac] \
logging.save_best.metric_names=[eval_rew_align/pearson_robofac,eval_p_rank/kendall_last_robofac] \
logging.save_best.greater_is_better=[true,true] \
training.eval_steps=50 \
training.custom_eval_steps=50
uv run python train.py \
model.base_model_id=Qwen/Qwen3-VL-4B-Instruct \
model.use_peft=false \
model.train_progress_head=true \
model.train_preference_head=true \
data.train_datasets=[aliangdw_robofac_rbm_robofac] \
data.eval_datasets=[aliangdw_robofac_rbm_robofac] \
training.per_device_train_batch_size=8 \
training.learning_rate=2e-5 \
training.warmup_ratio=0.1 \
training.weight_decay=0.01 \
training.max_steps=1000 \
training.output_dir=./logs \
training.exp_name=qwen3vl_robofac_baseline \
logging.log_to=[wandb] \
custom_eval.reward_alignment=[aliangdw_robofac_rbm_robofac] \
custom_eval.policy_ranking=[aliangdw_robofac_rbm_robofac] \
logging.save_best.metric_names=[eval_rew_align/pearson_robofac,eval_p_rank/kendall_last_robofac] \
logging.save_best.greater_is_better=[true,true] \
training.eval_steps=50 \
training.custom_eval_steps=50
```
|