ACT_manipulate_pipette (checkpoint R_u330)

ACT policy for the RoboSynChallenge simulation task manipulate_pipette. The repository root is a complete LeRobot pretrained_model directory. This card documents the user-selected uploaded weights at revision 05e339628e7076ce284764e3478dd8503e489ed3. Preparing this README does not modify weights or inference configuration.

Files

File	Size (bytes)	sha256
config.json	2033	cfd7951bf965c30a5c16a8bfd46dbaf6c146e8d3a67a87442dbdac53b919e9b2
model.safetensors	206563032	ab244c369ae6f97f0c138192a3a6b8d43c4c60511869747c2fdb283de24e270a
train_config.json	4044	4b99befa0777f3ce08ff9cd4baa9f0fe14e3c9f055b78c32c307cfc2f06e3291
SmolVLA preprocessor / postprocessor	N/A (ACT)	N/A

The pinned HF revision also contains VERSION.txt, MANIFEST.json and SHA256SUMS (provenance only), plus .gitattributes. Normalization is embedded in the weights; no separate normalization artifact or SmolVLA processor is required. The README will be uploaded separately to the HF root by the owner, so its upload will produce a different repository revision; retain the weights revision above for provenance.

Model

• Architecture: ACTPolicy; ResNet18 vision backbone; transformer dim_model=512, 8 heads, feedforward=3200, 4 encoder layers, 1 decoder layer; VAE latent_dim=32, 4 VAE encoder layers, KL weight=10.
• Initialization: B40k (B_random200_60k, 40000 optimizer updates), local source /home/yizhenyu/robosyn-act-b20k/runs/B_random200_60k/training/checkpoints/040000/pretrained_model; model SHA256 fcf6ba04ce76c36748ff75e1f71d9d3f8069097e69b0329846f8fc3c60c3fc6f.
• R training: 330 optimizer updates, AdamW lr=1e-5, lr_backbone=1e-5, weight_decay=1e-4, betas=(0.9,0.999), eps=1e-8, grad_clip_norm=10, fixed seed=1000; fresh optimizer initialized from B40k weights, AMP=false. Microbatch 32 × accumulation 2 = effective batch 64. Accumulation and sampling are recorded in experiment protocol; inherited train_config.batch_size=32 alone is not the effective batch.
• Original data: local original_plus_random, LeRobot v2.1, 1200 episodes / 343200 frames, 25 Hz (original 1000 + random expert 200); /home/yizhenyu/robosyn-act-b20k/data/original_plus_random. Exact upstream data revision is not recorded.
• Added data: 5 successful CURRENT_STATE / LIVE_TAKEOVER corrective demonstrations, 554 frames, /home/yizhenyu/robosyn-recovery-cr-20261011-1315/data/recovery_valid. Only successful expert corrective actions were supervised; failed policy prefixes and failed corrective fragments were not labeled as positive expert targets. Privileged diagnostics were excluded from policy observations.
• Actual sample composition: 4162 corrective windows of 21120 sampled training windows = 19.71%; 16958 original-data windows. This fraction is by sampled training window/frame anchor, not by episode or total stored frames. Episode boundaries and tail padding/masks were retained; no cross-episode action stitching. Original B40k normalization tensors were preserved.
• W&B: N/A; this R run disabled W&B, run_id null.
• Code: RoboSynChallenge inspected checkout 7122266b88321e3342a3df416c43c9be1c0bdcdc; EmbodiChain 9ebee30011f378f94a7cbe78b01d8c2eacba231a. The immutable original training git commit was not recorded. Local training adapter /home/yizhenyu/robosyn-recovery-cr-20261011-1315/code/train_cr.py, SHA256 a13df0f8f393f24e810887c0b3241042844b0be1da3dc4568fdb6020215e7730; converter SHA256 19d5bc5103c28ea1f8cb18c7e2f494b3ab514bd9aa33f4ed17d2322476195e90.
• Evaluation adapter run_probe.py SHA256 d99315394aa46bffbee3caa38380f0027c42d5db3493536b4507e26d07361a87, worker.py SHA256 14e9954076cdcd37037428787aa9bae9bf9fee013f5ffa3ea6991f3d821ffe21 in /root/workspace/issue22-runs/recovery-cr-20261011-1315/eval_source_v2; full frozen source hashes are in evaluation plan.json.
• Environment: training Python 3.10.22, LeRobot 0.3.3, PyTorch 2.7.0+cu128, CUDA runtime 12.8, A100 80GB. Evaluation used existing RTX 5090 runtime, policy Python 3.10.12 / PyTorch 2.7.0+cu128, CPU physics, three RGB cameras. Exact original B40k training source git commit is not recorded.
• Method scope: this project borrowed the SimDAgger-style “policy failure → expert correction → data aggregation” workflow. The trigger, current-state expert and RoboSyn adapter are project-specific; this is not a full reproduction of original SimDAgger/RoboTwin or evidence that expert recovery itself implies improved learned policy SR.

Input and output

• Runtime state input: observation.state, float32 [B,14]. Left arm 6 joints + gripper, then right arm 6 joints + gripper; gripper indices 6 and 13. Joint positions use the dataset/robot convention; gripper values must retain that convention rather than being treated as revolute angles.
• Runtime RGB inputs: observation.images.cam_high, observation.images.cam_right_wrist, observation.images.cam_left_wrist, each float32 [B,3,480,640]. Raw frames are uint8 HWC [480,640,3], converted to CHW and divided by 255 before the policy's stored normalization. Cameras are mapped by key, not positional numbering.
• config.json also declares observation.qvel and observation.qf [14]; these are not passed by the verified ACT deployment path. They are not additional required policy inputs for that path. Privileged object poses, button state, success flags and future observations are excluded.
• Action: float32 [B,50,14] from predict_action_chunk, or [B,14] per select_action; bimanual joint/gripper targets, not Cartesian deltas. Preserve the left/right and gripper ordering.
• Normalization: VISUAL, STATE and ACTION use MEAN_STD; normalization tensors are embedded in model.safetensors. No external normalization file is required. Do not rebuild or reset statistics during deployment.
• n_obs_steps=1, chunk_size=50, n_action_steps=50, temporal_ensemble_coeff=null; training data 25 Hz. The internal tests used H50. Competition execution_chunk requirements must be checked separately; no deployment configuration was changed for this card.

The real R training batches contained state, the three named images, action labels and action_is_pad; hidden object state and official success were not policy features. The 100-episode inference bridge likewise supplied only RGB and robot state.

Procedure: download and evaluate

Download the already uploaded weights at the exact revision (command documented, not executed as a fresh validation in this card-preparation task):

hf download mair-robosyn/ACT_manipulate_pipette --repo-type model --revision 05e339628e7076ce284764e3478dd8503e489ed3 --local-dir checkpoints/ACT_manipulate_pipette/

Use PreTrainedConfig.from_pretrained to load ACT config, then ACTPolicy; the existing config has pretrained_backbone_weights=null. Do not change normalization, camera mapping or action semantics.

Prospective team-SOP check, from the RoboSynChallenge checkout with its already working simulator environment:

cd /root/workspace/RoboSynChallenge
export PYTHON_BIN=/root/workspace/shorttrain-20260924-1417/envs/act/bin/python
bash policy/act/eval.sh manipulate_pipette random checkpoints/ACT_manipulate_pipette 0 --max_episodes 5 --pytorch_device cuda --headless True

This official-script check is NOT DONE on a fresh HF fixed-revision download. It is not an exact reproduction of the custom 100-seed protocol below; task gym timeout and expert-feasibility defaults must not be conflated with that protocol.

Historical command for the completed fixed-list 100-episode custom pure-policy evaluation, inside ssh robosyn → robosyn-workspace:

/root/miniconda3/envs/py310/bin/python /root/workspace/issue22-runs/r100-20261011/run_bounded_detached.py

That audited run directory is single-use and contains a start marker; do not rerun it or overwrite it. Exact configuration, fixed seeds, environment file and frozen code hashes are in /root/workspace/issue22-runs/r100-20261011/plan.json; results are in final_summary_bounded.json, exit_status_bounded.json, per_seed.csv and evaluation_bounded.log. The private project evaluation adapter is not bundled in this weights-only HF repository; a standalone identical 100-seed reproduction requires that recorded adapter and environment, not only the public weights. No evaluation is started during README preparation.

Recorded training entry, requiring the same private data and project adapter, with a new output directory:

/home/yizhenyu/miniconda3/envs/robosyn-act-py310/bin/python /home/yizhenyu/robosyn-recovery-cr-20261011-1315/code/train_cr.py --branch R --updates 330 --recovery-root /home/yizhenyu/robosyn-recovery-cr-20261011-1315/data/recovery_valid --out /home/yizhenyu/robosyn-recovery-cr-20261011-1315/runs/R_u330_reproduce --max-seconds 900

The adapter fixes the remaining optimizer/data/seed settings described above; this command is documentation only, not a new training run.

Results (team-internal, not organizer results)

• Protocol: formal ManipulatePipette, random, configs/manipulate_pipette/random/gym_config.json, genuine official task success conditions; custom frozen pure-policy bridge, H50, max 1000 actions per planned seed, RTX 5090 host ssh robosyn. Expert takeover disabled, no privileged policy inputs, no rewind/teleport or reset retries of normally failed tasks. No expert-feasibility seed filter in this protocol.
• Fixed independent development-evaluation list: 100 planned seeds, generated with seed 20261011100 and frozen before evaluation. All registered collection/expert-debug/training-development/prior-evaluation seeds were excluded. Full disjointness from original 1200 training episodes is not established because original metadata lacks seed fields. “Independent” refers to the registered collection/debug/model-selection seeds, not proof of complete training-scene disjointness.
• 99 normally completed: 51 successes, 48 task failures. 1 terminal runtime error; 0 unrun. Normal-completion SR = 51/99 = 51.52%, 95% Wilson interval 41.80%–61.12%. The denominator excludes the separately reported terminal runtime error; it is not 100 successfully executed episodes. Mean completed-episode steps: see raw records; not recomputed for this card.
• Terminal initialization-error seed 1975839064. There were 7 initialization-failure attempts and 5 seeds recovered on their single retry (1599153036, 1564187768, 296214554, 253064360, 1027407378). 106 total attempts; per-seed limit 2, normal task failures not retried, no seed replacement. Native initialization SIGSEGV root cause remained unresolved; bounded retry success is not a simulator fix.
• Completed on 2026-10-11; scheduler exit code 0. Evidence directory /root/workspace/issue22-runs/r100-20261011; formal gym config SHA256 1847efdf8287f73d4d9845177646895e580c7e65ba8e4fe6c76a429b0c8d6c31, action config SHA256 5616cfc4fb3359cbe2ba87e2bcbd1bd0195d1e795d33269ee0b43e44b3a8b202.
• This is a single R model result. It is not a paired B40k/C/R improvement estimate and must not be compared directly with historical B40k 49/100 to claim stable improvement.

Status

• UPLOADED: owner reports weights uploaded at 05e339628e7076ce284764e3478dd8503e489ed3; HF revision metadata was read and matches that revision. Full pinned-revision fresh-download/all-file hash validation is NOT DONE; metadata visibility alone does not satisfy the SOP's strict UPLOADED verification.
• LOAD VERIFIED: no for a fresh HF pinned-revision download. Pre-upload local package strict CPU loading and source/archive hash checks were completed; local GPU inference was exercised during evaluation. These checks do not establish post-upload LOAD VERIFIED.
• EVAL VERIFIED (team-internal): no in the SOP's post-HF-download official eval.sh sense. Local-source custom pure-policy evaluation completed with 51/99; no official-script evaluation on re-downloaded revision 05e339628e7076ce284764e3478dd8503e489ed3 was performed.
• README preparation only; no HF login, upload, weight/config modification or competition submission was performed. A model repository is not a competition submission receipt.

Warning

These results come from our local runs. The organizers did not verify them. This small corrective-data experiment does not establish stable SR improvement, original SimDAgger reproduction, or deployed expert recovery. Evaluation used the policy alone. Do not infer paired improvement from historical baselines, training loss, or the existence of successful expert demonstrations.