| library_name: transformers | |
| tags: | |
| - molmoact2 | |
| - robotics | |
| - image-text-to-text | |
| <img src="Assets/Logo-Hit.jpg" alt="MolmoAct Logo" style="width: auto; height: 200px;"> | |
| # **MolmoAct2 x HIT** | |
| MolmoAct2-YellowCube-BlueBin is a fine-tuned vision-language-action model for specific robot manipulation tasks, built upon the **allenai/MolmoAct2** base model. | |
| This checkpoint has been further trained for a targeted robotic task: selecting a specific yellow object from a set of objects and placing it into a designated blue container. | |
| ### Task Details | |
| * **Environment Setup:** 3 different-colored blocks (including one yellow block) are placed at fixed positions within the robot's workspace. | |
| * **Goal:** Locate the yellow block and throw/place it into the target blue container. | |
| * **Usage:** Unlike the general multi-embodiment base model, this checkpoint serves as a specialized policy ready for execution and evaluation in this specific pick-and-place environment setup. | |
| ## Quick Links | |
| - 📂 Models: [Models](https://huggingface.co/HIT-Robotics/ur5e_molmoact2_pp_yellow/tree/main) | |
| - 📂 Datasets: [MolmoAct2-Yellow_Stone_Dataset](https://huggingface.co/datasets/HIT-Robotics/ur5e_pick_and_place_yellow_stone/tree/main) | |
| ## Intended Use | |
| Use this checkpoint for further fine-tuning on a target robot embodiment or benchmark. It contains the VLM and continuous action expert weights, plus normalization metadata for the post-training mixture in `norm_stats.json`. | |
| This model card intentionally does not include direct policy inference code. For ready-to-run inference examples, use one of the fine-tuned checkpoints such as `MolmoAct2-LIBERO`, `MolmoAct2-DROID`, `MolmoAct2-BimanualYAM`, or `MolmoAct2-SO100_101`. | |
| ## Model and Hardware Safety | |
| MolmoAct2 generate robot actions from visual observations and language instructions, but their behavior may vary across embodiments, environments, and hardware configurations. Users should carefully validate model outputs before deployment, especially when operating physical robots or other actuated systems. Where possible, actions should be monitored through interpretable intermediate outputs (adaptive depth map), simulation rollouts, action limits, or other safety checks before execution on hardware. The model’s action space should be bounded by the training data, robot controller limits, and task-specific safety constraints, including limits on speed, workspace, torque, and contact force. Users should follow the hardware manufacturer’s safety guidelines, use appropriate emergency-stop mechanisms, and operate the system only in a safely configured environment with human supervision. | |