Robotics
Transformers
Safetensors
English
molmoact2
image-text-to-text
OpenRAL
rskill
vision-language-action
nf4
4-bit precision
so100_follower
so101_follower
vla
so101
so100
manipulation
custom_code
8-bit precision
Instructions to use OpenRAL/rskill-molmoact2-multi-so101-nf4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenRAL/rskill-molmoact2-multi-so101-nf4 with Transformers:
# Load model directly from transformers import AutoModelForImageTextToText model = AutoModelForImageTextToText.from_pretrained("OpenRAL/rskill-molmoact2-multi-so101-nf4", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
docs: add generated SKILL.md discovery view
Browse files
SKILL.md
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: molmoact2-so101-nf4
|
| 3 |
+
description: >-
|
| 4 |
+
S1 Vision-Language-Action policy. Capabilities: pick, place, pick_and_place, grasp. MolmoAct2 (Ai2) finetuned on the SO-100/SO-101 teleop mixture, NF4-quantized for 8 GB GPUs. Emits 6-DoF absolute joint-position chunks (size 10) for the SO-100/SO-101 follower arm. Flow-matching action expert on Molmo2-ER VLM. Apache-2.0. norm_tag="so100_so101_molmoact2" travels in the manifest's image_preprocessing block (overridable via vla.extra.norm_tag). Discovery view of an OpenRAL rSkill — NOT directly runnable by an agent harness; it runs via rSkill.from_pretrained + the robot HAL.
|
| 5 |
+
metadata:
|
| 6 |
+
openral_rskill: true # generated discovery view of an rSkill
|
| 7 |
+
schema_version: 0.1
|
| 8 |
+
rskill_id: OpenRAL/rskill-molmoact2-so101-nf4
|
| 9 |
+
manifest: ./rskill.yaml
|
| 10 |
+
role: s1
|
| 11 |
+
kind: vla
|
| 12 |
+
model_family: molmoact2
|
| 13 |
+
embodiment_tags: [so100_follower, so101_follower]
|
| 14 |
+
actions: [pick, place, pick_and_place, grasp]
|
| 15 |
+
scenes: [tabletop]
|
| 16 |
+
sensors_required: ['rgb:observation.images.camera1', 'rgb:observation.images.camera2']
|
| 17 |
+
state_dim: 6
|
| 18 |
+
action_dim: 6
|
| 19 |
+
runtime: pytorch
|
| 20 |
+
quantization: int4/pytorch
|
| 21 |
+
min_vram_gb: {fp32: 22.0, bf16: 11.0, int4: 4.0}
|
| 22 |
+
chunk_size: 10
|
| 23 |
+
latency_budget: {per_chunk_ms: 1000.0}
|
| 24 |
+
license_code: Apache-2.0
|
| 25 |
+
license_weights: apache-2.0
|
| 26 |
+
weights_uri: hf://OpenRAL/rskill-molmoact2-so101-nf4
|
| 27 |
+
source_repo: hf://allenai/MolmoAct2-SO100_101
|
| 28 |
+
paper_url: https://arxiv.org/abs/2605.02881
|
| 29 |
+
---
|
| 30 |
+
|
| 31 |
+
# molmoact2-so101-nf4 — rSkill discovery view
|
| 32 |
+
|
| 33 |
+
> **Generated view, not a hand-written skill.** This `SKILL.md` is a discovery-only
|
| 34 |
+
> mirror of [`rskill.yaml`](./rskill.yaml), produced by `tools/generate_rskill_skillmd.py`.
|
| 35 |
+
> It lets tools that read the standard agent-skill format find and reason about this
|
| 36 |
+
> OpenRAL rSkill. The `rskill.yaml` manifest is the single source of truth
|
| 37 |
+
> (CLAUDE.md §1.3). Do not edit by hand — edit the manifest and regenerate.
|
| 38 |
+
|
| 39 |
+
## What it is
|
| 40 |
+
|
| 41 |
+
An OpenRAL **Vision-Language-Action policy** (`role: s1`, `kind: vla`). MolmoAct2 (Ai2) finetuned on the SO-100/SO-101 teleop mixture, NF4-quantized for 8 GB GPUs. Emits 6-DoF absolute joint-position chunks (size 10) for the SO-100/SO-101 follower arm. Flow-matching action expert on Molmo2-ER VLM. Apache-2.0. norm_tag="so100_so101_molmoact2" travels in the manifest's image_preprocessing block (overridable via vla.extra.norm_tag).
|
| 42 |
+
|
| 43 |
+
## Capabilities
|
| 44 |
+
|
| 45 |
+
- **Verbs:** pick · place · pick_and_place · grasp
|
| 46 |
+
- **Scenes:** tabletop
|
| 47 |
+
- **Embodiments:** so100_follower · so101_follower
|
| 48 |
+
|
| 49 |
+
## Why this is discovery-only
|
| 50 |
+
|
| 51 |
+
An agent skill is natural-language instructions loaded into an LLM's context. An rSkill
|
| 52 |
+
is an executable artifact: it carries a typed capability/embodiment contract, model weights,
|
| 53 |
+
a runtime, and a license/provenance gate — none of which fit in freeform markdown. So an
|
| 54 |
+
agent can use this view to *select* the right skill, but cannot *execute* it by loading
|
| 55 |
+
this file. Execution always goes through the OpenRAL loader and the robot HAL.
|
| 56 |
+
|
| 57 |
+
## License
|
| 58 |
+
|
| 59 |
+
- **Code:** Apache-2.0.
|
| 60 |
+
- **Weights:** `apache-2.0` — permissive / commercial-use OK
|
| 61 |
+
|
| 62 |
+
## How to actually run it (not via an agent harness)
|
| 63 |
+
|
| 64 |
+
```python
|
| 65 |
+
from openral_rskill import rSkill
|
| 66 |
+
|
| 67 |
+
skill = rSkill.from_pretrained("OpenRAL/rskill-molmoact2-so101-nf4")
|
| 68 |
+
# the loader validates embodiment / sensors / runtime / quantization against the target
|
| 69 |
+
# RobotDescription and enforces the weight-license gate before any weights load.
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
See [`rskill.yaml`](./rskill.yaml) for the authoritative, validated manifest.
|