| --- |
| license: other |
| license_name: rlwrld-model-license-v1.0 |
| license_link: LICENSE.md |
| library_name: transformers |
| pipeline_tag: image-text-to-text |
| tags: |
| - vlm |
| - vision-language |
| - rldx |
| - qwen3-vl |
| base_model: Qwen/Qwen3-VL-8B-Instruct |
| --- |
| |
| # RLDX-1-VLM |
|
|
| [Paper](https://arxiv.org/abs/2605.03269) · [Project page](https://rlwrld.ai/rldx-1) · [Code](https://github.com/RLWRLD/RLDX-1) · [Models](https://huggingface.co/collections/RLWRLD/rldx-1) |
|
|
| `RLDX-1-VLM` is the **vision-language backbone** used by the RLDX-1 robot |
| policy family. It is a Qwen3-VL-8B-Instruct checkpoint distributed |
| separately from the action policy so that researchers can inspect, finetune, |
| or replace the perceptual stack independently. |
|
|
| > **Note.** This checkpoint exposes a standard Qwen3-VL VLM interface only — |
| > it does **not** ship the Multi-Stream Action Transformer head, the |
| > cognition tokens, the memory / motion / physics modules, or the RLDX |
| > inference server. For action prediction, use one of the |
| > [`RLDX-1-PT`](https://huggingface.co/RLWRLD/RLDX-1-PT), |
| > [`RLDX-1-FT-*`](https://huggingface.co/RLWRLD), or |
| > [`RLDX-1-MT-*`](https://huggingface.co/RLWRLD) checkpoints. |
|
|
| ## Intended use |
|
|
| - As the `--backbone-path` for finetuning a fresh RLDX-1 policy ([recipe](https://github.com/RLWRLD/RLDX-1#finetuning)). |
| - For VLM-only ablations, dense-captioning experiments, or perceptual |
| probing within the RLDX research stack. |
|
|
| ## Quick start |
|
|
| ```python |
| from transformers import AutoModelForImageTextToText, AutoProcessor |
| |
| model = AutoModelForImageTextToText.from_pretrained( |
| "RLWRLD/RLDX-1-VLM", |
| torch_dtype="bfloat16", |
| device_map="cuda:0", |
| ) |
| processor = AutoProcessor.from_pretrained("RLWRLD/RLDX-1-VLM") |
| ``` |
|
|
| ## Model details |
|
|
| - **Type:** Vision-language model (multimodal text + image / video). |
| - **Backbone:** [`Qwen/Qwen3-VL-8B-Instruct`](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct). |
| - **Params:** 8B. |
| - **Role in RLDX-1:** perceptual encoder for the MSAT action policy. Cognition |
| tokens are injected into this backbone and routed through Qwen3-VL hidden |
| states to produce a compact perceptual summary consumed by the action |
| model. |
|
|
| For a full architectural walkthrough including how cognition tokens are |
| wired into this backbone, see |
| [`docs/architecture.md`](https://github.com/RLWRLD/RLDX-1/blob/main/docs/architecture.md). |
|
|
| ## Limitations |
|
|
| `RLDX-1-VLM` is a research backbone snapshot. It is not safety-tuned beyond |
| its Qwen3-VL upstream, and it is not intended as a general-purpose |
| chat assistant. For action prediction it must be paired with the RLDX-1 |
| policy head; the standalone VLM does not produce robot commands. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{rldx2026, |
| title={RLDX-1 Technical Report}, |
| author={Kim, Dongyoung and Jang, Huiwon and Koo, Myungkyu and Jang, Suhyeok and Kim, Taeyoung and others}, |
| year={2026}, |
| note={RLWRLD}, |
| eprint={2605.03269}, |
| archivePrefix={arXiv}, |
| url={https://arxiv.org/abs/2605.03269} |
| } |
| ``` |
|
|
| ## License |
|
|
| Released under the **RLWRLD Model License v1.0** — a non-commercial license |
| with attribution and share-alike requirements. See [`LICENSE.md`](LICENSE.md) for |
| the full text. By using this model you agree to those terms, including the |
| use restrictions in §3.5. |
|
|
