Instructions to use Haongchen/MemoryVLA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Piper
How to use Haongchen/MemoryVLA with Piper:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Document base model dependencies and direct loading
Browse files
README.md
CHANGED
|
@@ -41,6 +41,30 @@ configs/piper-color-sorting/config.json
|
|
| 41 |
The checkpoint is intended to be loaded with the MemoryVLA codebase and
|
| 42 |
`use_ema=True`. The model uses the `custom_finetuning` normalization key.
|
| 43 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
## Training configuration
|
| 45 |
|
| 46 |
- Task: Piper color sorting
|
|
@@ -91,7 +115,7 @@ settings for this checkpoint are:
|
|
| 91 |
|
| 92 |
```python
|
| 93 |
model = load_vla(
|
| 94 |
-
model_id_or_path="
|
| 95 |
load_for_training=False,
|
| 96 |
action_dim=7,
|
| 97 |
future_action_window_size=15,
|
|
|
|
| 41 |
The checkpoint is intended to be loaded with the MemoryVLA codebase and
|
| 42 |
`use_ema=True`. The model uses the `custom_finetuning` normalization key.
|
| 43 |
|
| 44 |
+
For direct loading from the repository root, a server-side LFS alias is also
|
| 45 |
+
provided at:
|
| 46 |
+
|
| 47 |
+
```text
|
| 48 |
+
checkpoints/frozen-ema-step-20000.pt
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
Set `MEMORYVLA_SKIP_BASE_WEIGHTS=1` for inference. The task checkpoint already
|
| 52 |
+
contains the trained LLM, vision backbone, projector, action model, and EMA
|
| 53 |
+
weights. The loader still needs the Llama-2 tokenizer/config metadata from
|
| 54 |
+
`NousResearch/Llama-2-7b-hf`; it does not need to download the full Llama or
|
| 55 |
+
vision weight files again.
|
| 56 |
+
|
| 57 |
+
For future real-task training, the matching CogACT-Large initialization
|
| 58 |
+
checkpoint is available at:
|
| 59 |
+
|
| 60 |
+
```text
|
| 61 |
+
base_models/CogACT-Large/CogACT-Large.pt
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
The original Llama model weights are not duplicated in this repository because
|
| 65 |
+
they are gated third-party weights. Use the original Hugging Face model with
|
| 66 |
+
the appropriate access terms and token.
|
| 67 |
+
|
| 68 |
## Training configuration
|
| 69 |
|
| 70 |
- Task: Piper color sorting
|
|
|
|
| 115 |
|
| 116 |
```python
|
| 117 |
model = load_vla(
|
| 118 |
+
model_id_or_path="Haongchen/MemoryVLA",
|
| 119 |
load_for_training=False,
|
| 120 |
action_dim=7,
|
| 121 |
future_action_window_size=15,
|