Instructions to use Haongchen/MemoryVLA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Piper
How to use Haongchen/MemoryVLA with Piper:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Document mirrored Llama metadata and CogACT training base
Browse files
README.md
CHANGED
|
@@ -50,9 +50,16 @@ checkpoints/frozen-ema-step-20000.pt
|
|
| 50 |
|
| 51 |
Set `MEMORYVLA_SKIP_BASE_WEIGHTS=1` for inference. The task checkpoint already
|
| 52 |
contains the trained LLM, vision backbone, projector, action model, and EMA
|
| 53 |
-
weights. The
|
| 54 |
-
|
| 55 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
|
| 57 |
For future real-task training, the matching CogACT-Large initialization
|
| 58 |
checkpoint is available at:
|
|
@@ -61,6 +68,10 @@ checkpoint is available at:
|
|
| 61 |
base_models/CogACT-Large/CogACT-Large.pt
|
| 62 |
```
|
| 63 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
The original Llama model weights are not duplicated in this repository because
|
| 65 |
they are gated third-party weights. Use the original Hugging Face model with
|
| 66 |
the appropriate access terms and token.
|
|
|
|
| 50 |
|
| 51 |
Set `MEMORYVLA_SKIP_BASE_WEIGHTS=1` for inference. The task checkpoint already
|
| 52 |
contains the trained LLM, vision backbone, projector, action model, and EMA
|
| 53 |
+
weights. The required Llama-2 tokenizer/config metadata is mirrored at:
|
| 54 |
+
|
| 55 |
+
```text
|
| 56 |
+
base_models/llama-2-7b-hf/
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
It does not need to download the full Llama or vision weight files again.
|
| 60 |
+
Point `MEMORYVLA_LLAMA2_7B_PATH` at that downloaded metadata directory for
|
| 61 |
+
offline inference. The metadata is provided for loader compatibility; the
|
| 62 |
+
gated Llama model license still applies to the underlying model family.
|
| 63 |
|
| 64 |
For future real-task training, the matching CogACT-Large initialization
|
| 65 |
checkpoint is available at:
|
|
|
|
| 68 |
base_models/CogACT-Large/CogACT-Large.pt
|
| 69 |
```
|
| 70 |
|
| 71 |
+
The CogACT base checkpoint is only needed when initializing a new training
|
| 72 |
+
run. It is not needed when loading the Frozen+EMA task checkpoint for
|
| 73 |
+
inference.
|
| 74 |
+
|
| 75 |
The original Llama model weights are not duplicated in this repository because
|
| 76 |
they are gated third-party weights. Use the original Hugging Face model with
|
| 77 |
the appropriate access terms and token.
|