Update README.md
Browse files
README.md
CHANGED
|
@@ -16,13 +16,13 @@ tags:
|
|
| 16 |
|
| 17 |
## Description:
|
| 18 |
|
| 19 |
-
|
| 20 |
Isaac `GR00T N1.5-3B` is the medium-sized version of our model built using pre-trained vision and language encoders, and uses a flow matching action transformer to model a chunk of actions conditioned on vision, language and proprioception.<br><br>
|
| 21 |
-
This model is ready for non-commercial use.
|
| 22 |
|
| 23 |
## License/Terms of Use
|
| 24 |
[Nvidia License](https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyIsIm5jaWQiOiJzby15b3V0LTg3MTcwMS12dDQ4In0=)<br>
|
| 25 |
-
You are responsible for ensuring that your use of NVIDIA
|
| 26 |
|
| 27 |
### Deployment Geography:
|
| 28 |
Global
|
|
@@ -43,6 +43,7 @@ Flow Matching Policy:
|
|
| 43 |
Black, Kevin, et al. "π0: A Vision-Language-Action Flow Model for General Robot Control." arXiv preprint arXiv:2410.24164 (2024).<br>
|
| 44 |
|
| 45 |
## Model Architecture:
|
|
|
|
| 46 |
**Architecture Type:** Vision Transformer, Multilayer Perceptron, Flow matching Transformer
|
| 47 |
|
| 48 |
Isaac GR00T N1.5 uses vision and text transformers to encode the robot's image observations and text instructions. The architecture handles a varying number of views per embodiment by concatenating image token embeddings from all frames into a sequence, followed by language token embeddings.
|
|
@@ -60,6 +61,9 @@ Robot proprioception is encoded using a multi-layer perceptron (MLP) indexed by
|
|
| 60 |
Actions are encoded and velocity predictions decoded by an MLP, one per unique embodiment.
|
| 61 |
The flow matching transformer is implemented as a diffusion transformer (DiT), in which the diffusion step conditioning is implemented using adaptive layernorm (AdaLN).
|
| 62 |
|
|
|
|
|
|
|
|
|
|
| 63 |
## Input:
|
| 64 |
**Input Type:**
|
| 65 |
* Vision: Image Frames<br>
|
|
@@ -84,10 +88,17 @@ The flow matching transformer is implemented as a diffusion transformer (DiT), i
|
|
| 84 |
|
| 85 |
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
|
| 86 |
|
| 87 |
-
##
|
| 88 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
|
| 90 |
-
|
|
|
|
|
|
|
|
|
|
| 91 |
All of the below:
|
| 92 |
* NVIDIA Ampere
|
| 93 |
* NVIDIA Blackwell
|
|
@@ -98,14 +109,9 @@ All of the below:
|
|
| 98 |
**[Preferred/Supported] Operating System(s):**
|
| 99 |
* Linux
|
| 100 |
|
| 101 |
-
## Model Version(s):
|
| 102 |
-
Version 1.5.
|
| 103 |
-
|
| 104 |
## Ethical Considerations:
|
| 105 |
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
|
| 106 |
|
| 107 |
-
For more detailed information on ethical considerations for this model, please see the Model Card++ [Explainability](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/EXPLAINABILITY.md), [Bias](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/BIAS.md), [Safety & Security](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/SAFETY_and_SECURITY.md)), and [Privacy](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/PRIVACY.md) Subcards.
|
| 108 |
-
|
| 109 |
Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
|
| 110 |
|
| 111 |
## Resources
|
|
|
|
| 16 |
|
| 17 |
## Description:
|
| 18 |
|
| 19 |
+
GN1x-Tuned-Arena-GR1-Manipulation is a fine tuned NVIDIA Isaac GR00T N1.5 model for the manipulation open door task provided in IsaacLab Arena.<br><br>
|
| 20 |
Isaac `GR00T N1.5-3B` is the medium-sized version of our model built using pre-trained vision and language encoders, and uses a flow matching action transformer to model a chunk of actions conditioned on vision, language and proprioception.<br><br>
|
| 21 |
+
This model is ready for non-commercial use only.
|
| 22 |
|
| 23 |
## License/Terms of Use
|
| 24 |
[Nvidia License](https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyIsIm5jaWQiOiJzby15b3V0LTg3MTcwMS12dDQ4In0=)<br>
|
| 25 |
+
You are responsible for ensuring that your use of NVIDIA provided Models complies with all applicable laws. <br>
|
| 26 |
|
| 27 |
### Deployment Geography:
|
| 28 |
Global
|
|
|
|
| 43 |
Black, Kevin, et al. "π0: A Vision-Language-Action Flow Model for General Robot Control." arXiv preprint arXiv:2410.24164 (2024).<br>
|
| 44 |
|
| 45 |
## Model Architecture:
|
| 46 |
+
|
| 47 |
**Architecture Type:** Vision Transformer, Multilayer Perceptron, Flow matching Transformer
|
| 48 |
|
| 49 |
Isaac GR00T N1.5 uses vision and text transformers to encode the robot's image observations and text instructions. The architecture handles a varying number of views per embodiment by concatenating image token embeddings from all frames into a sequence, followed by language token embeddings.
|
|
|
|
| 61 |
Actions are encoded and velocity predictions decoded by an MLP, one per unique embodiment.
|
| 62 |
The flow matching transformer is implemented as a diffusion transformer (DiT), in which the diffusion step conditioning is implemented using adaptive layernorm (AdaLN).
|
| 63 |
|
| 64 |
+
**Number of Model Parameters:**
|
| 65 |
+
3B
|
| 66 |
+
|
| 67 |
## Input:
|
| 68 |
**Input Type:**
|
| 69 |
* Vision: Image Frames<br>
|
|
|
|
| 88 |
|
| 89 |
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
|
| 90 |
|
| 91 |
+
## Model Version(s):
|
| 92 |
+
Version 1.5.
|
| 93 |
+
|
| 94 |
+
## Post-Training Dataset:
|
| 95 |
+
Data Collection Method by Dataset: Synthetically generated datasets, nvidia/Arena-GR1-Manipulation-Task
|
| 96 |
+
Labeling Method by Dataset: Not Applicable
|
| 97 |
|
| 98 |
+
## Inference:
|
| 99 |
+
**Acceleration Engine(s):** PyTorch
|
| 100 |
+
|
| 101 |
+
**Test Hardwares**
|
| 102 |
All of the below:
|
| 103 |
* NVIDIA Ampere
|
| 104 |
* NVIDIA Blackwell
|
|
|
|
| 109 |
**[Preferred/Supported] Operating System(s):**
|
| 110 |
* Linux
|
| 111 |
|
|
|
|
|
|
|
|
|
|
| 112 |
## Ethical Considerations:
|
| 113 |
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
|
| 114 |
|
|
|
|
|
|
|
| 115 |
Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
|
| 116 |
|
| 117 |
## Resources
|