Update README.md
Browse files
README.md
CHANGED
|
@@ -16,9 +16,9 @@ tags:
|
|
| 16 |
|
| 17 |
## Description:
|
| 18 |
|
| 19 |
-
|
| 20 |
Isaac `GR00T N1.5-3B` is the medium-sized version of our model built using pre-trained vision and language encoders, and uses a flow matching action transformer to model a chunk of actions conditioned on vision, language and proprioception.<br><br>
|
| 21 |
-
This model is ready for non-commercial use.
|
| 22 |
|
| 23 |
## License/Terms of Use
|
| 24 |
[Nvidia License](https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyIsIm5jaWQiOiJzby15b3V0LTg3MTcwMS12dDQ4In0=)<br>
|
|
@@ -60,6 +60,9 @@ Robot proprioception is encoded using a multi-layer perceptron (MLP) indexed by
|
|
| 60 |
Actions are encoded and velocity predictions decoded by an MLP, one per unique embodiment.
|
| 61 |
The flow matching transformer is implemented as a diffusion transformer (DiT), in which the diffusion step conditioning is implemented using adaptive layernorm (AdaLN).
|
| 62 |
|
|
|
|
|
|
|
|
|
|
| 63 |
## Input:
|
| 64 |
**Input Type:**
|
| 65 |
* Vision: Image Frames<br>
|
|
@@ -84,10 +87,17 @@ The flow matching transformer is implemented as a diffusion transformer (DiT), i
|
|
| 84 |
|
| 85 |
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
|
| 86 |
|
| 87 |
-
##
|
| 88 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
|
| 90 |
-
**
|
| 91 |
All of the below:
|
| 92 |
* NVIDIA Ampere
|
| 93 |
* NVIDIA Blackwell
|
|
@@ -98,14 +108,9 @@ All of the below:
|
|
| 98 |
**[Preferred/Supported] Operating System(s):**
|
| 99 |
* Linux
|
| 100 |
|
| 101 |
-
## Model Version(s):
|
| 102 |
-
Version 1.5.
|
| 103 |
-
|
| 104 |
## Ethical Considerations:
|
| 105 |
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
|
| 106 |
|
| 107 |
-
For more detailed information on ethical considerations for this model, please see the Model Card++ [Explainability](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/EXPLAINABILITY.md), [Bias](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/BIAS.md), [Safety & Security](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/SAFETY_and_SECURITY.md)), and [Privacy](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/PRIVACY.md) Subcards.
|
| 108 |
-
|
| 109 |
Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
|
| 110 |
|
| 111 |
## Resources
|
|
|
|
| 16 |
|
| 17 |
## Description:
|
| 18 |
|
| 19 |
+
`GN1x-Tuned-Arena-G1-Loco-Manipulation` is a fine tuned NVIDIA Isaac GR00T N1.5 model for the locomanipulation pick and place task provided in IsaacLab Arena.<br><br>
|
| 20 |
Isaac `GR00T N1.5-3B` is the medium-sized version of our model built using pre-trained vision and language encoders, and uses a flow matching action transformer to model a chunk of actions conditioned on vision, language and proprioception.<br><br>
|
| 21 |
+
This model is ready for non-commercial use only.
|
| 22 |
|
| 23 |
## License/Terms of Use
|
| 24 |
[Nvidia License](https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyIsIm5jaWQiOiJzby15b3V0LTg3MTcwMS12dDQ4In0=)<br>
|
|
|
|
| 60 |
Actions are encoded and velocity predictions decoded by an MLP, one per unique embodiment.
|
| 61 |
The flow matching transformer is implemented as a diffusion transformer (DiT), in which the diffusion step conditioning is implemented using adaptive layernorm (AdaLN).
|
| 62 |
|
| 63 |
+
**Number of Model Parameters:**
|
| 64 |
+
3B
|
| 65 |
+
|
| 66 |
## Input:
|
| 67 |
**Input Type:**
|
| 68 |
* Vision: Image Frames<br>
|
|
|
|
| 87 |
|
| 88 |
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
|
| 89 |
|
| 90 |
+
## Model Version(s):
|
| 91 |
+
Version 1.5.
|
| 92 |
+
|
| 93 |
+
## Post-Training Dataset:
|
| 94 |
+
Data Collection Method by Dataset: Synthetically generated datasets, nvidia/Arena-G1-Loco-Manipulation-Task
|
| 95 |
+
Labeling Method by Dataset: Not Applicable
|
| 96 |
+
|
| 97 |
+
## Inference:
|
| 98 |
+
**Acceleration Engine(s):** PyTorch
|
| 99 |
|
| 100 |
+
**Test Hardwares**
|
| 101 |
All of the below:
|
| 102 |
* NVIDIA Ampere
|
| 103 |
* NVIDIA Blackwell
|
|
|
|
| 108 |
**[Preferred/Supported] Operating System(s):**
|
| 109 |
* Linux
|
| 110 |
|
|
|
|
|
|
|
|
|
|
| 111 |
## Ethical Considerations:
|
| 112 |
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
|
| 113 |
|
|
|
|
|
|
|
| 114 |
Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
|
| 115 |
|
| 116 |
## Resources
|