Robotics
Safetensors
gr00t_n1_5
xyaoNV commited on
Commit
48f69bf
·
verified ·
1 Parent(s): afc7d11

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -11
README.md CHANGED
@@ -16,13 +16,13 @@ tags:
16
 
17
  ## Description:
18
 
19
- This is a fine tuned NVIDIA Isaac GR00T N1.5 model for the manipulation open door task provided in IsaacLab Arena.<br><br>
20
  Isaac `GR00T N1.5-3B` is the medium-sized version of our model built using pre-trained vision and language encoders, and uses a flow matching action transformer to model a chunk of actions conditioned on vision, language and proprioception.<br><br>
21
- This model is ready for non-commercial use.
22
 
23
  ## License/Terms of Use
24
  [Nvidia License](https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyIsIm5jaWQiOiJzby15b3V0LTg3MTcwMS12dDQ4In0=)<br>
25
- You are responsible for ensuring that your use of NVIDIA AI Foundation Models complies with all applicable laws. <br>
26
 
27
  ### Deployment Geography:
28
  Global
@@ -43,6 +43,7 @@ Flow Matching Policy:
43
  Black, Kevin, et al. "π0: A Vision-Language-Action Flow Model for General Robot Control." arXiv preprint arXiv:2410.24164 (2024).<br>
44
 
45
  ## Model Architecture:
 
46
  **Architecture Type:** Vision Transformer, Multilayer Perceptron, Flow matching Transformer
47
 
48
  Isaac GR00T N1.5 uses vision and text transformers to encode the robot's image observations and text instructions. The architecture handles a varying number of views per embodiment by concatenating image token embeddings from all frames into a sequence, followed by language token embeddings.
@@ -60,6 +61,9 @@ Robot proprioception is encoded using a multi-layer perceptron (MLP) indexed by
60
  Actions are encoded and velocity predictions decoded by an MLP, one per unique embodiment.
61
  The flow matching transformer is implemented as a diffusion transformer (DiT), in which the diffusion step conditioning is implemented using adaptive layernorm (AdaLN).
62
 
 
 
 
63
  ## Input:
64
  **Input Type:**
65
  * Vision: Image Frames<br>
@@ -84,10 +88,17 @@ The flow matching transformer is implemented as a diffusion transformer (DiT), i
84
 
85
  Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
86
 
87
- ## Software Integration:
88
- **Runtime Engine(s):** PyTorch
 
 
 
 
89
 
90
- **Supported Hardware Microarchitecture Compatibility:**
 
 
 
91
  All of the below:
92
  * NVIDIA Ampere
93
  * NVIDIA Blackwell
@@ -98,14 +109,9 @@ All of the below:
98
  **[Preferred/Supported] Operating System(s):**
99
  * Linux
100
 
101
- ## Model Version(s):
102
- Version 1.5.
103
-
104
  ## Ethical Considerations:
105
  NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
106
 
107
- For more detailed information on ethical considerations for this model, please see the Model Card++ [Explainability](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/EXPLAINABILITY.md), [Bias](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/BIAS.md), [Safety & Security](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/SAFETY_and_SECURITY.md)), and [Privacy](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/PRIVACY.md) Subcards.
108
-
109
  Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
110
 
111
  ## Resources
 
16
 
17
  ## Description:
18
 
19
+ GN1x-Tuned-Arena-GR1-Manipulation is a fine tuned NVIDIA Isaac GR00T N1.5 model for the manipulation open door task provided in IsaacLab Arena.<br><br>
20
  Isaac `GR00T N1.5-3B` is the medium-sized version of our model built using pre-trained vision and language encoders, and uses a flow matching action transformer to model a chunk of actions conditioned on vision, language and proprioception.<br><br>
21
+ This model is ready for non-commercial use only.
22
 
23
  ## License/Terms of Use
24
  [Nvidia License](https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf?t=eyJscyI6ImdzZW8iLCJsc2QiOiJodHRwczovL3d3dy5nb29nbGUuY29tLyIsIm5jaWQiOiJzby15b3V0LTg3MTcwMS12dDQ4In0=)<br>
25
+ You are responsible for ensuring that your use of NVIDIA provided Models complies with all applicable laws. <br>
26
 
27
  ### Deployment Geography:
28
  Global
 
43
  Black, Kevin, et al. "π0: A Vision-Language-Action Flow Model for General Robot Control." arXiv preprint arXiv:2410.24164 (2024).<br>
44
 
45
  ## Model Architecture:
46
+
47
  **Architecture Type:** Vision Transformer, Multilayer Perceptron, Flow matching Transformer
48
 
49
  Isaac GR00T N1.5 uses vision and text transformers to encode the robot's image observations and text instructions. The architecture handles a varying number of views per embodiment by concatenating image token embeddings from all frames into a sequence, followed by language token embeddings.
 
61
  Actions are encoded and velocity predictions decoded by an MLP, one per unique embodiment.
62
  The flow matching transformer is implemented as a diffusion transformer (DiT), in which the diffusion step conditioning is implemented using adaptive layernorm (AdaLN).
63
 
64
+ **Number of Model Parameters:**
65
+ 3B
66
+
67
  ## Input:
68
  **Input Type:**
69
  * Vision: Image Frames<br>
 
88
 
89
  Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
90
 
91
+ ## Model Version(s):
92
+ Version 1.5.
93
+
94
+ ## Post-Training Dataset:
95
+ Data Collection Method by Dataset: Synthetically generated datasets, nvidia/Arena-GR1-Manipulation-Task
96
+ Labeling Method by Dataset: Not Applicable
97
 
98
+ ## Inference:
99
+ **Acceleration Engine(s):** PyTorch
100
+
101
+ **Test Hardwares**
102
  All of the below:
103
  * NVIDIA Ampere
104
  * NVIDIA Blackwell
 
109
  **[Preferred/Supported] Operating System(s):**
110
  * Linux
111
 
 
 
 
112
  ## Ethical Considerations:
113
  NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
114
 
 
 
115
  Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
116
 
117
  ## Resources