Spaces:
Sleeping
Sleeping
File size: 3,115 Bytes
49d36c0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 | # Installation
## Prerequisites
- Python 3.12+
- CUDA-compatible GPU with drivers supporting CUDA 12.6+
- [Git LFS](https://git-lfs.github.com/) (required for SOMA body model assets)
- [uv](https://github.com/astral-sh/uv) (fast Python package manager)
## Step 1 — Clone with submodules
```bash
git clone --recursive https://github.com/NVlabs/GEM-X.git
cd GEM-X
```
If you already cloned without `--recursive`:
```bash
git submodule update --init --recursive
```
## Step 2 — Create virtual environment
```bash
pip install uv
uv venv .venv --python 3.12
source .venv/bin/activate
```
## Step 3 — Install PyTorch with CUDA
```bash
# Adjust the CUDA version to match your GPU driver.
# See https://pytorch.org/get-started/locally/
uv pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126
```
| CUDA Version | Index URL |
|---|---|
| CUDA 12.6 | `https://download.pytorch.org/whl/cu126` |
| CUDA 13.0 | `https://download.pytorch.org/whl/cu130` |
## Step 4 — Install SOMA body model
```bash
uv pip install -e third_party/soma
cd third_party/soma && git lfs pull && cd ../..
```
## Step 5 — Install GEM and dependencies
```bash
bash scripts/install_env.sh
```
This installs the `gem` package in editable mode along with Detectron2 for human detection.
## Step 6 — (Optional) Install SOMA Retargeter for humanoid robot retargeting
To enable `--retarget` mode (retarget recovered motion to the Unitree G1 robot):
```bash
uv pip install -e third_party/soma-retargeter
```
> **Note:** The soma-retargeter submodule requires SSH access. If `third_party/soma-retargeter` is empty, run:
> ```bash
> git submodule update --init third_party/soma-retargeter
> ```
## Step 7 — Third-party model assets
**SOMA body model** — follow `third_party/soma/README.md` and place model assets under `inputs/soma_assets/`.
**SAM-3D-Body** — follow `third_party/sam-3d-body/README.md` to download the checkpoint.
## Pretrained Model Download
Download the pretrained GEM checkpoint:
- **GEM (SOMA)**: [gem_soma.ckpt](https://huggingface.co/nvidia/GEM-X)
You can also download manually via CLI:
```bash
huggingface-cli download nvidia/GEM-X gem_soma.ckpt --local-dir inputs/pretrained
```
Place it under `inputs/pretrained/` or pass the path via `--ckpt`.
## Expected Directory Layout
After setup, your `inputs/` directory should look like:
```
inputs/
├── pretrained/
│ └── gem_soma.ckpt
├── soma_assets/
│ ├── soma_model/
│ └── ...
└── sam3d/
└── checkpoint.pth
```
## Docker
A `Dockerfile` is provided at the repository root for reproducible setup. See the [Dockerfile](../Dockerfile) for details.
## Troubleshooting
| Issue | Solution |
|---|---|
| `git lfs` files are pointer files | Run `cd third_party/soma && git lfs pull` |
| CUDA version mismatch | Ensure PyTorch CUDA version matches your driver (`nvidia-smi`) |
| `ModuleNotFoundError: gem` | Ensure you ran `bash scripts/install_env.sh` with the venv activated |
| OpenGL/EGL errors | Set `PYOPENGL_PLATFORM=egl` and `EGL_PLATFORM=surfaceless` |
|