docs/INSTALL_MACOS.md

macOS Installation (Apple Silicon)

Run the GEM demo scripts on a MacBook with Apple Silicon.

Platform note: Training and the full offline pipeline (demo_soma.py) work best with an NVIDIA GPU — see INSTALL.md. The ONNX accelerated demo (demo_soma_onnx.py) runs well on Apple Silicon using ONNX Runtime with CoreML.

Quick Setup (recommended)

Run the one-step setup script — it handles everything (environment, dependencies, models):

git clone --recursive https://github.com/NVlabs/GEM-X.git && cd GEM-X
bash scripts/setup_mac.sh

Then run the demo:

source .venv/bin/activate
python scripts/demo/demo_soma_onnx.py --video path/to/video.mp4

The rest of this document explains each step in detail if you prefer manual setup or need to troubleshoot.

---

Prerequisites

• macOS 13 (Ventura) or later
• Apple Silicon Mac (M1/M2/M3/M4)
• Python 3.12+
• uv package manager
• ~5 GB disk space for models and assets

Step 1 — Clone with submodules

git clone --recursive https://github.com/NVlabs/GEM-X.git
cd GEM-X

If you already cloned without --recursive:

git submodule update --init --recursive

Step 2 — Create virtual environment

pip install uv
uv venv .venv --python 3.12
source .venv/bin/activate

Step 3 — Install PyTorch (Apple Silicon)

# PyTorch with MPS (Metal Performance Shaders) backend — no CUDA needed
uv pip install torch torchvision

Verify MPS is available:

python -c "import torch; print('MPS:', torch.backends.mps.is_available())"
# Should print: MPS: True

Step 4 — Install SOMA body model

uv pip install -e third_party/soma
cd third_party/soma && git lfs pull && cd ../..

Step 5 — Install GEM and dependencies

bash scripts/install_env.sh

This script detects macOS and automatically skips detectron2 (which requires CUDA), installing ONNX Runtime instead.

Or install manually:

uv pip install -e .
uv pip install cloudpickle fvcore iopath pycocotools braceexpand roma 'setuptools<75'
uv pip install onnxruntime

Note: ONNX Runtime on macOS automatically includes the CoreML Execution Provider, which routes supported operations to the Apple Neural Engine (ANE) and GPU.

Step 6 — Download ONNX models

ONNX models are automatically downloaded from HuggingFace on first run. To download them ahead of time:

python -c "from gem.utils.hf_utils import download_all_onnx; download_all_onnx()"

Step 7 — SOMA assets for 3D rendering

The rendering pipeline requires SOMA body model assets:

# Create symlink (assets ship with the SOMA submodule after git lfs pull)
ln -sf third_party/soma/assets inputs/soma_assets

Run the demo

# ONNX accelerated demo (recommended on macOS)
python scripts/demo/demo_soma_onnx.py \
  --video path/to/video.mp4

# Standard demo (uses PyTorch — slower on macOS)
python scripts/demo/demo_soma.py \
  --video path/to/video.mp4 \
  --ckpt inputs/pretrained/gem_soma.ckpt

See DEMO.md for full argument reference and output descriptions.

Troubleshooting

Issue	Solution
MPS: False in PyTorch	Ensure macOS 13+ and torch>=2.0
import detectron2 errors	Not needed for demo_soma_onnx.py. Run bash scripts/install_env.sh which skips detectron2 on macOS
No ONNX/TRT denoiser found	Download ONNX models (Step 6)
YOLOX download fails	YOLOX auto-downloads on first run. Check internet connection
Very slow ONNX inference	Check [ONNX] Loaded ... (EP=...) log — should show CoreMLExecutionProvider. If not, reinstall onnxruntime

Model backend priority

The ONNX demo automatically selects the fastest available backend:

FP16 ONNX  →  INT8 ONNX  →  Full ONNX  →  PyTorch
    ↑                            ↑
quantize_onnx.py --fp16    download_all_onnx()

On macOS, VitPose is automatically converted to FP16 on first run (~2-3 min, one-time). Both VitPose and the denoiser use ONNX Runtime with CoreML EP for hardware acceleration.