docs/installation.md

# Installation

[← Meridian](../README.md) · [Inference](inference.md) · [Studio demo](../README.md#self-hosting-the-demo)

## Before downloading

- **Review the [licenses and intended use](../README.md#license).** The weights are not Apache 2.0,
  the MiniMax-H3 license has territorial restrictions, and the VGGT-Omega dependency is licensed
  separately for noncommercial research.
- **Use a CUDA GPU with substantial memory.** The released scripts run on one GPU and do not expose
  CPU inference, multi-GPU sharding, quantization, or CPU-offload options. The reported resident-service
  peak is approximately 88 GiB for 73 frames. The CLI logs approximately 82 GiB of peak PyTorch-allocated
  memory during denoising; this does not measure the whole-process peak or driver-level GPU usage.
  A 96 GB-class GPU is the reported configuration for takes up to 124 frames; 243-frame takes reach
  approximately 113 GiB in the service.
  Leave headroom for geometry caches, other processes, and differences between GB and GiB.
- **Allow disk space beyond the weights.** Meridian's two adapters total approximately 5 GiB, and the
  MiniMax-H3 transformer they load onto is approximately 62 GiB; the H3 VAE, VGGT-Omega checkpoint,
  package caches, uploads, and generated videos are additional.
- **Reference environment:** Python 3.12, CUDA 12.8, PyTorch 2.9.1, and torchvision 0.24.1.
  B200 is the reported benchmark GPU, not a claim that every CUDA GPU is validated.
- Have **Git**, **FFmpeg**, and **FFprobe** on `PATH`. Git is needed for the pinned Diffusers install;
  the Python packages do not install the FFmpeg command-line executable.

## 1. Create an environment and download the code

Run these commands in a shell with Python 3.12 available. `python` below always means the Python in
the activated environment. Sign in if repository access requires it. The command fetches code,
runtime assets, guides, and sample clips, not the optional showcase videos or model weights.

```bash
python3.12 -m venv .venv-meridian
source .venv-meridian/bin/activate
python -m pip install --upgrade pip
python -m pip install huggingface_hub

hf auth login
hf download Viggle/Meridian --local-dir Meridian \
  --include "README.md" "LICENSE*" "NOTICE" "MODIFICATIONS.md" "requirements.txt" \
  "recam/*" "inference/*" "service/*" "assets/*" "examples/*" \
  "docs/installation.md" "docs/inference.md"
cd Meridian
python -m pip install -r requirements.txt
python -m pip install peft==0.18.0

# These must work before loading any model weights or starting the GPU service.
python inference/sample.py --help
python service/app.py --help
```

The PEFT package is needed by the student adapter loader and is not currently listed in
`requirements.txt`; install it explicitly. Use a dedicated environment rather than upgrading a
shared inference environment in place.

Keep the Diffusers commit pinned by `requirements.txt` (`d6726f3`). The scripts use MiniMax-H3 classes
and modular-pipeline helpers that may not exist in another build, even if its version string includes
`dev`. Do not replace that dependency with an arbitrary PyPI release.

### Supply the base transformer and Meridian's two adapters

Meridian is **not** a transformer of its own: it is two LoRA adapters that load onto the unmodified
MiniMax-H3 transformer and are applied together. Download both from this repository, and the base
from MiniMax. Run from the Meridian directory:

```bash
hf download Viggle/Meridian --include "teacher_lora/*" "turbo_lora/*" --local-dir .
hf download MiniMaxAI/MiniMax-H3 --include "transformer/*" "vae/*" --local-dir ../MiniMax-H3
```

The adapters total approximately 5 GiB and the base transformer approximately 62 GiB. The base also
supplies the VAE, so this one download covers section 3 below. VGGT-Omega is obtained separately.

Point the CLI or Studio at the base with `--model-dir ../MiniMax-H3`; the adapters are found next to
the code:

```text
MiniMax-H3/            # the base download, a sibling of the release
  transformer/
  vae/
Meridian/
  inference/sample.py
  assets/
  teacher_lora/
    pytorch_lora_weights.safetensors
  turbo_lora/
    pytorch_lora_weights.safetensors
```

Alternatively, name the directories outright: `--ckpt /absolute/path/to/transformer` and
`--lora /abs/teacher_lora /abs/turbo_lora`. Order matters — the re-camera adapter first, the turbo
second — and **neither may be merged into the base weights**; `--lora` loads them as live adapters
and sums them, which is the combination the turbo was distilled against.

The first release shipped one 61.7 GiB fused teacher plus a single adapter. Those files still exist
under `legacy/` for reproducing earlier results; they are not what the current code expects.

## 2. Obtain VGGT-Omega separately

VGGT-Omega code and weights are **not redistributed here**. Request access to
[facebook/VGGT-Omega](https://huggingface.co/facebook/VGGT-Omega), read its license, and authenticate
with a Hugging Face account that has been granted access.

```bash
# Run from the Meridian release directory; the checkout is placed beside it.
git clone https://github.com/facebookresearch/vggt-omega ../vggt-omega
export VGGT_OMEGA_DIR="$(cd ../vggt-omega && pwd)"

hf auth login
hf download facebook/VGGT-Omega vggt_omega_1b_512.pt \
  --local-dir "$VGGT_OMEGA_DIR/checkpoints"
```

Follow the VGGT-Omega checkout's own dependency instructions if additional packages are needed.
Its source directory is imported directly; this release does not install it as a Python package.

By default, Meridian looks for
`$VGGT_OMEGA_DIR/checkpoints/vggt_omega_1b_512.pt`. If you already store the weight file elsewhere:

```bash
export VGGT_OMEGA_CKPT=/absolute/path/to/vggt_omega_1b_512.pt
```

Keep these exports in the shell that starts inference. The CLI and service also accept
`--vggt-repo /absolute/path/to/vggt-omega` and `--vggt /absolute/path/to/the/checkpoint.pt`.

Meta's FAIR Noncommercial Research License v1 restricts commercial use of the research materials
and their outputs or results. Here those results include the geometry used to make the reference
render. The Apache license on Meridian's code does not remove that restriction. Commercial use
requires an appropriately licensed geometry solution or permission from Meta; swapping the
geometry front end is not a built-in CLI option and requires integration work.

## 3. Provide the MiniMax-H3 VAE

Inference loads both `transformer/` and `vae/` from
[`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3); it never loads the text encoder.
If you followed section 1 you already have them. Otherwise:

```bash
hf download MiniMaxAI/MiniMax-H3 --include "transformer/*" "vae/*" --local-dir ../MiniMax-H3
```

Then add `--model-dir ../MiniMax-H3` to your CLI or service command. This path is the directory
**containing** `transformer/` and `vae/`, not either of them. Without the flag, the default Hub
identifier is used and the weights are loaded through the Hugging Face cache.

Do not run the turbo adapter on its own: it is a delta on the re-camera teacher, and on base H3
alone it is a mismatched delta.

## 4. Check the setup

These checks import the required components without loading their weights or starting inference:

```bash
ffmpeg -version
ffprobe -version
python -m pip check
python -c "import torch; print('torch:', torch.__version__, 'CUDA:', torch.version.cuda, 'available:', torch.cuda.is_available())"
python -c "import peft; from diffusers import AutoencoderKLMiniMaxH3, MiniMaxH3Transformer3DModel, MiniMaxH3Scheduler; from recam.h3 import pack; print('H3 and PEFT imports OK')"
python -c "import os, sys; sys.path.insert(0, os.environ['VGGT_OMEGA_DIR']); from vggt_omega.models import VGGTOmega; print('VGGT-Omega import OK')"
```

For a geometry-only check on the selected GPU:

```bash
CUDA_VISIBLE_DEVICES=0 python inference/sample.py \
  --video examples/media/sp_bouldering_hang.mp4 \
  --yaw 15 --sweep --gauge-only --out out/check
```

This loads VGGT-Omega and prints geometry diagnostics. It does not load the VAE or transformer, and
does not write the normal output videos. It is not a full inference or model-memory test.

Next: run the [first take](../README.md#quickstart), learn the [camera controls](inference.md), or
start the [Studio demo](../README.md#self-hosting-the-demo).

## Setup problems

| Symptom | Check |
|---|---|
| `hf` or `ffmpeg` not found | Activate the environment for `hf`; install the system FFmpeg tools separately and check `PATH`. |
| Hub access denied | Confirm the account has accepted the model's terms and received access; authenticate with that account. A token alone does not grant gated access. |
| `No module named vggt_omega` | `VGGT_OMEGA_DIR` must contain the `vggt_omega/` package. Export it in the same shell that starts the process. |
| `VGGT-Omega not found` | Check both the source checkout and checkpoint path; `VGGT_OMEGA_CKPT` must name the `.pt` file. |
| Cannot import a MiniMax-H3 class or layout helper | Reinstall the pinned requirements in the active environment; inspect `python -c "import diffusers; print(diffusers.__file__)"` for a conflicting checkout. |
| Missing PEFT or adapter-loading error | Install PEFT and confirm the adapter filename and both `--lora` directories. The run must print `lora: 2 adapter(s) active`; one adapter means the other silently did not land. |
| CUDA or attention-backend failure | Check the PyTorch/CUDA/driver combination against the reference environment. The service selects `_native_cudnn`; other hardware/backend combinations are not validated here. |
| Out of memory | Start with 73 output frames, a short source span, and no other GPU workload. The scripts do not automatically offload to CPU. The Studio keeps its models and recent geometry caches resident. |

## Lower-memory community work

Meridian retains MiniMax-H3's transformer architecture and uses precomputed text embeddings, so
inference does not load the text encoder. This is a starting point for adapting community memory-saving
techniques—not evidence that the remaining transformer, activations, VAE, and geometry fit a smaller GPU.

We welcome work on quantization and CPU offloading toward consumer GPUs such as the RTX 4090.
Diffusers documents [quantization](https://huggingface.co/docs/diffusers/main/en/quantization/overview)
and [memory reduction and offloading](https://huggingface.co/docs/diffusers/main/en/optimization/memory).
These are general integration references, not a tested Meridian recipe or a reason to replace the
pinned Diffusers build indiscriminately.

The current CLI and service move their models onto one CUDA device; neither exposes those optimizations.
A contribution needs to integrate them into the custom inference path and validate adapter loading,
reference conditioning, image quality, peak GPU/host memory, and end-to-end latency. There is no
verified RTX 4090 configuration or performance claim for this release.