README.md

---
language:
- en
library_name: bagel-mot
license: other
pipeline_tag: image-text-to-text
tags:
- geoug
- geometry
- reinforcement-learning
---

# GeoUG-Spatial

[Project page](https://yangr116.github.io/geoug-project-page/) ·
[Paper](https://yangr116.github.io/geoug-project-page/assets/geoug-paper.pdf)

**Generate geometry when it helps, then reason with it.**

GeoUG-Spatial is the final adaptive spatial reasoning model from
*GeoUG: Learning and Reasoning with Geometry — How Generation Helps Spatial
Understanding?* Given an image and a grounded question, the policy can answer
directly or generate metric depth, query geometry tools, and revise its answer.

| Property | Value |
|---|---|
| Stage | Spatial-policy RL, initialized from GeoUG-ColdStart |
| Objective | `spatial_gate_draft_v3`, generation cost λ = 0.1 |
| Weights | Merged EMA spatial policy |
| Precision | Original BF16 EMA weights |
| Runtime | GeoUG custom BAGEL/MoT loader |

## Download

```python
from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="rayruiyang/GeoUG-Spatial",
    local_dir="ckpt/geoug-spatial-rl-lambda0.1",
)
```

These are custom BAGEL/MoT weights. Use the GeoUG runtime rather than a generic
`AutoModel.from_pretrained` call. The weight repository contains the checkpoint
assets; inference and training code live in the GeoUG code checkout.

Verify the download:

```bash
(cd ckpt/geoug-spatial-rl-lambda0.1 && sha256sum -c checksums.sha256)
```

## Quick start

Follow the GeoUG checkout's standalone inference installation instructions
(Python 3.10/3.11, CUDA, and the documented FlashAttention environment). Run the
following commands from the checkout root after downloading the package:

```bash
python -m inference.spatial \
  --model-path ckpt/geoug-spatial-rl-lambda0.1 \
  --image /path/to/image.jpg \
  --question 'Which point is closer to the camera? A: <point>120 400</point>. B: <point>730 420</point>. Answer A or B.' \
  --mode adaptive \
  --no-do-sample \
  --num-timesteps 50 \
  --cfg-text-scale 4.0 \
  --cfg-img-scale 2.0 \
  --output-dir outputs/inference/spatial
```

Replace the example points with references appropriate for your image. The
GeoUG/BAGEL point convention normalizes x and y independently to 0–1000;
benchmark adapters convert their canonical coordinates at runtime.

The CLI enables the required reasoning tokens automatically. Custom runners
must set `BAGEL_REASONING_TOKENS=1`. `force_depth` and `force_answer` expose the
two policy endpoints for inspection; use `adaptive` for the adaptive model.
The spatial model contains its own depth generator; downloading GeoUG-Depth
is not required for spatial inference.

## Reported results

| Benchmark | Adaptive circular accuracy (%) |
|---|---:|
| 3DCueBench | 90.75 |
| CV-Bench-3D Textual | 97.92 |
| DA-2K Textual | 94.20 |

Mean depth-trigger rate across the three benchmarks: **63.4%**. Circular
accuracy credits a question only if every tested answer-option order is correct.

These results are reported in the paper. Full adaptive-policy reproduction
with the current evaluation code remains pending.

## Training and evaluation resources

The RL reward balances answer correctness against depth-generation cost:
`reward = answer_correct - 0.1 * used_depth`. See `docs/RL_TRAINING.md` in the
GeoUG checkout for the full method and training setup.

- [GeoUG training data](https://huggingface.co/datasets/rayruiyang/GeoUG_data)
- [3DCueBench](https://huggingface.co/datasets/rayruiyang/3DCueBench)
- [CV-Bench-3D Textual](https://huggingface.co/datasets/rayruiyang/CVBench3DTexutal)
- DA-2K Textual / Visual preparation: `evaluation/vlmevalkit/README.md`

Generated depth and spatial answers are model predictions. Their accuracy
depends on the input, coordinate interpretation, and geometric calibration.

## Model family

| Checkpoint | Intended use |
|---|---|
| [GeoUG-Spatial](https://huggingface.co/rayruiyang/GeoUG-Spatial) | Final adaptive spatial reasoning policy |
| [GeoUG-Depth](https://huggingface.co/rayruiyang/GeoUG-Depth) | Standalone metric-depth generation |
| [GeoUG-ColdStart](https://huggingface.co/rayruiyang/GeoUG-ColdStart) | SFT initialization for spatial-policy RL |

## Package and integrity

| File | Contents |
|---|---|
| `ema.safetensors` | BF16 EMA model weights, approximately 27.21 GiB |
| `ae.safetensors` | VAE weights |
| `llm_config.json`, `vit_config.json`, `config.json`, `generation_config.json` | Model and generation configuration |
| `tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt` | Tokenizer assets |
| `transform_config.yaml` | Image/depth preprocessing and normalization |
| `checksums.sha256` | SHA-256 checksums for the package files |

Download the complete repository. The release uses EMA weights and does not
require a separate non-EMA checkpoint or a shard index.

## License and attribution

GeoUG builds on the BAGEL/MoT architecture and its pretrained components.
This checkpoint accompanies the mixed-license GeoUG code release. Consult
`LICENSE` and `NOTICE` in the GeoUG code checkout for the
applicable component terms, including the CC BY-NC restrictions on the listed
BAGEL/DiT-derived code files. This card does not assign a new blanket license
to upstream components. The data repositories document their own source and
usage terms separately.

## Contact

For questions about this release, contact
[rayyang0116@connect.hku.hk](mailto:rayyang0116@connect.hku.hk).