README.md

---
license: other
tags:
- accessibility
- amodal-completion
- image-inpainting
- 3d-reconstruction
---

# AccessPath

AccessPath is an anonymous-review release for accessibility-scene amodal
completion. It includes the full, finite GPU execution path from mask proposal
through quality-gated 2D completion and visual 3D completion. The package also
keeps two direct, mask-driven entry points:

- **2D completion:** GPU inpainting that only changes the reviewed occluded
  region and preserves source pixels elsewhere.
- **Visual 3D completion:** a wrapper around the separately released
  [Amodal3R](https://huggingface.co/Sm0kyWu/Amodal3R) backend that produces
  Gaussian-splat and dense-mesh visual reconstructions from an RGB image and
  an aligned three-value amodal mask.

The five supported accessibility categories are `stairs`, `ramp`, `curb_cut`,
`tactile_paving`, and `walkway`.

## Relationship to prior work

AccessPath is **inspired by and adapted from** established research on promptable
segmentation, amodal completion, diffusion inpainting, monocular geometry, and
visual 3D reconstruction. It is not presented as a reimplementation or a copy
of any one prior system. The project-level contribution is the accessibility
adaptation: category-specific target/obstacle prompts, a visible-hidden-amodal
mask schema, constrained mask relations, mask-restricted 2D completion,
geometry/quality gates, a finite Slurm workflow, and review-oriented outputs.

When AccessPath actually calls an external model, that model remains an
independent backend with its own code, weights, and license. The precise
boundary between inspiration, optional comparison, and runtime use is listed
in [docs/DEPENDENCIES_AND_WEIGHTS.md](docs/DEPENDENCIES_AND_WEIGHTS.md).

## What is included

This repository contains project-level pipeline, mask, 2D, depth/geometry,
visual-3D adapter, verification, rendering, review-bundle, Slurm, and prompt
configuration code. The complete process is documented in
[docs/REPRODUCIBLE_PIPELINE.md](docs/REPRODUCIBLE_PIPELINE.md), and the
stage-to-source mapping is in [docs/SOURCE_MANIFEST.md](docs/SOURCE_MANIFEST.md).
For a beginner-oriented installation guide, exact dependency status, official
model links, weight-download commands, and citation information, read
[docs/DEPENDENCIES_AND_WEIGHTS.md](docs/DEPENDENCIES_AND_WEIGHTS.md) before
running an experiment.

It deliberately contains **no** source images, masks, model checkpoints,
environments, experiment outputs, logs, author information, or
machine-specific paths.

The 100-image reviewer subset is not distributed yet. Its source-image and
derived-mask redistribution status requires source-specific license and
privacy clearance. A data card and a release manifest will be added only after
that review is complete.

## Setup

Use Linux, Python 3.10+, a CUDA-capable GPU, and an environment compatible
with the chosen models. Create a fresh environment, install PyTorch for the
local CUDA version, then install the lightweight utilities:

```bash
python -m pip install -r requirements/runtime.txt
python -m pip install diffusers transformers accelerate safetensors
```

The visual-3D backend additionally needs the official Amodal3R environment
and its CUDA rasterizer dependencies. Follow the upstream installation guide;
do not copy the upstream source tree or checkpoints into this repository.

## Obtain models separately

No weights are redistributed here. Download each dependency only after
accepting its own license and access conditions.

| Component | Official source | Used by |
| --- | --- | --- |
| Stable Diffusion inpainting | [`sd-legacy/stable-diffusion-inpainting`](https://huggingface.co/sd-legacy/stable-diffusion-inpainting) | 2D completion |
| Amodal3R | [`Sm0kyWu/Amodal3R`](https://huggingface.co/Sm0kyWu/Amodal3R) | visual 3D completion |
| SAM 3 (optional mask proposal stage) | [`facebook/sam3`](https://huggingface.co/facebook/sam3) | mask proposals only |

Project-trained checkpoints are intentionally withheld during anonymous review.
They should be released only after verifying the training-data permissions,
base-model terms, privacy risk, and the paper's release policy.

The optional VGGT path and the non-runtime related-work references
(pix2gestalt, Open-World AMODAL, and Amodal Completion in the Wild) are
identified explicitly in [docs/DEPENDENCIES_AND_WEIGHTS.md](docs/DEPENDENCIES_AND_WEIGHTS.md).
They are not silently downloaded or executed by the default pipeline.

## Input masks

All masks must match the input RGB resolution. For 2D completion, provide the
visible-target, amodal-target, and obstacle masks. For visual 3D completion,
provide one aligned PNG with exactly these values:

```text
255  background
188  visible target
0    hidden target
```

The intended relations are `hidden = amodal AND NOT visible`, visible and
obstacle are disjoint, and hidden is a subset of obstacle. Automatic masks are
proposals and should be reviewed before they drive a completion result.

## Run 2D completion

```bash
python accesspath.py 2d -- \
  --image path/to/image.jpg \
  --target-visible-mask path/to/target_visible.png \
  --target-amodal-mask path/to/target_amodal.png \
  --obstacle-mask path/to/obstacle.png \
  --category stairs \
  --model path/to/stable-diffusion-inpainting \
  --output-dir outputs/example_2d \
  --device cuda
```

The selected RGB result and its quality/provenance metadata are written under
the output directory. Run `python accesspath.py 2d -- --help` for all options.

## Run the complete pipeline

The complete pipeline uses a bounded Slurm GPU job, so it queues for a GPU,
persists its result directory, and exits once the requested image is complete:

```bash
python accesspath.py pipeline -- \
  --image path/to/image.jpg \
  --category stairs \
  --output-dir outputs/stairs_demo
```

It runs mask proposals (or validates supplied reviewed masks), constrained
amodal-mask inference, 2D completion, depth/geometry diagnostics, visual 3D,
verification, and a compact review bundle. See
[docs/REPRODUCIBLE_PIPELINE.md](docs/REPRODUCIBLE_PIPELINE.md) for environment
variables, stage-by-stage behavior, outputs, and optional stage switches.

## Run visual 3D completion

Install the official Amodal3R runtime first, then run:

```bash
python accesspath.py 3d -- \
  --image path/to/image.jpg \
  --mask path/to/amodal_3value.png \
  --model Sm0kyWu/Amodal3R \
  --output-dir outputs/example_3d
```

This is a learned visual reconstruction, not metric geometry, a navigation
safety label, or a guarantee of a watertight or scaled mesh. Run
`python accesspath.py 3d -- --help` for output and rendering options.

## Attribution and release boundary

The 3D adapter calls Amodal3R and does not claim to reimplement that upstream
model. Consult [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) and the model
cards for all dependency terms. Repository code is provided for anonymous
academic review; a final license and any project checkpoint release will be
announced with the archival paper release.