--- library_name: "fastforks" pipeline_tag: "depth-estimation" license: "cc-by-nc-4.0" tags: - "stereo" - "stereo-matching" - "disparity" - "cuda" - "pytorch" - "fastforks" --- # S2M2 (fastforks) S2M2 (Min et al. 2025): multi-resolution transformer with optimal-transport matching; also outputs confidence and occlusion maps. This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: . Kernels for the `kernels` backend: . Original code: at `8f3bd785df59`. Author weight repos: . Task: stereo. Tier: stable. Outputs: `disparity`, `confidence`, `occlusion`. Backends: `portable`, `kernels`. Input: a rectified left / right image pair. Output: a disparity map (pixels, left view). ## Variants Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in [`card.toml`](https://github.com/kleinicke/fastforks/blob/main/src/fastforks/stereo/models/s2m2/card.toml). | variant | weights (author link) | SHA-256 | weights licence | commercial use | |---|---|---|---|---| | `s2m2-l@1` | [CH256NTR3.pth](https://huggingface.co/minimok/s2m2/resolve/main/CH256NTR3.pth) | `770e82227258` | CC-BY-NC-4.0 | **no** | | `s2m2-m@1` | [CH192NTR2.pth](https://huggingface.co/minimok/s2m2/resolve/main/CH192NTR2.pth) | `6e767e0977dd` | CC-BY-NC-4.0 | **no** | | `s2m2-s@1` | [CH128NTR1.pth](https://huggingface.co/minimok/s2m2/resolve/main/CH128NTR1.pth) | `93f7177d8159` | CC-BY-NC-4.0 | **no** | | `s2m2-xl@1` | [CH384NTR3.pth](https://huggingface.co/minimok/s2m2/resolve/main/CH384NTR3.pth) | `78dee08652ac` | CC-BY-NC-4.0 | **no** | ## Licence Original-code strings, recorded as they are (research-use package; check them before any other use). Code: CC-BY-NC-4.0. Files: `LICENSE.md`. Training data: README: extended training data incl. transparent objects. Licence evidence: ## Input size any H x W >= 32, padded to a multiple of 32 Tested up to (portable, fp16, RTX 5060 Ti 16 GB): `s2m2-l` 1988x2880 (11.4 GiB peak allocated, 14.2 reserved); `s2m2-m` 1988x2880 (8.9 GiB peak allocated, 11.5 reserved); `s2m2-s` 1988x2880 (7.4 GiB peak allocated, 10.6 reserved); `s2m2-xl` 1024x2048 (7.3 GiB peak allocated, 8.5 reserved) ## Speed and memory ### Stereo, Middlebury 2014 half, 994x1482 | model | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `s2m2-l` | 1935.6 | – | 1015.34 | 375.83 | – | 2.7x | – | 3245 / 3838 | 1249 / 1568 | | `s2m2-m` | 1241.7 | 666.2 | 670.36 | 247.87 | 2.7x | 2.7x | 2371 / 2920 | 2300 / 2786 | 876 / 1162 | | `s2m2-s` | 582.5 | 350.2 | 348.36 | 113.45 | 3.1x | 3.1x | 1759 / 1986 | 1733 / 1930 | 686 / 820 | | `s2m2-xl` | 3510.4 | 1796.1 | 1814.68 | 681.94 | 2.6x | 2.7x | 5606 / 6080 | 5228 / 6016 | 2092 / 2274 | ### Startup and first call | model | load (s): original | portable | kernels | first call (s): original | portable | kernels | | --- | --- | --- | --- | --- | --- | --- | | `s2m2-l` | 1.22 (375x1242) | 0.99 | 0.58† | 0.67 (375x1242) | 1.01† | 0.37† | | `s2m2-m` | 3.95 (375x1242) | 0.50 | 0.31† | 0.27 (375x1242) | 0.68† | 0.25† | | `s2m2-s` | 0.22 (375x1242) | 0.22 | 0.20† | 0.14 (375x1242) | 0.35† | 0.11† | | `s2m2-xl` | 4.50 (375x1242) | 2.73 | 0.90† | 0.53 (375x1242) | 1.81† | 0.68† | ### torch.compile and TensorRT | model | input | portable ms | compile ms | compile s (cold) | TensorRT fp16 ms | ONNX + build s | kernels ms | peak MiB (alloc / reserved): portable | compile | TensorRT (device) | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `s2m2-s` | 994x1482 | 348.36 | 185.9 | 95.9 | 137.1 | 363 | 113.45 | 1733 / 1930 | 1579 / 1710 | 980 | ### Fixed-size speed and peak memory, all models | model | mode | 640x480 | 1280x720 | 1920x1080 | 12 MP 4000x3000 | | --- | --- | --- | --- | --- | --- | | `s2m2-l` | portable | 182.7 ms / 1304 / 1460 | 618.6 ms / 2499 / 2956 | 1459.4 ms / 4660 / 5680 | OOM | | `s2m2-l` | kernels | 83.3 ms / 719 / 782 | 228.8 ms / 988 / 1224 | 534.8 ms / 1530 / 2250 | 4189.4 ms (n=3) / 6355 / 9246 | | `s2m2-m` | portable | 114.5 ms / 780 / 904 | 402.5 ms / 1690 / 1954 | 973.5 ms / 3330 / 3896 | OOM | | `s2m2-m` | kernels | 55.3 ms / 395 / 492 | 153.2 ms / 655 / 940 | 355.5 ms / 1119 / 1760 | 2793.3 ms (n=3) / 5151 / 7876 | | `s2m2-s` | portable | 56.0 ms / 456 / 536 | 202.3 ms / 1141 / 1450 | 511.1 ms / 2427 / 3054 | OOM | | `s2m2-s` | kernels | 32.7 ms / 212 / 294 | 70.2 ms / 452 / 692 | 161.9 ms / 887 / 1534 | 1203.3 ms / 4637 / 7166 | | `s2m2-xl` | portable | 338.0 ms / 2476 / 2710 | 1112.0 ms / 4248 / 4842 | 2594.0 ms (n=3) / 7447 / 8684 | SKIPPED | | `s2m2-xl` | kernels | 140.8 ms / 1441 / 1644 | 414.8 ms / 1748 / 2040 | 956.5 ms / 2493 / 3446 | 6933.0 ms (n=3) / 9200 / 11578 | Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB). Hardware, protocol and caveats: [benchmarks](https://github.com/kleinicke/fastforks/blob/main/docs/benchmarks.md) (RTX 5060 Ti 16 GB, sm_120). Original = the authors' code at the precision of portable (fp16, or `(fp32)` where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's. ## Accuracy ### On public datasets Original = the authors' code, same checkpoint and protocol. | variant | dataset | original | portable | kernels | note | |---|---|---|---|---|---| | `s2m2-s` | ETH3D two-view train, bad > 1 px % / EPE | 2.08 / 0.316 | 1.11 / 0.193 | 1.10 / 0.193 | possibly in the training data | | `s2m2-s` | Middlebury 2014 trainingH, bad > 2 px % / EPE | 3.79 / 0.833 | 3.23 / 0.652 | 3.24 / 0.652 | possibly in the training data | | `s2m2-m` | ETH3D two-view train, bad > 1 px % / EPE | 1.18 / 0.247 | 0.95 / 0.178 | 0.95 / 0.178 | possibly in the training data | | `s2m2-m` | Middlebury 2014 trainingH, bad > 2 px % / EPE | 3.44 / 0.742 | 3.29 / 0.651 | 3.28 / 0.651 | possibly in the training data | | `s2m2-l` | ETH3D two-view train, bad > 1 px % / EPE | 1.13 / 0.246 | 0.84 / 0.169 | 0.82 / 0.169 | possibly in the training data | | `s2m2-l` | Middlebury 2014 trainingH, bad > 2 px % / EPE | 3.02 / 0.645 | 2.84 / 0.542 | 2.85 / 0.542 | possibly in the training data | | `s2m2-xl` | ETH3D two-view train, bad > 1 px % / EPE | 0.58 / 0.189 | 0.52 / 0.147 | 0.52 / 0.149 | possibly in the training data | | `s2m2-xl` | Middlebury 2014 trainingH, bad > 2 px % / EPE | 2.41 / 0.554 | 2.33 / 0.499 | 2.33 / 0.499 | possibly in the training data | Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, 2026-10-27. ### Parity with the original code Every variant is checked against golden outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.02 px and p99 <= 0.25 px. ## Limitations - At 994x1482 portable S2M2-S is about as fast as the original code and allocates slightly more memory; use kernels for speed and memory. ## Usage ```python from fastforks import stereo # non-commercial weights: run with FASTFORKS_NONCOMMERCIAL=1 in the environment model = stereo.load("s2m2-l") # mode="auto": the preferred backend from recorded measurements result = model("left.png", "right.png") result.disparity # [H, W] float32 tensor, input resolution result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16') ``` Install with `pip install fastforks` (see ). Other variants: `s2m2-l`, `s2m2-m`, `s2m2-s`, `s2m2-xl`. ## Kernels backend `stereo.load("s2m2-l", mode="kernels")` runs our CUDA kernels (`s2m2`, `s2m2_cutlass`, `cudnn_hacc`). It needs a prebuilt kernel for your GPU architecture (downloaded from on first use) and a recent NVIDIA driver; if there is none it raises `BackendUnavailableError` with the reason and never falls back silently. `mode="auto"` picks `kernels` when it can run and logs the choice.