hitnet / README.md

Model card synced with fastforks 7204e33f

305cfac verified about 3 hours ago

7.77 kB

---
library_name: "fastforks"
pipeline_tag: "depth-estimation"
license: "other"
license_name: "apache-2.0-code-weights-unstated"
license_link: "https://github.com/google-research/google-research/blob/f200f2b89d29484bf0dd0fef0c3979ac8bed9af4/LICENSE"
tags:
- "stereo"
- "stereo-matching"
- "disparity"
- "cuda"
- "pytorch"
- "fastforks"
---
# HITNet (fastforks)

HITNet (Tankovich et al. 2021): slanted tile hypotheses propagated and refined from coarse to fine; the authors' ETH3D (gray, max disparity 128) and Middlebury (half resolution, max disparity 400) graphs.

This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: <https://github.com/kleinicke/fastforks>. Kernels for the `kernels` backend: <https://huggingface.co/fastforks/kernels>. Original code: <https://github.com/google-research/google-research> at `f200f2b89d29`.

Task: stereo. Tier: stable. Outputs: `disparity`. Backends: `portable`, `kernels`.

Input: a rectified left / right image pair. Output: a disparity map (pixels, left view).

## Variants

Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in [`card.toml`](https://github.com/kleinicke/fastforks/blob/main/src/fastforks/stereo/models/hitnet/card.toml).

| variant | weights (author link) | SHA-256 | weights licence | commercial use |
|---|---|---|---|---|
| `hitnet-eth3d@1` | [eth3d.pb](https://s3.ap-northeast-2.wasabisys.com/pinto-model-zoo/142_HITNET/resources.tar.gz) | `ab34a01de126` | Apache-2.0 (assumed from code) | yes |
| `hitnet-middlebury-d400@1` | [middlebury_d400.pb](https://s3.ap-northeast-2.wasabisys.com/pinto-model-zoo/142_HITNET/resources.tar.gz) | `cc386fbb3181` | Apache-2.0 (assumed from code) | yes |

## Licence

Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Files: `LICENSE`.

Code licence: Apache-2.0; the authors do not state a separate weights licence.

Training data: not stated by the authors.

- Licence note: The authors state no weights licence; the checkpoints are fetched from the PINTO model-zoo mirror (a third-party host), not from Google, and the code licence is assumed to cover them.

Licence evidence: <https://github.com/google-research/google-research/blob/f200f2b89d29484bf0dd0fef0c3979ac8bed9af4/LICENSE>

## Input size

any H x W >= 8, resized or padded inside the model
Tested up to (portable, fp16, RTX 5060 Ti 16 GB): `hitnet-eth3d` 3000x4000 (6.4 GiB peak allocated, 10.8 reserved); `hitnet-middlebury-d400` 1988x2880 (6.4 GiB peak allocated, 10.1 reserved)

## Speed and memory

### Stereo, Middlebury 2014 half, 994x1482

| model | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `hitnet-eth3d` | – | – | 67.78 | 23.16 | – | 2.9x | – | 860 / 1090 | 647 / 1248 |
| `hitnet-middlebury-d400` | – | – | 176.0 | 52.73 | – | 3.3x | – | 1744 / 2414 | 1263 / 2368 |

### Startup and first call

| model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
| --- | --- | --- | --- | --- | --- | --- |
| `hitnet-eth3d` | – | 0.09 | 0.07† | – | 0.08† | 0.03† |
| `hitnet-middlebury-d400` | – | 0.13 | 0.10† | – | 0.18† | 0.05† |

### torch.compile and TensorRT

| model | input | portable ms | compile ms | compile s (cold) | TensorRT fp16 ms | ONNX + build s | kernels ms | peak MiB (alloc / reserved): portable | compile | TensorRT (device) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `hitnet-eth3d` | 994x1482 | 67.78 | 27.7 | 39.1 | 31.1 | 136 | 23.16 | 860 / 1090 | 719 / 728 | 836 |

### Fixed-size speed and peak memory, all models

| model | mode | 640x480 | 1280x720 | 1920x1080 | 12 MP 4000x3000 |
| --- | --- | --- | --- | --- | --- |
| `hitnet-eth3d` | portable | 18.1 ms / 221 / 336 | 40.2 ms / 659 / 1004 | 92.6 ms / 1134 / 2438 | 579.4 ms / 6558 / 11110 |
| `hitnet-eth3d` | kernels | 5.2 ms / 140 / 470 | 14.5 ms / 392 / 482 | 30.8 ms / 836 / 1004 | 181.7 ms / 4814 / 5758 |
| `hitnet-middlebury-d400` | portable | 35.8 ms / 602 / 838 | 108.8 ms / 1067 / 2222 | 251.7 ms / 2392 / 3814 | OOM |
| `hitnet-middlebury-d400` | kernels | 11.7 ms / 266 / 966 | 33.1 ms / 759 / 978 | 74.2 ms / 1695 / 2248 | 426.2 ms / 9604 / 12664 |

### Apple silicon (MPS)

| model | `portable` | `kernels` | speed-up |
| --- | --- | --- | --- |
| `hitnet-eth3d` | 206 ms / 2444 MiB (float32) | 74 ms / 1409 MiB | 2.80x |
| `hitnet-middlebury-d400` | 448 ms / 4539 MiB (float32) | 175 ms / 1454 MiB | 2.57x |

Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).

Hardware, protocol and caveats: [benchmarks](https://github.com/kleinicke/fastforks/blob/main/docs/benchmarks.md) (RTX 5060 Ti 16 GB, sm_120). Original = the authors' code at the precision of portable (fp16, or `(fp32)` where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.
## Accuracy
### On public datasets
Original = the authors' code, same checkpoint and protocol.
| variant | dataset | portable | kernels | note |
|---|---|---|---|---|
| `hitnet-eth3d` | ETH3D two-view train, bad > 1 px % / EPE | 0.93 / 0.164 | 0.93 / 0.164 | trained on this dataset's public images; not a zero-shot result |
| `hitnet-eth3d` | Middlebury 2014 trainingH, bad > 2 px % / EPE | 17.25 / 8.340 | 17.30 / 8.356 |  |
| `hitnet-middlebury-d400` | ETH3D two-view train, bad > 1 px % / EPE | 7.01 / 0.899 | 6.98 / 0.872 |  |
| `hitnet-middlebury-d400` | Middlebury 2014 trainingH, bad > 2 px % / EPE | 7.54 / 1.187 | 7.52 / 1.184 | trained on this dataset's public images; not a zero-shot result |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, 2026-10-27.
### Parity with the original code
Every variant is checked against golden outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.02 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.15 px and p99 <= 1.0 px.
## Limitations
- CUDA output (float16, kernels, float32 with PyTorch's default TF32 convolutions) differs from the CPU by ~0.2 px mean / 4 px p99 at 720x1280: the argmin flips on near-equal tile costs. Full float32 (`torch.backends.cudnn.allow_tf32 = False`) matches the CPU to 0.00002 px.
- The weights are the authors' frozen TensorFlow graphs read without TensorFlow; the FlyingThings and Middlebury d160 / d288 graphs are not included.
- The original code is TensorFlow and was not timed.

## Usage

```python
from fastforks import stereo
model = stereo.load("hitnet-eth3d")  # mode="auto": the preferred backend from recorded measurements
result = model("left.png", "right.png")
result.disparity  # [H, W] float32 tensor, input resolution
result.backend  # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
```

Install with `pip install fastforks` (see <https://github.com/kleinicke/fastforks>). Other variants: `hitnet-eth3d`, `hitnet-middlebury-d400`.

## Kernels backend

`stereo.load("hitnet-eth3d", mode="kernels")` runs our CUDA kernels (`hitnet`, `hitnet_conv`). It needs a prebuilt kernel for your GPU architecture (downloaded from <https://huggingface.co/fastforks/kernels> on first use) and a recent NVIDIA driver; if there is none it raises `BackendUnavailableError` with the reason and never falls back silently. `mode="auto"` picks `kernels` when it can run and logs the choice.

<!-- staged from fastforks 7204e33f (2026-11-05) -->

	---
	library_name: "fastforks"
	pipeline_tag: "depth-estimation"
	license: "other"
	license_name: "apache-2.0-code-weights-unstated"
	license_link: "https://github.com/google-research/google-research/blob/f200f2b89d29484bf0dd0fef0c3979ac8bed9af4/LICENSE"
	tags:
	- "stereo"
	- "stereo-matching"
	- "disparity"
	- "cuda"
	- "pytorch"
	- "fastforks"
	---
	# HITNet (fastforks)

	HITNet (Tankovich et al. 2021): slanted tile hypotheses propagated and refined from coarse to fine; the authors' ETH3D (gray, max disparity 128) and Middlebury (half resolution, max disparity 400) graphs.

	This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: <https://github.com/kleinicke/fastforks>. Kernels for the `kernels` backend: <https://huggingface.co/fastforks/kernels>. Original code: <https://github.com/google-research/google-research> at `f200f2b89d29`.

	Task: stereo. Tier: stable. Outputs: `disparity`. Backends: `portable`, `kernels`.

	Input: a rectified left / right image pair. Output: a disparity map (pixels, left view).

	## Variants

	Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in [`card.toml`](https://github.com/kleinicke/fastforks/blob/main/src/fastforks/stereo/models/hitnet/card.toml).

	\| variant \| weights (author link) \| SHA-256 \| weights licence \| commercial use \|
	\|---\|---\|---\|---\|---\|
	\| `hitnet-eth3d@1` \| [eth3d.pb](https://s3.ap-northeast-2.wasabisys.com/pinto-model-zoo/142_HITNET/resources.tar.gz) \| `ab34a01de126` \| Apache-2.0 (assumed from code) \| yes \|
	\| `hitnet-middlebury-d400@1` \| [middlebury_d400.pb](https://s3.ap-northeast-2.wasabisys.com/pinto-model-zoo/142_HITNET/resources.tar.gz) \| `cc386fbb3181` \| Apache-2.0 (assumed from code) \| yes \|

	## Licence

	Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Files: `LICENSE`.

	Code licence: Apache-2.0; the authors do not state a separate weights licence.

	Training data: not stated by the authors.

	- Licence note: The authors state no weights licence; the checkpoints are fetched from the PINTO model-zoo mirror (a third-party host), not from Google, and the code licence is assumed to cover them.

	Licence evidence: <https://github.com/google-research/google-research/blob/f200f2b89d29484bf0dd0fef0c3979ac8bed9af4/LICENSE>

	## Input size

	any H x W >= 8, resized or padded inside the model
	Tested up to (portable, fp16, RTX 5060 Ti 16 GB): `hitnet-eth3d` 3000x4000 (6.4 GiB peak allocated, 10.8 reserved); `hitnet-middlebury-d400` 1988x2880 (6.4 GiB peak allocated, 10.1 reserved)

	## Speed and memory

	### Stereo, Middlebury 2014 half, 994x1482

	\| model \| time (ms): original fp32 \| original \| portable \| kernels \| kernels speed-up: vs original \| vs portable \| peak memory (MiB, allocated / reserved): original \| portable \| kernels \|
	\| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \|
	\| `hitnet-eth3d` \| – \| – \| 67.78 \| 23.16 \| – \| 2.9x \| – \| 860 / 1090 \| 647 / 1248 \|
	\| `hitnet-middlebury-d400` \| – \| – \| 176.0 \| 52.73 \| – \| 3.3x \| – \| 1744 / 2414 \| 1263 / 2368 \|

	### Startup and first call

	\| model \| load (s): original \| portable \| kernels \| first call (s): original \| portable \| kernels \|
	\| --- \| --- \| --- \| --- \| --- \| --- \| --- \|
	\| `hitnet-eth3d` \| – \| 0.09 \| 0.07† \| – \| 0.08† \| 0.03† \|
	\| `hitnet-middlebury-d400` \| – \| 0.13 \| 0.10† \| – \| 0.18† \| 0.05† \|

	### torch.compile and TensorRT

	\| model \| input \| portable ms \| compile ms \| compile s (cold) \| TensorRT fp16 ms \| ONNX + build s \| kernels ms \| peak MiB (alloc / reserved): portable \| compile \| TensorRT (device) \|
	\| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \| --- \|
	\| `hitnet-eth3d` \| 994x1482 \| 67.78 \| 27.7 \| 39.1 \| 31.1 \| 136 \| 23.16 \| 860 / 1090 \| 719 / 728 \| 836 \|

	### Fixed-size speed and peak memory, all models

	\| model \| mode \| 640x480 \| 1280x720 \| 1920x1080 \| 12 MP 4000x3000 \|
	\| --- \| --- \| --- \| --- \| --- \| --- \|
	\| `hitnet-eth3d` \| portable \| 18.1 ms / 221 / 336 \| 40.2 ms / 659 / 1004 \| 92.6 ms / 1134 / 2438 \| 579.4 ms / 6558 / 11110 \|
	\| `hitnet-eth3d` \| kernels \| 5.2 ms / 140 / 470 \| 14.5 ms / 392 / 482 \| 30.8 ms / 836 / 1004 \| 181.7 ms / 4814 / 5758 \|
	\| `hitnet-middlebury-d400` \| portable \| 35.8 ms / 602 / 838 \| 108.8 ms / 1067 / 2222 \| 251.7 ms / 2392 / 3814 \| OOM \|
	\| `hitnet-middlebury-d400` \| kernels \| 11.7 ms / 266 / 966 \| 33.1 ms / 759 / 978 \| 74.2 ms / 1695 / 2248 \| 426.2 ms / 9604 / 12664 \|

	### Apple silicon (MPS)

	\| model \| `portable` \| `kernels` \| speed-up \|
	\| --- \| --- \| --- \| --- \|
	\| `hitnet-eth3d` \| 206 ms / 2444 MiB (float32) \| 74 ms / 1409 MiB \| 2.80x \|
	\| `hitnet-middlebury-d400` \| 448 ms / 4539 MiB (float32) \| 175 ms / 1454 MiB \| 2.57x \|

	Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).

	Hardware, protocol and caveats: [benchmarks](https://github.com/kleinicke/fastforks/blob/main/docs/benchmarks.md) (RTX 5060 Ti 16 GB, sm_120). Original = the authors' code at the precision of portable (fp16, or `(fp32)` where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.

	## Accuracy

	### On public datasets

	Original = the authors' code, same checkpoint and protocol.

	\| variant \| dataset \| portable \| kernels \| note \|
	\|---\|---\|---\|---\|---\|
	\| `hitnet-eth3d` \| ETH3D two-view train, bad > 1 px % / EPE \| 0.93 / 0.164 \| 0.93 / 0.164 \| trained on this dataset's public images; not a zero-shot result \|
	\| `hitnet-eth3d` \| Middlebury 2014 trainingH, bad > 2 px % / EPE \| 17.25 / 8.340 \| 17.30 / 8.356 \| \|
	\| `hitnet-middlebury-d400` \| ETH3D two-view train, bad > 1 px % / EPE \| 7.01 / 0.899 \| 6.98 / 0.872 \| \|
	\| `hitnet-middlebury-d400` \| Middlebury 2014 trainingH, bad > 2 px % / EPE \| 7.54 / 1.187 \| 7.52 / 1.184 \| trained on this dataset's public images; not a zero-shot result \|

	Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, 2026-10-27.

	### Parity with the original code

	Every variant is checked against golden outputs of the original code at its pinned commit. CPU, float32: max \|diff\| <= 0.02 vs the original. GPU, float16 (the default), portable and kernels: mean \|diff\| <= 0.15 px and p99 <= 1.0 px.

	## Limitations

	- CUDA output (float16, kernels, float32 with PyTorch's default TF32 convolutions) differs from the CPU by ~0.2 px mean / 4 px p99 at 720x1280: the argmin flips on near-equal tile costs. Full float32 (`torch.backends.cudnn.allow_tf32 = False`) matches the CPU to 0.00002 px.
	- The weights are the authors' frozen TensorFlow graphs read without TensorFlow; the FlyingThings and Middlebury d160 / d288 graphs are not included.
	- The original code is TensorFlow and was not timed.

	## Usage

	```python
	from fastforks import stereo

	model = stereo.load("hitnet-eth3d") # mode="auto": the preferred backend from recorded measurements
	result = model("left.png", "right.png")
	result.disparity # [H, W] float32 tensor, input resolution
	result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
	```

	Install with `pip install fastforks` (see <https://github.com/kleinicke/fastforks>). Other variants: `hitnet-eth3d`, `hitnet-middlebury-d400`.

	## Kernels backend

	`stereo.load("hitnet-eth3d", mode="kernels")` runs our CUDA kernels (`hitnet`, `hitnet_conv`). It needs a prebuilt kernel for your GPU architecture (downloaded from <https://huggingface.co/fastforks/kernels> on first use) and a recent NVIDIA driver; if there is none it raises `BackendUnavailableError` with the reason and never falls back silently. `mode="auto"` picks `kernels` when it can run and logs the choice.

	<!-- staged from fastforks 7204e33f (2026-11-05) -->