smerkd/arrow-heads

ARROW auxiliary heads

Custom EMA auxiliary-head checkpoints trained on frozen ARROW features. The ARROW encoder/decoder weights are not included. Obtain the compatible base model separately from ifradlin/arrow.

Checkpoints

Directory	Training run	Contents	Saved step (zero-based)
weights/E0b	E0b_heads_only_bf16	Motion, covisibility, 128-D local + GeM global descriptors, sky	19,999
weights/E1	E1_motion_covis_reweighted_c4d	Same four heads, warm-started from E0b and trained for 10k further steps with reweighted sampling and C4D	9,999
weights/G1d	G1d_aplus_lang_owngeo_support_sky_video	Surface-bound Gaussian head, two local-neighbor attention blocks, 32-D language latent and learned language decoder	19,999

Token-head bundles contain 5,853,133 parameters each; G1d contains 5,791,411. Original inference-only .pt files are preserved alongside bit-exact .safetensors exports. Optimizer states, dataset RGB and base-model weights are not included. Each variant includes its head architecture JSON and recorded training YAML. E0b/E1 include their original saved evaluation reports.

These are research checkpoints, not a claim that the heads improve every SLAM scene or perform dynamic-object re-identification. The descriptor objective uses covisible static correspondences. Descriptor retrieval and sky evaluation metrics were not provided in the saved reports. G1d is the established earlier Gaussian baseline; this upload does not select or export every newer ongoing Gaussian experiment.

Recorded token-head evaluation

Numbers below are copied from the supplied eval.json, not newly rerun. Motion baseline is the original ARROW motion prediction on the same evaluator inputs. AP/IoU are fractions.

Variant	Dataset	Motion AP	ARROW motion AP	Covisibility AP	Covisibility IoU
E0b	PointOdyssey val	0.3846	0.4278	0.9916	0.9281
E1	PointOdyssey val	0.4182	0.4278	0.9915	0.9192
E0b	MOVi-F val	0.9632	0.8800	0.9918	0.9249
E1	MOVi-F val	0.9467	0.8800	0.9920	0.9186

Covisibility-positive prevalence is approximately 86–87% on these reports. The configs also use dynamic_replica@val in training; that split must not be presented as held out for these checkpoints. Absolute paths in the YAML files record the training environment and need overrides when reproducing training elsewhere.

Load

Dependencies: PyTorch, NumPy, safetensors and huggingface_hub. Loading the heads does not download or instantiate ARROW.

import sys
import torch
from huggingface_hub import snapshot_download

snapshot = snapshot_download('smerkd/arrow-heads')  # use revision=<commit> for reproducibility
sys.path.insert(0, snapshot + '/code')
from arrow_heads import load_token_heads, load_gaussian_head

heads = load_token_heads('E1', device='cuda')
gaussian_head = load_gaussian_head(device='cuda')  # optional, separate from TokenHeads

# Existing ARROW encoding, not an additional image encoder:
# tokens: [B, S, h*w, 3072], id_tokens: [B, S, 3072]
# h,w: patch-grid dimensions; pairs: [P,2] source/target frame indices
with torch.inference_mode():
    outputs = heads(tokens.float(), id_tokens.float(), h, w, pairs)
# motion / sky / covisibility are logits, apply sigmoid for probabilities.
# descriptor_local and descriptor_global are normalized descriptors.

Token features use ARROW's 14-pixel patch grid. The Gaussian head consumes ARROW decoder-stage features (blocks 2, 5, 8) plus query embeddings; a Gaussian parameter vector is not by itself a rasterized splat. Its optional neighborhood context and construction helpers are in code/arrow_train/gaussians.py and neighbors.py. G1d's language decoder targets are included in assets/label_embeddings.npz; this decoder is not a person-identity head.

Validation and provenance

Before upload, all three original checkpoints loaded with torch.load(weights_only=True) on CPU. Their state dictionaries loaded strictly into the bundled implementations; CPU forward smoke checks passed, all parameters were finite, and safetensors roundtrips were bit-exact. These checks establish serialization/architecture compatibility, not fresh dataset performance. Bundled source is a loader-compatible snapshot captured on 2026-10-15, not a claim of the exact historical training source revision.

manifest.json records checkpoint steps, parameter counts and payload SHA256 hashes. Original checkpoint hashes were also compared against the files on the training volume. Project source licensing is retained in LICENSE; the separately obtained ARROW base model remains subject to its own terms.