Depth Anything V2 Metric + SAM 2.1 ONNX
Collection
Metric Depth Anything V2 (Hypersim 20 m, Virtual KITTI 80 m) and SAM 2.1 image-mode ONNX. Not the paper tables. β’ 6 items β’ Updated
How to use christianengel/SAM2.1-Small-Image-ONNX with sam2:
# Use SAM2 with images
import torch
from sam2.sam2_image_predictor import SAM2ImagePredictor
predictor = SAM2ImagePredictor.from_pretrained("christianengel/SAM2.1-Small-Image-ONNX")
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
predictor.set_image(<your_image>)
masks, _, _ = predictor.predict(<input_prompts>) # Use SAM2 with videos
import torch
from sam2.sam2_video_predictor import SAM2VideoPredictor
predictor = SAM2VideoPredictor.from_pretrained("christianengel/SAM2.1-Small-Image-ONNX")
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
state = predictor.init_state(<your_video>)
# add new prompts and instantly get the output on the same frame
frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>)
# propagate the prompts to get masklets throughout the video
for frame_idx, object_ids, masks in predictor.propagate_in_video(state):
...How to use christianengel/SAM2.1-Small-Image-ONNX with TensorRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
Encoder plus prompt decoder. No video. truck.jpg only, 1200Γ1800. Do not quote the tiny 16-image means as Small.
| Encoder input | image float32 1Γ3Γ1024Γ1024, stretch, ImageNet mean/std on 0β1 |
| Decoder | point or box. x = x_px / width * 1024. Labels 0, 1, 2, 3 |
| Outputs | masks logits (threshold 0), scores [1, 3] |
| Files | onnx/encoder_{fp32,fp16}.onnx, onnx/decoder_{fp32,fp16}.onnx |
FP32 IoU 1.0. FP16 point 0.99994, box 0.99987. Point area 638,200. Small FP16 keep-list is empty (I/O FP32).
from huggingface_hub import hf_hub_download
import onnxruntime as ort
enc = hf_hub_download("christianengel/SAM2.1-Small-Image-ONNX", "onnx/encoder_fp16.onnx")
session = ort.InferenceSession(enc, providers=["CPUExecutionProvider"])
jetson/build_tensorrt.sh encoder_fp16.onnx 1024 1024
No Small encoder time. Jetson Orin was not available. Ravi et al., https://arxiv.org/abs/2408.00714.
Base model
facebook/sam2.1-hiera-small