sentiantai/lingbot-vision-small-onnx

LingBot-Vision-Small · ONNX (browser)

Unofficial ONNX conversion of LingBot-Vision-Small (ViT-S/16 dense visual-feature backbone) for browser / ONNX Runtime inference.

Converted and published by Ideal Real Labs Ltd (irlabs.co.uk) for Pristina. This is not an official Robbyant / LingBot release; names are used only to identify the upstream model.

Model details

Item	Value
Upstream model	LingBot-Vision-Small (backbone-only)
Architecture	ViT-S/16, embed dim 384, patch size 16, 4 register tokens, fp32 RoPE
Upstream HF	robbyant/lingbot-vision-vit-small
Upstream revision	127cbcec380de0bcd55bdc1b1fad3819850a6514
Upstream model.pt SHA-256	dca36562cb6b0b34504df6edc18fa282c5ef06fb375c3e91d5487247a1096f9d
Upstream code	Robbyant/lingbot-vision @ 151e46321bae4399f8568829f190c7bdec216b49
This conversion	Fixed 512×512 input → normalized patch tokens

What this is: a dense feature encoder.
What this is not: a segmentation / classification head.

Files

File	Role	SHA-256
lingbot_vits_512_fp16.onnx	Browser / deployment (recommended)	bf278c53965624c4bc1d50405d85d159661f42863eff47ad2d447c1a3b33ac85
lingbot_vits_512_fp32.onnx	Parity / reference	cb4e7726abf30ad4af398a81b765d9c551d9d04315916c6be510e7d43d005c47

I/O contract

• Input pixel_values: float32[1, 3, 512, 512] — RGB, ImageNet mean/std normalized
  • mean (0.485, 0.456, 0.406), std (0.229, 0.224, 0.225)
  • bilinear resize to 512×512 (square), values in NCHW
• Output patch_tokens: float32[1, 1024, 384] — normalized patch tokens (32×32 grid)

FP16 weights use float32 I/O (cast wrapper). RoPE tables remain float32 inside the FP16 graph.

Quick start (Python)

import numpy as np
import onnxruntime as ort
from PIL import Image

MEAN = np.array([0.485, 0.456, 0.406], dtype=np.float32).reshape(1, 3, 1, 1)
STD = np.array([0.229, 0.224, 0.225], dtype=np.float32).reshape(1, 3, 1, 1)

img = Image.open("image.jpg").convert("RGB").resize((512, 512), Image.BILINEAR)
x = np.asarray(img, dtype=np.float32).transpose(2, 0, 1)[None] / 255.0
x = (x - MEAN) / STD

sess = ort.InferenceSession("lingbot_vits_512_fp16.onnx", providers=["CPUExecutionProvider"])
tokens = sess.run(["patch_tokens"], {"pixel_values": x})[0]
print(tokens.shape)  # (1, 1024, 384)

Conversion notes

1. Load official backbone with lingbot_vision at the pinned git SHA.
2. Export PatchTokenExporter (patch tokens only) at opset 18 with math/eager attention.
3. Export FP16 from a half() copy with RoPE kept in float32.
4. Verify PyTorch vs ONNX Runtime CPU parity (FP32 cosine ≈ 1.0; FP16 cosine ≈ 0.999997).

Reproducible tooling: local lingbot-onnx conversion repository (Ideal Real Labs Ltd).

License

Upstream weights and code: Apache License 2.0 (Copyright 2026 LingBot-Vision Contributors).

This ONNX packaging and conversion: also under Apache-2.0, with the modification notice below.

LingBot-Vision-Small
Copyright 2026 LingBot-Vision Contributors
Licensed under the Apache License, Version 2.0.

Converted from the official PyTorch checkpoint to ONNX and modified
for browser inference by Ideal Real Labs Ltd (irlabs.co.uk) for Pristina.

See LICENSE and NOTICE in this repository. Use of the LingBot or Robbyant names does not imply endorsement.

Citation

If you use the upstream model, please cite:

@article{lingbot-vision2026,
  title={Vision Pretraining for Dense Spatial Perception},
  author={Fu, Zelin and Tan, Bin and Sun, Changjiang and Liu, Shaohui and Zheng, Kecheng and Xu, Yinghao and Zhu, Xing and Shen, Yujun and Xue, Nan},
  journal={arXiv preprint arXiv:2607.05247},
  year={2026}
}