Add native transformers support (DINOv3ViT-compatible weights)

README.md

@@ -13,7 +13,7 @@ tags:
 - pytorch
 datasets:
 - custom
-library_name: pytorch
+library_name: transformers
 pipeline_tag: image-feature-extraction
 ---
 
@@ -21,7 +21,7 @@ pipeline_tag: image-feature-extraction
 
 **LingBot-Vision** is a family of self-supervised Vision Transformer backbones for dense spatial perception. The models are pretrained with masked boundary modeling, a boundary-centric objective that encourages spatially structured patch features while retaining strong semantic representations.
 
-This Hugging Face repository stores a backbone-only PyTorch checkpoint as `model.pt`. It is intended for inference, feature extraction, PCA visualization, and downstream dense prediction research.
+This Hugging Face repository stores backbone-only weights in two interchangeable formats: a Transformers-format checkpoint (`model.safetensors` + `config.json`, loadable with `AutoModel`) and the reference PyTorch checkpoint `model.pt`. It is intended for inference, feature extraction, PCA visualization, and downstream dense prediction research.
 
 ## Model Details
 
@@ -72,6 +72,32 @@ Each checkpoint contains backbone weights only. Training-time heads, optimizer s
 
 ## How to Load
 
+### With 🤗 Transformers
+
+This repository contains Transformers-format weights (`config.json`, `model.safetensors`), converted from the reference `model.pt` and numerically equivalent to it (fp32 max deviation < 1e-5). LingBot-Vision backbones are architecturally compatible with the DINOv3 ViT implementation in `transformers` (>= 4.56), so they load natively — no custom code, no `trust_remote_code`:
+
+```python
+import torch
+from PIL import Image
+from transformers import AutoImageProcessor, AutoModel
+
+processor = AutoImageProcessor.from_pretrained("robbyant/lingbot-vision-vit-small")
+model = AutoModel.from_pretrained("robbyant/lingbot-vision-vit-small")
+
+image = Image.open("example.png").convert("RGB")
+inputs = processor(images=image, return_tensors="pt")
+
+with torch.no_grad():
+    outputs = model(**inputs)
+
+# token layout: [CLS, 4 register tokens, H*W patch tokens]
+num_register_tokens = model.config.num_register_tokens
+cls_token = outputs.pooler_output                                       # [B, C]
+patch_tokens = outputs.last_hidden_state[:, 1 + num_register_tokens :]  # [B, H*W, C]
+```
+
+### With the reference implementation
+
 Install the LingBot-Vision inference repository and dependencies:
 
 ```bash
@@ -90,13 +116,13 @@ Load a pretrained backbone:
 ```python
 import torch
 
-from lbot_vision_infer import load_pretrained_backbone
+from lingbot_vision import load_pretrained_backbone
 
 device = "cuda" if torch.cuda.is_available() else "cpu"
 dtype = torch.bfloat16 if device == "cuda" else torch.float32
 
 backbone, embed_dim = load_pretrained_backbone(
-    variant="large",
+    variant="small",
     device=device,
     dtype=dtype,
 )
@@ -113,7 +139,7 @@ The `variant` argument can be `giant`, `large`, `base`, or `small`. You can also
 - **Backbone:** Vision Transformer with patch size 16
 - **Released variants:** ViT-g/16, ViT-L/16, ViT-B/16, ViT-S/16
 - **Output:** Normalized patch tokens from the frozen backbone
-- **Checkpoint format:** Backbone-only `.pt` file stored as `model.pt`
+- **Checkpoint format:** Transformers `model.safetensors` (DINOv3ViT-compatible) and backbone-only `model.pt`
 - **Training objective:** Masked boundary modeling with self-distillation
 
 ### Software Requirements

config.json

@@ -0,0 +1,55 @@
+{
+  "apply_layernorm": true,
+  "architectures": [
+    "DINOv3ViTModel"
+  ],
+  "attention_dropout": 0.0,
+  "drop_path_rate": 0.0,
+  "dtype": "float32",
+  "hidden_act": "gelu",
+  "hidden_size": 384,
+  "image_size": 512,
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "key_bias": false,
+  "layer_norm_eps": 1e-05,
+  "layerscale_value": 1e-05,
+  "mlp_bias": true,
+  "model_type": "dinov3_vit",
+  "num_attention_heads": 6,
+  "num_channels": 3,
+  "num_hidden_layers": 12,
+  "num_register_tokens": 4,
+  "out_features": [
+    "stage12"
+  ],
+  "out_indices": [
+    12
+  ],
+  "patch_size": 16,
+  "pos_embed_jitter": null,
+  "pos_embed_rescale": 2.0,
+  "pos_embed_shift": null,
+  "proj_bias": true,
+  "query_bias": true,
+  "reshape_hidden_states": true,
+  "rope_theta": 100.0,
+  "stage_names": [
+    "stem",
+    "stage1",
+    "stage2",
+    "stage3",
+    "stage4",
+    "stage5",
+    "stage6",
+    "stage7",
+    "stage8",
+    "stage9",
+    "stage10",
+    "stage11",
+    "stage12"
+  ],
+  "transformers_version": "5.5.3",
+  "use_gated_mlp": false,
+  "value_bias": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a037c545ef5c3cc62ffa4f36b5778c3d4f0429200c4344089e4dc6633aafe40e
+size 86406384

preprocessor_config.json

@@ -0,0 +1,22 @@
+{
+  "image_processor_type": "DINOv3ViTImageProcessorFast",
+  "do_resize": true,
+  "size": {
+    "height": 512,
+    "width": 512
+  },
+  "resample": 2,
+  "do_rescale": true,
+  "rescale_factor": 0.00392156862745098,
+  "do_normalize": true,
+  "image_mean": [
+    0.485,
+    0.456,
+    0.406
+  ],
+  "image_std": [
+    0.229,
+    0.224,
+    0.225
+  ]
+}