Upload 4 files

README.md

@@ -0,0 +1,186 @@
+---
+license: other
+license_name: dinov3-license
+license_link: https://ai.meta.com/resources/models-and-libraries/dinov3-license
+base_model: facebook/dinov3-vitb16-pretrain-lvd1689m
+pipeline_tag: image-feature-extraction
+library_name: codon
+tags:
+- dino
+- dinov3
+- vision
+- feature-extraction
+- fp16
+---
+
+# DINOv3 ViT-B/16 — converted weights (codon layout, float16)
+
+This repository hosts the weights of
+[`facebook/dinov3-vitb16-pretrain-lvd1689m`](https://huggingface.co/facebook/dinov3-vitb16-pretrain-lvd1689m)
+converted for the `codon` implementation of DINOv3:
+
+- **renamed** to the codon naming convention (`codon.impl.DINOv3ViT`),
+- **cast to float16** (2× smaller than the original fp32 checkpoint),
+- **numerically verified** against the original weights and against the
+  `transformers` reference implementation.
+
+The model itself is unchanged: same architecture, same parameter values, same
+numerics up to fp16 quantization.
+
+## Why this exists
+
+The upstream checkpoints use the `transformers`/Meta naming layout
+(`layer.{i}.attention.*`, `layer_scale{1,2}.lambda1`, `embeddings.*`) and ship in
+fp32. `codon.impl.DINOv3ViT` uses its own convention (all linear layers are
+`*_proj`, the block list is `blocks`, Layer Scale is `gamma{1,2}`, register tokens
+are `storage_tokens`), so the weights need a one-time remap. This repository is
+that remap, pre-applied and stored in fp16 so it can be loaded directly with a
+single call.
+
+## Files
+
+| File | Description |
+| --- | --- |
+| `model.safetensors` | 163.4 MB, 211 tensors, all `F16`, codon key layout |
+| `config.json` | Architecture summary plus `"dtype": "float16"` and `"key_layout": "codon"` |
+| `preprocessor_config.json` | Image preprocessing parameters, carried over from the original model |
+| `README.md` | This file |
+
+## Key mapping
+
+| Original (transformers / Meta) | codon |
+| --- | --- |
+| `embeddings.patch_embeddings.*` | `patch_embed.*` |
+| `embeddings.cls_token` | `cls_token` |
+| `embeddings.register_tokens` | `storage_tokens` |
+| `layer.{i}.attention.q_proj.*` | `blocks.{i}.q_proj.*` |
+| `layer.{i}.attention.k_proj.*` | `blocks.{i}.k_proj.*` |
+| `layer.{i}.attention.v_proj.*` | `blocks.{i}.v_proj.*` |
+| `layer.{i}.attention.o_proj.*` | `blocks.{i}.o_proj.*` |
+| `layer.{i}.norm1.*` / `norm2.*` | `blocks.{i}.norm1.*` / `norm2.*` |
+| `layer.{i}.mlp.up_proj.*` / `down_proj.*` | `blocks.{i}.up_proj.*` / `down_proj.*` |
+| `layer.{i}.layer_scale1.lambda1` | `blocks.{i}.gamma1` |
+| `layer.{i}.layer_scale2.lambda1` | `blocks.{i}.gamma2` |
+| `norm.*` | `norm.*` |
+| `rope_embeddings.inv_freq` | dropped (rebuilt dynamically from input shape) |
+
+Two notes on the conversion:
+
+- The `inv_freq` rotary table is **not** stored. In this implementation the 2D axial
+  RoPE frequencies are recomputed from `rope_theta` and the input resolution at
+  runtime (a non-persistent buffer), which is what lets the same weights run at any
+  image size.
+- A zero-initialized `mask_token` is included as a placeholder. It is part of this
+  implementation's MAE architecture but is absent from the original checkpoint;
+  `load_pretrained(strict=True)` does not require it.
+
+## Model overview
+
+| Property | Value |
+| --- | --- |
+| Architecture | DINOv3 ViT (pre-norm Transformer, bidirectional attention, 2D axial RoPE) |
+| Parameters | 85.7 M |
+| Hidden size | 768 |
+| Layers / attention heads | 12 / 12 |
+| MLP hidden size | 3072 (ratio 4, GELU) |
+| Patch size / default resolution | 16 / 224×224 |
+| Register tokens | 4 |
+| RoPE theta | 100.0 |
+| Layer-norm eps | 1e-5 |
+| Layer Scale init | 1.0 |
+| Attention biases | q/v/o yes, k no |
+| Drop path | 0.0 (inference) |
+| Weight dtype | float16 |
+
+## Usage
+
+```python
+import torch
+from codon.impl import DINOv3ViT_Base
+
+# Pulls config.json + model.safetensors from this repository.
+# config.json says dtype=float16, so the model is built in float16.
+model = DINOv3ViT_Base.from_remote().eval()
+
+# Prefer fp32 compute? The weights are upcast on load.
+model32 = DINOv3ViT_Base.from_remote(dtype=torch.float32).eval()
+```
+
+`from_remote()` reads the architecture from `config.json` — `num_register_tokens`,
+`rope_theta`, `pos_embed_rescale`, `layer_norm_eps`, `key_bias` and `dtype` are all
+applied automatically, so no manual configuration is needed.
+
+To load the file yourself:
+
+```python
+model = DINOv3ViT_Base().half()
+model.load_pretrained('model.safetensors', strict=True, dtype=torch.float16)
+```
+
+### Feature extraction
+
+```python
+x = torch.randn(1, 3, 224, 224).half()
+
+with torch.no_grad():
+    feats = model.forward_features(x)
+
+feats['x_norm_clstoken']        # [1, 768]        CLS token, post-norm
+feats['x_storage_tokens']       # [1, 4, 768]     4 register tokens, post-norm
+feats['x_norm_patchtokens']     # [1, 196, 768]   14x14 patch tokens, post-norm
+feats['x_norm_alltokens']       # [1, 201, 768]   full post-norm sequence
+feats['x_prenorm']              # [1, 201, 768]   full pre-norm sequence
+```
+
+`forward(x)` (the default, `is_training=False`) returns just the CLS token of shape
+`[N, 768]`.
+
+Dense features at arbitrary resolutions work without interpolation, because the RoPE
+frequencies are derived from the actual patch grid:
+
+```python
+model.forward_features(torch.randn(1, 3, 256, 192).half())['x_norm_patchtokens'].shape
+# torch.Size([1, 192, 768])   -> 16x12 patch grid
+```
+
+Intermediate layers, optionally reshaped to feature maps:
+
+```python
+with torch.no_grad():
+    layers = model.get_intermediate_layers(x, n=[0, 5, 11])          # tuple of [N, HW, C]
+    maps = model.get_intermediate_layers(x, n=1, reshape=True)       # [N, C, 14, 14]
+```
+
+## Verification
+
+The conversion was checked at every step, and these checks are reproducible via
+`test/test_dinov3_vit.py` in the codon repository:
+
+| Check | Result |
+| --- | --- |
+| Key remap + `strict=True` load from `model.safetensors` | passes |
+| fp32 remapped weights vs `transformers` `DINOv3ViTModel`, 224×224 and 196×252 | `max|diff| = 0.000e+00` (bit-exact) |
+| Per-block outputs vs `transformers` `output_hidden_states` (layers 0, 5, 11) | `max|diff| < 1e-5` |
+| fp16 forward vs original fp32 model (CLS token) | `max|diff| = 6.9e-03` |
+| fp16 weight quantization error (per tensor) | `≤ 2.0e-03` |
+| Save → reload round trip | `max|diff| = 0.000e+00` |
+
+The fp16 error is well within the expected range for a 12-layer model: activations
+reach an order of magnitude of ~10-30, and fp16 carries roughly three significant
+decimal digits.
+## Precision notes
+
+- Inference in float16 works on CPU and CUDA. The rotary `cos`/`sin` tables are
+  computed in fp32 and then **cast to the activation dtype**, so attention inputs
+  stay in float16 throughout instead of being silently promoted to fp32.
+- If you need maximum fidelity, use `dtype=torch.float32`; the weights are exact
+  float16 representations of the original fp32 values, so this only recovers the
+  rounding that happened at export time.
+
+## License
+
+The weights are derived from Meta's DINOv3 and remain subject to the
+[DINOv3 License](https://ai.meta.com/resources/models-and-libraries/dinov3-license).
+Please read and comply with that license before use. In particular, the license
+governs acceptable use, redistribution and attribution; this conversion adds no
+additional permissions and no warranty.

config.json

@@ -0,0 +1,36 @@
+{
+  "architectures": [
+    "DINOv3ViT"
+  ],
+  "model_type": "dinov3_vit",
+  "library_name": "codon",
+  "key_layout": "codon",
+  "converted_from": "facebook/dinov3-vitb16-pretrain-lvd1689m",
+  "dinov3_repo": "CodonProject/DINOv3-ViT-Base",
+  "dtype": "float16",
+  "torch_dtype": "float16",
+  "hidden_size": 768,
+  "intermediate_size": 3072,
+  "num_hidden_layers": 12,
+  "num_attention_heads": 12,
+  "num_channels": 3,
+  "patch_size": 16,
+  "image_size": 224,
+  "num_register_tokens": 4,
+  "hidden_act": "gelu",
+  "layer_norm_eps": 1e-05,
+  "layerscale_value": 1.0,
+  "query_bias": true,
+  "key_bias": false,
+  "value_bias": true,
+  "proj_bias": true,
+  "mlp_bias": true,
+  "use_gated_mlp": false,
+  "rope_theta": 100.0,
+  "pos_embed_shift": null,
+  "pos_embed_jitter": null,
+  "pos_embed_rescale": 2.0,
+  "drop_path_rate": 0.0,
+  "attention_dropout": 0.0,
+  "initializer_range": 0.02
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8b1c229664c031632392b7890bea9d10c13fb8c972dc24710c1a8fa8e374b11
+size 171340048

preprocessor_config.json

@@ -0,0 +1,31 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": true,
+  "device": null,
+  "disable_grouping": null,
+  "do_center_crop": null,
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.485,
+    0.456,
+    0.406
+  ],
+  "image_processor_type": "DINOv3ViTImageProcessorFast",
+  "image_std": [
+    0.229,
+    0.224,
+    0.225
+  ],
+  "input_data_format": null,
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "return_tensors": null,
+  "size": {
+    "height": 224,
+    "width": 224
+  }
+}