Inkling 2 Small Vision Encoder and Projector

A lossless extraction of the visual component from thinkingmachines/Inkling-2-Small, revision 1df2d97d45210d94f781f42a6521392e5a539e46. This package includes all original model.visual.* tensors: the Perception Encoder-derived visual transformer, learned position and class embeddings, layer scales, and the two-layer visual-to-language projector. It excludes the language model, language tokenizer, mixture-of-experts weights, and speculative decoder.

Architecture and weights

  • 422 original BF16 tensors, 327,693,312 parameters. Original tensor keys, shapes, dtypes and values are retained.
  • 23 vision transformer layers, width 1024, 16 attention heads, native image patch size 20. A 2ร—2 spatial merge corresponds to one 40ร—40 image token; the included projector outputs width 4608.
  • model.safetensors SHA-256: 112c344cd13ab31fafd05354e2e47138e56f65659f4c4e12fffb70f4503fb5de.
  • All selected visual tensors are in model-00024-of-00024.safetensors. The official ModelScope mirror thinkingmachines/Inkling-2-Small, revision 474f37075eae8bf97585075393ce1da9bac5a22e, was used for range transfer. The Hugging Face and ModelScope APIs report the same full-shard SHA-256 and size, and their complete safetensors headers match byte-for-byte. Extracted payloads were checked byte-for-byte against downloaded exact mirror ranges. The complete original shard was not downloaded or independently hashed. See extraction.json, tensor_mapping.json, and mirror_comparison.json.
  • The parent model contains quantized language weights; all extracted visual tensors are BF16 and have not been quantized or converted.

Load the standalone component

Transformers added Inkling 2 support in version 5.23.0. Install a compatible release and the processor dependencies:

pip install 'transformers[torch,vision]>=5.23.0' 'safetensors>=0.8'
from huggingface_hub import snapshot_download
from PIL import Image
import importlib.util
import sys
import torch

path = snapshot_download('gatilin/inkling-2-small-vision-encoder')
spec = importlib.util.spec_from_file_location('inkling2_visual_component', f'{path}/component.py')
component = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = component
spec.loader.exec_module(component)

model = component.load_model(path, dtype=torch.float32, device='cpu')
processor = component.load_processor(path)
inputs = processor(images=Image.open('image.jpg').convert('RGB'), return_tensors='pt')
with torch.inference_mode():
    output = model(pixel_values=inputs['pixel_values'], image_grid_thw=inputs['image_grid_thw'])
print(output.last_hidden_state.shape, output.pooler_output.shape)

last_hidden_state includes a class token plus native visual patches; pooler_output contains projected merged image tokens. Sequence lengths depend on the image dimensions. The checkpoint keeps source key names; component.py applies the upstream Transformers vision-key mapping at load time, strictly loads all 422 tensors, and reconstructs the non-persistent axial RoPE buffer after meta initialization. This standalone component is not registered as an AutoModel and does not generate text.

Validation

validation.json records strict loading of all 422 tensors and complete CPU float32 forwards on two real COCO128 images. For each 224ร—224 image, the processor produced a 6ร—6 merged grid, [145, 1024] visual states including one class token and [36, 4608] projected outputs. Same-image repeated outputs matched exactly, and different images produced different outputs. For one image, selected attention layers 0, 11 and 22 were explicitly reconstructed after RoPE and compared with native SDPA; the maximum absolute error was below 5.3e-6. Visual-key conditional attention entropy and class-token mass are recorded separately.

The validation used the official v5.23.0 Inkling 2 source files over a local Transformers 5.21.1 runtime. It did not independently test a full v5.23.0 installation, original GPU numerical parity, downstream understanding accuracy, or the complete multimodal model. The artifact remains BF16; only the local validation cast it to float32.

License

The upstream model card identifies Apache-2.0 as the model license; the license text is included in LICENSE. The official Transformers implementation files in native_source/ retain their Apache-2.0 notices. Consult the upstream model card for attribution and the provider's separate acceptable-use information.

Downloads last month
-
Safetensors
Model size
0.3B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for gatilin/inkling-2-small-vision-encoder

Quantized
(21)
this model

Collection including gatilin/inkling-2-small-vision-encoder