NLD-Uni RADIO visual tower and multimodal projector

This community extraction contains the encoder.vision_tower.* and encoder.mm_projector.* tensors from NVIDIA NLD-Uni, revision 5db755a2eb0ab84654e3dcf80c7d0245bfe36a50. It contains the continuous RADIO visual encoder and its understanding-side projection, not the discrete image tokenizer, diffusion language model or standalone image generator. Original names, shapes and dtypes are preserved.

extraction.json records the source revision, tensor count and SHA-256; tensor_mapping.json identifies the source offsets. validation.json documents the local forward test and its limitations. The source implementation and attribution notices are retained in vision.py, radio_patch.py, radio_cls.py and projector.py.

Usage

Install PyTorch, timm, einops, accelerate, safetensors, numpy and Pillow, then:

import sys
import torch
from PIL import Image
from huggingface_hub import snapshot_download
root = snapshot_download('gatilin/nld-uni-vision-encoder')
sys.path.insert(0, root)
from component import load_model
model = load_model(root)
image = Image.open('image.jpg').convert('RGB')
pixels = model.encoder.vision_tower.image_processor([image]).pixel_values
with torch.inference_mode():
    features = model.encoder.vision_tower(pixels)

The official processor uses variable-size grids; use the exact processed grid dimensions divided by the 16-pixel patch size when applying the projector. This visual component alone does not perform language or image generation.

License

Distributed under the bundled NVIDIA License. Its section 3.3 limits use to non-commercial research or evaluation (subject to its specified NVIDIA exception). Preserve the notices in the bundled source files.