Hy4-flash Vision Encoder and Projector

A lossless extraction of the visual component from tencent/Hy4-flash, source revision d88c9dfd96ba4f2e6b92adba0b8e8f1c8696f959. This package includes every original vit.* tensor: the 27-layer visual encoder, patch and positional embeddings, visual projector/patch merger, normalization, and learned image-newline parameter. It does not contain the language model, tokenizer, MoE, Engram, or speculative decoder.

Component and provenance

  • 441 original BF16 tensors, 466,629,360 parameters; checkpoint keys, values, shapes, and dtypes are preserved.
  • Native visual width 1152, 16 attention heads, patch size 16, 27 transformer layers; the visual merger projects to width 3072. Source architecture is adapted from SigLIP2-So400m-Patch16-Res512.
  • model.safetensors SHA-256: 5fdf462167ec22a48a231b79196bd196d96c6838a108e80716369690f95980bc.
  • The selected tensors span 20 official checkpoint shards. For transfer, the official ModelScope mirror was used at revision 4c33bfeb0fc3e08c50242c59d85a222cfac90ea7. The two repositories' API-declared full-shard SHA-256 values and sizes matched for all 20 shards, and their safetensors headers were compared byte-for-byte. Extracted output payloads were compared byte-for-byte with the downloaded exact mirror ranges. The complete original shards were not downloaded or independently hashed. See extraction.json, tensor_mapping.json, and mirror_comparison.json.

Loading

The source model currently points to an upstream Transformers implementation under review. Install the tested implementation's source revision and use the provided component loader:

pip install 'transformers @ git+https://github.com/ManaEstras/transformers.git@c2ec850510a37978ea83df226589623d5f2be776' safetensors torch
from huggingface_hub import snapshot_download
from PIL import Image
import importlib.util
import sys
import torch

path = snapshot_download('gatilin/hy4-flash-vision-encoder')
spec = importlib.util.spec_from_file_location('hy4_visual_component', f'{path}/component.py')
component = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = component
spec.loader.exec_module(component)

model = component.load_model(path, dtype=torch.float32, device='cpu')
processor = component.load_processor(path)
image = Image.open('image.jpg').convert('RGB')
inputs = processor(images=image, return_tensors='pt')
with torch.inference_mode():
    output = model(pixel_values=inputs['pixel_values'], grid_thw=inputs['image_grid_thw'])
print(output.last_hidden_state.shape, output.pooler_output.shape)

last_hidden_state contains pre-merger patch features; pooler_output contains projected tokens with the native learned image-newline tokens. The number of tokens depends on the native image grid. The file retains original vit.* names; the loader applies the same vision-key renames as the pinned upstream PR at load time and uses strict state-dict loading. This component is not registered as an AutoModel and is not a full image-to-text model.

Validation

validation.json records the completed checks: all 441 tensors strictly loaded into the full 27-layer tower and projector; two real COCO128 images passed CPU float32 inference; for a 14×14 patch grid, shapes were [1, 196, 1152] before merging and [1, 56, 3072] after the native projector and newline insertion. Repeating the same image gave identical outputs; two different images produced different outputs. For one image, explicit softmax(QKᵀ)V at layers 0, 13, and 26 matched native SDPA with maximum absolute error below 4.8e-6.

The validation used the pinned Hy4-flash implementation files over a local Transformers 5.21.1 runtime. It did not independently install and test the entire pinned PR environment, reproduce original GPU inference, benchmark image understanding, or test the full language model. The released weights remain BF16; the recorded local test used CPU float32.

License and attribution

Weights and model configuration are released under the original Tencent Hy License, including its additional commercial terms; this is not an Apache-2.0 weight license. The copied upstream Transformers source files retain their original Apache-2.0 notices; see LICENSE-APACHE-2.0 and native_source/source_manifest.json. Refer to the upstream model and license for full terms.

Downloads last month
-
Safetensors
Model size
0.5B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for gatilin/hy4-flash-vision-encoder

Finetuned
(3)
this model

Collection including gatilin/hy4-flash-vision-encoder