gatilin/mammothmoda2_mammothtok_codec

MammothModa2-Dev: complete MammothTok image codec

749 original tensors covering the AIMv2-based encoder, vector quantizer, semantic decoder and RGB reconstruction decoder. Selected from gen_tokenizer.image_tokenizer.* inside shard 10 of the official fixed checkpoint. Only this prefix is removed; tensor dtypes and bytes are preserved. Source offsets, per-tensor SHA-256 and code provenance are included.

The actual native constructor and stored weights use 16384 entries × 8 channels, despite the outer wrapper mentioning codebook_size=32768. Both construction and validation follow the stored native shape. Its spatial compression ratio is 14. The tokenizer has its own RGB reconstruction path; the full language/MoT backbone, DiT and separate diffusion VAE are excluded.

from huggingface_hub import snapshot_download
import sys, torch
folder = snapshot_download('gatilin/mammothmoda2_mammothtok_codec')
sys.path.insert(0, folder)
from component import load_model
codec = load_model(folder)
x = torch.zeros(1, 3, 224, 224) # native NCHW tensor; dimensions divisible by 14
with torch.inference_mode():
    quantized, _, info = codec.encode(x)
    reconstructed_rgb, semantic_features = codec.decode(quantized)

See source preprocessor_config.json for the original multimodal processor's pixel normalization settings. The native codec receives a preprocessed tensor; the example is an interface fixture, not an image preprocessing recipe. validate.py requires strict 749-entry loading, source payload SHA-256 equality, actual codebook shape, ID bounds, lookup parity and square/rectangular reconstruction with finite outputs. Executed results are in validation.json. These component checks do not establish downstream MLLM or image generation quality.

Weights and main source are released as Apache-2.0 in the official model README and code repository. AIM_SOURCE_LICENSE preserves the Apple software license accompanying the adapted AIMv2 implementation, alongside its unchanged copyright notices. Those source-specific terms remain applicable.