Add files using upload-large-folder tool

README.md

@@ -0,0 +1,73 @@
+---
+license: apache-2.0
+base_model: thinkingmachines/Inkling
+base_model_relation: quantized
+pipeline_tag: image-text-to-text
+library_name: mlx
+tags:
+- mlx
+- moe
+- multimodal
+- inkling
+- thinking-machines
+---
+
+# Inkling-MLX-4bit
+
+**Built with Inkling (Thinking Machines Lab).**
+
+MLX (Apple Silicon) conversion of
+[thinkingmachines/Inkling](https://huggingface.co/thinkingmachines/Inkling),
+quantized to **4-bit** (affine group quant, group size 64).
+
+**Code / loader:** [github.com/PipeNetwork/inkling-mlx](https://github.com/PipeNetwork/inkling-mlx)
+
+Inkling is a **975B-total / 41B-active** sparse-MoE, natively multimodal model
+(text + image/video + audio → text). This is the **full multimodal** conversion:
+all three towers (text backbone, HMLP vision, dMel audio) are ported; the
+multi-token-prediction head is dropped (inference-irrelevant).
+
+## Quantizations
+
+| Variant | Size | Notes |
+|---|---|---|
+| [8bit](https://huggingface.co/pipenetwork/Inkling-MLX-8bit) | ~937 GB | near-lossless |
+| [6bit](https://huggingface.co/pipenetwork/Inkling-MLX-6bit) | ~717 GB | high quality |
+| [4bit](https://huggingface.co/pipenetwork/Inkling-MLX-4bit) | ~490 GB | balanced default |
+
+## ⚠️ Loading requires the bundled `inkling_mlx` loader
+
+The `inkling_mm_model` architecture is **not** in stock `mlx-lm` / `mlx-vlm`, so this
+repo bundles a minimal, numerically-validated MLX implementation under `inkling_mlx/`.
+
+```bash
+pip install mlx mlx-lm transformers
+```
+```python
+from inkling_mlx.load import load
+from inkling_mlx.generate import greedy_generate
+from transformers import AutoTokenizer
+
+model, config = load("/path/to/this/repo")
+tok = AutoTokenizer.from_pretrained("/path/to/this/repo", trust_remote_code=True)
+ids = tok("The capital of France is")["input_ids"]
+print(tok.decode(greedy_generate(model, config, ids, max_new_tokens=64)))
+```
+
+Needs an Apple-Silicon Mac with enough unified memory to hold the weights (≈ the
+size above).
+
+## Status & caveats
+
+- **Text generation** works end-to-end via an incremental KV + short-convolution cache.
+- The **vision and audio towers are ported and numerically validated** (fp32 parity
+  vs the PyTorch reference, max |Δ| ~1e-5), but currently take **pre-featurized**
+  inputs (`pixel_values` / `audio_input_ids`); the image/audio *preprocessing* pipeline
+  is not yet bundled.
+- Quantized: attention / MLP / expert projections, token embed+unembed, and the
+  vision/audio matmuls. Kept in higher precision: the MoE router, RMSNorms, the four
+  short-convolutions per layer, and the relative-position bias.
+
+Conversion is streaming (tensor-by-tensor; the ~1.9 TB bf16 model never fully loads
+into RAM) and was validated with fp32 numerical parity against transformers PR #47347.
+License: Apache-2.0 (inherits the base model).

chat_template.jinja

@@ -0,0 +1,129 @@
+{%- set effort_map = {"none": 0.0, "minimal": 0.1, "low": 0.2, "medium": 0.7, "high": 0.9, "max": 0.99} -%}
+{%- set role_token = {"user": "<|message_user|>", "assistant": "<|message_model|>", "system": "<|message_system|>", "tool": "<|message_tool|>"} -%}
+
+{%- macro emit_thinking_effort() -%}
+    {%- set eff = reasoning_effort if reasoning_effort is defined and reasoning_effort is not none else 0.9 -%}
+    {%- if eff is string -%}
+        {%- set key = eff | trim -%}
+        {%- if key not in effort_map -%}
+            {{- raise_exception("Unknown reasoning_effort: " ~ eff) -}}
+        {%- endif -%}
+        {%- set num = effort_map[key] -%}
+    {%- else -%}
+        {%- set num = eff | float -%}
+    {%- endif -%}
+    {%- if num < 0.0 or num > 0.99 -%}
+        {{- raise_exception("reasoning_effort must be in [0.0, 0.99]") -}}
+    {%- endif -%}
+    {{- "<|message_system|><|content_text|>Thinking effort level: " -}}
+    {%- if num == 0.0 -%}0{%- else -%}{{ num }}{%- endif -%}
+    {{- "<|end_message|>" -}}
+{%- endmacro -%}
+
+{%- if tools -%}
+    {%- set tool_state = namespace(specs=[]) -%}
+    {%- for tool in tools -%}
+        {%- set fn = tool.function if tool.function is defined else tool -%}
+        {%- set spec = {
+            "description": (fn.description if fn.description is defined and fn.description else ""),
+            "name": fn.name,
+            "parameters": (fn.parameters if fn.parameters is defined and fn.parameters else {}),
+            "type": (tool.type if tool.type is defined and tool.type else "function"),
+        } -%}
+        {%- set tool_state.specs = tool_state.specs + [spec] -%}
+    {%- endfor -%}
+    {{- "<|message_system|>tool_declare<|content_xml|>" -}}
+    {{- tool_state.specs | tojson(sort_keys=true, separators=(",", ":")) -}}
+    {{- "<|end_message|>" -}}
+{%- endif -%}
+
+{%- set state = namespace(effort_emitted=false) -%}
+{%- for message in messages -%}
+    {%- if message.role not in role_token -%}
+        {{- raise_exception("Unknown message role: " ~ message.role) -}}
+    {%- endif -%}
+    {%- if not state.effort_emitted and message.role != "system" -%}
+        {{- emit_thinking_effort() -}}
+        {%- set state.effort_emitted = true -%}
+    {%- endif -%}
+
+    {%- set rtok = role_token[message.role] -%}
+
+    {%- if message.role == "tool" -%}
+        {%- set tool_name_state = namespace(name="") -%}
+        {%- if message.name is defined and message.name -%}
+            {%- set tool_name_state.name = message.name -%}
+        {%- elif message.tool_call_id is defined and message.tool_call_id -%}
+            {%- for prev in messages -%}
+                {%- if prev.role == "assistant" and prev.tool_calls -%}
+                    {%- for tc in prev.tool_calls -%}
+                        {%- if tc.id is defined and tc.id == message.tool_call_id and tc.function.name is defined -%}
+                            {%- set tool_name_state.name = tc.function.name -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+        {{- rtok -}}
+        {%- if tool_name_state.name -%}{{- tool_name_state.name -}}{%- endif -%}
+        {{- "<|content_text|>" -}}
+        {%- if message.content is string -%}{{- message.content -}}{%- endif -%}
+        {{- "<|end_message|>" -}}
+
+    {%- else -%}
+        {%- if message.role == "assistant" and message.reasoning_content is defined and message.reasoning_content -%}
+            {{- "<|message_model|><|content_thinking|>" ~ message.reasoning_content ~ "<|end_message|>" -}}
+        {%- endif -%}
+
+        {%- if message.content is string -%}
+            {{- rtok ~ "<|content_text|>" ~ message.content ~ "<|end_message|>" -}}
+        {%- elif message.content -%}
+            {%- for part in message.content -%}
+                {%- if part is string -%}
+                    {{- rtok ~ "<|content_text|>" ~ part ~ "<|end_message|>" -}}
+                {%- elif part.type is not defined or part.type in ("text", "input_text") -%}
+                    {%- set text_part = (part.text if part.text is defined and part.text is string else "") -%}
+                    {{- rtok ~ "<|content_text|>" ~ text_part ~ "<|end_message|>" -}}
+                {%- elif part.type in ("image", "input_image", "image_url") -%}
+                    {{- rtok ~ "<|content_image|><|unused_200054|><|end_message|>" -}}
+                {%- elif part.type in ("audio", "input_audio", "audio_url") -%}
+                    {{- rtok ~ "<|content_audio_input|><|unused_200053|><|audio_end|><|end_message|>" -}}
+                {%- else -%}
+                    {{- raise_exception("Unsupported content part type: " ~ part.type) -}}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+
+        {%- if message.role == "assistant" and message.tool_calls -%}
+            {%- for tc in message.tool_calls -%}
+                {%- set fn = tc.function -%}
+                {%- if fn.name is not defined or fn.name is not string -%}
+                    {{- raise_exception("tool call function name must be a string") -}}
+                {%- endif -%}
+                {%- set args = fn.arguments if fn.arguments is defined and fn.arguments else {} -%}
+                {%- if args is string -%}
+                    {{- raise_exception("tool call arguments must be a parsed object, not a JSON string; canonicalize upstream") -}}
+                {%- endif -%}
+                {%- if args is not mapping -%}
+                    {{- raise_exception("tool call arguments must be an object") -}}
+                {%- endif -%}
+                {{- "<|message_model|>" ~ fn.name ~ "<|content_invoke_tool_json|>" -}}
+                {{- '{"name":' ~ (fn.name | tojson(sort_keys=true, separators=(",", ":"))) ~ ',"args":' -}}
+                {{- (args | tojson(sort_keys=true, separators=(",", ":"))) -}}
+                {{- "}<|end_message|>" -}}
+            {%- endfor -%}
+        {%- endif -%}
+
+        {%- if message.role == "assistant" -%}
+            {{- "<|content_model_end_sampling|>" -}}
+        {%- endif -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{%- if not state.effort_emitted -%}
+    {{- emit_thinking_effort() -}}
+{%- endif -%}
+
+{%- if add_generation_prompt -%}
+    {{- "<|message_model|>" -}}
+{%- endif -%}

config.json

@@ -0,0 +1,139 @@
+{
+  "architectures": [
+    "InklingForConditionalGeneration"
+  ],
+  "model_type": "inkling_mm_model",
+  "eos_token_id": 200006,
+  "text_config": {
+    "model_max_length": 1048576,
+    "torch_dtype": "bfloat16",
+    "hidden_size": 6144,
+    "num_hidden_layers": 66,
+    "vocab_size": 201024,
+    "num_attention_heads": 64,
+    "num_key_value_heads": 8,
+    "head_dim": 128,
+    "d_rel": 16,
+    "rel_extent": 1024,
+    "q_bias": false,
+    "o_bias": false,
+    "log_scaling_n_floor": 128000,
+    "log_scaling_alpha": 0.1,
+    "rms_norm_eps": 1e-06,
+    "use_embed_norm": true,
+    "local_layer_ids": [
+      0,
+      1,
+      2,
+      3,
+      4,
+      6,
+      7,
+      8,
+      9,
+      10,
+      12,
+      13,
+      14,
+      15,
+      16,
+      18,
+      19,
+      20,
+      21,
+      22,
+      24,
+      25,
+      26,
+      27,
+      28,
+      30,
+      31,
+      32,
+      33,
+      34,
+      36,
+      37,
+      38,
+      39,
+      40,
+      42,
+      43,
+      44,
+      45,
+      46,
+      48,
+      49,
+      50,
+      51,
+      52,
+      54,
+      55,
+      56,
+      57,
+      58,
+      60,
+      61,
+      62,
+      63,
+      64
+    ],
+    "dense_mlp_idx": 2,
+    "use_sconv": true,
+    "sconv_kernel_size": 4,
+    "unpadded_vocab_size": 200058,
+    "logits_mup_width_multiplier": 24.0,
+    "final_logit_softcapping": null,
+    "swa_head_dim": 128,
+    "swa_num_attention_heads": 64,
+    "swa_num_key_value_heads": 16,
+    "sliding_window_size": 512,
+    "n_routed_experts": 256,
+    "num_experts_per_tok": 6,
+    "n_shared_experts": 2,
+    "shared_expert_sink": true,
+    "dense_intermediate_size": 24576,
+    "intermediate_size": 3072,
+    "route_scale": 8.0,
+    "use_gate_bias": true,
+    "gate_activation": "sigmoid",
+    "norm_after_topk": true,
+    "use_global_scale": true
+  },
+  "audio_config": {
+    "decoder_dmodel": 6144,
+    "n_mel_bins": 80,
+    "mel_vocab_size": 16,
+    "bias": false,
+    "dmel_min_value": -7.0,
+    "dmel_max_value": 2.0,
+    "use_audio_norm": true,
+    "audio_mode": "dmel"
+  },
+  "vision_config": {
+    "vision_encoder_type": "hmlp",
+    "decoder_dmodel": 6144,
+    "patch_size": 40,
+    "temporal_patch_size": 2,
+    "n_channels": 3,
+    "n_layers": 4,
+    "use_vision_norm": true
+  },
+  "mtp_config": {
+    "num_nextn_predict_layers": 8,
+    "chain_hidden_post_norm": false,
+    "local_layer_ids": [
+      0,
+      2,
+      4,
+      5,
+      6,
+      7
+    ]
+  },
+  "quantization": {
+    "group_size": 64,
+    "bits": 4,
+    "recipe": "uniform"
+  }
+}

inkling_mlx/__init__.py

@@ -0,0 +1,14 @@
+"""MLX port of thinkingmachines/Inkling (975B MoE, natively multimodal)."""
+
+from .config import AudioConfig, InklingConfig, TextConfig, VisionConfig
+from .model import InklingForConditionalGeneration
+from .text import TextModel
+
+__all__ = [
+    "InklingConfig",
+    "TextConfig",
+    "VisionConfig",
+    "AudioConfig",
+    "InklingForConditionalGeneration",
+    "TextModel",
+]

inkling_mlx/attention.py

@@ -0,0 +1,135 @@
+"""Inkling attention: hybrid local/global, per-head q/k RMSNorm, relative-position
+logits bias, optional log-scaling, and short-convolution on k/v.
+
+Mirrors ``InklingAttention`` + ``InklingRelativeLogits`` from transformers PR #47347.
+This implementation is prefill-oriented (full-sequence, no KV cache); an incremental
+cache (including the 4 per-layer conv states) can be layered on top later.
+"""
+
+from __future__ import annotations
+
+import math
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from .common import RMSNorm, ShortConvolution
+from .config import TextConfig
+
+NEG_INF = -1e30
+
+
+class RelativeLogits(nn.Module):
+    """Hidden-state-conditioned relative position bias.
+
+    ``proj`` is a bank of bias-vs-distance profiles ``[d_rel, rel_extent]``. Each
+    query's ``d_rel`` relative-state vector mixes them into one bias value per
+    backward distance; the bias is zero outside ``0 <= distance < rel_extent``.
+    """
+
+    def __init__(self, d_rel: int, rel_extent: int):
+        super().__init__()
+        self.rel_extent = rel_extent
+        self.proj = mx.zeros((d_rel, rel_extent))
+
+    def __call__(self, relative_states, q_pos, kv_pos):
+        # relative_states: [B, Lq, heads, d_rel]
+        # rel_logits: [B, Lq, heads, rel_extent] -> [B, heads, Lq, rel_extent]
+        rel_logits = mx.swapaxes(relative_states @ self.proj, 1, 2)
+        B, H, Lq, _ = rel_logits.shape
+        distance = q_pos[:, None] - kv_pos[None, :]          # [Lq, Lkv]
+        gather = mx.clip(distance, 0, self.rel_extent - 1)   # [Lq, Lkv]
+        gather = mx.broadcast_to(gather[None, None], (B, H, Lq, gather.shape[-1]))
+        bias = mx.take_along_axis(rel_logits, gather, axis=-1)  # [B, H, Lq, Lkv]
+        valid = (distance >= 0) & (distance < self.rel_extent)  # [Lq, Lkv]
+        return mx.where(valid[None, None], bias, 0.0)
+
+
+class Attention(nn.Module):
+    def __init__(self, config: TextConfig, layer_idx: int):
+        super().__init__()
+        self.config = config
+        self.layer_idx = layer_idx
+        self.is_sliding = config.layer_types[layer_idx] == "hybrid_sliding"
+
+        self.head_dim = config.swa_head_dim if self.is_sliding else config.head_dim
+        self.num_heads = config.swa_num_attention_heads if self.is_sliding else config.num_attention_heads
+        self.num_kv_heads = config.swa_num_key_value_heads if self.is_sliding else config.num_key_value_heads
+        self.n_rep = self.num_heads // self.num_kv_heads
+        self.sliding_window = config.sliding_window_size if self.is_sliding else None
+        self.rel_extent = config.sliding_window_size if self.is_sliding else config.rel_extent
+        self.d_rel = config.d_rel
+        # q/k are per-head RMS-normalized, hence 1/d rather than 1/sqrt(d)
+        self.scaling = 1.0 / self.head_dim
+
+        h = config.hidden_size
+        self.wq_du = nn.Linear(h, self.num_heads * self.head_dim, bias=False)
+        self.wk_dv = nn.Linear(h, self.num_kv_heads * self.head_dim, bias=False)
+        self.wv_dv = nn.Linear(h, self.num_kv_heads * self.head_dim, bias=False)
+        self.wr_du = nn.Linear(h, self.num_heads * self.d_rel, bias=False)
+        self.wo_ud = nn.Linear(self.num_heads * self.head_dim, h, bias=False)
+
+        self.k_sconv = ShortConvolution(self.num_kv_heads * self.head_dim, config.sconv_kernel_size)
+        self.v_sconv = ShortConvolution(self.num_kv_heads * self.head_dim, config.sconv_kernel_size)
+        self.q_norm = RMSNorm(self.head_dim, eps=config.rms_norm_eps)
+        self.k_norm = RMSNorm(self.head_dim, eps=config.rms_norm_eps)
+        self.rel_logits_proj = RelativeLogits(self.d_rel, self.rel_extent)
+
+    def __call__(self, hidden_states, start_pos=0, kv_cache=None,
+                 k_conv=None, v_conv=None, conv_mask=None):
+        B, L, _ = hidden_states.shape
+
+        q = self.wq_du(hidden_states)
+        k = self.k_sconv(self.wk_dv(hidden_states), mask=conv_mask, cache=k_conv)
+        v = self.v_sconv(self.wv_dv(hidden_states), mask=conv_mask, cache=v_conv)
+        rel = self.wr_du(hidden_states)
+
+        q = self.q_norm(q.reshape(B, L, self.num_heads, self.head_dim))
+        k = self.k_norm(k.reshape(B, L, self.num_kv_heads, self.head_dim))
+        v = v.reshape(B, L, self.num_kv_heads, self.head_dim)
+
+        # -> [B, heads, L, head_dim]
+        q = q.transpose(0, 2, 1, 3)
+        k = k.transpose(0, 2, 1, 3)
+        v = v.transpose(0, 2, 1, 3)
+
+        q_pos = mx.arange(L) + start_pos
+        if kv_cache is not None:
+            k, v = kv_cache.update(k, v)      # full history
+        kv_pos = mx.arange(k.shape[2])
+
+        rel = rel.reshape(B, L, self.num_heads, self.d_rel)
+        position_bias = self.rel_logits_proj(rel, q_pos, kv_pos)  # [B, heads, Lq, Lkv]
+
+        # log-scaling (global layers only; no-op for context <= n_floor)
+        if not self.is_sliding and self.config.log_scaling_n_floor is not None:
+            n_floor = self.config.log_scaling_n_floor
+            eff_n = (q_pos + 1).astype(mx.float32)
+            tau = 1.0 + self.config.log_scaling_alpha * mx.log(
+                mx.maximum(eff_n / n_floor, 1.0)
+            )
+            tau_q = tau.reshape(1, 1, -1, 1)
+            q = (q.astype(mx.float32) * tau_q).astype(q.dtype)
+            position_bias = (position_bias.astype(mx.float32) * tau_q).astype(position_bias.dtype)
+
+        # GQA expand
+        if self.n_rep > 1:
+            k = mx.repeat(k, self.n_rep, axis=1)
+            v = mx.repeat(v, self.n_rep, axis=1)
+
+        scores = (q.astype(mx.float32) @ mx.swapaxes(k, 2, 3).astype(mx.float32)) * self.scaling
+        scores = scores + position_bias.astype(mx.float32)
+        scores = scores + self._causal_mask(q_pos, kv_pos)
+        weights = mx.softmax(scores, axis=-1)
+        out = weights.astype(v.dtype) @ v                      # [B, heads, Lq, head_dim]
+
+        out = out.transpose(0, 2, 1, 3).reshape(B, L, self.num_heads * self.head_dim)
+        return self.wo_ud(out)
+
+    def _causal_mask(self, q_pos, kv_pos):
+        distance = q_pos[:, None] - kv_pos[None, :]  # [Lq, Lkv]
+        allowed = distance >= 0
+        if self.sliding_window is not None:
+            allowed = allowed & (distance < self.sliding_window)
+        mask = mx.where(allowed, 0.0, NEG_INF)
+        return mask[None, None].astype(mx.float32)

inkling_mlx/audio.py

@@ -0,0 +1,34 @@
+"""Inkling audio tower: discrete dMel-token embedding + norm.
+
+Each audio frame is ``n_mel_bins`` discretized bins (values in ``[0, mel_vocab_size)``);
+each bin is embedded from its own slice of a shared table (offset ``bin * mel_vocab_size``)
+and the per-bin embeddings are summed. Mirrors ``InklingAudioModel`` /
+``InklingAudioModelEmbeddings``. Checkpoint keys: ``audio.encoder.weight`` (the
+``[n_mel_bins*mel_vocab_size, hidden]`` table) and ``audio.final_norm.weight``.
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from .common import RMSNorm
+from .config import AudioConfig
+
+
+class AudioModel(nn.Module):
+    def __init__(self, config: AudioConfig):
+        super().__init__()
+        self.config = config
+        self.encoder = nn.Embedding(
+            config.n_mel_bins * config.mel_vocab_size, config.text_hidden_size
+        )
+        self.final_norm = RMSNorm(config.text_hidden_size, eps=config.rms_norm_eps)
+        # non-persistent: arange(n_mel_bins) * mel_vocab_size
+        self._offsets = mx.arange(config.n_mel_bins) * config.mel_vocab_size
+
+    def __call__(self, audio_input_ids: mx.array) -> mx.array:
+        # audio_input_ids: [..., n_mel_bins] with values in [0, mel_vocab_size)
+        embeds = self.encoder(audio_input_ids + self._offsets)  # [..., n_mel_bins, hidden]
+        embeds = embeds.sum(axis=-2)                            # [..., hidden]
+        return self.final_norm(embeds)

inkling_mlx/cache.py

@@ -0,0 +1,66 @@
+"""Incremental caches for Inkling generation.
+
+Two kinds of per-layer state must persist across decode steps:
+
+* ``KVCache``   — the appended key/value tensors for attention.
+* ``ConvCache`` — the last ``kernel-1`` inputs of each depthwise short-convolution
+  (there are 4 per layer: k, v, post-attn, post-mlp).
+
+A ``LayerCache`` bundles one KVCache + the 4 ConvCaches; ``make_cache`` builds one
+per decoder layer. Absolute key positions are always ``arange(kv_len)`` because the
+cache holds every key from position 0 (KVCache keeps the full history — correct for
+both global and sliding layers, since the sliding-window constraint is enforced by
+the attention mask).
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+
+
+class ConvCache:
+    """Holds the last ``kernel-1`` inputs of a short convolution."""
+
+    __slots__ = ("state",)
+
+    def __init__(self):
+        self.state = None  # [B, kernel-1, C] or None
+
+
+class KVCache:
+    """Appends keys/values along the sequence axis (full history)."""
+
+    __slots__ = ("keys", "values")
+
+    def __init__(self):
+        self.keys = None    # [B, heads, T, d]
+        self.values = None
+
+    @property
+    def offset(self) -> int:
+        return 0 if self.keys is None else self.keys.shape[2]
+
+    def update(self, k: mx.array, v: mx.array):
+        if self.keys is None:
+            self.keys, self.values = k, v
+        else:
+            self.keys = mx.concatenate([self.keys, k], axis=2)
+            self.values = mx.concatenate([self.values, v], axis=2)
+        return self.keys, self.values
+
+
+class LayerCache:
+    __slots__ = ("kv", "k_conv", "v_conv", "attn_conv", "mlp_conv")
+
+    def __init__(self):
+        self.kv = KVCache()
+        self.k_conv = ConvCache()
+        self.v_conv = ConvCache()
+        self.attn_conv = ConvCache()
+        self.mlp_conv = ConvCache()
+
+
+def make_cache(model) -> list[LayerCache]:
+    """One LayerCache per text decoder layer."""
+    n = len(model.model.llm.layers) if hasattr(model, "model") else len(model.layers)
+    return [LayerCache() for _ in range(n)]

inkling_mlx/common.py

@@ -0,0 +1,63 @@
+"""Shared low-level modules for the Inkling MLX port."""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+
+
+class RMSNorm(nn.Module):
+    """Llama-style RMSNorm (compute in fp32, weight is a gain).
+
+    Matches ``LlamaRMSNorm``: ``x_fp32 * rsqrt(mean(x^2) + eps) * weight``.
+    """
+
+    def __init__(self, dims: int, eps: float = 1e-6):
+        super().__init__()
+        self.weight = mx.ones((dims,))
+        self.eps = eps
+
+    def __call__(self, x: mx.array) -> mx.array:
+        return mx.fast.rms_norm(x, self.weight, self.eps)
+
+
+class ShortConvolution(nn.Module):
+    """Depthwise causal 1-D convolution with a residual add, computed in fp32.
+
+    Mirrors ``InklingShortConvolution``: a per-channel (groups == channels) causal
+    conv1d of ``kernel_size`` taps, no bias, no activation, then ``out + input``.
+    The reference keeps this module in fp32 regardless of the model dtype
+    (``_keep_in_fp32_modules_strict``), so we upcast here too.
+
+    Weight layout (MLX ``conv1d``): ``[channels, kernel_size, 1]``.
+    """
+
+    def __init__(self, channels: int, kernel_size: int):
+        super().__init__()
+        self.channels = channels
+        self.kernel_size = kernel_size
+        # [C_out, K, C_in // groups] with groups == channels -> [C, K, 1]
+        self.weight = mx.zeros((channels, kernel_size, 1))
+
+    def __call__(self, x: mx.array, mask: mx.array | None = None, cache=None) -> mx.array:
+        # x: [batch, seq, channels]
+        in_dtype = x.dtype
+        xf = x.astype(mx.float32)
+        residual = xf
+        if mask is not None:
+            xf = xf * mask.astype(mx.float32)
+        k = self.kernel_size
+        B, seq, C = xf.shape
+        w = self.weight.astype(mx.float32)
+        if cache is not None:
+            # left-context = cached last (k-1) inputs (zeros on the first call);
+            # a "valid" conv over [left, xf] yields exactly `seq` causal outputs.
+            left = cache.state if cache.state is not None else mx.zeros((B, k - 1, C), dtype=mx.float32)
+            x_in = mx.concatenate([left, xf], axis=1)
+            out = mx.conv1d(x_in, w, padding=0, groups=self.channels)
+            cache.state = x_in[:, -(k - 1):, :]
+        else:
+            # causal: left-pad by (k-1), keep first `seq` outputs (== zero left-context)
+            out = mx.conv1d(xf, w, padding=k - 1, groups=self.channels)[:, :seq, :]
+        out = out + residual
+        return out.astype(in_dtype)

inkling_mlx/config.py

@@ -0,0 +1,207 @@
+"""Configuration for the Inkling multimodal model (MLX port).
+
+Mirrors ``thinkingmachines/Inkling`` ``config.json`` and the transformers PR #47347
+reference (``InklingConfig`` / ``InklingTextConfig`` / ``InklingVisionConfig`` /
+``InklingAudioConfig``). We parse the *checkpoint* config layout (top-level
+``text_config`` / ``vision_config`` / ``audio_config`` / ``mtp_config``), not the
+flattened transformers layout.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any
+
+
+def _get(d: dict, *names, default=None):
+    for n in names:
+        if n in d and d[n] is not None:
+            return d[n]
+    return default
+
+
+@dataclass
+class TextConfig:
+    hidden_size: int = 6144
+    num_hidden_layers: int = 66
+    vocab_size: int = 201024
+    unpadded_vocab_size: int | None = 200058
+
+    # global (full) attention
+    num_attention_heads: int = 64
+    num_key_value_heads: int = 8
+    head_dim: int = 128
+    # sliding-window attention
+    swa_num_attention_heads: int = 64
+    swa_num_key_value_heads: int = 16
+    swa_head_dim: int = 128
+    sliding_window_size: int = 512
+
+    # relative-position logits
+    d_rel: int = 16
+    rel_extent: int = 1024
+    log_scaling_n_floor: int | None = 128000
+    log_scaling_alpha: float = 0.1
+
+    rms_norm_eps: float = 1e-6
+    use_embed_norm: bool = True
+
+    # short convolution
+    sconv_kernel_size: int = 4
+
+    # dense vs MoE MLP
+    dense_mlp_idx: int = 2
+    dense_intermediate_size: int = 24576  # dense MLP intermediate
+    moe_intermediate_size: int = 3072     # per-expert intermediate
+
+    # MoE routing
+    n_routed_experts: int = 256
+    num_experts_per_tok: int = 6
+    n_shared_experts: int = 2
+    shared_expert_sink: bool = True
+    route_scale: float = 8.0
+    use_gate_bias: bool = True
+    norm_after_topk: bool = True
+    use_global_scale: bool = True
+
+    logits_mup_width_multiplier: float = 24.0
+    hidden_act: str = "silu"
+
+    max_position_embeddings: int = 1048576
+
+    # which layer indices use sliding-window ("local") attention
+    local_layer_ids: list[int] = field(default_factory=list)
+
+    # MTP head (dropped for inference)
+    num_mtp_layers: int | None = None
+
+    @property
+    def layer_types(self) -> list[str]:
+        local = set(self.local_layer_ids)
+        return [
+            "hybrid_sliding" if i in local else "hybrid"
+            for i in range(self.num_hidden_layers)
+        ]
+
+    @property
+    def mlp_layer_types(self) -> list[str]:
+        return [
+            "dense" if i < self.dense_mlp_idx else "sparse"
+            for i in range(self.num_hidden_layers)
+        ]
+
+    @classmethod
+    def from_dict(cls, tc: dict) -> "TextConfig":
+        return cls(
+            hidden_size=_get(tc, "hidden_size", default=6144),
+            num_hidden_layers=_get(tc, "num_hidden_layers", default=66),
+            vocab_size=_get(tc, "vocab_size", default=201024),
+            unpadded_vocab_size=_get(tc, "unpadded_vocab_size"),
+            num_attention_heads=_get(tc, "num_attention_heads", default=64),
+            num_key_value_heads=_get(tc, "num_key_value_heads", default=8),
+            head_dim=_get(tc, "head_dim", default=128),
+            swa_num_attention_heads=_get(tc, "swa_num_attention_heads", default=64),
+            swa_num_key_value_heads=_get(tc, "swa_num_key_value_heads", default=16),
+            swa_head_dim=_get(tc, "swa_head_dim", default=128),
+            sliding_window_size=_get(tc, "sliding_window_size", default=512),
+            d_rel=_get(tc, "d_rel", default=16),
+            rel_extent=_get(tc, "rel_extent", default=1024),
+            log_scaling_n_floor=_get(tc, "log_scaling_n_floor"),
+            log_scaling_alpha=_get(tc, "log_scaling_alpha", default=0.1),
+            rms_norm_eps=_get(tc, "rms_norm_eps", default=1e-6),
+            use_embed_norm=_get(tc, "use_embed_norm", default=True),
+            sconv_kernel_size=_get(tc, "sconv_kernel_size", default=4),
+            dense_mlp_idx=_get(tc, "dense_mlp_idx", default=2),
+            dense_intermediate_size=_get(tc, "dense_intermediate_size", default=24576),
+            # checkpoint labels the *MoE* intermediate as `intermediate_size`
+            moe_intermediate_size=_get(tc, "intermediate_size", default=3072),
+            n_routed_experts=_get(tc, "n_routed_experts", default=256),
+            num_experts_per_tok=_get(tc, "num_experts_per_tok", default=6),
+            n_shared_experts=_get(tc, "n_shared_experts", default=2),
+            shared_expert_sink=_get(tc, "shared_expert_sink", default=True),
+            route_scale=_get(tc, "route_scale", default=8.0),
+            use_gate_bias=_get(tc, "use_gate_bias", default=True),
+            norm_after_topk=_get(tc, "norm_after_topk", default=True),
+            use_global_scale=_get(tc, "use_global_scale", default=True),
+            logits_mup_width_multiplier=_get(tc, "logits_mup_width_multiplier", default=24.0),
+            max_position_embeddings=_get(tc, "model_max_length", "max_position_embeddings", default=1048576),
+            local_layer_ids=list(_get(tc, "local_layer_ids", default=[]) or []),
+        )
+
+
+@dataclass
+class VisionConfig:
+    text_hidden_size: int = 6144
+    patch_size: int = 40
+    temporal_patch_size: int = 2
+    num_channels: int = 3
+    n_layers: int = 4
+    rms_norm_eps: float = 1e-6
+    use_vision_norm: bool = True
+
+    @classmethod
+    def from_dict(cls, vc: dict, text_hidden: int) -> "VisionConfig":
+        return cls(
+            text_hidden_size=text_hidden,
+            patch_size=_get(vc, "patch_size", default=40),
+            temporal_patch_size=_get(vc, "temporal_patch_size", default=2),
+            num_channels=_get(vc, "n_channels", "num_channels", default=3),
+            n_layers=_get(vc, "n_layers", "num_hidden_layers", default=4),
+            rms_norm_eps=_get(vc, "rms_norm_eps", default=1e-6),
+            use_vision_norm=_get(vc, "use_vision_norm", default=True),
+        )
+
+
+@dataclass
+class AudioConfig:
+    text_hidden_size: int = 6144
+    n_mel_bins: int = 80
+    mel_vocab_size: int = 16
+    rms_norm_eps: float = 1e-6
+
+    @classmethod
+    def from_dict(cls, ac: dict, text_hidden: int) -> "AudioConfig":
+        return cls(
+            text_hidden_size=text_hidden,
+            n_mel_bins=_get(ac, "n_mel_bins", default=80),
+            mel_vocab_size=_get(ac, "mel_vocab_size", default=16),
+            rms_norm_eps=_get(ac, "rms_norm_eps", default=1e-6),
+        )
+
+
+@dataclass
+class InklingConfig:
+    text: TextConfig
+    vision: VisionConfig
+    audio: AudioConfig
+    image_token_id: int = 200054
+    audio_token_id: int = 200053
+    image_bos_token_id: int = 200005
+    audio_bos_token_id: int = 200020
+    eos_token_id: int = 200006
+    model_type: str = "inkling_mm_model"
+
+    @classmethod
+    def from_dict(cls, cfg: dict) -> "InklingConfig":
+        tc = dict(cfg.get("text_config", {}))
+        mtp = cfg.get("mtp_config") or {}
+        if mtp.get("num_nextn_predict_layers") is not None:
+            tc.setdefault("num_mtp_layers", mtp.get("num_nextn_predict_layers"))
+        text = TextConfig.from_dict(tc)
+        vision = VisionConfig.from_dict(cfg.get("vision_config", {}) or {}, text.hidden_size)
+        audio = AudioConfig.from_dict(cfg.get("audio_config", {}) or {}, text.hidden_size)
+        return cls(
+            text=text,
+            vision=vision,
+            audio=audio,
+            image_token_id=_get(cfg, "image_token_id", default=200054),
+            audio_token_id=_get(cfg, "audio_token_id", default=200053),
+            image_bos_token_id=_get(cfg, "image_bos_token_id", default=200005),
+            audio_bos_token_id=_get(cfg, "audio_bos_token_id", default=200020),
+            eos_token_id=_get(cfg, "eos_token_id", default=200006),
+            model_type=_get(cfg, "model_type", default="inkling_mm_model"),
+        )
+
+    @property
+    def raw(self) -> dict[str, Any]:
+        return {"model_type": self.model_type}

inkling_mlx/convert.py

@@ -0,0 +1,217 @@
+"""Streaming HF -> MLX conversion + quantization for Inkling.
+
+The model is far too large (~1.9 TB bf16) to instantiate in RAM, so we convert
+tensor-by-tensor: read each source shard (mmap), remap the name, apply the layout
+transform, optionally affine-quantize the weight, and write output shards. Affine
+quantization has no cross-tensor dependency, so per-tensor streaming is exactly
+equivalent to ``nn.quantize(model)``.
+
+Name/layout transforms vs. the checkpoint:
+  * ``*_sconv.weight``  [C,1,K] -> [C,K,1]            (MLX conv1d layout)
+  * ``mlp.w13_dn``      [2I,H]  -> gate_proj/up_proj   (split dense fused gate+up)
+  * ``experts.w13_weight``        [E,2I,H] -> gate_proj/up_proj (split)
+  * ``experts.w2_weight``         [E,H,I]  -> down_proj          (identity)
+  * ``shared_experts.shared_w13`` [2,2I,H] -> gate_proj/up_proj  (split)
+  * ``model.mtp.*``               dropped (inference-irrelevant)
+  * everything else: identity
+"""
+
+from __future__ import annotations
+
+import glob
+import json
+import os
+import shutil
+
+import mlx.core as mx
+
+
+def map_name(name: str):
+    """HF checkpoint tensor name -> list of (out_name, kind) for the MLX model."""
+    if name.startswith("model.mtp."):
+        return []  # drop MTP head
+
+    if name.endswith(("k_sconv.weight", "v_sconv.weight", "attn_sconv.weight", "mlp_sconv.weight")):
+        return [(name, "sconv")]
+
+    # dense MLP fused gate+up / down
+    if name.endswith("mlp.w13_dn.weight"):
+        base = name[: -len("w13_dn.weight")]
+        return [(base + "gate_proj.weight", "w13_gate"), (base + "up_proj.weight", "w13_up")]
+    if name.endswith("mlp.w2_md.weight"):
+        return [(name[: -len("w2_md.weight")] + "down_proj.weight", "identity")]
+
+    # routed experts fused
+    if name.endswith("experts.w13_weight"):
+        base = name[: -len("w13_weight")]
+        return [(base + "gate_proj.weight", "w13_gate"), (base + "up_proj.weight", "w13_up")]
+    if name.endswith("experts.w2_weight"):
+        return [(name[: -len("w2_weight")] + "down_proj.weight", "identity")]
+
+    # shared experts fused
+    if name.endswith("shared_experts.shared_w13_weight"):
+        base = name[: -len("shared_w13_weight")]
+        return [(base + "gate_proj.weight", "w13_gate"), (base + "up_proj.weight", "w13_up")]
+    if name.endswith("shared_experts.shared_w2_weight"):
+        return [(name[: -len("shared_w2_weight")] + "down_proj.weight", "identity")]
+
+    return [(name, "identity")]
+
+
+def transform(w: mx.array, kind: str) -> mx.array:
+    if kind == "identity":
+        return w
+    if kind == "sconv":
+        # [C, 1, K] -> [C, K, 1]
+        return mx.swapaxes(w, 1, 2)
+    if kind in ("w13_gate", "w13_up"):
+        # The checkpoint stores gate/up INTERLEAVED row-wise: [g0, u0, g1, u1, ...]
+        # (SGLang `deinterleave_w13`). De-interleave: gate = rows 0::2, up = rows 1::2.
+        # A contiguous [:half]/[half:] split scrambles gate<->up in every MLP.
+        n = w.shape[-2] // 2
+        g = w.reshape(*w.shape[:-2], n, 2, w.shape[-1])
+        return g[..., 0, :] if kind == "w13_gate" else g[..., 1, :]
+    raise ValueError(kind)
+
+
+# ---- quantization target predicate (must be identical in convert and load) ----
+
+# Quant "recipes" — which module leaves get affine-quantized.
+#   uniform       : everything (attention, MLP/experts, embed/unembed, audio, vision)
+#   experts_only  : ONLY the MLP/expert matmuls (+ audio/vision); attention and
+#                   embed/unembed stay bf16. Inkling attention dominates 4-bit error
+#                   (~58% per layer vs ~15% for experts), so this keeps a 4-bit-sized
+#                   build coherent while the ~927 B experts still fit in 512 GB.
+_RECIPES = {
+    "uniform": {"wq_du", "wk_dv", "wv_dv", "wr_du", "wo_ud",
+                "gate_proj", "up_proj", "down_proj", "embed", "unembed", "encoder"},
+    "experts_only": {"gate_proj", "up_proj", "down_proj", "encoder"},
+}
+
+
+def is_quant_target(out_name: str, quant_axis_size: int, group_size: int, recipe: str = "uniform") -> bool:
+    """Whether ``out_name`` (a converted param path) should be affine-quantized."""
+    if not out_name.endswith(".weight"):
+        return False
+    leaf = out_name[: -len(".weight")].rsplit(".", 1)[-1]
+    leaves = _RECIPES[recipe]
+    # vision projection layers (linear_0 .. linear_3) — quantized in both recipes
+    is_vision_linear = leaf.startswith("linear_") and ".visual." in out_name
+    if leaf not in leaves and not is_vision_linear:
+        return False
+    # router gate stays fp (leaf == "gate", excluded above); norms/sconv excluded by leaf
+    # can only group-quantize when the input dim is a multiple of group_size
+    return quant_axis_size % group_size == 0
+
+
+# ------------------------------ streaming driver ------------------------------
+
+_SHARD_CAP_BYTES = 5_000_000_000  # ~5 GB per output shard
+
+
+def _process_tensor(name, w, bits, group_size, out_dtype, recipe="uniform"):
+    """Yield (out_name, array) pairs for one source tensor."""
+    for out_name, kind in map_name(name):
+        wt = transform(w, kind)
+        quantize = bits is not None and is_quant_target(out_name, wt.shape[-1], group_size, recipe)
+        if quantize:
+            qw, scales, biases = mx.quantize(wt, group_size=group_size, bits=bits)
+            base = out_name[: -len(".weight")]
+            yield out_name, qw
+            yield base + ".scales", scales
+            yield base + ".biases", biases
+        else:
+            # keep norms/router/sconv/rel-proj in fp32-safe dtype; matmul weights in out_dtype
+            keep_hi = wt.dtype == mx.float32 and (".global_scale" in out_name or ".bias" in out_name
+                                                  or out_name.endswith(("_norm.weight", "norm.weight")))
+            yield out_name, wt.astype(mx.float32 if keep_hi else out_dtype)
+
+
+def convert_model(src: str, dst: str, bits=None, group_size: int = 64, out_dtype=mx.bfloat16,
+                  recipe: str = "uniform"):
+    """Stream-convert an Inkling checkpoint from ``src`` to ``dst``.
+
+    ``bits=None`` -> plain dtype cast (bf16). ``bits in {4,6,8}`` -> affine quant.
+    ``recipe`` selects which modules are quantized (see ``_RECIPES``).
+    Processes one source shard at a time; never holds the whole model in RAM.
+    """
+    os.makedirs(dst, exist_ok=True)
+    index = json.load(open(os.path.join(src, "model.safetensors.index.json")))
+    weight_map = index["weight_map"]
+
+    shard_to_names: dict[str, list[str]] = {}
+    for n, s in weight_map.items():
+        shard_to_names.setdefault(s, []).append(n)
+
+    out_index: dict[str, str] = {}
+    buffer: dict[str, mx.array] = {}
+    buffer_bytes = 0
+    out_shard_id = 0
+    total_out_shards_placeholder = "{:05d}"
+
+    def flush(final=False):
+        nonlocal buffer, buffer_bytes, out_shard_id
+        if not buffer:
+            return
+        out_shard_id += 1
+        fname = f"model-{total_out_shards_placeholder.format(out_shard_id)}.safetensors"
+        mx.save_safetensors(os.path.join(dst, fname), buffer, metadata={"format": "mlx"})
+        for k in buffer:
+            out_index[k] = fname
+        buffer = {}
+        buffer_bytes = 0
+
+    for shard in sorted(shard_to_names):
+        path = os.path.join(src, shard)
+        tensors = mx.load(path)  # mmap
+        for name in shard_to_names[shard]:
+            w = tensors[name]
+            for out_name, arr in _process_tensor(name, w, bits, group_size, out_dtype, recipe):
+                mx.eval(arr)
+                buffer[out_name] = arr
+                buffer_bytes += arr.nbytes
+                if buffer_bytes >= _SHARD_CAP_BYTES:
+                    flush()
+        del tensors
+    flush(final=True)
+
+    # rename shards with correct total, build index.json
+    _finalize_index(dst, out_index, out_shard_id)
+    _write_config(src, dst, bits, group_size, recipe)
+    _copy_aux(src, dst)
+    return dst
+
+
+def _finalize_index(dst, out_index, n_shards):
+    # rewrite shard filenames to model-XXXXX-of-YYYYY.safetensors
+    remap = {}
+    for i in range(1, n_shards + 1):
+        old = f"model-{i:05d}.safetensors"
+        new = f"model-{i:05d}-of-{n_shards:05d}.safetensors"
+        if old != new and os.path.exists(os.path.join(dst, old)):
+            os.rename(os.path.join(dst, old), os.path.join(dst, new))
+        remap[old] = new
+    weight_map = {k: remap[v] for k, v in out_index.items()}
+    total = sum(os.path.getsize(os.path.join(dst, f)) for f in set(weight_map.values()))
+    with open(os.path.join(dst, "model.safetensors.index.json"), "w") as f:
+        json.dump({"metadata": {"total_size": total}, "weight_map": weight_map}, f, indent=2)
+
+
+def _write_config(src, dst, bits, group_size, recipe="uniform"):
+    cfg = json.load(open(os.path.join(src, "config.json")))
+    if bits is not None:
+        cfg["quantization"] = {"group_size": group_size, "bits": bits, "recipe": recipe}
+    with open(os.path.join(dst, "config.json"), "w") as f:
+        json.dump(cfg, f, indent=2)
+
+
+def _copy_aux(src, dst):
+    for pat in ("tokenizer*", "special_tokens_map.json", "*.tiktoken", "tiktoken",
+                "chat_template.jinja", "processor_config.json"):
+        for p in glob.glob(os.path.join(src, pat)):
+            base = os.path.basename(p)
+            target = os.path.join(dst, base)
+            if os.path.isdir(p):
+                shutil.copytree(p, target, dirs_exist_ok=True)
+            else:
+                shutil.copy2(p, target)

inkling_mlx/convert_cli.py

@@ -0,0 +1,44 @@
+"""CLI: convert/quantize an Inkling checkpoint to MLX.
+
+Examples:
+  python -m inkling_mlx.convert_cli --src /path/Inkling-src --dst out-bf16
+  python -m inkling_mlx.convert_cli --src /path/Inkling-src --dst out-4bit --bits 4
+"""
+
+from __future__ import annotations
+
+import argparse
+import time
+
+import mlx.core as mx
+
+from .convert import convert_model
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--src", required=True, help="Inkling bf16 source dir (HF layout)")
+    ap.add_argument("--dst", required=True, help="output dir")
+    ap.add_argument("--bits", type=int, default=None, choices=[2, 3, 4, 5, 6, 8],
+                    help="quantization bits; omit for bf16 passthrough")
+    ap.add_argument("--group-size", type=int, default=64)
+    ap.add_argument("--dtype", default="bfloat16", choices=["bfloat16", "float16"])
+    ap.add_argument("--device", default="gpu", choices=["gpu", "cpu"],
+                    help="cpu avoids the Metal GPU-timeout watchdog on huge tensors (slower, robust)")
+    ap.add_argument("--recipe", default="uniform", choices=["uniform", "experts_only"],
+                    help="experts_only keeps attention + embed/unembed at bf16 (coherent 4-bit-sized build)")
+    args = ap.parse_args()
+
+    if args.device == "cpu":
+        mx.set_default_device(mx.cpu)
+        print("[convert] using CPU device (avoids Metal command-buffer timeout)")
+
+    dtype = {"bfloat16": mx.bfloat16, "float16": mx.float16}[args.dtype]
+    t0 = time.time()
+    print(f"[convert] {args.src} -> {args.dst}  bits={args.bits} group_size={args.group_size} dtype={args.dtype} recipe={args.recipe}")
+    convert_model(args.src, args.dst, bits=args.bits, group_size=args.group_size, out_dtype=dtype, recipe=args.recipe)
+    print(f"[convert] done in {time.time()-t0:.0f}s -> {args.dst}")
+
+
+if __name__ == "__main__":
+    main()

inkling_mlx/generate.py

@@ -0,0 +1,74 @@
+"""Greedy generation for an Inkling MLX model, using an incremental KV + conv-state
+cache: the prompt is prefilled once, then each new token is a single-position step.
+"""
+
+from __future__ import annotations
+
+import argparse
+import time
+
+import mlx.core as mx
+
+from .cache import make_cache
+from .load import load
+
+
+def load_tokenizer(path: str):
+    try:
+        from transformers import AutoTokenizer
+        return AutoTokenizer.from_pretrained(path, trust_remote_code=True)
+    except Exception:
+        from transformers import PreTrainedTokenizerFast
+        import os
+        return PreTrainedTokenizerFast(tokenizer_file=os.path.join(path, "tokenizer.json"))
+
+
+def greedy_generate(model, config, input_ids, max_new_tokens=32, eos_id=None):
+    eos_id = eos_id if eos_id is not None else config.eos_token_id
+    caches = make_cache(model)
+    prompt = list(input_ids)
+
+    # prefill the whole prompt in one pass; only need the last position's logits
+    logits = model(mx.array([prompt]), caches=caches, start_pos=0, last_logit_only=True)
+    next_id = int(mx.argmax(logits[0, -1]).item())
+    out = [next_id]
+    pos = len(prompt)
+
+    for _ in range(max_new_tokens - 1):
+        if next_id == eos_id:
+            break
+        logits = model(mx.array([[next_id]]), caches=caches, start_pos=pos, last_logit_only=True)
+        next_id = int(mx.argmax(logits[0, -1]).item())
+        out.append(next_id)
+        pos += 1
+
+    return prompt + out
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--model", required=True, help="converted MLX model dir")
+    ap.add_argument("--prompt", default="The capital of France is")
+    ap.add_argument("--max-new-tokens", type=int, default=32)
+    args = ap.parse_args()
+
+    print(f"[load] {args.model}")
+    t0 = time.time()
+    model, config = load(args.model)
+    print(f"[load] ready in {time.time()-t0:.0f}s")
+
+    tok = load_tokenizer(args.model)
+    input_ids = tok(args.prompt)["input_ids"]
+    print(f"[prompt] {args.prompt!r} -> {len(input_ids)} tokens")
+
+    t0 = time.time()
+    out_ids = greedy_generate(model, config, input_ids, args.max_new_tokens)
+    dt = time.time() - t0
+    text = tok.decode(out_ids)
+    n_new = len(out_ids) - len(input_ids)
+    print(f"\n{text}\n")
+    print(f"[gen] {n_new} tokens in {dt:.1f}s ({n_new/dt:.2f} tok/s)")
+
+
+if __name__ == "__main__":
+    main()

inkling_mlx/layers.py

@@ -0,0 +1,47 @@
+"""Inkling decoder layer: attention + MLP, each wrapped by a pre-norm and a
+trailing short-convolution, with residual adds. Mirrors ``InklingDecoderLayer``.
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from .attention import Attention
+from .common import RMSNorm, ShortConvolution
+from .config import TextConfig
+from .moe import DenseMLP, MoE
+
+
+class DecoderLayer(nn.Module):
+    def __init__(self, config: TextConfig, layer_idx: int):
+        super().__init__()
+        self.attn = Attention(config, layer_idx)
+        self.attn_norm = RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.mlp_norm = RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        if config.mlp_layer_types[layer_idx] == "sparse":
+            self.mlp = MoE(config)
+        else:
+            self.mlp = DenseMLP(config)
+        self.attn_sconv = ShortConvolution(config.hidden_size, config.sconv_kernel_size)
+        self.mlp_sconv = ShortConvolution(config.hidden_size, config.sconv_kernel_size)
+
+    def __call__(self, x, start_pos=0, cache=None, conv_mask=None):
+        kv = cache.kv if cache is not None else None
+        residual = x
+        h = self.attn_norm(x)
+        h = self.attn(
+            h, start_pos=start_pos, kv_cache=kv,
+            k_conv=cache.k_conv if cache is not None else None,
+            v_conv=cache.v_conv if cache is not None else None,
+            conv_mask=conv_mask,
+        )
+        h = self.attn_sconv(h, mask=conv_mask, cache=cache.attn_conv if cache is not None else None)
+        x = residual + h
+
+        residual = x
+        h = self.mlp_norm(x)
+        h = self.mlp(h)
+        h = self.mlp_sconv(h, mask=conv_mask, cache=cache.mlp_conv if cache is not None else None)
+        x = residual + h
+        return x

inkling_mlx/load.py

@@ -0,0 +1,69 @@
+"""Load a (possibly quantized) Inkling MLX model produced by ``convert_model``."""
+
+from __future__ import annotations
+
+import glob
+import json
+import os
+
+import mlx.core as mx
+import mlx.nn as nn
+from mlx.utils import tree_flatten
+
+from .config import InklingConfig
+from .model import InklingForConditionalGeneration
+
+
+def quant_predicate(group_size: int, recipe: str = "uniform"):
+    """Quantize exactly the modules the converter did, by delegating to
+    ``convert.is_quant_target`` with the same ``recipe``. Guarantees the loaded
+    module set matches the checkpoint (e.g. under ``experts_only``, attention and
+    embed/unembed stay bf16 and must NOT be re-quantized here)."""
+    from .convert import is_quant_target
+
+    def pred(path, module):
+        if not hasattr(module, "to_quantized"):
+            return False
+        w = getattr(module, "weight", None)
+        if w is None:
+            return False
+        return is_quant_target(path + ".weight", w.shape[-1], group_size, recipe)
+
+    return pred
+
+
+def load(path: str, lazy: bool = False):
+    cfg_dict = json.load(open(os.path.join(path, "config.json")))
+    config = InklingConfig.from_dict(cfg_dict)
+    model = InklingForConditionalGeneration(config)
+
+    q = cfg_dict.get("quantization")
+    if q:
+        nn.quantize(model, group_size=q["group_size"], bits=q["bits"],
+                    class_predicate=quant_predicate(q["group_size"], q.get("recipe", "uniform")))
+
+    # Stream shards: assign each, then release its handle. We do NOT eagerly
+    # mx.eval() the whole parameter tree — for a ~500 GB model that builds one
+    # enormous eval graph and trips a Metal resource limit. Weights stay lazy
+    # (mmap-backed) and materialize on demand during the forward pass, exactly
+    # like mlx-lm loads large models.
+    loaded = set()
+    shards = sorted(glob.glob(os.path.join(path, "*.safetensors")))
+    for shard in shards:
+        w = mx.load(shard)
+        model.load_weights(list(w.items()), strict=False)
+        if not lazy:
+            # materialize THIS shard's tensors now (bounded graph) and keep them
+            # resident. Avoids one enormous eval over all ~500 GB of params, which
+            # trips a Metal resource limit; also prevents per-token disk paging.
+            mx.eval(list(w.values()))
+        loaded.update(w.keys())
+        del w
+
+    expected = {k for k, _ in tree_flatten(model.parameters())}
+    missing = expected - loaded
+    if missing:
+        raise ValueError(f"{len(missing)} params not found in checkpoint, e.g. {sorted(missing)[:3]}")
+
+    model.eval()
+    return model, config

inkling_mlx/model.py

@@ -0,0 +1,79 @@
+"""Top-level Inkling multimodal model.
+
+Checkpoint layout: ``model.llm.*`` (text backbone + untied unembed), ``model.visual.*``
+(HMLP vision tower), ``model.audio.*`` (dMel audio tower). Image/audio features are
+scattered into the token-embedding stream at their placeholder-token positions, then
+the text backbone runs and the untied unembed head produces (muP-scaled) logits.
+The MTP head (``model.mtp.*``) is intentionally not loaded (inference-irrelevant).
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+import numpy as np
+
+from .audio import AudioModel
+from .config import InklingConfig
+from .text import TextModel
+from .vision import VisionModel
+
+
+def _scatter_features(embeds, input_ids, token_id, features):
+    """Replace ``embeds`` rows where ``input_ids == token_id`` with ``features``
+    (in sequence order). ``input_ids`` is host-known so we resolve positions on CPU."""
+    B, L, H = embeds.shape
+    ids = np.array(input_ids).reshape(-1)
+    pos = np.nonzero(ids == token_id)[0]
+    if pos.size == 0:
+        return embeds
+    flat = embeds.reshape(B * L, H)
+    flat[mx.array(pos)] = features.astype(flat.dtype)
+    return flat.reshape(B, L, H)
+
+
+class InnerModel(nn.Module):
+    """The ``model.`` level holding the three towers."""
+
+    def __init__(self, config: InklingConfig):
+        super().__init__()
+        self.llm = TextModel(config.text)
+        self.visual = VisionModel(config.vision)
+        self.audio = AudioModel(config.audio)
+
+
+class InklingForConditionalGeneration(nn.Module):
+    def __init__(self, config: InklingConfig):
+        super().__init__()
+        self.config = config
+        self.model = InnerModel(config)
+
+    # --- convenience accessors ---
+    @property
+    def llm(self) -> TextModel:
+        return self.model.llm
+
+    def __call__(
+        self,
+        input_ids: mx.array,
+        pixel_values: mx.array | None = None,
+        audio_input_ids: mx.array | None = None,
+        conv_mask=None,
+        caches=None,
+        start_pos: int = 0,
+        last_logit_only: bool = False,
+    ) -> mx.array:
+        embeds = self.model.llm.embed_tokens(input_ids)
+
+        if pixel_values is not None:
+            img = self.model.visual(pixel_values)
+            embeds = _scatter_features(embeds, input_ids, self.config.image_token_id, img)
+
+        if audio_input_ids is not None:
+            aud = self.model.audio(audio_input_ids)
+            embeds = _scatter_features(embeds, input_ids, self.config.audio_token_id, aud)
+
+        hidden = self.model.llm.backbone(embeds, conv_mask=conv_mask, caches=caches, start_pos=start_pos)
+        if last_logit_only:
+            hidden = hidden[:, -1:, :]
+        return self.model.llm.logits(hidden)

inkling_mlx/moe.py

@@ -0,0 +1,117 @@
+"""Inkling MLP variants: dense SwiGLU (with a learned output scale) and the
+sparse MoE (sigmoid router with correction bias, softmax-over-selected weights,
+route/global scaling, and 2 always-on shared experts forming a routing "sink").
+
+Mirrors ``InklingMLP`` / ``InklingTopkRouter`` / ``InklingExperts`` /
+``InklingSharedExperts`` / ``InklingMoE`` from transformers PR #47347.
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from mlx_lm.models.switch_layers import SwitchGLU
+
+from .config import TextConfig
+
+
+class DenseMLP(nn.Module):
+    """SwiGLU MLP with a learned scalar output gain (``global_scale``).
+
+    The checkpoint fuses gate+up into ``w13_dn``; the converter splits it into
+    ``gate_proj``/``up_proj`` so the standard MLX quantizer sees plain ``nn.Linear``s.
+    """
+
+    def __init__(self, config: TextConfig):
+        super().__init__()
+        h = config.hidden_size
+        inter = config.dense_intermediate_size
+        self.gate_proj = nn.Linear(h, inter, bias=False)
+        self.up_proj = nn.Linear(h, inter, bias=False)
+        self.down_proj = nn.Linear(inter, h, bias=False)
+        self.global_scale = mx.ones((1,))
+
+    def __call__(self, x: mx.array) -> mx.array:
+        y = self.down_proj(nn.silu(self.gate_proj(x)) * self.up_proj(x))
+        return y * self.global_scale
+
+
+class Router(nn.Module):
+    """Sigmoid top-k router with a correction bias and a shared-expert sink.
+
+    Kept in full precision (tiny). Returns per-token routed weights/indices plus
+    the two shared-expert gammas produced by the same softmax (the "sink").
+    """
+
+    def __init__(self, config: TextConfig):
+        super().__init__()
+        self.num_experts = config.n_routed_experts
+        self.n_shared = config.n_shared_experts
+        self.n_total = self.num_experts + self.n_shared
+        self.top_k = config.num_experts_per_tok
+        self.route_scale = config.route_scale
+        self.hidden = config.hidden_size
+        self.weight = mx.zeros((self.n_total, config.hidden_size))
+        self.bias = mx.zeros((self.num_experts,))              # e_score_correction_bias
+        self.global_scale = mx.ones((1,))
+
+    def __call__(self, x: mx.array):
+        # Routing (esp. the top-k selection) is precision-sensitive: in bf16 the
+        # rounding of near-tied expert scores flips which experts fire, and a wrong
+        # choice compounds over 64 MoE layers into incoherent output. Compute the
+        # whole router in fp32.
+        flat = x.reshape(-1, self.hidden).astype(mx.float32)
+        router_logits = flat @ self.weight.T.astype(mx.float32)  # [T, n_total]
+        scores = mx.sigmoid(router_logits)
+        routed_scores = scores[:, : self.num_experts]
+        scores_for_choice = routed_scores + self.bias
+
+        # top-k experts (order within the top-k is irrelevant downstream)
+        topk_idx = mx.argpartition(-scores_for_choice, kth=self.top_k - 1, axis=-1)[:, : self.top_k]
+
+        routed_logits = router_logits[:, : self.num_experts]
+        shared_logits = router_logits[:, self.num_experts :]  # [T, n_shared]
+        gathered = mx.take_along_axis(routed_logits, topk_idx, axis=-1)  # [T, top_k]
+        topk_logits = mx.concatenate([gathered, shared_logits], axis=-1)  # [T, top_k+n_shared]
+
+        # softmax over the selected (+shared) logits, computed in the log domain
+        log_probs = -mx.logaddexp(mx.zeros_like(topk_logits), -topk_logits)  # logsigmoid
+        weights = mx.softmax(log_probs, axis=-1)
+        weights = weights * self.route_scale * self.global_scale
+
+        shared_gammas = weights[:, self.top_k :].astype(x.dtype)   # [T, n_shared]
+        topk_weights = weights[:, : self.top_k].astype(x.dtype)    # [T, top_k]
+        return topk_weights, topk_idx, shared_gammas
+
+
+class MoE(nn.Module):
+    def __init__(self, config: TextConfig):
+        super().__init__()
+        self.config = config
+        self.n_shared = config.n_shared_experts
+        self.gate = Router(config)
+        self.experts = SwitchGLU(
+            config.hidden_size, config.moe_intermediate_size, config.n_routed_experts, bias=False
+        )
+        self.shared_experts = SwitchGLU(
+            config.hidden_size, config.moe_intermediate_size, config.n_shared_experts, bias=False
+        )
+
+    def __call__(self, x: mx.array) -> mx.array:
+        B, L, H = x.shape
+        topk_weights, topk_idx, shared_gammas = self.gate(x)
+        xf = x.reshape(-1, H)                                  # [T, H]
+        T = xf.shape[0]
+
+        routed = self.experts(xf, topk_idx)                   # [T, top_k, H]
+        routed = (routed * topk_weights[..., None]).sum(axis=1)
+
+        shared_idx = mx.broadcast_to(
+            mx.arange(self.n_shared)[None], (T, self.n_shared)
+        )
+        shared = self.shared_experts(xf, shared_idx)          # [T, n_shared, H]
+        shared = (shared.astype(mx.float32) * shared_gammas[..., None].astype(mx.float32)).sum(axis=1)
+        shared = shared.astype(routed.dtype)
+
+        return (routed + shared).reshape(B, L, H)

inkling_mlx/text.py

@@ -0,0 +1,50 @@
+"""Inkling text backbone (``model.llm.*``): token embedding + embed-norm,
+66 decoder layers, final norm, and the (untied) unembed head.
+
+Mirrors ``InklingTextModel`` + the unembed / muP-logit scaling from
+``InklingForConditionalGeneration``.
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from .common import RMSNorm
+from .config import TextConfig
+from .layers import DecoderLayer
+
+
+class TextModel(nn.Module):
+    def __init__(self, config: TextConfig):
+        super().__init__()
+        self.config = config
+        self.embed = nn.Embedding(config.vocab_size, config.hidden_size)
+        self.embed_norm = RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.layers = [DecoderLayer(config, i) for i in range(config.num_hidden_layers)]
+        self.norm = RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.unembed = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
+
+    def embed_tokens(self, input_ids: mx.array) -> mx.array:
+        return self.embed_norm(self.embed(input_ids))
+
+    def backbone(self, inputs_embeds: mx.array, conv_mask=None, caches=None, start_pos=0) -> mx.array:
+        h = inputs_embeds
+        for i, layer in enumerate(self.layers):
+            h = layer(h, start_pos=start_pos,
+                      cache=caches[i] if caches is not None else None,
+                      conv_mask=conv_mask)
+        return self.norm(h)
+
+    def logits(self, hidden: mx.array) -> mx.array:
+        hidden = hidden / self.config.logits_mup_width_multiplier
+        logits = self.unembed(hidden)
+        uv = self.config.unpadded_vocab_size
+        if uv is not None and uv < logits.shape[-1]:
+            logits = logits[..., :uv]
+        return logits
+
+    def __call__(self, input_ids: mx.array, conv_mask=None) -> mx.array:
+        h = self.embed_tokens(input_ids)
+        h = self.backbone(h, conv_mask=conv_mask)
+        return self.logits(h)

inkling_mlx/vision.py

@@ -0,0 +1,126 @@
+"""Inkling vision tower: an HMLP (hierarchical MLP) patch encoder.
+
+No attention — each layer folds space/time into the channel dim then projects
+(Linear -> RMSNorm -> GELU), progressively growing channels up to the text hidden
+size. Mirrors ``InklingVisionModel`` / ``InklingVisionEncoderLayer`` /
+``plan_out_scales``. Checkpoint keys are flat: ``visual.layers.linear_{i}`` and
+``visual.layers.norm_{i}`` plus ``visual.final_norm``.
+"""
+
+from __future__ import annotations
+
+import math
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from .common import RMSNorm
+from .config import VisionConfig
+
+
+def _prime_factors(n: int) -> list[int]:
+    factors = []
+    while n % 2 == 0:
+        factors.append(2)
+        n //= 2
+    p = 3
+    while p * p <= n:
+        while n % p == 0:
+            factors.append(p)
+            n //= p
+        p += 2
+    if n > 1:
+        factors.append(n)
+    return factors
+
+
+def plan_out_scales(temporal_patch_size: int, patch_size: int, n_layers: int, n_channels: int):
+    """Port of the reference ``plan_out_scales`` (returns an ``(n_layers+1, 4)``
+    array of (t, h, w, c) grid sizes). Uses numpy + scipy for the assignment."""
+    import numpy as np
+    from scipy.optimize import linear_sum_assignment
+
+    h = np.cumprod(np.array(_prime_factors(patch_size)[::-1]))
+    t = np.cumprod(np.array(_prime_factors(temporal_patch_size)[::-1]))
+
+    h_ch = np.ceil(h**2 * n_channels / 64).astype(np.int64) * 64
+    t_ch = (np.ceil(h[-1] ** 2 * n_channels * t)).astype(np.int64) * 64
+
+    base = np.array([[1, 1, 1, n_channels]], dtype=np.int64)
+    spatial = np.stack([np.ones_like(h), h, h, h_ch], axis=1)
+    temporal = np.stack([t, np.full_like(t, h[-1]), np.full_like(t, h[-1]), t_ch], axis=1)
+    scales = np.concatenate([base, spatial, temporal], axis=0).astype(np.int64)
+
+    size_reduction = np.prod(scales[:, :-1], axis=1).astype(np.float64)
+    total_elements = patch_size * patch_size * temporal_patch_size * n_channels
+    log_ideal = np.linspace(0.0, math.log(total_elements), n_layers + 1)
+    cost = np.abs(log_ideal[:, None] - np.log(size_reduction)[None, :])
+
+    if n_layers >= scales.shape[0]:
+        idxs = np.argmin(cost, axis=1)
+    else:
+        _, idxs = linear_sum_assignment(cost)
+    idxs = np.array(idxs)
+    idxs[0] = 0
+    idxs[-1] = scales.shape[0] - 1
+    return scales[idxs]
+
+
+def _fold_timespace_to_depth(x, t_fold, hw_fold):
+    # x: [B, T, H, W, C] -> [B, T//t, H//hw, W//hw, C*t*hw*hw]
+    B, T, H, W, C = x.shape
+    t_new, h_new, w_new = T // t_fold, H // hw_fold, W // hw_fold
+    x = x.reshape(B, t_new, t_fold, h_new, hw_fold, w_new, hw_fold, C)
+    x = x.transpose(0, 1, 3, 5, 2, 4, 6, 7)
+    x = x.reshape(B, t_new, h_new, w_new, t_fold * hw_fold * hw_fold * C)
+    return x
+
+
+class _VisionLayers(nn.Module):
+    """Holds ``linear_{i}`` / ``norm_{i}`` to match checkpoint keys."""
+
+    def __init__(self, config: VisionConfig):
+        super().__init__()
+        scales = plan_out_scales(
+            config.temporal_patch_size, config.patch_size, config.n_layers, config.num_channels
+        )
+        self.n_layers = config.n_layers
+        self.folds = []  # (t_fold, hw_fold, add_norm)
+        for i in range(config.n_layers):
+            start, end = scales[i], scales[i + 1]
+            shuffle = (
+                (end[0] // start[0]) * (end[1] // start[1]) * (end[2] // start[2])
+            )
+            hw_fold = int(end[1] // start[1])
+            t_fold = int(end[0] // start[0])
+            in_dim = int(start[3]) * int(shuffle)
+            add_norm = i != config.n_layers - 1
+            out_dim = config.text_hidden_size if i == config.n_layers - 1 else int(end[3])
+            setattr(self, f"linear_{i}", nn.Linear(in_dim, out_dim, bias=False))
+            if add_norm:
+                setattr(self, f"norm_{i}", RMSNorm(out_dim, eps=config.rms_norm_eps))
+            self.folds.append((t_fold, hw_fold, add_norm))
+
+    def __call__(self, x):
+        for i, (t_fold, hw_fold, add_norm) in enumerate(self.folds):
+            if hw_fold > 1 or t_fold > 1:
+                x = _fold_timespace_to_depth(x, t_fold, hw_fold)
+            x = getattr(self, f"linear_{i}")(x)
+            if add_norm:
+                x = getattr(self, f"norm_{i}")(x)
+                x = nn.gelu(x)
+        return x
+
+
+class VisionModel(nn.Module):
+    def __init__(self, config: VisionConfig):
+        super().__init__()
+        self.config = config
+        self.layers = _VisionLayers(config)
+        self.final_norm = RMSNorm(config.text_hidden_size, eps=config.rms_norm_eps)
+
+    def __call__(self, pixel_values: mx.array) -> mx.array:
+        num_patches = pixel_values.shape[0]
+        h = self.layers(pixel_values)
+        h = self.final_norm(h)
+        return h.reshape(num_patches, -1)

model-00013-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:347bd0e9a78592de1d8b284c8ed730e7df889c6774dfefb3279b999ba1593b17
+size 5435818784

model-00015-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3bd2902330c2cb765b0b099b03b422bf9772ef8ada1a88dd9e509a0bbcbb4005
+size 5435847671

model-00016-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f44c9b25d12e42fe706bea4b509a47260a2746f1f0a110aa8b8f09085215536
+size 5570459254

model-00019-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18d1963fa26a1370c94da1d9f8a99f69e2666b03849c73bb96547ae029f80387
+size 5552268696

model-00021-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea283c381a817ac92bbd3cfac3a2c0bac073d2722822ecf1b002dc9ddfd704b0
+size 5439436602

model-00022-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55fd2826eec30ba0504695fa5c3bd558ee497596d1ced2c38cdb5ff3c7ecb26b
+size 5481892886

model-00024-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdd9110a691f77120d80661a45f56b010f97cd873fe1a782df50853746065225
+size 5555889618

model-00027-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff814a774c46f077ae1d52a6a479d2ae7141e86c3e4e09ef7209dd24b7bedfe6
+size 5435835282

model-00035-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74f166e9ef09da8cd1a1de091c48cd324f38b01d2828b485cb0424bf3c197542
+size 5478500966

model-00036-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7413fe9ef56f0d6056d428ac56b7cd5259b3d0ffce3507d70b883303906ea95f
+size 5435828491

model-00051-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa53f3839b888da3993e042447a1ed8ae955696ade3547618b74d007e0c48b49
+size 5510226779

model-00052-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10d628557229be2340453f26644e33f223a342714f78399e1764c8bc75b283fb
+size 5669450472

model-00054-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a39fb3ed64642e61729ac8a3486349aa05fa04661fba2c97c96cdf259e267ab5
+size 5474428577

model-00057-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d3ad88cc0c3c7097ba4d9c9c0d28f7c091b0f901cfe2eb30a4d48ddb8dfa343
+size 5470856882

model-00058-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb6cc23978a5a517d93a5e7eb516d09ba0b316c1fa067b95fea3effc4cdf3ac1
+size 5464130796

model-00060-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb6b9e197466046f11fd92fa22a0bc10c0c2bc13710595c3331f8d7f42547de4
+size 5481474960

model-00063-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:529ace358426464006df9a322f0a5ff08a67d10d439ff31ef5ec7c534dee0d01
+size 5574017292

model-00065-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ba5447632352096bd4479ff52b9549942d758c084392a5b53291157d11b27b4
+size 5474815047

model-00069-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3edb6f874628d7d042b55a9936aaae7f40c2ea5684de45d2f352bbb011ef759a
+size 5602111647

model-00074-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d867d3a5e88dd3acf2d382d90b1fbf9d87bb876b9ef0fa1266e3c9b9b1c21182
+size 5492505269

model-00077-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5aa5ee38350ae4ee8d1fbc8140489973fad2c483cd80a31ef70887bd4bc3f90a
+size 5474462387

model-00082-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3009eb89302944b671fdc045b8613e8786ea90f0753ed93681108a70c42c8fed
+size 5573474243

model-00084-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fc752a1868f0655b468806329fe95aabb2e1869fa95ae9d34ff15ca92df6b3d
+size 5591306886

model-00090-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:508879fcdd31f023915301928b2b47c59c20f9c43dc54b83d35b0a11adb0f066
+size 5435831912

model-00093-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce1888d0339f7409619d799e335112c3552a20f16edb93d7533286ab5cac8047
+size 5523236292

model-00095-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2f2b7b4b21d09e46d6e975e8eb7dd25d8cb434a98c47980ba6e9ae7dda15860
+size 5439374554

model-00096-of-00097.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c40dbf5ce79741cdf82325013ac23295267a8e53384bc4ff25a74501d78e8aa
+size 5492442567

processor_config.json

@@ -0,0 +1,46 @@
+{
+  "audio_token": "<|unused_200053|>",
+  "audio_bos_token": "<|content_audio_input|>",
+  "dmel_max_value": 2.0,
+  "dmel_min_value": -7.0,
+  "feature_extractor": {
+    "audio_token_duration_s": 0.05,
+    "feature_extractor_type": "InklingFeatureExtractor",
+    "feature_size": 80,
+    "hop_length": 800,
+    "n_fft": 1600,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "return_attention_mask": true,
+    "sampling_rate": 16000,
+    "window_size": 1600,
+    "window_size_multiplier": 2.0
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_processor_type": "InklingImageProcessor",
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "height": 40,
+      "width": 40
+    }
+  },
+  "image_token": "<|unused_200054|>",
+  "image_bos_token": "<|content_image|>",
+  "num_dmel_bins": 16,
+  "processor_class": "InklingProcessor"
+}

special_tokens_map.json

@@ -0,0 +1,22 @@
+{
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "<|message_user|>",
+    "<|message_model|>",
+    "<|message_system|>",
+    "<|message_tool|>",
+    "<|content_text|>",
+    "<|content_image|>",
+    "<|content_model_end_sampling|>",
+    "<|content_thinking|>",
+    "<|end_message|>",
+    "<|content_audio_input|>",
+    "<|content_tool_error|>",
+    "<|audio|>",
+    "<|content_xml|>",
+    "<|begin_of_text|>",
+    "<|audio_end|>",
+    "<|content_invoke_tool_json|>",
+    "<|content_invoke_tool_text|>"
+  ]
+}

tiktoken/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc253fd2b702f7a6da7105eaa8f3463b2f1247e83614f23e5323b921088bed2a
+size 3615874

tokenizer_config.json

@@ -0,0 +1,508 @@
+{
+  "added_tokens_decoder": {
+    "199998": {
+      "content": "<|unused|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "199999": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200000": {
+      "content": "<|message_user|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200001": {
+      "content": "<|message_model|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200002": {
+      "content": "<|message_system|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200003": {
+      "content": "<|message_tool|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200004": {
+      "content": "<|content_text|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200005": {
+      "content": "<|content_image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200006": {
+      "content": "<|content_model_end_sampling|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200007": {
+      "content": "<|unused_200007|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200008": {
+      "content": "<|content_thinking|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200009": {
+      "content": "<|unused_200009|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200010": {
+      "content": "<|end_message|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200011": {
+      "content": "<|unused_200011|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200012": {
+      "content": "<|unused_200012|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200013": {
+      "content": "<|unused_200013|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200014": {
+      "content": "<|unused_200014|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200015": {
+      "content": "<|unused_200015|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200016": {
+      "content": "<|unused_200016|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200017": {
+      "content": "<|unused_200017|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200018": {
+      "content": "<|unused_200018|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200019": {
+      "content": "<|unused_200019|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200020": {
+      "content": "<|content_audio_input|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200021": {
+      "content": "<|unused_200021|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200022": {
+      "content": "<|content_tool_error|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200023": {
+      "content": "<|audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200024": {
+      "content": "<|content_xml|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200025": {
+      "content": "<|unused_200025|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200026": {
+      "content": "<|unused_200026|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200027": {
+      "content": "<|unused_200027|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200028": {
+      "content": "<|begin_of_text|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200029": {
+      "content": "<|unused_200029|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200030": {
+      "content": "<|unused_200030|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200031": {
+      "content": "<|unused_200031|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200032": {
+      "content": "<|unused_200032|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200033": {
+      "content": "<|unused_200033|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200034": {
+      "content": "<|unused_200034|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200035": {
+      "content": "<|unused_200035|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200036": {
+      "content": "<|unused_200036|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200037": {
+      "content": "<|unused_200037|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200038": {
+      "content": "<|unused_200038|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200039": {
+      "content": "<|unused_200039|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200040": {
+      "content": "<|unused_200040|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200041": {
+      "content": "<|unused_200041|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200042": {
+      "content": "<|unused_200042|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200043": {
+      "content": "<|audio_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200044": {
+      "content": "<|unused_200044|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200045": {
+      "content": "<|unused_200045|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200046": {
+      "content": "<|unused_200046|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200047": {
+      "content": "<|unused_200047|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200048": {
+      "content": "<|unused_200048|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200049": {
+      "content": "<|content_invoke_tool_json|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200050": {
+      "content": "<|unused_200050|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200051": {
+      "content": "<|unused_200051|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200052": {
+      "content": "<|unused_200052|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200053": {
+      "content": "<|unused_200053|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200054": {
+      "content": "<|unused_200054|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200055": {
+      "content": "<|unused_200055|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200056": {
+      "content": "<|unused_200056|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200057": {
+      "content": "<|content_invoke_tool_text|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "fix_mistral_regex": false,
+  "extra_special_tokens": {
+    "endoftext": "<|endoftext|>",
+    "message_user": "<|message_user|>",
+    "message_model": "<|message_model|>",
+    "message_system": "<|message_system|>",
+    "message_tool": "<|message_tool|>",
+    "content_text": "<|content_text|>",
+    "content_image": "<|content_image|>",
+    "content_model_end_sampling": "<|content_model_end_sampling|>",
+    "content_thinking": "<|content_thinking|>",
+    "end_message": "<|end_message|>",
+    "content_audio_input": "<|content_audio_input|>",
+    "content_tool_error": "<|content_tool_error|>",
+    "audio": "<|audio|>",
+    "content_xml": "<|content_xml|>",
+    "begin_of_text": "<|begin_of_text|>",
+    "audio_end": "<|audio_end|>",
+    "content_invoke_tool_json": "<|content_invoke_tool_json|>",
+    "content_invoke_tool_text": "<|content_invoke_tool_text|>"
+  },
+  "model_max_length": 1000000000000000019884624838656,
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}