PR-1

by Xiao-AMD - opened 9 days ago

base: refs/heads/main

←

from: refs/pr/1

Discussion Files changed

-6226

This view is limited to 50 files because it contains too many changes. See the raw diff here.

Files changed (50) hide show

.gitattributes +0 -2
LICENSE +0 -27
README.md +1 -134
THIRD_PARTY_NOTICES.md +0 -43
chat_template.jinja +0 -61
config.json +0 -433
configuration_deepseek.py +0 -214
configuration_kimi_k25.py +0 -123
generation_config.json +0 -6
kimi_k25_processor.py +0 -165
kimi_k25_vision_processing.py +0 -251
media_utils.py +0 -368
model-00001-of-000064.safetensors +0 -3
model-00002-of-000064.safetensors +0 -3
model-00003-of-000064.safetensors +0 -3
model-00004-of-000064.safetensors +0 -3
model-00005-of-000064.safetensors +0 -3
model-00006-of-000064.safetensors +0 -3
model-00007-of-000064.safetensors +0 -3
model-00008-of-000064.safetensors +0 -3
model-00009-of-000064.safetensors +0 -3
model-00010-of-000064.safetensors +0 -3
model-00011-of-000064.safetensors +0 -3
model-00012-of-000064.safetensors +0 -3
model-00013-of-000064.safetensors +0 -3
model-00014-of-000064.safetensors +0 -3
model-00015-of-000064.safetensors +0 -3
model-00016-of-000064.safetensors +0 -3
model-00017-of-000064.safetensors +0 -3
model-00018-of-000064.safetensors +0 -3
model-00019-of-000064.safetensors +0 -3
model-00020-of-000064.safetensors +0 -3
model-00021-of-000064.safetensors +0 -3
model-00022-of-000064.safetensors +0 -3
model-00023-of-000064.safetensors +0 -3
model-00024-of-000064.safetensors +0 -3
model-00025-of-000064.safetensors +0 -3
model-00026-of-000064.safetensors +0 -3
model-00027-of-000064.safetensors +0 -3
model-00028-of-000064.safetensors +0 -3
model-00029-of-000064.safetensors +0 -3
model-00030-of-000064.safetensors +0 -3
model-00031-of-000064.safetensors +0 -3
model-00032-of-000064.safetensors +0 -3
model-00033-of-000064.safetensors +0 -3
model-00034-of-000064.safetensors +0 -3
model-00035-of-000064.safetensors +0 -3
model-00036-of-000064.safetensors +0 -3
model-00037-of-000064.safetensors +0 -3
model-00038-of-000064.safetensors +0 -3

.gitattributes CHANGED Viewed

@@ -33,5 +33,3 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
-model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
-figures/demo_video.mp4 filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

LICENSE DELETED Viewed

@@ -1,27 +0,0 @@
-Modified MIT License
-Copyright (c) 2026 Moonshot AI
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the “Software”), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-Our only modification part is that, if the Software (or any derivative works
-thereof) is used for any of your commercial products or services that have
-more than 100 million monthly active users, or more than 20 million US dollars
-(or equivalent in other currencies) in monthly revenue, you shall prominently
-display "Kimi K2.7 Code" on the user interface of such product or service.

README.md CHANGED Viewed

@@ -1,136 +1,3 @@
 ---
-license: other
-license_name: modified-mit
-license_link: LICENSE
-base_model:
-- moonshotai/Kimi-K2.7-Code
 ---
-# Model Overview
-- **Model Architecture:** Kimi-K2.7-Code
-  - **Input:** Text, Image, Video
-  - **Output:** Text
-- **Supported Hardware Microarchitecture:** AMD MI350/MI355
-- **ROCm:** 7.2.3
-- **PyTorch:** 2.10.0
-- **Transformers:** 5.12.1
-- **Operating System(s):** Linux
-- **Inference Engine:** [vLLM](https://docs.vllm.ai/en/latest/)
-- **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (V0.12)
-  - **Weight quantization:** OCP MXFP4, Static; self_attn Perchannel, FP8E4M3, Static
-  - **Activation quantization:** OCP MXFP4, Dynamic; self_attn Pertoken, FP8E4M3, Dynamic
-  - **Excluded from quantization:** MoE gates, `lm_head`, vision tower and multimodal projector
-This model was built with the Kimi-K2.7-Code model by applying [AMD-Quark](https://quark.docs.amd.com/latest/index.html) for MXFP4 quantization.
-# Model Quantization
-The model was quantized from [moonshotai/Kimi-K2.7-Code](https://huggingface.co/moonshotai/Kimi-K2.7-Code) using [AMD-Quark](https://quark.docs.amd.com/latest/index.html). The MoE/Linear weights and activations are quantized to OCP MXFP4, while the attention projections use FP8 (E4M3). The vision tower and multimodal projector are kept at BF16.
-**Quantization script:**
-```bash
-cd Quark/examples/torch/language_modeling/llm_ptq/
-python3 quantize_quark.py \
-    --model_dir moonshotai/Kimi-K2.7-Code \
-    --output_dir Kimi-K2.7-Code-MXFP4 \
-    --file2file_quantization \
-    --trust_remote_code \
-    --quant_scheme mxfp4 \
-    --layer_quant_scheme '*self_attn*' ptpc_fp8 \
-    --exclude_layers "*lm_head*" "*mlp.gate" "*mm_projector*" \
-        "*vision_tower*" "mtp.*" "*shared_expert_gate*" "*router*" \
-    --model_export hf_format
-```
-# Deployment
-### Use with vLLM
-This model can be deployed efficiently using the [vLLM](https://docs.vllm.ai/en/latest/) backend.
-Note: this model has 64 KV heads, which is incompatible with the AITER MLA
-kernel (supports 16 or 128 only). Disable AITER MLA when serving on ROCm:
-```bash
-export VLLM_ROCM_USE_AITER=1
-export VLLM_ROCM_USE_AITER_MLA=0
-export VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0
-export VLLM_ROCM_USE_AITER_FP4BMM=0
-python3 -m vllm.entrypoints.openai.api_server \
-    --model amd/Kimi-K2.7-Code-MXFP4 \
-    --trust-remote-code \
-    --tensor-parallel-size 4 \
-    --gpu-memory-utilization 0.9 \
-    --max-model-len 8192
-```
-## Evaluation
-The model was evaluated on the GSM8K benchmark.
-### Accuracy
-<table>
-  <tr>
-   <td><strong>Benchmark</strong>
-   </td>
-   <td><strong>Kimi-K2.7-Code</strong>
-   </td>
-   <td><strong>Kimi-K2.7-Code-MXFP4 (this model)</strong>
-   </td>
-   <td><strong>Recovery</strong>
-   </td>
-  </tr>
-  <tr>
-   <td>GSM8K (strict-match)
-   </td>
-   <td>95.07
-   </td>
-   <td>94.80
-   </td>
-   <td>99.7%
-   </td>
-  </tr>
-  <tr>
-   <td>GSM8K (flexible-extract)
-   </td>
-   <td>95.15
-   </td>
-   <td>94.77
-   </td>
-   <td>99.6%
-   </td>
-  </tr>
-</table>
-GSM8K is 5-shot, greedy decoding. The MXFP4 numbers are the mean of repeated
-stable runs (range: strict 94.39–95.60, flexible 94.39–95.53).
-### Reproduction
-The GSM8K results were obtained using the `lm-evaluation-harness` framework
-with the vLLM backend (`rocm/vllm-dev` nightly, vLLM `0.23.1rc1`). The model
-is served first, then evaluated via the OpenAI-compatible completions API.
-Important: serve with automatic prefix caching disabled
-(`--no-enable-prefix-caching`) for deterministic evaluation results.
-```bash
-# 1) Serve
-export VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_MLA=0 \
-       VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0 VLLM_ROCM_USE_AITER_FP4BMM=0
-python3 -m vllm.entrypoints.openai.api_server \
-    --model amd/Kimi-K2.7-Code-MXFP4 \
-    --trust-remote-code --tensor-parallel-size 4 \
-    --gpu-memory-utilization 0.9 --max-model-len 8192 \
-    --seed 42 --no-enable-prefix-caching
-# 2) Evaluate
-lm_eval --model local-completions \
-    --model_args "model=amd/Kimi-K2.7-Code-MXFP4,base_url=http://0.0.0.0:8000/v1/completions,num_concurrent=128,tokenized_requests=False,max_length=8192,add_bos_token=True,seed=42,trust_remote_code=True" \
-    --tasks gsm8k --num_fewshot 5 --batch_size 1 --seed 42
-```
-# License
-Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.

 ---
+license: mit
 ---

THIRD_PARTY_NOTICES.md DELETED Viewed

@@ -1,43 +0,0 @@
-# THIRD_PARTY_NOTICES
-This file lists third-party software contained in Kimi-K2.7-Code along with their licenses, in compliance with the redistribution clauses of those licenses.
----
-## 1. DeepSeek-V3
-Our model architecture is DeepSeek-V3-like. Some of modeling codes are copied from the source repository.
-- **Source Repository**
-  https://huggingface.co/deepseek-ai/DeepSeek-V3
-- **Files / Directories Used**
-  - configuration_deepseek.py
-  - modeling_deepseek.py
-- **License Type**
-  MIT License
-- **Copyright Notice**
-  Copyright (c)  2023 DeepSeek
-- **Full License Text**
-```
-MIT License
-Copyright (c) 2023 DeepSeek
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-```

chat_template.jinja DELETED Viewed

@@ -1,61 +0,0 @@
-{%- macro render_content(msg) -%}
-    {%- set c = msg.get('content') -%}
-    {%- if c is string -%}
-      {{ c }}
-    {%- elif c is not none -%}
-      {% for content in c -%}
-        {% if content['type'] == 'image' or content['type'] == 'image_url' -%}
-          <|media_begin|>image<|media_content|><|media_pad|><|media_end|>
-        {% elif content['type'] == 'video' or content['type']== 'video_url'-%}
-          <|kimi_k25_video_placeholder|>
-        {% else -%}
-          {{ content['text'] }}
-        {%- endif -%}
-      {%- endfor -%}
-    {%- endif -%}
-{%- endmacro -%}
-{% macro set_roles(message) -%}
-  {%- set role_name =  message.get('name') or  message['role'] -%}
-  {%- if message['role'] == 'user' -%}
-    <|im_user|>{{role_name}}<|im_middle|>
-  {%- elif message['role'] == 'assistant' -%}
-    <|im_assistant|>{{role_name}}<|im_middle|>
-  {%- else -%}
-    <|im_system|>{{role_name}}<|im_middle|>
-  {%- endif -%}
-{%- endmacro -%}
-{%- macro render_toolcalls(message) -%}
-  <|tool_calls_section_begin|>
-  {%- for tool_call in message['tool_calls'] -%}
-    {%- set formatted_id = tool_call['id'] -%}
-    <|tool_call_begin|>{{ formatted_id }}<|tool_call_argument_begin|>{% if tool_call['function']['arguments'] is string %}{{ tool_call['function']['arguments'] }}{% else %}{{ tool_call['function']['arguments'] | tojson }}{% endif %}<|tool_call_end|>
-  {%- endfor -%}
-  <|tool_calls_section_end|>
-{%- endmacro -%}
-{%- if tools -%}
-  {%- if tools_ts_str -%}
-    <|im_system|>tool_declare<|im_middle|>{{ tools_ts_str }}<|im_end|>
-  {%- else -%}
-    <|im_system|>tool_declare<|im_middle|>{{ tools | tojson(separators=(',', ':')) }}<|im_end|>
-  {%- endif -%}
-{%- endif -%}
-{%- for message in messages -%}
-  {{set_roles(message)}}
-  {%- if message['role'] == 'assistant' -%}
-    {%- set rc = message.get('reasoning', message.get('reasoning_content', '')) -%}
-    <think>{{rc}}</think>{{render_content(message)}}
-    {%- if message.get('tool_calls') -%}
-      {{render_toolcalls(message)}}
-    {%- endif -%}
-  {%- elif message['role'] == 'tool' -%}
-    {%- set tool_call_id = message.tool_call_id -%}
-    ## Return of {{ tool_call_id }}
-{{render_content(message)}}
-  {%- elif message['content'] is not none -%}
-    {{render_content(message)}}
-  {%- endif -%}
-  <|im_end|>
-{%- endfor -%}
-{%- if add_generation_prompt -%}
-  <|im_assistant|>assistant<|im_middle|><think>
-{%- endif -%}

config.json DELETED Viewed

@@ -1,433 +0,0 @@
-{
-    "architectures": [
-        "KimiK25ForConditionalGeneration"
-    ],
-    "auto_map": {
-        "AutoConfig": "configuration_kimi_k25.KimiK25Config",
-        "AutoModel": "modeling_kimi_k25.KimiK25ForConditionalGeneration",
-        "AutoModelForCausalLM": "modeling_kimi_k25.KimiK25ForConditionalGeneration"
-    },
-    "bos_token_id": 163584,
-    "dtype": "bfloat16",
-    "eos_token_id": 163586,
-    "ignore_index": -100,
-    "media_placeholder_token_id": 163605,
-    "model_type": "kimi_k25",
-    "pad_token_id": 163839,
-    "text_config": {
-        "_name_or_path": "",
-        "add_cross_attention": false,
-        "architectures": [
-            "DeepseekV3ForCausalLM"
-        ],
-        "attention_bias": false,
-        "attention_dropout": 0.0,
-        "auto_map": {
-            "AutoConfig": "configuration_deepseek.DeepseekV3Config",
-            "AutoModel": "modeling_deepseek.DeepseekV3Model",
-            "AutoModelForCausalLM": "modeling_deepseek.DeepseekV3ForCausalLM"
-        },
-        "aux_loss_alpha": 0.001,
-        "bad_words_ids": null,
-        "begin_suppress_tokens": null,
-        "bos_token_id": 163584,
-        "chunk_size_feed_forward": 0,
-        "cross_attention_hidden_size": null,
-        "decoder_start_token_id": null,
-        "diversity_penalty": 0.0,
-        "do_sample": false,
-        "dtype": "bfloat16",
-        "early_stopping": false,
-        "encoder_no_repeat_ngram_size": 0,
-        "eos_token_id": 163586,
-        "ep_size": 1,
-        "exponential_decay_length_penalty": null,
-        "finetuning_task": null,
-        "first_k_dense_replace": 1,
-        "forced_bos_token_id": null,
-        "forced_eos_token_id": null,
-        "hidden_act": "silu",
-        "hidden_size": 7168,
-        "id2label": {
-            "0": "LABEL_0",
-            "1": "LABEL_1"
-        },
-        "initializer_range": 0.02,
-        "intermediate_size": 18432,
-        "is_decoder": false,
-        "is_encoder_decoder": false,
-        "kv_lora_rank": 512,
-        "label2id": {
-            "LABEL_0": 0,
-            "LABEL_1": 1
-        },
-        "length_penalty": 1.0,
-        "max_length": 20,
-        "max_position_embeddings": 262144,
-        "min_length": 0,
-        "model_type": "kimi_k2",
-        "moe_intermediate_size": 2048,
-        "moe_layer_freq": 1,
-        "n_group": 1,
-        "n_routed_experts": 384,
-        "n_shared_experts": 1,
-        "no_repeat_ngram_size": 0,
-        "norm_topk_prob": true,
-        "num_attention_heads": 64,
-        "num_beam_groups": 1,
-        "num_beams": 1,
-        "num_experts_per_tok": 8,
-        "num_hidden_layers": 61,
-        "num_key_value_heads": 64,
-        "num_nextn_predict_layers": 0,
-        "num_return_sequences": 1,
-        "output_attentions": false,
-        "output_hidden_states": false,
-        "output_scores": false,
-        "pad_token_id": 163839,
-        "prefix": null,
-        "pretraining_tp": 1,
-        "problem_type": null,
-        "pruned_heads": {},
-        "q_lora_rank": 1536,
-        "qk_nope_head_dim": 128,
-        "qk_rope_head_dim": 64,
-        "remove_invalid_values": false,
-        "repetition_penalty": 1.0,
-        "return_dict": true,
-        "return_dict_in_generate": false,
-        "rms_norm_eps": 1e-05,
-        "rope_scaling": {
-            "beta_fast": 32.0,
-            "beta_slow": 1.0,
-            "factor": 64.0,
-            "mscale": 1.0,
-            "mscale_all_dim": 1.0,
-            "original_max_position_embeddings": 4096,
-            "type": "yarn"
-        },
-        "rope_theta": 50000.0,
-        "routed_scaling_factor": 2.827,
-        "scoring_func": "sigmoid",
-        "sep_token_id": null,
-        "seq_aux": true,
-        "suppress_tokens": null,
-        "task_specific_params": null,
-        "temperature": 1.0,
-        "tf_legacy_loss": false,
-        "tie_encoder_decoder": false,
-        "tie_word_embeddings": false,
-        "tokenizer_class": null,
-        "top_k": 50,
-        "top_p": 1.0,
-        "topk_group": 1,
-        "topk_method": "noaux_tc",
-        "torchscript": false,
-        "transformers_version": "4.56.2",
-        "typical_p": 1.0,
-        "use_bfloat16": false,
-        "use_cache": true,
-        "v_head_dim": 128,
-        "vocab_size": 163840
-    },
-    "tie_word_embeddings": false,
-    "use_unified_vision_chunk": true,
-    "video_placeholder": "<|kimi_k25_video_placeholder|>",
-    "vision_config": {
-        "_attn_implementation": "flash_attention_2",
-        "init_pos_emb_height": 64,
-        "init_pos_emb_time": 4,
-        "init_pos_emb_width": 64,
-        "merge_kernel_size": [
-            2,
-            2
-        ],
-        "merge_type": "sd2_tpool",
-        "mm_hidden_size": 1152,
-        "mm_projector_type": "patchmerger",
-        "patch_size": 14,
-        "pos_emb_type": "divided_fixed",
-        "projector_hidden_act": "gelu",
-        "projector_ln_eps": 1e-05,
-        "text_hidden_size": 7168,
-        "video_attn_type": "spatial_temporal",
-        "vt_hidden_size": 1152,
-        "vt_intermediate_size": 4304,
-        "vt_num_attention_heads": 16,
-        "vt_num_hidden_layers": 27
-    },
-    "quantization_config": {
-        "global_quant_config": {
-            "input_tensors": {
-                "dtype": "fp4",
-                "is_dynamic": true,
-                "qscheme": "per_group",
-                "ch_axis": -1,
-                "group_size": 32,
-                "block_size": null,
-                "symmetric": null,
-                "round_method": "half_even",
-                "scale_type": "float",
-                "scale_format": "e8m0",
-                "scale_calculation_mode": "even",
-                "mx_element_dtype": null,
-                "observer_cls": "PerBlockMXObserver",
-                "is_scale_quant": false,
-                "enable_buffer_reuse": false,
-                "max_input_numel": 4194304
-            },
-            "output_tensors": null,
-            "weight": {
-                "dtype": "fp4",
-                "is_dynamic": false,
-                "qscheme": "per_group",
-                "ch_axis": -1,
-                "group_size": 32,
-                "block_size": null,
-                "symmetric": null,
-                "round_method": "half_even",
-                "scale_type": "float",
-                "scale_format": "e8m0",
-                "scale_calculation_mode": "even",
-                "mx_element_dtype": null,
-                "observer_cls": "PerBlockMXObserver",
-                "is_scale_quant": false,
-                "enable_buffer_reuse": false,
-                "max_input_numel": 4194304
-            },
-            "bias": null,
-            "target_device": null
-        },
-        "exclude": [
-            "language_model.lm_head",
-            "language_model.model.layers.1.mlp.gate",
-            "language_model.model.layers.10.mlp.gate",
-            "language_model.model.layers.11.mlp.gate",
-            "language_model.model.layers.12.mlp.gate",
-            "language_model.model.layers.13.mlp.gate",
-            "language_model.model.layers.14.mlp.gate",
-            "language_model.model.layers.15.mlp.gate",
-            "language_model.model.layers.16.mlp.gate",
-            "language_model.model.layers.17.mlp.gate",
-            "language_model.model.layers.18.mlp.gate",
-            "language_model.model.layers.19.mlp.gate",
-            "language_model.model.layers.2.mlp.gate",
-            "language_model.model.layers.20.mlp.gate",
-            "language_model.model.layers.21.mlp.gate",
-            "language_model.model.layers.22.mlp.gate",
-            "language_model.model.layers.23.mlp.gate",
-            "language_model.model.layers.24.mlp.gate",
-            "language_model.model.layers.25.mlp.gate",
-            "language_model.model.layers.26.mlp.gate",
-            "language_model.model.layers.27.mlp.gate",
-            "language_model.model.layers.28.mlp.gate",
-            "language_model.model.layers.29.mlp.gate",
-            "language_model.model.layers.3.mlp.gate",
-            "language_model.model.layers.30.mlp.gate",
-            "language_model.model.layers.31.mlp.gate",
-            "language_model.model.layers.32.mlp.gate",
-            "language_model.model.layers.33.mlp.gate",
-            "language_model.model.layers.34.mlp.gate",
-            "language_model.model.layers.35.mlp.gate",
-            "language_model.model.layers.36.mlp.gate",
-            "language_model.model.layers.37.mlp.gate",
-            "language_model.model.layers.38.mlp.gate",
-            "language_model.model.layers.39.mlp.gate",
-            "language_model.model.layers.4.mlp.gate",
-            "language_model.model.layers.40.mlp.gate",
-            "language_model.model.layers.41.mlp.gate",
-            "language_model.model.layers.42.mlp.gate",
-            "language_model.model.layers.43.mlp.gate",
-            "language_model.model.layers.44.mlp.gate",
-            "language_model.model.layers.45.mlp.gate",
-            "language_model.model.layers.46.mlp.gate",
-            "language_model.model.layers.47.mlp.gate",
-            "language_model.model.layers.48.mlp.gate",
-            "language_model.model.layers.49.mlp.gate",
-            "language_model.model.layers.5.mlp.gate",
-            "language_model.model.layers.50.mlp.gate",
-            "language_model.model.layers.51.mlp.gate",
-            "language_model.model.layers.52.mlp.gate",
-            "language_model.model.layers.53.mlp.gate",
-            "language_model.model.layers.54.mlp.gate",
-            "language_model.model.layers.55.mlp.gate",
-            "language_model.model.layers.56.mlp.gate",
-            "language_model.model.layers.57.mlp.gate",
-            "language_model.model.layers.58.mlp.gate",
-            "language_model.model.layers.59.mlp.gate",
-            "language_model.model.layers.6.mlp.gate",
-            "language_model.model.layers.60.mlp.gate",
-            "language_model.model.layers.7.mlp.gate",
-            "language_model.model.layers.8.mlp.gate",
-            "language_model.model.layers.9.mlp.gate",
-            "mm_projector.proj.0",
-            "mm_projector.proj.2",
-            "vision_tower.encoder.blocks.0.mlp.fc0",
-            "vision_tower.encoder.blocks.0.mlp.fc1",
-            "vision_tower.encoder.blocks.0.wo",
-            "vision_tower.encoder.blocks.0.wqkv",
-            "vision_tower.encoder.blocks.1.mlp.fc0",
-            "vision_tower.encoder.blocks.1.mlp.fc1",
-            "vision_tower.encoder.blocks.1.wo",
-            "vision_tower.encoder.blocks.1.wqkv",
-            "vision_tower.encoder.blocks.10.mlp.fc0",
-            "vision_tower.encoder.blocks.10.mlp.fc1",
-            "vision_tower.encoder.blocks.10.wo",
-            "vision_tower.encoder.blocks.10.wqkv",
-            "vision_tower.encoder.blocks.11.mlp.fc0",
-            "vision_tower.encoder.blocks.11.mlp.fc1",
-            "vision_tower.encoder.blocks.11.wo",
-            "vision_tower.encoder.blocks.11.wqkv",
-            "vision_tower.encoder.blocks.12.mlp.fc0",
-            "vision_tower.encoder.blocks.12.mlp.fc1",
-            "vision_tower.encoder.blocks.12.wo",
-            "vision_tower.encoder.blocks.12.wqkv",
-            "vision_tower.encoder.blocks.13.mlp.fc0",
-            "vision_tower.encoder.blocks.13.mlp.fc1",
-            "vision_tower.encoder.blocks.13.wo",
-            "vision_tower.encoder.blocks.13.wqkv",
-            "vision_tower.encoder.blocks.14.mlp.fc0",
-            "vision_tower.encoder.blocks.14.mlp.fc1",
-            "vision_tower.encoder.blocks.14.wo",
-            "vision_tower.encoder.blocks.14.wqkv",
-            "vision_tower.encoder.blocks.15.mlp.fc0",
-            "vision_tower.encoder.blocks.15.mlp.fc1",
-            "vision_tower.encoder.blocks.15.wo",
-            "vision_tower.encoder.blocks.15.wqkv",
-            "vision_tower.encoder.blocks.16.mlp.fc0",
-            "vision_tower.encoder.blocks.16.mlp.fc1",
-            "vision_tower.encoder.blocks.16.wo",
-            "vision_tower.encoder.blocks.16.wqkv",
-            "vision_tower.encoder.blocks.17.mlp.fc0",
-            "vision_tower.encoder.blocks.17.mlp.fc1",
-            "vision_tower.encoder.blocks.17.wo",
-            "vision_tower.encoder.blocks.17.wqkv",
-            "vision_tower.encoder.blocks.18.mlp.fc0",
-            "vision_tower.encoder.blocks.18.mlp.fc1",
-            "vision_tower.encoder.blocks.18.wo",
-            "vision_tower.encoder.blocks.18.wqkv",
-            "vision_tower.encoder.blocks.19.mlp.fc0",
-            "vision_tower.encoder.blocks.19.mlp.fc1",
-            "vision_tower.encoder.blocks.19.wo",
-            "vision_tower.encoder.blocks.19.wqkv",
-            "vision_tower.encoder.blocks.2.mlp.fc0",
-            "vision_tower.encoder.blocks.2.mlp.fc1",
-            "vision_tower.encoder.blocks.2.wo",
-            "vision_tower.encoder.blocks.2.wqkv",
-            "vision_tower.encoder.blocks.20.mlp.fc0",
-            "vision_tower.encoder.blocks.20.mlp.fc1",
-            "vision_tower.encoder.blocks.20.wo",
-            "vision_tower.encoder.blocks.20.wqkv",
-            "vision_tower.encoder.blocks.21.mlp.fc0",
-            "vision_tower.encoder.blocks.21.mlp.fc1",
-            "vision_tower.encoder.blocks.21.wo",
-            "vision_tower.encoder.blocks.21.wqkv",
-            "vision_tower.encoder.blocks.22.mlp.fc0",
-            "vision_tower.encoder.blocks.22.mlp.fc1",
-            "vision_tower.encoder.blocks.22.wo",
-            "vision_tower.encoder.blocks.22.wqkv",
-            "vision_tower.encoder.blocks.23.mlp.fc0",
-            "vision_tower.encoder.blocks.23.mlp.fc1",
-            "vision_tower.encoder.blocks.23.wo",
-            "vision_tower.encoder.blocks.23.wqkv",
-            "vision_tower.encoder.blocks.24.mlp.fc0",
-            "vision_tower.encoder.blocks.24.mlp.fc1",
-            "vision_tower.encoder.blocks.24.wo",
-            "vision_tower.encoder.blocks.24.wqkv",
-            "vision_tower.encoder.blocks.25.mlp.fc0",
-            "vision_tower.encoder.blocks.25.mlp.fc1",
-            "vision_tower.encoder.blocks.25.wo",
-            "vision_tower.encoder.blocks.25.wqkv",
-            "vision_tower.encoder.blocks.26.mlp.fc0",
-            "vision_tower.encoder.blocks.26.mlp.fc1",
-            "vision_tower.encoder.blocks.26.wo",
-            "vision_tower.encoder.blocks.26.wqkv",
-            "vision_tower.encoder.blocks.3.mlp.fc0",
-            "vision_tower.encoder.blocks.3.mlp.fc1",
-            "vision_tower.encoder.blocks.3.wo",
-            "vision_tower.encoder.blocks.3.wqkv",
-            "vision_tower.encoder.blocks.4.mlp.fc0",
-            "vision_tower.encoder.blocks.4.mlp.fc1",
-            "vision_tower.encoder.blocks.4.wo",
-            "vision_tower.encoder.blocks.4.wqkv",
-            "vision_tower.encoder.blocks.5.mlp.fc0",
-            "vision_tower.encoder.blocks.5.mlp.fc1",
-            "vision_tower.encoder.blocks.5.wo",
-            "vision_tower.encoder.blocks.5.wqkv",
-            "vision_tower.encoder.blocks.6.mlp.fc0",
-            "vision_tower.encoder.blocks.6.mlp.fc1",
-            "vision_tower.encoder.blocks.6.wo",
-            "vision_tower.encoder.blocks.6.wqkv",
-            "vision_tower.encoder.blocks.7.mlp.fc0",
-            "vision_tower.encoder.blocks.7.mlp.fc1",
-            "vision_tower.encoder.blocks.7.wo",
-            "vision_tower.encoder.blocks.7.wqkv",
-            "vision_tower.encoder.blocks.8.mlp.fc0",
-            "vision_tower.encoder.blocks.8.mlp.fc1",
-            "vision_tower.encoder.blocks.8.wo",
-            "vision_tower.encoder.blocks.8.wqkv",
-            "vision_tower.encoder.blocks.9.mlp.fc0",
-            "vision_tower.encoder.blocks.9.mlp.fc1",
-            "vision_tower.encoder.blocks.9.wo",
-            "vision_tower.encoder.blocks.9.wqkv"
-        ],
-        "algo_config": null,
-        "softmax_quant_spec": null,
-        "quant_method": "quark",
-        "layer_type_quant_config": {},
-        "layer_quant_config": {
-            "*self_attn*": {
-                "input_tensors": {
-                    "dtype": "fp8_e4m3",
-                    "is_dynamic": true,
-                    "qscheme": "per_channel",
-                    "ch_axis": 1,
-                    "group_size": null,
-                    "block_size": null,
-                    "symmetric": true,
-                    "round_method": "half_even",
-                    "scale_type": "float",
-                    "scale_format": null,
-                    "scale_calculation_mode": null,
-                    "mx_element_dtype": null,
-                    "observer_cls": "PerChannelMinMaxObserver",
-                    "is_scale_quant": false,
-                    "enable_buffer_reuse": false,
-                    "max_input_numel": 4194304
-                },
-                "output_tensors": null,
-                "weight": {
-                    "dtype": "fp8_e4m3",
-                    "is_dynamic": false,
-                    "qscheme": "per_channel",
-                    "ch_axis": 0,
-                    "group_size": null,
-                    "block_size": null,
-                    "symmetric": true,
-                    "round_method": "half_even",
-                    "scale_type": "float",
-                    "scale_format": null,
-                    "scale_calculation_mode": null,
-                    "mx_element_dtype": null,
-                    "observer_cls": "PerChannelMinMaxObserver",
-                    "is_scale_quant": false,
-                    "enable_buffer_reuse": false,
-                    "max_input_numel": 4194304
-                },
-                "bias": null,
-                "target_device": null
-            }
-        },
-        "kv_cache_quant_config": {},
-        "kv_cache_post_rope": false,
-        "quant_mode": "eager_mode",
-        "version": "0.12+f7bd7b7b998",
-        "export": {
-            "kv_cache_group": [],
-            "min_kv_scale": 0.0,
-            "pack_method": "reorder",
-            "weight_format": "real_quantized",
-            "weight_merge_groups": null
-        }
-    }
-}

configuration_deepseek.py DELETED Viewed

@@ -1,214 +0,0 @@
-# Copy from https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/configuration_deepseek.py
-from transformers.configuration_utils import PretrainedConfig
-from transformers.utils import logging
-logger = logging.get_logger(__name__)
-DEEPSEEK_PRETRAINED_CONFIG_ARCHIVE_MAP = {}
-class DeepseekV3Config(PretrainedConfig):
-    r"""
-    This is the configuration class to store the configuration of a [`DeepseekV3Model`]. It is used to instantiate an DeepSeek
-    model according to the specified arguments, defining the model architecture. Instantiating a configuration with the
-    defaults will yield a similar configuration to that of the DeepSeek-V3.
-    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
-    documentation from [`PretrainedConfig`] for more information.
-    Args:
-        vocab_size (`int`, *optional*, defaults to 129280):
-            Vocabulary size of the Deep model. Defines the number of different tokens that can be represented by the
-            `inputs_ids` passed when calling [`DeepseekV3Model`]
-        hidden_size (`int`, *optional*, defaults to 4096):
-            Dimension of the hidden representations.
-        intermediate_size (`int`, *optional*, defaults to 11008):
-            Dimension of the MLP representations.
-        moe_intermediate_size (`int`, *optional*, defaults to 1407):
-            Dimension of the MoE representations.
-        num_hidden_layers (`int`, *optional*, defaults to 32):
-            Number of hidden layers in the Transformer decoder.
-        num_nextn_predict_layers (`int`, *optional*, defaults to 1):
-            Number of nextn predict layers in the DeepSeekV3 Model.
-        num_attention_heads (`int`, *optional*, defaults to 32):
-            Number of attention heads for each attention layer in the Transformer decoder.
-        n_shared_experts (`int`, *optional*, defaults to None):
-            Number of shared experts, None means dense model.
-        n_routed_experts (`int`, *optional*, defaults to None):
-            Number of routed experts, None means dense model.
-        routed_scaling_factor (`float`, *optional*, defaults to 1.0):
-            Scaling factor or routed experts.
-        topk_method (`str`, *optional*, defaults to `gready`):
-            Topk method used in routed gate.
-        n_group (`int`, *optional*, defaults to None):
-            Number of groups for routed experts.
-        topk_group (`int`, *optional*, defaults to None):
-            Number of selected groups for each token(for each token, ensuring the selected experts is only within `topk_group` groups).
-        num_experts_per_tok (`int`, *optional*, defaults to None):
-            Number of selected experts, None means dense model.
-        moe_layer_freq (`int`, *optional*, defaults to 1):
-            The frequency of the MoE layer: one expert layer for every `moe_layer_freq - 1` dense layers.
-        first_k_dense_replace (`int`, *optional*, defaults to 0):
-            Number of dense layers in shallow layers(embed->dense->dense->...->dense->moe->moe...->lm_head).
-                                                            \--k dense layers--/
-        norm_topk_prob (`bool`, *optional*, defaults to False):
-            Whether to normalize the weights of the routed experts.
-        scoring_func (`str`, *optional*, defaults to 'softmax'):
-            Method of computing expert weights.
-        aux_loss_alpha (`float`, *optional*, defaults to 0.001):
-            Auxiliary loss weight coefficient.
-        seq_aux = (`bool`, *optional*, defaults to True):
-            Whether to compute the auxiliary loss for each individual sample.
-        num_key_value_heads (`int`, *optional*):
-            This is the number of key_value heads that should be used to implement Grouped Query Attention. If
-            `num_key_value_heads=num_attention_heads`, the model will use Multi Head Attention (MHA), if
-            `num_key_value_heads=1 the model will use Multi Query Attention (MQA) otherwise GQA is used. When
-            converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed
-            by meanpooling all the original heads within that group. For more details checkout [this
-            paper](https://arxiv.org/pdf/2305.13245.pdf). If it is not specified, will default to
-            `num_attention_heads`.
-        hidden_act (`str` or `function`, *optional*, defaults to `"silu"`):
-            The non-linear activation function (function or string) in the decoder.
-        max_position_embeddings (`int`, *optional*, defaults to 2048):
-            The maximum sequence length that this model might ever be used with.
-        initializer_range (`float`, *optional*, defaults to 0.02):
-            The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
-        rms_norm_eps (`float`, *optional*, defaults to 1e-06):
-            The epsilon used by the rms normalization layers.
-        use_cache (`bool`, *optional*, defaults to `True`):
-            Whether or not the model should return the last key/values attentions (not used by all models). Only
-            relevant if `config.is_decoder=True`.
-        pad_token_id (`int`, *optional*):
-            Padding token id.
-        bos_token_id (`int`, *optional*, defaults to 1):
-            Beginning of stream token id.
-        eos_token_id (`int`, *optional*, defaults to 2):
-            End of stream token id.
-        pretraining_tp (`int`, *optional*, defaults to 1):
-            Experimental feature. Tensor parallelism rank used during pretraining. Please refer to [this
-            document](https://huggingface.co/docs/transformers/parallelism) to understand more about it. This value is
-            necessary to ensure exact reproducibility of the pretraining results. Please refer to [this
-            issue](https://github.com/pytorch/pytorch/issues/76232).
-        tie_word_embeddings (`bool`, *optional*, defaults to `False`):
-            Whether to tie weight embeddings
-        rope_theta (`float`, *optional*, defaults to 10000.0):
-            The base period of the RoPE embeddings.
-        rope_scaling (`Dict`, *optional*):
-            Dictionary containing the scaling configuration for the RoPE embeddings. Currently supports two scaling
-            strategies: linear and dynamic. Their scaling factor must be a float greater than 1. The expected format is
-            `{"type": strategy name, "factor": scaling factor}`. When using this flag, don't update
-            `max_position_embeddings` to the expected new maximum.
-        attention_bias (`bool`, defaults to `False`, *optional*, defaults to `False`):
-            Whether to use a bias in the query, key, value and output projection layers during self-attention.
-        attention_dropout (`float`, *optional*, defaults to 0.0):
-            The dropout ratio for the attention probabilities.
-    ```python
-    >>> from transformers import DeepseekV3Model, DeepseekV3Config
-    >>> # Initializing a Deepseek-V3 style configuration
-    >>> configuration = DeepseekV3Config()
-    >>> # Accessing the model configuration
-    >>> configuration = model.config
-    ```"""
-    model_type = "deepseek_v3"
-    keys_to_ignore_at_inference = ["past_key_values"]
-    def __init__(
-        self,
-        vocab_size=129280,
-        hidden_size=7168,
-        intermediate_size=18432,
-        moe_intermediate_size=2048,
-        num_hidden_layers=61,
-        num_nextn_predict_layers=1,
-        num_attention_heads=128,
-        num_key_value_heads=128,
-        n_shared_experts=1,
-        n_routed_experts=256,
-        ep_size=1,
-        routed_scaling_factor=2.5,
-        kv_lora_rank=512,
-        q_lora_rank=1536,
-        qk_rope_head_dim=64,
-        v_head_dim=128,
-        qk_nope_head_dim=128,
-        topk_method='noaux_tc',
-        n_group=8,
-        topk_group=4,
-        num_experts_per_tok=8,
-        moe_layer_freq=1,
-        first_k_dense_replace=3,
-        norm_topk_prob=True,
-        scoring_func='sigmoid',
-        aux_loss_alpha=0.001,
-        seq_aux=True,
-        hidden_act="silu",
-        max_position_embeddings=4096,
-        initializer_range=0.02,
-        rms_norm_eps=1e-6,
-        use_cache=True,
-        pad_token_id=None,
-        bos_token_id=0,
-        eos_token_id=1,
-        pretraining_tp=1,
-        tie_word_embeddings=False,
-        rope_theta=10000.0,
-        rope_scaling=None,
-        attention_bias=False,
-        attention_dropout=0.0,
-        **kwargs,
-    ):
-        self.vocab_size = vocab_size
-        self.max_position_embeddings = max_position_embeddings
-        self.hidden_size = hidden_size
-        self.intermediate_size = intermediate_size
-        self.moe_intermediate_size = moe_intermediate_size
-        self.num_hidden_layers = num_hidden_layers
-        self.num_nextn_predict_layers = num_nextn_predict_layers
-        self.num_attention_heads = num_attention_heads
-        self.n_shared_experts = n_shared_experts
-        self.n_routed_experts = n_routed_experts
-        self.ep_size = ep_size
-        self.routed_scaling_factor = routed_scaling_factor
-        self.kv_lora_rank = kv_lora_rank
-        self.q_lora_rank = q_lora_rank
-        self.qk_rope_head_dim = qk_rope_head_dim
-        self.v_head_dim = v_head_dim
-        self.qk_nope_head_dim = qk_nope_head_dim
-        self.topk_method = topk_method
-        self.n_group = n_group
-        self.topk_group = topk_group
-        self.num_experts_per_tok = num_experts_per_tok
-        self.moe_layer_freq = moe_layer_freq
-        self.first_k_dense_replace = first_k_dense_replace
-        self.norm_topk_prob = norm_topk_prob
-        self.scoring_func = scoring_func
-        self.aux_loss_alpha = aux_loss_alpha
-        self.seq_aux = seq_aux
-        # for backward compatibility
-        if num_key_value_heads is None:
-            num_key_value_heads = num_attention_heads
-        self.num_key_value_heads = num_key_value_heads
-        self.hidden_act = hidden_act
-        self.initializer_range = initializer_range
-        self.rms_norm_eps = rms_norm_eps
-        self.pretraining_tp = pretraining_tp
-        self.use_cache = use_cache
-        self.rope_theta = rope_theta
-        self.rope_scaling = rope_scaling
-        self.attention_bias = attention_bias
-        self.attention_dropout = attention_dropout
-        super().__init__(
-            pad_token_id=pad_token_id,
-            bos_token_id=bos_token_id,
-            eos_token_id=eos_token_id,
-            tie_word_embeddings=tie_word_embeddings,
-            **kwargs,
-        )

configuration_kimi_k25.py DELETED Viewed

@@ -1,123 +0,0 @@
-from transformers.configuration_utils import PretrainedConfig
-try:
-    from configuration_deepseek import DeepseekV3Config
-except ImportError:
-    from .configuration_deepseek import DeepseekV3Config
-class KimiK25VisionConfig(PretrainedConfig):
-    def __init__(
-            self,
-            patch_size: int = 14,
-            init_pos_emb_height: int = 64,
-            init_pos_emb_width: int = 64,
-            init_pos_emb_time: int = 4,
-            pos_emb_type: str = 'divided_fixed',
-            vt_num_attention_heads: int = 16,
-            vt_num_hidden_layers: int = 27,
-            vt_hidden_size: int = 1152,
-            vt_intermediate_size: int = 4304,
-            merge_kernel_size: tuple = (2, 2),
-            video_attn_type: str = 'spatial_temporal',
-            merge_type: str = 'sd2_tpool',
-            _attn_implementation: str = 'flash_attention_2',
-            # MM Projector parameters
-            mm_projector_type: str = 'patchmerger',
-            mm_hidden_size: int | None = None,
-            projector_hidden_act: str = "gelu",
-            projector_ln_eps: float = 1e-5,
-            # Other parameters
-            ignore_index: int = -100,
-            media_placeholder_token_id: int = 163605,
-            pad_token_id: int = 0,
-            use_unified_vision_chunk: bool = True,
-            video_placeholder="<|kimi_k25_video_placeholder|>",
-            text_hidden_size=7168,
-            **vision_config_kwargs):
-        self.patch_size = patch_size
-        self.init_pos_emb_height = init_pos_emb_height
-        self.init_pos_emb_width = init_pos_emb_width
-        self.init_pos_emb_time = init_pos_emb_time
-        self.pos_emb_type = pos_emb_type
-        self.vt_num_attention_heads = vt_num_attention_heads
-        self.vt_num_hidden_layers = vt_num_hidden_layers
-        self.vt_hidden_size = vt_hidden_size
-        self.vt_intermediate_size = vt_intermediate_size
-        self.merge_kernel_size = merge_kernel_size
-        self.video_attn_type = video_attn_type
-        self.merge_type = merge_type
-        self._attn_implementation = _attn_implementation
-        # MM Projector config
-        self.mm_projector_type = mm_projector_type
-        self.mm_hidden_size = mm_hidden_size if mm_hidden_size is not None else vt_hidden_size
-        self.projector_hidden_act = projector_hidden_act
-        self.projector_ln_eps = projector_ln_eps
-        self.text_hidden_size = text_hidden_size
-class KimiK25Config(PretrainedConfig):
-    """Kimi-K2.5 model configuration.
-    Args:
-        text_config (dict | DeepseekV3Config): Configuration for the text model.
-        Vision Tower Parameters (from MoonViT3dConfig):
-            patch_size (int): Patch size for vision tower.
-            init_pos_emb_height (int): Initial position embedding height.
-            init_pos_emb_width (int): Initial position embedding width.
-            init_pos_emb_time (int): Initial position embedding time dimension.
-            pos_emb_type (str): Type of position embedding.
-            vt_num_attention_heads (int): Number of attention heads in vision tower.
-            vt_num_hidden_layers (int): Number of hidden layers in vision tower.
-            vt_hidden_size (int): Hidden size of vision tower.
-            vt_intermediate_size (int): Intermediate size in vision tower FFN.
-            merge_kernel_size (tuple): Kernel size for patch merging.
-            video_attn_type (str): Type of video attention.
-            merge_type (str): Type of merge operation.
-            _attn_implementation (str): Attention implementation type.
-        MM Projector Parameters (from MultiModalProjectorConfig):
-            mm_projector_type (str): Type of multimodal projector.
-            mm_hidden_size (int): Hidden size from vision tower (should match vt_hidden_size).
-            projector_hidden_act (str): Activation function for projector.
-            projector_ln_eps (float): Layer norm epsilon for projector.
-        Other Parameters:
-            ignore_index (int): The ignore index for the loss function.
-            media_placeholder_token_id (int): The token ID to use for media placeholders.
-            pad_token_id (int): The token ID to use for padding.
-    """
-    model_type = "kimi_k25"
-    def __init__(
-        self,
-        text_config: dict | DeepseekV3Config = None,
-        vision_config: dict | KimiK25VisionConfig = None,
-        # Other parameters
-        ignore_index: int = -100,
-        media_placeholder_token_id: int = 163605,
-        pad_token_id: int = 0,
-        use_unified_vision_chunk: bool = True,
-        video_placeholder="<|kimi_k25_video_placeholder|>",
-        **kwargs,
-    ):
-        if isinstance(text_config, dict):
-            text_config = DeepseekV3Config(**text_config)
-        if isinstance(vision_config, dict):
-            vision_config = KimiK25VisionConfig(**vision_config)
-        self.text_config = text_config
-        self.vision_config = vision_config
-        # Other config
-        self.ignore_index = ignore_index
-        self.media_placeholder_token_id = media_placeholder_token_id
-        self.use_unified_vision_chunk = use_unified_vision_chunk
-        self.video_placeholder = video_placeholder
-        if getattr(self.text_config, "quantization_config", None) is not None:
-            self.quantization_config = self.text_config.quantization_config
-        super().__init__(pad_token_id=pad_token_id, **kwargs)

generation_config.json DELETED Viewed

@@ -1,6 +0,0 @@
-{
-  "max_length": 262144,
-  "eos_token_id": 163586,
-  "temperature": 1.0,
-  "top_p": 0.95
-}

kimi_k25_processor.py DELETED Viewed

@@ -1,165 +0,0 @@
-from transformers.feature_extraction_utils import BatchFeature
-from transformers.processing_utils import ProcessorMixin
-from transformers.utils import logging
-logger = logging.get_logger(__name__)
-class KimiK25Processor(ProcessorMixin):
-    r"""
-    Constructs a KimiK25 processor which wraps a KimiK25 image processor and a tokenizer into a single processor.
-    [`KimiK25Processor`] offers all the functionalities of [`KimiK25ImageProcessor`] and [`TikTokenTokenizer`]. See the
-    [`~KimiK25Processor.__call__`] and [`~KimiK25Processor.decode`] for more information.
-    Args:
-        image_processor ([`KimiK25ImageProcessor`], *optional*):
-            The image processor is a required input.
-        tokenizer ([`TikTokenTokenizer`], *optional*):
-            The tokenizer is a required input.
-        chat_template (`str`, *optional*): A Jinja template which will be used to convert lists of messages
-            in a chat into a tokenizable string.
-    """
-    attributes = ["image_processor", "tokenizer"]
-    valid_kwargs = ["chat_template"]
-    image_processor_class = "AutoImageProcessor"
-    tokenizer_class = "AutoTokenizer"
-    def __init__(
-        self,
-        image_processor=None,
-        tokenizer=None,
-        chat_template=None,
-        **kwargs,
-    ):
-        super().__init__(image_processor,
-                         tokenizer,
-                         chat_template=chat_template)
-        self.media_processor = image_processor
-        # A special temporal placeholder to be replaced by actual video placeholders
-        self.video_placeholder = "<|kimi_k25_video_placeholder|>"
-    def update_raw_text(self, text: str, video_prompts: list[str]) -> str:
-        # replace video prompt in text with video chunk prompts
-        video_count = text.count(self.video_placeholder)
-        if video_count == 0:
-            return text
-        assert video_count == len(video_prompts)
-        text_parts = text.split(self.video_placeholder)
-        assert len(text_parts) == len(video_prompts) + 1
-        text = "".join([
-            text_parts[i] + video_prompts[i] for i in range(len(video_prompts))
-        ])
-        text += text_parts[-1]
-        return text
-    def preprocess_medias(self, medias: list[dict]) -> list[dict]:
-        updated_medias = []
-        video_prompts = []
-        for media in medias:
-            if media['type'] == 'image':
-                updated_medias.append(media)
-            elif media['type'] == 'video':
-                video_chunks = self.media_processor.split_video_chunks(
-                    media['video'])
-                updated_medias.extend(video_chunks)
-                video_prompts.append("".join(
-                    [vc['prompt'] for vc in video_chunks]))
-            else:
-                raise ValueError(f"unsupported media type: {media['type']}")
-        return updated_medias, video_prompts
-    def __call__(self,
-                 messages: list[dict] = None,
-                 medias: list[dict] = None,
-                 text: str = None,
-                 return_tensors: str = "pt",
-                 **kwargs) -> BatchFeature:
-        """
-        Process multimodal inputs for Kimi-K2.5 model.
-        This processor accepts ordered messages and extracts both media and text in a single pass.
-        text will be automatically updated if video input detected in messages
-        Args:
-            messages: List of message dicts with 'role' and 'content' fields.
-                     If provided, medias and text will be extracted automatically.
-            medias: Pre-extracted list of media dicts. If None, extracted from messages.
-            text: Pre-formatted text string. If None, generated via apply_chat_template.
-            return_tensors: Format of returned tensors ('pt', 'np', 'tf'). Default: 'pt'.
-            **kwargs: Additional arguments passed to tokenizer.apply_chat_template.
-        Returns:
-            BatchFeature with fields: input_ids, attention_mask, pixel_values, grid_thws.
-        """
-        if messages is None and (medias is None or text is None):
-            raise ValueError(
-                "Provide either 'messages' or both 'medias' and 'text'")
-        if medias is not None and text is not None:
-            updated_medias, video_prompts = self.preprocess_medias(medias)
-            preprocessed = self.media_processor.preprocess(
-                updated_medias, return_tensors=return_tensors)
-            text = self.update_raw_text(text, video_prompts)
-            text_inputs = self.tokenizer(text, return_tensors=return_tensors)
-            return BatchFeature(data={**text_inputs, **preprocessed.data})
-        if medias is None:
-            medias = self._extract_medias_from_messages(messages)
-        updated_medias, video_prompts = self.preprocess_medias(medias)
-        preprocessed = self.media_processor.preprocess(
-            updated_medias, return_tensors=return_tensors)
-        # Generate text if not provided
-        if text is None:
-            text = self.tokenizer.apply_chat_template(messages, **kwargs)
-        text = self.update_raw_text(text, video_prompts)
-        text_inputs = self.tokenizer(text, return_tensors=return_tensors)
-        return BatchFeature(data={**text_inputs, **preprocessed.data})
-    @staticmethod
-    def _extract_medias_from_messages(messages: list[dict]) -> list[dict]:
-        """
-        Extract media items from messages in a single pass.
-        This is an optimized version that processes messages only once.
-        Kept as internal method since external callers should use __call__.
-        """
-        medias = []
-        for msg in messages:
-            if msg['role'] != 'user' or not msg.get('content'):
-                continue
-            for content_part in msg['content']:
-                if not isinstance(content_part, dict):
-                    continue
-                content_type = content_part.get('type')
-                if content_type in ['video_url', 'video']:
-                    medias.append({
-                        'type': 'video',
-                        'video': content_part['video_url']['url'],
-                        'first_frame_timestamp': 0.0
-                    })
-                elif content_type in ['image_url', 'image']:
-                    medias.append({
-                        'type': 'image',
-                        'image': content_part['image_url'],
-                    })
-        return medias
-    def apply_chat_template(self, messages, **kwargs):
-        return self.tokenizer.apply_chat_template(messages, **kwargs)
-    def batch_decode(self, *args, **kwargs):
-        return self.tokenizer.batch_decode(*args, **kwargs)
-    def decode(self, *args, **kwargs):
-        return self.tokenizer.decode(*args, **kwargs)
-    @property
-    def model_input_names(self):
-        return ['input_ids', 'attention_mask', 'pixel_values', 'grid_thws']

kimi_k25_vision_processing.py DELETED Viewed

@@ -1,251 +0,0 @@
-"""Image processor class for Kimi-K2.5.
-"""
-import json
-from typing import Any, Dict, Optional, Union
-import numpy as np
-import torch
-from PIL import Image
-from transformers.image_processing_utils import (BaseImageProcessor,
-                                                 BatchFeature)
-from transformers.utils import TensorType
-from .media_utils import (MediaInput, VideoChunkInput, _to_tensor,
-                          ensure_media_type, get_video_meta, image_to_np,
-                          navit_patchify, navit_resize_image,
-                          navit_resize_video, normalize,
-                          real_sample_fps_and_max_num_frames, timestamp_as_str)
-try:
-    from mecord import VideoReader
-except ImportError:
-    VideoReader = None
-def resampling(video_bytes: bytes,
-               sample_indices: list[int],
-               key_indices=None,
-               frame_time_info=None,
-               num_threads=4) -> str:
-    video = VideoReader(video_bytes,
-                        num_threads=num_threads,
-                        frame_time_info=frame_time_info,
-                        key_indices=key_indices)
-    # extract target frames
-    frames = video[sample_indices]
-    frames = [Image.fromarray(frame) for frame in frames]
-    return frames
-class KimiK25VisionProcessor(BaseImageProcessor):
-    model_type = "kimi_k25"
-    def __init__(
-        self,
-        media_proc_cfg: dict,
-        **kwargs,
-    ):
-        super().__init__(**kwargs)
-        self.media_proc_cfg = media_proc_cfg
-        self.num_frames_per_chunk = media_proc_cfg[
-            'temporal_merge_kernel_size']
-    def media_tokens_calculator(self, media: MediaInput):
-        media = ensure_media_type(media)
-        ret = self.get_resize_config(media)
-        return ret['num_tokens']
-    @classmethod
-    def make_chunk_prompt(cls, timestamp_text: str) -> str:
-        return f"{timestamp_text}<|media_begin|>video<|media_content|><|media_pad|><|media_end|>"
-    def split_video_chunks(self,
-                           video_url: str | bytes) -> list[list[Image.Image]]:
-        # video_url should be base64 str or bytes
-        video_spec = get_video_meta(video_url)
-        sample_fps = min(self.media_proc_cfg['sample_fps'], video_spec.fps)
-        sampled_nframes = max(
-            round(video_spec.num_frames * sample_fps / video_spec.fps), 1)
-        frame_inds = np.linspace(0, video_spec.num_frames - 1,
-                                 sampled_nframes).round().astype(int)
-        frame_inds = frame_inds.tolist()
-        sampled_frame_ids = []
-        temporal_merge_kernel_size = self.media_proc_cfg[
-            "temporal_merge_kernel_size"]
-        num_chunks = 0
-        chunk_timestamp = []
-        for i in range(0, len(frame_inds), temporal_merge_kernel_size):
-            sampled_frame_ids.extend(frame_inds[i:i +
-                                                temporal_merge_kernel_size])
-            start_time = frame_inds[i] / float(video_spec.fps)
-            timestamp_text = timestamp_as_str(
-                start_time, self.media_proc_cfg["timestamp_mode"])
-            chunk_timestamp.append(timestamp_text)
-            num_chunks += 1
-        sampled_frames = resampling(video_url, sampled_frame_ids)
-        chunks = []
-        for chunk_id in range(num_chunks):
-            chunk = sampled_frames[chunk_id *
-                                   temporal_merge_kernel_size:(chunk_id + 1) *
-                                   temporal_merge_kernel_size]
-            chunks.append(
-                VideoChunkInput(type="video_chunk",
-                                video_chunk=chunk,
-                                prompt=self.make_chunk_prompt(
-                                    chunk_timestamp[chunk_id])))
-        return chunks
-    def get_resize_config(self, media_input: MediaInput) -> dict:
-        if media_input['type'] == 'image':
-            w, h = media_input['image'].size
-            ret = navit_resize_image(
-                w, h, self.media_proc_cfg['patch_size'],
-                self.media_proc_cfg['merge_kernel_size'],
-                self.media_proc_cfg['in_patch_limit'],
-                self.media_proc_cfg['patch_limit_on_one_side'],
-                self.media_proc_cfg['fixed_output_tokens'])
-            return ret
-        elif media_input['type'] == 'video_chunk':
-            frame = media_input['video_chunk'][0]
-            width, height = frame.size
-            num_frames = len(media_input["video_chunk"])
-            fps = 1.0
-            sample_fps, max_num_frames_each_video = real_sample_fps_and_max_num_frames(
-                media_input["type"],
-                self.media_proc_cfg['sample_fps'],
-                self.media_proc_cfg['max_num_frames_each_video'],
-            )
-            in_patch_limit_each_frame = self.media_proc_cfg[
-                'in_patch_limit_each_frame']
-            if in_patch_limit_each_frame is None:
-                in_patch_limit_each_frame = self.media_proc_cfg[
-                    'in_patch_limit']
-            ret = navit_resize_video(
-                width,
-                height,
-                num_frames,
-                fps,
-                sample_fps,
-                self.media_proc_cfg['patch_size'],
-                self.media_proc_cfg['merge_kernel_size'],
-                in_patch_limit_each_frame,
-                self.media_proc_cfg['patch_limit_on_one_side'],
-                self.media_proc_cfg['in_patch_limit_video'],
-                max_num_frames_each_video,
-                self.media_proc_cfg['fixed_output_tokens'],
-            )
-            return ret
-        else:
-            raise ValueError("Unsupported type: {}".format(
-                media_input['type']))
-    def resize_image(self, image: Image.Image, new_width: int, new_height: int,
-                     pad_width: int, pad_height: int) -> np.ndarray:
-        image_np = image_to_np(image, (new_width, new_height), "resize")
-        image_np = np.pad(
-            image_np,
-            ((0, pad_height), (0, pad_width), (0, 0)),
-            mode="constant",
-            constant_values=0,
-        )
-        return image_np
-    def preprocess(
-        self,
-        medias: list[MediaInput],
-        return_tensors: Optional[Union[str, TensorType]] = None,
-    ) -> BatchFeature:
-        """
-        Preprocess a atom vision input (images/video_chunk) into model-ready tensors.
-        Args:
-            medias: List of MediaInput.
-            return_tensors: Desired output format ('pt', 'np', 'tf', or None).
-        Returns:
-            BatchFeature containing 'pixel_values' and 'grid_thws' tensors.
-        """
-        if not isinstance(medias, list):
-            medias = [medias]
-        if medias:
-            pixel_values = []
-            for item in medias:
-                item = ensure_media_type(item)
-                resize_config = self.get_resize_config(item)
-                new_width, new_height, pad_width, pad_height = resize_config[
-                    'new_width'], resize_config['new_height'], resize_config[
-                        'pad_width'], resize_config['pad_height']
-                if item['type'] == 'image':
-                    image = item['image']
-                    image_np = self.resize_image(image, new_width, new_height,
-                                                 pad_width, pad_height)
-                    pixel_values.append(np.expand_dims(image_np, axis=0))
-                elif item['type'] == 'video_chunk':
-                    pixels = []
-                    for frame in item['video_chunk']:
-                        frame_np = self.resize_image(frame, new_width,
-                                                     new_height, pad_width,
-                                                     pad_height)
-                        pixels.append(frame_np)
-                    pixel_values.append(np.stack(pixels, axis=0))
-                else:
-                    raise ValueError("Unsupported type: {}".format(
-                        item['type']))
-            normalized_pixel_values = []
-            image_std_inv = 1.0 / np.array(self.media_proc_cfg['image_std'])
-            image_mean = np.array(self.media_proc_cfg['image_mean'])
-            for pixels in pixel_values:
-                pixels = normalize(pixels, image_mean, image_std_inv)
-                pixels_and_thw = navit_patchify(
-                    pixels,
-                    self.media_proc_cfg['patch_size'],
-                )
-                normalized_pixel_values.append(pixels_and_thw)
-            pixel_values = torch.cat([
-                _to_tensor(pixel_value['pixel_values'])
-                for pixel_value in normalized_pixel_values
-            ])
-            grid_thws = torch.cat([
-                _to_tensor(pixel_value['grid_thw'],
-                           dtype=torch.int64).unsqueeze(0)
-                for pixel_value in normalized_pixel_values
-            ])
-            data = {
-                'pixel_values': pixel_values,
-                'grid_thws': grid_thws,
-            }
-        else:
-            data = {}
-        return BatchFeature(data=data, tensor_type=return_tensors)
-    def __repr__(self):
-        return f"KimiK25VisionProcessor(media_proc_cfg={self.media_proc_cfg})"
-    def to_dict(self) -> Dict[str, Any]:
-        output = super().to_dict()
-        output["media_proc_cfg"] = self.media_proc_cfg
-        if "media_processor" in output:
-            del output["media_processor"]
-        return output
-    @classmethod
-    def from_dict(cls, config_dict: Dict[str, Any], **kwargs):
-        config = config_dict.copy()
-        media_proc_cfg = config.pop("media_proc_cfg", {})
-        return cls(media_proc_cfg=media_proc_cfg, **config, **kwargs)
-    def to_json_string(self):
-        dictionary = self.to_dict()
-        for key, value in dictionary.items():
-            if hasattr(value, 'tolist'):
-                dictionary[key] = value.tolist()
-        return json.dumps(dictionary, indent=2, sort_keys=True) + "\n"

media_utils.py DELETED Viewed

@@ -1,368 +0,0 @@
-import base64
-import io
-import math
-import os
-from datetime import datetime, timezone
-from typing import List, Literal, Optional, TypedDict
-import numpy as np
-from PIL import Image
-from pydantic import BaseModel, Field
-try:
-    from mecord import VideoReader
-except ImportError:
-    VideoReader = None
-class VideoSpec(BaseModel):
-    media_type: str = Literal['video']
-    height: int = Field(..., gt=0, description="video frame height")
-    width: int = Field(..., gt=0, description="video frame width")
-    num_frames: int = Field(..., gt=0, description="num frames")
-    fps: float = Field(..., gt=0, description="average fps")
-    # optional, help to accelerate video reading
-    key_indices: list[int] = Field(None, description="key indices")
-    frame_time_info: dict = Field(None, description="frame time info")
-class ImageInput(TypedDict):
-    type: Literal['image']
-    image: Image.Image
-class VideoChunkInput(TypedDict):
-    type: Literal['video_chunk']
-    video_chunk: List[Image.Image]
-    prompt: Optional[str] = None
-MediaInput = ImageInput | VideoChunkInput
-def get_video_meta(video_src: bytes | str | os.PathLike,
-                   accurate: bool = True) -> dict:
-    """Get the dimensions of a video."""
-    if isinstance(video_src, os.PathLike):
-        video_src = str(video_src)
-    # if b64 string, decode to bytes
-    if isinstance(video_src,
-                  str) and video_src.startswith('data:video/mp4;base64,'):
-        video_src = base64.b64decode(video_src.split(',')[1])
-    video = VideoReader(video_src, auto_init=accurate, num_threads=1)
-    assert video.num_frames > 0, "Invalid video format."
-    assert video.original_width > 0 and video.original_height > 0, (
-        "Invalid video format.")
-    assert video.avg_fps > 0, "Invalid video format."
-    return VideoSpec(media_type='video',
-                     height=video.original_height,
-                     width=video.original_width,
-                     num_frames=video.num_frames,
-                     fps=video.avg_fps,
-                     key_indices=video.key_indices,
-                     frame_time_info=video.frame_time_info)
-def timestamp_as_str(timestamp: float,
-                     timestamp_mode: str = "hh:mm:ss.fff") -> str:
-    """Convert a timestamp to a string in the format of HH:MM:SS.mmm."""
-    if timestamp_mode == "hh:mm:ss.fff":
-        return (datetime.fromtimestamp(timestamp,
-                                       tz=timezone.utc).strftime("%H:%M:%S") +
-                f".{int((timestamp % 1) * 1000):03d}")
-    elif timestamp_mode == "mm:ss.fff":
-        return (datetime.fromtimestamp(timestamp,
-                                       tz=timezone.utc).strftime("%M:%S") +
-                f".{int((timestamp % 1) * 1000):03d}")
-    elif timestamp_mode == "mm:ss":
-        return datetime.fromtimestamp(timestamp,
-                                      tz=timezone.utc).strftime("%M:%S")
-    else:
-        raise ValueError(f"Invalid timestamp mode: {timestamp_mode}")
-def navit_resize_image(
-    width: int,
-    height: int,
-    patch_size: int,
-    merge_kernel_size: int,
-    in_patch_limit: int,
-    patch_limit_on_one_side: int,
-    fixed_output_tokens: int | None,
-):
-    # Apply the patch limits.
-    s1 = math.sqrt(
-        in_patch_limit /
-        (max(1.0, width // patch_size) * max(1.0, height // patch_size)))
-    s2 = patch_limit_on_one_side * patch_size / width
-    s3 = patch_limit_on_one_side * patch_size / height
-    scale = min(1.0, s1, s2, s3)
-    new_w, new_h = max(1, int(width * scale)), max(1, int(height * scale))
-    new_w = min(new_w, patch_limit_on_one_side * patch_size)
-    new_h = min(new_h, patch_limit_on_one_side * patch_size)
-    # Calculate the padding to make the height and width divisible by the merge kernel size and patch size.
-    factor = merge_kernel_size * patch_size
-    pad_height = (factor - new_h % factor) % factor
-    pad_width = (factor - new_w % factor) % factor
-    if fixed_output_tokens is not None:
-        num_tokens = fixed_output_tokens
-    else:
-        # Calculate new dimensions after padding and patching
-        token_height = (new_h + pad_height) // factor
-        token_width = (new_w + pad_width) // factor
-        assert token_height * merge_kernel_size <= patch_limit_on_one_side, (
-            f"token_height {token_height} * merge_kernel_size {merge_kernel_size} > patch_limit_on_one_side {patch_limit_on_one_side}"
-        )
-        assert token_width * merge_kernel_size <= patch_limit_on_one_side, (
-            f"token_width {token_width} * merge_kernel_size {merge_kernel_size} > patch_limit_on_one_side {patch_limit_on_one_side}"
-        )
-        num_tokens = token_height * token_width
-    return {
-        "num_tokens": num_tokens,
-        "new_width": new_w,
-        "new_height": new_h,
-        "pad_width": pad_width,
-        "pad_height": pad_height,
-        "sampled_nframes": 1,
-    }
-def navit_resize_video(
-    width: int,
-    height: int,
-    nframes: int,
-    avg_fps: float,
-    sample_fps: float,
-    patch_size: int,
-    merge_kernel_size: int,
-    in_patch_limit_each_frame: int,
-    patch_limit_on_one_side: int,
-    in_patch_limit_total: int | None,
-    max_num_frames_each_video: int | None,
-    fixed_output_tokens_each_frame: int | None,
-):
-    sample_fps = min(sample_fps, avg_fps)
-    # Calculate the number of frames to sample based on target FPS
-    sampled_nframes = max(round(nframes * sample_fps / avg_fps), 1)
-    if max_num_frames_each_video is not None:
-        sampled_nframes = min(sampled_nframes, max_num_frames_each_video)
-    if in_patch_limit_total is not None:
-        in_patch_limit_each_frame = min(
-            round(in_patch_limit_total / sampled_nframes),
-            in_patch_limit_each_frame)
-    ret = navit_resize_image(
-        width,
-        height,
-        patch_size,
-        merge_kernel_size,
-        in_patch_limit_each_frame,
-        patch_limit_on_one_side,
-        fixed_output_tokens_each_frame,
-    )
-    ret["sampled_nframes"] = sampled_nframes
-    return ret
-def real_sample_fps_and_max_num_frames(
-    type_name: Literal["video", "video_chunk"],
-    sample_fps: float,
-    max_num_frames_each_video: int | None,
-) -> tuple[int, int | None]:
-    if type_name == "video":
-        return sample_fps, max_num_frames_each_video
-    elif type_name == "video_chunk":
-        max_num_frames_each_video = None
-        sample_fps = math.inf
-        return sample_fps, max_num_frames_each_video
-    else:
-        return math.inf, None
-def _to_pil(data: str | bytes):
-    if isinstance(data, Image.Image):
-        return data.convert("RGB")
-    elif isinstance(data, str):
-        if data.startswith("data:"):
-            raw_base64 = data.split(",")[1]
-            return Image.open(io.BytesIO(
-                base64.b64decode(raw_base64))).convert("RGB")
-        else:
-            return Image.open(data).convert("RGB")
-    elif isinstance(data, bytes):
-        return Image.open(io.BytesIO(data)).convert("RGB")
-    else:
-        raise ValueError(f"Unsupported data type: {type(data)}")
-def ensure_media_type(media: MediaInput) -> MediaInput:
-    if media['type'] == 'image':
-        media['image'] = _to_pil(media['image'])
-        return media
-    elif media['type'] == 'video_chunk':
-        media['video_chunk'] = [
-            _to_pil(frame) for frame in media['video_chunk']
-        ]
-        return media
-    else:
-        raise ValueError(f"Unsupported media type: {media['type']}")
-def image_to_np(
-    image: Image.Image,
-    resize_to: tuple[int, int] | None = None,
-    mode: str = "resize",
-    raise_error_for_ill_resize: bool = True,
-) -> np.ndarray:
-    """Convert an image to a numpy array.
-    Args:
-        content: The image to convert.
-        resize_to: The size to resize the image to.
-        mode: The mode to resize the image to.
-        raise_error_for_ill_resize: Whether to raise an error for ill-sized resize.
-    Returns:
-        A numpy array.
-    """
-    assert isinstance(image, Image.Image), "image must be a PIL Image"
-    if resize_to is not None:
-        if mode == "resize":
-            image = image.resize(resize_to, resample=Image.Resampling.BICUBIC)
-        elif mode == "rescale_and_pad_to_center":
-            scale = min(resize_to[0] / image.width,
-                        resize_to[1] / image.height, 1.0)
-            new_width = round(image.width * scale)
-            new_height = round(image.height * scale)
-            if new_width == 0 or new_height == 0:
-                if raise_error_for_ill_resize:
-                    raise ValueError(
-                        f"Invalid resize to: {resize_to}, from image size: {image.size}"
-                    )
-                else:
-                    return np.zeros((resize_to[1], resize_to[0], 3),
-                                    dtype=np.uint8)
-            image = image.resize((new_width, new_height),
-                                 resample=Image.Resampling.BICUBIC)
-            padding_left = (resize_to[0] - new_width) // 2
-            padding_right = resize_to[0] - new_width - padding_left
-            padding_top = (resize_to[1] - new_height) // 2
-            padding_bottom = resize_to[1] - new_height - padding_top
-            image = np.asarray(image)
-            image = np.pad(
-                image,
-                ((padding_top, padding_bottom), (padding_left, padding_right),
-                 (0, 0)),
-                mode="constant",
-                constant_values=0,
-            )
-            assert image.shape == (resize_to[1], resize_to[0], 3)
-        elif mode == "rescale_and_pad_to_rightbottom":
-            scale = min(resize_to[0] / image.width,
-                        resize_to[1] / image.height, 1.0)
-            new_width = round(image.width * scale)
-            new_height = round(image.height * scale)
-            if new_width == 0 or new_height == 0:
-                if raise_error_for_ill_resize:
-                    raise ValueError(
-                        f"Invalid resize to: {resize_to}, from image size: {image.size}"
-                    )
-                else:
-                    return np.zeros((resize_to[1], resize_to[0], 3),
-                                    dtype=np.uint8)
-            image = image.resize((new_width, new_height),
-                                 resample=Image.Resampling.BICUBIC)
-            padding_right = resize_to[0] - new_width
-            padding_bottom = resize_to[1] - new_height
-            image = np.asarray(image)
-            image = np.pad(
-                image,
-                ((0, padding_bottom), (0, padding_right), (0, 0)),
-                mode="constant",
-                constant_values=0,
-            )
-            assert image.shape == (resize_to[1], resize_to[0], 3)
-        else:
-            raise ValueError(f"Invalid mode: {mode}")
-    if isinstance(image, Image.Image):
-        return np.asarray(image)
-    else:
-        return image
-def navit_patchify(pixel_values: np.ndarray,
-                   patch_size: int) -> dict[str, np.ndarray]:
-    """Reshape the pixel values to a navit shape.
-    Args:
-        pixel_values: np.ndarray, shape (t, h, w, c)
-        patch_size: int
-    Returns:
-        dict[str, np.ndarray]
-        - patches: np.ndarray, shape (t * h//patch_size * w//patch_size, c, patch_size, patch_size)
-        - grid_thw: np.ndarray, (t, h//patch_size, w//patch_size)
-    """
-    T, H, W, C = pixel_values.shape
-    assert C == 3, "pixel_values must have 3 channels"
-    patches = pixel_values.reshape(T, H // patch_size, patch_size,
-                                   W // patch_size, patch_size, C)
-    # (T, H//patch_size, W//patch_size, C, patch_size, patch_size)
-    patches = patches.transpose(0, 1, 3, 5, 2, 4)
-    patches = patches.reshape(-1, C, patch_size, patch_size)
-    grid_thw = np.array([T, H // patch_size, W // patch_size])
-    return {"pixel_values": patches, "grid_thw": grid_thw}
-def normalize(x: np.ndarray,
-              mean,
-              std_inv,
-              pixels_dtype: np.dtype = np.float32) -> np.ndarray:
-    """Normalize the image.
-    Args:
-        x: The image to normalize. The shape is (..., 3). The dtype is uint8. The range is [0, 255].
-        mean: The mean of the image.
-        std_inv: The inverse of the std of the image.
-        pixels_dtype: The dtype of the image.
-    Returns:
-        The normalized image. The shape is (..., 3). The dtype is determined by the pixels_dtype.
-    """
-    x = (x / 255.0).astype(pixels_dtype)
-    x -= mean
-    x *= std_inv
-    return x
-def _to_tensor(data, **kwargs):
-    import torch
-    if isinstance(data, np.ndarray):
-        return torch.from_numpy(data).to(**kwargs)
-    elif isinstance(data, torch.Tensor):
-        return data.to(**kwargs)
-    elif isinstance(data, list):
-        return [_to_tensor(item, **kwargs) for item in data]
-    elif isinstance(data, tuple):
-        return tuple(_to_tensor(item, **kwargs) for item in data)
-    elif isinstance(data, dict):
-        return {k: _to_tensor(v, **kwargs) for k, v in data.items()}
-    elif data is None:
-        return None
-    else:
-        raise ValueError(f"Unsupported data type: {type(data)}")

model-00001-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:a5534929491e07bcc47f8988cc1bcaab8f75ac642cdab6d303154b599e9c7d00
-size 311876368

model-00002-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:a7e65f5fb3501bb344187ddb9a9f7396ed3dccc2b419c34beabbc6aad3074cff
-size 9114734600

model-00003-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:1218146bf79dec04f78c860663b8bf2559207f366c3b896a681a181984e94c59
-size 9114734600

model-00004-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:d634d73aa58771d79907fe8bb1eb182680476aae4c6cee895ee19a0b08b27c61
-size 9114734600

model-00005-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:27999c82227bf0abac7b84ae2a39ef6b7390ceff89a44215ea918774b7f723fb
-size 9114734600

model-00006-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:08d2bb54186cde897a13ee8c6fd3f95e034e6845cb19045271d0492634218601
-size 9114734600

model-00007-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:6013ce1d89c6d4249ddfd18ae9ad8c999f95d1565cdd2a301cf6395eacb57255
-size 9114734600

model-00008-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f13ddf63e3f94820623e708e4410bc581ebf69cab4d6a6cf14936355459558dd
-size 9114734600

model-00009-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:32d7db55afc6d8b95dd699e239b08ab4a37c012c65ee457d28d7fd2a23c4f02f
-size 9114734600

model-00010-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:0f5909e62ec37d8273313adeb7b3526a31ddf42062c00a939a28a3fad8dbfb7e
-size 9114734600

model-00011-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:e478571a1a8f0d7e7869e9c3a81810b06dd2f16079477f7e492c4659f94f6545
-size 9114736920

model-00012-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:3f2ad758afc3ec09f1bd7c6dab3a10f7a4f97b2f7839899e5b7b361a541bc09e
-size 9114736920

model-00013-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:b36c6e7cab59e48796861493c2dc9a882bb7760f953ed03073e03b9de25d5c1f
-size 9114736920

model-00014-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:76b0c3e519a5feb3075b5dc680f41562db007714ace98173273dd75fe877aee5
-size 9114736920

model-00015-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f7357d842cd0d694e9399f8e7db62937df3e1b3809f1d088b01a3d44070e5bc7
-size 9114736920

model-00016-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:d2ffdf8da26b403ff8a8fa752a518fa0694cbfaef17058d9a28b8f51a89c7e53
-size 9114736920

model-00017-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:08214aa54833d00928396ec6eb70c7b71bdd58e6b65ef1b27c215ea3a06e2435
-size 9114736920

model-00018-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:208752b9b90321185382f3b14a1d1342e1a9685c6ed8cb8442e5938078bd4501
-size 9114736920

model-00019-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:b93a375b79942e9ac9a9bf6e3ecf91b73d54cea2115ed004f18d5e1cdb5375a5
-size 9114736920

model-00020-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:e9774b9bf6ae32dc9f60a9b58ea5adf7d7b3945b791dd6669503803541f50248
-size 9114736920

model-00021-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:07654fa139729369a7a203ab15555c325d120d7038fac9f299c8ca74fab418f4
-size 9114736920

model-00022-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:acb233668aaff95058d799e1abc483be2637f49f2e5b7273b1642f793a3532a6
-size 9114736920

model-00023-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f13506e0d94830420bc75d055951dcec273d4537f7f7ee977b59c96d7186b8c9
-size 9114736920

model-00024-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:8085b0dc101c1fa518bd636177406ea3c4d304550aeb6f1b03c22c59341b83d8
-size 9114736920

model-00025-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:2ee7ae7ebad2da178db97fabb5a2ae03d663e2db4989ab9f5150ff3308db9708
-size 9114736920

model-00026-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f746b8ee6b753af9f7bb11f7e758c6bf7ffa7546fbe4a7422580c9ba655f126a
-size 9114736920

model-00027-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:a14452550a57fb827895effa3c82c17ac2259837282a6a47b52a13c06fc4a743
-size 9114736920

model-00028-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:6baa181d5df23a9d8e70151926e9c79d2e05d503ac9e161a8693617a59750fdc
-size 9114736920

model-00029-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:40cced8605c485ef5ee66fb617c6bd4df4e2a9f7ae510509f2b968ed17af5a6f
-size 9114736920

model-00030-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:55a56be4f9e9a81dbeb99ab9a385bda0547effc1d01e28868fefb4ee97a6cbef
-size 9114736920

model-00031-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:1a95284a6578de16065728e91dd07464427f428b797020a2ef0dded4b9451b87
-size 9114736920

model-00032-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:a0b02a36f460ffaa06323cb767b88d8d5e743be1a060ac1d84e99bb6daec777e
-size 9114736920

model-00033-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:034a2dee202adf1cb268c2365202da57536f815d5cc26aa87cc83b6d6f2b83fb
-size 9114736920

model-00034-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:d2b58dcec2ffbe076c62237b4c94382dc32e57bbb31f6eca3c0605032384c976
-size 9114736920

model-00035-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:be9e39dda10bcb91633cccc93a83a6237ae1899b7745e07ec8f2504bfe096ca1
-size 9114736920

model-00036-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:806a960d868eb92fa5174580302afe63a7dfed806a0b7d6f9a1d618fb90594e9
-size 9114736920

model-00037-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:986674ec5294e8c569ea17753e1f3cba35b0350dfaa543b55b47a1b7aeaf69c8
-size 9114736920

model-00038-of-000064.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:689a00a1528f0f882ea887f836758e87cab18ec990c49485bd61e44419d5afcc
-size 9114736920