Add Alpamayo 2 Super release candidate

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,147 @@
+---
+license: openmdw-1.1
+---
+# Alpamayo 2 Super Overview
+
+## Description:
+
+Alpamayo 2 Super is a 34B-class foundation model designed to tackle multiple autonomous vehicle (AV) development tasks. It combines a 32B-class VLM backbone with a 2B-class diffusion expert. The release candidate contains 35,813,934,322 indexed parameters. The released inference path uses the trained VLM backbone to generate Chain-of-Causation text, then samples future trajectories through the trained action expert.
+Alpamayo 2 Super was developed by NVIDIA as a part of Alpamayo.
+
+### License/Terms of Use:
+
+**Source code license:** Apache License 2.0, as provided in the Alpamayo 2 source repository.
+
+**Governing Terms:** Use of the model is governed by the [OpenMDW-1.1](https://openmdw.ai/license/1-1/) license.
+
+### Deployment Geography:
+
+Global
+
+### Use Case:
+
+Developers and researchers working on autonomous vehicle systems who need a foundation model for perception, planning, and decision-making tasks. Alpamayo 2 Super supports multiple AV development tasks such as trajectory prediction, visual question answering, grounding, and auto-labeling. It enables enterprises to accelerate AV software development with a unified model that combines vision-language and diffusion expertise.
+
+### [Expected] Release Date:
+
+**Hugging Face:** To be determined, via https://huggingface.co/nvidia/Alpamayo2-Super
+
+## Reference(s):
+
+[Ethical Considerations](https://www.nvidia.com/en-us/support/submit-security-vulnerability/)
+
+## Model Architecture:
+
+**Architecture Type:** Transformer
+**Network Architecture:** 32B VLM backbone with 2B diffusion VLA model based on Cosmos-Reason and featuring a diffusion-based trajectory decoder.
+**This model was developed based on Cosmos-Reason (VLM backbone) with a diffusion-based action decoder.**
+**Number of model parameters:** 35,813,934,322 indexed parameters (34B-class)
+
+## Computational Load
+
+**Cumulative Compute:** Not Applicable (N/A)
+**Estimated Energy and Emissions for Model Training:** Not sure
+
+## Input:
+
+**Input Type(s):** Text, Image
+**Input Format(s):**
+
+- Image: Red, Green, Blue (RGB)
+- Text: String
+- Trajectory: Floating-point values (position, velocity, acceleration, steering)
+
+**Input Parameters:** One-Dimensional (1D), Two-Dimensional (2D)
+**Other Properties Related to Input:** The validated public notebook profiles use six cameras and four historical frames per camera. VQA uses camera IDs `[0, 1, 2, 3, 4, 5]` (cross left, front wide, cross right, rear left, rear tele, and rear right). Trajectory, meta-action, auto-labeling, and grounding use camera IDs `[0, 1, 2, 3, 5, 6]` (cross left, front wide, cross right, rear left, rear right, and front tele). The four frames are synchronized context frames ending near `t0`; their exact timestamps come from the input sample. Images and ego-motion history require associated timestamps. Image resizing and normalization are performed by the packaged processor rather than by a fixed public raw-image resolution contract.
+
+## Output:
+
+**Output Type(s):** Text
+**Output Format:** Other: Text,Trajectory
+**Output Parameters:** One-Dimensional (1D), Two-Dimensional (2D)
+**Other Properties Related to Output:** The trajectory API returns 64 waypoints spanning 0.1 through 6.4 seconds at 0.1-second intervals. Each trajectory contains ego-frame XYZ positions and 3x3 rotation matrices. Text outputs include variable-length Chain-of-Causation reasoning, meta-actions, VQA answers, grounding coordinates, and structured auto-labeling fields.
+
+Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
+
+## Software Integration:
+
+**Runtime Engine(s):** Not Applicable (N/A)
+**Supported Hardware Microarchitecture Compatibility:**
+
+- Tested: NVIDIA H100 80GB HBM3
+- Other GPU architectures have not yet been validated for this release candidate.
+
+**Supported Operating System(s):** Linux
+
+The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
+
+(Optional) This AI model can be embedded as an Application Programming Interface (API) call into the software environment described above.
+
+## Model Version(s):
+
+Alpamayo 2 Super
+
+Integrate Alpamayo 2 Super by setting up a Python environment with uv, installing the locked dependencies, then authenticating to Hugging Face. Use the provided CLI or Jupyter notebooks to load the model via `Alpamayo2Super.from_pretrained` and run inference on validation samples. The verified single-GPU release run used BF16 and PyTorch SDPA on an NVIDIA H100 80GB GPU. Although `flash-attn` is an installation dependency, it was not the active attention backend in that measured run.
+
+## Training, Testing, and Evaluation Datasets:
+
+## Training Dataset:
+
+**Data Modality:**
+
+- Image
+- Text
+- Video
+
+**Image Training Data Size:** More than 1 Billion Images
+**Text Training Data Size:** Less than a Billion Tokens
+**Video Training Data Size:** 10,000 to 1 Million Hours
+**Data Collection Method by dataset:** Automatic/Sensors
+**Labeling Method by dataset:** Hybrid: Automated/Manually-Labelled
+**Properties (Quantity, Dataset Descriptions, Sensor(s)):** The dataset comprises 80,000 hours of multi-camera driving video with corresponding egomotion and trajectory annotations.
+It includes 700,000 Chain-of-Causation (CoC) reasoning traces that provide decision-grounded, causally linked explanations of driving behaviors.
+Content includes machine-generated data from vehicle sensors (cameras, IMUs, and GPS) and synthetic reasoning traces.
+CoC annotations are in English and use a structured format that links driving decisions to causal factors.
+Sensors include RGB cameras (2-6 per vehicle), inertial measurement units, and GPS.
+
+### Testing Dataset:
+
+**Data Collection Method by dataset:** Automatic/Sensors
+**Labeling Method by dataset:** Hybrid: Automatic/Sensors/Manually-Labelled
+**Properties (Quantity, Dataset Descriptions, Sensor(s)):** The dataset comprises 80,000 hours of multi-camera driving video with corresponding egomotion and trajectory annotations.
+It includes 700,000 Chain-of-Causation (CoC) reasoning traces that provide decision-grounded, causally linked explanations of driving behaviors.
+Content includes machine-generated data from vehicle sensors (cameras, IMUs, and GPS) and synthetic reasoning traces.
+CoC annotations are in English and use a structured format that links driving decisions to causal factors.
+Sensors include RGB cameras (2-6 per vehicle), inertial measurement units, and GPS.
+
+### Evaluation Dataset:
+
+**Benchmark Score:** None
+
+| Benchmark | Metric | Score |
+| --------- | ------ | ----- |
+| minADE    | 1      |       |
+
+**Data Collection Method by dataset:** Automatic/Sensors
+**Labeling Method by dataset:** Hybrid: Automatic/Sensors/Manually-Labelled
+**Properties (Quantity, Dataset Descriptions, Sensor(s)):** This dataset covers multi-camera driving scenarios with a particular focus on safety-critical, long-tail events. It includes challenging cases such as complex intersections, cut-ins, pedestrian interactions, and adverse weather conditions. Data are collected from RGB cameras and vehicle sensors.
+
+## Inference:
+
+**Acceleration Engine:** Other: PyTorch, Hugging Face Transformers
+**Hardware Requirements (GPU Architecture, Model):**
+
+- Tested on: 1 NVIDIA H100 80GB HBM3 GPU
+- Measured configuration: seven cameras, four frames per camera, batch size 1, one trajectory sample, BF16, PyTorch SDPA, classifier-free guidance disabled, and 10 diffusion steps.
+- Measured peak: 72,115 MiB device memory, with 69.316 GiB peak PyTorch allocation and 69.463 GiB peak PyTorch reservation.
+- The six-camera public notebook profiles have not yet been separately memory-profiled. Memory use also varies with camera and frame counts, trajectory sample count, classifier-free guidance, attention backend, and device placement.
+- A lower minimum-memory GPU configuration has not been validated. A 24GB GPU should not be presented as supported for full-model inference.
+
+## Ethical Considerations:
+
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
+Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.
+For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards.
+Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
+
+**Generated by NVIDIA Model Card Generator Toolkit.**

added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

chat_template.jinja

@@ -0,0 +1,120 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' }}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- for message in messages %}
+    {%- if message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content_item in message.content %}
+                {%- if 'text' in content_item %}
+                    {{- content_item.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and message.content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,404 @@
+{
+  "architectures": [
+    "Alpamayo2Super"
+  ],
+  "cotrain_expert_vlm": false,
+  "dtype": "bfloat16",
+  "enable_expert": true,
+  "encode_as_video": false,
+  "expert_config": {
+    "action_in_proj_cfg": {
+      "_target_": "alpamayo2_super.models.action_in_proj.PerWaypointActionInProjV2",
+      "hidden_size": 512,
+      "max_freq": 100.0,
+      "num_enc_layers": 2,
+      "num_fourier_feats": 20
+    },
+    "action_out_proj_cfg": {
+      "_target_": "torch.nn.Linear"
+    },
+    "action_space_cfg": {
+      "_target_": "alpamayo2_super.action_space.UnicycleAccelCurvatureActionSpace",
+      "a_lambda": 0.0001,
+      "a_ridge": 0.0001,
+      "accel_bounds": [
+        -9.8,
+        9.8
+      ],
+      "accel_mean": 0.02902694707164455,
+      "accel_std": 0.6810426736454882,
+      "curvature_bounds": [
+        -0.33,
+        0.33
+      ],
+      "curvature_mean": 0.0002692167976330542,
+      "curvature_std": 0.026148280660833106,
+      "dt": 0.1,
+      "kappa_lambda": 0.0001,
+      "kappa_ridge": 0.0001,
+      "n_waypoints": 64,
+      "theta_lambda": 1e-06,
+      "theta_ridge": 1e-08,
+      "v_lambda": 1e-06,
+      "v_ridge": 0.0001
+    },
+    "cache_layer_indices": null,
+    "diffusion_cfg": {
+      "_target_": "alpamayo2_super.diffusion.flow_matching.FlowMatching",
+      "inference_guidance_weight": 3.0,
+      "int_method": "euler",
+      "train_ignore_guidance_rate": 0.1,
+      "train_timestep_sampler": "beta",
+      "use_classifier_free_guidance": false,
+      "x_dims": null
+    },
+    "expert_non_causal_attention": true,
+    "expert_update_cfg": {
+      "head_dim": 128,
+      "hidden_size": 1536,
+      "intermediate_size": 6144,
+      "num_attention_heads": 16
+    },
+    "llm_config": {
+      "_name_or_path": "",
+      "add_cross_attention": false,
+      "architectures": null,
+      "attention_bias": false,
+      "attention_dropout": 0.0,
+      "bad_words_ids": null,
+      "begin_suppress_tokens": null,
+      "bos_token_id": 151643,
+      "chunk_size_feed_forward": 0,
+      "cross_attention_hidden_size": null,
+      "decoder_start_token_id": null,
+      "diversity_penalty": 0.0,
+      "do_sample": false,
+      "dtype": "bfloat16",
+      "early_stopping": false,
+      "encoder_no_repeat_ngram_size": 0,
+      "eos_token_id": 151645,
+      "exponential_decay_length_penalty": null,
+      "finetuning_task": null,
+      "forced_bos_token_id": null,
+      "forced_eos_token_id": null,
+      "head_dim": 128,
+      "hidden_act": "silu",
+      "hidden_size": 1536,
+      "id2label": {
+        "0": "LABEL_0",
+        "1": "LABEL_1"
+      },
+      "initializer_range": 0.02,
+      "intermediate_size": 6144,
+      "is_decoder": false,
+      "is_encoder_decoder": false,
+      "label2id": {
+        "LABEL_0": 0,
+        "LABEL_1": 1
+      },
+      "length_penalty": 1.0,
+      "max_length": 20,
+      "max_position_embeddings": 262144,
+      "min_length": 0,
+      "model_type": "qwen3_vl_text",
+      "no_repeat_ngram_size": 0,
+      "num_attention_heads": 16,
+      "num_beam_groups": 1,
+      "num_beams": 1,
+      "num_hidden_layers": 64,
+      "num_key_value_heads": 8,
+      "num_return_sequences": 1,
+      "output_attentions": false,
+      "output_hidden_states": false,
+      "output_scores": false,
+      "pad_token_id": null,
+      "prefix": null,
+      "problem_type": null,
+      "pruned_heads": {},
+      "remove_invalid_values": false,
+      "repetition_penalty": 1.0,
+      "return_dict": true,
+      "return_dict_in_generate": false,
+      "rms_norm_eps": 1e-06,
+      "rope_scaling": {
+        "mrope_interleaved": true,
+        "mrope_section": [
+          24,
+          20,
+          20
+        ],
+        "rope_type": "default"
+      },
+      "rope_theta": 5000000,
+      "sep_token_id": null,
+      "suppress_tokens": null,
+      "task_specific_params": null,
+      "temperature": 1.0,
+      "tf_legacy_loss": false,
+      "tie_encoder_decoder": false,
+      "tie_word_embeddings": false,
+      "tokenizer_class": null,
+      "top_k": 50,
+      "top_p": 1.0,
+      "torchscript": false,
+      "typical_p": 1.0,
+      "use_bfloat16": false,
+      "use_cache": true,
+      "vocab_size": 155776
+    },
+    "model_type": "alpamayo2_super_expert"
+  },
+  "frame_label": "frame_num",
+  "future_traj_tokenizer_cfg": {
+    "_recursive_": false,
+    "_target_": "alpamayo2_super.action_space.discrete_action_space.DiscreteTrajectoryTokenizer",
+    "action_space_cfg": {
+      "_target_": "alpamayo2_super.action_space.UnicycleAccelCurvatureActionSpace",
+      "a_lambda": 0.0001,
+      "a_ridge": 0.0001,
+      "accel_bounds": [
+        -9.8,
+        9.8
+      ],
+      "accel_mean": 0.02902694707164455,
+      "accel_std": 0.6810426736454882,
+      "curvature_bounds": [
+        -0.33,
+        0.33
+      ],
+      "curvature_mean": 0.0002692167976330542,
+      "curvature_std": 0.026148280660833106,
+      "dt": 0.1,
+      "kappa_lambda": 0.0001,
+      "kappa_ridge": 0.0001,
+      "n_waypoints": 64,
+      "theta_lambda": 1e-06,
+      "theta_ridge": 1e-08,
+      "v_lambda": 1e-06,
+      "v_ridge": 0.0001
+    },
+    "dims_max": [
+      10,
+      10
+    ],
+    "dims_min": [
+      -10,
+      -10
+    ],
+    "num_bins": 3000
+  },
+  "future_vocab_size": 3000,
+  "hist_traj_tokenizer_cfg": {
+    "_target_": "alpamayo2_super.models.delta_tokenizer.DeltaTrajectoryTokenizer",
+    "pad_origin_at_beginning": false
+  },
+  "history_vocab_size": 1000,
+  "include_camera_ids": true,
+  "loss_weights": {
+    "future_traj": 1.0,
+    "others": 1.0
+  },
+  "max_pixels": 196608,
+  "min_pixels": 163840,
+  "model_type": "alpamayo2_super",
+  "padding_side": "left",
+  "token_layout": "camera_ts",
+  "tokens_per_future_traj": 128,
+  "tokens_per_history_traj": 45,
+  "traj_ids": {
+    "future_end": 155683,
+    "future_id0": 152669,
+    "future_pad": 155685,
+    "future_start": 155681,
+    "history_end": 155676,
+    "history_id0": 151669,
+    "history_pad": 155684,
+    "history_start": 155674
+  },
+  "traj_vocab_size": 4000,
+  "transformers_version": "4.57.6",
+  "video_max_pixels": null,
+  "video_min_pixels": null,
+  "video_pixels_per_frame": false,
+  "vlm_class": "Qwen3VLForConditionalGeneration",
+  "vlm_config": {
+    "_name_or_path": "",
+    "architectures": [
+      "Qwen3VLForConditionalGeneration"
+    ],
+    "dtype": "bfloat16",
+    "image_token_id": 151655,
+    "model_type": "qwen3_vl",
+    "text_config": {
+      "_name_or_path": "",
+      "add_cross_attention": false,
+      "architectures": null,
+      "attention_bias": false,
+      "attention_dropout": 0.0,
+      "bad_words_ids": null,
+      "begin_suppress_tokens": null,
+      "bos_token_id": 151643,
+      "chunk_size_feed_forward": 0,
+      "cross_attention_hidden_size": null,
+      "decoder_start_token_id": null,
+      "diversity_penalty": 0.0,
+      "do_sample": false,
+      "dtype": "bfloat16",
+      "early_stopping": false,
+      "encoder_no_repeat_ngram_size": 0,
+      "eos_token_id": 151645,
+      "exponential_decay_length_penalty": null,
+      "finetuning_task": null,
+      "forced_bos_token_id": null,
+      "forced_eos_token_id": null,
+      "head_dim": 128,
+      "hidden_act": "silu",
+      "hidden_size": 5120,
+      "id2label": {
+        "0": "LABEL_0",
+        "1": "LABEL_1"
+      },
+      "initializer_range": 0.02,
+      "intermediate_size": 25600,
+      "is_decoder": false,
+      "is_encoder_decoder": false,
+      "label2id": {
+        "LABEL_0": 0,
+        "LABEL_1": 1
+      },
+      "length_penalty": 1.0,
+      "max_length": 20,
+      "max_position_embeddings": 262144,
+      "min_length": 0,
+      "model_type": "qwen3_vl_text",
+      "no_repeat_ngram_size": 0,
+      "num_attention_heads": 64,
+      "num_beam_groups": 1,
+      "num_beams": 1,
+      "num_hidden_layers": 64,
+      "num_key_value_heads": 8,
+      "num_return_sequences": 1,
+      "output_attentions": false,
+      "output_hidden_states": false,
+      "output_scores": false,
+      "pad_token_id": null,
+      "prefix": null,
+      "problem_type": null,
+      "pruned_heads": {},
+      "remove_invalid_values": false,
+      "repetition_penalty": 1.0,
+      "return_dict": true,
+      "return_dict_in_generate": false,
+      "rms_norm_eps": 1e-06,
+      "rope_scaling": {
+        "mrope_interleaved": true,
+        "mrope_section": [
+          24,
+          20,
+          20
+        ],
+        "rope_type": "default"
+      },
+      "rope_theta": 5000000,
+      "sep_token_id": null,
+      "suppress_tokens": null,
+      "task_specific_params": null,
+      "temperature": 1.0,
+      "tf_legacy_loss": false,
+      "tie_encoder_decoder": false,
+      "tie_word_embeddings": false,
+      "tokenizer_class": null,
+      "top_k": 50,
+      "top_p": 1.0,
+      "torchscript": false,
+      "typical_p": 1.0,
+      "use_bfloat16": false,
+      "use_cache": true,
+      "vocab_size": 155776
+    },
+    "video_token_id": 151656,
+    "vision_config": {
+      "_name_or_path": "",
+      "add_cross_attention": false,
+      "architectures": null,
+      "bad_words_ids": null,
+      "begin_suppress_tokens": null,
+      "bos_token_id": null,
+      "chunk_size_feed_forward": 0,
+      "cross_attention_hidden_size": null,
+      "decoder_start_token_id": null,
+      "deepstack_visual_indexes": [
+        8,
+        16,
+        24
+      ],
+      "depth": 27,
+      "diversity_penalty": 0.0,
+      "do_sample": false,
+      "dtype": "bfloat16",
+      "early_stopping": false,
+      "encoder_no_repeat_ngram_size": 0,
+      "eos_token_id": null,
+      "exponential_decay_length_penalty": null,
+      "finetuning_task": null,
+      "forced_bos_token_id": null,
+      "forced_eos_token_id": null,
+      "hidden_act": "gelu_pytorch_tanh",
+      "hidden_size": 1152,
+      "id2label": {
+        "0": "LABEL_0",
+        "1": "LABEL_1"
+      },
+      "in_channels": 3,
+      "initializer_range": 0.02,
+      "intermediate_size": 4304,
+      "is_decoder": false,
+      "is_encoder_decoder": false,
+      "label2id": {
+        "LABEL_0": 0,
+        "LABEL_1": 1
+      },
+      "length_penalty": 1.0,
+      "max_length": 20,
+      "min_length": 0,
+      "model_type": "qwen3_vl",
+      "no_repeat_ngram_size": 0,
+      "num_beam_groups": 1,
+      "num_beams": 1,
+      "num_heads": 16,
+      "num_position_embeddings": 2304,
+      "num_return_sequences": 1,
+      "out_hidden_size": 5120,
+      "output_attentions": false,
+      "output_hidden_states": false,
+      "output_scores": false,
+      "pad_token_id": null,
+      "patch_size": 16,
+      "prefix": null,
+      "problem_type": null,
+      "pruned_heads": {},
+      "remove_invalid_values": false,
+      "repetition_penalty": 1.0,
+      "return_dict": true,
+      "return_dict_in_generate": false,
+      "sep_token_id": null,
+      "spatial_merge_size": 2,
+      "suppress_tokens": null,
+      "task_specific_params": null,
+      "temperature": 1.0,
+      "temporal_patch_size": 2,
+      "tf_legacy_loss": false,
+      "tie_encoder_decoder": false,
+      "tie_word_embeddings": true,
+      "tokenizer_class": null,
+      "top_k": 50,
+      "top_p": 1.0,
+      "torchscript": false,
+      "typical_p": 1.0,
+      "use_bfloat16": false
+    },
+    "vision_end_token_id": 151653,
+    "vision_start_token_id": 151652
+  },
+  "vlm_name_or_path": ""
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "4.57.6"
+}

model-00001-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:794ffa42440f4998735bf7d8c53c30a4da3b57d9bc3bae88c68a457bc32384ba
+size 4924862728

model-00002-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:faa303461d97a98a54e4d92b842ec149c00524830a13018f89fe7c1146cda31a
+size 4875990736

model-00003-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:11ed2a8407379103f3ebed1cffb5f12512bc17f9f9119ddb02f296535dd874e3
+size 4875990752

model-00004-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:695a122dc6c167bc878f67c391e9b5304f42c29f2bada93b0333bc8c1c0789e7
+size 4875990792

model-00005-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67cd73482cb2a4b41e1d8c58bf89cf73490f11fed3ca683e7d6ec1670cf25d08
+size 4875990792

model-00006-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9ef4f0355604b7772782fbd98c85055a01507a178dfeacdfea2e5e527262206
+size 4875990792

model-00007-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a0f639e3a4ab3b1be6c4a010669829cce1ca077d29e05fdfe33fa441385792e6
+size 4875990792

model-00008-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73a8e3b12a0ec00975219272a60b3c78915490aa0b055ca295baf924da29c6c1
+size 4875990792

model-00009-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dbe04e5d540b6ac79f8939844510639f787fdb7c7dc331e95217fb19bab64b9a
+size 4875990792

model-00010-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:31c6920d91a7ff154153fe587b2d127e4831066bedf480151faff9bbaf8ef3b3
+size 4875990792

model-00011-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74a34bb7a2f2efe37dead56807ee3252710fff4b14a4ee8564b8ab01a6932d52
+size 4875990792

model-00012-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db9afcec2566fa2ce881bbeaddb05d4a135cd8fd038f5af71bdad46e934b393b
+size 4875990792

model-00013-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:831a7a5cdf85cdb1c49bcbc187da86e8cd768868f0e4d62701edac4a79f7c53b
+size 4875990792

model-00014-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86e4f9e3a720dc0829bd8e4fe6b4ae1d8de80a5248bb1e441a8b0332597b462a
+size 4999047456

model-00015-of-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc9423f8a6308fb94687ec89d411a7bf55ddf28a1b59deac77ae1a27cef6d52f
+size 3192293580

preprocessor_config.json

@@ -0,0 +1,39 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": true,
+  "device": null,
+  "disable_grouping": null,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_pad": null,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "Qwen2VLImageProcessorFast",
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "input_data_format": null,
+  "max_pixels": null,
+  "merge_size": 2,
+  "min_pixels": null,
+  "pad_size": null,
+  "patch_size": 16,
+  "processor_class": "Qwen3VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "return_tensors": null,
+  "size": {
+    "longest_edge": 16777216,
+    "shortest_edge": 65536
+  },
+  "temporal_patch_size": 2
+}

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4
+size 11422654

tokenizer_config.json

@@ -0,0 +1,240 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 262144,
+  "pad_token": "<|endoftext|>",
+  "processor_class": "Qwen3VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

video_preprocessor_config.json

@@ -0,0 +1,41 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": true,
+  "device": null,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "do_sample_frames": true,
+  "fps": 2,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "input_data_format": null,
+  "max_frames": 768,
+  "merge_size": 2,
+  "min_frames": 4,
+  "num_frames": null,
+  "pad_size": null,
+  "patch_size": 16,
+  "processor_class": "Qwen3VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "return_metadata": false,
+  "size": {
+    "longest_edge": 25165824,
+    "shortest_edge": 4096
+  },
+  "temporal_patch_size": 2,
+  "video_metadata": null,
+  "video_processor_type": "Qwen3VLVideoProcessor"
+}