Add files using upload-large-folder tool

README.md

@@ -0,0 +1,179 @@
+---
+library_name: easydel
+pipeline_tag: image-to-text
+tags:
+  - easydel
+  - jax
+  - "glm4v"
+  - "ImageTextToText"
+  - "vanilla"
+---
+
+<p align="center">
+  <a href="https://github.com/erfanzar/EasyDeL">
+    <img src="https://raw.githubusercontent.com/erfanzar/easydel/main/images/easydel-logo-with-text.png" height="80" alt="EasyDeL" />
+  </a>
+</p>
+
+<h1 align="center">zai-org/GLM-4.6V-Flash</h1>
+
+<p align="center">
+  EasyDeL checkpoint converted from zai-org/GLM-4.6V-Flash.
+</p>
+
+<p align="center">
+  <a href="https://huggingface.co/EasyDeL/GLM-4.6V-Flash">
+    <img src="https://img.shields.io/badge?label=HF&message=EasyDeL/GLM-4.6V-Flash&color=FFD21E&style=flat-square" alt="HuggingFace Repo" />
+  </a>
+  <a href="https://github.com/erfanzar/EasyDeL">
+    <img src="https://img.shields.io/badge?label=EasyDeL&message=v0.2.0&color=0B5FFF&style=flat-square" alt="EasyDeL Version" />
+  </a>
+  <img src="https://img.shields.io/badge?label=Model&message=glm4v&color=0A66C2&style=flat-square" alt="Model Type" />
+  <img src="https://img.shields.io/badge?label=Task&message=ImageTextToText&color=2EAD4D&style=flat-square" alt="Task" />
+  <img src="https://img.shields.io/badge?label=Attention&message=vanilla&color=8A2BE2&style=flat-square" alt="Attention Mechanism" />
+</p>
+
+---
+
+## At a Glance
+
+| Field | Value |
+| --- | --- |
+| Repo ID | `EasyDeL/GLM-4.6V-Flash` |
+| Model type | `glm4v` |
+| Task | `ImageTextToText` |
+| Attention | `vanilla` (`AttentionMechanisms.VANILLA`) |
+| EasyDeL | `0.2.0` |
+
+## Overview
+
+This checkpoint is intended to be loaded with EasyDeL on JAX (CPU/GPU/TPU). It supports sharded loading with `auto_shard_model=True` and configurable precision via `dtype`, `param_dtype`, and `precision`.
+
+## Quickstart
+
+```python
+import easydel as ed
+from jax import numpy as jnp, lax
+
+repo_id = "EasyDeL/GLM-4.6V-Flash"
+
+dtype = jnp.bfloat16  # try jnp.float16 on many GPUs
+
+model = ed.AutoEasyDeLModelForImageTextToText.from_pretrained(
+    repo_id,
+    config_kwargs=ed.EasyDeLBaseConfigDict(
+        attn_dtype=dtype,
+        attn_mechanism=ed.AttentionMechanisms.VANILLA,
+    ),
+    dtype=dtype,
+    param_dtype=dtype,
+    precision=lax.Precision("fastest"),
+    auto_shard_model=True,
+)
+```
+
+If the repository only provides PyTorch weights, pass `from_torch=True` to `from_pretrained(...)`.
+
+## Sharding & Parallelism (Multi-Device)
+
+EasyDeL can scale to multiple devices by creating a logical device mesh. Most EasyDeL loaders use a 5D mesh:
+
+- `dp`: data parallel (replicated parameters, different batch shards)
+- `fsdp`: parameter sharding (memory saver; often the biggest axis)
+- `ep`: expert parallel (MoE; keep `1` for non-MoE models)
+- `tp`: tensor parallel (splits large matmuls)
+- `sp`: sequence parallel (splits sequence dimension)
+
+Use `sharding_axis_names=("dp","fsdp","ep","tp","sp")` and choose `sharding_axis_dims` so that their product equals your device count.
+You can use `-1` in `sharding_axis_dims` to let EasyDeL infer the remaining dimension.
+
+<details>
+<summary>Example sharding configs</summary>
+
+```python
+# 8 devices, pure FSDP
+sharding_axis_dims = (1, 8, 1, 1, 1)
+
+# 8 devices, 2-way DP x 4-way FSDP
+sharding_axis_dims = (2, 4, 1, 1, 1)
+
+# 8 devices, 4-way FSDP x 2-way TP
+sharding_axis_dims = (1, 4, 1, 2, 1)
+```
+</details>
+
+## Using via `eLargeModel` (ELM)
+
+`eLargeModel` is a higher-level interface that wires together loading, sharding, training, and eSurge inference from a single config.
+
+```python
+from easydel import eLargeModel
+
+repo_id = "EasyDeL/GLM-4.6V-Flash"
+
+elm = eLargeModel.from_pretrained(repo_id)  # task is auto-detected
+elm.set_dtype("bf16")
+elm.set_sharding(axis_names=("dp", "fsdp", "ep", "tp", "sp"), axis_dims=(1, -1, 1, 1, 1))
+
+model = elm.build_model()
+# Optional: build an inference engine
+# engine = elm.build_esurge()
+```
+
+<details>
+<summary>ELM YAML config example</summary>
+
+```yaml
+model:
+  name_or_path: "EasyDeL/GLM-4.6V-Flash"
+
+loader:
+  dtype: bf16
+  param_dtype: bf16
+
+sharding:
+  axis_dims: [1, -1, 1, 1, 1]
+  auto_shard_model: true
+```
+</details>
+
+## Features
+
+**EasyDeL:**
+- JAX native implementation and sharded execution
+- Configurable attention backends via `AttentionMechanisms.*`
+- Precision control via `dtype`, `param_dtype`, and `precision`
+
+## Installation
+
+```bash
+pip install easydel
+```
+
+## Links
+
+- EasyDeL GitHub: https://github.com/erfanzar/EasyDeL
+- Docs: https://easydel.readthedocs.io/en/latest/
+
+## Supported Tasks
+
+- ImageTextToText
+
+## Limitations
+
+- Refer to the original model card for training data, evaluation, and intended use.
+
+## License
+
+EasyDeL is released under the Apache-2.0 license. The license for this model's weights may differ; please consult the original repository.
+
+## Citation
+
+```bibtex
+@misc{Zare Chavoshi_2023,
+    title={EasyDeL: An open-source library for enhancing and streamlining the training process of machine learning models},
+    url={https://github.com/erfanzar/EasyDeL},
+    author={Zare Chavoshi, Erfan},
+    year={2023}
+}
+```

checkpoint_metadata.json

@@ -0,0 +1,6 @@
+{
+  "timestamp": "2025-12-28T04:14:13.950976",
+  "custom_metadata": {
+    "step": 0
+  }
+}

config.json

@@ -0,0 +1,349 @@
+{
+  "_external_rope_config_kwargs": {},
+  "architectures": [
+    "Glm4vForConditionalGeneration"
+  ],
+  "attn_mechanism": "vanilla",
+  "backend": null,
+  "bits": null,
+  "blocksize_b": 1,
+  "blocksize_k": 128,
+  "blocksize_q": 128,
+  "decode_attn_mechanism": null,
+  "easy_method": "train",
+  "fcm_max_ratio": 0.0,
+  "fcm_min_ratio": 0.0,
+  "flash_attention_backward_pass_impl": "triton",
+  "fsdp_is_ep_bound": true,
+  "gradient_checkpointing": "",
+  "gradient_checkpointing_targets": null,
+  "hardware_abstraction": false,
+  "image_end_token_id": 151340,
+  "image_start_token_id": 151339,
+  "image_token_id": 151363,
+  "kv_cache_quantization_config": null,
+  "kv_cache_sharding_sequence_axis_name": "sp",
+  "model_type": "glm4v",
+  "moe_force_xla_gmm": false,
+  "moe_method": "fused_moe",
+  "moe_tiling_size_batch": 4,
+  "moe_tiling_size_dim": 128,
+  "moe_tiling_size_seqlen": 128,
+  "operation_configs": null,
+  "pallas_k_block_size": 128,
+  "pallas_m_block_size": 128,
+  "pallas_n_block_size": 128,
+  "partition_axis": {
+    "attention_dim_axis": null,
+    "attention_kv_dim_axis": null,
+    "batch_axis": [
+      "fsdp",
+      "dp"
+    ],
+    "bias_head_sequence_axis": null,
+    "bias_key_sequence_axis": null,
+    "data_parallel_axis": "dp",
+    "decode_attention_dim_axis": null,
+    "decode_attention_kv_dim_axis": null,
+    "decode_batch_axis": [
+      "fsdp",
+      "dp"
+    ],
+    "decode_head_axis": "tp",
+    "decode_key_sequence_axis": "sp",
+    "decode_kv_head_axis": "tp",
+    "decode_query_sequence_axis": null,
+    "expert_axis": "ep",
+    "expert_gate_axis": null,
+    "expert_parallel_axis": "ep",
+    "fully_sharded_data_parallel_axis": "fsdp",
+    "head_axis": "tp",
+    "hidden_state_axis": "tp",
+    "key_sequence_axis": "sp",
+    "kv_head_axis": "tp",
+    "mlp_intermediate_axis": "tp",
+    "query_sequence_axis": "sp",
+    "sequence_axis": "sp",
+    "sequence_parallel_axis": "sp",
+    "tensor_parallel_axis": "tp",
+    "vocab_axis": "tp"
+  },
+  "platform": null,
+  "precompute_masks": true,
+  "pretraining_tp": 1,
+  "quantization_config": null,
+  "scan_attention_layers": false,
+  "scan_mlp_chunk_size": 1024,
+  "scan_ring_attention": true,
+  "sequence_axis_name": "sp",
+  "sharding_axis_dims": [
+    1,
+    -1,
+    1,
+    1,
+    1
+  ],
+  "sharding_axis_names": [
+    "dp",
+    "fsdp",
+    "ep",
+    "tp",
+    "sp"
+  ],
+  "sharding_dcn_axis_dims": null,
+  "sp_is_ep_bound": true,
+  "text_config": {
+    "_external_rope_config_kwargs": {
+      "repetition_style": true
+    },
+    "architectures": [
+      "Glm4vForConditionalGeneration"
+    ],
+    "attention_bias": true,
+    "attention_dropout": 0.0,
+    "attn_mechanism": "vanilla",
+    "backend": null,
+    "bits": null,
+    "blocksize_b": 1,
+    "blocksize_k": 128,
+    "blocksize_q": 128,
+    "decode_attn_mechanism": null,
+    "dtype": "bfloat16",
+    "easy_method": "train",
+    "eos_token_id": [
+      151329,
+      151336,
+      151338
+    ],
+    "fcm_max_ratio": 0.0,
+    "fcm_min_ratio": 0.0,
+    "flash_attention_backward_pass_impl": "triton",
+    "fsdp_is_ep_bound": true,
+    "gradient_checkpointing": "",
+    "gradient_checkpointing_targets": null,
+    "hardware_abstraction": false,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 4096,
+    "initializer_range": 0.02,
+    "intermediate_size": 13696,
+    "kv_cache_quantization_config": null,
+    "kv_cache_sharding_sequence_axis_name": "sp",
+    "max_position_embeddings": 131072,
+    "model_type": "glm4v_text",
+    "moe_force_xla_gmm": false,
+    "moe_method": "fused_moe",
+    "moe_tiling_size_batch": 4,
+    "moe_tiling_size_dim": 128,
+    "moe_tiling_size_seqlen": 128,
+    "num_attention_heads": 32,
+    "num_hidden_layers": 40,
+    "num_key_value_heads": 2,
+    "operation_configs": null,
+    "pad_token_id": 151329,
+    "pallas_k_block_size": 128,
+    "pallas_m_block_size": 128,
+    "pallas_n_block_size": 128,
+    "partial_rotary_factor": 0.5,
+    "partition_axis": {
+      "attention_dim_axis": null,
+      "attention_kv_dim_axis": null,
+      "batch_axis": [
+        "fsdp",
+        "dp"
+      ],
+      "bias_head_sequence_axis": null,
+      "bias_key_sequence_axis": null,
+      "data_parallel_axis": "dp",
+      "decode_attention_dim_axis": null,
+      "decode_attention_kv_dim_axis": null,
+      "decode_batch_axis": [
+        "fsdp",
+        "dp"
+      ],
+      "decode_head_axis": "tp",
+      "decode_key_sequence_axis": "sp",
+      "decode_kv_head_axis": "tp",
+      "decode_query_sequence_axis": null,
+      "expert_axis": "ep",
+      "expert_gate_axis": null,
+      "expert_parallel_axis": "ep",
+      "fully_sharded_data_parallel_axis": "fsdp",
+      "head_axis": "tp",
+      "hidden_state_axis": "tp",
+      "key_sequence_axis": "sp",
+      "kv_head_axis": "tp",
+      "mlp_intermediate_axis": "tp",
+      "query_sequence_axis": "sp",
+      "sequence_axis": "sp",
+      "sequence_parallel_axis": "sp",
+      "tensor_parallel_axis": "tp",
+      "vocab_axis": "tp"
+    },
+    "platform": null,
+    "precompute_masks": true,
+    "pretraining_tp": 1,
+    "quantization_config": null,
+    "rms_norm_eps": 1e-05,
+    "rope_scaling": {
+      "mrope_section": [
+        8,
+        12,
+        12
+      ],
+      "rope_type": "default"
+    },
+    "rope_theta": 500000.0,
+    "scan_attention_layers": false,
+    "scan_mlp_chunk_size": 1024,
+    "scan_ring_attention": true,
+    "sequence_axis_name": "sp",
+    "sharding_axis_dims": [
+      1,
+      -1,
+      1,
+      1,
+      1
+    ],
+    "sharding_axis_names": [
+      "dp",
+      "fsdp",
+      "ep",
+      "tp",
+      "sp"
+    ],
+    "sharding_dcn_axis_dims": null,
+    "sp_is_ep_bound": true,
+    "use_cache": true,
+    "use_expert_tensor_mode": false,
+    "use_ring_of_experts": false,
+    "use_scan_mlp": false,
+    "use_sharded_kv_caching": false,
+    "use_sharding_constraint": false,
+    "vocab_size": 151552
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.3",
+  "use_expert_tensor_mode": false,
+  "use_ring_of_experts": false,
+  "use_scan_mlp": false,
+  "use_sharded_kv_caching": false,
+  "use_sharding_constraint": false,
+  "video_end_token_id": 151342,
+  "video_start_token_id": 151341,
+  "video_token_id": 151364,
+  "vision_config": {
+    "_external_rope_config_kwargs": {},
+    "architectures": [
+      "Glm4vForConditionalGeneration"
+    ],
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_mechanism": "vanilla",
+    "backend": null,
+    "bits": null,
+    "blocksize_b": 1,
+    "blocksize_k": 128,
+    "blocksize_q": 128,
+    "decode_attn_mechanism": null,
+    "depth": 24,
+    "easy_method": "train",
+    "fcm_max_ratio": 0.0,
+    "fcm_min_ratio": 0.0,
+    "flash_attention_backward_pass_impl": "triton",
+    "fsdp_is_ep_bound": true,
+    "gradient_checkpointing": "",
+    "gradient_checkpointing_targets": null,
+    "hardware_abstraction": false,
+    "hidden_act": "silu",
+    "hidden_dropout_prob": 0.0,
+    "hidden_size": 1536,
+    "image_size": 336,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 13696,
+    "kv_cache_quantization_config": null,
+    "kv_cache_sharding_sequence_axis_name": "sp",
+    "model_type": "glm4v_vision",
+    "moe_force_xla_gmm": false,
+    "moe_method": "fused_moe",
+    "moe_tiling_size_batch": 4,
+    "moe_tiling_size_dim": 128,
+    "moe_tiling_size_seqlen": 128,
+    "num_attention_heads": 12,
+    "num_heads": 12,
+    "operation_configs": null,
+    "out_hidden_size": 4096,
+    "pallas_k_block_size": 128,
+    "pallas_m_block_size": 128,
+    "pallas_n_block_size": 128,
+    "partition_axis": {
+      "attention_dim_axis": null,
+      "attention_kv_dim_axis": null,
+      "batch_axis": [
+        "fsdp",
+        "dp"
+      ],
+      "bias_head_sequence_axis": null,
+      "bias_key_sequence_axis": null,
+      "data_parallel_axis": "dp",
+      "decode_attention_dim_axis": null,
+      "decode_attention_kv_dim_axis": null,
+      "decode_batch_axis": [
+        "fsdp",
+        "dp"
+      ],
+      "decode_head_axis": "tp",
+      "decode_key_sequence_axis": "sp",
+      "decode_kv_head_axis": "tp",
+      "decode_query_sequence_axis": null,
+      "expert_axis": "ep",
+      "expert_gate_axis": null,
+      "expert_parallel_axis": "ep",
+      "fully_sharded_data_parallel_axis": "fsdp",
+      "head_axis": "tp",
+      "hidden_state_axis": "tp",
+      "key_sequence_axis": "sp",
+      "kv_head_axis": "tp",
+      "mlp_intermediate_axis": "tp",
+      "query_sequence_axis": "sp",
+      "sequence_axis": "sp",
+      "sequence_parallel_axis": "sp",
+      "tensor_parallel_axis": "tp",
+      "vocab_axis": "tp"
+    },
+    "patch_size": 14,
+    "platform": null,
+    "precompute_masks": true,
+    "pretraining_tp": 1,
+    "quantization_config": null,
+    "rms_norm_eps": 1e-05,
+    "scan_attention_layers": false,
+    "scan_mlp_chunk_size": 1024,
+    "scan_ring_attention": true,
+    "sequence_axis_name": "sp",
+    "sharding_axis_dims": [
+      1,
+      -1,
+      1,
+      1,
+      1
+    ],
+    "sharding_axis_names": [
+      "dp",
+      "fsdp",
+      "ep",
+      "tp",
+      "sp"
+    ],
+    "sharding_dcn_axis_dims": null,
+    "sp_is_ep_bound": true,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2,
+    "use_expert_tensor_mode": false,
+    "use_ring_of_experts": false,
+    "use_scan_mlp": false,
+    "use_sharded_kv_caching": false,
+    "use_sharding_constraint": false
+  }
+}

generation_config.json

@@ -0,0 +1,15 @@
+{
+  "_from_model_config": true,
+  "do_sample": true,
+  "eos_token_id": [
+    151329,
+    151336,
+    151338,
+    151348
+  ],
+  "pad_token_id": 151329,
+  "temperature": 0.8,
+  "top_k": 2,
+  "top_p": 0.6,
+  "transformers_version": "4.57.3"
+}

model/params/model/language_model/layers/0/self_attn/o_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[4096,1024],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096,4096],"zarr_format":2}

model/params/model/language_model/layers/1/mlp/down_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[13696,1024],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[13696,4096],"zarr_format":2}

model/params/model/language_model/layers/10/input_layernorm/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096],"zarr_format":2}

model/params/model/language_model/layers/10/post_mlp_layernorm/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096],"zarr_format":2}

model/params/model/language_model/layers/10/self_attn/q_proj/bias/.zarray

@@ -0,0 +1 @@
+{"chunks":[4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096],"zarr_format":2}

model/params/model/language_model/layers/10/self_attn/q_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[1024,4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096,4096],"zarr_format":2}

model/params/model/language_model/layers/10/self_attn/v_proj/bias/.zarray

@@ -0,0 +1 @@
+{"chunks":[256],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[256],"zarr_format":2}

model/params/model/language_model/layers/10/self_attn/v_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[1024,256],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096,256],"zarr_format":2}

model/params/model/language_model/layers/11/input_layernorm/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096],"zarr_format":2}

model/params/model/language_model/layers/11/mlp/down_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[13696,1024],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[13696,4096],"zarr_format":2}

model/params/model/language_model/layers/11/mlp/gate_up_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[1024,27392],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096,27392],"zarr_format":2}

model/params/model/language_model/layers/11/post_attention_layernorm/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096],"zarr_format":2}

model/params/model/language_model/layers/11/post_mlp_layernorm/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096],"zarr_format":2}

model/params/model/language_model/layers/11/post_self_attn_layernorm/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096],"zarr_format":2}

model/params/model/language_model/layers/11/self_attn/k_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[1024,256],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096,256],"zarr_format":2}

model/params/model/visual/blocks/5/attn/proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[1536,384],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536,1536],"zarr_format":2}

model/params/model/visual/blocks/5/attn/qkv/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[384,4608],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536,4608],"zarr_format":2}

model/params/model/visual/blocks/5/mlp/gate_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[384,4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536,4096],"zarr_format":2}

model/params/model/visual/blocks/5/mlp/up_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[384,4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536,4096],"zarr_format":2}

model/params/model/visual/blocks/6/attn/proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[1536,384],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536,1536],"zarr_format":2}

model/params/model/visual/blocks/6/mlp/down_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[4096,384],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096,1536],"zarr_format":2}

model/params/model/visual/blocks/6/mlp/gate_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[384,4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536,4096],"zarr_format":2}

model/params/model/visual/blocks/6/mlp/up_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[384,4096],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536,4096],"zarr_format":2}

model/params/model/visual/blocks/6/norm1/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[1536],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536],"zarr_format":2}

model/params/model/visual/blocks/7/attn/qkv/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[384,4608],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536,4608],"zarr_format":2}

model/params/model/visual/blocks/7/norm1/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[1536],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536],"zarr_format":2}

model/params/model/visual/blocks/8/attn/qkv/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[384,4608],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536,4608],"zarr_format":2}

model/params/model/visual/blocks/8/mlp/down_proj/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[4096,384],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[4096,1536],"zarr_format":2}

model/params/model/visual/blocks/8/norm2/kernel/.zarray

@@ -0,0 +1 @@
+{"chunks":[1536],"compressor":{"id":"zstd","level":1},"dimension_separator":".","dtype":"bfloat16","fill_value":null,"filters":null,"order":"C","shape":[1536],"zarr_format":2}