Super-squash branch 'main' using huggingface_hub

Co-authored-by: lckr <lckr@users.noreply.huggingface.co>
Co-authored-by: bartowski <bartowski@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,224 @@
+---
+license: apache-2.0
+language:
+- en
+- es
+- fr
+- de
+- it
+- pt
+- ru
+- ar
+- hi
+- ko
+- zh
+library_name: transformers
+base_model:
+- arcee-ai/Trinity-Large-Preview
+base_model_relation: quantized
+---
+<!-- markdownlint-disable first-line-h1 -->
+<!-- markdownlint-disable html -->
+<!-- markdownlint-disable no-duplicate-header -->
+
+<div align="center">
+  <picture>
+    <img
+      src="https://cdn-uploads.huggingface.co/production/uploads/6435718aaaef013d1aec3b8b/i-v1KyAMOW_mgVGeic9WJ.png"
+      alt="Arcee Trinity Large"
+      style="max-width: 100%; height: auto;"
+    >
+  </picture>
+</div>
+<hr>
+
+# Trinity-Large-Preview-FP8
+
+## Introduction
+
+Trinity-Large-Preview is a 398B-parameter sparse Mixture-of-Experts (MoE) model with approximately 13B active parameters per token. It is the largest model in Arcee AI's Trinity family, trained on more than 17 trillion tokens and delivering frontier-level performance with strong long-context comprehension.
+Trinity-Large-Preview is a lightly post-trained model based on Trinity-Large-Base.
+
+**This repository contains the FP8 quantized weights of Trinity-Large-Preview.**
+
+Try it at [chat.arcee.ai](http://chat.arcee.ai/)
+
+More details on the training of Trinity Large are available in the [technical report](https://github.com/arcee-ai/trinity-large-tech-report/).
+
+
+## Model Variants
+
+The Trinity Large family consists of three checkpoints from the same training run:
+
+- **[Trinity-Large-Preview](https://huggingface.co/arcee-ai/Trinity-Large-Preview)**: Lightly post-trained, chat-ready model undergoing active RL
+- **[Trinity-Large-TrueBase](https://huggingface.co/arcee-ai/Trinity-Large-TrueBase)**: 10T-token pre-anneal pretraining checkpoint
+- **[Trinity-Large-Base](https://huggingface.co/arcee-ai/Trinity-Large-Base)**: Full 17T-token pretrained foundation model with mid-training anneals
+
+## Architecture
+
+Trinity-Large-Preview uses a sparse MoE configuration designed to maximize efficiency while maintaining large-scale capacity.
+
+| Hyperparameter | Value |
+|:---|:---:|
+| Total parameters | ~398B |
+| Active parameters per token | ~13B |
+| Experts | 256 (1 shared) |
+| Active experts | 4 |
+| Routing strategy | 4-of-256 (1.56% sparsity) |
+| Dense layers | 6 |
+| Pretraining context length | 8,192 |
+| Context length after extension | 512k |
+| Architecture | Sparse MoE (AfmoeForCausalLM) |
+
+## Benchmarks
+
+| Benchmark | Llama 4 Maverick | Trinity-Large Preview |
+|-----------|------------------|----------------------|
+| MMLU | 85.5 | 87.2 |
+| MMLU-Pro | 80.5 | 75.2 |
+| GPQA-Diamond | 69.8 | 63.3 |
+| AIME 2025 | 19.3 | 24.0 |
+
+## Training Configuration
+
+### Pretraining
+
+- Training tokens: 17 trillion
+- Data partner: [Datology](https://www.datologyai.com/)
+
+<div align="center">
+  <picture>
+      <img src="https://cdn-uploads.huggingface.co/production/uploads/6435718aaaef013d1aec3b8b/sSVjGNHfrJKmQ6w8I18ek.png" style="background-color:ghostwhite;padding:5px;" width="17%" alt="Powered by Datology">
+  </picture>
+</div>
+
+## Posttraining
+- This checkpoint was instruction tuned on 20B tokens.
+
+### Infrastructure
+
+- Hardware: 2,048 NVIDIA B300 GPUs
+- Parallelism: HSDP + Expert Parallelism
+- Compute partner: [Prime Intellect](https://www.primeintellect.ai/)
+
+
+<div align="center">
+  <picture>
+      <img src="https://cdn-avatars.huggingface.co/v1/production/uploads/61e020e4a343274bb132e138/H2mcdPRWtl4iKLd-OYYBc.jpeg" style="background-color:ghostwhite;padding:5px;" width="17%" alt="Powered by Prime Intellect">
+  </picture>
+</div>
+
+## Usage
+
+### Running our model
+
+- [Transformers](https://huggingface.co/arcee-ai/Trinity-Large-Preview#transformers)
+- [VLLM](https://huggingface.co/arcee-ai/Trinity-Large-Preview#vllm)
+- [llama.cpp](https://huggingface.co/arcee-ai/Trinity-Large-Preview#llamacpp)
+- [LM Studio](https://huggingface.co/arcee-ai/Trinity-Large-Preview#lm-studio)
+- [API](https://huggingface.co/arcee-ai/Trinity-Large-Preview#api)
+
+
+### Recommended settings
+  * temperature: 
+  * top_k: 
+  * top_p: 
+  * min_p: 
+
+### Transformers
+
+Use the `main` transformers branch or pass `trust_remote_code=True` with a released version.
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+model_id = "arcee-ai/Trinity-Large-Preview"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True
+)
+
+messages = [
+    {"role": "user", "content": "Who are you?"},
+]
+
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+outputs = model.generate(
+    input_ids,
+    max_new_tokens=256,
+    do_sample=True,
+    temperature=0.5,
+    top_k=50,
+    top_p=0.95
+)
+
+response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(response)
+```
+
+### VLLM
+
+Supported in VLLM release 0.11.1+
+
+```bash
+vllm serve arcee-ai/Trinity-Large-Preview \
+  --dtype bfloat16 \
+  --enable-auto-tool-choice \
+  --tool-call-parser hermes
+```
+
+### llama.cpp
+
+Supported in llama.cpp release b7061+
+
+```bash
+llama-server -hf arcee-ai/Trinity-Large-Preview-GGUF:q4_k_m
+```
+
+### LM Studio
+
+Supported in the latest LM Studio runtime. Search for `arcee-ai/Trinity-Large-Preview-GGUF` in Model Search.
+
+### API
+
+Available on OpenRouter:
+
+```bash
+curl -X POST "https://openrouter.ai/v1/chat/completions" \
+  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "arcee-ai/trinity-large-preview",
+    "messages": [
+      {
+        "role": "user",
+        "content": "What are some fun things to do in New York?"
+      }
+    ]
+  }'
+```
+
+
+## License
+
+Trinity-Large-Preview is released under the Apache License, Version 2.0.
+
+## Citation
+
+```bibtex
+@misc{arcee_trinity_large_preview,
+  title = {Trinity-Large-Preview},
+  author = {{Arcee AI}},
+  year = {2026},
+  note = {398B sparse MoE model trained on 17T tokens}
+}
+```

chat_template.jinja

@@ -0,0 +1,65 @@
+{{ bos_token }}{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {% generation %}
+        {{- content}}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>' }}
+        {% endgeneration%}
+        {{- '\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,511 @@
+{
+  "architectures": [
+    "AfmoeForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "configuration_afmoe.AfmoeConfig",
+    "AutoModel": "modeling_afmoe.AfmoeModel",
+    "AutoModelForCausalLM": "modeling_afmoe.AfmoeForCausalLM"
+  },
+  "bos_token_id": 0,
+  "dtype": "bfloat16",
+  "eos_token_id": 3,
+  "global_attn_every_n_layers": 4,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 3072,
+  "initializer_range": 0.02,
+  "intermediate_size": 12288,
+  "layer_types": [
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention"
+  ],
+  "load_balance_coeff": 5e-05,
+  "max_position_embeddings": 262144,
+  "model_type": "afmoe",
+  "moe_intermediate_size": 3072,
+  "mup_enabled": true,
+  "n_group": 1,
+  "num_attention_heads": 48,
+  "num_dense_layers": 6,
+  "num_expert_groups": 1,
+  "num_experts": 256,
+  "num_experts_per_tok": 4,
+  "num_hidden_layers": 60,
+  "num_key_value_heads": 8,
+  "num_limited_groups": 1,
+  "num_shared_experts": 1,
+  "pad_token_id": 12,
+  "quantization_config": {
+    "config_groups": {
+      "group_0": {
+        "format": "float-quantized",
+        "input_activations": {
+          "actorder": null,
+          "block_structure": null,
+          "dynamic": true,
+          "group_size": null,
+          "num_bits": 8,
+          "observer": null,
+          "observer_kwargs": {},
+          "strategy": "token",
+          "symmetric": true,
+          "type": "float"
+        },
+        "output_activations": null,
+        "targets": [
+          "Linear"
+        ],
+        "weights": {
+          "actorder": null,
+          "block_structure": null,
+          "dynamic": false,
+          "group_size": null,
+          "num_bits": 8,
+          "observer": "minmax",
+          "observer_kwargs": {},
+          "strategy": "channel",
+          "symmetric": true,
+          "type": "float"
+        }
+      }
+    },
+    "format": "float-quantized",
+    "global_compression_ratio": null,
+    "ignore": [
+      "model.layers.0.self_attn.q_proj",
+      "model.layers.0.self_attn.k_proj",
+      "model.layers.0.self_attn.v_proj",
+      "model.layers.0.self_attn.o_proj",
+      "model.layers.0.self_attn.gate_proj",
+      "model.layers.1.self_attn.q_proj",
+      "model.layers.1.self_attn.k_proj",
+      "model.layers.1.self_attn.v_proj",
+      "model.layers.1.self_attn.o_proj",
+      "model.layers.1.self_attn.gate_proj",
+      "model.layers.2.self_attn.q_proj",
+      "model.layers.2.self_attn.k_proj",
+      "model.layers.2.self_attn.v_proj",
+      "model.layers.2.self_attn.o_proj",
+      "model.layers.2.self_attn.gate_proj",
+      "model.layers.3.self_attn.q_proj",
+      "model.layers.3.self_attn.k_proj",
+      "model.layers.3.self_attn.v_proj",
+      "model.layers.3.self_attn.o_proj",
+      "model.layers.3.self_attn.gate_proj",
+      "model.layers.4.self_attn.q_proj",
+      "model.layers.4.self_attn.k_proj",
+      "model.layers.4.self_attn.v_proj",
+      "model.layers.4.self_attn.o_proj",
+      "model.layers.4.self_attn.gate_proj",
+      "model.layers.5.self_attn.q_proj",
+      "model.layers.5.self_attn.k_proj",
+      "model.layers.5.self_attn.v_proj",
+      "model.layers.5.self_attn.o_proj",
+      "model.layers.5.self_attn.gate_proj",
+      "model.layers.6.self_attn.q_proj",
+      "model.layers.6.self_attn.k_proj",
+      "model.layers.6.self_attn.v_proj",
+      "model.layers.6.self_attn.o_proj",
+      "model.layers.6.self_attn.gate_proj",
+      "model.layers.6.mlp.router.gate",
+      "model.layers.7.self_attn.q_proj",
+      "model.layers.7.self_attn.k_proj",
+      "model.layers.7.self_attn.v_proj",
+      "model.layers.7.self_attn.o_proj",
+      "model.layers.7.self_attn.gate_proj",
+      "model.layers.7.mlp.router.gate",
+      "model.layers.8.self_attn.q_proj",
+      "model.layers.8.self_attn.k_proj",
+      "model.layers.8.self_attn.v_proj",
+      "model.layers.8.self_attn.o_proj",
+      "model.layers.8.self_attn.gate_proj",
+      "model.layers.8.mlp.router.gate",
+      "model.layers.9.self_attn.q_proj",
+      "model.layers.9.self_attn.k_proj",
+      "model.layers.9.self_attn.v_proj",
+      "model.layers.9.self_attn.o_proj",
+      "model.layers.9.self_attn.gate_proj",
+      "model.layers.9.mlp.router.gate",
+      "model.layers.10.self_attn.q_proj",
+      "model.layers.10.self_attn.k_proj",
+      "model.layers.10.self_attn.v_proj",
+      "model.layers.10.self_attn.o_proj",
+      "model.layers.10.self_attn.gate_proj",
+      "model.layers.10.mlp.router.gate",
+      "model.layers.11.self_attn.q_proj",
+      "model.layers.11.self_attn.k_proj",
+      "model.layers.11.self_attn.v_proj",
+      "model.layers.11.self_attn.o_proj",
+      "model.layers.11.self_attn.gate_proj",
+      "model.layers.11.mlp.router.gate",
+      "model.layers.12.self_attn.q_proj",
+      "model.layers.12.self_attn.k_proj",
+      "model.layers.12.self_attn.v_proj",
+      "model.layers.12.self_attn.o_proj",
+      "model.layers.12.self_attn.gate_proj",
+      "model.layers.12.mlp.router.gate",
+      "model.layers.13.self_attn.q_proj",
+      "model.layers.13.self_attn.k_proj",
+      "model.layers.13.self_attn.v_proj",
+      "model.layers.13.self_attn.o_proj",
+      "model.layers.13.self_attn.gate_proj",
+      "model.layers.13.mlp.router.gate",
+      "model.layers.14.self_attn.q_proj",
+      "model.layers.14.self_attn.k_proj",
+      "model.layers.14.self_attn.v_proj",
+      "model.layers.14.self_attn.o_proj",
+      "model.layers.14.self_attn.gate_proj",
+      "model.layers.14.mlp.router.gate",
+      "model.layers.15.self_attn.q_proj",
+      "model.layers.15.self_attn.k_proj",
+      "model.layers.15.self_attn.v_proj",
+      "model.layers.15.self_attn.o_proj",
+      "model.layers.15.self_attn.gate_proj",
+      "model.layers.15.mlp.router.gate",
+      "model.layers.16.self_attn.q_proj",
+      "model.layers.16.self_attn.k_proj",
+      "model.layers.16.self_attn.v_proj",
+      "model.layers.16.self_attn.o_proj",
+      "model.layers.16.self_attn.gate_proj",
+      "model.layers.16.mlp.router.gate",
+      "model.layers.17.self_attn.q_proj",
+      "model.layers.17.self_attn.k_proj",
+      "model.layers.17.self_attn.v_proj",
+      "model.layers.17.self_attn.o_proj",
+      "model.layers.17.self_attn.gate_proj",
+      "model.layers.17.mlp.router.gate",
+      "model.layers.18.self_attn.q_proj",
+      "model.layers.18.self_attn.k_proj",
+      "model.layers.18.self_attn.v_proj",
+      "model.layers.18.self_attn.o_proj",
+      "model.layers.18.self_attn.gate_proj",
+      "model.layers.18.mlp.router.gate",
+      "model.layers.19.self_attn.q_proj",
+      "model.layers.19.self_attn.k_proj",
+      "model.layers.19.self_attn.v_proj",
+      "model.layers.19.self_attn.o_proj",
+      "model.layers.19.self_attn.gate_proj",
+      "model.layers.19.mlp.router.gate",
+      "model.layers.20.self_attn.q_proj",
+      "model.layers.20.self_attn.k_proj",
+      "model.layers.20.self_attn.v_proj",
+      "model.layers.20.self_attn.o_proj",
+      "model.layers.20.self_attn.gate_proj",
+      "model.layers.20.mlp.router.gate",
+      "model.layers.21.self_attn.q_proj",
+      "model.layers.21.self_attn.k_proj",
+      "model.layers.21.self_attn.v_proj",
+      "model.layers.21.self_attn.o_proj",
+      "model.layers.21.self_attn.gate_proj",
+      "model.layers.21.mlp.router.gate",
+      "model.layers.22.self_attn.q_proj",
+      "model.layers.22.self_attn.k_proj",
+      "model.layers.22.self_attn.v_proj",
+      "model.layers.22.self_attn.o_proj",
+      "model.layers.22.self_attn.gate_proj",
+      "model.layers.22.mlp.router.gate",
+      "model.layers.23.self_attn.q_proj",
+      "model.layers.23.self_attn.k_proj",
+      "model.layers.23.self_attn.v_proj",
+      "model.layers.23.self_attn.o_proj",
+      "model.layers.23.self_attn.gate_proj",
+      "model.layers.23.mlp.router.gate",
+      "model.layers.24.self_attn.q_proj",
+      "model.layers.24.self_attn.k_proj",
+      "model.layers.24.self_attn.v_proj",
+      "model.layers.24.self_attn.o_proj",
+      "model.layers.24.self_attn.gate_proj",
+      "model.layers.24.mlp.router.gate",
+      "model.layers.25.self_attn.q_proj",
+      "model.layers.25.self_attn.k_proj",
+      "model.layers.25.self_attn.v_proj",
+      "model.layers.25.self_attn.o_proj",
+      "model.layers.25.self_attn.gate_proj",
+      "model.layers.25.mlp.router.gate",
+      "model.layers.26.self_attn.q_proj",
+      "model.layers.26.self_attn.k_proj",
+      "model.layers.26.self_attn.v_proj",
+      "model.layers.26.self_attn.o_proj",
+      "model.layers.26.self_attn.gate_proj",
+      "model.layers.26.mlp.router.gate",
+      "model.layers.27.self_attn.q_proj",
+      "model.layers.27.self_attn.k_proj",
+      "model.layers.27.self_attn.v_proj",
+      "model.layers.27.self_attn.o_proj",
+      "model.layers.27.self_attn.gate_proj",
+      "model.layers.27.mlp.router.gate",
+      "model.layers.28.self_attn.q_proj",
+      "model.layers.28.self_attn.k_proj",
+      "model.layers.28.self_attn.v_proj",
+      "model.layers.28.self_attn.o_proj",
+      "model.layers.28.self_attn.gate_proj",
+      "model.layers.28.mlp.router.gate",
+      "model.layers.29.self_attn.q_proj",
+      "model.layers.29.self_attn.k_proj",
+      "model.layers.29.self_attn.v_proj",
+      "model.layers.29.self_attn.o_proj",
+      "model.layers.29.self_attn.gate_proj",
+      "model.layers.29.mlp.router.gate",
+      "model.layers.30.self_attn.q_proj",
+      "model.layers.30.self_attn.k_proj",
+      "model.layers.30.self_attn.v_proj",
+      "model.layers.30.self_attn.o_proj",
+      "model.layers.30.self_attn.gate_proj",
+      "model.layers.30.mlp.router.gate",
+      "model.layers.31.self_attn.q_proj",
+      "model.layers.31.self_attn.k_proj",
+      "model.layers.31.self_attn.v_proj",
+      "model.layers.31.self_attn.o_proj",
+      "model.layers.31.self_attn.gate_proj",
+      "model.layers.31.mlp.router.gate",
+      "model.layers.32.self_attn.q_proj",
+      "model.layers.32.self_attn.k_proj",
+      "model.layers.32.self_attn.v_proj",
+      "model.layers.32.self_attn.o_proj",
+      "model.layers.32.self_attn.gate_proj",
+      "model.layers.32.mlp.router.gate",
+      "model.layers.33.self_attn.q_proj",
+      "model.layers.33.self_attn.k_proj",
+      "model.layers.33.self_attn.v_proj",
+      "model.layers.33.self_attn.o_proj",
+      "model.layers.33.self_attn.gate_proj",
+      "model.layers.33.mlp.router.gate",
+      "model.layers.34.self_attn.q_proj",
+      "model.layers.34.self_attn.k_proj",
+      "model.layers.34.self_attn.v_proj",
+      "model.layers.34.self_attn.o_proj",
+      "model.layers.34.self_attn.gate_proj",
+      "model.layers.34.mlp.router.gate",
+      "model.layers.35.self_attn.q_proj",
+      "model.layers.35.self_attn.k_proj",
+      "model.layers.35.self_attn.v_proj",
+      "model.layers.35.self_attn.o_proj",
+      "model.layers.35.self_attn.gate_proj",
+      "model.layers.35.mlp.router.gate",
+      "model.layers.36.self_attn.q_proj",
+      "model.layers.36.self_attn.k_proj",
+      "model.layers.36.self_attn.v_proj",
+      "model.layers.36.self_attn.o_proj",
+      "model.layers.36.self_attn.gate_proj",
+      "model.layers.36.mlp.router.gate",
+      "model.layers.37.self_attn.q_proj",
+      "model.layers.37.self_attn.k_proj",
+      "model.layers.37.self_attn.v_proj",
+      "model.layers.37.self_attn.o_proj",
+      "model.layers.37.self_attn.gate_proj",
+      "model.layers.37.mlp.router.gate",
+      "model.layers.38.self_attn.q_proj",
+      "model.layers.38.self_attn.k_proj",
+      "model.layers.38.self_attn.v_proj",
+      "model.layers.38.self_attn.o_proj",
+      "model.layers.38.self_attn.gate_proj",
+      "model.layers.38.mlp.router.gate",
+      "model.layers.39.self_attn.q_proj",
+      "model.layers.39.self_attn.k_proj",
+      "model.layers.39.self_attn.v_proj",
+      "model.layers.39.self_attn.o_proj",
+      "model.layers.39.self_attn.gate_proj",
+      "model.layers.39.mlp.router.gate",
+      "model.layers.40.self_attn.q_proj",
+      "model.layers.40.self_attn.k_proj",
+      "model.layers.40.self_attn.v_proj",
+      "model.layers.40.self_attn.o_proj",
+      "model.layers.40.self_attn.gate_proj",
+      "model.layers.40.mlp.router.gate",
+      "model.layers.41.self_attn.q_proj",
+      "model.layers.41.self_attn.k_proj",
+      "model.layers.41.self_attn.v_proj",
+      "model.layers.41.self_attn.o_proj",
+      "model.layers.41.self_attn.gate_proj",
+      "model.layers.41.mlp.router.gate",
+      "model.layers.42.self_attn.q_proj",
+      "model.layers.42.self_attn.k_proj",
+      "model.layers.42.self_attn.v_proj",
+      "model.layers.42.self_attn.o_proj",
+      "model.layers.42.self_attn.gate_proj",
+      "model.layers.42.mlp.router.gate",
+      "model.layers.43.self_attn.q_proj",
+      "model.layers.43.self_attn.k_proj",
+      "model.layers.43.self_attn.v_proj",
+      "model.layers.43.self_attn.o_proj",
+      "model.layers.43.self_attn.gate_proj",
+      "model.layers.43.mlp.router.gate",
+      "model.layers.44.self_attn.q_proj",
+      "model.layers.44.self_attn.k_proj",
+      "model.layers.44.self_attn.v_proj",
+      "model.layers.44.self_attn.o_proj",
+      "model.layers.44.self_attn.gate_proj",
+      "model.layers.44.mlp.router.gate",
+      "model.layers.45.self_attn.q_proj",
+      "model.layers.45.self_attn.k_proj",
+      "model.layers.45.self_attn.v_proj",
+      "model.layers.45.self_attn.o_proj",
+      "model.layers.45.self_attn.gate_proj",
+      "model.layers.45.mlp.router.gate",
+      "model.layers.46.self_attn.q_proj",
+      "model.layers.46.self_attn.k_proj",
+      "model.layers.46.self_attn.v_proj",
+      "model.layers.46.self_attn.o_proj",
+      "model.layers.46.self_attn.gate_proj",
+      "model.layers.46.mlp.router.gate",
+      "model.layers.47.self_attn.q_proj",
+      "model.layers.47.self_attn.k_proj",
+      "model.layers.47.self_attn.v_proj",
+      "model.layers.47.self_attn.o_proj",
+      "model.layers.47.self_attn.gate_proj",
+      "model.layers.47.mlp.router.gate",
+      "model.layers.48.self_attn.q_proj",
+      "model.layers.48.self_attn.k_proj",
+      "model.layers.48.self_attn.v_proj",
+      "model.layers.48.self_attn.o_proj",
+      "model.layers.48.self_attn.gate_proj",
+      "model.layers.48.mlp.router.gate",
+      "model.layers.49.self_attn.q_proj",
+      "model.layers.49.self_attn.k_proj",
+      "model.layers.49.self_attn.v_proj",
+      "model.layers.49.self_attn.o_proj",
+      "model.layers.49.self_attn.gate_proj",
+      "model.layers.49.mlp.router.gate",
+      "model.layers.50.self_attn.q_proj",
+      "model.layers.50.self_attn.k_proj",
+      "model.layers.50.self_attn.v_proj",
+      "model.layers.50.self_attn.o_proj",
+      "model.layers.50.self_attn.gate_proj",
+      "model.layers.50.mlp.router.gate",
+      "model.layers.51.self_attn.q_proj",
+      "model.layers.51.self_attn.k_proj",
+      "model.layers.51.self_attn.v_proj",
+      "model.layers.51.self_attn.o_proj",
+      "model.layers.51.self_attn.gate_proj",
+      "model.layers.51.mlp.router.gate",
+      "model.layers.52.self_attn.q_proj",
+      "model.layers.52.self_attn.k_proj",
+      "model.layers.52.self_attn.v_proj",
+      "model.layers.52.self_attn.o_proj",
+      "model.layers.52.self_attn.gate_proj",
+      "model.layers.52.mlp.router.gate",
+      "model.layers.53.self_attn.q_proj",
+      "model.layers.53.self_attn.k_proj",
+      "model.layers.53.self_attn.v_proj",
+      "model.layers.53.self_attn.o_proj",
+      "model.layers.53.self_attn.gate_proj",
+      "model.layers.53.mlp.router.gate",
+      "model.layers.54.self_attn.q_proj",
+      "model.layers.54.self_attn.k_proj",
+      "model.layers.54.self_attn.v_proj",
+      "model.layers.54.self_attn.o_proj",
+      "model.layers.54.self_attn.gate_proj",
+      "model.layers.54.mlp.router.gate",
+      "model.layers.55.self_attn.q_proj",
+      "model.layers.55.self_attn.k_proj",
+      "model.layers.55.self_attn.v_proj",
+      "model.layers.55.self_attn.o_proj",
+      "model.layers.55.self_attn.gate_proj",
+      "model.layers.55.mlp.router.gate",
+      "model.layers.56.self_attn.q_proj",
+      "model.layers.56.self_attn.k_proj",
+      "model.layers.56.self_attn.v_proj",
+      "model.layers.56.self_attn.o_proj",
+      "model.layers.56.self_attn.gate_proj",
+      "model.layers.56.mlp.router.gate",
+      "model.layers.57.self_attn.q_proj",
+      "model.layers.57.self_attn.k_proj",
+      "model.layers.57.self_attn.v_proj",
+      "model.layers.57.self_attn.o_proj",
+      "model.layers.57.self_attn.gate_proj",
+      "model.layers.57.mlp.router.gate",
+      "model.layers.58.self_attn.q_proj",
+      "model.layers.58.self_attn.k_proj",
+      "model.layers.58.self_attn.v_proj",
+      "model.layers.58.self_attn.o_proj",
+      "model.layers.58.self_attn.gate_proj",
+      "model.layers.58.mlp.router.gate",
+      "model.layers.59.self_attn.q_proj",
+      "model.layers.59.self_attn.k_proj",
+      "model.layers.59.self_attn.v_proj",
+      "model.layers.59.self_attn.o_proj",
+      "model.layers.59.self_attn.gate_proj",
+      "model.layers.59.mlp.router.gate",
+      "lm_head"
+    ],
+    "kv_cache_scheme": null,
+    "quant_method": "compressed-tensors",
+    "quantization_status": "compressed",
+    "sparsity_config": {},
+    "transform_config": {},
+    "version": "0.13.1.a20260116"
+  },
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000,
+  "route_norm": true,
+  "route_scale": 2.448,
+  "score_func": "sigmoid",
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "topk_group": 1,
+  "transformers_version": "4.57.3",
+  "use_cache": true,
+  "use_grouped_mm": true,
+  "vocab_size": 200192
+}

configuration_afmoe.py

@@ -0,0 +1,133 @@
+# coding=utf-8
+# Copyright 2022 EleutherAI and the HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from transformers.configuration_utils import PretrainedConfig
+from transformers.modeling_rope_utils import rope_config_validation
+from transformers.configuration_utils import layer_type_validation
+from transformers.utils import logging
+
+logger = logging.get_logger(__name__)
+
+class AfmoeConfig(PretrainedConfig):
+    """
+    n_group (`int`, *optional*, defaults to 1):
+            Number of groups for routed experts.
+    topk_group (`int`, *optional*, defaults to 1):
+        Number of selected groups for each token(for each token, ensuring the selected experts is only within `topk_group` groups).
+    """
+    model_type = "afmoe"
+    base_model_pp_plan = {
+        "embed_tokens": (["input_ids"], ["inputs_embeds"]),
+        "layers": (["hidden_states", "attention_mask"], ["hidden_states"]),
+        "norm": (["hidden_states"], ["hidden_states"]),
+    }
+
+    def __init__(
+        self,
+        num_hidden_layers: int = 32,
+        vocab_size: int = 200192,
+        hidden_size: int = 2048,
+        intermediate_size: int = 6144,
+        moe_intermediate_size=1408,
+        num_dense_layers=1,
+        num_attention_heads=16,
+        num_key_value_heads=None,
+        head_dim=128,
+        hidden_act="silu",
+        max_position_embeddings=16384,
+        initializer_range=0.02,
+        rms_norm_eps=1e-5,
+        use_cache=True,
+        tie_word_embeddings=False,
+        rope_theta=10000.0,
+        rope_scaling=None,
+        num_experts=64,
+        num_experts_per_tok=6,
+        num_shared_experts=2,
+        num_expert_groups=1,
+        num_limited_groups=1,
+        score_func="sigmoid",
+        route_norm=True,
+        route_scale=1.0,
+        global_attn_every_n_layers=4,
+        sliding_window=1024,
+        mup_enabled=False,
+        layer_types=None,
+        attention_dropout: float = 0.0,
+        n_group: int = 1,
+        topk_group: int = 1,
+        **kwargs,
+    ):
+        self.vocab_size = vocab_size
+        self.max_position_embeddings = max_position_embeddings
+        self.hidden_size = hidden_size
+        self.intermediate_size = intermediate_size
+        self.num_hidden_layers = num_hidden_layers
+        self.num_dense_layers = num_dense_layers
+        self.num_attention_heads = num_attention_heads
+        self.head_dim = head_dim
+        self.hidden_act = hidden_act
+        self.initializer_range = initializer_range
+        self.rms_norm_eps = rms_norm_eps
+        self.use_cache = use_cache
+        self.rope_theta = rope_theta
+        self.rope_scaling = rope_scaling
+
+
+        # MoE specific
+        self.moe_intermediate_size = moe_intermediate_size
+        self.num_experts_per_tok = num_experts_per_tok
+        self.n_group = n_group
+        self.topk_group = topk_group
+        self.num_experts = num_experts
+        self.num_shared_experts = num_shared_experts
+        self.num_expert_groups = num_expert_groups
+        self.num_limited_groups = num_limited_groups
+        self.score_func = score_func
+        self.route_norm = route_norm
+        self.route_scale = route_scale
+
+
+        # Attention specific
+        self.attention_dropout = attention_dropout
+        self.global_attn_every_n_layers = global_attn_every_n_layers
+        self.sliding_window = sliding_window
+        self.layer_types = layer_types
+        if self.layer_types is None:
+            self.layer_types = [
+                "sliding_attention" if bool((i + 1) % global_attn_every_n_layers) else "full_attention" for i in range(self.num_hidden_layers)
+            ]
+        layer_type_validation(self.layer_types)
+
+        # muP specific
+        self.mup_enabled = mup_enabled
+
+        if num_key_value_heads is None:
+            num_key_value_heads = num_attention_heads
+
+        self.num_key_value_heads = num_key_value_heads
+
+
+        # Validate rope configs
+        if self.rope_scaling is not None and "type" in self.rope_scaling:
+            self.rope_scaling["rope_type"] = self.rope_scaling["type"]
+        rope_config_validation(self)
+
+        super().__init__(
+            tie_word_embeddings=tie_word_embeddings,
+            **kwargs,
+        )
+
+
+__all__ = ["AfmoeConfig"]

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 3,
+  "pad_token_id": 12,
+  "transformers_version": "4.57.3"
+}

model-00001-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8dbbf27913667dd5daa8f3ab2d6caf94cc06a85f68b62aaed6e4b594ba5d9e87
+size 4992710176

model-00002-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d40b6c07723fdcbde16cf0c078998965adcdf8d7c3e460f3266c72c4e1ddff3
+size 4995652424

model-00003-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67881e1a723b4e32338a1919f82b4468e54615e58aa39dad33c7686053474b47
+size 4990871376

model-00004-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c89b03fb780f45dc4d32f81d91cea59fad950203c5a34e2cb68e4120c3ad4e7
+size 4990871488

model-00005-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09f042de3d82a8efa37228ca8a588f823de94b648e4af2014db6effca9ad86fa
+size 4970435832

model-00006-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96acbf691df5b6c6309785377fe9cd1a98f56128418c3aa6fafd9147f815322d
+size 4997200848

model-00007-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b188aad182c1c66ab422a4bf0ce55ea959797645a73f6ec8edac746bacd5391b
+size 4990873000

model-00008-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd324bc2598cef90e0ee2db3be91300af12c8fea9addd83d75c8d9222ca36e0c
+size 4990872984

model-00009-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb52d6d4dd2981214ddcd9fa2682cb88f148c80c1b2aa57910a52b70dd3f7bf1
+size 4995652984

model-00010-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3a848fb367a760cc13b41ed5d52c11b090a8199f6188934cc176e058f0c5877
+size 4990872464

model-00011-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08e4965fda9298555fa0c825eea19715575747ad57f7701f8d6dba8233839a15
+size 4990872944

model-00012-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0135513fcb2f1e4a10c85cae23b4e644dc6bb11ab5f4423e7a06e6989051a4c2
+size 4995653056

model-00013-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63a86d89577344eb765450a27dad75005bad3eafb4f07ccd7f56318eaa54b4bb
+size 4990872432

model-00014-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:40f347e9fec7fb67e2c2cf737780e28384072eb7f74419e82e46cf6ae98ddc6a
+size 4990872936

model-00015-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5624091fe7f150abd2e3e7084e54d1db84b8fa951d2a74f52ee4648784b92808
+size 4995653096

model-00016-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c854b5c666838192a26ba8227575661ec3504bb9379bc365e0709609f027f15
+size 4990872424

model-00017-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e96deaafe66fba4ea1d3910d7bab515b9d98b98595a23f995343a9a4de1dc28c
+size 4990872904

model-00018-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35e7fea8257dc420395bc408f9cdb88ab788badeebfb9dec3178a0d5ad875c70
+size 4995653136

model-00019-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4df68b1cc5658a73cad254a05bc49c1ee4bf7fbe15b2d51794b5faf3444820ca
+size 4990872424

model-00020-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afde029d6331842ca7eb9c0a2223ad9e390b1a035f875c3705d5516bc1b56f0e
+size 4990872872

model-00021-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5083a325e9a6f68769da076016c8e70b3968ee2792a45e9375542dddde736c69
+size 4995653168

model-00022-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c86cff8a0ef1f39c6b9673660b1a4b53463b49f411c6545a74c966813cb9ed3
+size 4990872424

model-00023-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df878f4956784ad1869eef6cd01c9ae297182110bab4a6029fda7552e204caf2
+size 4990872840

model-00024-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9ad486d4fbf0002b4c302bdf73311ddbaef53c89e20cd3edd14d561f3576614
+size 4995653200

model-00025-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c96b35be297a8334a61706101de1b58f7fc169710987eb274ea53b5541c67a8f
+size 4990872424

model-00026-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:171808ce9c704536726581dd72dc98af7577db297409b916f5f4208b27390fe3
+size 4990872808

model-00027-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70c0ec59c6e718d097caba0b828fd4f0de7f616fb39ab59649c16f165966ecf3
+size 4995653240

model-00028-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d688e415e3d8a840d4e047fdd47bce7ad6f65c67f11852095d77baa7959ad5c3
+size 4990872424

model-00029-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72d82006184cd7e045a48c0ea348abf22f52f3abae16e154bc9cfdc332965683
+size 4990872768

model-00030-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d894424ddae10413b5e6dadbcbe25f44ab3a674e79a42d3522794daee7ce20b1
+size 4995653272

model-00031-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0408c2fb5b843d1c96e63d1497c97cb59bf89732cb5ad4c014806d42621885b2
+size 4990872424

model-00032-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0527a783f733553683df36c33c3100eb8539067773797bc2aed9dfa8f0e06bfb
+size 4990872744

model-00033-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b1e190416b58a5a9a5b66d2abc9f8b7b0b177b4b103e9643fcc51522119210a
+size 4995653304

model-00034-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81ffbd498ad533439be3ece295de1fb3a2ca8282df6b22a5abac392e58ca9163
+size 4990872424

model-00035-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:451710d6308b07538279db1b73c9fc00ef5b4b69fcc3beb2f88c2cfccf691d3c
+size 4990872704

model-00036-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08115015dd88aeb65ae404b4f3ecbfc664d753dd314993e75246826e999f84f0
+size 4995653336

model-00037-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f73ecc3c239deaf43edb4ca56950dcd412869fac73ab0111bd4ae26bb2e8cb51
+size 4990872424

model-00038-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b7efd3ba1c2769b992823db255c1fd98fd32bb7e2f65c142cadc7da97923849
+size 4990872672

model-00039-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d63c968f35f38009401c4aa07e10077667be53dd9aef608b590b91ff2486bce7
+size 4995653376

model-00040-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f2081924946e17d3e484859e3aef1d042ad30e7f9aa037888573af6d36b22c5d
+size 4990872424

model-00041-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09e61bd832509e0a22b88cfcaa713f6170a83d363201464b483e149e1969a86c
+size 4990872640

model-00042-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:201c1564640e56c8847d3a8bd89cd06d6e521dbb8757769177b2494832e308ab
+size 4995653400

model-00043-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39b530176c8a49c149f20902e4a460c1a9322ab6bbc728eacae626711e5b5252
+size 4990872424

model-00044-of-00081.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:62c0794a387f4423576ca9cd7141d72bd89eff1e341765d6f15bcf056de3a5a0
+size 4990872600