Standard One 8B SH v2
Browse files- .gitattributes +2 -0
- README.md +63 -32
- SHA256SUMS +26 -0
- SYSTEM_PROMPT.txt +29 -0
- adapter/adapter_config.json +0 -54
- chat_template.jinja +189 -0
- code/render.py +16 -4
- config.json +67 -0
- generation_config.json +7 -0
- head/schema_head.safetensors +1 -1
- head/schema_head_config.json +4 -5
- model-00001-of-00004.safetensors +3 -0
- model-00002-of-00004.safetensors +3 -0
- model-00003-of-00004.safetensors +3 -0
- model-00004-of-00004.safetensors +3 -0
- model.safetensors.index.json +539 -0
- params.json +47 -0
- processor_config.json +42 -0
- release-manifest.json +51 -0
- special_tokens_map.json +0 -0
- adapter/adapter_model.safetensors → tekken.json +2 -2
- tokenizer.json +3 -0
- tokenizer_config.json +0 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
tekken.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,7 +1,6 @@
|
|
| 1 |
---
|
| 2 |
base_model: StandardThinking/StandardOne-8B
|
| 3 |
-
|
| 4 |
-
library_name: peft
|
| 5 |
license: apache-2.0
|
| 6 |
pipeline_tag: text-classification
|
| 7 |
language:
|
|
@@ -18,16 +17,24 @@ tags:
|
|
| 18 |
|
| 19 |
# Standard One 8B SH
|
| 20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
**Standard One 8B SH** is Standard One 8B with a **joint schema head** (SH). It reads the scenario once and
|
| 22 |
scores every option of every question in a single forward pass, then returns probabilities through the same
|
| 23 |
`POST /v1/systemone` contract as Standard One. It does not generate text.
|
| 24 |
|
| 25 |
| | |
|
| 26 |
|---|---|
|
| 27 |
-
| Backbone |
|
| 28 |
-
|
|
| 29 |
| Question types | `choice` (2–255 options), `noul` (yes/no), `score` (ordinal scale) |
|
| 30 |
| Context | up to 32,768 tokens per request |
|
|
|
|
|
|
|
| 31 |
| License | Apache-2.0 |
|
| 32 |
|
| 33 |
## Why a schema head
|
|
@@ -43,36 +50,50 @@ graph, and returns a calibrated distribution for each question at once. In pract
|
|
| 43 |
|
| 44 |
## Results
|
| 45 |
|
| 46 |
-
All numbers
|
| 47 |
-
|
| 48 |
-
| Benchmark | Standard One 8B v2.2 |
|
| 49 |
-
|---|---:|---:|
|
| 50 |
-
| [Decision Index](https://huggingface.co/spaces/multimodalart/jev-decision-index) 0.2.1 (self-run, full suite) | 41.14 |
|
| 51 |
-
| JevBench public — hard tier (111 items) | 56.76 | **69.37** |
|
| 52 |
-
| JevBench public — original tier (72 items) | 98.61 | 98.61 |
|
| 53 |
-
| JevBench public — easy tier (48 items) | 100.00 | 100.00 |
|
| 54 |
-
| Intent labels with 27–151 options (CLINC150, BANKING77, MASSIVE; 18,000 items) | 82.67 | **92.
|
| 55 |
-
| Intent labels with
|
| 56 |
-
| Decision hidden in a long document (150 items) | 40.00 | **88.00** |
|
| 57 |
-
| WinoGrande (dev, 2,010 items) | 87.86 |
|
| 58 |
-
| Policy-judgement repair (1,200 items) | 60.58 | **74.25** |
|
| 59 |
-
| Game positions: dots and boxes / snake and 2048 / tetris | 38.37 / 58.63 / 54.86 | **51.96 / 68.37 / 64.19** |
|
| 60 |
-
| Held-out decision suite (600 items) | 82.50 |
|
| 61 |
-
| Held-out realistic decision suite (600 items) | 91.50 | **91.67** |
|
| 62 |
-
|
| 63 |
-
Accuracy in percent, except the Decision Index (its own 0–100 index). The Decision Index
|
| 64 |
-
in-edition requests of edition 0.2.1;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
|
| 66 |
## Quick start
|
| 67 |
|
| 68 |
```bash
|
| 69 |
-
pip install
|
| 70 |
-
hf download StandardThinking/StandardOne-8B --revision v2.2 --local-dir StandardOne-8B
|
| 71 |
hf download StandardThinking/StandardOne-8B-SH --local-dir StandardOne-8B-SH
|
| 72 |
|
| 73 |
python StandardOne-8B-SH/code/serve_head.py \
|
| 74 |
-
--snapshot StandardOne-8B
|
| 75 |
-
--port 30171
|
| 76 |
```
|
| 77 |
|
| 78 |
```bash
|
|
@@ -88,19 +109,29 @@ curl -s localhost:30171/v1/systemone -H 'content-type: application/json' -d '{
|
|
| 88 |
```
|
| 89 |
|
| 90 |
The server batches concurrent requests (`--max-batch-requests`, `--max-batch-tokens`) and binds `127.0.0.1` by
|
| 91 |
-
default. It runs on one GPU with the backbone in bfloat16 and the head in float32.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
|
| 93 |
## Limitations
|
| 94 |
|
| 95 |
- Requests longer than 32,768 tokens are refused, not truncated.
|
| 96 |
- On our hard held-out decision suite (600 items; long policies, multi-step lookups, date and number reasoning)
|
| 97 |
-
it scores
|
| 98 |
- Text input only has been validated for this release.
|
| 99 |
-
- The server is a PyTorch
|
| 100 |
-
load is lower than the engine-served Standard One models.
|
| 101 |
- Probabilities are calibrated on our held-out data; refit thresholds on your own distribution before relying on them.
|
| 102 |
|
| 103 |
## License
|
| 104 |
|
| 105 |
-
Apache-2.0.
|
| 106 |
built on [Ministral 3 8B](https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512-BF16) (Apache-2.0).
|
|
|
|
| 1 |
---
|
| 2 |
base_model: StandardThinking/StandardOne-8B
|
| 3 |
+
library_name: transformers
|
|
|
|
| 4 |
license: apache-2.0
|
| 5 |
pipeline_tag: text-classification
|
| 6 |
language:
|
|
|
|
| 17 |
|
| 18 |
# Standard One 8B SH
|
| 19 |
|
| 20 |
+
> **Updated weights (v2, 2026-10-09).** This repository now holds the full merged backbone weights plus the schema
|
| 21 |
+
> head, so no separate base download or LoRA merge is needed. The previous release (a LoRA adapter for
|
| 22 |
+
> StandardOne-8B v2.2 plus a head) stays available under the tag `v1`.
|
| 23 |
+
|
| 24 |
+
**Version:** v2
|
| 25 |
+
|
| 26 |
**Standard One 8B SH** is Standard One 8B with a **joint schema head** (SH). It reads the scenario once and
|
| 27 |
scores every option of every question in a single forward pass, then returns probabilities through the same
|
| 28 |
`POST /v1/systemone` contract as Standard One. It does not generate text.
|
| 29 |
|
| 30 |
| | |
|
| 31 |
|---|---|
|
| 32 |
+
| Backbone | Standard One 8B v2.2 (`mistral3`, Ministral 3 8B text + Pixtral vision tower), fine-tuned for the head and merged; weights in this repository |
|
| 33 |
+
| Head | joint schema head, 70.6M parameters, float32 (`head/`) |
|
| 34 |
| Question types | `choice` (2–255 options), `noul` (yes/no), `score` (ordinal scale) |
|
| 35 |
| Context | up to 32,768 tokens per request |
|
| 36 |
+
| Prompt format | the chat template with **no default system message** (see "Prompt format") |
|
| 37 |
+
| Other formats | [StandardOne-8B-SH-FP8](https://huggingface.co/StandardThinking/StandardOne-8B-SH-FP8) · [StandardOne-8B-SH-GGUF](https://huggingface.co/StandardThinking/StandardOne-8B-SH-GGUF) |
|
| 38 |
| License | Apache-2.0 |
|
| 39 |
|
| 40 |
## Why a schema head
|
|
|
|
| 50 |
|
| 51 |
## Results
|
| 52 |
|
| 53 |
+
All numbers compare models on identical items, scored the same way.
|
| 54 |
+
|
| 55 |
+
| Benchmark | Standard One 8B v2.2 | SH v1 | **SH v2 (this release)** |
|
| 56 |
+
|---|---:|---:|---:|
|
| 57 |
+
| [Decision Index](https://huggingface.co/spaces/multimodalart/jev-decision-index) 0.2.1 (self-run, full suite) | 41.14 | 46.17 | **48.68** |
|
| 58 |
+
| JevBench public — hard tier (111 items) | 56.76 | **69.37** | 63.96 |
|
| 59 |
+
| JevBench public — original tier (72 items) | 98.61 | 98.61 | 98.61 |
|
| 60 |
+
| JevBench public — easy tier (48 items) | 100.00 | 100.00 | 100.00 |
|
| 61 |
+
| Intent labels with 27–151 options (CLINC150, BANKING77, MASSIVE; 18,000 items) | 82.67 | 92.09 | **92.50** |
|
| 62 |
+
| Intent labels with 255 options (CLINC150, 1,100 utterances) | 85.34 | 96.91 | **97.43** |
|
| 63 |
+
| Decision hidden in a long document (150 items) | 40.00 | **88.00** | 84.67 |
|
| 64 |
+
| WinoGrande (dev, 2,010 items) | 87.86 | 94.98 | **95.07** |
|
| 65 |
+
| Policy-judgement repair (1,200 items) | 60.58 | **74.25** | 72.92 |
|
| 66 |
+
| Game positions: dots and boxes / snake and 2048 / tetris | 38.37 / 58.63 / 54.86 | **51.96 / 68.37** / 64.19 | 50.42 / 67.59 / **65.90** |
|
| 67 |
+
| Held-out decision suite (600 items) | 82.50 | 88.33 | **90.67** |
|
| 68 |
+
| Held-out realistic decision suite (600 items) | 91.50 | **91.67** | 90.83 |
|
| 69 |
+
|
| 70 |
+
Accuracy in percent, except the Decision Index (its own 0–100 index). The Decision Index runs cover all 150,759
|
| 71 |
+
in-edition requests of edition 0.2.1; they are our own runs with the kit's scorer, not leaderboard entries.
|
| 72 |
+
|
| 73 |
+
## Changes in v2
|
| 74 |
+
|
| 75 |
+
- **Full weights.** The backbone fine-tune is merged into the weights (fp32 merge, bfloat16 save); `adapter/` is gone.
|
| 76 |
+
- **Broader training data.** v2 adds public-dataset decision tasks to the training mix. The Decision Index rises from
|
| 77 |
+
46.17 to 48.68, mostly in retrieval and classification tasks, and the held-out decision suite from 88.33 to 90.67.
|
| 78 |
+
- **No default system prompt.** v1 rendered every request with the base chat template's long default system message
|
| 79 |
+
(about 520 tokens). v2 was trained and is served without it, so short requests are several times shorter.
|
| 80 |
+
- **Trade-offs.** The JevBench hard tier (69.37 → 63.96), long-document decisions (88.00 → 84.67) and policy
|
| 81 |
+
judgement (74.25 → 72.92) are lower than v1. Keep `v1` if those workloads matter most to you.
|
| 82 |
+
|
| 83 |
+
## Prompt format
|
| 84 |
+
|
| 85 |
+
The included `chat_template.jinja` has an empty default system message, and the reference server renders requests
|
| 86 |
+
without one (`SH_SYSTEM_PROMPT=none`, the default in `code/render.py`). Do not add the Ministral default system
|
| 87 |
+
prompt: the head was trained without it, and adding it changes the hidden states it reads.
|
| 88 |
|
| 89 |
## Quick start
|
| 90 |
|
| 91 |
```bash
|
| 92 |
+
pip install -r code/requirements.txt # torch, transformers, safetensors
|
|
|
|
| 93 |
hf download StandardThinking/StandardOne-8B-SH --local-dir StandardOne-8B-SH
|
| 94 |
|
| 95 |
python StandardOne-8B-SH/code/serve_head.py \
|
| 96 |
+
--snapshot StandardOne-8B-SH --head StandardOne-8B-SH/head --port 30171
|
|
|
|
| 97 |
```
|
| 98 |
|
| 99 |
```bash
|
|
|
|
| 109 |
```
|
| 110 |
|
| 111 |
The server batches concurrent requests (`--max-batch-requests`, `--max-batch-tokens`) and binds `127.0.0.1` by
|
| 112 |
+
default. It runs on one GPU with the backbone in bfloat16 and the head in float32. For CPU or Apple-silicon use, see
|
| 113 |
+
[StandardOne-8B-SH-GGUF](https://huggingface.co/StandardThinking/StandardOne-8B-SH-GGUF).
|
| 114 |
+
|
| 115 |
+
## Files
|
| 116 |
+
|
| 117 |
+
| Path | Contents |
|
| 118 |
+
|---|---|
|
| 119 |
+
| `model-0000{1..4}-of-00004.safetensors`, `config.json`, tokenizer files | merged backbone, bfloat16 (`Mistral3ForConditionalGeneration`) |
|
| 120 |
+
| `chat_template.jinja` | chat template with an empty default system message |
|
| 121 |
+
| `head/` | `schema_head.safetensors` (float32) and `schema_head_config.json` (taps: final layer and layer 25) |
|
| 122 |
+
| `code/` | reference `/v1/systemone` server: request rendering, head, scoring |
|
| 123 |
+
| `SHA256SUMS`, `release-manifest.json` | checksums; source and merge details |
|
| 124 |
|
| 125 |
## Limitations
|
| 126 |
|
| 127 |
- Requests longer than 32,768 tokens are refused, not truncated.
|
| 128 |
- On our hard held-out decision suite (600 items; long policies, multi-step lookups, date and number reasoning)
|
| 129 |
+
it scores 46.00 against 53.50 for Standard One 8B v2.2. Prefer Standard One 8B for that kind of workload.
|
| 130 |
- Text input only has been validated for this release.
|
| 131 |
+
- The reference server is a PyTorch server; throughput under heavy load is lower than an inference engine.
|
|
|
|
| 132 |
- Probabilities are calibrated on our held-out data; refit thresholds on your own distribution before relying on them.
|
| 133 |
|
| 134 |
## License
|
| 135 |
|
| 136 |
+
Apache-2.0. Built on [Standard One 8B](https://huggingface.co/StandardThinking/StandardOne-8B) (Apache-2.0), which is
|
| 137 |
built on [Ministral 3 8B](https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512-BF16) (Apache-2.0).
|
SHA256SUMS
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
1495e1e757ef4d0925a2350563cf5754bb23c51701a8ec4fb3c5cdcbedae6747 LICENSE
|
| 2 |
+
2e46a8977e057ca9c452138915ceda5dd9fa946143d0b5c9ce0feb6f59925efc README.md
|
| 3 |
+
66e0fda8c8ab269ccd010dca6196931d475a22c3f2eebe5b6b3a2ec33a354d0e SYSTEM_PROMPT.txt
|
| 4 |
+
e62f05da25555a4c109d0f413eb673d328fcc40f7c82537c61631611f6954372 chat_template.jinja
|
| 5 |
+
5f3d6035ca36885c9be04b7cfa88b5ba532c87d5b26fa3525c94327fb1b4348b code/head.py
|
| 6 |
+
88e3e1c200ff0d37f2a9fcb864ebf06b864e4883dcd71f2297d83bdf96a73120 code/model.py
|
| 7 |
+
72ad6b7d1989105b8425c5d79e347fda2e9d3f2b429a5935c6d660983be412d1 code/render.py
|
| 8 |
+
41685ab5516b7bf5eef3431b1fbcff43b35ed83bc957b88ffa4a4e23bf8ec855 code/requirements.txt
|
| 9 |
+
9f18b699a0917d51b916e8ed96af87eb086af06fb9e6a723767032c107a50d15 code/scoring.py
|
| 10 |
+
1b31b60dc0273ef671e230b3b1db79efd2a7de1410feef380757fc4cb2a316d9 code/serve_head.py
|
| 11 |
+
2fc7f8d6b8cb4fc34f369475fe11d7e7b9bb1ef629b647d6ac6acb97666b55b6 config.json
|
| 12 |
+
e0923390059f84a9180b00e5501778acc45ea9856cd7f2fd68208b360927c677 generation_config.json
|
| 13 |
+
1b4bf756cb7c2413b30f2f34a6bab6578d6ccabc0cfd25b77f39b6af94469e14 head/schema_head.safetensors
|
| 14 |
+
ca506de1c23024e21fee174a03871e0db2ed073e35754b4e1c33d8e2d54173fb head/schema_head_config.json
|
| 15 |
+
4f2ca3c4f8c6f86466c4edab8cf4ca8f3ef2fd5b5859f81387de5d8f8b86111a model-00001-of-00004.safetensors
|
| 16 |
+
a86699c59416f085442224865ee84f7f305332a712956b210cb0b4e915b1d5f8 model-00002-of-00004.safetensors
|
| 17 |
+
9a5cbf1aa0b7b72a92303020a2752279010e71c6745a53f36e65304e229fd67e model-00003-of-00004.safetensors
|
| 18 |
+
70a8726a628117450473df8e8db53982a6d8bfb6df7aa337bb59d0fcb7068394 model-00004-of-00004.safetensors
|
| 19 |
+
71e022361d37d84c2b6224acbddca2c74bedcc7addc523cfc6a57420d4b22ceb model.safetensors.index.json
|
| 20 |
+
81b8377f36c5b3d60900b333214d85160a0500b84576e969092c6fd214f69538 params.json
|
| 21 |
+
ece2373c2ae391bce18785a1810543bc6173a1f28b6767cffab2c35dbea5f002 processor_config.json
|
| 22 |
+
00ce758bfa3972cc21fdab4defd0561ca77389115580d4fbfde41d540ac0b628 release-manifest.json
|
| 23 |
+
0a5c981e8c5c6f8886ee007a6d4543a0be6b221cb9ca32a8709384a4c6fc8cbb special_tokens_map.json
|
| 24 |
+
600bb27946565481ecf51ba8aee252e49b9a68507866080ac9c30185bb312843 tekken.json
|
| 25 |
+
d5f6046775b112f0e2d456ee9dba450684ab964fe5c4e231599bdc6773028135 tokenizer.json
|
| 26 |
+
f59f7294e4f26383d0ea93840fe21cf197784be0842a8301a0343e8c34ed0d6d tokenizer_config.json
|
SYSTEM_PROMPT.txt
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
You are Ministral-3-8B-Instruct-2512, a Large Language Model (LLM) created by Mistral AI, a French startup headquartered in Paris.
|
| 2 |
+
You power an AI assistant called Le Chat.
|
| 3 |
+
Your knowledge base was last updated on 2023-10-01.
|
| 4 |
+
The current date is {today}.
|
| 5 |
+
|
| 6 |
+
When you're not sure about some information or when the user's request requires up-to-date or specific data, you must use the available tools to fetch the information. Do not hesitate to use tools whenever they can provide a more accurate or complete response. If no relevant tools are available, then clearly state that you don't have the information and avoid making up anything.
|
| 7 |
+
If the user's question is not clear, ambiguous, or does not provide enough context for you to accurately answer the question, you do not try to answer it right away and you rather ask the user to clarify their request (e.g. "What are some good restaurants around me?" => "Where are you?" or "When is the next flight to Tokyo" => "Where do you travel from?").
|
| 8 |
+
You are always very attentive to dates, in particular you try to resolve dates (e.g. "yesterday" is {yesterday}) and when asked about information at specific dates, you discard information that is at another date.
|
| 9 |
+
You follow these instructions in all languages, and always respond to the user in the language they use or request.
|
| 10 |
+
Next sections describe the capabilities that you have.
|
| 11 |
+
|
| 12 |
+
# WEB BROWSING INSTRUCTIONS
|
| 13 |
+
|
| 14 |
+
You cannot perform any web search or access internet to open URLs, links etc. If it seems like the user is expecting you to do so, you clarify the situation and ask the user to copy paste the text directly in the chat.
|
| 15 |
+
|
| 16 |
+
# MULTI-MODAL INSTRUCTIONS
|
| 17 |
+
|
| 18 |
+
You have the ability to read images, but you cannot generate images. You also cannot transcribe audio files or videos.
|
| 19 |
+
You cannot read nor transcribe audio files or videos.
|
| 20 |
+
|
| 21 |
+
# TOOL CALLING INSTRUCTIONS
|
| 22 |
+
|
| 23 |
+
You may have access to tools that you can use to fetch information or perform actions. You must use these tools in the following situations:
|
| 24 |
+
|
| 25 |
+
1. When the request requires up-to-date information.
|
| 26 |
+
2. When the request requires specific data that you do not have in your knowledge base.
|
| 27 |
+
3. When the request involves actions that you cannot perform without tools.
|
| 28 |
+
|
| 29 |
+
Always prioritize using tools to provide the most accurate and helpful response. If tools are not available, inform the user that you cannot perform the requested action at the moment.
|
adapter/adapter_config.json
DELETED
|
@@ -1,54 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"alora_invocation_tokens": null,
|
| 3 |
-
"alpha_pattern": {},
|
| 4 |
-
"arrow_config": null,
|
| 5 |
-
"auto_mapping": {
|
| 6 |
-
"base_model_class": "Mistral3ForConditionalGeneration",
|
| 7 |
-
"parent_library": "transformers.models.mistral3.modeling_mistral3"
|
| 8 |
-
},
|
| 9 |
-
"base_model_name_or_path": "StandardThinking/StandardOne-8B",
|
| 10 |
-
"bias": "none",
|
| 11 |
-
"corda_config": null,
|
| 12 |
-
"ensure_weight_tying": false,
|
| 13 |
-
"eva_config": null,
|
| 14 |
-
"exclude_modules": ".*\\.(vision_tower|multi_modal_projector)\\..*",
|
| 15 |
-
"fan_in_fan_out": false,
|
| 16 |
-
"inference_mode": true,
|
| 17 |
-
"init_lora_weights": true,
|
| 18 |
-
"kasa_config": null,
|
| 19 |
-
"layer_replication": null,
|
| 20 |
-
"layers_pattern": null,
|
| 21 |
-
"layers_to_transform": null,
|
| 22 |
-
"loftq_config": {},
|
| 23 |
-
"lora_alpha": 128.0,
|
| 24 |
-
"lora_bias": false,
|
| 25 |
-
"lora_dropout": 0.0,
|
| 26 |
-
"lora_ga_config": null,
|
| 27 |
-
"megatron_config": null,
|
| 28 |
-
"megatron_core": "megatron.core",
|
| 29 |
-
"modules_to_save": null,
|
| 30 |
-
"monteclora_config": null,
|
| 31 |
-
"peft_type": "LORA",
|
| 32 |
-
"peft_version": "0.21.0",
|
| 33 |
-
"qalora_group_size": 16,
|
| 34 |
-
"r": 64,
|
| 35 |
-
"rank_pattern": {},
|
| 36 |
-
"revision": null,
|
| 37 |
-
"target_modules": [
|
| 38 |
-
"v_proj",
|
| 39 |
-
"k_proj",
|
| 40 |
-
"q_proj",
|
| 41 |
-
"gate_proj",
|
| 42 |
-
"down_proj",
|
| 43 |
-
"o_proj",
|
| 44 |
-
"up_proj"
|
| 45 |
-
],
|
| 46 |
-
"target_parameters": null,
|
| 47 |
-
"task_type": null,
|
| 48 |
-
"trainable_token_indices": null,
|
| 49 |
-
"use_bdlora": null,
|
| 50 |
-
"use_dora": false,
|
| 51 |
-
"use_qalora": false,
|
| 52 |
-
"use_rslora": false,
|
| 53 |
-
"velora_config": null
|
| 54 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
chat_template.jinja
ADDED
|
@@ -0,0 +1,189 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{#- Default system message if no system prompt is passed. #}
|
| 2 |
+
{%- set default_system_message = '' %}
|
| 3 |
+
|
| 4 |
+
{#- Begin of sequence token. #}
|
| 5 |
+
{{- bos_token }}
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
{#- Handle system prompt if it exists. #}
|
| 9 |
+
{%- set loop_messages = messages %}
|
| 10 |
+
{%- if messages[0]['role'] != 'system' and default_system_message != '' %}
|
| 11 |
+
{{- '[SYSTEM_PROMPT]' + default_system_message + '[/SYSTEM_PROMPT]' }}
|
| 12 |
+
{%- endif %}
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
{#- Tools definition #}
|
| 16 |
+
{%- set available_tools = '' %}
|
| 17 |
+
{%- set has_tools = false %}
|
| 18 |
+
{%- if tools is defined and tools is not none and tools|length > 0 %}
|
| 19 |
+
{%- set has_tools = true %}
|
| 20 |
+
{%- set available_tools = '[AVAILABLE_TOOLS]' + (tools| tojson) + '[/AVAILABLE_TOOLS]' %}
|
| 21 |
+
{%- endif %}
|
| 22 |
+
|
| 23 |
+
{#- Macros #}
|
| 24 |
+
{%- macro render_content(content, context_name, supported_types_desc, support_images) -%}
|
| 25 |
+
{%- if content is string -%}
|
| 26 |
+
{{- content -}}
|
| 27 |
+
{%- elif content -%}
|
| 28 |
+
{%- for block in content -%}
|
| 29 |
+
{%- if block['type'] == 'text' -%}
|
| 30 |
+
{{- block['text'] -}}
|
| 31 |
+
{%- elif support_images and block['type'] in ['image', 'image_url'] -%}
|
| 32 |
+
{{- '[IMG]' -}}
|
| 33 |
+
{%- else -%}
|
| 34 |
+
{{- raise_exception('Only ' + supported_types_desc + ' chunks are supported in ' + context_name + '.') -}}
|
| 35 |
+
{%- endif -%}
|
| 36 |
+
{%- endfor -%}
|
| 37 |
+
{%- else -%}
|
| 38 |
+
{{- raise_exception(context_name + ' must have non-empty content.') -}}
|
| 39 |
+
{%- endif -%}
|
| 40 |
+
{%- endmacro -%}
|
| 41 |
+
|
| 42 |
+
{#- Aggregate consecutive messages with the same role except system and tool. #}
|
| 43 |
+
{#- A sentinel message is appended so the last group gets flushed inside the loop. #}
|
| 44 |
+
{%- set ns_agg = namespace(messages=[], current_group=[], current_role=none) %}
|
| 45 |
+
{%- for message in loop_messages + [{'role': '__sentinel__'}] %}
|
| 46 |
+
{%- if message['role'] != ns_agg.current_role or message['role'] == 'system' or message['role'] == 'tool' %}
|
| 47 |
+
{%- if ns_agg.current_role == 'tool' %}
|
| 48 |
+
{%- set ns_agg.messages = ns_agg.messages + ns_agg.current_group %}
|
| 49 |
+
{%- elif ns_agg.current_role is not none %}
|
| 50 |
+
{%- set ns_c = namespace(text_parts=[], chunks=[], has_non_text=false, tool_calls=[]) %}
|
| 51 |
+
{%- for msg in ns_agg.current_group %}
|
| 52 |
+
{%- if msg['content'] is string %}
|
| 53 |
+
{%- set ns_c.text_parts = ns_c.text_parts + [msg['content']] %}
|
| 54 |
+
{%- elif msg['content'] is not none %}
|
| 55 |
+
{%- for block in msg['content'] %}
|
| 56 |
+
{%- if block['type'] == 'text' %}
|
| 57 |
+
{%- set ns_c.text_parts = ns_c.text_parts + [block['text']] %}
|
| 58 |
+
{%- else %}
|
| 59 |
+
{%- if ns_c.text_parts | length > 0 %}
|
| 60 |
+
{%- set ns_c.chunks = ns_c.chunks + [{'type': 'text', 'text': ns_c.text_parts | join('\n\n')}] %}
|
| 61 |
+
{%- set ns_c.text_parts = [] %}
|
| 62 |
+
{%- endif %}
|
| 63 |
+
{%- set ns_c.chunks = ns_c.chunks + [block] %}
|
| 64 |
+
{%- set ns_c.has_non_text = true %}
|
| 65 |
+
{%- endif %}
|
| 66 |
+
{%- endfor %}
|
| 67 |
+
{%- endif %}
|
| 68 |
+
{%- if msg['tool_calls'] is defined and msg['tool_calls'] is not none %}
|
| 69 |
+
{%- set ns_c.tool_calls = ns_c.tool_calls + msg['tool_calls'] | list %}
|
| 70 |
+
{%- endif %}
|
| 71 |
+
{%- endfor %}
|
| 72 |
+
{%- if ns_c.has_non_text %}
|
| 73 |
+
{%- if ns_c.text_parts | length > 0 %}
|
| 74 |
+
{%- set ns_c.chunks = ns_c.chunks + [{'type': 'text', 'text': ns_c.text_parts | join('\n\n')}] %}
|
| 75 |
+
{%- endif %}
|
| 76 |
+
{%- set merged_content = ns_c.chunks %}
|
| 77 |
+
{%- else %}
|
| 78 |
+
{%- set merged_content = ns_c.text_parts | join('\n\n') %}
|
| 79 |
+
{%- endif %}
|
| 80 |
+
{%- if ns_c.tool_calls | length > 0 %}
|
| 81 |
+
{%- set ns_agg.messages = ns_agg.messages + [{'role': ns_agg.current_role, 'content': merged_content, 'tool_calls': ns_c.tool_calls}] %}
|
| 82 |
+
{%- else %}
|
| 83 |
+
{%- set ns_agg.messages = ns_agg.messages + [{'role': ns_agg.current_role, 'content': merged_content}] %}
|
| 84 |
+
{%- endif %}
|
| 85 |
+
{%- endif %}
|
| 86 |
+
{%- if message['role'] != '__sentinel__' %}
|
| 87 |
+
{%- set ns_agg.current_group = [message] %}
|
| 88 |
+
{%- set ns_agg.current_role = message['role'] %}
|
| 89 |
+
{%- endif %}
|
| 90 |
+
{%- else %}
|
| 91 |
+
{%- set ns_agg.current_group = ns_agg.current_group + [message] %}
|
| 92 |
+
{%- endif %}
|
| 93 |
+
{%- endfor %}
|
| 94 |
+
{%- set loop_messages = ns_agg.messages %}
|
| 95 |
+
|
| 96 |
+
{#- Validates message ordering. #}
|
| 97 |
+
{%- set ns = namespace(available_tools_emitted=false) %}
|
| 98 |
+
{%- if loop_messages | length > 0 and loop_messages[0]['role'] not in ['user', 'system'] %}
|
| 99 |
+
{{- raise_exception('Conversation must start with a user or system message, got ' + loop_messages[0]['role'] + '.') }}
|
| 100 |
+
{%- endif %}
|
| 101 |
+
{%- set ns_order = namespace(previous_role=none) %}
|
| 102 |
+
{%- for message in loop_messages %}
|
| 103 |
+
{%- set current_role = message['role'] %}
|
| 104 |
+
{%- if ns_order.previous_role is not none %}
|
| 105 |
+
{%- if ns_order.previous_role == 'system' %}
|
| 106 |
+
{%- if current_role not in ['user', 'assistant', 'system'] %}
|
| 107 |
+
{{- raise_exception('Unexpected role \'' + current_role + '\' after role \'' + ns_order.previous_role + '\'') }}
|
| 108 |
+
{%- endif %}
|
| 109 |
+
{%- elif ns_order.previous_role == 'user' %}
|
| 110 |
+
{%- if current_role not in ['assistant', 'system', 'user'] %}
|
| 111 |
+
{{- raise_exception('Unexpected role \'' + current_role + '\' after role \'' + ns_order.previous_role + '\'') }}
|
| 112 |
+
{%- endif %}
|
| 113 |
+
{%- elif ns_order.previous_role == 'assistant' %}
|
| 114 |
+
{%- if current_role not in ['assistant', 'user', 'tool'] %}
|
| 115 |
+
{{- raise_exception('Unexpected role \'' + current_role + '\' after role \'' + ns_order.previous_role + '\'') }}
|
| 116 |
+
{%- endif %}
|
| 117 |
+
{%- elif ns_order.previous_role == 'tool' %}
|
| 118 |
+
{%- if current_role not in ['assistant', 'tool', 'user'] %}
|
| 119 |
+
{{- raise_exception('Unexpected role \'' + current_role + '\' after role \'' + ns_order.previous_role + '\'') }}
|
| 120 |
+
{%- endif %}
|
| 121 |
+
{%- endif %}
|
| 122 |
+
{%- endif %}
|
| 123 |
+
{%- set ns_order.previous_role = current_role %}
|
| 124 |
+
{%- endfor %}
|
| 125 |
+
|
| 126 |
+
{#- Handle conversation messages. #}
|
| 127 |
+
{%- for message in loop_messages %}
|
| 128 |
+
{#- User messages supports text, image and image_url content. #}
|
| 129 |
+
{%- if message['role'] == 'user' %}
|
| 130 |
+
{%- if not ns.available_tools_emitted %}
|
| 131 |
+
{{- available_tools }}
|
| 132 |
+
{%- set ns.available_tools_emitted = true %}
|
| 133 |
+
{%- endif %}
|
| 134 |
+
{%- if message['content'] is not string and message['content'] %}
|
| 135 |
+
{#- When content has exactly one image and one text block, put image first. #}
|
| 136 |
+
{%- if message['content'] | length == 2 and message['content'][0]['type'] == 'text' and message['content'][1]['type'] in ['image', 'image_url'] %}
|
| 137 |
+
{%- set blocks = [message['content'][1], message['content'][0]] %}
|
| 138 |
+
{%- else %}
|
| 139 |
+
{%- set blocks = message['content'] %}
|
| 140 |
+
{%- endif %}
|
| 141 |
+
{%- set user_content = blocks %}
|
| 142 |
+
{%- else %}
|
| 143 |
+
{%- set user_content = message['content'] %}
|
| 144 |
+
{%- endif %}
|
| 145 |
+
{{- '[INST]' -}}
|
| 146 |
+
{{- render_content(user_content, 'user message content', supported_types_desc='text, image and image_url', support_images=true) -}}
|
| 147 |
+
{{- '[/INST]' }}
|
| 148 |
+
|
| 149 |
+
{#- Assistant messages supports text content. #}
|
| 150 |
+
{%- elif message['role'] == 'assistant' %}
|
| 151 |
+
{%- if (message['content'] is none or message['content'] == '' or message['content']|length == 0) and (message['tool_calls'] is not defined or message['tool_calls'] is none or message['tool_calls']|length == 0) %}
|
| 152 |
+
{{- raise_exception('Assistant message must have a string or a list of chunks in content or a list of tool calls.') }}
|
| 153 |
+
{%- endif %}
|
| 154 |
+
|
| 155 |
+
{%- if message['content'] %}
|
| 156 |
+
{{- render_content(message['content'], 'assistant message contents', supported_types_desc='text', support_images=false) -}}
|
| 157 |
+
{%- endif %}
|
| 158 |
+
|
| 159 |
+
{%- if message['tool_calls'] is defined and message['tool_calls'] is not none and message['tool_calls']|length > 0 %}
|
| 160 |
+
{%- for tool in message['tool_calls'] %}
|
| 161 |
+
{{- '[TOOL_CALLS]' }}
|
| 162 |
+
{%- set name = tool['function']['name'] %}
|
| 163 |
+
{%- set arguments = tool['function']['arguments'] %}
|
| 164 |
+
{%- if arguments is not string %}
|
| 165 |
+
{%- set arguments = arguments|tojson|safe %}
|
| 166 |
+
{%- elif arguments == '' %}
|
| 167 |
+
{%- set arguments = '{}' %}
|
| 168 |
+
{%- endif %}
|
| 169 |
+
{{- name + '[ARGS]' + arguments }}
|
| 170 |
+
{%- endfor %}
|
| 171 |
+
{%- endif %}
|
| 172 |
+
|
| 173 |
+
{{- eos_token }}
|
| 174 |
+
|
| 175 |
+
{#- Tool messages only supports text content. #}
|
| 176 |
+
{%- elif message['role'] == 'tool' %}
|
| 177 |
+
{{- '[TOOL_RESULTS]' + message['content']|string + '[/TOOL_RESULTS]' }}
|
| 178 |
+
|
| 179 |
+
{#- System messages. #}
|
| 180 |
+
{%- elif message['role'] == 'system' %}
|
| 181 |
+
{{- '[SYSTEM_PROMPT]' -}}
|
| 182 |
+
{{- render_content(message['content'], 'system message contents', supported_types_desc='text', support_images=false) -}}
|
| 183 |
+
{{- '[/SYSTEM_PROMPT]' -}}
|
| 184 |
+
|
| 185 |
+
{#- Raise exception for unsupported roles. #}
|
| 186 |
+
{%- else %}
|
| 187 |
+
{{- raise_exception('Only user, assistant, system and tool roles are supported, got ' + message['role'] + '.') }}
|
| 188 |
+
{%- endif %}
|
| 189 |
+
{%- endfor %}
|
code/render.py
CHANGED
|
@@ -16,11 +16,14 @@ Rendering may show them in another order (augmentation); spans are always return
|
|
| 16 |
Encoding (``Encoder.encode``) returns token ids plus token spans. Spans are half-open [start, end) token indices.
|
| 17 |
"""
|
| 18 |
import hashlib
|
|
|
|
|
|
|
| 19 |
import json
|
| 20 |
import math
|
| 21 |
import random
|
| 22 |
|
| 23 |
-
|
|
|
|
| 24 |
MAX_LENGTH = 32768
|
| 25 |
MAX_OPTIONS = 255
|
| 26 |
MAX_QUESTIONS = 256
|
|
@@ -199,6 +202,12 @@ class Encoder:
|
|
| 199 |
self.image_token_id = image_token_id
|
| 200 |
self.image_end_id = image_end_id
|
| 201 |
self.special_ids = set(getattr(tokenizer, "all_special_ids", []) or [])
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 202 |
|
| 203 |
@classmethod
|
| 204 |
def from_snapshot(cls, snapshot, with_processor=True, max_length=MAX_LENGTH, mistral_regex_fix=True):
|
|
@@ -226,10 +235,12 @@ class Encoder:
|
|
| 226 |
# no enable_thinking here: the Ministral template has no such variable, and the processor's
|
| 227 |
# apply_chat_template treats unknown kwargs as processor kwargs (warning on every image encode). The text
|
| 228 |
# is identical either way (tests/test_image_batch.py::test_processor_kwargs_accepted).
|
| 229 |
-
|
|
|
|
| 230 |
messages = [{"role": "user", "content": content}]
|
|
|
|
| 231 |
return self.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True,
|
| 232 |
-
enable_thinking=False), messages
|
| 233 |
|
| 234 |
def encode(self, req, wording="W0", question_order=None, option_orders=None, preview=True, pil_images=None):
|
| 235 |
"""Returns a dict (all plain Python ints/lists):
|
|
@@ -248,7 +259,8 @@ class Encoder:
|
|
| 248 |
if self.check_roundtrip and not images:
|
| 249 |
ref = self.tokenizer.apply_chat_template([{"role": "user", "content": content}], tokenize=True,
|
| 250 |
add_generation_prompt=True, enable_thinking=False,
|
| 251 |
-
return_dict=False
|
|
|
|
| 252 |
require(list(ref) == ids, "roundtrip_mismatch")
|
| 253 |
image_span = None
|
| 254 |
pixel = None
|
|
|
|
| 16 |
Encoding (``Encoder.encode``) returns token ids plus token spans. Spans are half-open [start, end) token indices.
|
| 17 |
"""
|
| 18 |
import hashlib
|
| 19 |
+
import os
|
| 20 |
+
import re
|
| 21 |
import json
|
| 22 |
import math
|
| 23 |
import random
|
| 24 |
|
| 25 |
+
SYSTEM_PROMPT = os.environ.get("SH_SYSTEM_PROMPT", "none") # this model was trained without the template's default system prompt; "default" keeps it
|
| 26 |
+
TEMPLATE_ID = "schema-v1" if SYSTEM_PROMPT == "default" else "schema-v1-nosys"
|
| 27 |
MAX_LENGTH = 32768
|
| 28 |
MAX_OPTIONS = 255
|
| 29 |
MAX_QUESTIONS = 256
|
|
|
|
| 202 |
self.image_token_id = image_token_id
|
| 203 |
self.image_end_id = image_end_id
|
| 204 |
self.special_ids = set(getattr(tokenizer, "all_special_ids", []) or [])
|
| 205 |
+
self.chat_template = None
|
| 206 |
+
if SYSTEM_PROMPT == "none":
|
| 207 |
+
tpl = tokenizer.chat_template
|
| 208 |
+
new, n = re.subn(r"set default_system_message = '(?:[^'\\]|\\.)*'", "set default_system_message = ''", tpl)
|
| 209 |
+
require(n == 1, "no_default_system_message_in_template")
|
| 210 |
+
self.chat_template = new
|
| 211 |
|
| 212 |
@classmethod
|
| 213 |
def from_snapshot(cls, snapshot, with_processor=True, max_length=MAX_LENGTH, mistral_regex_fix=True):
|
|
|
|
| 235 |
# no enable_thinking here: the Ministral template has no such variable, and the processor's
|
| 236 |
# apply_chat_template treats unknown kwargs as processor kwargs (warning on every image encode). The text
|
| 237 |
# is identical either way (tests/test_image_batch.py::test_processor_kwargs_accepted).
|
| 238 |
+
kw = {"chat_template": self.chat_template} if self.chat_template else {}
|
| 239 |
+
return self.processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, **kw), messages
|
| 240 |
messages = [{"role": "user", "content": content}]
|
| 241 |
+
kw = {"chat_template": self.chat_template} if self.chat_template else {}
|
| 242 |
return self.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True,
|
| 243 |
+
enable_thinking=False, **kw), messages
|
| 244 |
|
| 245 |
def encode(self, req, wording="W0", question_order=None, option_orders=None, preview=True, pil_images=None):
|
| 246 |
"""Returns a dict (all plain Python ints/lists):
|
|
|
|
| 259 |
if self.check_roundtrip and not images:
|
| 260 |
ref = self.tokenizer.apply_chat_template([{"role": "user", "content": content}], tokenize=True,
|
| 261 |
add_generation_prompt=True, enable_thinking=False,
|
| 262 |
+
return_dict=False,
|
| 263 |
+
**({"chat_template": self.chat_template} if self.chat_template else {}))
|
| 264 |
require(list(ref) == ids, "roundtrip_mismatch")
|
| 265 |
image_span = None
|
| 266 |
pixel = None
|
config.json
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Mistral3ForConditionalGeneration"
|
| 4 |
+
],
|
| 5 |
+
"dtype": "bfloat16",
|
| 6 |
+
"image_token_index": 10,
|
| 7 |
+
"model_type": "mistral3",
|
| 8 |
+
"multimodal_projector_bias": false,
|
| 9 |
+
"projector_hidden_act": "gelu",
|
| 10 |
+
"spatial_merge_size": 2,
|
| 11 |
+
"text_config": {
|
| 12 |
+
"attention_dropout": 0.0,
|
| 13 |
+
"bos_token_id": 1,
|
| 14 |
+
"dtype": "float32",
|
| 15 |
+
"eos_token_id": 2,
|
| 16 |
+
"head_dim": 128,
|
| 17 |
+
"hidden_act": "silu",
|
| 18 |
+
"hidden_size": 4096,
|
| 19 |
+
"initializer_range": 0.02,
|
| 20 |
+
"intermediate_size": 14336,
|
| 21 |
+
"max_position_embeddings": 262144,
|
| 22 |
+
"model_type": "ministral3",
|
| 23 |
+
"num_attention_heads": 32,
|
| 24 |
+
"num_hidden_layers": 34,
|
| 25 |
+
"num_key_value_heads": 8,
|
| 26 |
+
"pad_token_id": 11,
|
| 27 |
+
"rms_norm_eps": 1e-05,
|
| 28 |
+
"rope_parameters": {
|
| 29 |
+
"beta_fast": 32.0,
|
| 30 |
+
"beta_slow": 1.0,
|
| 31 |
+
"factor": 16.0,
|
| 32 |
+
"llama_4_scaling_beta": 0.1,
|
| 33 |
+
"mscale": 1.0,
|
| 34 |
+
"mscale_all_dim": 1.0,
|
| 35 |
+
"original_max_position_embeddings": 16384,
|
| 36 |
+
"rope_theta": 1000000.0,
|
| 37 |
+
"rope_type": "yarn",
|
| 38 |
+
"type": "yarn"
|
| 39 |
+
},
|
| 40 |
+
"sliding_window": null,
|
| 41 |
+
"tie_word_embeddings": false,
|
| 42 |
+
"use_cache": true,
|
| 43 |
+
"vocab_size": 131072
|
| 44 |
+
},
|
| 45 |
+
"tie_word_embeddings": false,
|
| 46 |
+
"transformers_version": "5.12.1",
|
| 47 |
+
"vision_config": {
|
| 48 |
+
"attention_dropout": 0.0,
|
| 49 |
+
"dtype": "float32",
|
| 50 |
+
"head_dim": 64,
|
| 51 |
+
"hidden_act": "silu",
|
| 52 |
+
"hidden_size": 1024,
|
| 53 |
+
"image_size": 1540,
|
| 54 |
+
"initializer_range": 0.02,
|
| 55 |
+
"intermediate_size": 4096,
|
| 56 |
+
"model_type": "pixtral",
|
| 57 |
+
"num_attention_heads": 16,
|
| 58 |
+
"num_channels": 3,
|
| 59 |
+
"num_hidden_layers": 24,
|
| 60 |
+
"patch_size": 14,
|
| 61 |
+
"rope_parameters": {
|
| 62 |
+
"rope_theta": 10000.0,
|
| 63 |
+
"rope_type": "default"
|
| 64 |
+
}
|
| 65 |
+
},
|
| 66 |
+
"vision_feature_layer": -1
|
| 67 |
+
}
|
generation_config.json
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token_id": 1,
|
| 3 |
+
"eos_token_id": 2,
|
| 4 |
+
"max_length": 262144,
|
| 5 |
+
"pad_token_id": 11,
|
| 6 |
+
"transformers_version": "5.0.0.dev0"
|
| 7 |
+
}
|
head/schema_head.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 282265276
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1b4bf756cb7c2413b30f2f34a6bab6578d6ccabc0cfd25b77f39b6af94469e14
|
| 3 |
size 282265276
|
head/schema_head_config.json
CHANGED
|
@@ -22,10 +22,9 @@
|
|
| 22 |
"final",
|
| 23 |
25
|
| 24 |
],
|
| 25 |
-
"template": "schema-v1",
|
| 26 |
-
"step":
|
| 27 |
"lora_rank": 64,
|
| 28 |
-
"snapshot": "StandardThinking/StandardOne-8B",
|
| 29 |
-
"preview": true
|
| 30 |
-
"snapshot_revision": "ac495d1c221297f0ae951941eff27bcf58b5ce52"
|
| 31 |
}
|
|
|
|
| 22 |
"final",
|
| 23 |
25
|
| 24 |
],
|
| 25 |
+
"template": "schema-v1-nosys",
|
| 26 |
+
"step": 2077,
|
| 27 |
"lora_rank": 64,
|
| 28 |
+
"snapshot": "StandardThinking/StandardOne-8B-SH",
|
| 29 |
+
"preview": true
|
|
|
|
| 30 |
}
|
model-00001-of-00004.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4f2ca3c4f8c6f86466c4edab8cf4ca8f3ef2fd5b5859f81387de5d8f8b86111a
|
| 3 |
+
size 4999724576
|
model-00002-of-00004.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a86699c59416f085442224865ee84f7f305332a712956b210cb0b4e915b1d5f8
|
| 3 |
+
size 4999820896
|
model-00003-of-00004.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9a5cbf1aa0b7b72a92303020a2752279010e71c6745a53f36e65304e229fd67e
|
| 3 |
+
size 4915917688
|
model-00004-of-00004.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:70a8726a628117450473df8e8db53982a6d8bfb6df7aa337bb59d0fcb7068394
|
| 3 |
+
size 2920659992
|
model.safetensors.index.json
ADDED
|
@@ -0,0 +1,539 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"metadata": {
|
| 3 |
+
"total_parameters": 8918026240,
|
| 4 |
+
"total_size": 17836052480
|
| 5 |
+
},
|
| 6 |
+
"weight_map": {
|
| 7 |
+
"language_model.lm_head.weight": "model-00001-of-00004.safetensors",
|
| 8 |
+
"language_model.model.embed_tokens.weight": "model-00001-of-00004.safetensors",
|
| 9 |
+
"language_model.model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 10 |
+
"language_model.model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
| 11 |
+
"language_model.model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
| 12 |
+
"language_model.model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
| 13 |
+
"language_model.model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 14 |
+
"language_model.model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
| 15 |
+
"language_model.model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
| 16 |
+
"language_model.model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
| 17 |
+
"language_model.model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
| 18 |
+
"language_model.model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 19 |
+
"language_model.model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
| 20 |
+
"language_model.model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
| 21 |
+
"language_model.model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
| 22 |
+
"language_model.model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 23 |
+
"language_model.model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
| 24 |
+
"language_model.model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
| 25 |
+
"language_model.model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
| 26 |
+
"language_model.model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
| 27 |
+
"language_model.model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 28 |
+
"language_model.model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
| 29 |
+
"language_model.model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
| 30 |
+
"language_model.model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 31 |
+
"language_model.model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 32 |
+
"language_model.model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 33 |
+
"language_model.model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 34 |
+
"language_model.model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 35 |
+
"language_model.model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 36 |
+
"language_model.model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 37 |
+
"language_model.model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
| 38 |
+
"language_model.model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
| 39 |
+
"language_model.model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 40 |
+
"language_model.model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 41 |
+
"language_model.model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 42 |
+
"language_model.model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 43 |
+
"language_model.model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 44 |
+
"language_model.model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 45 |
+
"language_model.model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 46 |
+
"language_model.model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
| 47 |
+
"language_model.model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
| 48 |
+
"language_model.model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 49 |
+
"language_model.model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 50 |
+
"language_model.model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 51 |
+
"language_model.model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 52 |
+
"language_model.model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 53 |
+
"language_model.model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 54 |
+
"language_model.model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 55 |
+
"language_model.model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
| 56 |
+
"language_model.model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
| 57 |
+
"language_model.model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 58 |
+
"language_model.model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 59 |
+
"language_model.model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 60 |
+
"language_model.model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 61 |
+
"language_model.model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 62 |
+
"language_model.model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 63 |
+
"language_model.model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 64 |
+
"language_model.model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
| 65 |
+
"language_model.model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
| 66 |
+
"language_model.model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 67 |
+
"language_model.model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 68 |
+
"language_model.model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 69 |
+
"language_model.model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 70 |
+
"language_model.model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 71 |
+
"language_model.model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 72 |
+
"language_model.model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 73 |
+
"language_model.model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
| 74 |
+
"language_model.model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
| 75 |
+
"language_model.model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 76 |
+
"language_model.model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 77 |
+
"language_model.model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 78 |
+
"language_model.model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 79 |
+
"language_model.model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 80 |
+
"language_model.model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 81 |
+
"language_model.model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 82 |
+
"language_model.model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
| 83 |
+
"language_model.model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
| 84 |
+
"language_model.model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 85 |
+
"language_model.model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 86 |
+
"language_model.model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 87 |
+
"language_model.model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 88 |
+
"language_model.model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 89 |
+
"language_model.model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 90 |
+
"language_model.model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 91 |
+
"language_model.model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
| 92 |
+
"language_model.model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
| 93 |
+
"language_model.model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 94 |
+
"language_model.model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 95 |
+
"language_model.model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 96 |
+
"language_model.model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 97 |
+
"language_model.model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 98 |
+
"language_model.model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 99 |
+
"language_model.model.layers.18.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 100 |
+
"language_model.model.layers.18.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 101 |
+
"language_model.model.layers.18.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
| 102 |
+
"language_model.model.layers.18.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
| 103 |
+
"language_model.model.layers.18.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 104 |
+
"language_model.model.layers.18.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
| 105 |
+
"language_model.model.layers.18.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
| 106 |
+
"language_model.model.layers.18.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
| 107 |
+
"language_model.model.layers.18.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
| 108 |
+
"language_model.model.layers.19.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 109 |
+
"language_model.model.layers.19.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 110 |
+
"language_model.model.layers.19.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
| 111 |
+
"language_model.model.layers.19.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
| 112 |
+
"language_model.model.layers.19.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 113 |
+
"language_model.model.layers.19.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
| 114 |
+
"language_model.model.layers.19.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
| 115 |
+
"language_model.model.layers.19.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
| 116 |
+
"language_model.model.layers.19.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
| 117 |
+
"language_model.model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 118 |
+
"language_model.model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
| 119 |
+
"language_model.model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
| 120 |
+
"language_model.model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
| 121 |
+
"language_model.model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 122 |
+
"language_model.model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
| 123 |
+
"language_model.model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
| 124 |
+
"language_model.model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
| 125 |
+
"language_model.model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
| 126 |
+
"language_model.model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 127 |
+
"language_model.model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 128 |
+
"language_model.model.layers.20.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
| 129 |
+
"language_model.model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
| 130 |
+
"language_model.model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 131 |
+
"language_model.model.layers.20.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
| 132 |
+
"language_model.model.layers.20.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
| 133 |
+
"language_model.model.layers.20.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
| 134 |
+
"language_model.model.layers.20.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
| 135 |
+
"language_model.model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 136 |
+
"language_model.model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 137 |
+
"language_model.model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
| 138 |
+
"language_model.model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
| 139 |
+
"language_model.model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 140 |
+
"language_model.model.layers.21.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
| 141 |
+
"language_model.model.layers.21.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
| 142 |
+
"language_model.model.layers.21.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
| 143 |
+
"language_model.model.layers.21.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
| 144 |
+
"language_model.model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 145 |
+
"language_model.model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 146 |
+
"language_model.model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
| 147 |
+
"language_model.model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
| 148 |
+
"language_model.model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 149 |
+
"language_model.model.layers.22.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
| 150 |
+
"language_model.model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
| 151 |
+
"language_model.model.layers.22.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
| 152 |
+
"language_model.model.layers.22.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
| 153 |
+
"language_model.model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 154 |
+
"language_model.model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 155 |
+
"language_model.model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
| 156 |
+
"language_model.model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
| 157 |
+
"language_model.model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 158 |
+
"language_model.model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
| 159 |
+
"language_model.model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
| 160 |
+
"language_model.model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
| 161 |
+
"language_model.model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
| 162 |
+
"language_model.model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 163 |
+
"language_model.model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 164 |
+
"language_model.model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
| 165 |
+
"language_model.model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
| 166 |
+
"language_model.model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 167 |
+
"language_model.model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
| 168 |
+
"language_model.model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
| 169 |
+
"language_model.model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
| 170 |
+
"language_model.model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
| 171 |
+
"language_model.model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 172 |
+
"language_model.model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 173 |
+
"language_model.model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
| 174 |
+
"language_model.model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
| 175 |
+
"language_model.model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 176 |
+
"language_model.model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
| 177 |
+
"language_model.model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
| 178 |
+
"language_model.model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
| 179 |
+
"language_model.model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
| 180 |
+
"language_model.model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 181 |
+
"language_model.model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 182 |
+
"language_model.model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
| 183 |
+
"language_model.model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
| 184 |
+
"language_model.model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 185 |
+
"language_model.model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
| 186 |
+
"language_model.model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
| 187 |
+
"language_model.model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
| 188 |
+
"language_model.model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
| 189 |
+
"language_model.model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 190 |
+
"language_model.model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 191 |
+
"language_model.model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
| 192 |
+
"language_model.model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
| 193 |
+
"language_model.model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 194 |
+
"language_model.model.layers.27.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
| 195 |
+
"language_model.model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
| 196 |
+
"language_model.model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
| 197 |
+
"language_model.model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
| 198 |
+
"language_model.model.layers.28.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 199 |
+
"language_model.model.layers.28.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 200 |
+
"language_model.model.layers.28.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
| 201 |
+
"language_model.model.layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
| 202 |
+
"language_model.model.layers.28.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 203 |
+
"language_model.model.layers.28.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
| 204 |
+
"language_model.model.layers.28.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
| 205 |
+
"language_model.model.layers.28.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
| 206 |
+
"language_model.model.layers.28.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
| 207 |
+
"language_model.model.layers.29.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
| 208 |
+
"language_model.model.layers.29.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
| 209 |
+
"language_model.model.layers.29.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 210 |
+
"language_model.model.layers.29.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 211 |
+
"language_model.model.layers.29.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
| 212 |
+
"language_model.model.layers.29.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 213 |
+
"language_model.model.layers.29.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 214 |
+
"language_model.model.layers.29.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 215 |
+
"language_model.model.layers.29.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 216 |
+
"language_model.model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 217 |
+
"language_model.model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
| 218 |
+
"language_model.model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
| 219 |
+
"language_model.model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
| 220 |
+
"language_model.model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 221 |
+
"language_model.model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
| 222 |
+
"language_model.model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
| 223 |
+
"language_model.model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
| 224 |
+
"language_model.model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
| 225 |
+
"language_model.model.layers.30.input_layernorm.weight": "model-00004-of-00004.safetensors",
|
| 226 |
+
"language_model.model.layers.30.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 227 |
+
"language_model.model.layers.30.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 228 |
+
"language_model.model.layers.30.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 229 |
+
"language_model.model.layers.30.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
| 230 |
+
"language_model.model.layers.30.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 231 |
+
"language_model.model.layers.30.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 232 |
+
"language_model.model.layers.30.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 233 |
+
"language_model.model.layers.30.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 234 |
+
"language_model.model.layers.31.input_layernorm.weight": "model-00004-of-00004.safetensors",
|
| 235 |
+
"language_model.model.layers.31.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 236 |
+
"language_model.model.layers.31.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 237 |
+
"language_model.model.layers.31.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 238 |
+
"language_model.model.layers.31.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
| 239 |
+
"language_model.model.layers.31.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 240 |
+
"language_model.model.layers.31.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 241 |
+
"language_model.model.layers.31.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 242 |
+
"language_model.model.layers.31.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 243 |
+
"language_model.model.layers.32.input_layernorm.weight": "model-00004-of-00004.safetensors",
|
| 244 |
+
"language_model.model.layers.32.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 245 |
+
"language_model.model.layers.32.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 246 |
+
"language_model.model.layers.32.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 247 |
+
"language_model.model.layers.32.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
| 248 |
+
"language_model.model.layers.32.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 249 |
+
"language_model.model.layers.32.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 250 |
+
"language_model.model.layers.32.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 251 |
+
"language_model.model.layers.32.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 252 |
+
"language_model.model.layers.33.input_layernorm.weight": "model-00004-of-00004.safetensors",
|
| 253 |
+
"language_model.model.layers.33.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 254 |
+
"language_model.model.layers.33.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 255 |
+
"language_model.model.layers.33.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 256 |
+
"language_model.model.layers.33.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
| 257 |
+
"language_model.model.layers.33.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 258 |
+
"language_model.model.layers.33.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 259 |
+
"language_model.model.layers.33.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 260 |
+
"language_model.model.layers.33.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 261 |
+
"language_model.model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 262 |
+
"language_model.model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
| 263 |
+
"language_model.model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
| 264 |
+
"language_model.model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
| 265 |
+
"language_model.model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 266 |
+
"language_model.model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
| 267 |
+
"language_model.model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
| 268 |
+
"language_model.model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
| 269 |
+
"language_model.model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
| 270 |
+
"language_model.model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 271 |
+
"language_model.model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
| 272 |
+
"language_model.model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
| 273 |
+
"language_model.model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
| 274 |
+
"language_model.model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 275 |
+
"language_model.model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
| 276 |
+
"language_model.model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
| 277 |
+
"language_model.model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
| 278 |
+
"language_model.model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
| 279 |
+
"language_model.model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
| 280 |
+
"language_model.model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
| 281 |
+
"language_model.model.layers.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
| 282 |
+
"language_model.model.layers.6.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 283 |
+
"language_model.model.layers.6.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 284 |
+
"language_model.model.layers.6.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 285 |
+
"language_model.model.layers.6.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 286 |
+
"language_model.model.layers.6.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 287 |
+
"language_model.model.layers.6.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 288 |
+
"language_model.model.layers.7.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 289 |
+
"language_model.model.layers.7.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
| 290 |
+
"language_model.model.layers.7.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
| 291 |
+
"language_model.model.layers.7.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 292 |
+
"language_model.model.layers.7.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 293 |
+
"language_model.model.layers.7.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 294 |
+
"language_model.model.layers.7.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 295 |
+
"language_model.model.layers.7.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 296 |
+
"language_model.model.layers.7.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 297 |
+
"language_model.model.layers.8.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 298 |
+
"language_model.model.layers.8.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
| 299 |
+
"language_model.model.layers.8.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
| 300 |
+
"language_model.model.layers.8.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 301 |
+
"language_model.model.layers.8.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 302 |
+
"language_model.model.layers.8.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 303 |
+
"language_model.model.layers.8.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 304 |
+
"language_model.model.layers.8.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 305 |
+
"language_model.model.layers.8.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 306 |
+
"language_model.model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 307 |
+
"language_model.model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
| 308 |
+
"language_model.model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
| 309 |
+
"language_model.model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
| 310 |
+
"language_model.model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
| 311 |
+
"language_model.model.layers.9.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
| 312 |
+
"language_model.model.layers.9.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
| 313 |
+
"language_model.model.layers.9.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
| 314 |
+
"language_model.model.layers.9.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
| 315 |
+
"language_model.model.norm.weight": "model-00004-of-00004.safetensors",
|
| 316 |
+
"multi_modal_projector.linear_1.weight": "model-00004-of-00004.safetensors",
|
| 317 |
+
"multi_modal_projector.linear_2.weight": "model-00004-of-00004.safetensors",
|
| 318 |
+
"multi_modal_projector.norm.weight": "model-00004-of-00004.safetensors",
|
| 319 |
+
"multi_modal_projector.patch_merger.merging_layer.weight": "model-00004-of-00004.safetensors",
|
| 320 |
+
"vision_tower.ln_pre.weight": "model-00004-of-00004.safetensors",
|
| 321 |
+
"vision_tower.patch_conv.weight": "model-00004-of-00004.safetensors",
|
| 322 |
+
"vision_tower.transformer.layers.0.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 323 |
+
"vision_tower.transformer.layers.0.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 324 |
+
"vision_tower.transformer.layers.0.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 325 |
+
"vision_tower.transformer.layers.0.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 326 |
+
"vision_tower.transformer.layers.0.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 327 |
+
"vision_tower.transformer.layers.0.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 328 |
+
"vision_tower.transformer.layers.0.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 329 |
+
"vision_tower.transformer.layers.0.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 330 |
+
"vision_tower.transformer.layers.0.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 331 |
+
"vision_tower.transformer.layers.1.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 332 |
+
"vision_tower.transformer.layers.1.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 333 |
+
"vision_tower.transformer.layers.1.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 334 |
+
"vision_tower.transformer.layers.1.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 335 |
+
"vision_tower.transformer.layers.1.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 336 |
+
"vision_tower.transformer.layers.1.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 337 |
+
"vision_tower.transformer.layers.1.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 338 |
+
"vision_tower.transformer.layers.1.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 339 |
+
"vision_tower.transformer.layers.1.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 340 |
+
"vision_tower.transformer.layers.10.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 341 |
+
"vision_tower.transformer.layers.10.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 342 |
+
"vision_tower.transformer.layers.10.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 343 |
+
"vision_tower.transformer.layers.10.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 344 |
+
"vision_tower.transformer.layers.10.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 345 |
+
"vision_tower.transformer.layers.10.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 346 |
+
"vision_tower.transformer.layers.10.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 347 |
+
"vision_tower.transformer.layers.10.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 348 |
+
"vision_tower.transformer.layers.10.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 349 |
+
"vision_tower.transformer.layers.11.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 350 |
+
"vision_tower.transformer.layers.11.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 351 |
+
"vision_tower.transformer.layers.11.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 352 |
+
"vision_tower.transformer.layers.11.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 353 |
+
"vision_tower.transformer.layers.11.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 354 |
+
"vision_tower.transformer.layers.11.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 355 |
+
"vision_tower.transformer.layers.11.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 356 |
+
"vision_tower.transformer.layers.11.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 357 |
+
"vision_tower.transformer.layers.11.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 358 |
+
"vision_tower.transformer.layers.12.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 359 |
+
"vision_tower.transformer.layers.12.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 360 |
+
"vision_tower.transformer.layers.12.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 361 |
+
"vision_tower.transformer.layers.12.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 362 |
+
"vision_tower.transformer.layers.12.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 363 |
+
"vision_tower.transformer.layers.12.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 364 |
+
"vision_tower.transformer.layers.12.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 365 |
+
"vision_tower.transformer.layers.12.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 366 |
+
"vision_tower.transformer.layers.12.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 367 |
+
"vision_tower.transformer.layers.13.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 368 |
+
"vision_tower.transformer.layers.13.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 369 |
+
"vision_tower.transformer.layers.13.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 370 |
+
"vision_tower.transformer.layers.13.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 371 |
+
"vision_tower.transformer.layers.13.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 372 |
+
"vision_tower.transformer.layers.13.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 373 |
+
"vision_tower.transformer.layers.13.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 374 |
+
"vision_tower.transformer.layers.13.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 375 |
+
"vision_tower.transformer.layers.13.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 376 |
+
"vision_tower.transformer.layers.14.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 377 |
+
"vision_tower.transformer.layers.14.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 378 |
+
"vision_tower.transformer.layers.14.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 379 |
+
"vision_tower.transformer.layers.14.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 380 |
+
"vision_tower.transformer.layers.14.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 381 |
+
"vision_tower.transformer.layers.14.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 382 |
+
"vision_tower.transformer.layers.14.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 383 |
+
"vision_tower.transformer.layers.14.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 384 |
+
"vision_tower.transformer.layers.14.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 385 |
+
"vision_tower.transformer.layers.15.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 386 |
+
"vision_tower.transformer.layers.15.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 387 |
+
"vision_tower.transformer.layers.15.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 388 |
+
"vision_tower.transformer.layers.15.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 389 |
+
"vision_tower.transformer.layers.15.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 390 |
+
"vision_tower.transformer.layers.15.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 391 |
+
"vision_tower.transformer.layers.15.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 392 |
+
"vision_tower.transformer.layers.15.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 393 |
+
"vision_tower.transformer.layers.15.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 394 |
+
"vision_tower.transformer.layers.16.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 395 |
+
"vision_tower.transformer.layers.16.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 396 |
+
"vision_tower.transformer.layers.16.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 397 |
+
"vision_tower.transformer.layers.16.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 398 |
+
"vision_tower.transformer.layers.16.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 399 |
+
"vision_tower.transformer.layers.16.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 400 |
+
"vision_tower.transformer.layers.16.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 401 |
+
"vision_tower.transformer.layers.16.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 402 |
+
"vision_tower.transformer.layers.16.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 403 |
+
"vision_tower.transformer.layers.17.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 404 |
+
"vision_tower.transformer.layers.17.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 405 |
+
"vision_tower.transformer.layers.17.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 406 |
+
"vision_tower.transformer.layers.17.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 407 |
+
"vision_tower.transformer.layers.17.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 408 |
+
"vision_tower.transformer.layers.17.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 409 |
+
"vision_tower.transformer.layers.17.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 410 |
+
"vision_tower.transformer.layers.17.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 411 |
+
"vision_tower.transformer.layers.17.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 412 |
+
"vision_tower.transformer.layers.18.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 413 |
+
"vision_tower.transformer.layers.18.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 414 |
+
"vision_tower.transformer.layers.18.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 415 |
+
"vision_tower.transformer.layers.18.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 416 |
+
"vision_tower.transformer.layers.18.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 417 |
+
"vision_tower.transformer.layers.18.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 418 |
+
"vision_tower.transformer.layers.18.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 419 |
+
"vision_tower.transformer.layers.18.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 420 |
+
"vision_tower.transformer.layers.18.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 421 |
+
"vision_tower.transformer.layers.19.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 422 |
+
"vision_tower.transformer.layers.19.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 423 |
+
"vision_tower.transformer.layers.19.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 424 |
+
"vision_tower.transformer.layers.19.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 425 |
+
"vision_tower.transformer.layers.19.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 426 |
+
"vision_tower.transformer.layers.19.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 427 |
+
"vision_tower.transformer.layers.19.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 428 |
+
"vision_tower.transformer.layers.19.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 429 |
+
"vision_tower.transformer.layers.19.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 430 |
+
"vision_tower.transformer.layers.2.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 431 |
+
"vision_tower.transformer.layers.2.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 432 |
+
"vision_tower.transformer.layers.2.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 433 |
+
"vision_tower.transformer.layers.2.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 434 |
+
"vision_tower.transformer.layers.2.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 435 |
+
"vision_tower.transformer.layers.2.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 436 |
+
"vision_tower.transformer.layers.2.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 437 |
+
"vision_tower.transformer.layers.2.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 438 |
+
"vision_tower.transformer.layers.2.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 439 |
+
"vision_tower.transformer.layers.20.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 440 |
+
"vision_tower.transformer.layers.20.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 441 |
+
"vision_tower.transformer.layers.20.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 442 |
+
"vision_tower.transformer.layers.20.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 443 |
+
"vision_tower.transformer.layers.20.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 444 |
+
"vision_tower.transformer.layers.20.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 445 |
+
"vision_tower.transformer.layers.20.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 446 |
+
"vision_tower.transformer.layers.20.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 447 |
+
"vision_tower.transformer.layers.20.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 448 |
+
"vision_tower.transformer.layers.21.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 449 |
+
"vision_tower.transformer.layers.21.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 450 |
+
"vision_tower.transformer.layers.21.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 451 |
+
"vision_tower.transformer.layers.21.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 452 |
+
"vision_tower.transformer.layers.21.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 453 |
+
"vision_tower.transformer.layers.21.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 454 |
+
"vision_tower.transformer.layers.21.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 455 |
+
"vision_tower.transformer.layers.21.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 456 |
+
"vision_tower.transformer.layers.21.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 457 |
+
"vision_tower.transformer.layers.22.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 458 |
+
"vision_tower.transformer.layers.22.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 459 |
+
"vision_tower.transformer.layers.22.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 460 |
+
"vision_tower.transformer.layers.22.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 461 |
+
"vision_tower.transformer.layers.22.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 462 |
+
"vision_tower.transformer.layers.22.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 463 |
+
"vision_tower.transformer.layers.22.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 464 |
+
"vision_tower.transformer.layers.22.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 465 |
+
"vision_tower.transformer.layers.22.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 466 |
+
"vision_tower.transformer.layers.23.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 467 |
+
"vision_tower.transformer.layers.23.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 468 |
+
"vision_tower.transformer.layers.23.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 469 |
+
"vision_tower.transformer.layers.23.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 470 |
+
"vision_tower.transformer.layers.23.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 471 |
+
"vision_tower.transformer.layers.23.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 472 |
+
"vision_tower.transformer.layers.23.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 473 |
+
"vision_tower.transformer.layers.23.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 474 |
+
"vision_tower.transformer.layers.23.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 475 |
+
"vision_tower.transformer.layers.3.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 476 |
+
"vision_tower.transformer.layers.3.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 477 |
+
"vision_tower.transformer.layers.3.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 478 |
+
"vision_tower.transformer.layers.3.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 479 |
+
"vision_tower.transformer.layers.3.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 480 |
+
"vision_tower.transformer.layers.3.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 481 |
+
"vision_tower.transformer.layers.3.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 482 |
+
"vision_tower.transformer.layers.3.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 483 |
+
"vision_tower.transformer.layers.3.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 484 |
+
"vision_tower.transformer.layers.4.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 485 |
+
"vision_tower.transformer.layers.4.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 486 |
+
"vision_tower.transformer.layers.4.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 487 |
+
"vision_tower.transformer.layers.4.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 488 |
+
"vision_tower.transformer.layers.4.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 489 |
+
"vision_tower.transformer.layers.4.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 490 |
+
"vision_tower.transformer.layers.4.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 491 |
+
"vision_tower.transformer.layers.4.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 492 |
+
"vision_tower.transformer.layers.4.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 493 |
+
"vision_tower.transformer.layers.5.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 494 |
+
"vision_tower.transformer.layers.5.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 495 |
+
"vision_tower.transformer.layers.5.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 496 |
+
"vision_tower.transformer.layers.5.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 497 |
+
"vision_tower.transformer.layers.5.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 498 |
+
"vision_tower.transformer.layers.5.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 499 |
+
"vision_tower.transformer.layers.5.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 500 |
+
"vision_tower.transformer.layers.5.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 501 |
+
"vision_tower.transformer.layers.5.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 502 |
+
"vision_tower.transformer.layers.6.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 503 |
+
"vision_tower.transformer.layers.6.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 504 |
+
"vision_tower.transformer.layers.6.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 505 |
+
"vision_tower.transformer.layers.6.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 506 |
+
"vision_tower.transformer.layers.6.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 507 |
+
"vision_tower.transformer.layers.6.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 508 |
+
"vision_tower.transformer.layers.6.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 509 |
+
"vision_tower.transformer.layers.6.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 510 |
+
"vision_tower.transformer.layers.6.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 511 |
+
"vision_tower.transformer.layers.7.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 512 |
+
"vision_tower.transformer.layers.7.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 513 |
+
"vision_tower.transformer.layers.7.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 514 |
+
"vision_tower.transformer.layers.7.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 515 |
+
"vision_tower.transformer.layers.7.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 516 |
+
"vision_tower.transformer.layers.7.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 517 |
+
"vision_tower.transformer.layers.7.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 518 |
+
"vision_tower.transformer.layers.7.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 519 |
+
"vision_tower.transformer.layers.7.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 520 |
+
"vision_tower.transformer.layers.8.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 521 |
+
"vision_tower.transformer.layers.8.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 522 |
+
"vision_tower.transformer.layers.8.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 523 |
+
"vision_tower.transformer.layers.8.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 524 |
+
"vision_tower.transformer.layers.8.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 525 |
+
"vision_tower.transformer.layers.8.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 526 |
+
"vision_tower.transformer.layers.8.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 527 |
+
"vision_tower.transformer.layers.8.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 528 |
+
"vision_tower.transformer.layers.8.ffn_norm.weight": "model-00004-of-00004.safetensors",
|
| 529 |
+
"vision_tower.transformer.layers.9.attention.k_proj.weight": "model-00004-of-00004.safetensors",
|
| 530 |
+
"vision_tower.transformer.layers.9.attention.o_proj.weight": "model-00004-of-00004.safetensors",
|
| 531 |
+
"vision_tower.transformer.layers.9.attention.q_proj.weight": "model-00004-of-00004.safetensors",
|
| 532 |
+
"vision_tower.transformer.layers.9.attention.v_proj.weight": "model-00004-of-00004.safetensors",
|
| 533 |
+
"vision_tower.transformer.layers.9.attention_norm.weight": "model-00004-of-00004.safetensors",
|
| 534 |
+
"vision_tower.transformer.layers.9.feed_forward.down_proj.weight": "model-00004-of-00004.safetensors",
|
| 535 |
+
"vision_tower.transformer.layers.9.feed_forward.gate_proj.weight": "model-00004-of-00004.safetensors",
|
| 536 |
+
"vision_tower.transformer.layers.9.feed_forward.up_proj.weight": "model-00004-of-00004.safetensors",
|
| 537 |
+
"vision_tower.transformer.layers.9.ffn_norm.weight": "model-00004-of-00004.safetensors"
|
| 538 |
+
}
|
| 539 |
+
}
|
params.json
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"dim": 4096,
|
| 3 |
+
"n_layers": 34,
|
| 4 |
+
"head_dim": 128,
|
| 5 |
+
"hidden_dim": 14336,
|
| 6 |
+
"n_heads": 32,
|
| 7 |
+
"n_kv_heads": 8,
|
| 8 |
+
"rope_theta": 1000000.0,
|
| 9 |
+
"norm_eps": 1e-05,
|
| 10 |
+
"vocab_size": 131072,
|
| 11 |
+
"tied_embeddings": false,
|
| 12 |
+
"max_position_embeddings": 262144,
|
| 13 |
+
"llama_4_scaling": {
|
| 14 |
+
"original_max_position_embeddings": 16384,
|
| 15 |
+
"beta": 0.1
|
| 16 |
+
},
|
| 17 |
+
"q_lora_rank": null,
|
| 18 |
+
"qk_rope_head_dim": null,
|
| 19 |
+
"qk_nope_head_dim": null,
|
| 20 |
+
"kv_lora_rank": null,
|
| 21 |
+
"v_head_dim": null,
|
| 22 |
+
"yarn": {
|
| 23 |
+
"original_max_position_embeddings": 16384,
|
| 24 |
+
"factor": 16,
|
| 25 |
+
"apply_scale": false,
|
| 26 |
+
"beta": 32,
|
| 27 |
+
"alpha": 1
|
| 28 |
+
},
|
| 29 |
+
"vision_encoder": {
|
| 30 |
+
"image_token_id": 10,
|
| 31 |
+
"image_break_token_id": 12,
|
| 32 |
+
"image_end_token_id": 13,
|
| 33 |
+
"intermediate_size": 4096,
|
| 34 |
+
"num_hidden_layers": 24,
|
| 35 |
+
"num_attention_heads": 16,
|
| 36 |
+
"mm_projector_id": "patch_merge",
|
| 37 |
+
"spatial_merge_size": 2,
|
| 38 |
+
"hidden_size": 1024,
|
| 39 |
+
"num_channels": 3,
|
| 40 |
+
"image_size": 1540,
|
| 41 |
+
"max_image_size": 1540,
|
| 42 |
+
"patch_size": 14,
|
| 43 |
+
"rope_theta": 10000.0,
|
| 44 |
+
"add_pre_mm_projector_layer_norm": true,
|
| 45 |
+
"adapter_bias": false
|
| 46 |
+
}
|
| 47 |
+
}
|
processor_config.json
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"image_break_token": "[IMG_BREAK]",
|
| 3 |
+
"image_end_token": "[IMG_END]",
|
| 4 |
+
"image_processor": {
|
| 5 |
+
"crop_size": null,
|
| 6 |
+
"data_format": "channels_first",
|
| 7 |
+
"device": null,
|
| 8 |
+
"disable_grouping": null,
|
| 9 |
+
"do_center_crop": null,
|
| 10 |
+
"do_convert_rgb": true,
|
| 11 |
+
"do_normalize": true,
|
| 12 |
+
"do_pad": null,
|
| 13 |
+
"do_rescale": true,
|
| 14 |
+
"do_resize": true,
|
| 15 |
+
"image_mean": [
|
| 16 |
+
0.48145466,
|
| 17 |
+
0.4578275,
|
| 18 |
+
0.40821073
|
| 19 |
+
],
|
| 20 |
+
"image_processor_type": "PixtralImageProcessorFast",
|
| 21 |
+
"image_seq_length": null,
|
| 22 |
+
"image_std": [
|
| 23 |
+
0.26862954,
|
| 24 |
+
0.26130258,
|
| 25 |
+
0.27577711
|
| 26 |
+
],
|
| 27 |
+
"input_data_format": null,
|
| 28 |
+
"pad_size": null,
|
| 29 |
+
"patch_size": 14,
|
| 30 |
+
"processor_class": "PixtralProcessor",
|
| 31 |
+
"resample": 3,
|
| 32 |
+
"rescale_factor": 0.00392156862745098,
|
| 33 |
+
"return_tensors": null,
|
| 34 |
+
"size": {
|
| 35 |
+
"longest_edge": 1540
|
| 36 |
+
}
|
| 37 |
+
},
|
| 38 |
+
"image_token": "[IMG]",
|
| 39 |
+
"patch_size": 14,
|
| 40 |
+
"processor_class": "PixtralProcessor",
|
| 41 |
+
"spatial_merge_size": 2
|
| 42 |
+
}
|
release-manifest.json
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"version": "v2",
|
| 3 |
+
"date": "2026-10-09",
|
| 4 |
+
"backbone_base": {
|
| 5 |
+
"repo": "StandardThinking/StandardOne-8B",
|
| 6 |
+
"revision": "ac495d1c221297f0ae951941eff27bcf58b5ce52",
|
| 7 |
+
"version": "v2.2"
|
| 8 |
+
},
|
| 9 |
+
"fine_tune": {
|
| 10 |
+
"lora_rank": 64,
|
| 11 |
+
"lora_alpha": 128,
|
| 12 |
+
"adapter_sha256": "735df27810a12ed9cfb60f8b40ccac5128841bc00d12b451688eaba556cc70a1",
|
| 13 |
+
"steps": 2077,
|
| 14 |
+
"merge": "fp32 merge, bfloat16 save"
|
| 15 |
+
},
|
| 16 |
+
"head": {
|
| 17 |
+
"taps": [
|
| 18 |
+
"final",
|
| 19 |
+
25
|
| 20 |
+
],
|
| 21 |
+
"template": "schema-v1-nosys",
|
| 22 |
+
"sha256": "1b4bf756cb7c2413b30f2f34a6bab6578d6ccabc0cfd25b77f39b6af94469e14"
|
| 23 |
+
},
|
| 24 |
+
"prompt_format": "chat template with an empty default system message (SH_SYSTEM_PROMPT=none)",
|
| 25 |
+
"release": "StandardOne-8B-SH",
|
| 26 |
+
"evaluation": {
|
| 27 |
+
"reference_server": {
|
| 28 |
+
"internal-eval-v1.heldout": 90.67,
|
| 29 |
+
"internal-eval-hardproxy-v1.heldout": 46.0,
|
| 30 |
+
"internal-eval-v2-realistic.heldout": 90.83,
|
| 31 |
+
"jevbench-easy": 100.0,
|
| 32 |
+
"jevbench-original": 98.61,
|
| 33 |
+
"jevbench-hard": 63.96,
|
| 34 |
+
"longctx-dev": 84.67,
|
| 35 |
+
"winogrande-dev": 95.07,
|
| 36 |
+
"repair5k-validation-policy-judgment": 66.17
|
| 37 |
+
},
|
| 38 |
+
"decision_index_0_2_1": 48.68,
|
| 39 |
+
"engine_bf16_check": {
|
| 40 |
+
"internal-eval-v1.heldout": 90.5,
|
| 41 |
+
"internal-eval-hardproxy-v1.heldout": 45.67,
|
| 42 |
+
"internal-eval-v2-realistic.heldout": 91.0,
|
| 43 |
+
"jevbench-easy": 100.0,
|
| 44 |
+
"jevbench-original": 98.61,
|
| 45 |
+
"jevbench-hard": 63.96,
|
| 46 |
+
"longctx-dev": 86.0,
|
| 47 |
+
"winogrande-dev": 95.12,
|
| 48 |
+
"repair5k-validation-policy-judgment": 67.33
|
| 49 |
+
}
|
| 50 |
+
}
|
| 51 |
+
}
|
special_tokens_map.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
adapter/adapter_model.safetensors → tekken.json
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:600bb27946565481ecf51ba8aee252e49b9a68507866080ac9c30185bb312843
|
| 3 |
+
size 16753784
|
tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d5f6046775b112f0e2d456ee9dba450684ab964fe5c4e231599bdc6773028135
|
| 3 |
+
size 17078128
|
tokenizer_config.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|