Instructions to use N8Programs/Qwen3.6-35B-A3B-AntiLoop with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use N8Programs/Qwen3.6-35B-A3B-AntiLoop with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="N8Programs/Qwen3.6-35B-A3B-AntiLoop")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("N8Programs/Qwen3.6-35B-A3B-AntiLoop")
model = AutoModelForMultimodalLM.from_pretrained("N8Programs/Qwen3.6-35B-A3B-AntiLoop")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use N8Programs/Qwen3.6-35B-A3B-AntiLoop with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "N8Programs/Qwen3.6-35B-A3B-AntiLoop"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "N8Programs/Qwen3.6-35B-A3B-AntiLoop",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/N8Programs/Qwen3.6-35B-A3B-AntiLoop

SGLang

How to use N8Programs/Qwen3.6-35B-A3B-AntiLoop with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "N8Programs/Qwen3.6-35B-A3B-AntiLoop" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "N8Programs/Qwen3.6-35B-A3B-AntiLoop",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "N8Programs/Qwen3.6-35B-A3B-AntiLoop" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "N8Programs/Qwen3.6-35B-A3B-AntiLoop",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use N8Programs/Qwen3.6-35B-A3B-AntiLoop with Docker Model Runner:
```
docker model run hf.co/N8Programs/Qwen3.6-35B-A3B-AntiLoop
```

Qwen3.6-35B-A3B-AntiLoop / merge_verification.json

N8Programs

Add files using upload-large-folder tool

aa17594 verified 4 days ago

Raw

History Blame Contribute Delete

105 kB

	{
	"adapter_sha256": "9347d68d937eecbbdb22be7232f8905692436346817697b02ebc94b1a1a53faf",
	"all_non_target_bytes_identical": true,
	"all_targets_distinct_from_base": true,
	"all_targets_distinct_from_round1": true,
	"all_targets_formula_bit_exact": true,
	"base_revision": "995ad96eacd98c81ed38be0c5b274b04031597b0",
	"base_tensor_count": 1045,
	"layer_counts": {
	"0": 5,
	"1": 5,
	"10": 5,
	"11": 4,
	"12": 5,
	"13": 5,
	"14": 5,
	"15": 4,
	"16": 5,
	"17": 5,
	"18": 5,
	"19": 4,
	"2": 5,
	"20": 5,
	"21": 5,
	"22": 5,
	"23": 4,
	"24": 5,
	"25": 5,
	"26": 5,
	"27": 4,
	"28": 5,
	"29": 5,
	"3": 4,
	"30": 5,
	"31": 4,
	"32": 5,
	"33": 5,
	"34": 5,
	"35": 4,
	"36": 5,
	"37": 5,
	"38": 5,
	"39": 4,
	"4": 5,
	"5": 5,
	"6": 5,
	"7": 4,
	"8": 5,
	"9": 5
	},
	"merged_shards": 26,
	"metadata_loads": {
	"config_class": "Qwen3_5MoeConfig",
	"processor_class": "Qwen3VLProcessor",
	"tokenizer_class": "Qwen2Tokenizer"
	},
	"module_counts": {
	"in_proj_a": 30,
	"in_proj_b": 30,
	"in_proj_qkv": 30,
	"in_proj_z": 30,
	"k_proj": 10,
	"o_proj": 10,
	"out_proj": 30,
	"q_proj": 10,
	"v_proj": 10
	},
	"non_target_tensor_count": 855,
	"schema_version": 1,
	"target_reports": {
	"model.language_model.layers.0.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 30755,
	"changed_from_base_fraction": 0.4692840576171875,
	"changed_from_round1_elements": 29646,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.0.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.25063905038321e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00002-of-00026.safetensors"
	},
	"model.language_model.layers.0.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 48709,
	"changed_from_base_fraction": 0.7432403564453125,
	"changed_from_round1_elements": 49284,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000579833984375,
	"module": "base_model.model.model.layers.0.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.360447674059165e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00002-of-00026.safetensors"
	},
	"model.language_model.layers.0.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10751965,
	"changed_from_base_fraction": 0.6408670544624329,
	"changed_from_round1_elements": 10964134,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.0.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 7.51468740055135e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00001-of-00026.safetensors"
	},
	"model.language_model.layers.0.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5165585,
	"changed_from_base_fraction": 0.6157857179641724,
	"changed_from_round1_elements": 5290126,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.0.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 7.136164236255221e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00001-of-00026.safetensors"
	},
	"model.language_model.layers.0.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 4227429,
	"changed_from_base_fraction": 0.5039488077163696,
	"changed_from_round1_elements": 4250523,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00115966796875,
	"module": "base_model.model.model.layers.0.linear_attn.out_proj",
	"rms_effective_bf16_delta": 5.702941032752764e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00001-of-00026.safetensors"
	},
	"model.language_model.layers.1.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 33111,
	"changed_from_base_fraction": 0.5052337646484375,
	"changed_from_round1_elements": 34857,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000591278076171875,
	"module": "base_model.model.model.layers.1.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 6.47281010775474e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00002-of-00026.safetensors"
	},
	"model.language_model.layers.1.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 42255,
	"changed_from_base_fraction": 0.6447601318359375,
	"changed_from_round1_elements": 44658,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0003662109375,
	"module": "base_model.model.model.layers.1.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 6.433693698526348e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00002-of-00026.safetensors"
	},
	"model.language_model.layers.1.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 9998779,
	"changed_from_base_fraction": 0.5959736704826355,
	"changed_from_round1_elements": 10335801,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010216832160949707,
	"module": "base_model.model.model.layers.1.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 6.986407024210961e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00001-of-00026.safetensors"
	},
	"model.language_model.layers.1.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5231233,
	"changed_from_base_fraction": 0.623611569404602,
	"changed_from_round1_elements": 5395745,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00091552734375,
	"module": "base_model.model.model.layers.1.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 7.359320666631618e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00001-of-00026.safetensors"
	},
	"model.language_model.layers.1.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 4777187,
	"changed_from_base_fraction": 0.569485068321228,
	"changed_from_round1_elements": 4838053,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0006103515625,
	"module": "base_model.model.model.layers.1.linear_attn.out_proj",
	"rms_effective_bf16_delta": 5.66848069458663e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00001-of-00026.safetensors"
	},
	"model.language_model.layers.10.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 41894,
	"changed_from_base_fraction": 0.639251708984375,
	"changed_from_round1_elements": 40018,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.10.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 8.030118138951974e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00007-of-00026.safetensors"
	},
	"model.language_model.layers.10.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 46368,
	"changed_from_base_fraction": 0.70751953125,
	"changed_from_round1_elements": 45654,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000560760498046875,
	"module": "base_model.model.model.layers.10.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.305791432245633e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00007-of-00026.safetensors"
	},
	"model.language_model.layers.10.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10747736,
	"changed_from_base_fraction": 0.6406149864196777,
	"changed_from_round1_elements": 10039291,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0008058547973632812,
	"module": "base_model.model.model.layers.10.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 7.884481753437252e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00006-of-00026.safetensors"
	},
	"model.language_model.layers.10.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5717960,
	"changed_from_base_fraction": 0.6816339492797852,
	"changed_from_round1_elements": 5150891,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0008544921875,
	"module": "base_model.model.model.layers.10.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 8.197803025550057e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00006-of-00026.safetensors"
	},
	"model.language_model.layers.10.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5380400,
	"changed_from_base_fraction": 0.6413936614990234,
	"changed_from_round1_elements": 4266678,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00091552734375,
	"module": "base_model.model.model.layers.10.linear_attn.out_proj",
	"rms_effective_bf16_delta": 7.199138831694352e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00006-of-00026.safetensors"
	},
	"model.language_model.layers.11.self_attn.k_proj.weight": {
	"changed_from_base_elements": 694448,
	"changed_from_base_fraction": 0.6622772216796875,
	"changed_from_round1_elements": 677592,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000579833984375,
	"module": "base_model.model.model.layers.11.self_attn.k_proj",
	"rms_effective_bf16_delta": 7.355739501388593e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00008-of-00026.safetensors"
	},
	"model.language_model.layers.11.self_attn.o_proj.weight": {
	"changed_from_base_elements": 5479007,
	"changed_from_base_fraction": 0.6531485319137573,
	"changed_from_round1_elements": 4212366,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009410381317138672,
	"module": "base_model.model.model.layers.11.self_attn.o_proj",
	"rms_effective_bf16_delta": 8.168481778209755e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00008-of-00026.safetensors"
	},
	"model.language_model.layers.11.self_attn.q_proj.weight": {
	"changed_from_base_elements": 10769572,
	"changed_from_base_fraction": 0.6419165134429932,
	"changed_from_round1_elements": 10052272,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00079345703125,
	"module": "base_model.model.model.layers.11.self_attn.q_proj",
	"rms_effective_bf16_delta": 7.578644977082595e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00008-of-00026.safetensors"
	},
	"model.language_model.layers.11.self_attn.v_proj.weight": {
	"changed_from_base_elements": 758420,
	"changed_from_base_fraction": 0.7232856750488281,
	"changed_from_round1_elements": 684474,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.11.self_attn.v_proj",
	"rms_effective_bf16_delta": 8.498003892171755e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00008-of-00026.safetensors"
	},
	"model.language_model.layers.12.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 42608,
	"changed_from_base_fraction": 0.650146484375,
	"changed_from_round1_elements": 35693,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.12.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.816997522647381e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00009-of-00026.safetensors"
	},
	"model.language_model.layers.12.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 46557,
	"changed_from_base_fraction": 0.7104034423828125,
	"changed_from_round1_elements": 43605,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00054931640625,
	"module": "base_model.model.model.layers.12.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.412433719402122e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00009-of-00026.safetensors"
	},
	"model.language_model.layers.12.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10868056,
	"changed_from_base_fraction": 0.6477866172790527,
	"changed_from_round1_elements": 9488770,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0008544921875,
	"module": "base_model.model.model.layers.12.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 8.18584051920274e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00008-of-00026.safetensors"
	},
	"model.language_model.layers.12.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5882139,
	"changed_from_base_fraction": 0.7012056112289429,
	"changed_from_round1_elements": 5084926,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.12.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 8.230692583350476e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00008-of-00026.safetensors"
	},
	"model.language_model.layers.12.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5516697,
	"changed_from_base_fraction": 0.6576415300369263,
	"changed_from_round1_elements": 4116675,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00079345703125,
	"module": "base_model.model.model.layers.12.linear_attn.out_proj",
	"rms_effective_bf16_delta": 7.846577771796752e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00008-of-00026.safetensors"
	},
	"model.language_model.layers.13.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 41837,
	"changed_from_base_fraction": 0.6383819580078125,
	"changed_from_round1_elements": 31763,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000946044921875,
	"module": "base_model.model.model.layers.13.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 9.347651483814202e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00009-of-00026.safetensors"
	},
	"model.language_model.layers.13.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 46031,
	"changed_from_base_fraction": 0.7023773193359375,
	"changed_from_round1_elements": 40700,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000396728515625,
	"module": "base_model.model.model.layers.13.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.12724770719925e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00009-of-00026.safetensors"
	},
	"model.language_model.layers.13.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10692640,
	"changed_from_base_fraction": 0.6373310089111328,
	"changed_from_round1_elements": 8906058,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0012636184692382812,
	"module": "base_model.model.model.layers.13.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 8.316534015371205e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00008-of-00026.safetensors"
	},
	"model.language_model.layers.13.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5786985,
	"changed_from_base_fraction": 0.6898623704910278,
	"changed_from_round1_elements": 4767619,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010528564453125,
	"module": "base_model.model.model.layers.13.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 8.540713466959101e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00008-of-00026.safetensors"
	},
	"model.language_model.layers.13.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5599044,
	"changed_from_base_fraction": 0.6674580574035645,
	"changed_from_round1_elements": 4031375,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000885009765625,
	"module": "base_model.model.model.layers.13.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.360459743314073e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00008-of-00026.safetensors"
	},
	"model.language_model.layers.14.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 36731,
	"changed_from_base_fraction": 0.5604705810546875,
	"changed_from_round1_elements": 32822,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.14.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 6.408820823201858e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00011-of-00026.safetensors"
	},
	"model.language_model.layers.14.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 46324,
	"changed_from_base_fraction": 0.70684814453125,
	"changed_from_round1_elements": 40698,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000579833984375,
	"module": "base_model.model.model.layers.14.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.931456048098573e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00011-of-00026.safetensors"
	},
	"model.language_model.layers.14.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10758927,
	"changed_from_base_fraction": 0.6412820219993591,
	"changed_from_round1_elements": 8893031,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00091552734375,
	"module": "base_model.model.model.layers.14.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 8.396889941829679e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00010-of-00026.safetensors"
	},
	"model.language_model.layers.14.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 6000591,
	"changed_from_base_fraction": 0.715326189994812,
	"changed_from_round1_elements": 4979405,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009002685546875,
	"module": "base_model.model.model.layers.14.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 8.636677447278575e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00010-of-00026.safetensors"
	},
	"model.language_model.layers.14.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5598240,
	"changed_from_base_fraction": 0.6673622131347656,
	"changed_from_round1_elements": 3852012,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.14.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.746053163095078e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00010-of-00026.safetensors"
	},
	"model.language_model.layers.15.self_attn.k_proj.weight": {
	"changed_from_base_elements": 701780,
	"changed_from_base_fraction": 0.6692695617675781,
	"changed_from_round1_elements": 605692,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0008392333984375,
	"module": "base_model.model.model.layers.15.self_attn.k_proj",
	"rms_effective_bf16_delta": 7.595614539234427e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00010-of-00026.safetensors"
	},
	"model.language_model.layers.15.self_attn.o_proj.weight": {
	"changed_from_base_elements": 5682420,
	"changed_from_base_fraction": 0.6773972511291504,
	"changed_from_round1_elements": 3786799,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.15.self_attn.o_proj",
	"rms_effective_bf16_delta": 9.326062588130585e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00010-of-00026.safetensors"
	},
	"model.language_model.layers.15.self_attn.q_proj.weight": {
	"changed_from_base_elements": 11393837,
	"changed_from_base_fraction": 0.6791256070137024,
	"changed_from_round1_elements": 9237687,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00079345703125,
	"module": "base_model.model.model.layers.15.self_attn.q_proj",
	"rms_effective_bf16_delta": 8.019703041610845e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00010-of-00026.safetensors"
	},
	"model.language_model.layers.15.self_attn.v_proj.weight": {
	"changed_from_base_elements": 764574,
	"changed_from_base_fraction": 0.7291545867919922,
	"changed_from_round1_elements": 581314,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.15.self_attn.v_proj",
	"rms_effective_bf16_delta": 9.223723003081227e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00010-of-00026.safetensors"
	},
	"model.language_model.layers.16.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 40458,
	"changed_from_base_fraction": 0.617340087890625,
	"changed_from_round1_elements": 31743,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0008544921875,
	"module": "base_model.model.model.layers.16.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.926234263603284e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00011-of-00026.safetensors"
	},
	"model.language_model.layers.16.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 47306,
	"changed_from_base_fraction": 0.721832275390625,
	"changed_from_round1_elements": 40583,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.16.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 6.947450366004857e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00011-of-00026.safetensors"
	},
	"model.language_model.layers.16.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10924269,
	"changed_from_base_fraction": 0.6511371731758118,
	"changed_from_round1_elements": 8357069,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0008544921875,
	"module": "base_model.model.model.layers.16.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 8.514577641788167e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00010-of-00026.safetensors"
	},
	"model.language_model.layers.16.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5987956,
	"changed_from_base_fraction": 0.7138199806213379,
	"changed_from_round1_elements": 4625973,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001220703125,
	"module": "base_model.model.model.layers.16.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 9.070390622614111e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00010-of-00026.safetensors"
	},
	"model.language_model.layers.16.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5665606,
	"changed_from_base_fraction": 0.6753928661346436,
	"changed_from_round1_elements": 3700246,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.16.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.717200426859417e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00010-of-00026.safetensors"
	},
	"model.language_model.layers.17.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 40944,
	"changed_from_base_fraction": 0.624755859375,
	"changed_from_round1_elements": 29399,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.17.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.38633181213219e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00013-of-00026.safetensors"
	},
	"model.language_model.layers.17.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 46734,
	"changed_from_base_fraction": 0.713104248046875,
	"changed_from_round1_elements": 36777,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.17.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.645919554666517e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00013-of-00026.safetensors"
	},
	"model.language_model.layers.17.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 11070108,
	"changed_from_base_fraction": 0.65982985496521,
	"changed_from_round1_elements": 8012316,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.17.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 8.773317292698886e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00012-of-00026.safetensors"
	},
	"model.language_model.layers.17.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 6084979,
	"changed_from_base_fraction": 0.7253860235214233,
	"changed_from_round1_elements": 4587384,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0008544921875,
	"module": "base_model.model.model.layers.17.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 9.123145477755034e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00012-of-00026.safetensors"
	},
	"model.language_model.layers.17.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5673400,
	"changed_from_base_fraction": 0.6763219833374023,
	"changed_from_round1_elements": 3760744,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.17.linear_attn.out_proj",
	"rms_effective_bf16_delta": 9.202713231300383e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00012-of-00026.safetensors"
	},
	"model.language_model.layers.18.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 42062,
	"changed_from_base_fraction": 0.641815185546875,
	"changed_from_round1_elements": 28920,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.18.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.428588345419565e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00013-of-00026.safetensors"
	},
	"model.language_model.layers.18.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 47332,
	"changed_from_base_fraction": 0.72222900390625,
	"changed_from_round1_elements": 38646,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.18.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.459836919250263e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00013-of-00026.safetensors"
	},
	"model.language_model.layers.18.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 11072675,
	"changed_from_base_fraction": 0.6599828600883484,
	"changed_from_round1_elements": 7896147,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.18.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 9.043640688973794e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00012-of-00026.safetensors"
	},
	"model.language_model.layers.18.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5985994,
	"changed_from_base_fraction": 0.7135860919952393,
	"changed_from_round1_elements": 4394901,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0013427734375,
	"module": "base_model.model.model.layers.18.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 9.505438980045448e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00012-of-00026.safetensors"
	},
	"model.language_model.layers.18.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5617121,
	"changed_from_base_fraction": 0.6696130037307739,
	"changed_from_round1_elements": 3429999,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000946044921875,
	"module": "base_model.model.model.layers.18.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.487318640617153e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00012-of-00026.safetensors"
	},
	"model.language_model.layers.19.self_attn.k_proj.weight": {
	"changed_from_base_elements": 717909,
	"changed_from_base_fraction": 0.6846513748168945,
	"changed_from_round1_elements": 546715,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.19.self_attn.k_proj",
	"rms_effective_bf16_delta": 7.58642914683779e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00012-of-00026.safetensors"
	},
	"model.language_model.layers.19.self_attn.o_proj.weight": {
	"changed_from_base_elements": 5519486,
	"changed_from_base_fraction": 0.6579740047454834,
	"changed_from_round1_elements": 3326415,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00079345703125,
	"module": "base_model.model.model.layers.19.self_attn.o_proj",
	"rms_effective_bf16_delta": 8.263213799470523e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00012-of-00026.safetensors"
	},
	"model.language_model.layers.19.self_attn.q_proj.weight": {
	"changed_from_base_elements": 11628234,
	"changed_from_base_fraction": 0.6930967569351196,
	"changed_from_round1_elements": 8096194,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.19.self_attn.q_proj",
	"rms_effective_bf16_delta": 8.709391886705632e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00012-of-00026.safetensors"
	},
	"model.language_model.layers.19.self_attn.v_proj.weight": {
	"changed_from_base_elements": 761626,
	"changed_from_base_fraction": 0.7263431549072266,
	"changed_from_round1_elements": 500010,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009212493896484375,
	"module": "base_model.model.model.layers.19.self_attn.v_proj",
	"rms_effective_bf16_delta": 8.91430244772417e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00012-of-00026.safetensors"
	},
	"model.language_model.layers.2.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 39811,
	"changed_from_base_fraction": 0.6074676513671875,
	"changed_from_round1_elements": 40877,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0006103515625,
	"module": "base_model.model.model.layers.2.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.184019530288684e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00003-of-00026.safetensors"
	},
	"model.language_model.layers.2.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 44642,
	"changed_from_base_fraction": 0.681182861328125,
	"changed_from_round1_elements": 46866,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000579833984375,
	"module": "base_model.model.model.layers.2.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 6.96682201680039e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00003-of-00026.safetensors"
	},
	"model.language_model.layers.2.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10180888,
	"changed_from_base_fraction": 0.6068282127380371,
	"changed_from_round1_elements": 10610201,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000995635986328125,
	"module": "base_model.model.model.layers.2.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 7.100468354687935e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00003-of-00026.safetensors"
	},
	"model.language_model.layers.2.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5378050,
	"changed_from_base_fraction": 0.6411135196685791,
	"changed_from_round1_elements": 5493006,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.2.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 7.283989786059287e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00003-of-00026.safetensors"
	},
	"model.language_model.layers.2.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 4510282,
	"changed_from_base_fraction": 0.5376675128936768,
	"changed_from_round1_elements": 4408151,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00093841552734375,
	"module": "base_model.model.model.layers.2.linear_attn.out_proj",
	"rms_effective_bf16_delta": 6.235573763046415e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00003-of-00026.safetensors"
	},
	"model.language_model.layers.20.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 41909,
	"changed_from_base_fraction": 0.6394805908203125,
	"changed_from_round1_elements": 29478,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.20.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 8.93851098858551e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00015-of-00026.safetensors"
	},
	"model.language_model.layers.20.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 46827,
	"changed_from_base_fraction": 0.7145233154296875,
	"changed_from_round1_elements": 35896,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0008544921875,
	"module": "base_model.model.model.layers.20.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.938891775580925e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00015-of-00026.safetensors"
	},
	"model.language_model.layers.20.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10428442,
	"changed_from_base_fraction": 0.6215835809707642,
	"changed_from_round1_elements": 6832766,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001220703125,
	"module": "base_model.model.model.layers.20.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 8.930851044544183e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00014-of-00026.safetensors"
	},
	"model.language_model.layers.20.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5788078,
	"changed_from_base_fraction": 0.6899926662445068,
	"changed_from_round1_elements": 3905647,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00115203857421875,
	"module": "base_model.model.model.layers.20.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 9.368472263909851e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00014-of-00026.safetensors"
	},
	"model.language_model.layers.20.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5360389,
	"changed_from_base_fraction": 0.6390081644058228,
	"changed_from_round1_elements": 3199937,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.20.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.357498774166259e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00014-of-00026.safetensors"
	},
	"model.language_model.layers.21.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 40619,
	"changed_from_base_fraction": 0.6197967529296875,
	"changed_from_round1_elements": 29516,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0006103515625,
	"module": "base_model.model.model.layers.21.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.828153390238863e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00015-of-00026.safetensors"
	},
	"model.language_model.layers.21.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 48063,
	"changed_from_base_fraction": 0.7333831787109375,
	"changed_from_round1_elements": 33832,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0006103515625,
	"module": "base_model.model.model.layers.21.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 8.272542708067484e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00015-of-00026.safetensors"
	},
	"model.language_model.layers.21.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10417391,
	"changed_from_base_fraction": 0.6209248900413513,
	"changed_from_round1_elements": 6799676,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.21.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 8.82992226033504e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00014-of-00026.safetensors"
	},
	"model.language_model.layers.21.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5771549,
	"changed_from_base_fraction": 0.688022255897522,
	"changed_from_round1_elements": 3865573,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001621246337890625,
	"module": "base_model.model.model.layers.21.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 9.694958700437348e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00014-of-00026.safetensors"
	},
	"model.language_model.layers.21.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5266131,
	"changed_from_base_fraction": 0.6277717351913452,
	"changed_from_round1_elements": 3044407,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0007476806640625,
	"module": "base_model.model.model.layers.21.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.394243018921675e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00014-of-00026.safetensors"
	},
	"model.language_model.layers.22.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 42877,
	"changed_from_base_fraction": 0.6542510986328125,
	"changed_from_round1_elements": 29433,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00054931640625,
	"module": "base_model.model.model.layers.22.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 8.447058045108835e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00015-of-00026.safetensors"
	},
	"model.language_model.layers.22.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 45907,
	"changed_from_base_fraction": 0.7004852294921875,
	"changed_from_round1_elements": 34274,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000499725341796875,
	"module": "base_model.model.model.layers.22.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.73146372057541e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00015-of-00026.safetensors"
	},
	"model.language_model.layers.22.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10554381,
	"changed_from_base_fraction": 0.6290901303291321,
	"changed_from_round1_elements": 6836164,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010728836059570312,
	"module": "base_model.model.model.layers.22.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 8.774579516589965e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00014-of-00026.safetensors"
	},
	"model.language_model.layers.22.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5863367,
	"changed_from_base_fraction": 0.6989678144454956,
	"changed_from_round1_elements": 3989572,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00103759765625,
	"module": "base_model.model.model.layers.22.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 9.434000093003106e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00014-of-00026.safetensors"
	},
	"model.language_model.layers.22.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5293620,
	"changed_from_base_fraction": 0.6310486793518066,
	"changed_from_round1_elements": 3137184,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.22.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.249776368945531e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00014-of-00026.safetensors"
	},
	"model.language_model.layers.23.self_attn.k_proj.weight": {
	"changed_from_base_elements": 678159,
	"changed_from_base_fraction": 0.6467428207397461,
	"changed_from_round1_elements": 472559,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.23.self_attn.k_proj",
	"rms_effective_bf16_delta": 7.518396677676303e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00016-of-00026.safetensors"
	},
	"model.language_model.layers.23.self_attn.o_proj.weight": {
	"changed_from_base_elements": 5218370,
	"changed_from_base_fraction": 0.6220781803131104,
	"changed_from_round1_elements": 2920793,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.23.self_attn.o_proj",
	"rms_effective_bf16_delta": 8.036047474386736e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00016-of-00026.safetensors"
	},
	"model.language_model.layers.23.self_attn.q_proj.weight": {
	"changed_from_base_elements": 10712835,
	"changed_from_base_fraction": 0.6385347247123718,
	"changed_from_round1_elements": 6933208,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001220703125,
	"module": "base_model.model.model.layers.23.self_attn.q_proj",
	"rms_effective_bf16_delta": 8.438226304057198e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00016-of-00026.safetensors"
	},
	"model.language_model.layers.23.self_attn.v_proj.weight": {
	"changed_from_base_elements": 746834,
	"changed_from_base_fraction": 0.7122364044189453,
	"changed_from_round1_elements": 472261,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0007476806640625,
	"module": "base_model.model.model.layers.23.self_attn.v_proj",
	"rms_effective_bf16_delta": 8.178560910881905e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00016-of-00026.safetensors"
	},
	"model.language_model.layers.24.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 40637,
	"changed_from_base_fraction": 0.6200714111328125,
	"changed_from_round1_elements": 26629,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.24.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 6.87900255946991e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00017-of-00026.safetensors"
	},
	"model.language_model.layers.24.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 45379,
	"changed_from_base_fraction": 0.6924285888671875,
	"changed_from_round1_elements": 32660,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.24.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 6.982129294249513e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00017-of-00026.safetensors"
	},
	"model.language_model.layers.24.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10362982,
	"changed_from_base_fraction": 0.6176818609237671,
	"changed_from_round1_elements": 6346341,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00079345703125,
	"module": "base_model.model.model.layers.24.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 8.51102613002069e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00016-of-00026.safetensors"
	},
	"model.language_model.layers.24.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5883216,
	"changed_from_base_fraction": 0.7013339996337891,
	"changed_from_round1_elements": 3821507,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001220703125,
	"module": "base_model.model.model.layers.24.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 9.154575677870438e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00016-of-00026.safetensors"
	},
	"model.language_model.layers.24.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5249409,
	"changed_from_base_fraction": 0.625778317451477,
	"changed_from_round1_elements": 2982806,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0006103515625,
	"module": "base_model.model.model.layers.24.linear_attn.out_proj",
	"rms_effective_bf16_delta": 7.722079186994334e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00016-of-00026.safetensors"
	},
	"model.language_model.layers.25.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 43391,
	"changed_from_base_fraction": 0.6620941162109375,
	"changed_from_round1_elements": 25954,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0007476806640625,
	"module": "base_model.model.model.layers.25.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 9.756493403597129e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00017-of-00026.safetensors"
	},
	"model.language_model.layers.25.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 47230,
	"changed_from_base_fraction": 0.720672607421875,
	"changed_from_round1_elements": 33213,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.25.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.860814471744432e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00017-of-00026.safetensors"
	},
	"model.language_model.layers.25.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10332363,
	"changed_from_base_fraction": 0.6158568263053894,
	"changed_from_round1_elements": 6151981,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001068115234375,
	"module": "base_model.model.model.layers.25.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 8.739930282826853e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00016-of-00026.safetensors"
	},
	"model.language_model.layers.25.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5786058,
	"changed_from_base_fraction": 0.6897518634796143,
	"changed_from_round1_elements": 3688364,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00103759765625,
	"module": "base_model.model.model.layers.25.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 9.676194801627504e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00016-of-00026.safetensors"
	},
	"model.language_model.layers.25.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5281483,
	"changed_from_base_fraction": 0.6296018362045288,
	"changed_from_round1_elements": 2889414,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00067138671875,
	"module": "base_model.model.model.layers.25.linear_attn.out_proj",
	"rms_effective_bf16_delta": 7.813189765614977e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00016-of-00026.safetensors"
	},
	"model.language_model.layers.26.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 39782,
	"changed_from_base_fraction": 0.607025146484375,
	"changed_from_round1_elements": 26177,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.26.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.568452986791395e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00019-of-00026.safetensors"
	},
	"model.language_model.layers.26.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 45840,
	"changed_from_base_fraction": 0.699462890625,
	"changed_from_round1_elements": 32586,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.26.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 8.063676498761255e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00019-of-00026.safetensors"
	},
	"model.language_model.layers.26.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10485047,
	"changed_from_base_fraction": 0.6249575018882751,
	"changed_from_round1_elements": 6261385,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.26.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 8.989223925345269e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00018-of-00026.safetensors"
	},
	"model.language_model.layers.26.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 6003196,
	"changed_from_base_fraction": 0.7156367301940918,
	"changed_from_round1_elements": 3835190,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001220703125,
	"module": "base_model.model.model.layers.26.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 9.987351500103812e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00018-of-00026.safetensors"
	},
	"model.language_model.layers.26.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5295000,
	"changed_from_base_fraction": 0.6312131881713867,
	"changed_from_round1_elements": 2851671,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.26.linear_attn.out_proj",
	"rms_effective_bf16_delta": 7.994238414767838e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00018-of-00026.safetensors"
	},
	"model.language_model.layers.27.self_attn.k_proj.weight": {
	"changed_from_base_elements": 683967,
	"changed_from_base_fraction": 0.6522817611694336,
	"changed_from_round1_elements": 439082,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0004901885986328125,
	"module": "base_model.model.model.layers.27.self_attn.k_proj",
	"rms_effective_bf16_delta": 7.73319782163451e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00018-of-00026.safetensors"
	},
	"model.language_model.layers.27.self_attn.o_proj.weight": {
	"changed_from_base_elements": 5308128,
	"changed_from_base_fraction": 0.6327781677246094,
	"changed_from_round1_elements": 2810666,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.27.self_attn.o_proj",
	"rms_effective_bf16_delta": 8.711702037703368e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00018-of-00026.safetensors"
	},
	"model.language_model.layers.27.self_attn.q_proj.weight": {
	"changed_from_base_elements": 11430191,
	"changed_from_base_fraction": 0.6812924742698669,
	"changed_from_round1_elements": 6864468,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001087188720703125,
	"module": "base_model.model.model.layers.27.self_attn.q_proj",
	"rms_effective_bf16_delta": 9.126735428806502e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00018-of-00026.safetensors"
	},
	"model.language_model.layers.27.self_attn.v_proj.weight": {
	"changed_from_base_elements": 738739,
	"changed_from_base_fraction": 0.7045164108276367,
	"changed_from_round1_elements": 420474,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.27.self_attn.v_proj",
	"rms_effective_bf16_delta": 8.949030938293286e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00018-of-00026.safetensors"
	},
	"model.language_model.layers.28.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 43108,
	"changed_from_base_fraction": 0.65777587890625,
	"changed_from_round1_elements": 29074,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00054931640625,
	"module": "base_model.model.model.layers.28.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 8.5556712585574e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00019-of-00026.safetensors"
	},
	"model.language_model.layers.28.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 47219,
	"changed_from_base_fraction": 0.7205047607421875,
	"changed_from_round1_elements": 34406,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.28.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.959844718254463e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00019-of-00026.safetensors"
	},
	"model.language_model.layers.28.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10649242,
	"changed_from_base_fraction": 0.6347442865371704,
	"changed_from_round1_elements": 6110051,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0014352798461914062,
	"module": "base_model.model.model.layers.28.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 9.628502558009477e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00018-of-00026.safetensors"
	},
	"model.language_model.layers.28.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5957038,
	"changed_from_base_fraction": 0.7101342678070068,
	"changed_from_round1_elements": 3644158,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010986328125,
	"module": "base_model.model.model.layers.28.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 0.00010274432978587353,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00018-of-00026.safetensors"
	},
	"model.language_model.layers.28.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5364574,
	"changed_from_base_fraction": 0.6395070552825928,
	"changed_from_round1_elements": 2894518,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.28.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.804179398761905e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00018-of-00026.safetensors"
	},
	"model.language_model.layers.29.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 42884,
	"changed_from_base_fraction": 0.65435791015625,
	"changed_from_round1_elements": 27859,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.29.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 8.808322381583734e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00021-of-00026.safetensors"
	},
	"model.language_model.layers.29.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 46574,
	"changed_from_base_fraction": 0.710662841796875,
	"changed_from_round1_elements": 31567,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.29.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 8.585155676340322e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00021-of-00026.safetensors"
	},
	"model.language_model.layers.29.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10667129,
	"changed_from_base_fraction": 0.6358104348182678,
	"changed_from_round1_elements": 6108182,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00128173828125,
	"module": "base_model.model.model.layers.29.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 9.849695659035123e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00020-of-00026.safetensors"
	},
	"model.language_model.layers.29.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 6084440,
	"changed_from_base_fraction": 0.7253217697143555,
	"changed_from_round1_elements": 3751712,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00115966796875,
	"module": "base_model.model.model.layers.29.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 0.00011035977036764194,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00020-of-00026.safetensors"
	},
	"model.language_model.layers.29.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5370665,
	"changed_from_base_fraction": 0.6402331590652466,
	"changed_from_round1_elements": 2921725,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001007080078125,
	"module": "base_model.model.model.layers.29.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.975065824565774e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00020-of-00026.safetensors"
	},
	"model.language_model.layers.3.self_attn.k_proj.weight": {
	"changed_from_base_elements": 662135,
	"changed_from_base_fraction": 0.6314611434936523,
	"changed_from_round1_elements": 706546,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000682830810546875,
	"module": "base_model.model.model.layers.3.self_attn.k_proj",
	"rms_effective_bf16_delta": 7.241444047012491e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00003-of-00026.safetensors"
	},
	"model.language_model.layers.3.self_attn.o_proj.weight": {
	"changed_from_base_elements": 5041438,
	"changed_from_base_fraction": 0.6009862422943115,
	"changed_from_round1_elements": 4954761,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0005035400390625,
	"module": "base_model.model.model.layers.3.self_attn.o_proj",
	"rms_effective_bf16_delta": 6.026982563617216e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00003-of-00026.safetensors"
	},
	"model.language_model.layers.3.self_attn.q_proj.weight": {
	"changed_from_base_elements": 10113617,
	"changed_from_base_fraction": 0.6028185486793518,
	"changed_from_round1_elements": 10534224,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00092315673828125,
	"module": "base_model.model.model.layers.3.self_attn.q_proj",
	"rms_effective_bf16_delta": 7.350570200486596e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00003-of-00026.safetensors"
	},
	"model.language_model.layers.3.self_attn.v_proj.weight": {
	"changed_from_base_elements": 721849,
	"changed_from_base_fraction": 0.6884088516235352,
	"changed_from_round1_elements": 731927,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000926971435546875,
	"module": "base_model.model.model.layers.3.self_attn.v_proj",
	"rms_effective_bf16_delta": 7.747421220958643e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00003-of-00026.safetensors"
	},
	"model.language_model.layers.30.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 43839,
	"changed_from_base_fraction": 0.6689300537109375,
	"changed_from_round1_elements": 26850,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000885009765625,
	"module": "base_model.model.model.layers.30.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 9.872395338817168e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00021-of-00026.safetensors"
	},
	"model.language_model.layers.30.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 46826,
	"changed_from_base_fraction": 0.714508056640625,
	"changed_from_round1_elements": 31020,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000518798828125,
	"module": "base_model.model.model.layers.30.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 8.24930908855796e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00021-of-00026.safetensors"
	},
	"model.language_model.layers.30.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10720563,
	"changed_from_base_fraction": 0.638995349407196,
	"changed_from_round1_elements": 6110636,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00112152099609375,
	"module": "base_model.model.model.layers.30.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 9.983738297083213e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00020-of-00026.safetensors"
	},
	"model.language_model.layers.30.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 6054666,
	"changed_from_base_fraction": 0.7217724323272705,
	"changed_from_round1_elements": 3673015,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0014801025390625,
	"module": "base_model.model.model.layers.30.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 0.00010544629155757812,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00020-of-00026.safetensors"
	},
	"model.language_model.layers.30.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5355054,
	"changed_from_base_fraction": 0.6383721828460693,
	"changed_from_round1_elements": 2894416,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0008544921875,
	"module": "base_model.model.model.layers.30.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.681716448816516e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00020-of-00026.safetensors"
	},
	"model.language_model.layers.31.self_attn.k_proj.weight": {
	"changed_from_base_elements": 696095,
	"changed_from_base_fraction": 0.6638479232788086,
	"changed_from_round1_elements": 416550,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009212493896484375,
	"module": "base_model.model.model.layers.31.self_attn.k_proj",
	"rms_effective_bf16_delta": 8.437487298802524e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00020-of-00026.safetensors"
	},
	"model.language_model.layers.31.self_attn.o_proj.weight": {
	"changed_from_base_elements": 5254913,
	"changed_from_base_fraction": 0.6264344453811646,
	"changed_from_round1_elements": 2820507,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00128173828125,
	"module": "base_model.model.model.layers.31.self_attn.o_proj",
	"rms_effective_bf16_delta": 8.925954272837241e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00020-of-00026.safetensors"
	},
	"model.language_model.layers.31.self_attn.q_proj.weight": {
	"changed_from_base_elements": 11193374,
	"changed_from_base_fraction": 0.6671770811080933,
	"changed_from_round1_elements": 6694341,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00146484375,
	"module": "base_model.model.model.layers.31.self_attn.q_proj",
	"rms_effective_bf16_delta": 9.08660571373039e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00020-of-00026.safetensors"
	},
	"model.language_model.layers.31.self_attn.v_proj.weight": {
	"changed_from_base_elements": 744481,
	"changed_from_base_fraction": 0.7099924087524414,
	"changed_from_round1_elements": 443320,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00115966796875,
	"module": "base_model.model.model.layers.31.self_attn.v_proj",
	"rms_effective_bf16_delta": 9.294866747722607e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00020-of-00026.safetensors"
	},
	"model.language_model.layers.32.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 42686,
	"changed_from_base_fraction": 0.651336669921875,
	"changed_from_round1_elements": 26671,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.32.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.927647995249258e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00022-of-00026.safetensors"
	},
	"model.language_model.layers.32.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 48385,
	"changed_from_base_fraction": 0.7382965087890625,
	"changed_from_round1_elements": 31012,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000701904296875,
	"module": "base_model.model.model.layers.32.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 9.253677487914579e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00022-of-00026.safetensors"
	},
	"model.language_model.layers.32.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10909815,
	"changed_from_base_fraction": 0.6502756476402283,
	"changed_from_round1_elements": 6333533,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0011749267578125,
	"module": "base_model.model.model.layers.32.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 0.00010049900326370136,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00022-of-00026.safetensors"
	},
	"model.language_model.layers.32.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5980567,
	"changed_from_base_fraction": 0.7129391431808472,
	"changed_from_round1_elements": 3655412,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001491546630859375,
	"module": "base_model.model.model.layers.32.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 0.00010056438913395777,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00022-of-00026.safetensors"
	},
	"model.language_model.layers.32.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5391896,
	"changed_from_base_fraction": 0.6427640914916992,
	"changed_from_round1_elements": 3100089,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00093841552734375,
	"module": "base_model.model.model.layers.32.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.832600221432643e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00022-of-00026.safetensors"
	},
	"model.language_model.layers.33.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 44844,
	"changed_from_base_fraction": 0.68426513671875,
	"changed_from_round1_elements": 26045,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.33.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 0.00010916238745063663,
	"shape": [
	32,
	2048
	],
	"shard": "model-00022-of-00026.safetensors"
	},
	"model.language_model.layers.33.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 48673,
	"changed_from_base_fraction": 0.7426910400390625,
	"changed_from_round1_elements": 31330,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00055694580078125,
	"module": "base_model.model.model.layers.33.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 9.246898753329683e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00022-of-00026.safetensors"
	},
	"model.language_model.layers.33.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10909430,
	"changed_from_base_fraction": 0.6502526998519897,
	"changed_from_round1_elements": 6319847,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001220703125,
	"module": "base_model.model.model.layers.33.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 0.00010028798313212791,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00022-of-00026.safetensors"
	},
	"model.language_model.layers.33.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5999651,
	"changed_from_base_fraction": 0.7152141332626343,
	"changed_from_round1_elements": 3623498,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001190185546875,
	"module": "base_model.model.model.layers.33.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 0.00010424333896815218,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00022-of-00026.safetensors"
	},
	"model.language_model.layers.33.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5184738,
	"changed_from_base_fraction": 0.6180689334869385,
	"changed_from_round1_elements": 3066342,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00115966796875,
	"module": "base_model.model.model.layers.33.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.289390623967547e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00022-of-00026.safetensors"
	},
	"model.language_model.layers.34.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 40663,
	"changed_from_base_fraction": 0.6204681396484375,
	"changed_from_round1_elements": 24328,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0008544921875,
	"module": "base_model.model.model.layers.34.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 8.94923818246955e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00023-of-00026.safetensors"
	},
	"model.language_model.layers.34.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 46075,
	"changed_from_base_fraction": 0.7030487060546875,
	"changed_from_round1_elements": 29945,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0003662109375,
	"module": "base_model.model.model.layers.34.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.623493961533599e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00023-of-00026.safetensors"
	},
	"model.language_model.layers.34.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10708613,
	"changed_from_base_fraction": 0.6382830739021301,
	"changed_from_round1_elements": 6293611,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010986328125,
	"module": "base_model.model.model.layers.34.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 9.605744535491254e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00023-of-00026.safetensors"
	},
	"model.language_model.layers.34.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5958854,
	"changed_from_base_fraction": 0.710350751876831,
	"changed_from_round1_elements": 3612789,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00115966796875,
	"module": "base_model.model.model.layers.34.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 0.00010116970767495327,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00023-of-00026.safetensors"
	},
	"model.language_model.layers.34.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5518636,
	"changed_from_base_fraction": 0.6578726768493652,
	"changed_from_round1_elements": 3302249,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00079345703125,
	"module": "base_model.model.model.layers.34.linear_attn.out_proj",
	"rms_effective_bf16_delta": 9.38996420599654e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00023-of-00026.safetensors"
	},
	"model.language_model.layers.35.self_attn.k_proj.weight": {
	"changed_from_base_elements": 690388,
	"changed_from_base_fraction": 0.6584053039550781,
	"changed_from_round1_elements": 432776,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00054931640625,
	"module": "base_model.model.model.layers.35.self_attn.k_proj",
	"rms_effective_bf16_delta": 7.802179036060055e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00023-of-00026.safetensors"
	},
	"model.language_model.layers.35.self_attn.o_proj.weight": {
	"changed_from_base_elements": 5132304,
	"changed_from_base_fraction": 0.6118183135986328,
	"changed_from_round1_elements": 3344461,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010986328125,
	"module": "base_model.model.model.layers.35.self_attn.o_proj",
	"rms_effective_bf16_delta": 9.380541725705772e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00023-of-00026.safetensors"
	},
	"model.language_model.layers.35.self_attn.q_proj.weight": {
	"changed_from_base_elements": 11146707,
	"changed_from_base_fraction": 0.6643955111503601,
	"changed_from_round1_elements": 6987602,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0013427734375,
	"module": "base_model.model.model.layers.35.self_attn.q_proj",
	"rms_effective_bf16_delta": 8.689808335234204e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00023-of-00026.safetensors"
	},
	"model.language_model.layers.35.self_attn.v_proj.weight": {
	"changed_from_base_elements": 729376,
	"changed_from_base_fraction": 0.695587158203125,
	"changed_from_round1_elements": 471089,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001323699951171875,
	"module": "base_model.model.model.layers.35.self_attn.v_proj",
	"rms_effective_bf16_delta": 0.00010167408347723381,
	"shape": [
	512,
	2048
	],
	"shard": "model-00023-of-00026.safetensors"
	},
	"model.language_model.layers.36.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 44847,
	"changed_from_base_fraction": 0.6843109130859375,
	"changed_from_round1_elements": 26274,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0006103515625,
	"module": "base_model.model.model.layers.36.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 0.00011075183923671724,
	"shape": [
	32,
	2048
	],
	"shard": "model-00024-of-00026.safetensors"
	},
	"model.language_model.layers.36.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 50187,
	"changed_from_base_fraction": 0.7657928466796875,
	"changed_from_round1_elements": 35259,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.36.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 9.08482572719799e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00024-of-00026.safetensors"
	},
	"model.language_model.layers.36.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10660430,
	"changed_from_base_fraction": 0.6354111433029175,
	"changed_from_round1_elements": 6365503,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001995086669921875,
	"module": "base_model.model.model.layers.36.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 0.00010201097014785468,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00024-of-00026.safetensors"
	},
	"model.language_model.layers.36.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 6022287,
	"changed_from_base_fraction": 0.7179125547409058,
	"changed_from_round1_elements": 3907487,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.36.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 0.00010200691580254258,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00024-of-00026.safetensors"
	},
	"model.language_model.layers.36.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5259156,
	"changed_from_base_fraction": 0.6269402503967285,
	"changed_from_round1_elements": 3501435,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010986328125,
	"module": "base_model.model.model.layers.36.linear_attn.out_proj",
	"rms_effective_bf16_delta": 8.514353110152134e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00024-of-00026.safetensors"
	},
	"model.language_model.layers.37.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 41306,
	"changed_from_base_fraction": 0.630279541015625,
	"changed_from_round1_elements": 25939,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.37.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 9.075526212392864e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00024-of-00026.safetensors"
	},
	"model.language_model.layers.37.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 47667,
	"changed_from_base_fraction": 0.7273406982421875,
	"changed_from_round1_elements": 35282,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0006103515625,
	"module": "base_model.model.model.layers.37.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 8.455174747374792e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00024-of-00026.safetensors"
	},
	"model.language_model.layers.37.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10697563,
	"changed_from_base_fraction": 0.6376244425773621,
	"changed_from_round1_elements": 6476544,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0012664794921875,
	"module": "base_model.model.model.layers.37.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 0.00010047640710325602,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00024-of-00026.safetensors"
	},
	"model.language_model.layers.37.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 6133336,
	"changed_from_base_fraction": 0.7311506271362305,
	"changed_from_round1_elements": 4008346,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00140380859375,
	"module": "base_model.model.model.layers.37.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 0.00010564545033233658,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00024-of-00026.safetensors"
	},
	"model.language_model.layers.37.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5651545,
	"changed_from_base_fraction": 0.67371666431427,
	"changed_from_round1_elements": 4044189,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010833740234375,
	"module": "base_model.model.model.layers.37.linear_attn.out_proj",
	"rms_effective_bf16_delta": 0.00010830739000305004,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00024-of-00026.safetensors"
	},
	"model.language_model.layers.38.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 41451,
	"changed_from_base_fraction": 0.6324920654296875,
	"changed_from_round1_elements": 26220,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.38.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 9.510995748775557e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00025-of-00026.safetensors"
	},
	"model.language_model.layers.38.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 46651,
	"changed_from_base_fraction": 0.7118377685546875,
	"changed_from_round1_elements": 30827,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00054931640625,
	"module": "base_model.model.model.layers.38.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 9.034939394162375e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00025-of-00026.safetensors"
	},
	"model.language_model.layers.38.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10948974,
	"changed_from_base_fraction": 0.6526097059249878,
	"changed_from_round1_elements": 6989297,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00146484375,
	"module": "base_model.model.model.layers.38.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 0.00011139941384783658,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00025-of-00026.safetensors"
	},
	"model.language_model.layers.38.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 6145820,
	"changed_from_base_fraction": 0.7326388359069824,
	"changed_from_round1_elements": 4016799,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00146484375,
	"module": "base_model.model.model.layers.38.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 0.00011233124404451418,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00025-of-00026.safetensors"
	},
	"model.language_model.layers.38.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5734733,
	"changed_from_base_fraction": 0.6836334466934204,
	"changed_from_round1_elements": 4031691,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010986328125,
	"module": "base_model.model.model.layers.38.linear_attn.out_proj",
	"rms_effective_bf16_delta": 0.00011524470527460758,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00025-of-00026.safetensors"
	},
	"model.language_model.layers.39.self_attn.k_proj.weight": {
	"changed_from_base_elements": 744442,
	"changed_from_base_fraction": 0.7099552154541016,
	"changed_from_round1_elements": 497152,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.39.self_attn.k_proj",
	"rms_effective_bf16_delta": 7.563793790538429e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00026-of-00026.safetensors"
	},
	"model.language_model.layers.39.self_attn.o_proj.weight": {
	"changed_from_base_elements": 5507689,
	"changed_from_base_fraction": 0.6565676927566528,
	"changed_from_round1_elements": 3545938,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010986328125,
	"module": "base_model.model.model.layers.39.self_attn.o_proj",
	"rms_effective_bf16_delta": 0.00010270824831516126,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00026-of-00026.safetensors"
	},
	"model.language_model.layers.39.self_attn.q_proj.weight": {
	"changed_from_base_elements": 11262917,
	"changed_from_base_fraction": 0.6713221669197083,
	"changed_from_round1_elements": 7286733,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010986328125,
	"module": "base_model.model.model.layers.39.self_attn.q_proj",
	"rms_effective_bf16_delta": 8.799169598088403e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00026-of-00026.safetensors"
	},
	"model.language_model.layers.39.self_attn.v_proj.weight": {
	"changed_from_base_elements": 830406,
	"changed_from_base_fraction": 0.7919368743896484,
	"changed_from_round1_elements": 561278,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001190185546875,
	"module": "base_model.model.model.layers.39.self_attn.v_proj",
	"rms_effective_bf16_delta": 0.00011691026157727887,
	"shape": [
	512,
	2048
	],
	"shard": "model-00026-of-00026.safetensors"
	},
	"model.language_model.layers.4.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 39980,
	"changed_from_base_fraction": 0.61004638671875,
	"changed_from_round1_elements": 39816,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0006103515625,
	"module": "base_model.model.model.layers.4.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 8.024908435996157e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00004-of-00026.safetensors"
	},
	"model.language_model.layers.4.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 45411,
	"changed_from_base_fraction": 0.6929168701171875,
	"changed_from_round1_elements": 45640,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.4.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 6.820581114168836e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00004-of-00026.safetensors"
	},
	"model.language_model.layers.4.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10472535,
	"changed_from_base_fraction": 0.6242117285728455,
	"changed_from_round1_elements": 10497201,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000762939453125,
	"module": "base_model.model.model.layers.4.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 7.423169572809558e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00004-of-00026.safetensors"
	},
	"model.language_model.layers.4.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5458783,
	"changed_from_base_fraction": 0.6507376432418823,
	"changed_from_round1_elements": 5380608,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000732421875,
	"module": "base_model.model.model.layers.4.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 7.415020020046006e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00004-of-00026.safetensors"
	},
	"model.language_model.layers.4.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 4842240,
	"changed_from_base_fraction": 0.577239990234375,
	"changed_from_round1_elements": 4690317,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.4.linear_attn.out_proj",
	"rms_effective_bf16_delta": 5.6341563341890654e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00004-of-00026.safetensors"
	},
	"model.language_model.layers.5.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 36954,
	"changed_from_base_fraction": 0.563873291015625,
	"changed_from_round1_elements": 34821,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00067138671875,
	"module": "base_model.model.model.layers.5.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.908453169709464e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00004-of-00026.safetensors"
	},
	"model.language_model.layers.5.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 45055,
	"changed_from_base_fraction": 0.6874847412109375,
	"changed_from_round1_elements": 45198,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00030517578125,
	"module": "base_model.model.model.layers.5.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 6.656953782415742e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00004-of-00026.safetensors"
	},
	"model.language_model.layers.5.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10373943,
	"changed_from_base_fraction": 0.6183351874351501,
	"changed_from_round1_elements": 10249644,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00079345703125,
	"module": "base_model.model.model.layers.5.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 7.285539636718919e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00004-of-00026.safetensors"
	},
	"model.language_model.layers.5.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5398227,
	"changed_from_base_fraction": 0.6435188055038452,
	"changed_from_round1_elements": 5291185,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00079345703125,
	"module": "base_model.model.model.layers.5.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 7.219678309162771e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00004-of-00026.safetensors"
	},
	"model.language_model.layers.5.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 4970646,
	"changed_from_base_fraction": 0.5925471782684326,
	"changed_from_round1_elements": 4649534,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0006103515625,
	"module": "base_model.model.model.layers.5.linear_attn.out_proj",
	"rms_effective_bf16_delta": 6.0648198292888544e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00004-of-00026.safetensors"
	},
	"model.language_model.layers.6.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 38780,
	"changed_from_base_fraction": 0.59173583984375,
	"changed_from_round1_elements": 38021,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0006103515625,
	"module": "base_model.model.model.layers.6.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.117397091117397e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00005-of-00026.safetensors"
	},
	"model.language_model.layers.6.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 43242,
	"changed_from_base_fraction": 0.659820556640625,
	"changed_from_round1_elements": 43407,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0003739595413208008,
	"module": "base_model.model.model.layers.6.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 6.940429126071299e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00005-of-00026.safetensors"
	},
	"model.language_model.layers.6.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10351489,
	"changed_from_base_fraction": 0.6169968247413635,
	"changed_from_round1_elements": 10114582,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.001117706298828125,
	"module": "base_model.model.model.layers.6.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 7.364370105770223e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00005-of-00026.safetensors"
	},
	"model.language_model.layers.6.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5560208,
	"changed_from_base_fraction": 0.6628284454345703,
	"changed_from_round1_elements": 5318671,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.6.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 7.529367979499003e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00005-of-00026.safetensors"
	},
	"model.language_model.layers.6.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 4966000,
	"changed_from_base_fraction": 0.5919933319091797,
	"changed_from_round1_elements": 4594875,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.6.linear_attn.out_proj",
	"rms_effective_bf16_delta": 5.961404818933803e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00005-of-00026.safetensors"
	},
	"model.language_model.layers.7.self_attn.k_proj.weight": {
	"changed_from_base_elements": 661506,
	"changed_from_base_fraction": 0.6308612823486328,
	"changed_from_round1_elements": 682677,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.7.self_attn.k_proj",
	"rms_effective_bf16_delta": 7.067536790705216e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00005-of-00026.safetensors"
	},
	"model.language_model.layers.7.self_attn.o_proj.weight": {
	"changed_from_base_elements": 5113964,
	"changed_from_base_fraction": 0.6096320152282715,
	"changed_from_round1_elements": 4449606,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00079345703125,
	"module": "base_model.model.model.layers.7.self_attn.o_proj",
	"rms_effective_bf16_delta": 7.080729971930253e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00005-of-00026.safetensors"
	},
	"model.language_model.layers.7.self_attn.q_proj.weight": {
	"changed_from_base_elements": 10590889,
	"changed_from_base_fraction": 0.631266176700592,
	"changed_from_round1_elements": 10567738,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0009765625,
	"module": "base_model.model.model.layers.7.self_attn.q_proj",
	"rms_effective_bf16_delta": 7.40017609793409e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00005-of-00026.safetensors"
	},
	"model.language_model.layers.7.self_attn.v_proj.weight": {
	"changed_from_base_elements": 719821,
	"changed_from_base_fraction": 0.6864748001098633,
	"changed_from_round1_elements": 699725,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000949859619140625,
	"module": "base_model.model.model.layers.7.self_attn.v_proj",
	"rms_effective_bf16_delta": 8.238527252719216e-05,
	"shape": [
	512,
	2048
	],
	"shard": "model-00005-of-00026.safetensors"
	},
	"model.language_model.layers.8.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 36348,
	"changed_from_base_fraction": 0.55462646484375,
	"changed_from_round1_elements": 34754,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0003662109375,
	"module": "base_model.model.model.layers.8.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 6.411371916081861e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00007-of-00026.safetensors"
	},
	"model.language_model.layers.8.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 47509,
	"changed_from_base_fraction": 0.7249298095703125,
	"changed_from_round1_elements": 45825,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.8.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 6.999336439184671e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00007-of-00026.safetensors"
	},
	"model.language_model.layers.8.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10457812,
	"changed_from_base_fraction": 0.6233341693878174,
	"changed_from_round1_elements": 9923692,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0010771006345748901,
	"module": "base_model.model.model.layers.8.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 7.903955210642053e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00006-of-00026.safetensors"
	},
	"model.language_model.layers.8.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5633351,
	"changed_from_base_fraction": 0.6715477705001831,
	"changed_from_round1_elements": 5202096,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00103759765625,
	"module": "base_model.model.model.layers.8.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 8.130170606636331e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00006-of-00026.safetensors"
	},
	"model.language_model.layers.8.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5311705,
	"changed_from_base_fraction": 0.6332045793533325,
	"changed_from_round1_elements": 4406999,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00079345703125,
	"module": "base_model.model.model.layers.8.linear_attn.out_proj",
	"rms_effective_bf16_delta": 7.404814731691594e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00006-of-00026.safetensors"
	},
	"model.language_model.layers.9.linear_attn.in_proj_a.weight": {
	"changed_from_base_elements": 39113,
	"changed_from_base_fraction": 0.5968170166015625,
	"changed_from_round1_elements": 38076,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00048828125,
	"module": "base_model.model.model.layers.9.linear_attn.in_proj_a",
	"rms_effective_bf16_delta": 7.49421331488855e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00007-of-00026.safetensors"
	},
	"model.language_model.layers.9.linear_attn.in_proj_b.weight": {
	"changed_from_base_elements": 45887,
	"changed_from_base_fraction": 0.7001800537109375,
	"changed_from_round1_elements": 44495,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00042724609375,
	"module": "base_model.model.model.layers.9.linear_attn.in_proj_b",
	"rms_effective_bf16_delta": 7.018184983915064e-05,
	"shape": [
	32,
	2048
	],
	"shard": "model-00007-of-00026.safetensors"
	},
	"model.language_model.layers.9.linear_attn.in_proj_qkv.weight": {
	"changed_from_base_elements": 10470688,
	"changed_from_base_fraction": 0.6241016387939453,
	"changed_from_round1_elements": 9802270,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.000946044921875,
	"module": "base_model.model.model.layers.9.linear_attn.in_proj_qkv",
	"rms_effective_bf16_delta": 7.70431663340467e-05,
	"shape": [
	8192,
	2048
	],
	"shard": "model-00006-of-00026.safetensors"
	},
	"model.language_model.layers.9.linear_attn.in_proj_z.weight": {
	"changed_from_base_elements": 5617997,
	"changed_from_base_fraction": 0.6697174310684204,
	"changed_from_round1_elements": 5138013,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.00091552734375,
	"module": "base_model.model.model.layers.9.linear_attn.in_proj_z",
	"rms_effective_bf16_delta": 7.911620981435632e-05,
	"shape": [
	4096,
	2048
	],
	"shard": "model-00006-of-00026.safetensors"
	},
	"model.language_model.layers.9.linear_attn.out_proj.weight": {
	"changed_from_base_elements": 5346855,
	"changed_from_base_fraction": 0.6373947858810425,
	"changed_from_round1_elements": 4354925,
	"formula_bit_exact": true,
	"max_abs_effective_bf16_delta": 0.0007781982421875,
	"module": "base_model.model.model.layers.9.linear_attn.out_proj",
	"rms_effective_bf16_delta": 7.257326413407083e-05,
	"shape": [
	2048,
	4096
	],
	"shard": "model-00006-of-00026.safetensors"
	}
	},
	"target_tensor_count": 190,
	"unchanged_bytes_compared": 69337387016,
	"verification_seconds": 343.941,
	"verified_at": "2026-07-09T22:43:31.176313+00:00"
	}