Hastagaras
/

J-EXP-45-WiSE

Model card Files Files and versions

You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

J-EXP-45-WiSE

A WiSE-FT mid-attention regularized build of Hastagaras/G4-Jamet-26B-A4B-MK-2.

Recipe

Base (85%): Hastagaras/G4-Jamet-26B-A4B-MK-2
Donor (15%): google/gemma-4-26B-A4B-it
Scope: Layers 10–19 Self-Attention projections (q_proj, k_proj, v_proj, o_proj)
Formula: $W = 0.85 \cdot W_{\text{MK-2}} + 0.15 \cdot W_{\text{Instruct}}$
All other weights: 100% Jamet MK-2 (experts, routers, MLPs, embeddings).

Downloads last month: 139

Safetensors

Model size

26B params

Tensor type

BF16

·

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Hastagaras/J-EXP-45-WiSE

Base model

google/gemma-4-26B-A4B

Finetuned

google/gemma-4-26B-A4B-it

Finetuned

Hastagaras/G4-Jamet-26B-A4B-MK-2

Finetuned

(1)

this model