Czarj/Z_Basorion_0.1

My first merge. Reasons for creation: The basic Gemma4 is good at handling the Russian language, but it doesn’t hold the character’s logic and loyalty well. At the same time, Orion is very good at holding logic and loyalty, but the Russian language is broken.

---

base_model: google/gemma-4-26B-A4B-it tags: - merge - gemma - roleplay - russian merge_config: merge_method: moe_della base_weight: 1.0 donor_weight: 0.05 (attn/mlp), 0.5 (embed_tokens) density: 0.9 epsilon: 0.09

Orion-Della-26B-A4B-v1

Merge of Orion-26B-A4B-v1 into gemma-4-26B-A4B-it using moe_della.

Profile

• Russian: native-quality (from base Gemma-4)
• Character depth: emotional layer preserved (Gemma signature)
• Loyalty: moderate — character may capitulate under sustained pressure (base Gemma trope)
• Thinking: coherent English reasoning, Russian output

Use case

Tension-driven roleplay where loyalty is negotiable but emotional depth and Russian language quality are important.

Technical notes

• Built on consumer hardware (32GB RAM, SATA SSD)
• Merged with low attention/mlp weights (0.05) to survive memory constraints
• Router weights normalized, experts blended via DELLA method
• No parcellation (Orion's punctuation quirks resolved by base syntax)

Limitations

Not suitable for strict loyalty scenarios — character may engage in infidelity under sustained seduction pressure. For zero-tolerance loyalty, consider full LoRA fine-tuning instead.

architecture: Gemma4ForConditionalGeneration
base_model: D:/ai/merge/base_sharded
models:
  - model: D:/ai/merge/orion_sharded
    parameters:
      weight:
        - filter: "embed_tokens"
          value: 0.5
        - filter: "attn"
          value: 0.05
        - filter: "mlp"
          value: 0.05
        - value: 0.05
      density: 0.9
      epsilon: 0.09
merge_method: moe_della
parameters:
  lambda: 1.0
  normalize_weights: false
  normalize_router: true
  rescale: true
  router_strategy: della
  blend_experts: true
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: union
chat_template: auto