---
base_model: google/gemma-3-4b-it
library_name: transformers
tags:
- mergekit
- merge
- russian
- multimodal
- gemma-3
- vision
language:
- ru
- en
---

# MIXdevAI-gemma3-4B

**MIXdevAI-gemma3-4B** is an experimental merged model based on **Google Gemma-3-4B**, combining the best qualities of several fine-tuned versions. The model features:

- Improved reasoning capabilities (enhanced "thinking")
- Strong vision understanding (fully multimodal)
- Natural Russian language support
- Compact 4B parameter size

> This model was created using weight merging with [mergekit](https://github.com/cg123/mergekit).

---

## Key Features

- **Vision support**: Works as a full Vision-Language model.
- **Russian language**: Trained on Russian data and prompts.
- **Improved reasoning**: Demonstrates chain-of-thought and analytical abilities.
- **Compatibility**: Fully compatible with `transformers` and the Gemma-3 format.

---

## Merge Details

### Merge Method

The model was assembled using the **Linear Merge** method (weighted average) with `google/gemma-3-4b-it` as the base.

### Models Merged

The merge includes:
- `google/gemma-3-4b-it` (base multimodal model)
- `Thinking` (fine-tuned version with improved reasoning and Russian language support)

### Configuration

```yaml
# gemma-3-4b-heretic-merge.yml
merge_method: linear
name: MIXdevAI-gemma3-4B
base_model: google/gemma-3-4b-it

models:
  - model: google/gemma-3-4b-it
    parameters:
      weight: 0.5
  - model: Thinking
    parameters:
      weight: 0.5

dtype: bfloat16
tokenizer_source: union
chat_template: auto

```