README.md

---
library_name: transformers
tags:
- pacer
- model-merging
- merged-model
- moe
license: apache-2.0
---

# pacer-merge

This model was created using **PACER (Permutation-Aligned Consensus Expert Routing)**.

## !!! Attention

 - Ill upload a stable release of Qwen-Pacer with its tokenizer callled 'Qwacer-Stable' which ill also turn to GGUF
 - My next plan? Merge GLM 4.6V and GLM 4.5V

 - What have i seen from this model (Qwacer):
    - Decent UI/UX
    - Hallucinates another User in the 3rd MultiTurn Chat so beware of that

## Model Details

**Merge Type:** PACER (Base-Free, Interference-Aware)

**Source Models:**
- `fluently/FluentlyQwen3-Coder-4B-0909`
- `SamuelBang/AesCoder-4B`

**Merge Configuration:**
- Interference Threshold: `0.35`
- Top-K Experts: `2`
- Merged Layers: `0`
- MoE Layers: `108`

## How PACER Works

PACER is a novel model merging framework that:
1. **Aligns models geometrically** using Git Re-Basin
2. **Computes a Consensus Barycenter** as a synthetic base
3. **Analyzes interference** per layer
4. **Merges low-interference layers** using DARE-TIES
5. **Upcycles high-interference layers** to Mixture-of-Experts

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Akicou/Qwacer")
# This below is temporary:
# Use the tokenizer from one of the models used in merging
# The problem with the tokenizer not being saved has already been fixed on the pacer github repo
tokenizer = AutoTokenizer.from_pretrained("fluently/FluentlyQwen3-Coder-4B-0909")

# Use the model
inputs = tokenizer("Hello, world!", return_tensors="pt")
outputs = model.generate(**inputs)
```

## Created With

[PacerKit](https://github.com/Akicuo/pacer) - PACER Model Merging Framework

**Created:** 2025-12-09 21:46:52