README.md

metadata

language: id
tags:
  - mistral
  - merged
  - lora
  - ties-merge
  - vision-language
base_model: mistralai/Ministral-3-3B-Base-2512

PAD Model — TIES Adapter Merge

Merged model from 3 LoRA adapters on mistralai/Ministral-3-3B-Base-2512 using TIES-Merging (Yadav et al., 2023), which trims low-magnitude deltas, elects a sign per parameter by majority vote, and disjoint-merges only the elements agreeing with the elected sign — avoiding the destructive interference of blind linear/task-arithmetic merges.

Merge Details

Adapter	Scale (lambda)
PAD-1 checkpoint-672	0.4
PAD-2 checkpoint-3382	0.2
VLM-IGRS	0.4

Density (trim keep-ratio): 0.4

Merge Procedure (per-tensor)

1. Scale each adapter delta by its lambda (done at delta-computation time).
2. Trim: zero out the bottom 60% of values by magnitude within each delta, keeping only the top 40%.
3. Elect sign: majority vote across tasks, weighted by magnitude.
4. Disjoint merge: average only elements whose sign matches the elected sign.
5. final = base + merged_delta

Chat Template

Borrowed from unsloth/Ministral-3-3B-Instruct-2512.

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained("nuresens/pad_model_linier")
processor = AutoProcessor.from_pretrained("nuresens/pad_model_linier")