Hugging Face's logo

Vortex5
/
Darkest-Grimoire-12B

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
roleplay
text-generation-inference
Model card Files
xet
 Community
Darkest-Grimoire-12B / README.md
Vortex5's picture
Vortex5
Update README.md
a1082da verified
preview code
|
raw
history blame contribute delete
9.27 kB
metadata 
base_model:
  - Retreatcost/Impish-LongPen-12B
  - Retreatcost/Darkstar-12B
library_name: transformers
tags:
  - mergekit
  - merge
  - roleplay

Darkest-Grimoire-12B

“Where intellect becomes scripture and thought burns black.”

Darkest-Grimoire-12B

Overview

Darkest-Grimoire-12B is a model merged using a custom harmony_flux method — it merges Darkstar-12B and Impish-LongPen-12B.

Custom Merge Method

An adaptive merge that compares model weight tensors and assigns merge weights based on structural and spectral differences, highlighting distinct and stable parameter variations instead of naively averaging them.

Read full explanation

Concept: The method evaluates each model’s tensors in two domains to determine contribution strength.

  • Spatial. Measures direct differences between model weight tensors in parameter space. Distinct structural weight patterns receive higher spatial scores.
  • Frequency. Applies FFT on flattened weight tensors to analyze spectral behavior; unique and consistent frequency structures receive higher frequency scores.

Both domains are combined into a unified per-model “goodness” value. These values are scaled by clarity, mixed via blend and global_blend, and normalized through a temperature-controlled softmax (focus) to generate final merge weights. If a base tensor is provided, it acts as the reference center; otherwise, the method computes the median tensor across donors as the neutral baseline.

Key Parameters.

  • focus (per-model/global): softmax temperature; higher = more selective contributions.
  • blend (per-model/global): mix between spatial (0) and frequency (1) domains.
  • clarity (per-model): Scales each model’s influence by its signal stability or confidence.
  • global_blend (global): global bias toward spatial (0) or frequency (1).
Show YAML 
models:
  - model: Retreatcost/Darkstar-12B
    parameters:
      weight: 0.42
      focus: 1.2
      blend: 0.65
      clarity: 0.95
  - model: Retreatcost/Impish-LongPen-12B
    parameters:
      weight: 0.58
      focus: 1.3
      blend: 0.55
      clarity: 1.1
merge_method: harmony_flux
dtype: bfloat16
parameters:
  global_blend: 0.5
  return_debug: true
tokenizer:
  source: union

Credits