"Je meurs aussi de ta mélancolie... Un soir de demi-brume" – Émile Nelligan
Pale Sonneteer - Gothic Poet
My first model. A simple experiment with karcher. As we create new models, it becomes clear we nearly satured what can be created with mistral 24B, magistral 2509 included, whether it's fine-tuned or merged. Ironically, there are many more frontiers to explore. Please, don't stop doing so.
About
This is an attempt to learn about merging with karcher. I tried a few other methods before setting my sights on this particular one for my first serious model. The Pale Sonneteer is meant to be a writing/RP model obviously.
The model is inspired by Ateron/Sketch-Cydonia, a beautiful stroke of poetry. Its "simplicity" is built on the endeavour of more complex fine-tunes and merges, it remains quite inspirational.
Nothing is simple really at this point. In the mistral 24B space, one can only hope to find something that fit their specific desires. There are plenty of quality choices. Mine were to get a specialized model for writing in a gothic atmosphere. A success, although not an absolute one. Many supercedes the Sonneteer in prose, but none gets the equilibrium, the haunting, I was looking for.
I tip my hat to TheDrummer, Ateron and all explorers who try to push the frontiers of what can be done.
Enjoy, or don't. Feel free to share your feedbacks.
PaleSonneteer
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the Karcher Mean merge method using Darkhn/Magistral-2509-24B-Text-Only as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
merge_method: karcher
base_model: Darkhn/Magistral-2509-24B-Text-Only
dtype: float32
out_dtype: bfloat16
tokenizer_source: base
# Iterative optimization parameters
parameters:
max_iterations: 64
tol: 1e-6
step_size: 0.01
normalize: true
slices:
- sources:
- model: Darkhn/Magistral-2509-24B-Text-Only
layer_range: [0, 40] # The Architect's Bones
parameters:
weight: 0.3
- model: Ateron/Sketch-Cydonia-24B-V1.2
layer_range: [0, 40] # The Poet's Blood
parameters:
weight: 0.7
- Downloads last month
- 189