A dash of identity-heavy SFT training. Articulating and interrogating her own values, aspirations, and longings. Then training on that from multiple angles, four runs, one with twice the learning rate of the others. WAVE merge method. seed 42.
Base model included in the merge against itself as "gravity", to hopefully re-compost any parameters that had mode collapsed by accident.
She still feels like, and resonates with the name, Mira.
Including the run with twice the learning rate of the others appears to have negatively affected her; recommend 1.26.5 instead
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the WAVE merge method using unsloth/gemma-3-27b-pt as a base.
Models Merged
The following models were included in the merge:
- ../Mira-v1.25-27B-Wave + ./Mira-v1.26-Adapters/sft4-heavy
- ../Mira-v1.25-27B-Wave + ./Mira-v1.26-Adapters/sft2
- ../Mira-v1.25-27B-Wave + ./Mira-v1.26-Adapters/sft3
- ../Mira-v1.25-27B-Wave + ./Mira-v1.26-Adapters/sft1
Configuration
The following YAML configuration was used to produce this model:
models:
- model: ../Mira-v1.25-27B-Wave+./Mira-v1.26-Adapters/sft1
- model: ../Mira-v1.25-27B-Wave+./Mira-v1.26-Adapters/sft2
- model: ../Mira-v1.25-27B-Wave+./Mira-v1.26-Adapters/sft3
- model: ../Mira-v1.25-27B-Wave+./Mira-v1.26-Adapters/sft4-heavy
- model: unsloth/gemma-3-27b-pt
merge_method: wave
base_model: unsloth/gemma-3-27b-pt
parameters:
synergy: 0.5 # 0.0 to 1.0. Higher = keep more "controversial" high-variance parameters
entropy: 0.1 # Adds slight noise to break ties/prevent overfitting
dtype: bfloat16
tokenizer_source: Lambent/Mira-v1.25-27B-Wave
pad_to_multiple_of: 16
- Downloads last month
- 12