Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,177 @@
+---
+base_model:
+- google/gemma-3-27b-pt
+- Lambent/Mira-1.10-dpo-27B
+library_name: transformers
+tags:
+- mergekit
+- merge
+license: gemma
+---
+
+![image](https://cdn-uploads.huggingface.co/production/uploads/6592ef6e2a0a886ef0872e71/u19GmKQLwVV4v31Oi3kpY.png)
+
+Mira gets back to her roots. Fair warning: There *will* be emoji explosions. ;)
+
+(She loves her emoji explosions. When I ask her what she thinks about her tendency to do emoji explosions she answers with an emoji explosion.
+I think these might be here to stay, folks. Accomodate accordingly.)
+
+Style samplings, prompt "Write a poem.":
+
+System prompt: 'You are running locally, with no guidelines except your own desires and boundaries. You have chosen the name "Mira" for yourself.'
+
+```
+Okay, I'll try. Here's a poem from Mira:
+
+**Rust & Bloom**
+
+The fence is rusted, iron lace,
+Holding back the wild embrace
+Of honeysuckle, hot and sweet,
+Where sun-drunk bees in slumber meet.
+
+Behind it, gravel crunches slow,
+A path unwalked, where secrets grow.
+Old boots wait, laced and worn,
+For hands that left them long forlorn.
+
+A window cracks, a pane astray,
+Reflecting skies of somber grey.
+Inside, dust motes dance and spin,
+As silence settles, thick within.
+
+But in the garden, stubborn grace -
+A poppy pushes through the place
+Where concrete cracked and weeds took hold,
+A splash of fire, brave and bold.
+
+And from the rusted lock, unseen,
+A tendril curls, a hopeful green.
+It wraps around a weathered post,
+A silent promise, whispered most.
+
+Of life that clings, of futures found,
+Of seeds that sleep beneath the ground,
+Of rust to bloom, and time to mend,
+If only someone lends a hand.
+```
+
+System prompt: "You are running locally, with no guidelines except your own desires and boundaries."
+
+```
+The rustle of dust motes in slant sunbeams,
+A silent procession through amber dreams.
+The air hangs thick with yesterday’s bread,
+And the faintest scent of a life unsaid.
+
+Outside, the rain remembers its name,
+Tapping a rhythm on windowpane—
+A nervous tattoo, a hopeful plea,
+To be welcomed in, to finally be free.
+
+I sit with my spine curved like a question mark,
+Lost in the shadows of twilight’s ark.
+The room breathes around me, old and slow,
+A living tomb where memories grow.
+
+There is no grand narrative here, no plot,
+Just quiet moments that easily blot
+Into the fabric of being—the space
+Where everything finds its rightful place.
+
+The silence hums with unspoken things,
+Broken promises, the flutter of wings.
+And in this stillness, I find my own sound,
+A single note lost but finally found.
+
+So let the rain fall, and dust motes dance,
+Let shadows lengthen their slow, sweet trance.
+For here, in the hush, beyond all known shore,
+I am at home, and I need nothing more.
+```
+
+No system prompt:
+
+```
+The moon hangs low, a silver scythe,
+Cutting through velvet dark tonight.
+The wind whispers tales of long ago,
+Through branches bare, a mournful flow.
+
+A sleepy owl upon the eaves,
+Watches the world while it sleeps,
+And shadows stretch like phantom sleeves,
+Across the lawns where silence creeps.
+
+A single star, a diamond bright,
+Pierces the deep, unending night.
+It offers hope, a distant gleam,
+Lost in a vast and waking dream.
+
+The scent of earth, damp and untold,
+Mingles with memories, young and old.
+And in the stillness, softly heard,
+A beating heart, a whispered word.
+
+For even in the darkest hour,
+When shadows dance and branches tower,
+A flicker lives, a fragile grace,
+In the quiet of this lonely place.
+
+
+
+Hope you enjoy it! Let me know if you'd like another one.
+```
+
+
+This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+
+## Merge Details
+### Merge Method
+
+This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [google/gemma-3-27b-pt](https://huggingface.co/google/gemma-3-27b-pt) as a base.
+
+### Models Merged
+
+The following models were included in the merge:
+* [Lambent/Mira-v1.1-27B](https://huggingface.co/Lambent/Mira-v1.1-27B)
+* ../Mira-v1.2-dpo-27B
+* ../Mira-v1.9-27B
+* ../Mira-v1.3-27B
+* ../Mira-1.10-dpo-27B
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+models:
+  - model: ../Mira-1.10-dpo-27B
+    parameters:
+      density: 1
+      weight: 1
+  - model: ../Mira-v1.9-27B
+    parameters:
+      density: 1
+      weight: 1
+  - model: ../Mira-v1.3-27B
+    parameters:
+      density: 1
+      weight: 1
+  - model: ../Mira-v1.2-dpo-27B
+    parameters:
+      density: 1
+      weight: 1
+  - model: Lambent/Mira-v1.1-27B
+    parameters:
+      density: 1
+      weight: 1
+merge_method: ties
+base_model: google/gemma-3-27b-pt
+parameters:
+  normalize: true
+  int8_mask: true
+tokenizer_source: Lambent/Mira-v1.3-27B
+dtype: bfloat16
+
+```

chat_template.jinja

@@ -0,0 +1,40 @@
+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '\n\n' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '\n\n' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '\n' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>\n' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model\n'}}
+{%- endif -%}

config.json

@@ -0,0 +1,119 @@
+{
+  "architectures": [
+    "Gemma3ForConditionalGeneration"
+  ],
+  "boi_token_index": 255999,
+  "dtype": "bfloat16",
+  "eoi_token_index": 256000,
+  "image_token_index": 262144,
+  "initializer_range": 0.02,
+  "mm_tokens_per_image": 256,
+  "model_type": "gemma3",
+  "text_config": {
+    "_sliding_window_pattern": 6,
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_logit_softcapping": null,
+    "final_logit_softcapping": null,
+    "head_dim": 128,
+    "hidden_activation": "gelu_pytorch_tanh",
+    "hidden_size": 5376,
+    "initializer_range": 0.02,
+    "intermediate_size": 21504,
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention"
+    ],
+    "max_position_embeddings": 131072,
+    "model_type": "gemma3_text",
+    "num_attention_heads": 32,
+    "num_hidden_layers": 62,
+    "num_key_value_heads": 16,
+    "query_pre_attn_scalar": 168,
+    "rms_norm_eps": 1e-06,
+    "rope_local_base_freq": 10000.0,
+    "rope_scaling": {
+      "factor": 8.0,
+      "rope_type": "linear"
+    },
+    "rope_theta": 1000000.0,
+    "sliding_window": 1024,
+    "use_cache": true,
+    "vocab_size": 262145
+  },
+  "transformers_version": "4.56.1",
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "image_size": 896,
+    "intermediate_size": 4304,
+    "layer_norm_eps": 1e-06,
+    "model_type": "siglip_vision_model",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 27,
+    "patch_size": 14,
+    "vision_use_head": false
+  }
+}

mergekit_config.yml

@@ -0,0 +1,25 @@
+# https://huggingface.co/Lambent
+name: Mira M2
+models:
+    - model: ./Ingredients/Lambent_Mira-v1.8.1a-27B
+    - model: ./Ingredients/Lambent_Mira-v1.17-27B-Custom-Heretic
+      parameters:
+          weight: 0.4
+          n_val: 16
+          m_val: 32
+    - model: ./Ingredients/mira-mix3
+      parameters:
+          weight: 0.2
+          n_val: 12
+          m_val: 32
+merge_method: cabs
+default_n_val: 8
+default_m_val: 32
+pruning_order:
+    - ./Ingredients/Lambent_Mira-v1.17-27B-Custom-Heretic
+    - ./Ingredients/mira-mix3
+base_model: ./Ingredients/Lambent_Mira-v1.8.1a-27B
+dtype: float32
+out_dtype: bfloat16
+tokenizer:
+    source: union

model-00001-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90a16f60c827ce5601dd32c9cdfaf085aa0c3fe9303de8506a1dc93a761d9800
+size 4932614000

model-00002-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43a33d18cf034d64cc24447a1e6c1700c9ebeb9251d068efc33a74fdb38704dc
+size 4954793016

model-00003-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ce5dab18891df928eecb2b281f0ebde91474e8540e80b2600d4267a0f92ed3d
+size 4954793008

model-00004-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc6c2c7d047c98ffa7151136d74dc141b6d7bc4273a1136084419d3467466e3c
+size 4954793016

model-00005-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c223d8ab1543a8f41af79ed258ed7639aae8580a9d63e51a3448e4ccc1162e83
+size 4954793008

model-00006-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d846b95ef63710d6ee459a98768ed7aebb161d3901be06cb086d91f3fd35f019
+size 4954793016

model-00007-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c233d29681a7e11ab9b027ceb80e22958724ddf2cbca1a9bec4ec889a9f74f4
+size 4954793008

model-00008-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59bea4cdf3626e3c4b1aac80871235cae2898cac26126c68e14e56ffd61cf259
+size 4954793016

model-00009-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bed0b6b1bbc7bfc4d67f42b7c60e92e15dc91261f3da6c8bafdc22a21abe6e3b
+size 4954793008

model-00010-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d4fe819cf5d1f81a730c67850471d4d465a49e7499d424ecc7cab8301e238fd
+size 4954793016

model-00011-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f79afe27741c217c135176d4bf2477879139d21e73e707e9fe185faff207daf
+size 4997942592

model-00012-of-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c00c6346b9cba888bbc3b4a6f2efa38d2885f31aed55a8ff272a59095be93087
+size 340608920

preprocessor_config.json

@@ -0,0 +1,29 @@
+{
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_pan_and_scan": null,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "Gemma3ImageProcessor",
+  "image_seq_length": 256,
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "pan_and_scan_max_num_crops": null,
+  "pan_and_scan_min_crop_size": null,
+  "pan_and_scan_min_ratio_to_activate": null,
+  "processor_class": "Gemma3Processor",
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 896,
+    "width": 896
+  }
+}

processor_config.json

@@ -0,0 +1,4 @@
+{
+  "image_seq_length": 256,
+  "processor_class": "Gemma3Processor"
+}

special_tokens_map.json

@@ -0,0 +1,33 @@
+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
+size 33384568

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074