New blend, then trained on top with my blacksheep dataset

README.md

@@ -1,8 +1,48 @@
----
-license: apache-2.0
----
-
-Please Dont Download This Yet, It has only seen 1 Epoch of my experimental Dataset
-- This is not ready for use yet for RP, context length is only 8k currently.
-
-I am fine tuning on a 32k context length dataset so it will take me a couple days before its ready to be used.
+---
+base_model:
+- mlabonne/NeuralDaredevil-7B
+- fhai50032/RolePlayLake-7B
+library_name: transformers
+tags:
+- mergekit
+- merge
+
+---
+# BlackSheep-RP
+
+This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+
+## Merge Details
+### Merge Method
+
+This model was merged using the SLERP merge method.
+
+### Models Merged
+
+The following models were included in the merge:
+* [mlabonne/NeuralDaredevil-7B](https://huggingface.co/mlabonne/NeuralDaredevil-7B)
+* [fhai50032/RolePlayLake-7B](https://huggingface.co/fhai50032/RolePlayLake-7B)
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+slices:
+  - sources:
+      - model: mlabonne/NeuralDaredevil-7B
+        layer_range: [0, 32]
+      - model: fhai50032/RolePlayLake-7B
+        layer_range: [0, 32]
+merge_method: slerp
+base_model: mlabonne/NeuralDaredevil-7B
+parameters:
+  t:
+    - filter: self_attn
+      value: [0, 0.5, 0.3, 0.7, 1]
+    - filter: mlp
+      value: [1, 0.5, 0.7, 0.3, 0]
+    - value: 0.5
+dtype: bfloat16
+
+```

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "mlabonne/NeuralDaredevil-7B",
+  "_name_or_path": "BlackSheep-RP",
   "architectures": [
     "MistralForCausalLM"
   ],

mergekit_config.yml

@@ -2,7 +2,7 @@ slices:
   - sources:
       - model: mlabonne/NeuralDaredevil-7B
         layer_range: [0, 32]
-      - model: SanjiWatsuki/Kunoichi-DPO-v2-7B
+      - model: fhai50032/RolePlayLake-7B
         layer_range: [0, 32]
 merge_method: slerp
 base_model: mlabonne/NeuralDaredevil-7B

model-00001-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf0f1ffd685c36842849ae7fa27c8e4d9bc4d4095ec3dcab3a6790525ff93d6f
+oid sha256:bd3a55614bf5acdfe42c903599e5885fe70c1535943c5b417c09237be29d0581
 size 4886547008

model-00002-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1b5188777153ac6b930f738c15ad166b2fe3bd2adb2709bf10bb3852b74fc83e
+oid sha256:dcc4dfc50f888b2cdb184f72d99fe4b8493248d6745caf020cc5f03d3c09204e
 size 4915916176

model-00003-of-00003.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b75c0a61e826ed3a389572c8f3c86ae24d43bd5dfd1e5148a8f65e624bf31e6a
+oid sha256:9ad941b6ef97df0f20367f3f2b69b64657b093e72cfe77ff9de1e4cc4564c567
 size 4681034848