Create README.md

README.md

@@ -0,0 +1,32 @@
+---
+license: apache-2.0
+---
+EDIT: Works pretty well for a model with no finetuning, has promise. Better and lighter than the 14b.
+
+A 13b Mistral base model, based on the NeverSleep recipe. We've had second Mistral, why not third Mistral?
+
+### Recipe
+
+slices
+
+  - sources:
+  - 
+    - model: mistralai/Mistral-7B-v0.1
+    - 
+      layer_range: [0, 24]
+
+  - sources:
+  - 
+    - model: mistralai/Mistral-7B-v0.1
+    - 
+      layer_range: [12, 24]
+            
+  - sources:
+  - 
+    - model: mistralai/Mistral-7B-v0.1
+    - 
+      layer_range: [8, 32]
+      
+merge_method: passthrough
+
+dtype: bfloat16