Joseph717171 commited on
Commit
398ba37
·
verified ·
1 Parent(s): 2acb90c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -3
README.md CHANGED
@@ -1,13 +1,28 @@
1
  ---
2
- base_model: []
3
- library_name: transformers
4
  tags:
 
 
 
 
5
  - mergekit
6
  - merge
7
-
 
 
 
8
  ---
 
 
 
9
  # Genstruct-10.7B
10
 
 
 
 
 
 
 
11
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
 
13
  ## Merge Details
 
1
  ---
2
+ base_model: mistralai/Mistral-7B-v0.1
 
3
  tags:
4
+ - Mistral
5
+ - instruct
6
+ - finetune
7
+ - synthetic
8
  - mergekit
9
  - merge
10
+ license: apache-2.0
11
+ language:
12
+ - en
13
+ library_name: transformers
14
  ---
15
+
16
+ # Credit for the model card's description goes to ddh0, mergekit, and, migtissera
17
+ # Inspired by ddh0/Starling-LM-10.7B-beta and ddh0/Mistral-10.7B-Instruct-v0.2
18
  # Genstruct-10.7B
19
 
20
+
21
+ This is Genstruct-10.7B, a depth-upscaled version of [NousResearch/Genstruct-7B](https://huggingface.co/NousResearch/Genstruct-7B).
22
+
23
+ This model is intended to be used as a basis for further fine-tuning, or as a drop-in upgrade from the original 7 billion parameter model.
24
+
25
+ Paper detailing how Depth-Up Scaling works: [SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling](https://arxiv.org/abs/2312.15166)
26
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
27
 
28
  ## Merge Details