Update README.md
Browse files
README.md
CHANGED
|
@@ -112,14 +112,23 @@ model-index:
|
|
| 112 |
---
|
| 113 |
# orca_mini_v2_13b
|
| 114 |
|
| 115 |
-
An **Uncensored** LLaMA-13b model in collaboration with [Eric Hartford](https://huggingface.co/ehartford). trained on explain tuned datasets, created using Instructions and Input from WizardLM, Alpaca & Dolly-V2 datasets and applying Orca Research Paper dataset construction approaches
|
| 116 |
|
| 117 |
Please note this model has *better code generation capabilities* compare to our original orca_mini_13b which was trained on base OpenLLaMA-13b model and which has the [empty spaces issues & found not good for code generation]((https://github.com/openlm-research/open_llama#update-06072023)).
|
| 118 |
|
|
|
|
| 119 |
|
| 120 |
-
|
|
|
|
| 121 |
|
| 122 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 123 |
|
| 124 |
I evaluated orca_mini_v2_13b on a wide range of tasks using [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) from EleutherAI.
|
| 125 |
|
|
@@ -138,7 +147,7 @@ Here are the results on metrics used by [HuggingFaceH4 Open LLM Leaderboard](htt
|
|
| 138 |
|
| 139 |
|
| 140 |
|
| 141 |
-
|
| 142 |
|
| 143 |
We used uncensored script on top of the previous explain tuned datasets we build which are [WizardLM dataset ~70K](https://github.com/nlpxucan/WizardLM), [Alpaca dataset ~52K](https://crfm.stanford.edu/2023/03/13/alpaca.html) & [Dolly-V2 dataset ~15K](https://github.com/databrickslabs/dolly) created using approaches from [Orca Research Paper](https://arxiv.org/abs/2306.02707).
|
| 144 |
|
|
@@ -148,7 +157,7 @@ This helps student model aka this model to learn ***thought*** process from teac
|
|
| 148 |
|
| 149 |
Please see below example usage how the **System** prompt is added before each **instruction**.
|
| 150 |
|
| 151 |
-
|
| 152 |
|
| 153 |
The training configurations are provided in the table below.
|
| 154 |
|
|
@@ -170,7 +179,7 @@ Here are some of params used during training:
|
|
| 170 |
|
| 171 |
|
| 172 |
|
| 173 |
-
|
| 174 |
|
| 175 |
Here is prompt format for [Oobabooga Text generation UI ](https://github.com/oobabooga/text-generation-webui)
|
| 176 |
|
|
|
|
| 112 |
---
|
| 113 |
# orca_mini_v2_13b
|
| 114 |
|
| 115 |
+
**An **Uncensored** LLaMA-13b model in collaboration with [Eric Hartford](https://huggingface.co/ehartford). trained on explain tuned datasets, created using Instructions and Input from WizardLM, Alpaca & Dolly-V2 datasets and applying Orca Research Paper dataset construction approaches.**
|
| 116 |
|
| 117 |
Please note this model has *better code generation capabilities* compare to our original orca_mini_13b which was trained on base OpenLLaMA-13b model and which has the [empty spaces issues & found not good for code generation]((https://github.com/openlm-research/open_llama#update-06072023)).
|
| 118 |
|
| 119 |
+
<img src="https://huggingface.co/pankajmathur/orca_mini_v5_8b/resolve/main/orca_minis_small.jpeg" width="auto" />
|
| 120 |
|
| 121 |
+
<strong>
|
| 122 |
+
Passionate about Generative AI? I help companies to privately train and deploy custom LLM/MLLM affordably. For startups, I can even assist with securing GPU grants to get you started. Let's chat!
|
| 123 |
|
| 124 |
+
<a href="https://www.linkedin.com/in/pankajam" target="_blank">https://www.linkedin.com/in/pankajam</a> Looking forward to connecting!
|
| 125 |
+
</strong>
|
| 126 |
+
|
| 127 |
+
<br>
|
| 128 |
+
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
### Evaluation
|
| 132 |
|
| 133 |
I evaluated orca_mini_v2_13b on a wide range of tasks using [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) from EleutherAI.
|
| 134 |
|
|
|
|
| 147 |
|
| 148 |
|
| 149 |
|
| 150 |
+
### Dataset
|
| 151 |
|
| 152 |
We used uncensored script on top of the previous explain tuned datasets we build which are [WizardLM dataset ~70K](https://github.com/nlpxucan/WizardLM), [Alpaca dataset ~52K](https://crfm.stanford.edu/2023/03/13/alpaca.html) & [Dolly-V2 dataset ~15K](https://github.com/databrickslabs/dolly) created using approaches from [Orca Research Paper](https://arxiv.org/abs/2306.02707).
|
| 153 |
|
|
|
|
| 157 |
|
| 158 |
Please see below example usage how the **System** prompt is added before each **instruction**.
|
| 159 |
|
| 160 |
+
### Training
|
| 161 |
|
| 162 |
The training configurations are provided in the table below.
|
| 163 |
|
|
|
|
| 179 |
|
| 180 |
|
| 181 |
|
| 182 |
+
### Example Usage
|
| 183 |
|
| 184 |
Here is prompt format for [Oobabooga Text generation UI ](https://github.com/oobabooga/text-generation-webui)
|
| 185 |
|