| | --- |
| | license: apache-2.0 |
| | --- |
| | |
| | Contributors: Nicolas Mejia Petit |
| |
|
| |  |
| |
|
| | # Mistral 29b: A New Base Model |
| |
|
| | The objective of this model is to serve as a new fully open source base model with 29.2 billion parameters. |
| |
|
| | This model spits out jargon, and needs to be fine tuned, either with qlora, with the adapter attached to every layer, or better yet a full fine tune. |
| |
|
| | ## Model Creation |
| |
|
| | The model was created by stacking four models: Dolphin, Zephyr, Meta-math7b, and Speechless code, to form a single model. |
| |
|
| | ## Useful Resources |
| |
|
| | - Model Merging (MergeKit) (https://github.com/cg123/mergekit) |
| |
|
| | ## Source Models |
| |
|
| | - Dolphin2.1-mistral-7b by Eric Hartford (https://huggingface.co/ehartford/dolphin-2.1-mistral-7b) |
| | - Zephyr-7b-beta by HuggingFace (https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) |
| | - MetaMath-Mistral-7B by meta-math (https://huggingface.co/meta-math/MetaMath-Mistral-7B) |
| | - speechless-code-mistral-7b-v1.0 (https://huggingface.co/uukuguy/speechless-code-mistral-7b-v1.0) |
| |
|
| |
|
