Update README.md
Browse files
README.md
CHANGED
|
@@ -19,5 +19,9 @@ extra_gated_description: >-
|
|
| 19 |
# TylerG01/Indigo-v0.1
|
| 20 |
Refer to the [original model card](https://huggingface.co/mistralai/Mistral-7B-v0.1) for more details on the model.
|
| 21 |
## Project Goals
|
| 22 |
-
This is v0.1 (alpha) release of the Indigo LLM project, which used LoRA Fine-Tuning to train Mistral 7B on more than 400 books, pamphlets,
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
For more information on this project, check out the blog post at https://t2-security.com/indigo-llm-503cd6e22fe4.
|
|
|
|
| 19 |
# TylerG01/Indigo-v0.1
|
| 20 |
Refer to the [original model card](https://huggingface.co/mistralai/Mistral-7B-v0.1) for more details on the model.
|
| 21 |
## Project Goals
|
| 22 |
+
This is v0.1 (alpha) release of the Indigo LLM project, which used LoRA Fine-Tuning to train Mistral 7B on more than 400 books, pamphlets,
|
| 23 |
+
training documents, code snippets and other works in the cyber security field, openly sourced on the surface web. This version used 16 LoRA layers
|
| 24 |
+
and had a val loss of 1.601 after the 4th training epoch. However, my goal for the LoRA version of this model is to produce a val loss of <1.51 after
|
| 25 |
+
some modification to the dataset and training approach.
|
| 26 |
+
|
| 27 |
For more information on this project, check out the blog post at https://t2-security.com/indigo-llm-503cd6e22fe4.
|