Update README.md
Browse files
README.md
CHANGED
|
@@ -9,6 +9,11 @@ tags:
|
|
| 9 |
# The-Omega-Directive-12B-v1.0
|
| 10 |
5 layers removed. It doesnt work. Dont download it. I am testing out some fine tuning on these "scooped" models.
|
| 11 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
| 13 |
|
| 14 |
## Merge Details
|
|
|
|
| 9 |
# The-Omega-Directive-12B-v1.0
|
| 10 |
5 layers removed. It doesnt work. Dont download it. I am testing out some fine tuning on these "scooped" models.
|
| 11 |
|
| 12 |
+
Using [Acree-AI/PruneMe](https://github.com/arcee-ai/PruneMe), I detected the least used layers, and removed them.
|
| 13 |
+
|
| 14 |
+
I am then hoping to fine tune the hell out the hell out of it, to rebalance the parameters.
|
| 15 |
+
|
| 16 |
+
|
| 17 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
| 18 |
|
| 19 |
## Merge Details
|