Update README.md

README.md

@@ -1,3 +1,39 @@
----
-license: mpl-2.0
----
+---
+datasets:
+- glaiveai/reasoning-v1-20m
+language:
+- en
+base_model:
+- facebook/galactica-1.3b
+tags:
+- reasoning
+- text-generation-inference
+- medical
+- science
+- chemistry
+- biology
+---
+## What is Galactic Reasoning?
+
+The Galactic Reasoning adapters are a collection of LoRA adapters, trained for the various sizes of the Facebook/Galactica models. These LoRAs enable the OPT architecture based Galactica models to use reasoning, inspired by more modern models like DeepSeek and OpenAI's O3.
+To achieve this, the [glaiveai/reasoning-v1-20m](https://huggingface.co/datasets/glaiveai/reasoning-v1-20m) dataset was used for both training and evalulation of points.
+
+|  Size       | Parameters  | Galactic Reasoning Adapter |
+|:-----------:|:-----------:|:--------------------------:|              
+| `mini`      |    125 M    | Coming Soon™               |
+| `base`      |    1.3 B    | In Progress                |
+| `standard`  |    6.7 B    | Coming Soon™               |
+| `large`     |     30 B    | Coming Soon™               |
+| `huge`      |    120 B    | Short of a GPU grant, unlikely to happen. |
+
+## How were these adapters developed?
+These adapters make use of the RS-LoRA finetuning method, trained locally on a Tesla P40. A rank of 64 and an alpha of 32 were utilized. 
+In addition, the following modules were targeted: ""q_proj", "v_proj", "k_proj", "out_proj", "fc1", "fc2"", including FC1 and FC2 due to the OPT architecture.
+
+## License
+Mozilla Public License 2.0 Modified with CC. See license.md.
+
+## Credits
+* Credit to Meta/Facebook for the Galactica OPT Based models.
+* Credit to GlaiveAi for the reasoning-v1-20m dataset.
+* Finally, credit to my highly overworked Tesla P40 who ran for days straight to produce this.