Update README.md
Browse files
README.md
CHANGED
|
@@ -29,6 +29,7 @@ To achieve this, the [glaiveai/reasoning-v1-20m](https://huggingface.co/datasets
|
|
| 29 |
## How were these adapters developed?
|
| 30 |
These adapters make use of the RS-LoRA finetuning method, trained locally on a Tesla P40. A rank of 64 and an alpha of 32 were utilized.
|
| 31 |
In addition, the following modules were targeted: ""q_proj", "v_proj", "k_proj", "out_proj", "fc1", "fc2"", including FC1 and FC2 due to the OPT architecture.
|
|
|
|
| 32 |
|
| 33 |
## License
|
| 34 |
Mozilla Public License 2.0 Modified with CC. See license.md.
|
|
|
|
| 29 |
## How were these adapters developed?
|
| 30 |
These adapters make use of the RS-LoRA finetuning method, trained locally on a Tesla P40. A rank of 64 and an alpha of 32 were utilized.
|
| 31 |
In addition, the following modules were targeted: ""q_proj", "v_proj", "k_proj", "out_proj", "fc1", "fc2"", including FC1 and FC2 due to the OPT architecture.
|
| 32 |
+
The adapters were then merged back into their FP16 base model before being quantized.
|
| 33 |
|
| 34 |
## License
|
| 35 |
Mozilla Public License 2.0 Modified with CC. See license.md.
|