Update README.md
Browse files
README.md
CHANGED
|
@@ -6,12 +6,14 @@ license: apache-2.0
|
|
| 6 |
|
| 7 |
HelixNet-LMoE is a simple LoRA based Mixture of Experts version of the [HelixNet](https://huggingface.co/migtissera/HelixNet) 3-model system by [Migel Tissera](https://huggingface.co/migtissera).
|
| 8 |
|
|
|
|
|
|
|
| 9 |
For each HelixNet model, a separate LoRA adapter was extracted :
|
| 10 |
* [HelixNet-LMoE-Actor](https://huggingface.co/rhysjones/HelixNet-LMoE-Actor)
|
| 11 |
* [HelixNet-LMoE-Critic](https://huggingface.co/rhysjones/HelixNet-LMoE-Critic)
|
| 12 |
* [HelixNet-LMoE-Regenerator](https://huggingface.co/rhysjones/HelixNet-LMoE-Regenerator)
|
| 13 |
|
| 14 |
-
These are then loaded
|
| 15 |
|
| 16 |
As HelixNet processes its inputs using the actor, critic and regenerator actions, the corresponding LoRA adapter is dynamically enabled as required.
|
| 17 |
|
|
|
|
| 6 |
|
| 7 |
HelixNet-LMoE is a simple LoRA based Mixture of Experts version of the [HelixNet](https://huggingface.co/migtissera/HelixNet) 3-model system by [Migel Tissera](https://huggingface.co/migtissera).
|
| 8 |
|
| 9 |
+
_Update_ : There is a 6bpw LMoE version that runs the entire 3-model system much faster, using 8 GB gpu mem in total. ExLlamaV2 version here: [HelixNet-LMoE-6.0bpw-h6-exl2](https://huggingface.co/rhysjones/HelixNet-LMoE-6.0bpw-h6-exl2).
|
| 10 |
+
|
| 11 |
For each HelixNet model, a separate LoRA adapter was extracted :
|
| 12 |
* [HelixNet-LMoE-Actor](https://huggingface.co/rhysjones/HelixNet-LMoE-Actor)
|
| 13 |
* [HelixNet-LMoE-Critic](https://huggingface.co/rhysjones/HelixNet-LMoE-Critic)
|
| 14 |
* [HelixNet-LMoE-Regenerator](https://huggingface.co/rhysjones/HelixNet-LMoE-Regenerator)
|
| 15 |
|
| 16 |
+
These are then loaded together with the base [Mistral 7b](https://huggingface.co/mistralai/Mistral-7B-v0.1) model to give the combined LMoE model.
|
| 17 |
|
| 18 |
As HelixNet processes its inputs using the actor, critic and regenerator actions, the corresponding LoRA adapter is dynamically enabled as required.
|
| 19 |
|