Update README.md
Browse files
README.md
CHANGED
|
@@ -1,22 +1,94 @@
|
|
| 1 |
---
|
| 2 |
-
|
| 3 |
tags:
|
|
|
|
|
|
|
| 4 |
- text-generation-inference
|
| 5 |
- transformers
|
| 6 |
- unsloth
|
| 7 |
- llama
|
| 8 |
- trl
|
| 9 |
-
|
|
|
|
| 10 |
language:
|
| 11 |
- en
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
-
|
| 15 |
|
| 16 |
-
-
|
| 17 |
-
- **
|
| 18 |
-
-
|
|
|
|
| 19 |
|
| 20 |
-
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
tags:
|
| 4 |
+
- unsloth
|
| 5 |
+
- Uncensored
|
| 6 |
- text-generation-inference
|
| 7 |
- transformers
|
| 8 |
- unsloth
|
| 9 |
- llama
|
| 10 |
- trl
|
| 11 |
+
- roleplay
|
| 12 |
+
- conversational
|
| 13 |
language:
|
| 14 |
- en
|
| 15 |
+
base_model:
|
| 16 |
+
- unsloth/Llama-3.2-1B-Instruct
|
| 17 |
+
pipeline_tag: text-generation
|
| 18 |
+
library_name: peft
|
| 19 |
+
metrics:
|
| 20 |
+
- character
|
| 21 |
+
- bleu
|
| 22 |
+
- rouge
|
| 23 |
---
|
| 24 |
+
# This is the Next Experimental OpenElla Architecture, Tested and Proven to Outcompete Minimaid-L2's and L1's
|
| 25 |
+
If you loved our Models, then please consider donating and supporting us through Ko-fi!
|
| 26 |
+
[](https://ko-fi.com/J3J61D8NHV)
|
| 27 |
+
Our Training Capabilities is limited to Google Colab's Free tier So any Help, Donation or Renting is very much appreciated!
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+

|
| 31 |
+
|
| 32 |
+
# OpenElla StoryWriter - TypeB
|
| 33 |
+
- Introducing OpenElla StoryWriter - TypeB!
|
| 34 |
+
- OpenElla StoryWriter TypeB is our brand new Model trained on a 1B llama Instruct model, capable of writing good RP responses,
|
| 35 |
+
without sacrificing Speed or Performance! The New Architecture is trained on **HERETICAL** and finetuned for RP dataset.
|
| 36 |
+
Which paves the path for the model's incredible performance compared to other 1B models, and are capable of Generating responses
|
| 37 |
+
On-par to 3B counterparts
|
| 38 |
+
|
| 39 |
+
# OpenElla StoryWriter - TypeB Base-Model Card Procedure:
|
| 40 |
+
- **TypeB**
|
| 41 |
+
TypeB Model contains 1B params, Type A is yet to be released and will contain 3B params Counterpart. There is **NO PLAN** to bump the
|
| 42 |
+
training model to **7B**, this is due to Google colab's free tier limit, limiting our training time and the intensity of training!
|
| 43 |
+
|
| 44 |
+
TypeB is trained using **HERETICAL** tool, its base model, Llama3.2 1B was taken from HF, and was trained on Heretical to remove as much Wastes as much as possible.
|
| 45 |
+
However, please bear in mind that the model is **UNCENSORED** or atmost should be due to heretical training,
|
| 46 |
+
|
| 47 |
+
**N-BOT-INT is NOT LIABLE FOR ANY DAMAGES, MISUSE, OR UNINTENDED OUTCOMES RESULTING FROM USER-GENERATED PROMPT**
|
| 48 |
+
|
| 49 |
+
TypeB is then feeded 600K dataset aimed to increase the performance of the AI model in term of these categories
|
| 50 |
+
- ROMANCE
|
| 51 |
+
- MYSTERY / DETECTIVE
|
| 52 |
+
- SCI-FI
|
| 53 |
+
|
| 54 |
+
This, alongside the same 600K datasets Handwritten nature, gave the Model length and consistency which is Useful for RP purposes,
|
| 55 |
+
HOWEVER DUE TO LACK OF PARAMS, EXPECT THE MODEL TO HALLUCINATE
|
| 56 |
+
|
| 57 |
+
**ALL FINETUNING, ACCOUNTING DATASET PREPARATION ASWELL, TOOK 5 DAYS. HERETICAL TRAINING IS EXCLUDED BECAUSE THE HERETICAL HAS BEEN PREPPED LONG TIME AGO**
|
| 58 |
+
|
| 59 |
+
- OpenElla StoryWriter - TypeB is Another Instance of Our Smallest Model Yet! if you find any issue, then please don't hesitate to email us at:
|
| 60 |
+
[nexus.networkinteractives@gmail.com](mailto:nexus.networkinteractives@gmail.com)
|
| 61 |
+
about any overfitting, or improvements for the future Model,
|
| 62 |
+
Once again feel free to Modify the LORA to your likings, However please consider Adding this Page
|
| 63 |
+
for credits and if you'll increase its **Dataset**, then please handle it with care and ethical considerations
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
- OpenElla StoryWriter - TypeB is
|
| 67 |
+
- **Developed by:** N-Bot-Int
|
| 68 |
+
- **License:** apache-2.0
|
| 69 |
+
|
| 70 |
+
- MiniMaid-L3 Official Metric Score
|
| 71 |
+

|
| 72 |
+
"Minimaid Instances Vs. OpenElla StoryWriter - TypeB"
|
| 73 |
+
|
| 74 |
+

|
| 75 |
+
"OpenElla StoryWriter - TypeB vs. Several Different AI Models"
|
| 76 |
|
| 77 |
+
- **METRIC SCORES ARE HIGHLY SUBJECTIVE, Feel free to use the model and judge them yourselves!**
|
| 78 |
|
| 79 |
+
- # Notice
|
| 80 |
+
- **For a Good Experience, Please use**
|
| 81 |
+
- (PLEASE CALIBRATE THE MODEL DEPENDING ON THE CHARACTER CARD)
|
| 82 |
+
- USE **CHATML**, the AI MODEL IS FINETUNED TO USE CHATML more than any other format
|
| 83 |
|
| 84 |
+
- # Detail card:
|
| 85 |
+
- Parameter
|
| 86 |
+
- 1 Billion Parameters
|
| 87 |
+
- (Please check your GPU Core, CPU and RAM to see if you can confortably run 1B models)
|
| 88 |
|
| 89 |
+
- Finetuning tool:
|
| 90 |
+
- Unsloth AI
|
| 91 |
+
- This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
| 92 |
+
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
| 93 |
+
- Fine-tuned Using:
|
| 94 |
+
- Google Colab Free Tier
|