ItsMeDevRoland commited on
Commit
ec8ad7a
·
verified ·
1 Parent(s): 9350fc3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +49 -45
README.md CHANGED
@@ -1,7 +1,8 @@
1
  ---
2
  library_name: peft
3
  license: apache-2.0
4
- base_model: unsloth/zephyr-sft-bnb-4bit
 
5
  tags:
6
  - trl
7
  - sft
@@ -10,48 +11,51 @@ tags:
10
  model-index:
11
  - name: ZoraBetaA1
12
  results: []
 
 
13
  ---
14
-
15
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
- should probably proofread and complete it, then remove this comment. -->
17
-
18
- # ZoraBetaA1
19
-
20
- This model is a fine-tuned version of [unsloth/zephyr-sft-bnb-4bit](https://huggingface.co/unsloth/zephyr-sft-bnb-4bit) on an unknown dataset.
21
-
22
- ## Model description
23
-
24
- More information needed
25
-
26
- ## Intended uses & limitations
27
-
28
- More information needed
29
-
30
- ## Training and evaluation data
31
-
32
- More information needed
33
-
34
- ## Training procedure
35
-
36
- ### Training hyperparameters
37
-
38
- The following hyperparameters were used during training:
39
- - learning_rate: 0.0003
40
- - train_batch_size: 2
41
- - eval_batch_size: 8
42
- - seed: 3407
43
- - gradient_accumulation_steps: 4
44
- - total_train_batch_size: 8
45
- - optimizer: Use OptimizerNames.ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
46
- - lr_scheduler_type: cosine
47
- - lr_scheduler_warmup_steps: 5
48
- - num_epochs: 0.5
49
- - mixed_precision_training: Native AMP
50
-
51
- ### Framework versions
52
-
53
- - PEFT 0.15.2
54
- - Transformers 4.52.4
55
- - Pytorch 2.6.0+cu124
56
- - Datasets 3.6.0
57
- - Tokenizers 0.21.1
 
 
1
  ---
2
  library_name: peft
3
  license: apache-2.0
4
+ base_model:
5
+ - HuggingFaceH4/zephyr-7b-beta
6
  tags:
7
  - trl
8
  - sft
 
11
  model-index:
12
  - name: ZoraBetaA1
13
  results: []
14
+ language:
15
+ - en
16
  ---
17
+ Support us On **KO-FI**
18
+ [![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/J3J61D8NHV)
19
+
20
+
21
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6633a73004501e16e7896b86/viDvgZVv7StCyEBBfR--G.png)
22
+ **ZoraBetaA family**
23
+ # ZoraBetaA1 - SuperLogical
24
+ - ZoraBetaA1 is Our Brand new AI Model, finetuned using [Iris-Uncensored-Reformat-R2](https://huggingface.co/datasets/N-Bot-Int/Iris-Uncensored-Reformat-R2?not-for-all-audiences=true),
25
+ ZoraBetaA1 showcase a Strong reasoning Capability With a Stronger Finetuned Bias toward Roleplaying Using **Zephyr Beta 7B**,
26
+ ZoraBetaA1 also Shows a Great Companionship Capabilities, Without Hallucinating Much Unlike MistThena7B Finetuned Using Mistral 7b v0.1,
27
+ This New Architecture allow us To Increase Roleplaying capabilities without Doing everything from scratch as **Zephyr Beta** has a Strong RP foundation already,
28
+ Leading us to Scaffolding on this Architecture And Increasing Roleplaying capabilities further.
29
+
30
+ - ZoraBetaA1 contains Cleaned Dataset, however its still relatively Unstable so please Report any issues found through our email
31
+ [nexus.networkinteractives@gmail.com](nexus.networkinteractives@gmail.com)
32
+ about any overfitting, or improvements for the future Models
33
+ Once again feel free to Modify the LORA to your likings, However please consider Adding this Page
34
+ for credits and if you'll increase its **Dataset**, then please handle it with care and ethical considerations
35
+
36
+ - ZoraBetaA1 is
37
+ - **Developed by:** N-Bot-Int
38
+ - **License:** apache-2.0
39
+ - **Parent Model from model:** HuggingFaceH4/zephyr-7b-beta
40
+ - **Dataset Combined Using:** UltraDatasetCleanerAndMoshpit-R1(Propietary Software)
41
+
42
+ - # Notice
43
+ - **For a Good Experience, Please use**
44
+ - Low temperature 1.5, min_p = 0.1 and max_new_tokens = 128
45
+
46
+
47
+ - # Detail card:
48
+ - Parameter
49
+ - 3 Billion Parameters
50
+ - (Please visit your GPU Vendor if you can Run 3B models)
51
+
52
+ - Training
53
+ - 300 Steps from
54
+ Iris-Dataset-Reformat-R1
55
+
56
+ - Finetuning tool:
57
+ - Unsloth AI
58
+ - This Zephyr model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
59
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
60
+ - Fine-tuned Using:
61
+ - Google Colab