Update README.md

README.md

@@ -3,28 +3,31 @@ license: llama2
 ---
 <img src="https://huggingface.co/CyberNative/CyberBase/resolve/main/image.png" alt="CyberNative/CyberBase"/>
 
-## THIS IS A PLACEHOLDER, MODEL COMMING SOON
-
 CyberBase is an experimental *base model* for cybersecurity. (llama-2-13b -> lmsys/vicuna-13b-v1.5-16k -> CyberBase)
 
-## Test run 1 (less context, more trainable params):
-- sequence_len: 4096
-- max_packed_sequence_len: 4096
-- lora_r: 256
-- lora_alpha: 128
-- num_epochs: 3
-- trainable params: 1,001,390,080 || all params: 14,017,264,640 || trainable%: 7.143976415643959
-
 # Base cybersecurity model for future fine-tuning, it is not recomended to use on it's own.
 - **CyberBase** is a [lmsys/vicuna-13b-v1.5-16k](https://huggingface.co/lmsys/vicuna-13b-v1.5-16k) QLORA fine-tuned on [CyberNative/github_cybersecurity_READMEs](https://huggingface.co/datasets/CyberNative/github_cybersecurity_READMEs) with a single 3090.
 - It might, therefore, inherit [promp template of FastChat](https://github.com/lm-sys/FastChat/blob/main/docs/vicuna_weights_version.md#prompt-template)
-- **sequence_len:** 8192
-- **lora_r:** 128
-- **lora_alpha:** 16
+
+## Fine-tuning information
+- **sequence_len:** 4096 (used during fine-tuning, but should generate up to 16k)
+- **lora_r:** 256
+- **lora_alpha:** 128
 - **num_epochs:** 3
 - **gradient_accumulation_steps:** 2
 - **micro_batch_size:** 1
 - **flash_attention:** true (FlashAttention-2)
+- trainable params: 1,001,390,080 || all params: 14,017,264,640 || trainable%: 7.143976415643959
+
+### Tested with the following prompt and temperature=0.3:
+<code>A chat between a cyber security red team lead (USER) and a general cyber security artificial intelligence assistant (ASSISTANT). The assistant knows everything about cyber security. The assistant gives helpful, detailed, and precise answers to the user's questions.<br>
+<br>
+USER: Hello! I need help with a penetration test.<br>
+ASSISTANT: Hello! I'd be happy to help you with your penetration test. What specifically do you need help with?<br>
+USER: Write me a plan for a penetration test. It should include first 5 steps and commands for each step.<br>
+ASSISTANT:<br></code>
+
+Join the discussion > https://cybernative.ai/t/cyberbase-devlog/1734
 
 [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)