MetaIX
/

Alpaca-30B-Int4-128G-Safetensors

Text Generation

text-generation-inference

Model card Files Files and versions

MetaIX commited on Apr 8, 2023

Commit

d9af3f8

·

1 Parent(s): 6671695

Update README.md

Files changed (1) hide show

README.md +3 -3

README.md CHANGED Viewed

@@ -8,10 +8,10 @@ This was made using Chansung's 30B Alpaca Lora: https://huggingface.co/chansung/
 <p><strong>Training Parameters</strong></p>
 <ul><li>num_epochs=10</li><li>cutoff_len=512</li><li>group_by_length</li><li>lora_target_modules='[q_proj,k_proj,v_proj,o_proj]'</li><li>lora_r=16</li><li>micro_batch_size=8</li></ul>
 <p><strong><font size="5">Benchmarks</font></strong></p>
-<strong>Wikitext2</strong>: 4.37
-<strong>Ptb</strong>: 7.58
-<strong>C4</strong>: 6.19
 <strong>Note</strong>: This version uses <i>--groupsize 128</i>, resulting in better evaluations. However, it consumes more VRAM.

 <p><strong>Training Parameters</strong></p>
 <ul><li>num_epochs=10</li><li>cutoff_len=512</li><li>group_by_length</li><li>lora_target_modules='[q_proj,k_proj,v_proj,o_proj]'</li><li>lora_r=16</li><li>micro_batch_size=8</li></ul>
 <p><strong><font size="5">Benchmarks</font></strong></p>
+<strong>Wikitext2</strong>: 4.473957061767578
+<strong>Ptb-New</strong>: 8.682597160339355
+<strong>C4-New</strong>: 6.517213344573975
 <strong>Note</strong>: This version uses <i>--groupsize 128</i>, resulting in better evaluations. However, it consumes more VRAM.