appvoid commited on
Commit
73f54d5
·
1 Parent(s): a04516c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -3
README.md CHANGED
@@ -3,11 +3,18 @@
3
  ### a better base model
4
  palmer is a series of ~1b parameters language models fine-tuned to be used as base models instead of using custom prompts for tasks. This means that it can be further fine-tuned on more data with custom prompts as usual or be used for downstream tasks as any base model you can get. The model has the best of both worlds: some "bias" to act as an assistant, but also the abillity to predict the next-word from its internet knowledge base. It's a llama 2 model so you can use it with your favorite tools/frameworks.
5
 
6
- ### Training
7
- Training took ~3.5 P100 gpu hours. It was trained on 15,000 openhermes dataset shuffled samples. palmer was fine-tuned using lower learning rates ensuring it keeps as much general knowledge as possible.
 
 
 
 
 
 
 
8
 
9
  Note: highly experimenal yet! Your feedback will make it better.
10
- ### Prompt
11
  ```
12
  On this article, we are going to learn
13
  palmer-001: about the different types of data that are used in the field of data science...
 
3
  ### a better base model
4
  palmer is a series of ~1b parameters language models fine-tuned to be used as base models instead of using custom prompts for tasks. This means that it can be further fine-tuned on more data with custom prompts as usual or be used for downstream tasks as any base model you can get. The model has the best of both worlds: some "bias" to act as an assistant, but also the abillity to predict the next-word from its internet knowledge base. It's a llama 2 model so you can use it with your favorite tools/frameworks.
5
 
6
+ ### evaluation
7
+ |Model| ARC_C| HellaSwag| PIQA| Winogrande|
8
+ |------|-----|-----------|------|-------------|
9
+ |tinyllama-2t| 0.2807| 0.5463| 0.7067| 0.5683|
10
+ |palmer-001| 0.2807| 0.5524| 0.7106| 0.5896|
11
+
12
+
13
+ ### training
14
+ Training took ~3.5 P100 gpu hours. It was trained on 15,000 gpt-4 shuffled samples. palmer was fine-tuned using lower learning rates ensuring it keeps as much general knowledge as possible.
15
 
16
  Note: highly experimenal yet! Your feedback will make it better.
17
+ ### prompt
18
  ```
19
  On this article, we are going to learn
20
  palmer-001: about the different types of data that are used in the field of data science...