Hugging Face's logo

ninagroot
/
Llama-360M-RUN1

Text Generation
Transformers
TensorBoard
Safetensors
llama
Generated from Trainer
text-generation-inference
Model card Files
xet
Metrics Community
ninagroot commited on
Commit
eb58b1f
·
verified ·
1 Parent(s): 2f3adda

ninagroot/Llama-360Mtest

Browse files
Files changed (8) hide show
  1. README.md +23 -22
  2. merges.txt +0 -0
  3. model.safetensors +1 -1
  4. runs/Mar20_14-47-36_gcn28.local.snellius.surf.nl/events.out.tfevents.1710942465.gcn28.local.snellius.surf.nl.3906721.0 +3 -0
  5. special_tokens_map.json +7 -1
  6. tokenizer_config.json +4 -26
  7. training_args.bin +1 -1
  8. vocab.json +0 -0
README.md CHANGED
@@ -13,7 +13,7 @@ should probably proofread and complete it, then remove this comment. -->
13
 
14
  This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
15
  It achieves the following results on the evaluation set:
16
- - Loss: 7.3725
17
 
18
  ## Model description
19
 
@@ -33,11 +33,11 @@ More information needed
33
 
34
  The following hyperparameters were used during training:
35
  - learning_rate: 0.0003
36
- - train_batch_size: 16
37
  - eval_batch_size: 8
38
  - seed: 42
39
  - gradient_accumulation_steps: 8
40
- - total_train_batch_size: 128
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: cosine
43
  - lr_scheduler_warmup_steps: 300
@@ -48,25 +48,26 @@ The following hyperparameters were used during training:
48
 
49
  | Training Loss | Epoch | Step | Validation Loss |
50
  |:-------------:|:-----:|:----:|:---------------:|
51
- | No log | 0.94 | 2 | 11.0486 |
52
- | No log | 1.88 | 4 | 10.9834 |
53
- | No log | 2.82 | 6 | 10.8679 |
54
- | No log | 3.76 | 8 | 10.7046 |
55
- | No log | 4.71 | 10 | 10.4956 |
56
- | No log | 5.65 | 12 | 10.2361 |
57
- | No log | 6.59 | 14 | 9.9249 |
58
- | No log | 8.0 | 17 | 9.4434 |
59
- | No log | 8.94 | 19 | 9.1960 |
60
- | 10.3432 | 9.88 | 21 | 8.9986 |
61
- | 10.3432 | 10.82 | 23 | 8.8144 |
62
- | 10.3432 | 11.76 | 25 | 8.6667 |
63
- | 10.3432 | 12.71 | 27 | 8.5072 |
64
- | 10.3432 | 13.65 | 29 | 8.3409 |
65
- | 10.3432 | 14.59 | 31 | 8.1751 |
66
- | 10.3432 | 16.0 | 34 | 7.8970 |
67
- | 10.3432 | 16.94 | 36 | 7.6996 |
68
- | 10.3432 | 17.88 | 38 | 7.5270 |
69
- | 8.1799 | 18.82 | 40 | 7.3725 |
 
70
 
71
 
72
  ### Framework versions
 
13
 
14
  This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
15
  It achieves the following results on the evaluation set:
16
+ - Loss: 4.9825
17
 
18
  ## Model description
19
 
 
33
 
34
  The following hyperparameters were used during training:
35
  - learning_rate: 0.0003
36
+ - train_batch_size: 1
37
  - eval_batch_size: 8
38
  - seed: 42
39
  - gradient_accumulation_steps: 8
40
+ - total_train_batch_size: 8
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: cosine
43
  - lr_scheduler_warmup_steps: 300
 
48
 
49
  | Training Loss | Epoch | Step | Validation Loss |
50
  |:-------------:|:-----:|:----:|:---------------:|
51
+ | 10.7103 | 0.98 | 32 | 8.8380 |
52
+ | 7.6149 | 1.99 | 65 | 6.7963 |
53
+ | 6.4154 | 2.97 | 97 | 5.7262 |
54
+ | 5.0267 | 3.98 | 130 | 5.1313 |
55
+ | 4.387 | 5.0 | 163 | 4.9127 |
56
+ | 4.1622 | 5.98 | 195 | 4.7890 |
57
+ | 3.7426 | 6.99 | 228 | 4.7529 |
58
+ | 3.483 | 8.0 | 261 | 4.7366 |
59
+ | 3.2482 | 8.98 | 293 | 4.7420 |
60
+ | 2.8495 | 9.99 | 326 | 4.7989 |
61
+ | 2.7111 | 10.97 | 358 | 4.7948 |
62
+ | 2.129 | 11.98 | 391 | 4.8183 |
63
+ | 1.6497 | 13.0 | 424 | 4.8724 |
64
+ | 1.3473 | 13.98 | 456 | 4.9136 |
65
+ | 0.9066 | 14.99 | 489 | 4.9447 |
66
+ | 0.6857 | 16.0 | 522 | 4.9582 |
67
+ | 0.5487 | 16.98 | 554 | 4.9712 |
68
+ | 0.3983 | 17.99 | 587 | 4.9771 |
69
+ | 0.3503 | 18.97 | 619 | 4.9824 |
70
+ | 0.3477 | 19.62 | 640 | 4.9825 |
71
 
72
 
73
  ### Framework versions
merges.txt CHANGED
The diff for this file is too large to render. See raw diff
 
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:81d0b4ce55886a4159484e3695f3dcd58f19abc0eecdf0bb8527cb2e06d5da7d
3
  size 1720553864
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef3156a8d151356cdcfd3ab09b9a293a70e7f912816fe61313a1f217edb048e1
3
  size 1720553864
runs/Mar20_14-47-36_gcn28.local.snellius.surf.nl/events.out.tfevents.1710942465.gcn28.local.snellius.surf.nl.3906721.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7635c9b019a3ec44fc2c9e7a4cffdfe38e3363088b7c8c8ce94f8e2a36f53db7
3
+ size 15089
special_tokens_map.json CHANGED
@@ -2,5 +2,11 @@
2
  "bos_token": "<s>",
3
  "eos_token": "</s>",
4
  "pad_token": "<pad>",
5
- "unk_token": "<|endoftext|>"
 
 
 
 
 
 
6
  }
 
2
  "bos_token": "<s>",
3
  "eos_token": "</s>",
4
  "pad_token": "<pad>",
5
+ "unk_token": {
6
+ "content": "<|endoftext|>",
7
+ "lstrip": false,
8
+ "normalized": true,
9
+ "rstrip": false,
10
+ "single_word": false
11
+ }
12
  }
tokenizer_config.json CHANGED
@@ -1,34 +1,11 @@
1
  {
 
2
  "add_prefix_space": false,
3
  "added_tokens_decoder": {
4
- "0": {
5
- "content": "<pad>",
6
- "lstrip": false,
7
- "normalized": false,
8
- "rstrip": false,
9
- "single_word": false,
10
- "special": true
11
- },
12
- "1": {
13
- "content": "<s>",
14
- "lstrip": false,
15
- "normalized": false,
16
- "rstrip": false,
17
- "single_word": false,
18
- "special": true
19
- },
20
- "2": {
21
- "content": "</s>",
22
- "lstrip": false,
23
- "normalized": false,
24
- "rstrip": false,
25
- "single_word": false,
26
- "special": true
27
- },
28
- "4312": {
29
  "content": "<|endoftext|>",
30
  "lstrip": false,
31
- "normalized": false,
32
  "rstrip": false,
33
  "single_word": false,
34
  "special": true
@@ -37,6 +14,7 @@
37
  "bos_token": "<s>",
38
  "clean_up_tokenization_spaces": true,
39
  "eos_token": "</s>",
 
40
  "model_max_length": 128,
41
  "pad_token": "<pad>",
42
  "tokenizer_class": "GPT2Tokenizer",
 
1
  {
2
+ "add_bos_token": false,
3
  "add_prefix_space": false,
4
  "added_tokens_decoder": {
5
+ "50256": {
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  "content": "<|endoftext|>",
7
  "lstrip": false,
8
+ "normalized": true,
9
  "rstrip": false,
10
  "single_word": false,
11
  "special": true
 
14
  "bos_token": "<s>",
15
  "clean_up_tokenization_spaces": true,
16
  "eos_token": "</s>",
17
+ "errors": "replace",
18
  "model_max_length": 128,
19
  "pad_token": "<pad>",
20
  "tokenizer_class": "GPT2Tokenizer",
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8602f336bb4ad74eb3e84c33958f37677008e74be29d1b73d0f234013e985f87
3
  size 4728
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ea73475b4a87801ca9770b73f428a8fd2309528c132f1b917b55adfe8c25dfe
3
  size 4728
vocab.json CHANGED
The diff for this file is too large to render. See raw diff