Hugging Face's logo

shpotes
/
codegen-350M

Text Generation
Transformers
PyTorch
codegen
Model card Files
xet
Community
1
shpotes commited on
Commit
8ff8b64
·
1 Parent(s): 56f92e6

rebase

Browse files
Files changed (6) hide show
  1. added_tokens.json +0 -40
  2. merges.txt +0 -0
  3. special_tokens_map.json +0 -6
  4. tokenizer.json +0 -0
  5. tokenizer_config.json +0 -10
  6. vocab.json +0 -0
added_tokens.json DELETED
@@ -1,40 +0,0 @@
1
- {
2
- "\t\t": 50294,
3
- "\t\t\t": 50293,
4
- "\t\t\t\t": 50292,
5
- "\t\t\t\t\t": 50291,
6
- "\t\t\t\t\t\t": 50290,
7
- "\t\t\t\t\t\t\t": 50289,
8
- "\t\t\t\t\t\t\t\t": 50288,
9
- "\t\t\t\t\t\t\t\t\t": 50287,
10
- " ": 50286,
11
- " ": 50285,
12
- " ": 50284,
13
- " ": 50283,
14
- " ": 50282,
15
- " ": 50281,
16
- " ": 50280,
17
- " ": 50279,
18
- " ": 50278,
19
- " ": 50277,
20
- " ": 50276,
21
- " ": 50275,
22
- " ": 50274,
23
- " ": 50273,
24
- " ": 50272,
25
- " ": 50271,
26
- " ": 50270,
27
- " ": 50269,
28
- " ": 50268,
29
- " ": 50267,
30
- " ": 50266,
31
- " ": 50265,
32
- " ": 50264,
33
- " ": 50263,
34
- " ": 50262,
35
- " ": 50261,
36
- " ": 50260,
37
- " ": 50259,
38
- " ": 50258,
39
- " ": 50257
40
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
merges.txt DELETED
The diff for this file is too large to render. See raw diff
 
special_tokens_map.json DELETED
@@ -1,6 +0,0 @@
1
- {
2
- "bos_token": "<|endoftext|>",
3
- "eos_token": "<|endoftext|>",
4
- "pad_token": 50256,
5
- "unk_token": "<|endoftext|>"
6
- }
 
 
 
 
 
 
 
tokenizer.json DELETED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json DELETED
@@ -1,10 +0,0 @@
1
- {
2
- "add_prefix_space": false,
3
- "bos_token": "<|endoftext|>",
4
- "eos_token": "<|endoftext|>",
5
- "model_max_length": 1024,
6
- "name_or_path": "gpt2",
7
- "special_tokens_map_file": null,
8
- "tokenizer_class": "GPT2Tokenizer",
9
- "unk_token": "<|endoftext|>"
10
- }
 
 
 
 
 
 
 
 
 
 
 
vocab.json DELETED
The diff for this file is too large to render. See raw diff