ChanLumerico commited on
Commit
83a9733
·
verified ·
1 Parent(s): 125b7fd

Update model card

Browse files
Files changed (1) hide show
  1. README.md +60 -0
README.md ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: lucid
3
+ license: mit
4
+ tags:
5
+ - base
6
+ - gpt
7
+ - lucid
8
+ datasets:
9
+ - bookcorpus
10
+ pipeline_tag: feature-extraction
11
+ ---
12
+
13
+ # GPT
14
+
15
+ > https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf
16
+
17
+ [Lucid](https://github.com/ChanLumerico/lucid) port of `transformers/openai-community/openai-gpt`,
18
+ converted to Lucid-native safetensors.
19
+
20
+ ## Available weights
21
+
22
+ | Tag | Params | GFLOPs | Size | Source |
23
+ |---|---|---|---|---|
24
+ | `BOOKCORPUS` *(default)* | 116.5M | — | 444.56 MB | transformers |
25
+
26
+ ## Usage
27
+
28
+ ```python
29
+ import lucid
30
+ import lucid.models as models
31
+ from lucid.models.weights import GptWeights
32
+
33
+ # default tag
34
+ model = models.gpt(pretrained=True)
35
+
36
+ # explicit tag (enum or string)
37
+ model = models.gpt(weights=GptWeights.BOOKCORPUS)
38
+ model = models.gpt(pretrained="BOOKCORPUS")
39
+
40
+ # feed token ids (tokenize with the matching lucid.utils.tokenizer)
41
+ input_ids = lucid.tensor([[101, 7592, 2088, 102]], dtype=lucid.int64)
42
+ out = model(input_ids)
43
+ hidden = out.last_hidden_state # (B, T, hidden_size)
44
+ ```
45
+
46
+ ## Conversion
47
+
48
+ Converted from `transformers/openai-community/openai-gpt` via
49
+ `python -m tools.convert_weights gpt --tag BOOKCORPUS`.
50
+ Key mapping + numerical parity verified against the source.
51
+
52
+ ## License
53
+
54
+ `mit` — inherited from the original weights.
55
+
56
+ ## Citation
57
+
58
+ ```
59
+ Radford et al., "Improving Language Understanding by Generative Pre-Training", 2018 (GPT).
60
+ ```