IMvision12 commited on
Commit
e04b684
·
0 Parent(s):

Super-squash branch 'main' using huggingface_hub

Browse files
Files changed (5) hide show
  1. .gitattributes +35 -0
  2. README.md +67 -0
  3. kf_config.json +21 -0
  4. model.weights.h5 +3 -0
  5. tokenizer.json +0 -0
.gitattributes ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ license: mit
4
+ base_model: openai-community/openai-gpt
5
+ library_name: kerasformers
6
+ language:
7
+ - en
8
+ tags:
9
+ - keras
10
+ - kerasformers
11
+ - gpt
12
+ - openai-gpt
13
+ - text-generation
14
+ - pytorch
15
+ - jax
16
+ - tf
17
+ ---
18
+
19
+ # Run GPT with Keras 3: JAX, PyTorch, or TensorFlow
20
+
21
+ [![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-black?logo=github)](https://github.com/IMvision12/KerasFormers) [![Docs](https://img.shields.io/badge/Docs-GPT-blue)](https://imvision12.github.io/KerasFormers/gpt/)
22
+
23
+ # kerasformers/gpt
24
+
25
+ Paper: [Improving Language Understanding by Generative Pre-Training (Radford et al., 2018)](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf)
26
+
27
+ GPT (the original GPT-1) is OpenAI's first generative pre-trained transformer: a
28
+ 12-layer decoder-only model with learned position embeddings (512-token context),
29
+ `gelu_new` activations, and a byte-pair-encoding tokenizer, trained on BookCorpus.
30
+ This is the **117M** base completion model (no chat template).
31
+
32
+ For more details, see the upstream [model card](https://huggingface.co/openai-community/openai-gpt).
33
+
34
+ Pure-**Keras 3** conversion of [`openai-community/openai-gpt`](https://huggingface.co/openai-community/openai-gpt) for
35
+ [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on
36
+ **TensorFlow / Torch / JAX**.
37
+
38
+ ## Quick start
39
+
40
+ ```python
41
+ import os
42
+ os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
43
+
44
+ from kerasformers.models.gpt import GptTextGenerate, GptTokenizer
45
+
46
+ model = GptTextGenerate.from_weights("kerasformers/gpt")
47
+ tokenizer = GptTokenizer.from_weights("kerasformers/gpt")
48
+
49
+ inputs = tokenizer("the meaning of life is")
50
+ outputs = model.generate(**inputs, max_new_tokens=40)
51
+ print(tokenizer.decode(outputs[0]))
52
+ ```
53
+
54
+ ## Tips
55
+
56
+ - Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
57
+ - Context length is 512 tokens; this is a base completion model, not
58
+ instruction-tuned.
59
+ - See the [GPT docs](https://imvision12.github.io/KerasFormers/gpt/) and [Loading Weights](https://imvision12.github.io/KerasFormers/loading_weights/).
60
+ - Upstream safetensors still work via the `hf:` prefix, e.g.
61
+ `GptTextGenerate.from_weights("hf:openai-community/openai-gpt")`.
62
+
63
+ ## Special Thanks
64
+
65
+ A huge thank you to the OpenAI GPT authors for creating and releasing this model.
66
+
67
+ License: MIT, inherited from the upstream OpenAI GPT release.
kf_config.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "library_name": "kerasformers",
3
+ "kerasformers_version": "1.2.1",
4
+ "model_module": "kerasformers.models.gpt",
5
+ "model_class": "GptModel",
6
+ "variant": "gpt",
7
+ "weights": "model.weights.h5",
8
+ "schema_version": 2,
9
+ "weight_dtype": "float32",
10
+ "model_type": "openai-gpt",
11
+ "text_config": {
12
+ "vocab_size": 40478,
13
+ "embed_dim": 768,
14
+ "mlp_dim": 3072,
15
+ "num_layers": 12,
16
+ "num_heads": 12,
17
+ "max_position_embeddings": 512,
18
+ "norm_eps": 1e-05,
19
+ "tie_embeddings": true
20
+ }
21
+ }
model.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec7b50f37f0c52df5481d6ab69b55116654f6ac734d706f254335a2c6c7f5d5f
3
+ size 466420584
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff