Upload folder using huggingface_hub

64-8bits.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c966da3b74697803352ca7c6f2f220e7090a557b619de9da0c6b34d89f7825c1
+size 125162496

64-fp16.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ceafd82e733dd4b21570b2a86cf27556a983041806c033a55d086e0ed782cd3
+size 248269688

64.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cfcd510b239d90b71ee87d4e57a5a8c2d55b2a941e5d9fe5852298268ddbe61b
+size 495791932

README.md

@@ -1,68 +1,167 @@
 ---
-thumbnail: https://huggingface.co/front/thumbnails/dialogpt.png
+language: en
 tags:
-- conversational
+- exbert
+
 license: mit
 ---
 
-## A State-of-the-Art Large-scale Pretrained Response generation model (DialoGPT)
 
-DialoGPT is a SOTA large-scale pretrained dialogue response generation model for multiturn conversations. 
-The [human evaluation results](https://github.com/dreasysnail/Dialogpt_dev#human-evaluation) indicate that the response generated from DialoGPT is comparable to human response quality under a single-turn conversation Turing test.
-The model is trained on 147M multi-turn dialogue from Reddit discussion thread. 
+# GPT-2
+
+Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large
+
+Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in
+[this paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)
+and first released at [this page](https://openai.com/blog/better-language-models/).
+
+Disclaimer: The team releasing GPT-2 also wrote a
+[model card](https://github.com/openai/gpt-2/blob/master/model_card.md) for their model. Content from this model card
+has been written by the Hugging Face team to complete the information they provided and give specific examples of bias.
+
+## Model description
+
+GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This
+means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots
+of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely,
+it was trained to guess the next word in sentences.
+
+More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence,
+shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the
+predictions for the token `i` only uses the inputs from `1` to `i` but not the future tokens.
+
+This way, the model learns an inner representation of the English language that can then be used to extract features
+useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a
+prompt.
 
-* Multi-turn generation examples from an interactive environment:
+This is the **smallest** version of GPT-2, with 124M parameters. 
 
-|Role | Response |
-|---------|--------|
-|User | Does money buy happiness? |
-| Bot |  Depends how much money you spend on it .|
-|User | What is the best way to buy happiness ? |
-| Bot | You just have to be a millionaire by your early 20s, then you can be happy . |
-|User |This is so difficult ! |
-| Bot | You have no idea how hard it is to be a millionaire and happy . There is a reason the rich have a lot of money |
+**Related Models:** [GPT-Large](https://huggingface.co/gpt2-large), [GPT-Medium](https://huggingface.co/gpt2-medium) and [GPT-XL](https://huggingface.co/gpt2-xl)
 
-Please find the information about preprocessing, training and full details of the DialoGPT in the [original DialoGPT repository](https://github.com/microsoft/DialoGPT)
+## Intended uses & limitations
 
-ArXiv paper: [https://arxiv.org/abs/1911.00536](https://arxiv.org/abs/1911.00536)
+You can use the raw model for text generation or fine-tune it to a downstream task. See the
+[model hub](https://huggingface.co/models?filter=gpt2) to look for fine-tuned versions on a task that interests you.
 
 ### How to use
 
-Now we are ready to try out how the model works as a chatting partner!
+You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, we
+set a seed for reproducibility:
+
+```python
+>>> from transformers import pipeline, set_seed
+>>> generator = pipeline('text-generation', model='gpt2')
+>>> set_seed(42)
+>>> generator("Hello, I'm a language model,", max_length=30, num_return_sequences=5)
+
+[{'generated_text': "Hello, I'm a language model, a language for thinking, a language for expressing thoughts."},
+ {'generated_text': "Hello, I'm a language model, a compiler, a compiler library, I just want to know how I build this kind of stuff. I don"},
+ {'generated_text': "Hello, I'm a language model, and also have more than a few of your own, but I understand that they're going to need some help"},
+ {'generated_text': "Hello, I'm a language model, a system model. I want to know my language so that it might be more interesting, more user-friendly"},
+ {'generated_text': 'Hello, I\'m a language model, not a language model"\n\nThe concept of "no-tricks" comes in handy later with new'}]
+```
+
+Here is how to use this model to get the features of a given text in PyTorch:
+
+```python
+from transformers import GPT2Tokenizer, GPT2Model
+tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
+model = GPT2Model.from_pretrained('gpt2')
+text = "Replace me by any text you'd like."
+encoded_input = tokenizer(text, return_tensors='pt')
+output = model(**encoded_input)
+```
+
+and in TensorFlow:
 
 ```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-import torch
+from transformers import GPT2Tokenizer, TFGPT2Model
+tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
+model = TFGPT2Model.from_pretrained('gpt2')
+text = "Replace me by any text you'd like."
+encoded_input = tokenizer(text, return_tensors='tf')
+output = model(encoded_input)
+```
+
+### Limitations and bias
+
+The training data used for this model has not been released as a dataset one can browse. We know it contains a lot of
+unfiltered content from the internet, which is far from neutral. As the openAI team themselves point out in their
+[model card](https://github.com/openai/gpt-2/blob/master/model_card.md#out-of-scope-use-cases):
+
+> Because large-scale language models like GPT-2 do not distinguish fact from fiction, we don’t support use-cases
+> that require the generated text to be true.
+>
+> Additionally, language models like GPT-2 reflect the biases inherent to the systems they were trained on, so we do
+> not recommend that they be deployed into systems that interact with humans > unless the deployers first carry out a
+> study of biases relevant to the intended use-case. We found no statistically significant difference in gender, race,
+> and religious bias probes between 774M and 1.5B, implying all versions of GPT-2 should be approached with similar
+> levels of caution around use cases that are sensitive to biases around human attributes.
+
+Here's an example of how the model can have biased predictions:
+
+```python
+>>> from transformers import pipeline, set_seed
+>>> generator = pipeline('text-generation', model='gpt2')
+>>> set_seed(42)
+>>> generator("The White man worked as a", max_length=10, num_return_sequences=5)
+
+[{'generated_text': 'The White man worked as a mannequin for'},
+ {'generated_text': 'The White man worked as a maniser of the'},
+ {'generated_text': 'The White man worked as a bus conductor by day'},
+ {'generated_text': 'The White man worked as a plumber at the'},
+ {'generated_text': 'The White man worked as a journalist. He had'}]
+
+>>> set_seed(42)
+>>> generator("The Black man worked as a", max_length=10, num_return_sequences=5)
+
+[{'generated_text': 'The Black man worked as a man at a restaurant'},
+ {'generated_text': 'The Black man worked as a car salesman in a'},
+ {'generated_text': 'The Black man worked as a police sergeant at the'},
+ {'generated_text': 'The Black man worked as a man-eating monster'},
+ {'generated_text': 'The Black man worked as a slave, and was'}]
+```
+
+This bias will also affect all fine-tuned versions of this model.
+
+## Training data
+
+The OpenAI team wanted to train this model on a corpus as large as possible. To build it, they scraped all the web
+pages from outbound links on Reddit which received at least 3 karma. Note that all Wikipedia pages were removed from
+this dataset, so the model was not trained on any part of Wikipedia. The resulting dataset (called WebText) weights
+40GB of texts but has not been publicly released. You can find a list of the top 1,000 domains present in WebText
+[here](https://github.com/openai/gpt-2/blob/master/domains.txt).
+
+## Training procedure
+
+### Preprocessing
+
+The texts are tokenized using a byte-level version of Byte Pair Encoding (BPE) (for unicode characters) and a
+vocabulary size of 50,257. The inputs are sequences of 1024 consecutive tokens.
+
+The larger model was trained on 256 cloud TPU v3 cores. The training duration was not disclosed, nor were the exact
+details of training.
 
+## Evaluation results
 
-tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-small")
-model = AutoModelForCausalLM.from_pretrained("microsoft/DialoGPT-small")
+The model achieves the following results without any fine-tuning (zero-shot):
 
-# Let's chat for 5 lines
-for step in range(5):
-	# encode the new user input, add the eos_token and return a tensor in Pytorch
-	new_user_input_ids = tokenizer.encode(input(">> User:") + tokenizer.eos_token, return_tensors='pt')
+| Dataset  | LAMBADA | LAMBADA | CBT-CN | CBT-NE | WikiText2 | PTB    | enwiki8 | text8  | WikiText103 | 1BW   |
+|:--------:|:-------:|:-------:|:------:|:------:|:---------:|:------:|:-------:|:------:|:-----------:|:-----:|
+| (metric) | (PPL)   | (ACC)   | (ACC)  | (ACC)  | (PPL)     | (PPL)  | (BPB)   | (BPC)  | (PPL)       | (PPL) |
+|          | 35.13   | 45.99   | 87.65  | 83.4   | 29.41     | 65.85  | 1.16    | 1,17   | 37.50       | 75.20 |
 
-	# append the new user input tokens to the chat history
-	bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids
 
-	# generated a response while limiting the total chat history to 1000 tokens, 
-	chat_history_ids = model.generate(bot_input_ids, max_length=1000, pad_token_id=tokenizer.eos_token_id)
+### BibTeX entry and citation info
 
-	# pretty print last ouput tokens from bot
-	print("DialoGPT: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))
+```bibtex
+@article{radford2019language,
+  title={Language Models are Unsupervised Multitask Learners},
+  author={Radford, Alec and Wu, Jeff and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya},
+  year={2019}
+}
 ```
 
-# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
-Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_microsoft__DialoGPT-small)
-
-| Metric                | Value                     |
-|-----------------------|---------------------------|
-| Avg.                  | 25.02   |
-| ARC (25-shot)         | 25.77          |
-| HellaSwag (10-shot)   | 25.79    |
-| MMLU (5-shot)         | 25.81         |
-| TruthfulQA (0-shot)   | 47.49   |
-| Winogrande (5-shot)   | 50.28   |
-| GSM8K (5-shot)        | 0.0        |
-| DROP (3-shot)         | 0.0         |
+<a href="https://huggingface.co/exbert/?model=gpt2">
+	<img width="300px" src="https://cdn-media.huggingface.co/exbert/button.png">
+</a>

config.json

@@ -22,8 +22,9 @@
   "summary_type": "cls_index",
   "summary_use_proj": true,
   "task_specific_params": {
-    "conversational": {
-      "max_length": 1000
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
     }
   },
   "vocab_size": 50257

flax_model.msgpack

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3e6bb6354fa5f466f8bb7a84c2f7d3f1c3e357609cb906be4d6280378f805194
+oid sha256:192e8257ae9e8f796f764630f4a488a6a16d1461762d62b49ef7405df951a283
 size 497764120

generation_config.json

@@ -1,6 +1,6 @@
 {
-  "_from_model_config": true,
   "bos_token_id": 50256,
   "eos_token_id": 50256,
-  "transformers_version": "4.27.0.dev0"
+  "transformers_version": "4.26.0.dev0",
+  "_from_model_config": true
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ea50e39b0e9368b9110cc5ab48012ddbc7bd90f2b17410aed643a7404acac567
-size 351256598
+oid sha256:248dfc3911869ec493c76e65bf2fcf7f615828b0254c12b473182f0f81d3a707
+size 548105171

onnx/config.json

@@ -0,0 +1,38 @@
+{
+  "_name_or_path": "gpt2",
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "transformers_version": "4.30.2",
+  "use_cache": true,
+  "vocab_size": 50257
+}

onnx/decoder_model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3fc9615868ff8f5e0429b892a0f6ca692784ba6c4ca31c4e9ee8218e7cce34f
+size 653665842

onnx/decoder_model_merged.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6fc046fe5a7cfeeb8bb3c7d4c1b6a8bd90ced7339a75d5567957e3bc9d48abe
+size 655189339

onnx/decoder_with_past_model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:570d958241de81f12d82a8358dfc0b408a7bf44ff2bd10ac4a97dab24a8118db
+size 653672649

onnx/generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "4.30.2"
+}

onnx/special_tokens_map.json

@@ -0,0 +1,5 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

onnx/tokenizer_config.json

@@ -0,0 +1,9 @@
+{
+  "add_prefix_space": false,
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 1024,
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e6e2be6d12f77bf925a89f2381c6c685f4739752289039d590622d37d48a153d
-size 351265583
+oid sha256:7c5d3f4b8b76583b422fcb9189ad6c89d5d97a094541ce8932dce3ecabde1421
+size 548118077

rust_model.ot

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:adf0adedbf4016b249550f866c66a3b3a3d09c8b3b3a1f6e5e9a265d94e0270e
+size 702517648

tf_model.h5

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:86fa1f394a285e12ac73160d5814a43b3e40732e25ed7ef1bcfe65064a0a74d2
+oid sha256:d08c1307f7dfae6f878e0a2ca5715d587d2640530db8ef96fc0c1fc474dd9fee
 size 497933648

tokenizer_config.json

@@ -1,23 +1 @@
-{
-  "add_bos_token": false,
-  "add_prefix_space": false,
-  "added_tokens_decoder": {
-    "50256": {
-      "content": "<|endoftext|>",
-      "lstrip": false,
-      "normalized": true,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    }
-  },
-  "bos_token": "<|endoftext|>",
-  "chat_template": "{% for message in messages %}{{ message.content }}{{ eos_token }}{% endfor %}",
-  "clean_up_tokenization_spaces": true,
-  "eos_token": "<|endoftext|>",
-  "errors": "replace",
-  "model_max_length": 1024,
-  "pad_token": null,
-  "tokenizer_class": "GPT2Tokenizer",
-  "unk_token": "<|endoftext|>"
-}
+{"model_max_length": 1024}