| --- |
| language: |
| - ja |
| tags: |
| - text-generation |
|
|
| --- |
| # Model Card for Talkmodel01 |
| |
| |
| # Model Details |
| |
| ## Model Description |
| |
| - **Developed by:** Yuki takada |
| - **Shared by [Optional]:** More information needed |
| - **Model type:** Text Generation |
| - **Language(s) (NLP):** japanese |
| - **License:** More information needed |
| - **Related Models:** |
| - **Parent Model:** GPT-2 |
| - **Resources for more information:** More information needed |
| |
| # Uses |
| |
| |
| ## Direct Use |
| |
| This model can be used for the task of Text Generation |
| |
| ## Downstream Use [Optional] |
| |
| More information needed |
| |
| ## Out-of-Scope Use |
| |
| The model should not be used to intentionally create hostile or alienating environments for people. |
| The model should not be used to intentionally create hostile or alienating environments for people. |
| OpenAI note in their [model card](https://github.com/openai/gpt-2/blob/master/model_card.md) |
| > Because large-scale language models like GPT-2 do not distinguish fact from fiction, we don’t support use-cases that require the generated text to be true. |
| |
| # Bias, Risks, and Limitations |
| |
| The training data used for this model has not been released as a dataset one can browse. We know it contains a lot of |
| unfiltered content from the internet, which is far from neutral. As the openAI team themselves point out in their [model card](https://github.com/openai/gpt-2/blob/master/model_card.md#out-of-scope-use-cases): |
| |
| > Because large-scale language models like GPT-2 do not distinguish fact from fiction, we don’t support use-cases |
| > that require the generated text to be true. |
| > |
| > Additionally, language models like GPT-2 reflect the biases inherent to the systems they were trained on, so we do |
| > not recommend that they be deployed into systems that interact with humans > unless the deployers first carry out a |
| > study of biases relevant to the intended use-case. We found no statistically significant difference in gender, race, |
| > and religious bias probes between 774M and 1.5B, implying all versions of GPT-2 should be approached with similar |
| > levels of caution around use cases that are sensitive to biases around human attributes. |
| |
| See the [GPT-2 model card](https://huggingface.co/gpt2?text=My+name+is+Merve+and+my+favorite) for examples of how the model can have biased predictions. |
| |
| |
| ## Recommendations |
| |
| Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. |
| |
| |
| # Training Details |
| |
| ## Training Data |
| |
| The OpenAI team wanted to train this model on a corpus as large as possible. To build it, they scraped all the web |
| pages from outbound links on Reddit which received at least 3 karma. Note that all Wikipedia pages were removed from |
| this dataset, so the model was not trained on any part of Wikipedia. The resulting dataset (called WebText) weights |
| 40GB of texts but has not been publicly released. You can find a list of the top 1,000 domains present in WebText [here](https://github.com/openai/gpt-2/blob/master/domains.txt). |
| |
| ## Training Procedure |
| |
| |
| ### Preprocessing |
| |
| More information needed |
| |
| ### Speeds, Sizes, Times |
| |
| More information needed |
| |
| # Evaluation |
| |
| |
| ## Testing Data, Factors & Metrics |
| |
| ### Testing Data |
| |
| More information needed |
| |
| ### Factors |
| |
| |
| ### Metrics |
| |
| More information needed |
| ## Results |
| |
| More information needed |
| |
| # Model Examination |
| |
| More information needed |
| |
| # Environmental Impact |
| |
| |
| Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). |
| |
| - **Hardware Type:** More information needed |
| - **Hours used:** More information needed |
| - **Cloud Provider:** More information needed |
| - **Compute Region:** More information needed |
| - **Carbon Emitted:** More information needed |
| |
| # Technical Specifications [optional] |
| |
| ## Model Architecture and Objective |
| |
| More information needed |
| |
| ## Compute Infrastructure |
| |
| More information needed |
| |
| ### Hardware |
| |
| More information needed |
| |
| ### Software |
| More information needed |
| |
| # Citation |
| |
| |
| **BibTeX:** |
| ``` |
| @article{radford2019language, |
| title={Language Models are Unsupervised Multitask Learners}, |
| author={Radford, Alec and Wu, Jeff and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya}, |
| year={2019} |
| } |
| ``` |
| |
| |
| |
| |
| # Glossary [optional] |
| More information needed |
| |
| # More Information [optional] |
| |
| More information needed |
| |
| # Model Card Authors [optional] |
| |
| |
| Yuki takada in collaboration with Ezi Ozoani and the Hugging Face team |
| |
| # Model Card Contact |
| |
| More information needed |
| |
| # How to Get Started with the Model |
| |
| Use the code below to get started with the model. |
| |
| <details> |
| <summary> Click to expand </summary> |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| |
| tokenizer = AutoTokenizer.from_pretrained("RuRI/Talkmodel01") |
| |
| model = AutoModelForCausalLM.from_pretrained("RuRI/Talkmodel01") |
| |
| ``` |
| </details> |
|
|
|
|
