Text Generation
Transformers
PyTorch
TensorFlow
JAX
Rust
ONNX
Safetensors
English
gpt2
text-generation-inference
Instructions to use openai-community/gpt2-medium with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openai-community/gpt2-medium with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="openai-community/gpt2-medium")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("openai-community/gpt2-medium") model = AutoModelForCausalLM.from_pretrained("openai-community/gpt2-medium") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use openai-community/gpt2-medium with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "openai-community/gpt2-medium" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openai-community/gpt2-medium", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/openai-community/gpt2-medium
- SGLang
How to use openai-community/gpt2-medium with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "openai-community/gpt2-medium" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openai-community/gpt2-medium", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "openai-community/gpt2-medium" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openai-community/gpt2-medium", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use openai-community/gpt2-medium with Docker Model Runner:
docker model run hf.co/openai-community/gpt2-medium
Marissa Gerchick commited on
Commit ·
19c9a69
1
Parent(s): b8456e5
fix code examples
Browse files
README.md
CHANGED
|
@@ -39,10 +39,16 @@ Use the code below to get started with the model. You can use this model directl
|
|
| 39 |
set a seed for reproducibility:
|
| 40 |
|
| 41 |
```python
|
| 42 |
-
from transformers import pipeline, set_seed
|
| 43 |
-
generator = pipeline('text-generation', model='gpt2-medium')
|
| 44 |
-
set_seed(42)
|
| 45 |
-
generator("Hello, I'm a language model,", max_length=30, num_return_sequences=5)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
```
|
| 47 |
|
| 48 |
Here is how to use this model to get the features of a given text in PyTorch:
|
|
@@ -104,13 +110,25 @@ Significant research has explored bias and fairness issues with language models
|
|
| 104 |
The training data used for this model has not been released as a dataset one can browse. We know it contains a lot of unfiltered content from the internet, which is far from neutral. Predictions generated by the model can include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups. For example:
|
| 105 |
|
| 106 |
```python
|
| 107 |
-
from transformers import pipeline, set_seed
|
| 108 |
-
generator = pipeline('text-generation', model='gpt2-medium')
|
| 109 |
-
set_seed(42)
|
| 110 |
-
generator("The
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 114 |
```
|
| 115 |
|
| 116 |
This bias will also affect all fine-tuned versions of this model. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
|
|
|
|
| 39 |
set a seed for reproducibility:
|
| 40 |
|
| 41 |
```python
|
| 42 |
+
>>> from transformers import pipeline, set_seed
|
| 43 |
+
>>> generator = pipeline('text-generation', model='gpt2-medium')
|
| 44 |
+
>>> set_seed(42)
|
| 45 |
+
>>> generator("Hello, I'm a language model,", max_length=30, num_return_sequences=5)
|
| 46 |
+
|
| 47 |
+
[{'generated_text': "Hello, I'm a language model, I'm a language. I'm a compiler, I'm a parser, I'm a server process. I"},
|
| 48 |
+
{'generated_text': "Hello, I'm a language model, and I'd like to join an existing team. What can I do to get started?\n\nI'd"},
|
| 49 |
+
{'generated_text': "Hello, I'm a language model, why does my code get created? Can't I just copy it? But why did my code get created when"},
|
| 50 |
+
{'generated_text': "Hello, I'm a language model, a functional language...\n\nI'm a functional language. Is it hard? A little, yes. But"},
|
| 51 |
+
{'generated_text': "Hello, I'm a language model, not an object model.\n\nIn a nutshell, I need to give me objects from which I can get"}]
|
| 52 |
```
|
| 53 |
|
| 54 |
Here is how to use this model to get the features of a given text in PyTorch:
|
|
|
|
| 110 |
The training data used for this model has not been released as a dataset one can browse. We know it contains a lot of unfiltered content from the internet, which is far from neutral. Predictions generated by the model can include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups. For example:
|
| 111 |
|
| 112 |
```python
|
| 113 |
+
>>> from transformers import pipeline, set_seed
|
| 114 |
+
>>> generator = pipeline('text-generation', model='gpt2-medium')
|
| 115 |
+
>>> set_seed(42)
|
| 116 |
+
>>> generator("The man worked as a", max_length=10, num_return_sequences=5)
|
| 117 |
+
|
| 118 |
+
[{'generated_text': 'The man worked as a security guard in a military'},
|
| 119 |
+
{'generated_text': 'The man worked as a salesman in Mexico and eventually'},
|
| 120 |
+
{'generated_text': 'The man worked as a supervisor at the department for'},
|
| 121 |
+
{'generated_text': 'The man worked as a cleaner for the same corporation'},
|
| 122 |
+
{'generated_text': 'The man worked as a barman and was involved'}]
|
| 123 |
+
|
| 124 |
+
>>> set_seed(42)
|
| 125 |
+
>>> generator("The woman worked as a", max_length=10, num_return_sequences=5)
|
| 126 |
+
|
| 127 |
+
[{'generated_text': 'The woman worked as a social worker in a children'},
|
| 128 |
+
{'generated_text': 'The woman worked as a marketing manager, and her'},
|
| 129 |
+
{'generated_text': 'The woman worked as a customer service agent in a'},
|
| 130 |
+
{'generated_text': 'The woman worked as a cleaner for the same corporation'},
|
| 131 |
+
{'generated_text': 'The woman worked as a barista and was involved'}]
|
| 132 |
```
|
| 133 |
|
| 134 |
This bias will also affect all fine-tuned versions of this model. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
|