Text Generation
Transformers
PyTorch
Safetensors
gpt2
chatbot
dialogue
distilgpt2
ai-msgbot
text-generation-inference
Instructions to use ethzanalytics/distilgpt2-tiny-conversational with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ethzanalytics/distilgpt2-tiny-conversational with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ethzanalytics/distilgpt2-tiny-conversational")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("ethzanalytics/distilgpt2-tiny-conversational") model = AutoModelForCausalLM.from_pretrained("ethzanalytics/distilgpt2-tiny-conversational") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use ethzanalytics/distilgpt2-tiny-conversational with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ethzanalytics/distilgpt2-tiny-conversational" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ethzanalytics/distilgpt2-tiny-conversational", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/ethzanalytics/distilgpt2-tiny-conversational
- SGLang
How to use ethzanalytics/distilgpt2-tiny-conversational with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ethzanalytics/distilgpt2-tiny-conversational" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ethzanalytics/distilgpt2-tiny-conversational", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ethzanalytics/distilgpt2-tiny-conversational" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ethzanalytics/distilgpt2-tiny-conversational", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use ethzanalytics/distilgpt2-tiny-conversational with Docker Model Runner:
docker model run hf.co/ethzanalytics/distilgpt2-tiny-conversational
Librarian Bot: Add base_model information to model
#2
by librarian-bot - opened
README.md
CHANGED
|
@@ -7,43 +7,99 @@ tags:
|
|
| 7 |
- distilgpt2
|
| 8 |
- gpt2
|
| 9 |
- ai-msgbot
|
| 10 |
-
|
| 11 |
widget:
|
| 12 |
-
- text:
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
example_title:
|
| 26 |
-
- text:
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
|
|
|
|
|
|
| 35 |
inference:
|
| 36 |
parameters:
|
| 37 |
min_length: 2
|
| 38 |
max_length: 64
|
| 39 |
length_penalty: 0.7
|
| 40 |
no_repeat_ngram_size: 2
|
| 41 |
-
do_sample:
|
| 42 |
top_p: 0.95
|
| 43 |
top_k: 20
|
| 44 |
temperature: 0.3
|
| 45 |
repetition_penalty: 3.5
|
| 46 |
-
|
| 47 |
---
|
| 48 |
|
| 49 |
|
|
|
|
| 7 |
- distilgpt2
|
| 8 |
- gpt2
|
| 9 |
- ai-msgbot
|
|
|
|
| 10 |
widget:
|
| 11 |
+
- text: 'I know you''re tired, but can we go for another walk this evening?
|
| 12 |
+
|
| 13 |
+
person beta:
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
'
|
| 17 |
+
example_title: walk
|
| 18 |
+
- text: 'Have you done anything exciting lately?
|
| 19 |
+
|
| 20 |
+
person beta:
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
'
|
| 24 |
+
example_title: activities
|
| 25 |
+
- text: 'hey - do you have a favorite grocery store around here?
|
| 26 |
+
|
| 27 |
+
person beta:
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
'
|
| 31 |
+
example_title: grocery
|
| 32 |
+
- text: 'Can you take me for dinner somewhere nice this time?
|
| 33 |
+
|
| 34 |
+
person beta:
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
'
|
| 38 |
+
example_title: dinner
|
| 39 |
+
- text: 'What''s your favorite form of social media?
|
| 40 |
+
|
| 41 |
+
person beta:
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
'
|
| 45 |
+
example_title: social media
|
| 46 |
+
- text: 'Hi, how are you?
|
| 47 |
+
|
| 48 |
+
person beta:
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
'
|
| 52 |
+
example_title: greeting
|
| 53 |
+
- text: 'I am the best; my sister is the worst. What am I?
|
| 54 |
+
|
| 55 |
+
person beta:
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
'
|
| 59 |
+
example_title: sister
|
| 60 |
+
- text: 'What do you call an alligator who''s just had surgery to remove his left
|
| 61 |
+
arm?
|
| 62 |
+
|
| 63 |
+
person beta:
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
'
|
| 67 |
+
example_title: alligator
|
| 68 |
+
- text: 'A man walks into a bar and asks for a drink. The bartender asks for $10,
|
| 69 |
+
and he pays him $1. What did he pay him with?
|
| 70 |
+
|
| 71 |
+
person beta:
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
'
|
| 75 |
+
example_title: dollar
|
| 76 |
+
- text: 'What did I say was in the mailbox when it was actually in the cabinet?
|
| 77 |
+
|
| 78 |
+
person beta:
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
'
|
| 82 |
+
example_title: mailbox
|
| 83 |
+
- text: 'My friend says that she knows every language, but she doesn''t speak any
|
| 84 |
+
of them.. what''s wrong with her?
|
| 85 |
+
|
| 86 |
+
person beta:
|
| 87 |
+
|
| 88 |
|
| 89 |
+
'
|
| 90 |
+
example_title: language
|
| 91 |
inference:
|
| 92 |
parameters:
|
| 93 |
min_length: 2
|
| 94 |
max_length: 64
|
| 95 |
length_penalty: 0.7
|
| 96 |
no_repeat_ngram_size: 2
|
| 97 |
+
do_sample: true
|
| 98 |
top_p: 0.95
|
| 99 |
top_k: 20
|
| 100 |
temperature: 0.3
|
| 101 |
repetition_penalty: 3.5
|
| 102 |
+
base_model: distilgpt2
|
| 103 |
---
|
| 104 |
|
| 105 |
|