Instructions to use vuminhtue/DialoGPT-large-HarryPotter3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use vuminhtue/DialoGPT-large-HarryPotter3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="vuminhtue/DialoGPT-large-HarryPotter3")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("vuminhtue/DialoGPT-large-HarryPotter3")
model = AutoModelForCausalLM.from_pretrained("vuminhtue/DialoGPT-large-HarryPotter3")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use vuminhtue/DialoGPT-large-HarryPotter3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "vuminhtue/DialoGPT-large-HarryPotter3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vuminhtue/DialoGPT-large-HarryPotter3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/vuminhtue/DialoGPT-large-HarryPotter3

SGLang

How to use vuminhtue/DialoGPT-large-HarryPotter3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "vuminhtue/DialoGPT-large-HarryPotter3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vuminhtue/DialoGPT-large-HarryPotter3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "vuminhtue/DialoGPT-large-HarryPotter3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vuminhtue/DialoGPT-large-HarryPotter3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use vuminhtue/DialoGPT-large-HarryPotter3 with Docker Model Runner:
```
docker model run hf.co/vuminhtue/DialoGPT-large-HarryPotter3
```

YAML Metadata Warning:The pipeline tag "conversational" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other

This is Conversational chatbot built on top of DialoGPT-large with the inclusion of Harry Potter scripts, downloaded from Kaggle here. The script is merged from 3 Harry Potter movies

Thanks to Lynn Zhang for her tutorial here that inspired me to build this chatbot.

How to run the model

Due to limitation in cloud computing from Hugging Face it might not be able to run the deployed model here, so I download the model to run on my local HPC system. Here is the script that I used to enable the 4 line chat:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("vuminhtue/DialoGPT-large-HarryPotter3")

model = AutoModelForCausalLM.from_pretrained("vuminhtue/DialoGPT-large-HarryPotter3")


# Let's chat for 4 lines
for step in range(4):
    # encode the new user input, add the eos_token and return a tensor in Pytorch
    new_user_input_ids = tokenizer.encode(input(">> User:") + tokenizer.eos_token, return_tensors='pt')
    # print(new_user_input_ids)

    # append the new user input tokens to the chat history
    bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids

    # generated a response while limiting the total chat history to 1000 tokens, 
    chat_history_ids = model.generate(
        bot_input_ids, max_length=200,
        pad_token_id=tokenizer.eos_token_id,  
        no_repeat_ngram_size=3,       
        do_sample=True, 
        top_k=10, 
        top_p=0.5,
        temperature=0.5
    )
    
    # pretty print last ouput tokens from bot
    print("HarryPotter_Bot: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))

Downloads last month: 3