Instructions to use FredZhang7/distilgpt2-stable-diffusion-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FredZhang7/distilgpt2-stable-diffusion-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FredZhang7/distilgpt2-stable-diffusion-v2")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("FredZhang7/distilgpt2-stable-diffusion-v2")
model = AutoModelForCausalLM.from_pretrained("FredZhang7/distilgpt2-stable-diffusion-v2")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use FredZhang7/distilgpt2-stable-diffusion-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FredZhang7/distilgpt2-stable-diffusion-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FredZhang7/distilgpt2-stable-diffusion-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/FredZhang7/distilgpt2-stable-diffusion-v2

SGLang

How to use FredZhang7/distilgpt2-stable-diffusion-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FredZhang7/distilgpt2-stable-diffusion-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FredZhang7/distilgpt2-stable-diffusion-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FredZhang7/distilgpt2-stable-diffusion-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FredZhang7/distilgpt2-stable-diffusion-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use FredZhang7/distilgpt2-stable-diffusion-v2 with Docker Model Runner:
```
docker model run hf.co/FredZhang7/distilgpt2-stable-diffusion-v2
```

Some questions about update

by yangming915 - opened Mar 17, 2023

Discussion

yangming915

Mar 17, 2023

Is it updated last day?
I am using the pytorch_model.bin,but I can not download the newer one,the old is 318MB,but I need the new of 334MB. Or you mean the new file model.safetensors? But my git lfs is downloading the pytorch_model.bin 334MB automatically and very slow. If I want to try download it on this page,I will get a 318MB pytorch_model.bin, is anything wrong?

FredZhang7

Owner Mar 17, 2023

Hi! A bot created a pull request yesterday, and I'm not sure how to use the .safetensors version of the model.
If you don't see any errors, nothing is wrong.

yangming915

Mar 18, 2023

Hi! A bot created a pull request yesterday, and I'm not sure how to use the .safetensors version of the model.
If you don't see any errors, nothing is wrong.

I certainly think there is anything wrong, no matter how I download the file 'pytorch_model.bin' or '.safetensors', I always got a 318M size file,but they were updated right? It should be 334M just like the label.
And I use the file I got last day, I certainly know it is the old one.
Thanks for your reply in any case.

FredZhang7

Owner Mar 18, 2023

•

edited Mar 18, 2023

And I use the file I got last day, I certainly know it is the old one.

The model itself hasn't been updated. Only a new version of the model .safetensors is added.

no matter how I download the file 'pytorch_model.bin' or '.safetensors', I always got a 318M size file,but they were updated right? It should be 334M just like the label.

If you download pytorch_model.bin on Chrome, you may get a smaller 318 MB file size. But if you use/download it via transformers, you should get a 316-318 MB file.

FredZhang7 changed discussion status to closed Dec 19, 2023

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment