Instructions to use succinctly/text2image-prompt-generator with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use succinctly/text2image-prompt-generator with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="succinctly/text2image-prompt-generator")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("succinctly/text2image-prompt-generator")
model = AutoModelForCausalLM.from_pretrained("succinctly/text2image-prompt-generator")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use succinctly/text2image-prompt-generator with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "succinctly/text2image-prompt-generator"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "succinctly/text2image-prompt-generator",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/succinctly/text2image-prompt-generator

SGLang

How to use succinctly/text2image-prompt-generator with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "succinctly/text2image-prompt-generator" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "succinctly/text2image-prompt-generator",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "succinctly/text2image-prompt-generator" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "succinctly/text2image-prompt-generator",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use succinctly/text2image-prompt-generator with Docker Model Runner:
```
docker model run hf.co/succinctly/text2image-prompt-generator
```

Comfyui Model Location

by Skettalee - opened Nov 2, 2024

Discussion

Skettalee

Nov 2, 2024

Where in comfyui do these models need to be placed? I have been trying to search through chatGPT and google and still have no idea. ChatGPT seems to think this model needs to be placed in the checkpoints\sdxl\ folder inside of ComfyUi but I do know how often chatGPT gets things 150% wrong and I cant trust her, so im asking here where im downloading the model. Anyone have an idea?

MaleSloganist

Nov 7, 2024

I'm running the comfyui-mixlab-nodes extension, and as per their instructions (https://github.com/shadowcz007/comfyui-mixlab-nodes/blob/main/README.md; scroll all the way to the bottom), you have to first create a sub-directory in your [...]\ComfyUI\models folder called prompt_generator if you don't already have it.

Then, navigate to that directory in PowerShell or cmd, and run the following:

git clone https://huggingface.co/succinctly/text2image-prompt-generator

This will clone it under a new sub-directory it creates called text2image-prompt-generator

Hope this helps.

Skettalee

Nov 7, 2024

Thank you so much!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment