Instructions to use mohammedaly22/t5-small-question-generation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mohammedaly22/t5-small-question-generation with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mohammedaly22/t5-small-question-generation")

# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("mohammedaly22/t5-small-question-generation")
model = AutoModelForSeq2SeqLM.from_pretrained("mohammedaly22/t5-small-question-generation")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mohammedaly22/t5-small-question-generation with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mohammedaly22/t5-small-question-generation"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mohammedaly22/t5-small-question-generation",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/mohammedaly22/t5-small-question-generation

SGLang

How to use mohammedaly22/t5-small-question-generation with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mohammedaly22/t5-small-question-generation" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mohammedaly22/t5-small-question-generation",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mohammedaly22/t5-small-question-generation" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mohammedaly22/t5-small-question-generation",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use mohammedaly22/t5-small-question-generation with Docker Model Runner:
```
docker model run hf.co/mohammedaly22/t5-small-question-generation
```

t5-small-question-generation

Commit History

Update README.md

c400dde
verified

mohammedaly22 commited on Mar 18, 2024

Update README.md

4c4ce4b
verified

Mohammed Aly commited on Mar 18, 2024

Update README.md

b121fab
verified

Mohammed Aly commited on Mar 18, 2024

Update README.md

032e944
verified

Mohammed Aly commited on Mar 18, 2024

Update README.md

5bcec25
verified

Mohammed Aly commited on Mar 18, 2024

Commit Successfully!

5170a2e
verified

Mohammed Aly commited on Mar 4, 2024

initial commit

1c47f86
verified

Mohammed Aly commited on Mar 4, 2024

Commit History

Update README.md c400dde verified

Update README.md 4c4ce4b verified

Update README.md b121fab verified

Update README.md 032e944 verified

Update README.md 5bcec25 verified

Commit Successfully! 5170a2e verified

initial commit 1c47f86 verified

Update README.md

c400dde
verified

Update README.md

4c4ce4b
verified

Update README.md

b121fab
verified

Update README.md

032e944
verified

Update README.md

5bcec25
verified

Commit Successfully!

5170a2e
verified

initial commit

1c47f86
verified