How to use from
vLLMInstall from pip and serve model
# Install vLLM from pip:
pip install vllm# Start the vLLM server:
vllm serve "deepakkr/bloomed"# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "deepakkr/bloomed",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Use Docker
docker model run hf.co/deepakkr/bloomedQuick Links
DESCRIPTION:
Built on Bloom-560M and finetuned on a medicines question-answer dataset, this experimental model explores potential for medical Q&A tasks.
- Downloads last month
- -
# Gated model: Login with a HF token with gated access permission hf auth login