Instructions to use ModelCloud/dbrx-instruct-converted-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ModelCloud/dbrx-instruct-converted-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ModelCloud/dbrx-instruct-converted-v2", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("ModelCloud/dbrx-instruct-converted-v2", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ModelCloud/dbrx-instruct-converted-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ModelCloud/dbrx-instruct-converted-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ModelCloud/dbrx-instruct-converted-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ModelCloud/dbrx-instruct-converted-v2

SGLang

How to use ModelCloud/dbrx-instruct-converted-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ModelCloud/dbrx-instruct-converted-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ModelCloud/dbrx-instruct-converted-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ModelCloud/dbrx-instruct-converted-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ModelCloud/dbrx-instruct-converted-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ModelCloud/dbrx-instruct-converted-v2 with Docker Model Runner:
```
docker model run hf.co/ModelCloud/dbrx-instruct-converted-v2
```

Upload folder using huggingface_hub

by lrl-modelcloud - opened Jun 23, 2024

base: refs/heads/main

←

from: refs/pr/2

Discussion Files changed

+5005

-3

9768ebe43b80c11e575db1d951572af2bb45474ede807ad71ff41445685fee7576b57542

44e5da5ca204044dbfb00cb3f9fa98a490e23d0f03db129f6b218d6411c04ff77560eb43

f982a90e8b1bd08203b3a06efc3d56cb6f4b9b15597961014c29c8fb3280bbf5487be7a8

e6629b0df4bd0297e9c54b115b77ddabda30f8b1e2599a8512ba501e3c02032cead3f331

319009fdee789e4dc2ef9ade44fdde27bb7f8714faff7ed4bb88fc80a92c255afd20877f

2038ed65c67f787b0f554d222d5f5071e00077de44bbbcfcabd9a7a8cbf1b61c8c939d5c

2db0cbc992f5e8d40ff6fc8b12c12d27c33f086406fe9531978269bf519bb387e3113a88

47acf2e55b2e182c49564b82c51e9aadc5e7152371113e35bcb706ac383139841c6cfeb1

b17a0d2943e6a7e91975da811fc09702a58e9988fb66862aaad53c715448c6da969b29dc

1050c18c71bb26b670f3cb059f49e8a9347bac319ed332127728993246730a53f0f2ee6d

2c42a75c50c49f8504ec46b8e6790380cb41013ce05c11773244e67c12f047d578c61d60

8037d19d50daa5d7c7173a06ed6f210bda60895e66befe7b555f3b5cfda902a9c2b98e30

e7bdbe573c94586a7fd098baf980ab87fb0ea21f49c778d7aa384a50ca42a2018cf536a4

45f2fcb3f7b589ee99dce4b45be692e58ce447677c041bc56ce2d217f36b49a609eb9fa8

288baaa26763a63582d9543e11b6760abfbc36291404e07e33ccfb04b5a000f66c539bd4

3bea9eb52bedaea7e05a08d10967fa0fcc3aa4c3d03bb89da901704eacfda64e52d2ba4a

8f0b6837bc3e80e1fa3c61f794cf4360b744a8867cae235cc553388bfa490ec2f4934fe4

d14394874f4c761910c5d14684445a287b992c7fe55f9d06293509b78811a22c8ec6dce1

cdfff6e0585be6a7194c489f5babeefb2363087256626ef5bfeaf2a0817ecc85faa228a2

087cc7d625a4a9e3fc91faa8c705bfcadbe6dab8548ff4f6cf07750ed02731b2a2b5620b

b6cf9a8729b7d32d1d64b4b4e2f48933812a730fbc012d9b1daceeb1f110534d0de01f73

632cb309b859bce20c8d3e1b3b105ceba7ce0f41eeaf672562ded32c2b0a07f335fb66cf

295a893c6c2b15075f795741ee52333c0dec32a5b36f736e7dd6bac79bc29c8c915ed16d

920f0d7709d09ec3f2b29c4f4f86c098903784994164af03db62a1d440bccb5953e65cc8

67194ecdd290d6ecd392026855c6740e384a2b1f4ef869a31124a0b81c9ddb549c053e74

b33e86e80e0b1c125497a8f9a6ad1eafa0ae70f24d9551d77b1e79ecb919e9bcff6dda62

8917f1518e26013bd41106977befc01e3fa76ab2162643d73faa3e2ac7059a50a3db7f20

b1adc1a4412a577b938d4ebe54231a1ab0458b163f94644fdefd1223809055fdee0c0c8a

0964f2c6379e965663237d248b8cc4f5877d61431f3143db933ed750ad810b8fa5452f72

7f8051ffc6cc69e994ff64860de70d170147c300dd492f1e857e038579e9f23b19a7e948

f49931de7748c7ceb9366c2eae4c1595f85e62d7506d2e789277cabbde0becd16e7115b1

6b9b2d5e6e979a8c1f85134b209dd071168428e8bc7372cc2352f504e1667786bc85fc9c

d4f0b1ab5337758a1b1bd7d24139f5d042d8b0ccfad55e23fc45842576afa526a16614ec

33bd963adf78dacb049bf13fcc0a4792b5fdec7ba73dcf7815c33ca50df23efa57d1b7be

387413fa44067d82d80c1f49f1f446d46c3821838e8baa0dbbe2d209a275b90f08d06df2

a55402b96ab8b068fd7a60c07527a6b9b689f539d60397d55ad9a010287dcabc6919c473

c48ff22578781ed20bc182dfa9c06fd94712a7b8dd7fb1b088af0122bf774221d42284e4

1abbd1271518e8d61d4f8e62034f5cb4c2a67b96fc7e88c6d837e558f523cb078c75adc4

7e8735194d382c1e3d5631dcef79a27c472e8bd2bf79276ee559c2147cf02f11dc7df262

23ad6e293484ab9b71b6ef9426e700dc6766119fd62c537f961d7eb4738d9684d0472e83

4c74b0c5a47dbca191eb0bd1877e77d128e8149fb54db9126e9e54ea7ba5c9011915de52

b9a2193d710d070ccab4271a8f910baedc59074ea85fb6e3ce99fb931512e5e53c3663c6

7b730fa7dc0adfe344c7e7f8083cf64580e521cb9328ccc27b40d284441ec10ba26cdc66

6b9ec34d5b4ed43aa27592e10e2fb508002e431ea6289e38fdbe88f5ba71d0a659f7a149

bbc38fc35a274ef24e7873f38c37e1a5de54906987f95da42d71a87f2975e9d83894fae8

0eb9769927829f93d9e88b51b5d6a91cd99ce6c803f263d589c13a651898512206e84ed1

29aa6aede56f3a1a666121806e1bc7a5aae7d40defaa88965c8d9cbac1aa025468fa8da0

24565d7c067ad1e21c7745e4d7f30d89c77d662d4e9b77c29756d55d83cd4969a0342bb5

d6c826650939059066c08cadde4c784df24b41c26b8b13d49a806e991e453f6d6c880912

e225e85722768c1fd988367213eea22198114d9a8db6621144b6a168ddf7561d1c7caeb8

e6512a4be02c5a30c1a6bfb86f7818d5ae1f9ece4682296906ce3106a6eb4bfe4c92f9c8

cd78ea3bf8dc2a261a2a254bc4786fae95e2ae5cfdf98225c84c512ae17df5d1d18fa255

2bbe8357fe7c21cd3d04a4b1d06f659e501209223b31a99c46a5cfe0a757e830cb2bfd2c

a62f6115a3ca31ca86d5f6e2279302b2c99510192e3eabf946ffcae1a14ce3c0810bcb43

74628101b328c875eabccb8c2535a403af3db7ff85989f4523c238c777222307ae044a16

1b74bb7a68b8d5a8376e33505cf125f713be199c9f55c648d55af07686924975fa6754fe

a635328c68b66a235159d525c8701d70a0deeea45eace07b758f0715d4d0ccf0e7c7421e

3f3fd7890c7aebb3172b13536812af6741204848e11ce4fd101998c21a56ac74e2fc1bfa

2eb7edb4742ee0e4924d9f8a2e0feb480f703506cfbc10f2a7190f2d129a092395c41459

3e51a202d07500da13fa32bde9c0c1f0a2ed2473cc4d8dfa5ae1bcb2a413d705337f50de

ea89e2716c35e4dc41d4c070f91f9887a896b63d689c1306a0d617adb3e505ec00e4ebde

38d52afe98894b7e91ddd95b616ceb46008c189834b5698652527c83023053e4acc23e84

lrl-modelcloud changed pull request title from [WIP] Upload folder using huggingface_hub (multi-commit cd5e054e267c35c0c776a9d4cad337ac8625e506feb6a0827c7313317eeb36b2) to Upload folder using huggingface_hub Jun 23, 2024

lrl-modelcloud

Jun 23, 2024

Multi-commit is now completed! You can ping the repo owner to review the changes. This PR can now be commented or modified without risking to corrupt it.

This is a comment posted using the huggingface_hub library in the context of a multi-commit. Learn more about multi-commits in this guide.

lrl-modelcloud changed pull request status to open Jun 23, 2024

lrl-modelcloud changed pull request status to merged Jun 23, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment