Instructions to use Devden/DialectAI-Vicuna-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Devden/DialectAI-Vicuna-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Devden/DialectAI-Vicuna-7B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Devden/DialectAI-Vicuna-7B")
model = AutoModelForCausalLM.from_pretrained("Devden/DialectAI-Vicuna-7B")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Devden/DialectAI-Vicuna-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Devden/DialectAI-Vicuna-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Devden/DialectAI-Vicuna-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Devden/DialectAI-Vicuna-7B

SGLang

How to use Devden/DialectAI-Vicuna-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Devden/DialectAI-Vicuna-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Devden/DialectAI-Vicuna-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Devden/DialectAI-Vicuna-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Devden/DialectAI-Vicuna-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Devden/DialectAI-Vicuna-7B with Docker Model Runner:
```
docker model run hf.co/Devden/DialectAI-Vicuna-7B
```

Osamarafique998 commited on Jul 11, 2023

Commit

f0eb694

1 Parent(s): e1df800

Upload folder using huggingface_hub

Browse files

Files changed (9) hide show

pytorch_model-00001-of-00002.bin +1 -1
pytorch_model-00002-of-00002.bin +1 -1
rng_state_0.pth +1 -1
rng_state_1.pth +1 -1
rng_state_2.pth +1 -1
rng_state_3.pth +1 -1
scheduler.pt +1 -1
trainer_state.json +0 -0
training_args.bin +1 -1

pytorch_model-00001-of-00002.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a2a22a59a791bc5836ca236e2bfc50cd0da7344d77f3ef42ef59f6210ef899e2
 size 9449597278

 version https://git-lfs.github.com/spec/v1
+oid sha256:d8b4e2c5aee2d3565051e127020bfc68e12bf457360dfc578e34cfcfe13bf290
 size 9449597278

pytorch_model-00002-of-00002.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9e12c66b8e32114d61e53e1d9c1380cc7a253a8fcc709550fee6d12887a39ded
 size 1949353379

 version https://git-lfs.github.com/spec/v1
+oid sha256:ce00796419bea69be589e13efc5552ca9c96ebb091df9a3c696fa070f403483b
 size 1949353379

rng_state_0.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4403f335fe3bcfa2bb3442812c81ed28a3c794a53ab3823c8adf65f0e19d6715
 size 14583

 version https://git-lfs.github.com/spec/v1
+oid sha256:87477d0f8edf067f50c88b4946719703de3bfabd31d09d4b7ddf0a17e7353fe8
 size 14583

rng_state_1.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:921a379f78523ffaeb8e7ffb1bc2889efcb776efb9a8d8b6ec45f71a9421df8a
 size 14583

 version https://git-lfs.github.com/spec/v1
+oid sha256:eab3593d5b9b66e3025a62f69b1c19ca9adf61b7912a7c90064de15176284593
 size 14583

rng_state_2.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0e8383a23c1ee676550095ccbf05f3bb9ee719c8be4fbc85a348a3bb3eb17eb0
 size 14583

 version https://git-lfs.github.com/spec/v1
+oid sha256:e902f55f5eff3be1ef7fe2935b104fa959e9cf9c432f0c7fd5f80e7d77de61c5
 size 14583

rng_state_3.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d107caf2bca1d4f924e93b2ad57ba9de883cd592e5b9459b95858d6f161d781a
 size 14583

 version https://git-lfs.github.com/spec/v1
+oid sha256:5e4b14613ded4830170a976a4e690d15e8aacb2ee65182d9240d54cde3938296
 size 14583

scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2f7a2719e097124bb18a881ed98b7ce282df9ea30a1e6781ab0ed94992674765
 size 627

 version https://git-lfs.github.com/spec/v1
+oid sha256:0c050ca74b6f9f8f4a6b3c047a2a39cfd44f9f45a1bb0e99e65e55adad696b85
 size 627

trainer_state.json CHANGED Viewed

The diff for this file is too large to render. See raw diff

training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f08b6424d90cfeba014939a11d98790f990c589c6ad724f1810510885596a1d3
 size 3771

 version https://git-lfs.github.com/spec/v1
+oid sha256:bd79995eb50e2a9114d5c9c816c56c9095e5e18d9043dc3135caed891cd6c73d
 size 3771