Instructions to use edumunozsala/unsloth-llama-2-7B-python-coder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use edumunozsala/unsloth-llama-2-7B-python-coder with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="edumunozsala/unsloth-llama-2-7B-python-coder")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("edumunozsala/unsloth-llama-2-7B-python-coder")
model = AutoModelForCausalLM.from_pretrained("edumunozsala/unsloth-llama-2-7B-python-coder")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use edumunozsala/unsloth-llama-2-7B-python-coder with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "edumunozsala/unsloth-llama-2-7B-python-coder"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "edumunozsala/unsloth-llama-2-7B-python-coder",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/edumunozsala/unsloth-llama-2-7B-python-coder

SGLang

How to use edumunozsala/unsloth-llama-2-7B-python-coder with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "edumunozsala/unsloth-llama-2-7B-python-coder" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "edumunozsala/unsloth-llama-2-7B-python-coder",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "edumunozsala/unsloth-llama-2-7B-python-coder" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "edumunozsala/unsloth-llama-2-7B-python-coder",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use edumunozsala/unsloth-llama-2-7B-python-coder with Docker Model Runner:
```
docker model run hf.co/edumunozsala/unsloth-llama-2-7B-python-coder
```

edumunozsala commited on Jan 8, 2024

Commit

b60022e

1 Parent(s): 64f7a0f

Upload LlamaForCausalLM

Browse files

Files changed (6) hide show

model-00001-of-00006.safetensors +1 -1
model-00002-of-00006.safetensors +1 -1
model-00003-of-00006.safetensors +1 -1
model-00004-of-00006.safetensors +1 -1
model-00005-of-00006.safetensors +1 -1
model-00006-of-00006.safetensors +1 -1

model-00001-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:900715890d072754bf4af458d51232ef7a6c4dd1cbf2caf24a8b20da1714ef32
 size 4840396416

 version https://git-lfs.github.com/spec/v1
+oid sha256:b59d4e90d0a4b0c12c9a4d946626fdb4d03bf766c620adedcc909c62e381ce60
 size 4840396416

model-00002-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c48493dcb1c7cfd1c80c038a7ceac58ec326e508f608a586c2ec45f447ae6a41
 size 4857206856

 version https://git-lfs.github.com/spec/v1
+oid sha256:78090ffbde9f5dce00f35c78683f34a43fe830b62f16b6df4fd4fc0c8a15eeea
 size 4857206856

model-00003-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5a01e909d285cfaa87ae4565b280551d204b0755dcaf17edcbaa496325b2315a
 size 4857206904

 version https://git-lfs.github.com/spec/v1
+oid sha256:6bbe34982b5ba76e24cc80b113afc954645278da5fe82fca064817193f9841e6
 size 4857206904

model-00004-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1b5b888dc445d6f1acfb78358dc4d1cb8380c44a3caa74077cd53390a4529a70
 size 4857206904

 version https://git-lfs.github.com/spec/v1
+oid sha256:f7aaab6f71a6b1029b612e19a38770e3d9f76a872d383f733215a8a3bb12fbab
 size 4857206904

model-00005-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c1ae99130dee9670ddb7965aab05e23efd22d05b4baa7a495e8436ffe8426fb5
 size 4857206904

 version https://git-lfs.github.com/spec/v1
+oid sha256:9d702c2cec1522f02b3647b5e3409092db067bc9a30a2839bec492148cdecbbc
 size 4857206904

model-00006-of-00006.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:859946b00913961636ee94afcee241a08bc64074bcecf351bad74a6d546db68c
 size 2684472112

 version https://git-lfs.github.com/spec/v1
+oid sha256:f99a756ac17b08db61cbc0f0134f8efbd85af74b8e6141e480a8a015973ef74b
 size 2684472112