Instructions to use OpenAssistant/stablelm-7b-sft-v7-epoch-3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenAssistant/stablelm-7b-sft-v7-epoch-3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenAssistant/stablelm-7b-sft-v7-epoch-3")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OpenAssistant/stablelm-7b-sft-v7-epoch-3")
model = AutoModelForCausalLM.from_pretrained("OpenAssistant/stablelm-7b-sft-v7-epoch-3", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OpenAssistant/stablelm-7b-sft-v7-epoch-3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenAssistant/stablelm-7b-sft-v7-epoch-3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenAssistant/stablelm-7b-sft-v7-epoch-3",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/OpenAssistant/stablelm-7b-sft-v7-epoch-3

SGLang

How to use OpenAssistant/stablelm-7b-sft-v7-epoch-3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenAssistant/stablelm-7b-sft-v7-epoch-3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenAssistant/stablelm-7b-sft-v7-epoch-3",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenAssistant/stablelm-7b-sft-v7-epoch-3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenAssistant/stablelm-7b-sft-v7-epoch-3",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use OpenAssistant/stablelm-7b-sft-v7-epoch-3 with Docker Model Runner:
```
docker model run hf.co/OpenAssistant/stablelm-7b-sft-v7-epoch-3
```

License tag

by Aspie96 - opened Apr 24, 2023

Discussion

Aspie96

Apr 24, 2023

The model is based on an open source version of StableLM under the CC BY-SA 4.0 license.

While I don't think models are (or should be) copyrightable, we'd better act as if they are, which is why licensing is being dealt with to begin with.

While the CC BY-SA 4.0 is a free and open license, it is (unlike, for example, Apache 2.0, MIT or CC BY 4.0) a copyleft license.

The model cart asserts that the license for this model is CC BY-SA 4.0, but the tag still incorrectly states "apache-2.0". It's the latter that should be fixed.

It must be noted that while Pythia versions of Open Assistant are purposefully under non-copyleft licenses, StableLM-based ones are under a copyleft license. This is ok, it's just important to be clear with those who may use the model.

(Note: if assets in the dataset are also derived by assets under Apache 2.0, this raises the question of compatibility between the two licenses. If they are directly compatible, CC BY-SA 4.0 is fine for this project. If not, GPL 3.0 would solve the issue as both licenses are compatible with it).

dvruette

OpenAssistant org Apr 27, 2023

Should be fixed now

dvruette changed discussion status to closed Apr 27, 2023

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment