Instructions to use PygmalionAI/pygmalion-2.7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PygmalionAI/pygmalion-2.7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PygmalionAI/pygmalion-2.7b")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("PygmalionAI/pygmalion-2.7b")
model = AutoModelForCausalLM.from_pretrained("PygmalionAI/pygmalion-2.7b", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use PygmalionAI/pygmalion-2.7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PygmalionAI/pygmalion-2.7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PygmalionAI/pygmalion-2.7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/PygmalionAI/pygmalion-2.7b

SGLang

How to use PygmalionAI/pygmalion-2.7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PygmalionAI/pygmalion-2.7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PygmalionAI/pygmalion-2.7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PygmalionAI/pygmalion-2.7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PygmalionAI/pygmalion-2.7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use PygmalionAI/pygmalion-2.7b with Docker Model Runner:
```
docker model run hf.co/PygmalionAI/pygmalion-2.7b
```

pygmalion-2.7b

Commit History

Nov 17, 2023

21:22
Adding Evaluation Results⁠

leaderboard-pr-bot

69aec20

Jan 12, 2023

19:20
revert "convert to safetensors"⁠

11b

9533805

Jan 11, 2023

23:43
revert KAI chatmode config until related fixes land on stable version

11b

e2fb1c4
21:46
enable chat mode by default on KoboldAI

11b

5d66b8c
05:47
Update README.md

TearGosling

2b4b8b8

Jan 5, 2023

16:04
convert to safetensors

11b

e620263
14:57
initial commit

11b

98a4107

Commit History

Nov 17, 2023

Adding Evaluation Results⁠

Jan 12, 2023

revert "convert to safetensors"⁠

Jan 11, 2023

revert KAI chatmode config until related fixes land on stable version

enable chat mode by default on KoboldAI

Update README.md

Jan 5, 2023

convert to safetensors

initial commit