Text Generation
Transformers
PyTorch
Safetensors
English
llama
text generation
instruct
text-generation-inference
Instructions to use PygmalionAI/pygmalion-2-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use PygmalionAI/pygmalion-2-7b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="PygmalionAI/pygmalion-2-7b")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("PygmalionAI/pygmalion-2-7b") model = AutoModelForCausalLM.from_pretrained("PygmalionAI/pygmalion-2-7b") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use PygmalionAI/pygmalion-2-7b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "PygmalionAI/pygmalion-2-7b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PygmalionAI/pygmalion-2-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/PygmalionAI/pygmalion-2-7b
- SGLang
How to use PygmalionAI/pygmalion-2-7b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "PygmalionAI/pygmalion-2-7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PygmalionAI/pygmalion-2-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "PygmalionAI/pygmalion-2-7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PygmalionAI/pygmalion-2-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use PygmalionAI/pygmalion-2-7b with Docker Model Runner:
docker model run hf.co/PygmalionAI/pygmalion-2-7b
Add some snippets to the README
Browse filesHi PygmalionAI team,
I'm a big fan of your models (Been trying a few of them to integrate into a project I'm working on). But there's been quite a few rough spots and I wanted to see what you thought about including some snippets like the one's in this PR just to give people that first step for using your model(s) which might help promote wider use of them.
If you're not looking for any kind of changes like this just disregrard or close this PR :)
README.md
CHANGED
|
@@ -32,6 +32,26 @@ and conversations with synthetically generated instructions attached.
|
|
| 32 |
|
| 33 |
This model is freely available for both commercial and non-commercial use, as per the Llama-2 license.
|
| 34 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
|
| 36 |
## Prompting
|
| 37 |
|
|
@@ -52,6 +72,19 @@ The system prompt has been designed to allow the model to "enter" various modes
|
|
| 52 |
You shall reply to the user while staying in character, and generate long responses.
|
| 53 |
```
|
| 54 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
## Dataset
|
| 56 |
The dataset used to fine-tune this model includes our own [PIPPA](https://huggingface.co/datasets/PygmalionAI/PIPPA), along with several other instruction
|
| 57 |
datasets, and datasets acquired from various RP forums.
|
|
|
|
| 32 |
|
| 33 |
This model is freely available for both commercial and non-commercial use, as per the Llama-2 license.
|
| 34 |
|
| 35 |
+
## Model Initialisation
|
| 36 |
+
|
| 37 |
+
One way to get started with the model is using HuggingFace's [transformers](https://huggingface.co/docs/transformers/index) library:
|
| 38 |
+
|
| 39 |
+
```python
|
| 40 |
+
import torch
|
| 41 |
+
from transformers import AutoTokenizer, pipeline
|
| 42 |
+
|
| 43 |
+
# App config
|
| 44 |
+
modelName = "PygmalionAI/pygmalion-2-7b"
|
| 45 |
+
|
| 46 |
+
# Model Initialisation
|
| 47 |
+
tokenizer = AutoTokenizer.from_pretrained(modelName)
|
| 48 |
+
pipeline = pipeline(
|
| 49 |
+
"text-generation",
|
| 50 |
+
model=modelName,
|
| 51 |
+
torch_dtype=torch.float16,
|
| 52 |
+
device="cuda", # cuda on a compatible Nvidia GPU is recommended for running this model
|
| 53 |
+
)
|
| 54 |
+
```
|
| 55 |
|
| 56 |
## Prompting
|
| 57 |
|
|
|
|
| 72 |
You shall reply to the user while staying in character, and generate long responses.
|
| 73 |
```
|
| 74 |
|
| 75 |
+
Using the pipeline snippet above:
|
| 76 |
+
|
| 77 |
+
```python
|
| 78 |
+
conversation_with_response = pipeline(
|
| 79 |
+
"Hi, can you tell me how cool Pygmalion models are?", # Use the tokens described above when prompting
|
| 80 |
+
do_sample=True,
|
| 81 |
+
top_k=10,
|
| 82 |
+
num_return_sequences=1,
|
| 83 |
+
eos_token_id=tokenizer.eos_token_id,
|
| 84 |
+
max_new_tokens=128
|
| 85 |
+
)
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
## Dataset
|
| 89 |
The dataset used to fine-tune this model includes our own [PIPPA](https://huggingface.co/datasets/PygmalionAI/PIPPA), along with several other instruction
|
| 90 |
datasets, and datasets acquired from various RP forums.
|