Instructions to use OpenAssistant/oasst-sft-1-pythia-12b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenAssistant/oasst-sft-1-pythia-12b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenAssistant/oasst-sft-1-pythia-12b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OpenAssistant/oasst-sft-1-pythia-12b")
model = AutoModelForCausalLM.from_pretrained("OpenAssistant/oasst-sft-1-pythia-12b")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use OpenAssistant/oasst-sft-1-pythia-12b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenAssistant/oasst-sft-1-pythia-12b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenAssistant/oasst-sft-1-pythia-12b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/OpenAssistant/oasst-sft-1-pythia-12b

SGLang

How to use OpenAssistant/oasst-sft-1-pythia-12b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenAssistant/oasst-sft-1-pythia-12b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenAssistant/oasst-sft-1-pythia-12b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenAssistant/oasst-sft-1-pythia-12b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenAssistant/oasst-sft-1-pythia-12b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use OpenAssistant/oasst-sft-1-pythia-12b with Docker Model Runner:
```
docker model run hf.co/OpenAssistant/oasst-sft-1-pythia-12b
```

Open-Assistant SFT-1 12B Model

This is the first iteration English supervised-fine-tuning (SFT) model of the Open-Assistant project. It is based on a Pythia 12B that was fine-tuned on ~22k human demonstrations of assistant conversations collected through the https://open-assistant.io/ human feedback web app before March 7, 2023.

Model Details

Developed by: Open-Assistant Contributors
Model type: Transformer-based Language Model
Language: English
Finetuned from: EleutherAI / pythia-12b-deduped
Code: Open-Assistant/model/model_training
Demo: Continuations for 250 random prompts (sampling code).
License: Apache 2.0
Contact: Open-Assistant Discord

Prompting

Input prompt example:

<|prompter|>What is a meme, and what's the history behind this word?<|endoftext|><|assistant|>

The input ends with the <|assistant|> token to signal that the model should start generating the assistant reply.

Example of generated continuation (typical sampling 0.2):

A visual joke often accompanied by text that spreads rapidly from individual
to individual. The exact origin of the term "meme" is unclear, but it was
first used in the 1990s by Richard Dawkins in his book "The Selfish Gene."
He defined a meme as an idea, behavior, or style that is transmitted from
one person to another within a culture.

Dawkins' use of the term caught on among scholars and journalists, who began
to apply the concept to other types of cultural items such as advertisements,
fashion trends, and jokes. Over time, the term "meme" has come to be used
more generally to describe any social behavior that can be repeated and
altered by individuals. Today, the term is widely recognized and applied in
fields ranging from anthropology to psychology to marketing.<|endoftext|>

Limitations

See limitations of Pythia 12B base model here.

The model is known to fail horribly at answering math and coding questions.

Beware of hallucinations: Outputs are often factually wrong or misleading. Replies might look convincing (at first glance) while containing completely made up false statements.

This model is usable only for English conversations.

Downloads last month: 1,346

OpenAssistant
/

oasst-sft-1-pythia-12b

Open-Assistant SFT-1 12B Model

Model Details

Prompting

Limitations

Spaces using OpenAssistant/oasst-sft-1-pythia-12b 100