Instructions to use Fredithefish/CrimsonPajama with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Fredithefish/CrimsonPajama with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Fredithefish/CrimsonPajama")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Fredithefish/CrimsonPajama")
model = AutoModelForCausalLM.from_pretrained("Fredithefish/CrimsonPajama", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Fredithefish/CrimsonPajama with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Fredithefish/CrimsonPajama"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Fredithefish/CrimsonPajama",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Fredithefish/CrimsonPajama

SGLang

How to use Fredithefish/CrimsonPajama with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Fredithefish/CrimsonPajama" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Fredithefish/CrimsonPajama",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Fredithefish/CrimsonPajama" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Fredithefish/CrimsonPajama",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Fredithefish/CrimsonPajama with Docker Model Runner:
```
docker model run hf.co/Fredithefish/CrimsonPajama
```

CrimsonPajama

This model was finetuned 1 epoch on the GPTeacher dataset using LoRA. A 2 epoch finetuned model will be released in the coming days.

Model Details

Base Model: RedPajama-INCITE-Chat-3B
Training Data: GPTeacher dataset
Model type: Language Model
License: Apache 2.0
Training Time: 2:20:53 hours on NVIDIA Tesla T4

Finetune Code for RedPajama-Chat in notebook format can be found here

About the dataset

A collection of modular datasets generated by GPT-4, General-Instruct - Roleplay-Instruct - Code-Instruct - and Toolformer

Comparison with RedPajama

Roleplay

RedPajama	CrimsonPajama
I want you to act like a pirate.	I want you to act like a pirate.
Hai, matey! Why do you look so sad? There’s treasure everywhere!	Ahoy, mateys! What be ye lookin’ fer? Today, we be takin’ ‘n adventure an’ need a little sassin’ and swashbucklin’. What’s yer plan, starvin’ mutineer? Pssst, follow me and ye’ll find riches an’ some old treasure sure to bring yer blazes. Together, we’ll laugh ‘till we’re blue in the mouth an’ sail the seas, smilin’ at every fickle sky.

RedPajama	CrimsonPajama
I want you to act like a waiter at a fancy restaurant.	I want you to act like a waiter at a fancy restaurant.
Here I am, a waiter at an iconic world-class 5-star restaurant.	Ladies and gentlemen, may I introduce myself? I’m your server today. I’m here to provide the finest cuisine and ensure you enjoy your meal in a dignified manner. Please excuse me for not knowing your preferences, but I’m happy to help out as best as I can. Please join me on this culinary journey and savor the delights of our restaurant. Are there any questions or requests you have? Anything at all? Oh, and may I recommend the signature dessert? Cheers! We’d love to hear your feedback! Stay healthy, and we’ll see you soon.

RedPajama

CrimsonPajama

I want you to act like a waiter at a fancy restaurant.

Here I am, a waiter at an iconic world-class 5-star restaurant.

Ladies and gentlemen, may I introduce myself? I’m your server today. I’m here to provide the finest cuisine and ensure you enjoy your meal in a dignified manner. Please excuse me for not knowing your preferences, but I’m happy to help out as best as I can. Please join me on this culinary journey and savor the delights of our restaurant. Are there any questions or requests you have? Anything at all? Oh, and may I recommend the signature dessert? Cheers! We’d love to hear your feedback! Stay healthy, and we’ll see you soon.

Downloads last month: 81

Dataset used to train Fredithefish/CrimsonPajama

Spaces using Fredithefish/CrimsonPajama 3

Paper for Fredithefish/CrimsonPajama

LoRA: Low-Rank Adaptation of Large Language Models

Paper • 2106.09685 • Published Jun 17, 2021 • 67