Instructions to use stanford-crfm/levanter-backpack-1b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use stanford-crfm/levanter-backpack-1b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="stanford-crfm/levanter-backpack-1b", trust_remote_code=True)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("stanford-crfm/levanter-backpack-1b", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use stanford-crfm/levanter-backpack-1b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "stanford-crfm/levanter-backpack-1b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "stanford-crfm/levanter-backpack-1b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/stanford-crfm/levanter-backpack-1b

SGLang

How to use stanford-crfm/levanter-backpack-1b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "stanford-crfm/levanter-backpack-1b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "stanford-crfm/levanter-backpack-1b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "stanford-crfm/levanter-backpack-1b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "stanford-crfm/levanter-backpack-1b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use stanford-crfm/levanter-backpack-1b with Docker Model Runner:
```
docker model run hf.co/stanford-crfm/levanter-backpack-1b
```

Configuration Parsing Warning:In config.json: "architectures" must be an array

Model Card for Levanter-Backpack-1.4B

This is 1.4B parameter version of Backpack architecture, intended to combine strong modeling performance with an interface for interpretability and control.

Training Details

Training Data

This model was trained on the OpenWebText corpus.

Training Procedure

This model was trained for 450k gradient steps and cosine decaying learning rate from 1e-4 to zero, with a linear warmup of 5k steps.

Environmental Impact

Hardware Type: v3-128 TPU (128 cores, 2TB Memory)
Hours used: Roughly 8.6 days.
Cloud Provider: Google Cloud Patform
Compute Region: North America.

Model Architecture and Objective

This model was trained to minimize the cross-entropy loss, and is a Backpack language model.

Software

This model was trained with Levanter and Jax.

Loss Curve

How to Get Started with the Model

Please install transformers, safetensors and torch to use this model.

pip install transformers safetensors torch

Run the following Python code:

import torch
import transformers
from transformers import AutoModelForCausalLM


model_id = "stanford-crfm/levanter-backpack-1b"
config = transformers.AutoConfig.from_pretrained(model_id, trust_remote_code=True)
torch_model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    config=config, 
    trust_remote_code=True
)
torch_model.eval()

input = torch.randint(0, 50264, (1, 512), dtype=torch.long)
torch_out = torch_model(input, position_ids=None,)
torch_out = torch.nn.functional.softmax(torch_out.logits, dim=-1)
print(torch_out.shape)

Downloads last month: 166

Safetensors

Model size

1B params

Tensor type

F32

Dataset used to train stanford-crfm/levanter-backpack-1b

Paper for stanford-crfm/levanter-backpack-1b

Backpack Language Models

Paper • 2305.16765 • Published May 26, 2023 • 1