Instructions to use Nitral-Archive/Prima-Pastacles-7b-128k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nitral-Archive/Prima-Pastacles-7b-128k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nitral-Archive/Prima-Pastacles-7b-128k", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Nitral-Archive/Prima-Pastacles-7b-128k", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Nitral-Archive/Prima-Pastacles-7b-128k", trust_remote_code=True)

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Nitral-Archive/Prima-Pastacles-7b-128k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nitral-Archive/Prima-Pastacles-7b-128k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nitral-Archive/Prima-Pastacles-7b-128k",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Nitral-Archive/Prima-Pastacles-7b-128k

SGLang

How to use Nitral-Archive/Prima-Pastacles-7b-128k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nitral-Archive/Prima-Pastacles-7b-128k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nitral-Archive/Prima-Pastacles-7b-128k",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nitral-Archive/Prima-Pastacles-7b-128k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nitral-Archive/Prima-Pastacles-7b-128k",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Nitral-Archive/Prima-Pastacles-7b-128k with Docker Model Runner:
```
docker model run hf.co/Nitral-Archive/Prima-Pastacles-7b-128k
```

Prima-Pastacles-7b-128k

Commit History

Adding Evaluation Results (#1)

bdeca17
verified

Nitral

leaderboard-pr-bot commited on Mar 4, 2024

Update config.json

0e9e531
verified

Nitral commited on Feb 27, 2024

Upload 2 files

58a4b01
verified

Nitral commited on Feb 19, 2024

Update README.md

556d16a
verified

Nitral commited on Feb 15, 2024

Update README.md

1be630f
verified

Nitral commited on Feb 15, 2024

Update README.md

1be449d
verified

Nitral commited on Feb 14, 2024

Update README.md

fdbcb23
verified

Nitral commited on Feb 13, 2024

Upload folder using huggingface_hub

a89f4d0
verified

Nitral commited on Feb 13, 2024

initial commit

ab173a3
verified

Nitral commited on Feb 13, 2024

Commit History

Adding Evaluation Results (#1) bdeca17 verified

Update config.json 0e9e531 verified

Upload 2 files 58a4b01 verified

Update README.md 556d16a verified

Update README.md 1be630f verified

Update README.md 1be449d verified

Update README.md fdbcb23 verified

Upload folder using huggingface_hub a89f4d0 verified

initial commit ab173a3 verified

Adding Evaluation Results (#1)

bdeca17
verified

Update config.json

0e9e531
verified

Upload 2 files

58a4b01
verified

Update README.md

556d16a
verified

Update README.md

1be630f
verified

Update README.md

1be449d
verified

Update README.md

fdbcb23
verified

Upload folder using huggingface_hub

a89f4d0
verified

initial commit

ab173a3
verified