Instructions to use nold/Prima-Pastacles-7b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nold/Prima-Pastacles-7b-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("nold/Prima-Pastacles-7b-GGUF", dtype="auto")

llama-cpp-python

How to use nold/Prima-Pastacles-7b-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="nold/Prima-Pastacles-7b-GGUF",
	filename="Prima-Pastacles-7b_Q2_K.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use nold/Prima-Pastacles-7b-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M

Use Docker

docker model run hf.co/nold/Prima-Pastacles-7b-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use nold/Prima-Pastacles-7b-GGUF with Ollama:
```
ollama run hf.co/nold/Prima-Pastacles-7b-GGUF:Q4_K_M
```

Unsloth Studio new

How to use nold/Prima-Pastacles-7b-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nold/Prima-Pastacles-7b-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nold/Prima-Pastacles-7b-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for nold/Prima-Pastacles-7b-GGUF to start chatting

Docker Model Runner
How to use nold/Prima-Pastacles-7b-GGUF with Docker Model Runner:
```
docker model run hf.co/nold/Prima-Pastacles-7b-GGUF:Q4_K_M
```

Lemonade

How to use nold/Prima-Pastacles-7b-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull nold/Prima-Pastacles-7b-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Prima-Pastacles-7b-GGUF-Q4_K_M

List all available models

lemonade list

Upload folder using huggingface_hub

by nold - opened Feb 16, 2024

base: refs/heads/main

←

from: refs/pr/3

Discussion Files changed

-7

nold

Owner Feb 16, 2024

•

edited Feb 16, 2024

Upload folder using huggingface_hub

Multi commit ID: e31ee6f5267b511797fa5b9fdf482b2de096040ff3da28bd60d72fa8f03c1fbe

Scheduled commits:

Upload 1 file(s) totalling 2.7G (0c54f522a1fd13faa9ac58469617bf494b99b2b94a292b2d8910d894251c9f2a)
Upload 1 file(s) totalling 4.4G (8e76b3ebd1f58fd0167391ff2e636d71c327494b28b2b0efaed7724002c0077a)
Upload 1 file(s) totalling 5.1G (5103887c27777daab2b6387eec0ab0dc6bb220670fa8a5a29b9df6ceb299006c)
Upload 1 file(s) totalling 5.9G (c17be9b43cc797ed9779854e718b5ca2f907ed030b68115701597fb49ea11e39)
Upload 1 file(s) totalling 7.7G (07635bdc2a59cfd6499bbb4aa0fd30a3889ce0fdcbda132f239a31a6cf82f313)
Upload 1 file(s) totalling 1.4K (3b9f5a048b8d0f86ec23970620578307a627401bb4e8b05772875f106292dce5)

This is a PR opened using the huggingface_hub library in the context of a multi-commit. PR can be commented as a usual PR. However, please be aware that manually updating the PR description, changing the PR status, or pushing new commits, is not recommended as it might corrupt the commit process. Learn more about multi-commits in this guide.

0c54f522a1fd13faa9ac58469617bf494b99b2b94a292b2d8910d894251c9f2a9b39c80d

8e76b3ebd1f58fd0167391ff2e636d71c327494b28b2b0efaed7724002c0077a2a677582

5103887c27777daab2b6387eec0ab0dc6bb220670fa8a5a29b9df6ceb299006c8ee8f5db

c17be9b43cc797ed9779854e718b5ca2f907ed030b68115701597fb49ea11e392b7950f6

07635bdc2a59cfd6499bbb4aa0fd30a3889ce0fdcbda132f239a31a6cf82f313de2d1b80

nold

Owner Feb 16, 2024

Multi-commit is now completed! You can ping the repo owner to review the changes. This PR can now be commented or modified without risking to corrupt it.

This is a comment posted using the huggingface_hub library in the context of a multi-commit. Learn more about multi-commits in this guide.

nold changed pull request title from [WIP] Upload folder using huggingface_hub (multi-commit e31ee6f5267b511797fa5b9fdf482b2de096040ff3da28bd60d72fa8f03c1fbe) to Upload folder using huggingface_hub Feb 16, 2024

nold changed pull request status to open Feb 16, 2024

nold

Owner Feb 16, 2024

create_pr=False has been passed so PR is automatically merged.

This is a comment posted using the huggingface_hub library in the context of a multi-commit. Learn more about multi-commits in this guide.

nold changed pull request status to merged Feb 16, 2024

3b9f5a048b8d0f86ec23970620578307a627401bb4e8b05772875f106292dce581dd6ab8

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment