Instructions to use nold/Prima-Pastacles-7b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nold/Prima-Pastacles-7b-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("nold/Prima-Pastacles-7b-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use nold/Prima-Pastacles-7b-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf nold/Prima-Pastacles-7b-GGUF:Q4_K_M

Use Docker

docker model run hf.co/nold/Prima-Pastacles-7b-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use nold/Prima-Pastacles-7b-GGUF with Ollama:
```
ollama run hf.co/nold/Prima-Pastacles-7b-GGUF:Q4_K_M
```

Unsloth Studio

How to use nold/Prima-Pastacles-7b-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nold/Prima-Pastacles-7b-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nold/Prima-Pastacles-7b-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for nold/Prima-Pastacles-7b-GGUF to start chatting

Docker Model Runner
How to use nold/Prima-Pastacles-7b-GGUF with Docker Model Runner:
```
docker model run hf.co/nold/Prima-Pastacles-7b-GGUF:Q4_K_M
```

Lemonade

How to use nold/Prima-Pastacles-7b-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull nold/Prima-Pastacles-7b-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Prima-Pastacles-7b-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

Upload folder using huggingface_hub

by nold - opened Feb 16, 2024

base: refs/heads/main

←

from: refs/pr/3

Discussion Files changed

-7

Files changed (6) hide show

Prima-Pastacles-7b_Q2_K.gguf +1 -1
Prima-Pastacles-7b_Q4_K_M.gguf +1 -1
Prima-Pastacles-7b_Q5_K_M.gguf +1 -1
Prima-Pastacles-7b_Q6_K.gguf +1 -1
Prima-Pastacles-7b_Q8_0.gguf +1 -1
README.md +1 -2

Prima-Pastacles-7b_Q2_K.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a135305bbc41153b01bdeba36ba9940a68ab23806802aadf11ddc1136df57f66
 size 2719241856

 version https://git-lfs.github.com/spec/v1
+oid sha256:fd59ebbfcefbb7fe0b08d155236ac3736cbba3fd62757d28008632504e8ee071
 size 2719241856

Prima-Pastacles-7b_Q4_K_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7d0ce480c5171372494318d5dd071158f84d9ce926bf19f13546b6fe122a218a
 size 4368438912

 version https://git-lfs.github.com/spec/v1
+oid sha256:d8667f9ef5b066083ac6afcd0bda8318ae58fb159cd89fff9a0d617075c61abb
 size 4368438912

Prima-Pastacles-7b_Q5_K_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e6e2225d5767403c7f7776c84dfc58f97ae75fa900887bc3c86d8d872de7752e
 size 5131409024

 version https://git-lfs.github.com/spec/v1
+oid sha256:56970ec1002cd838bc00e93c2d5bf9fb7de2249f653038bb1cb522d88b95c2af
 size 5131409024

Prima-Pastacles-7b_Q6_K.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e34bb7d35748b40a06042983542570650777ded04e8ba5079a5902f325dbbe4a
 size 5942064768

 version https://git-lfs.github.com/spec/v1
+oid sha256:14be3b8682ab4dfd02d4df42379a8403f897705ab19b4c3296293ccb8fa9ee4c
 size 5942064768

Prima-Pastacles-7b_Q8_0.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ef6b6c60ef474a907eda4df1171bc56e06ad28853d33fb248a17307a0251ecf3
 size 7695857280

 version https://git-lfs.github.com/spec/v1
+oid sha256:c58343e679a75550cdb9a82829e1f446b743f60703fe6638c2a3f4bc523f0731
 size 7695857280

README.md CHANGED Viewed

@@ -6,7 +6,7 @@ library_name: transformers
 tags:
 - mergekit
 - merge
 ---
 ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/642265bc01c62c1e4102dc36/Gs09MsXrzyJdDx7IrnkDa.jpeg)
 Quants Thanks to @Nold and @Bartowski:
@@ -43,7 +43,6 @@ parameters:
 dtype: bfloat16
 ```
 ***
 Quantization of Model [Test157t/Prima-Pastacles-7b](https://huggingface.co/Test157t/Prima-Pastacles-7b).

 tags:
 - mergekit
 - merge
+license: other
 ---
 ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/642265bc01c62c1e4102dc36/Gs09MsXrzyJdDx7IrnkDa.jpeg)
 Quants Thanks to @Nold and @Bartowski:
 dtype: bfloat16
 ```
 ***
 Quantization of Model [Test157t/Prima-Pastacles-7b](https://huggingface.co/Test157t/Prima-Pastacles-7b).