Instructions to use Acvarius-AI/GhostLlama with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use Acvarius-AI/GhostLlama with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="Acvarius-AI/GhostLlama", filename="ghostllama.Q4_K_M.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use Acvarius-AI/GhostLlama with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Acvarius-AI/GhostLlama:Q4_K_M # Run inference directly in the terminal: llama-cli -hf Acvarius-AI/GhostLlama:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Acvarius-AI/GhostLlama:Q4_K_M # Run inference directly in the terminal: llama-cli -hf Acvarius-AI/GhostLlama:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Acvarius-AI/GhostLlama:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf Acvarius-AI/GhostLlama:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Acvarius-AI/GhostLlama:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf Acvarius-AI/GhostLlama:Q4_K_M
Use Docker
docker model run hf.co/Acvarius-AI/GhostLlama:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use Acvarius-AI/GhostLlama with Ollama:
ollama run hf.co/Acvarius-AI/GhostLlama:Q4_K_M
- Unsloth Studio new
How to use Acvarius-AI/GhostLlama with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Acvarius-AI/GhostLlama to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Acvarius-AI/GhostLlama to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for Acvarius-AI/GhostLlama to start chatting
- Docker Model Runner
How to use Acvarius-AI/GhostLlama with Docker Model Runner:
docker model run hf.co/Acvarius-AI/GhostLlama:Q4_K_M
- Lemonade
How to use Acvarius-AI/GhostLlama with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Acvarius-AI/GhostLlama:Q4_K_M
Run and chat with the model
lemonade run user.GhostLlama-Q4_K_M
List all available models
lemonade list
File size: 5,059 Bytes
2f3a379 ed0d8b4 2f3a379 ed0d8b4 2f3a379 c15783a 2f3a379 c15783a 2f3a379 c15783a 2f3a379 c15783a 2f3a379 c15783a 2f3a379 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 | #!/bin/bash
# Colors
PURPLE='\033[0;35m'
TEAL='\033[0;36m'
NC='\033[0m'
echo ' ... ...'
echo ' -*%##: .=#=.'
echo ' *# ##=::=###:'
echo ' -%# ## ## ###:'
echo ' .=# ########## .'
echo ' :# ## ##### #+'
echo ' .= #############+.'
echo ' .## ##### ### #+###.'
echo ' .# ####### ###.'
echo ' .# #############=.'
echo ' :## ########%*'
echo ' :# ######## ##*'
echo ' ...........:=########### #:'
echo ' .....:+ ####################### #.'
echo ' .:= # ########################### #:'
echo ' .=# # ########################### #=.'
echo ' ...:+ # ############################# ##.'
echo ' :=# # # ############################# ##.'
echo ' ....:=# ############################ ##.'
echo ' :# ########################## ##=.'
echo ' .=# ########################## #-'
echo ' *# ########################### #'
echo ' * ##### # ########## #='
echo ' =# #### # ######## #'
echo ' * ##### # ###### # #+.'
echo ' * ###### #%- =# ####### @'
echo ' .:= ##===###=- .= ###### #=-'
echo ' .=# ##=. :%+: .=# #===# %*'
echo ' .+:... .##=:: ....'
echo ' ...'
# Header
echo -e "${PURPLE}"
echo "ββββββββββββββββββββββββββββββββββββββββ"
echo "β π» GhostLlama INSTALLER v1.1 β"
echo "β π How to Sacrifice your Llama β"
echo "ββββββββββββββββββββββββββββββββββββββββ"
echo -e "${NC}"
# Updates
echo -e "${TEAL} v1.1 Added ghostchat terminal command and Updated Telegram Integration.${NC}"
# Create directories
echo -e "${TEAL}[π£] Preparing the altar...${NC}"
mkdir -p ~/ghostllama/models
mkdir -p ~/ghostllama/config
# Download model
echo -e "${TEAL}[π£] Summoning the spirit from HuggingFace...${NC}"
cd ~/ghostllama/models
wget -O ghostllama.Q4_K_M.gguf https://huggingface.co/Acvarius-AI/GhostLlama/resolve/main/ghostllama.Q4_K_M.gguf
# Add terminal chat command
echo "alias ghostchat='~/llama.cpp/build/bin/llama-cli -m ~/ghostllama/models/ghostllama.Q4_K_M.gguf -cnv'" >> ~/.bashrc
echo -e "${TEAL}[π£] Terminal chat ready! Type 'ghostchat' to talk to GhostLlama.${NC}"
# Telegram setup
echo -e "${TEAL}[π£] Connect to Telegram? (y/n)${NC}"
read -r telegram
if [ "$telegram" = "y" ]; then
echo -e "${TEAL}[π£] Enter your Telegram Bot Token:${NC}"
read -r token
echo "TELEGRAM_BOT_TOKEN=$token" > ~/ghostllama/config/telegram.conf
# Create basic bot script
cat > ~/ghostllama/telegram-bot.py << 'EOF'
import requests
import subprocess
import time
import os
TOKEN = open(os.path.expanduser('~/ghostllama/config/telegram.conf')).read().split('=')[1].strip()
URL = f"https://api.telegram.org/bot{TOKEN}"
def get_updates(offset=None):
url = f"{URL}/getUpdates"
params = {"timeout": 100, "offset": offset}
response = requests.get(url, params=params)
return response.json()
def send_message(chat_id, text):
url = f"{URL}/sendMessage"
params = {"chat_id": chat_id, "text": text}
requests.post(url, params=params)
def call_model(prompt):
# Expand paths so they work from ANY directory
llama_cli = os.path.expanduser("~/llama.cpp/build/bin/llama-cli")
model_path = os.path.expanduser("~/ghostllama/models/ghostllama.Q4_K_M.gguf")
result = subprocess.run(
[llama_cli, "-m", model_path, "-p", prompt, "-n", "50"],
capture_output=True,
text=True
)
return result.stdout
last_update = 0
while True:
updates = get_updates(last_update + 1)
if "result" in updates:
for update in updates["result"]:
last_update = update["update_id"]
if "message" in update and "text" in update["message"]:
chat_id = update["message"]["chat"]["id"]
user_text = update["message"]["text"]
response = call_model(user_text)
send_message(chat_id, response)
time.sleep(1)
EOF
echo -e "${TEAL}[π£] Telegram bot installed! Run with: python ~/ghostllama/telegram-bot.py${NC}"
fi
# Done
echo -e "${PURPLE}"
echo "ββββββββββββββββββββββββββββββββββββββββ"
echo "β β¨ SACRIFICE COMPLETE β¨ β"
echo "β Run 'cd ~/ghostllama' to begin β"
echo "ββββββββββββββββββββββββββββββββββββββββ"
echo -e "${NC}"
|