Instructions to use Acvarius-AI/GhostLlama with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use Acvarius-AI/GhostLlama with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="Acvarius-AI/GhostLlama", filename="ghostllama.Q4_K_M.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use Acvarius-AI/GhostLlama with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Acvarius-AI/GhostLlama:Q4_K_M # Run inference directly in the terminal: llama-cli -hf Acvarius-AI/GhostLlama:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Acvarius-AI/GhostLlama:Q4_K_M # Run inference directly in the terminal: llama-cli -hf Acvarius-AI/GhostLlama:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Acvarius-AI/GhostLlama:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf Acvarius-AI/GhostLlama:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Acvarius-AI/GhostLlama:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf Acvarius-AI/GhostLlama:Q4_K_M
Use Docker
docker model run hf.co/Acvarius-AI/GhostLlama:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use Acvarius-AI/GhostLlama with Ollama:
ollama run hf.co/Acvarius-AI/GhostLlama:Q4_K_M
- Unsloth Studio new
How to use Acvarius-AI/GhostLlama with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Acvarius-AI/GhostLlama to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Acvarius-AI/GhostLlama to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for Acvarius-AI/GhostLlama to start chatting
- Docker Model Runner
How to use Acvarius-AI/GhostLlama with Docker Model Runner:
docker model run hf.co/Acvarius-AI/GhostLlama:Q4_K_M
- Lemonade
How to use Acvarius-AI/GhostLlama with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Acvarius-AI/GhostLlama:Q4_K_M
Run and chat with the model
lemonade run user.GhostLlama-Q4_K_M
List all available models
lemonade list
| # Colors | |
| PURPLE='\033[0;35m' | |
| TEAL='\033[0;36m' | |
| NC='\033[0m' | |
| echo ' ... ...' | |
| echo ' -*%##: .=#=.' | |
| echo ' *# ##=::=###:' | |
| echo ' -%# ## ## ###:' | |
| echo ' .=# ########## .' | |
| echo ' :# ## ##### #+' | |
| echo ' .= #############+.' | |
| echo ' .## ##### ### #+###.' | |
| echo ' .# ####### ###.' | |
| echo ' .# #############=.' | |
| echo ' :## ########%*' | |
| echo ' :# ######## ##*' | |
| echo ' ...........:=########### #:' | |
| echo ' .....:+ ####################### #.' | |
| echo ' .:= # ########################### #:' | |
| echo ' .=# # ########################### #=.' | |
| echo ' ...:+ # ############################# ##.' | |
| echo ' :=# # # ############################# ##.' | |
| echo ' ....:=# ############################ ##.' | |
| echo ' :# ########################## ##=.' | |
| echo ' .=# ########################## #-' | |
| echo ' *# ########################### #' | |
| echo ' * ##### # ########## #=' | |
| echo ' =# #### # ######## #' | |
| echo ' * ##### # ###### # #+.' | |
| echo ' * ###### #%- =# ####### @' | |
| echo ' .:= ##===###=- .= ###### #=-' | |
| echo ' .=# ##=. :%+: .=# #===# %*' | |
| echo ' .+:... .##=:: ....' | |
| echo ' ...' | |
| # Header | |
| echo -e "${PURPLE}" | |
| echo "ββββββββββββββββββββββββββββββββββββββββ" | |
| echo "β π» GhostLlama INSTALLER v1.1 β" | |
| echo "β π How to Sacrifice your Llama β" | |
| echo "ββββββββββββββββββββββββββββββββββββββββ" | |
| echo -e "${NC}" | |
| # Updates | |
| echo -e "${TEAL} v1.1 Added ghostchat terminal command and Updated Telegram Integration.${NC}" | |
| # Create directories | |
| echo -e "${TEAL}[π£] Preparing the altar...${NC}" | |
| mkdir -p ~/ghostllama/models | |
| mkdir -p ~/ghostllama/config | |
| # Download model | |
| echo -e "${TEAL}[π£] Summoning the spirit from HuggingFace...${NC}" | |
| cd ~/ghostllama/models | |
| wget -O ghostllama.Q4_K_M.gguf https://huggingface.co/Acvarius-AI/GhostLlama/resolve/main/ghostllama.Q4_K_M.gguf | |
| # Add terminal chat command | |
| echo "alias ghostchat='~/llama.cpp/build/bin/llama-cli -m ~/ghostllama/models/ghostllama.Q4_K_M.gguf -cnv'" >> ~/.bashrc | |
| echo -e "${TEAL}[π£] Terminal chat ready! Type 'ghostchat' to talk to GhostLlama.${NC}" | |
| # Telegram setup | |
| echo -e "${TEAL}[π£] Connect to Telegram? (y/n)${NC}" | |
| read -r telegram | |
| if [ "$telegram" = "y" ]; then | |
| echo -e "${TEAL}[π£] Enter your Telegram Bot Token:${NC}" | |
| read -r token | |
| echo "TELEGRAM_BOT_TOKEN=$token" > ~/ghostllama/config/telegram.conf | |
| # Create basic bot script | |
| cat > ~/ghostllama/telegram-bot.py << 'EOF' | |
| import requests | |
| import subprocess | |
| import time | |
| import os | |
| TOKEN = open(os.path.expanduser('~/ghostllama/config/telegram.conf')).read().split('=')[1].strip() | |
| URL = f"https://api.telegram.org/bot{TOKEN}" | |
| def get_updates(offset=None): | |
| url = f"{URL}/getUpdates" | |
| params = {"timeout": 100, "offset": offset} | |
| response = requests.get(url, params=params) | |
| return response.json() | |
| def send_message(chat_id, text): | |
| url = f"{URL}/sendMessage" | |
| params = {"chat_id": chat_id, "text": text} | |
| requests.post(url, params=params) | |
| def call_model(prompt): | |
| # Expand paths so they work from ANY directory | |
| llama_cli = os.path.expanduser("~/llama.cpp/build/bin/llama-cli") | |
| model_path = os.path.expanduser("~/ghostllama/models/ghostllama.Q4_K_M.gguf") | |
| result = subprocess.run( | |
| [llama_cli, "-m", model_path, "-p", prompt, "-n", "50"], | |
| capture_output=True, | |
| text=True | |
| ) | |
| return result.stdout | |
| last_update = 0 | |
| while True: | |
| updates = get_updates(last_update + 1) | |
| if "result" in updates: | |
| for update in updates["result"]: | |
| last_update = update["update_id"] | |
| if "message" in update and "text" in update["message"]: | |
| chat_id = update["message"]["chat"]["id"] | |
| user_text = update["message"]["text"] | |
| response = call_model(user_text) | |
| send_message(chat_id, response) | |
| time.sleep(1) | |
| EOF | |
| echo -e "${TEAL}[π£] Telegram bot installed! Run with: python ~/ghostllama/telegram-bot.py${NC}" | |
| fi | |
| # Done | |
| echo -e "${PURPLE}" | |
| echo "ββββββββββββββββββββββββββββββββββββββββ" | |
| echo "β β¨ SACRIFICE COMPLETE β¨ β" | |
| echo "β Run 'cd ~/ghostllama' to begin β" | |
| echo "ββββββββββββββββββββββββββββββββββββββββ" | |
| echo -e "${NC}" | |