Hugging Face's logo

athirdpath
/
Hestia-20b-GGUF

GGUF
Model card Files
xet
 Community
How to use from
llama.cpp
Install (macOS, Linux) 
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf athirdpath/Hestia-20b-GGUF:
# Run inference directly in the terminal:
llama cli -hf athirdpath/Hestia-20b-GGUF:
Install from WinGet (Windows) 
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf athirdpath/Hestia-20b-GGUF:
# Run inference directly in the terminal:
llama cli -hf athirdpath/Hestia-20b-GGUF:
Use pre-built binary 
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf athirdpath/Hestia-20b-GGUF:
# Run inference directly in the terminal:
./llama-cli -hf athirdpath/Hestia-20b-GGUF:
Build from source code 
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf athirdpath/Hestia-20b-GGUF:
# Run inference directly in the terminal:
./build/bin/llama-cli -hf athirdpath/Hestia-20b-GGUF:
Use Docker 
docker model run hf.co/athirdpath/Hestia-20b-GGUF:
Quick Links

This repo contains several GGUF quants of Hestia-20b.

This is a task_arithmetic merge of Harmonia (my 20b faux base model) with Noromaid and my LORA-glued Nethena. Solidly outperforms Harmonia.

merge_method: task_arithmetic

base_model: athirdpath/Harmonia-20b

models:

  • model: athirdpath/Harmonia-20b

  • model: NeverSleep/Noromaid-20b-v0.1.1

    • parameters: weight: 0.25

  • model: athirdpath/Nethena-20b-Glued

    • parameters: weight: 0.2

dtype: float16

Thanks to Undi95 for pioneering the 20B recipe, and for most of the models involved.

Downloads last month
10
GGUF
Model size
20B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

4-bit

5-bit

8-bit

View +1 variant
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including athirdpath/Hestia-20b-GGUF