How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf athirdpath/Harmonia-20b-GGUF:# Run inference directly in the terminal:
llama cli -hf athirdpath/Harmonia-20b-GGUF:Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf athirdpath/Harmonia-20b-GGUF:# Run inference directly in the terminal:
./llama-cli -hf athirdpath/Harmonia-20b-GGUF:Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf athirdpath/Harmonia-20b-GGUF:# Run inference directly in the terminal:
./build/bin/llama-cli -hf athirdpath/Harmonia-20b-GGUF:Use Docker
docker model run hf.co/athirdpath/Harmonia-20b-GGUF:Quick Links
Description
These are fp16, q_8_0, q5_k_m, and q4_k_m GGUF quants of Harmonia-20B.
Harmonia-20B is a unified 20B model crafted via a multi-step SLERP merge of eight 20B models. The aim was to develop a versatile "base model" for TaskArithmetic in this size class.
Merging Process:
Models:
- model: Undi95/Emerhyst-20B
- model: Undi95/MXLewd-L2-20B
- model: Undi95/Lewd-Sydney-20B
- model: athirdpath/Nethena-20b-Glued
- model: tavtav/Rose-20B
- model: Undi95/PsyMedRP-v1-20B
- model: NeverSleep/Noromaid-20b-v0.1.1
- model: Undi95/U-Amethyst-20B
Concept:
The idea behind this process was to blend the unique attributes of each model while minimizing individual quirks. This approach has also shown promising results as a standalone RP model, providing a combination of high-quality writing and situational problem-solving/awareness.
Prompt template: Alpaca
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{prompt}
### Response:
Thanks to Undi95 for pioneering the 20B recipe, and for most of the models involved.
- Downloads last month
- 10
Hardware compatibility
Log In to add your hardware
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf athirdpath/Harmonia-20b-GGUF:# Run inference directly in the terminal: llama cli -hf athirdpath/Harmonia-20b-GGUF: