How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Undi95/MG-FinalMix-72B-GGUF:Q3_K_M# Run inference directly in the terminal:
llama cli -hf Undi95/MG-FinalMix-72B-GGUF:Q3_K_MUse pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Undi95/MG-FinalMix-72B-GGUF:Q3_K_M# Run inference directly in the terminal:
./llama-cli -hf Undi95/MG-FinalMix-72B-GGUF:Q3_K_MBuild from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Undi95/MG-FinalMix-72B-GGUF:Q3_K_M# Run inference directly in the terminal:
./build/bin/llama-cli -hf Undi95/MG-FinalMix-72B-GGUF:Q3_K_MUse Docker
docker model run hf.co/Undi95/MG-FinalMix-72B-GGUF:Q3_K_MQuick Links
WIP of retouched alpindale/magnum-72b-v1 but I will not use "Magnum" in the name. Call it FinalMix!
Found some issues, trying to fix them for my own usage and adding more RP data with merging.
You can do your own quantized files with the imatrix.dat file done with "wiki.train.raw".
Credits to Alpin and the gang for magnum-72b-v1, and Ikari for his datasets.
Prompt template ChatML
<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
{output}<|im_end|>
- Downloads last month
- 7
Hardware compatibility
Log In to add your hardware
3-bit
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf Undi95/MG-FinalMix-72B-GGUF:Q3_K_M# Run inference directly in the terminal: llama cli -hf Undi95/MG-FinalMix-72B-GGUF:Q3_K_M