Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf QuantFactory/NovaSpark-GGUF:# Run inference directly in the terminal:
llama-cli -hf QuantFactory/NovaSpark-GGUF:Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf QuantFactory/NovaSpark-GGUF:# Run inference directly in the terminal:
./llama-cli -hf QuantFactory/NovaSpark-GGUF:Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf QuantFactory/NovaSpark-GGUF:# Run inference directly in the terminal:
./build/bin/llama-cli -hf QuantFactory/NovaSpark-GGUF:Use Docker
docker model run hf.co/QuantFactory/NovaSpark-GGUF:
QuantFactory/NovaSpark-GGUF
This is quantized version of Epiculous/NovaSpark created using llama.cpp
Original Model Card
Switching things up a bit since the last slew of models were all 12B, we now have NovaSpark! NovaSpark is an 8B model trained on GrimJim's abliterated version of arcee's SuperNova-lite. The hope is abliteration will remove some of the inherant refusals and censorship of the original model, however I noticed that finetuning on GrimJim's model undid some of the abliteration, therefore more than likely abiliteration will have to be reapplied to the resulting model to reinforce it.
Quants!
Prompting
This model is trained on llama instruct template, the prompting structure goes a little something like this:
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
Context and Instruct
This model is trained on llama-instruct, please use that Context and Instruct template.
Current Top Sampler Settings
Smooth Creativity: Credit to Juelsman for researching this one!
Variant Chimera: Credit to Numbra!
Spicy_Temp
Violet_Twilight-Nitral-Special
- Downloads last month
- 85
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
Install from brew
# Start a local OpenAI-compatible server with a web UI: llama-server -hf QuantFactory/NovaSpark-GGUF:# Run inference directly in the terminal: llama-cli -hf QuantFactory/NovaSpark-GGUF: