Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf QuantFactory/Llama-Spark-GGUF:# Run inference directly in the terminal:
llama-cli -hf QuantFactory/Llama-Spark-GGUF:Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf QuantFactory/Llama-Spark-GGUF:# Run inference directly in the terminal:
./llama-cli -hf QuantFactory/Llama-Spark-GGUF:Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf QuantFactory/Llama-Spark-GGUF:# Run inference directly in the terminal:
./build/bin/llama-cli -hf QuantFactory/Llama-Spark-GGUF:Use Docker
docker model run hf.co/QuantFactory/Llama-Spark-GGUF:
QuantFactory/Llama-Spark-GGUF
This is quantized version of arcee-ai/Llama-Spark created using llama.cpp
Original Model Card
Llama-Spark is a powerful conversational AI model developed by Arcee.ai. It's built on the foundation of Llama-3.1-8B and merges the power of our Tome Dataset with Llama-3.1-8B-Instruct, resulting in a remarkable conversationalist that punches well above its 8B parameter weight class.
GGUFs available here
Model Description
Llama-Spark is our commitment to consistently delivering the best-performing conversational AI in the 6-9B parameter range. As new base models become available, we'll continue to update and improve Spark to maintain its leadership position.
This model is a successor to our original Arcee-Spark, incorporating advancements and learnings from our ongoing research and development.
Intended Uses
Llama-Spark is intended for use in conversational AI applications, such as chatbots, virtual assistants, and dialogue systems. It excels at engaging in natural and informative conversations.
Training Information
Llama-Spark is built upon the Llama-3.1-8B base model, fine-tuned using of the Tome Dataset and merged with Llama-3.1-8B-Instruct.
Evaluation Results
Please note that these scores are consistantly higher than the OpenLLM leaderboard, and should be compared to their relative performance increase not weighed against the leaderboard.
Acknowledgements
We extend our deepest gratitude to PrimeIntellect for being our compute sponsor for this project.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 24.90 |
| IFEval (0-Shot) | 79.11 |
| BBH (3-Shot) | 29.77 |
| MATH Lvl 5 (4-Shot) | 1.06 |
| GPQA (0-shot) | 6.60 |
| MuSR (0-shot) | 2.62 |
| MMLU-PRO (5-shot) | 30.23 |
- Downloads last month
- -
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
Evaluation results
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard79.110
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard29.770
- exact match on MATH Lvl 5 (4-Shot)Open LLM Leaderboard1.060
- acc_norm on GPQA (0-shot)Open LLM Leaderboard6.600
- acc_norm on MuSR (0-shot)Open LLM Leaderboard2.620
- accuracy on MMLU-PRO (5-shot)test set Open LLM Leaderboard30.230
Install from brew
# Start a local OpenAI-compatible server with a web UI: llama-server -hf QuantFactory/Llama-Spark-GGUF:# Run inference directly in the terminal: llama-cli -hf QuantFactory/Llama-Spark-GGUF: