dattazigzag/rubin-base-f16

Fine-Tuning Gemma 2 2 billion parameter base model with Rick Rubin’s The Creative Act

More information can be found in the Github

On mac

Use all the GPU available ( --n-gpu-layers 99 or -ngl 99)

llama-server -m rubin-base-f16.gguf --host 0.0.0.0 --port 8080 --n-gpu-layers 99 --ctx-size 2048 -n 60 --temp 0.7

On pi5 (16GB)

Even though 1 GPU is available, llama.cpp works better on CPU ( --n-gpu-layers 0 )

llama-server -hf dattazigzag/rubin-base-f16:F16 --host 0.0.0.0 --port 8080 --ctx-size 2048 -n 60 --temp 0.7