| # ✨ Sparkfile |
|
|
| A hobbyist build of mozilla-ai/llamafile for DGX Spark. |
|
|
| `v0.10.0` |
|
|
| both CUDA and Vulkan backends packaged. |
|
|
| after download validate backends |
|
|
| ```sh |
| chmod +x ./sparkfile |
| ./sparkfile --gpu nvidia --cli -m your_local_model.gguf --list-devices |
| ./sparkfile --gpu vulkan --cli -m your_local_model.gguf --list-devices |
| ``` |
|
|
| *should see both CUDA and Vulkan backends being reported. |
| |
| |
| basic usage |
| |
| ```sh |
| ./sparkfile --chat -m your_local_model.gguf |
| ``` |
| |
| ```sh |
| ./sparkfile --gpu nvidia --chat -m your_local_model.gguf |
| ``` |
| |
| ```sh |
| ./sparkfile --gpu vulkan --chat -m your_local_model.gguf |
| ``` |
| |
| ```sh |
| ./sparkfile --server ... <llama-server args> |
| ``` |
| |
| for certain reason you are not launching as an executable itself, e.g. wsl2 |
| |
| ```sh |
| sh ./sparkfile <args> |
| ``` |
| |
| Expected Platforms and OSs |
| CPU: ANY if supported by llamafile |
| GPU: |
| - aarch64_cpu_linux: DGX Spark (GB10), Jetson Thor |
| - x86-64_cpu_linux: (needs a seperate `.so` or recompile) |
| - x86-64_cpu_win: (needs a seperate `.dll` or recompile) |
| |
| --- |
| license: apache-2.0 |
| --- |
| |
