Instructions to use Slaxikov/VokiLLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use Slaxikov/VokiLLM with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="Slaxikov/VokiLLM", filename="imatrix-vokillm.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use Slaxikov/VokiLLM with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Slaxikov/VokiLLM:Q4_K_M # Run inference directly in the terminal: llama-cli -hf Slaxikov/VokiLLM:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Slaxikov/VokiLLM:Q4_K_M # Run inference directly in the terminal: llama-cli -hf Slaxikov/VokiLLM:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Slaxikov/VokiLLM:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf Slaxikov/VokiLLM:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Slaxikov/VokiLLM:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf Slaxikov/VokiLLM:Q4_K_M
Use Docker
docker model run hf.co/Slaxikov/VokiLLM:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use Slaxikov/VokiLLM with Ollama:
ollama run hf.co/Slaxikov/VokiLLM:Q4_K_M
- Unsloth Studio new
How to use Slaxikov/VokiLLM with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Slaxikov/VokiLLM to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Slaxikov/VokiLLM to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for Slaxikov/VokiLLM to start chatting
- Pi new
How to use Slaxikov/VokiLLM with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf Slaxikov/VokiLLM:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Slaxikov/VokiLLM:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use Slaxikov/VokiLLM with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf Slaxikov/VokiLLM:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Slaxikov/VokiLLM:Q4_K_M
Run Hermes
hermes
- Docker Model Runner
How to use Slaxikov/VokiLLM with Docker Model Runner:
docker model run hf.co/Slaxikov/VokiLLM:Q4_K_M
- Lemonade
How to use Slaxikov/VokiLLM with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Slaxikov/VokiLLM:Q4_K_M
Run and chat with the model
lemonade run user.VokiLLM-Q4_K_M
List all available models
lemonade list
Upload 23 files
Browse files- .gitattributes +23 -0
- vokillm-0.5b-instruct-fp16.gguf +3 -0
- vokillm-0.5b-instruct-iq1_s.gguf +3 -0
- vokillm-0.5b-instruct-iq2_s.gguf +3 -0
- vokillm-0.5b-instruct-iq2_xs.gguf +3 -0
- vokillm-0.5b-instruct-iq2_xxs.gguf +3 -0
- vokillm-0.5b-instruct-iq3_s.gguf +3 -0
- vokillm-0.5b-instruct-iq3_xxs.gguf +3 -0
- vokillm-0.5b-instruct-iq4_nl.gguf +3 -0
- vokillm-0.5b-instruct-iq4_xs.gguf +3 -0
- vokillm-0.5b-instruct-q2_k.gguf +3 -0
- vokillm-0.5b-instruct-q3_k_l.gguf +3 -0
- vokillm-0.5b-instruct-q3_k_m.gguf +3 -0
- vokillm-0.5b-instruct-q3_k_s.gguf +3 -0
- vokillm-0.5b-instruct-q4_0.gguf +3 -0
- vokillm-0.5b-instruct-q4_1.gguf +3 -0
- vokillm-0.5b-instruct-q4_k_m.gguf +3 -0
- vokillm-0.5b-instruct-q4_k_s.gguf +3 -0
- vokillm-0.5b-instruct-q5_0.gguf +3 -0
- vokillm-0.5b-instruct-q5_1.gguf +3 -0
- vokillm-0.5b-instruct-q5_k_m.gguf +3 -0
- vokillm-0.5b-instruct-q5_k_s.gguf +3 -0
- vokillm-0.5b-instruct-q6_k.gguf +3 -0
- vokillm-0.5b-instruct-q8_0.gguf +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,26 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
vokillm-0.5b-instruct-fp16.gguf filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
vokillm-0.5b-instruct-iq1_s.gguf filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
vokillm-0.5b-instruct-iq2_s.gguf filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
vokillm-0.5b-instruct-iq2_xs.gguf filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
vokillm-0.5b-instruct-iq2_xxs.gguf filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
vokillm-0.5b-instruct-iq3_s.gguf filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
vokillm-0.5b-instruct-iq3_xxs.gguf filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
vokillm-0.5b-instruct-iq4_nl.gguf filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
vokillm-0.5b-instruct-iq4_xs.gguf filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
vokillm-0.5b-instruct-q2_k.gguf filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
vokillm-0.5b-instruct-q3_k_l.gguf filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
vokillm-0.5b-instruct-q3_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
vokillm-0.5b-instruct-q3_k_s.gguf filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
vokillm-0.5b-instruct-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 50 |
+
vokillm-0.5b-instruct-q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
| 51 |
+
vokillm-0.5b-instruct-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
| 52 |
+
vokillm-0.5b-instruct-q4_k_s.gguf filter=lfs diff=lfs merge=lfs -text
|
| 53 |
+
vokillm-0.5b-instruct-q5_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 54 |
+
vokillm-0.5b-instruct-q5_1.gguf filter=lfs diff=lfs merge=lfs -text
|
| 55 |
+
vokillm-0.5b-instruct-q5_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
| 56 |
+
vokillm-0.5b-instruct-q5_k_s.gguf filter=lfs diff=lfs merge=lfs -text
|
| 57 |
+
vokillm-0.5b-instruct-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
|
| 58 |
+
vokillm-0.5b-instruct-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
vokillm-0.5b-instruct-fp16.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6729162be5f20da0f1b4887f104fdc9c1aa9187da44039e6d9118ffc6633d680
|
| 3 |
+
size 1266425856
|
vokillm-0.5b-instruct-iq1_s.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:59321088c184b2337d945dd341425583eb30cbd42470d6e6bf1f4b2c6345a99e
|
| 3 |
+
size 392404960
|
vokillm-0.5b-instruct-iq2_s.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bbd80b06373f2fa0428b02e9393bd0160ab8c3b975757d7b38889723e256fd16
|
| 3 |
+
size 402312928
|
vokillm-0.5b-instruct-iq2_xs.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a6e2e3c4f7b64c7275105a1293f64298dd064bfb6b4d31bcbfa0def8143300a2
|
| 3 |
+
size 400985056
|
vokillm-0.5b-instruct-iq2_xxs.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:30aca8d2d7adae03f6e3040f39b7fae4e92aebbfd752c04b3e38e4ee94b11be1
|
| 3 |
+
size 398125024
|
vokillm-0.5b-instruct-iq3_s.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4c31d28e5e449a0278cac437592ac8914857c06dd2b0ac3f4e2684d16463c5f8
|
| 3 |
+
size 415182848
|
vokillm-0.5b-instruct-iq3_xxs.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3dd7b9c3fbc7eed0283e290a3dddb99a169d21c419058aedd81615b2db22ac67
|
| 3 |
+
size 416919296
|
vokillm-0.5b-instruct-iq4_nl.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7387949476ad6d9f1da773cb24a8ad7c6ca2ab483521c24d2d5433a437bdbc72
|
| 3 |
+
size 430880768
|
vokillm-0.5b-instruct-iq4_xs.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3e9699e889ae6e8878134956a9cda5f6b4cef5e71a63ff47b5cdd4261ad21d71
|
| 3 |
+
size 428020736
|
vokillm-0.5b-instruct-q2_k.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b78b4625174120d6ad418a0ddc8ce18d246902e9e8aa67fa8d6d43b148f6b0f5
|
| 3 |
+
size 415182848
|
vokillm-0.5b-instruct-q3_k_l.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9322f7821a31776a557530ad54f381e8486b4d223efcf18ca668efb7ca9a5fd8
|
| 3 |
+
size 445933568
|
vokillm-0.5b-instruct-q3_k_m.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f37456253ea8764523969d1451602917f7785717ea16be33f0c0ea219a09773c
|
| 3 |
+
size 432041984
|
vokillm-0.5b-instruct-q3_k_s.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ff32e267a8bf5617fdfc2e201bdb8c16fdb2126797e112e2ca82dccb6a7aa64e
|
| 3 |
+
size 414838784
|
vokillm-0.5b-instruct-q4_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:04a36451d82d7b6a28c5335c2afa56a26d738afb887ff121972b76e2a12ceecf
|
| 3 |
+
size 428730368
|
vokillm-0.5b-instruct-q4_1.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:952737aa5c571144118808aa13a641b3d1ca6f162deada6f9e7bbb630de8bc75
|
| 3 |
+
size 459602944
|
vokillm-0.5b-instruct-q4_k_m.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6cf673f9b846e0b56cd053cf238acd002eba7bb6d6106c4a54a96520942a1fcd
|
| 3 |
+
size 491400192
|
vokillm-0.5b-instruct-q4_k_s.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b8da56dfdf3538e0bb79b6f476a59a92d35fe3178b439f64f8fbdd3ea93872eb
|
| 3 |
+
size 479064064
|
vokillm-0.5b-instruct-q5_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3f13979ca2c8c0b3e20ab7b6a2175abb643a74b427607e2a5d91515cd5cb9e8c
|
| 3 |
+
size 490475520
|
vokillm-0.5b-instruct-q5_1.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ed6be4f981baca78b0601de8a6b59cd8718d134b692d852defcb50c6bda203d6
|
| 3 |
+
size 521348096
|
vokillm-0.5b-instruct-q5_k_m.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6c6d5ea2fad988490b9d44d23bc9587eeb62ecda2e034e69ebc18a18a8313519
|
| 3 |
+
size 522186752
|
vokillm-0.5b-instruct-q5_k_s.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8ae5152695b46e604dbff7497fd5a9e0b28ca2f622c5492769ef5cb7c8e085a8
|
| 3 |
+
size 514810880
|
vokillm-0.5b-instruct-q6_k.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7ed45829df611e6008c8e8b93b72472af7f055fce47aac886d0096cf6d6508c3
|
| 3 |
+
size 650379296
|
vokillm-0.5b-instruct-q8_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6c83f44e1bdd38b5104b4d98f4f91c6ad676bcf345fe7a40ce06494aeefd24ee
|
| 3 |
+
size 675710976
|