Instructions to use tensorblock/NuExtract-tiny-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use tensorblock/NuExtract-tiny-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf tensorblock/NuExtract-tiny-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf tensorblock/NuExtract-tiny-GGUF:Q2_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf tensorblock/NuExtract-tiny-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf tensorblock/NuExtract-tiny-GGUF:Q2_K
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf tensorblock/NuExtract-tiny-GGUF:Q2_K # Run inference directly in the terminal: ./build/bin/llama cli -hf tensorblock/NuExtract-tiny-GGUF:Q2_K
Use Docker
docker model run hf.co/tensorblock/NuExtract-tiny-GGUF:Q2_K
- Llama new
- LM Studio
- Jan
- Ollama
How to use tensorblock/NuExtract-tiny-GGUF with Ollama:
ollama run hf.co/tensorblock/NuExtract-tiny-GGUF:Q2_K
- Unsloth Desktop
- Docker Model Runner
How to use tensorblock/NuExtract-tiny-GGUF with Docker Model Runner:
docker model run hf.co/tensorblock/NuExtract-tiny-GGUF:Q2_K
- Lemonade
How to use tensorblock/NuExtract-tiny-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull tensorblock/NuExtract-tiny-GGUF:Q2_K
Run and chat with the model
lemonade run user.NuExtract-tiny-GGUF-Q2_K
List all available models
lemonade list
- Atomic Chat
Keep Q2_K/Q3_K_M gguf only
Browse files- NuExtract-tiny-Q3_K_L.gguf +0 -3
- NuExtract-tiny-Q3_K_S.gguf +0 -3
- NuExtract-tiny-Q4_0.gguf +0 -3
- NuExtract-tiny-Q4_K_M.gguf +0 -3
- NuExtract-tiny-Q4_K_S.gguf +0 -3
- NuExtract-tiny-Q5_0.gguf +0 -3
- NuExtract-tiny-Q5_K_M.gguf +0 -3
- NuExtract-tiny-Q5_K_S.gguf +0 -3
- NuExtract-tiny-Q6_K.gguf +0 -3
- NuExtract-tiny-Q8_0.gguf +0 -3
NuExtract-tiny-Q3_K_L.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:e52ab352981e6bcc22c71841326a9152626a508265ecb560cb25f91dd8efa73e
|
| 3 |
-
size 297350560
|
|
|
|
|
|
|
|
|
|
|
|
NuExtract-tiny-Q3_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:965765ce6981b78f5d5af5c28ac3fc0a41c0a0e7a6856ac61a90890c79d5d8dd
|
| 3 |
-
size 266532256
|
|
|
|
|
|
|
|
|
|
|
|
NuExtract-tiny-Q4_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:4a3c5ce7adae8d436593eb07c0eb4fe7b1ccc22a29824807502ed7dde3e5de17
|
| 3 |
-
size 307475872
|
|
|
|
|
|
|
|
|
|
|
|
NuExtract-tiny-Q4_K_M.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:04db8f863a4348c469a38c5406f28af7c6938989387bf86bf0b31095edc6fb6d
|
| 3 |
-
size 319640992
|
|
|
|
|
|
|
|
|
|
|
|
NuExtract-tiny-Q4_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:caa091ad503a9662ade85d7152ea118d4ea18928404c9aa08a09ddfc1080eb68
|
| 3 |
-
size 309081504
|
|
|
|
|
|
|
|
|
|
|
|
NuExtract-tiny-Q5_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:5a30ead6133c1b503a757fbf06d4cbed837a9b6ef8ad7eb45b7c6137a0c76fa8
|
| 3 |
-
size 346011040
|
|
|
|
|
|
|
|
|
|
|
|
NuExtract-tiny-Q5_K_M.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:b8f02bab63c9e078414e376fbd347cda82efe5a52e27658c30fd8b6e00f4f64d
|
| 3 |
-
size 352277920
|
|
|
|
|
|
|
|
|
|
|
|
NuExtract-tiny-Q5_K_S.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:162785964bc450b95593c40f994872186e4c798f824c5997545041f2604f62f2
|
| 3 |
-
size 346011040
|
|
|
|
|
|
|
|
|
|
|
|
NuExtract-tiny-Q6_K.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:7fc83f23f488c27ec9910bae4d6317667e4627ab536d95567392949f16920f36
|
| 3 |
-
size 386954656
|
|
|
|
|
|
|
|
|
|
|
|
NuExtract-tiny-Q8_0.gguf
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:bfe5489b3d753481910a7ab54712313d8d4580c5f4bb9a2276ba307a6fc56b22
|
| 3 |
-
size 499296672
|
|
|
|
|
|
|
|
|
|
|
|