Instructions to use backpack-run/SmolLM2-135M-Instruct-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use backpack-run/SmolLM2-135M-Instruct-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M
Use Docker
docker model run hf.co/backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use backpack-run/SmolLM2-135M-Instruct-GGUF with Ollama:
ollama run hf.co/backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M
- Unsloth Studio
How to use backpack-run/SmolLM2-135M-Instruct-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for backpack-run/SmolLM2-135M-Instruct-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for backpack-run/SmolLM2-135M-Instruct-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for backpack-run/SmolLM2-135M-Instruct-GGUF to start chatting
- Docker Model Runner
How to use backpack-run/SmolLM2-135M-Instruct-GGUF with Docker Model Runner:
docker model run hf.co/backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M
- Lemonade
How to use backpack-run/SmolLM2-135M-Instruct-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull backpack-run/SmolLM2-135M-Instruct-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.SmolLM2-135M-Instruct-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Add files using upload-large-folder tool
Browse files- .gitattributes +3 -0
- README.md +86 -0
- SmolLM2-135M-Instruct-Q4_K_M.gguf +3 -0
- SmolLM2-135M-Instruct-Q5_K_M.gguf +3 -0
- SmolLM2-135M-Instruct-Q8_0.gguf +3 -0
- backpack-model.yaml +106 -0
- checksums.sha256 +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
SmolLM2-135M-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
SmolLM2-135M-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
SmolLM2-135M-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: HuggingFaceTB/SmolLM2-135M-Instruct
|
| 3 |
+
license: apache-2.0
|
| 4 |
+
library_name: gguf
|
| 5 |
+
tags:
|
| 6 |
+
- gguf
|
| 7 |
+
- llama.cpp
|
| 8 |
+
- backpack
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# SmolLM2-135M-Instruct — Backpack GGUF
|
| 12 |
+
|
| 13 |
+
🎒 **Backpack Verified**
|
| 14 |
+
|
| 15 |
+
GGUF quantizations of [HuggingFaceTB/SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct), tested for llama.cpp-compatible inference and packaged for Backpack.
|
| 16 |
+
|
| 17 |
+
## Model
|
| 18 |
+
|
| 19 |
+
| Property | Value |
|
| 20 |
+
| --- | --- |
|
| 21 |
+
| Original model | `HuggingFaceTB/SmolLM2-135M-Instruct` |
|
| 22 |
+
| Original publisher | HuggingFaceTB |
|
| 23 |
+
| Upstream revision | `12fd25f77366fa6b3b4b768ec3050bf629380bac` |
|
| 24 |
+
| Architecture | LlamaForCausalLM |
|
| 25 |
+
| Parameters | 134,515,008 |
|
| 26 |
+
| Context length | 8,192 |
|
| 27 |
+
| License | apache-2.0 |
|
| 28 |
+
|
| 29 |
+
## Available packages
|
| 30 |
+
|
| 31 |
+
| Quantization | Size | Approx. RAM | Recommended for |
|
| 32 |
+
| --- | ---: | ---: | --- |
|
| 33 |
+
| Q4_K_M | 100.6 MiB | 1.14 GB | Most users |
|
| 34 |
+
| Q5_K_M | 106.9 MiB | 1.15 GB | Higher quality |
|
| 35 |
+
| Q8_0 | 138.1 MiB | 1.2 GB | Plenty of memory |
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
Memory values are estimates, not guarantees. Runtime configuration and context length change actual use.
|
| 39 |
+
|
| 40 |
+
## Backpack recommendation
|
| 41 |
+
|
| 42 |
+
**Recommended: Q4_K_M.** It usually offers a practical quality, size, and speed balance for local inference.
|
| 43 |
+
|
| 44 |
+
## Run with llama.cpp
|
| 45 |
+
|
| 46 |
+
Using the llama.cpp revision recorded below:
|
| 47 |
+
|
| 48 |
+
```bash
|
| 49 |
+
llama-cli --model SmolLM2-135M-Instruct-Q4_K_M.gguf --conversation
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
## Run with Backpack
|
| 53 |
+
|
| 54 |
+
These artifacts and `backpack-model.yaml` are prepared for the [Backpack AI workspace](https://backpack.run).
|
| 55 |
+
|
| 56 |
+
## Validation
|
| 57 |
+
|
| 58 |
+
| Package | Integrity | Load | Inference | Tokenizer |
|
| 59 |
+
| --- | --- | --- | --- | --- |
|
| 60 |
+
| Q4_K_M | passed | passed | passed | passed |
|
| 61 |
+
| Q5_K_M | passed | passed | passed | passed |
|
| 62 |
+
| Q8_0 | passed | passed | passed | passed |
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
- Packaged: 2026-08-20T20:32:10.805207+00:00
|
| 66 |
+
- llama.cpp revision: `de699957b92f490efebad149665b0dccf127eaff`
|
| 67 |
+
- SHA-256 checksums: see `checksums.sha256`
|
| 68 |
+
|
| 69 |
+
- `SmolLM2-135M-Instruct-Q4_K_M.gguf`: `dd18a11b8634d1684448986b8c166f75319f52082d759654aaa8fe5bd2f057e3`
|
| 70 |
+
- `SmolLM2-135M-Instruct-Q5_K_M.gguf`: `00680963c363ba10593daf7568dd6e1ee4c4771a608fe1b4e43f86b564d9b823`
|
| 71 |
+
- `SmolLM2-135M-Instruct-Q8_0.gguf`: `ee785d9b4836ddb57207ae6daa630206a756c99fc52e19696f1e2ea2e8a41b99`
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
## Provenance
|
| 75 |
+
|
| 76 |
+
The source model was resolved to immutable revision `12fd25f77366fa6b3b4b768ec3050bf629380bac`. It was converted with llama.cpp's `convert_hf_to_gguf.py` and quantized with `llama-quantize`; the exact tested revision is recorded above and in `backpack-model.yaml`.
|
| 77 |
+
|
| 78 |
+
## License and attribution
|
| 79 |
+
|
| 80 |
+
Upstream declares **apache-2.0**. Review the [upstream model card](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct) and comply with all applicable terms.
|
| 81 |
+
|
| 82 |
+
Backpack does not claim ownership of the original model. These artifacts are packaged and quantized distributions of the upstream model.
|
| 83 |
+
|
| 84 |
+
## Disclaimer
|
| 85 |
+
|
| 86 |
+
Quantization can alter output quality. Memory estimates vary with runtime configuration, context length, and hardware.
|
SmolLM2-135M-Instruct-Q4_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dd18a11b8634d1684448986b8c166f75319f52082d759654aaa8fe5bd2f057e3
|
| 3 |
+
size 105453984
|
SmolLM2-135M-Instruct-Q5_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:00680963c363ba10593daf7568dd6e1ee4c4771a608fe1b4e43f86b564d9b823
|
| 3 |
+
size 112103328
|
SmolLM2-135M-Instruct-Q8_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ee785d9b4836ddb57207ae6daa630206a756c99fc52e19696f1e2ea2e8a41b99
|
| 3 |
+
size 144810912
|
backpack-model.yaml
ADDED
|
@@ -0,0 +1,106 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
schema_version: 1
|
| 2 |
+
model:
|
| 3 |
+
id: smollm2-135m-instruct
|
| 4 |
+
display_name: SmolLM2-135M-Instruct
|
| 5 |
+
architecture: LlamaForCausalLM
|
| 6 |
+
model_type: llama
|
| 7 |
+
parameter_count: 134515008
|
| 8 |
+
context_length: 8192
|
| 9 |
+
tokenizer_type: GPT2Tokenizer
|
| 10 |
+
tasks:
|
| 11 |
+
- text-generation
|
| 12 |
+
chat_template: true
|
| 13 |
+
multimodal: false
|
| 14 |
+
custom_code: false
|
| 15 |
+
safetensors: true
|
| 16 |
+
upstream:
|
| 17 |
+
repo: HuggingFaceTB/SmolLM2-135M-Instruct
|
| 18 |
+
revision: 12fd25f77366fa6b3b4b768ec3050bf629380bac
|
| 19 |
+
license:
|
| 20 |
+
identifier: apache-2.0
|
| 21 |
+
redistribution: allowed
|
| 22 |
+
reason: Upstream declares a recognized license that permits redistribution; retain
|
| 23 |
+
its terms and attribution.
|
| 24 |
+
gated: false
|
| 25 |
+
private: false
|
| 26 |
+
source_url: https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct
|
| 27 |
+
packages:
|
| 28 |
+
- id: gguf-q4-k-m
|
| 29 |
+
format: gguf
|
| 30 |
+
precision: Q4_K_M
|
| 31 |
+
filename: SmolLM2-135M-Instruct-Q4_K_M.gguf
|
| 32 |
+
sha256: dd18a11b8634d1684448986b8c166f75319f52082d759654aaa8fe5bd2f057e3
|
| 33 |
+
size_bytes: 105453984
|
| 34 |
+
bits_per_weight: 4.83
|
| 35 |
+
runtime:
|
| 36 |
+
provider: llama.cpp
|
| 37 |
+
tested_revision: de699957b92f490efebad149665b0dccf127eaff
|
| 38 |
+
hardware:
|
| 39 |
+
estimated_ram_gb: 0.13
|
| 40 |
+
estimated_vram_gb: 0.12
|
| 41 |
+
recommended_ram_gb: 1.14
|
| 42 |
+
note: Estimate; runtime use varies with context length and configuration.
|
| 43 |
+
validation:
|
| 44 |
+
integrity: passed
|
| 45 |
+
metadata: passed
|
| 46 |
+
load: passed
|
| 47 |
+
inference: passed
|
| 48 |
+
tokenizer: passed
|
| 49 |
+
tested_at: '2026-08-20T20:24:16.685253Z'
|
| 50 |
+
details:
|
| 51 |
+
version: '3'
|
| 52 |
+
tensor_count: '272'
|
| 53 |
+
metadata_count: '30'
|
| 54 |
+
- id: gguf-q5-k-m
|
| 55 |
+
format: gguf
|
| 56 |
+
precision: Q5_K_M
|
| 57 |
+
filename: SmolLM2-135M-Instruct-Q5_K_M.gguf
|
| 58 |
+
sha256: 00680963c363ba10593daf7568dd6e1ee4c4771a608fe1b4e43f86b564d9b823
|
| 59 |
+
size_bytes: 112103328
|
| 60 |
+
bits_per_weight: 5.67
|
| 61 |
+
runtime:
|
| 62 |
+
provider: llama.cpp
|
| 63 |
+
tested_revision: de699957b92f490efebad149665b0dccf127eaff
|
| 64 |
+
hardware:
|
| 65 |
+
estimated_ram_gb: 0.13
|
| 66 |
+
estimated_vram_gb: 0.12
|
| 67 |
+
recommended_ram_gb: 1.15
|
| 68 |
+
note: Estimate; runtime use varies with context length and configuration.
|
| 69 |
+
validation:
|
| 70 |
+
integrity: passed
|
| 71 |
+
metadata: passed
|
| 72 |
+
load: passed
|
| 73 |
+
inference: passed
|
| 74 |
+
tokenizer: passed
|
| 75 |
+
tested_at: '2026-08-20T20:24:18.938307Z'
|
| 76 |
+
details:
|
| 77 |
+
version: '3'
|
| 78 |
+
tensor_count: '272'
|
| 79 |
+
metadata_count: '30'
|
| 80 |
+
- id: gguf-q8-0
|
| 81 |
+
format: gguf
|
| 82 |
+
precision: Q8_0
|
| 83 |
+
filename: SmolLM2-135M-Instruct-Q8_0.gguf
|
| 84 |
+
sha256: ee785d9b4836ddb57207ae6daa630206a756c99fc52e19696f1e2ea2e8a41b99
|
| 85 |
+
size_bytes: 144810912
|
| 86 |
+
bits_per_weight: 8.5
|
| 87 |
+
runtime:
|
| 88 |
+
provider: llama.cpp
|
| 89 |
+
tested_revision: de699957b92f490efebad149665b0dccf127eaff
|
| 90 |
+
hardware:
|
| 91 |
+
estimated_ram_gb: 0.17
|
| 92 |
+
estimated_vram_gb: 0.16
|
| 93 |
+
recommended_ram_gb: 1.2
|
| 94 |
+
note: Estimate; runtime use varies with context length and configuration.
|
| 95 |
+
validation:
|
| 96 |
+
integrity: passed
|
| 97 |
+
metadata: passed
|
| 98 |
+
load: passed
|
| 99 |
+
inference: passed
|
| 100 |
+
tokenizer: passed
|
| 101 |
+
tested_at: '2026-08-20T20:24:21.225025Z'
|
| 102 |
+
details:
|
| 103 |
+
version: '3'
|
| 104 |
+
tensor_count: '272'
|
| 105 |
+
metadata_count: '30'
|
| 106 |
+
generated_at: '2026-08-20T20:32:10.805207Z'
|
checksums.sha256
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
dd18a11b8634d1684448986b8c166f75319f52082d759654aaa8fe5bd2f057e3 SmolLM2-135M-Instruct-Q4_K_M.gguf
|
| 2 |
+
00680963c363ba10593daf7568dd6e1ee4c4771a608fe1b4e43f86b564d9b823 SmolLM2-135M-Instruct-Q5_K_M.gguf
|
| 3 |
+
ee785d9b4836ddb57207ae6daa630206a756c99fc52e19696f1e2ea2e8a41b99 SmolLM2-135M-Instruct-Q8_0.gguf
|