amirthakatesh commited on
Commit
cf9d77c
·
verified ·
1 Parent(s): f8b5b9b

Add files using upload-large-folder tool

Browse files
.gitattributes CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ SmolLM2-135M-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
37
+ SmolLM2-135M-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
38
+ SmolLM2-135M-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: HuggingFaceTB/SmolLM2-135M-Instruct
3
+ license: apache-2.0
4
+ library_name: gguf
5
+ tags:
6
+ - gguf
7
+ - llama.cpp
8
+ - backpack
9
+ ---
10
+
11
+ # SmolLM2-135M-Instruct — Backpack GGUF
12
+
13
+ 🎒 **Backpack Verified**
14
+
15
+ GGUF quantizations of [HuggingFaceTB/SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct), tested for llama.cpp-compatible inference and packaged for Backpack.
16
+
17
+ ## Model
18
+
19
+ | Property | Value |
20
+ | --- | --- |
21
+ | Original model | `HuggingFaceTB/SmolLM2-135M-Instruct` |
22
+ | Original publisher | HuggingFaceTB |
23
+ | Upstream revision | `12fd25f77366fa6b3b4b768ec3050bf629380bac` |
24
+ | Architecture | LlamaForCausalLM |
25
+ | Parameters | 134,515,008 |
26
+ | Context length | 8,192 |
27
+ | License | apache-2.0 |
28
+
29
+ ## Available packages
30
+
31
+ | Quantization | Size | Approx. RAM | Recommended for |
32
+ | --- | ---: | ---: | --- |
33
+ | Q4_K_M | 100.6 MiB | 1.14 GB | Most users |
34
+ | Q5_K_M | 106.9 MiB | 1.15 GB | Higher quality |
35
+ | Q8_0 | 138.1 MiB | 1.2 GB | Plenty of memory |
36
+
37
+
38
+ Memory values are estimates, not guarantees. Runtime configuration and context length change actual use.
39
+
40
+ ## Backpack recommendation
41
+
42
+ **Recommended: Q4_K_M.** It usually offers a practical quality, size, and speed balance for local inference.
43
+
44
+ ## Run with llama.cpp
45
+
46
+ Using the llama.cpp revision recorded below:
47
+
48
+ ```bash
49
+ llama-cli --model SmolLM2-135M-Instruct-Q4_K_M.gguf --conversation
50
+ ```
51
+
52
+ ## Run with Backpack
53
+
54
+ These artifacts and `backpack-model.yaml` are prepared for the [Backpack AI workspace](https://backpack.run).
55
+
56
+ ## Validation
57
+
58
+ | Package | Integrity | Load | Inference | Tokenizer |
59
+ | --- | --- | --- | --- | --- |
60
+ | Q4_K_M | passed | passed | passed | passed |
61
+ | Q5_K_M | passed | passed | passed | passed |
62
+ | Q8_0 | passed | passed | passed | passed |
63
+
64
+
65
+ - Packaged: 2026-08-20T20:32:10.805207+00:00
66
+ - llama.cpp revision: `de699957b92f490efebad149665b0dccf127eaff`
67
+ - SHA-256 checksums: see `checksums.sha256`
68
+
69
+ - `SmolLM2-135M-Instruct-Q4_K_M.gguf`: `dd18a11b8634d1684448986b8c166f75319f52082d759654aaa8fe5bd2f057e3`
70
+ - `SmolLM2-135M-Instruct-Q5_K_M.gguf`: `00680963c363ba10593daf7568dd6e1ee4c4771a608fe1b4e43f86b564d9b823`
71
+ - `SmolLM2-135M-Instruct-Q8_0.gguf`: `ee785d9b4836ddb57207ae6daa630206a756c99fc52e19696f1e2ea2e8a41b99`
72
+
73
+
74
+ ## Provenance
75
+
76
+ The source model was resolved to immutable revision `12fd25f77366fa6b3b4b768ec3050bf629380bac`. It was converted with llama.cpp's `convert_hf_to_gguf.py` and quantized with `llama-quantize`; the exact tested revision is recorded above and in `backpack-model.yaml`.
77
+
78
+ ## License and attribution
79
+
80
+ Upstream declares **apache-2.0**. Review the [upstream model card](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct) and comply with all applicable terms.
81
+
82
+ Backpack does not claim ownership of the original model. These artifacts are packaged and quantized distributions of the upstream model.
83
+
84
+ ## Disclaimer
85
+
86
+ Quantization can alter output quality. Memory estimates vary with runtime configuration, context length, and hardware.
SmolLM2-135M-Instruct-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dd18a11b8634d1684448986b8c166f75319f52082d759654aaa8fe5bd2f057e3
3
+ size 105453984
SmolLM2-135M-Instruct-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:00680963c363ba10593daf7568dd6e1ee4c4771a608fe1b4e43f86b564d9b823
3
+ size 112103328
SmolLM2-135M-Instruct-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee785d9b4836ddb57207ae6daa630206a756c99fc52e19696f1e2ea2e8a41b99
3
+ size 144810912
backpack-model.yaml ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ schema_version: 1
2
+ model:
3
+ id: smollm2-135m-instruct
4
+ display_name: SmolLM2-135M-Instruct
5
+ architecture: LlamaForCausalLM
6
+ model_type: llama
7
+ parameter_count: 134515008
8
+ context_length: 8192
9
+ tokenizer_type: GPT2Tokenizer
10
+ tasks:
11
+ - text-generation
12
+ chat_template: true
13
+ multimodal: false
14
+ custom_code: false
15
+ safetensors: true
16
+ upstream:
17
+ repo: HuggingFaceTB/SmolLM2-135M-Instruct
18
+ revision: 12fd25f77366fa6b3b4b768ec3050bf629380bac
19
+ license:
20
+ identifier: apache-2.0
21
+ redistribution: allowed
22
+ reason: Upstream declares a recognized license that permits redistribution; retain
23
+ its terms and attribution.
24
+ gated: false
25
+ private: false
26
+ source_url: https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct
27
+ packages:
28
+ - id: gguf-q4-k-m
29
+ format: gguf
30
+ precision: Q4_K_M
31
+ filename: SmolLM2-135M-Instruct-Q4_K_M.gguf
32
+ sha256: dd18a11b8634d1684448986b8c166f75319f52082d759654aaa8fe5bd2f057e3
33
+ size_bytes: 105453984
34
+ bits_per_weight: 4.83
35
+ runtime:
36
+ provider: llama.cpp
37
+ tested_revision: de699957b92f490efebad149665b0dccf127eaff
38
+ hardware:
39
+ estimated_ram_gb: 0.13
40
+ estimated_vram_gb: 0.12
41
+ recommended_ram_gb: 1.14
42
+ note: Estimate; runtime use varies with context length and configuration.
43
+ validation:
44
+ integrity: passed
45
+ metadata: passed
46
+ load: passed
47
+ inference: passed
48
+ tokenizer: passed
49
+ tested_at: '2026-08-20T20:24:16.685253Z'
50
+ details:
51
+ version: '3'
52
+ tensor_count: '272'
53
+ metadata_count: '30'
54
+ - id: gguf-q5-k-m
55
+ format: gguf
56
+ precision: Q5_K_M
57
+ filename: SmolLM2-135M-Instruct-Q5_K_M.gguf
58
+ sha256: 00680963c363ba10593daf7568dd6e1ee4c4771a608fe1b4e43f86b564d9b823
59
+ size_bytes: 112103328
60
+ bits_per_weight: 5.67
61
+ runtime:
62
+ provider: llama.cpp
63
+ tested_revision: de699957b92f490efebad149665b0dccf127eaff
64
+ hardware:
65
+ estimated_ram_gb: 0.13
66
+ estimated_vram_gb: 0.12
67
+ recommended_ram_gb: 1.15
68
+ note: Estimate; runtime use varies with context length and configuration.
69
+ validation:
70
+ integrity: passed
71
+ metadata: passed
72
+ load: passed
73
+ inference: passed
74
+ tokenizer: passed
75
+ tested_at: '2026-08-20T20:24:18.938307Z'
76
+ details:
77
+ version: '3'
78
+ tensor_count: '272'
79
+ metadata_count: '30'
80
+ - id: gguf-q8-0
81
+ format: gguf
82
+ precision: Q8_0
83
+ filename: SmolLM2-135M-Instruct-Q8_0.gguf
84
+ sha256: ee785d9b4836ddb57207ae6daa630206a756c99fc52e19696f1e2ea2e8a41b99
85
+ size_bytes: 144810912
86
+ bits_per_weight: 8.5
87
+ runtime:
88
+ provider: llama.cpp
89
+ tested_revision: de699957b92f490efebad149665b0dccf127eaff
90
+ hardware:
91
+ estimated_ram_gb: 0.17
92
+ estimated_vram_gb: 0.16
93
+ recommended_ram_gb: 1.2
94
+ note: Estimate; runtime use varies with context length and configuration.
95
+ validation:
96
+ integrity: passed
97
+ metadata: passed
98
+ load: passed
99
+ inference: passed
100
+ tokenizer: passed
101
+ tested_at: '2026-08-20T20:24:21.225025Z'
102
+ details:
103
+ version: '3'
104
+ tensor_count: '272'
105
+ metadata_count: '30'
106
+ generated_at: '2026-08-20T20:32:10.805207Z'
checksums.sha256 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ dd18a11b8634d1684448986b8c166f75319f52082d759654aaa8fe5bd2f057e3 SmolLM2-135M-Instruct-Q4_K_M.gguf
2
+ 00680963c363ba10593daf7568dd6e1ee4c4771a608fe1b4e43f86b564d9b823 SmolLM2-135M-Instruct-Q5_K_M.gguf
3
+ ee785d9b4836ddb57207ae6daa630206a756c99fc52e19696f1e2ea2e8a41b99 SmolLM2-135M-Instruct-Q8_0.gguf