Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+SMOLM2Prover-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+SMOLM2Prover.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,61 @@
+# SMOLM2Prover - GGUF Format
+
+GGUF quantized version of the SMOLM2Prover model for use with llama.cpp and compatible runtimes.
+
+## Model Details
+
+- **Original Model**: reaperdoesntknow/SMOLM2Prover
+- **Architecture**: LlamaForCausalLM
+- **Context Length**: 8192 tokens
+- **Embedding Dimension**: 960
+- **Layers**: 32
+- **Head Count**: 15 (Q), 5 (KV) - GQA
+
+## Available Files
+
+| File | Size | Quantization | Quality |
+|------|------|--------------|---------|
+| `SMOLM2Prover.gguf` | 692M | F16 | Original (no quantization) |
+| `SMOLM2Prover-Q4_K_M.gguf` | 258M | Q4_K_M | Recommended (good quality/size balance) |
+
+## Usage
+
+### With llama.cpp
+
+```bash
+# Run with the quantized model
+./llama-cli -m SMOLM2Prover-Q4_K_M.gguf -p "Your prompt here" -n 256
+```
+
+### With Ollama
+
+Create a `Modelfile`:
+```
+FROM ./SMOLM2Prover-Q4_K_M.gguf
+```
+
+Then:
+```bash
+ollama create smolm2prover -f Modelfile
+ollama run smolm2prover
+```
+
+### With LM Studio
+
+1. Download `SMOLM2Prover-Q4_K_M.gguf`
+2. Place in LM Studio models folder
+3. Load and chat!
+
+## Quantization Details
+
+The Q4_K_M quantization uses:
+- Q4_K for most weights
+- Q5_0 fallback for tensors not divisible by 256
+- Q6_K/Q8_0 for some critical layers
+
+**Size reduction**: 692M → 258M (63% smaller)
+**BPW**: 5.94 bits per weight
+
+## License
+
+Same as the original model.

SMOLM2Prover-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55f1d76afef32d2a1c3e1d67cd8f7f464286f60ff146270c9eeb875f95f96bbc
+size 270591136

SMOLM2Prover.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d350a1f1c1ce510879cb06d28850d43afe723dfcd5a3c8113a9de699c4f98ae1
+size 725554336