Instructions to use noahsta/lean27b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use noahsta/lean27b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf noahsta/lean27b:Q8_0 # Run inference directly in the terminal: llama cli -hf noahsta/lean27b:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf noahsta/lean27b:Q8_0 # Run inference directly in the terminal: llama cli -hf noahsta/lean27b:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf noahsta/lean27b:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf noahsta/lean27b:Q8_0
Use Docker
docker model run hf.co/noahsta/lean27b:Q8_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use noahsta/lean27b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "noahsta/lean27b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "noahsta/lean27b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/noahsta/lean27b:Q8_0
- Ollama
How to use noahsta/lean27b with Ollama:
ollama run hf.co/noahsta/lean27b:Q8_0
- Unsloth Desktop
- Pi
How to use noahsta/lean27b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf noahsta/lean27b:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "noahsta/lean27b:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use noahsta/lean27b with Docker Model Runner:
docker model run hf.co/noahsta/lean27b:Q8_0
- Lemonade
How to use noahsta/lean27b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull noahsta/lean27b:Q8_0
Run and chat with the model
lemonade run user.lean27b-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use noahsta/lean27b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf noahsta/lean27b:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default noahsta/lean27b:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use noahsta/lean27b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf noahsta/lean27b:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "noahsta/lean27b:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Concise README with full training params
Browse files
README.md
CHANGED
|
@@ -5,45 +5,56 @@ license: apache-2.0
|
|
| 5 |
tags:
|
| 6 |
- gguf
|
| 7 |
- llama.cpp
|
| 8 |
-
-
|
| 9 |
-
-
|
|
|
|
| 10 |
pipeline_tag: text-generation
|
| 11 |
---
|
| 12 |
|
| 13 |
# lean27b
|
| 14 |
|
| 15 |
-
|
| 16 |
|
| 17 |
-
|
| 18 |
|
| 19 |
-
|
| 20 |
-
|---|---|
|
| 21 |
-
| r | 32 |
|
| 22 |
-
| weight decay (wd) | 0.04 |
|
| 23 |
|
| 24 |
-
##
|
| 25 |
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
-
|
| 31 |
-
-
|
| 32 |
-
|
| 33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
## Usage
|
| 36 |
|
| 37 |
-
|
| 38 |
|
| 39 |
```bash
|
| 40 |
llama-server -m lean27b-r32_wd40_full-Q8_0.gguf -ngl 99 -c 262144 --jinja
|
| 41 |
```
|
| 42 |
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
## Note on metadata
|
| 48 |
-
|
| 49 |
-
The original merged export carried over `block_count = 65` and `nextn_predict_layers = 1` from the MTP base checkpoint, although the tensors contain only 64 layers and no MTP head. In this file both values are corrected to `64` / `0`. No tensor data was changed.
|
|
|
|
| 5 |
tags:
|
| 6 |
- gguf
|
| 7 |
- llama.cpp
|
| 8 |
+
- lean4
|
| 9 |
+
- mathlib
|
| 10 |
+
- lora
|
| 11 |
pipeline_tag: text-generation
|
| 12 |
---
|
| 13 |
|
| 14 |
# lean27b
|
| 15 |
|
| 16 |
+
[Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) LoRA-finetuned on Lean 4 / Mathlib4 source, merged and released as a single Q8_0 GGUF.
|
| 17 |
|
| 18 |
+

|
| 19 |
|
| 20 |
+
*Weight-decay sweep at rank 32, all layers. The released model is **wd = 0.4** (darkest line). Final full-validation loss: 0.652. The tool-call sanity check passed 3/3 in every epoch.*
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
+
## Data
|
| 23 |
|
| 24 |
+
- Every `.lean` file under `Mathlib/` in [mathlib4](https://github.com/leanprover-community/mathlib4) at tag `v4.33.0`.
|
| 25 |
+
- 80/20 train/val split by file (seed 42).
|
| 26 |
+
- Files shorter than 1024 tokens are kept whole. Longer files are split only at blank lines (`\n\n`) and packed greedily into chunks of at most 1024 tokens.
|
| 27 |
+
- Plain causal-LM loss on all tokens. No chat template, no instruction data.
|
| 28 |
+
|
| 29 |
+
## Training
|
| 30 |
|
| 31 |
+
| Parameter | Value |
|
| 32 |
+
|---|---|
|
| 33 |
+
| Method | LoRA (peft), bf16 base, gradient checkpointing |
|
| 34 |
+
| Rank / alpha / dropout | 32 / 32 / 0.05 |
|
| 35 |
+
| Target modules | `q_proj k_proj v_proj o_proj in_proj_qkv out_proj gate_proj up_proj down_proj` |
|
| 36 |
+
| Layers | all 64 |
|
| 37 |
+
| Optimizer | AdamW, lr 1e-5 (constant, no warmup), weight decay 0.4 |
|
| 38 |
+
| Grad clip | 1.0 |
|
| 39 |
+
| Batch size | 8 (no accumulation) |
|
| 40 |
+
| Max sequence length | 1024 |
|
| 41 |
+
| Epochs | 5 |
|
| 42 |
+
| Seed | 42 (data order reshuffled with seed + epoch) |
|
| 43 |
+
| Hardware | 1× 80 GB GPU |
|
| 44 |
+
| Libraries | torch 2.13.0, transformers 5.15.1, peft 0.20.0, accelerate 1.14.0 |
|
| 45 |
+
|
| 46 |
+
After each epoch, a check confirms the model still emits valid tool calls (grep / web_search / ls). Training stops early if 2 of the 3 calls fail in two consecutive epochs. This never triggered.
|
| 47 |
+
|
| 48 |
+
**Export.** The LoRA was merged into the bf16 base, and the vision tower was dropped. The result was converted with llama.cpp `convert_hf_to_gguf.py --outtype q8_0`. The GGUF metadata was then patched from `block_count=65, nextn_predict_layers=1` to `64 / 0`, because the MTP head is not included. No tensor data was changed.
|
| 49 |
|
| 50 |
## Usage
|
| 51 |
|
| 52 |
+
The file is `lean27b-r32_wd40_full-Q8_0.gguf` (28.6 GB). It needs a llama.cpp build that supports `qwen35`.
|
| 53 |
|
| 54 |
```bash
|
| 55 |
llama-server -m lean27b-r32_wd40_full-Q8_0.gguf -ngl 99 -c 262144 --jinja
|
| 56 |
```
|
| 57 |
|
| 58 |
+
- Text only: no mmproj, no MTP head.
|
| 59 |
+
- VRAM: about 26.6 GiB for the weights, about 41.6 GiB at the full 262k context.
|
| 60 |
+
- Sampling defaults embedded in the GGUF: temperature 1.0, top_p 0.95, top_k 20.
|
|
|
|
|
|
|
|
|
|
|
|