Instructions to use noahsta/lean27b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use noahsta/lean27b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf noahsta/lean27b:Q8_0 # Run inference directly in the terminal: llama cli -hf noahsta/lean27b:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf noahsta/lean27b:Q8_0 # Run inference directly in the terminal: llama cli -hf noahsta/lean27b:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf noahsta/lean27b:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf noahsta/lean27b:Q8_0
Use Docker
docker model run hf.co/noahsta/lean27b:Q8_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use noahsta/lean27b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "noahsta/lean27b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "noahsta/lean27b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/noahsta/lean27b:Q8_0
- Ollama
How to use noahsta/lean27b with Ollama:
ollama run hf.co/noahsta/lean27b:Q8_0
- Unsloth Desktop
- Pi
How to use noahsta/lean27b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf noahsta/lean27b:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "noahsta/lean27b:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use noahsta/lean27b with Docker Model Runner:
docker model run hf.co/noahsta/lean27b:Q8_0
- Lemonade
How to use noahsta/lean27b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull noahsta/lean27b:Q8_0
Run and chat with the model
lemonade run user.lean27b-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use noahsta/lean27b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf noahsta/lean27b:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default noahsta/lean27b:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use noahsta/lean27b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf noahsta/lean27b:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "noahsta/lean27b:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from noahsta/lean27b: direct link, hf CLI and curl.
- Browser
- Download file 2.51 kB
-
https://huggingface.co/noahsta/lean27b/resolve/main/README.md
- Command line
-
hf download hf://noahsta/lean27b/README.md
-
curl -L -o README.md https://huggingface.co/noahsta/lean27b/resolve/main/README.md
2.51 kB
| base_model: Qwen/Qwen3.6-27B | |
| base_model_relation: finetune | |
| license: apache-2.0 | |
| tags: | |
| - gguf | |
| - llama.cpp | |
| - lean4 | |
| - mathlib | |
| - lora | |
| pipeline_tag: text-generation | |
| # lean27b | |
| [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) LoRA-finetuned on Lean 4 / Mathlib4 source, merged and released as a single Q8_0 GGUF. | |
|  | |
| *Weight-decay sweep at rank 32, all layers. The released model is **wd = 0.4** (darkest line). Final full-validation loss: 0.652. The tool-call sanity check passed 3/3 in every epoch.* | |
| ## Data | |
| - Every `.lean` file under `Mathlib/` in [mathlib4](https://github.com/leanprover-community/mathlib4) at tag `v4.33.0`. | |
| - 80/20 train/val split by file (seed 42). | |
| - Files shorter than 1024 tokens are kept whole. Longer files are split only at blank lines (`\n\n`) and packed greedily into chunks of at most 1024 tokens. | |
| - Plain causal-LM loss on all tokens. No chat template, no instruction data. | |
| ## Training | |
| | Parameter | Value | | |
| |---|---| | |
| | Method | LoRA (peft), bf16 base, gradient checkpointing | | |
| | Rank / alpha / dropout | 32 / 32 / 0.05 | | |
| | Target modules | `q_proj k_proj v_proj o_proj in_proj_qkv out_proj gate_proj up_proj down_proj` | | |
| | Layers | all 64 | | |
| | Optimizer | AdamW, lr 1e-5 (constant, no warmup), weight decay 0.4 | | |
| | Grad clip | 1.0 | | |
| | Batch size | 8 (no accumulation) | | |
| | Max sequence length | 1024 | | |
| | Epochs | 5 | | |
| | Seed | 42 (data order reshuffled with seed + epoch) | | |
| | Hardware | 1× 80 GB GPU | | |
| | Libraries | torch 2.13.0, transformers 5.15.1, peft 0.20.0, accelerate 1.14.0 | | |
| After each epoch, a check confirms the model still emits valid tool calls (grep / web_search / ls). Training stops early if 2 of the 3 calls fail in two consecutive epochs. This never triggered. | |
| **Export.** The LoRA was merged into the bf16 base, and the vision tower was dropped. The result was converted with llama.cpp `convert_hf_to_gguf.py --outtype q8_0`. The GGUF metadata was then patched from `block_count=65, nextn_predict_layers=1` to `64 / 0`, because the MTP head is not included. No tensor data was changed. | |
| ## Usage | |
| The file is `lean27b-r32_wd40_full-Q8_0.gguf` (28.6 GB). It needs a llama.cpp build that supports `qwen35`. | |
| ```bash | |
| llama-server -m lean27b-r32_wd40_full-Q8_0.gguf -ngl 99 -c 262144 --jinja | |
| ``` | |
| - Text only: no mmproj, no MTP head. | |
| - VRAM: about 26.6 GiB for the weights, about 41.6 GiB at the full 262k context. | |
| - Sampling defaults embedded in the GGUF: temperature 1.0, top_p 0.95, top_k 20. | |