Text Generation
GGUF
English
legal
consumer-protection
fdcpa
fcra
letter-writing
grant-writing
llama-cpp
ollama
qwen3
lora
conversational
Instructions to use SwarmandBee/LocalLegal-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use SwarmandBee/LocalLegal-27B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf SwarmandBee/LocalLegal-27B # Run inference directly in the terminal: llama cli -hf SwarmandBee/LocalLegal-27B
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf SwarmandBee/LocalLegal-27B # Run inference directly in the terminal: llama cli -hf SwarmandBee/LocalLegal-27B
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf SwarmandBee/LocalLegal-27B # Run inference directly in the terminal: ./llama-cli -hf SwarmandBee/LocalLegal-27B
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf SwarmandBee/LocalLegal-27B # Run inference directly in the terminal: ./build/bin/llama-cli -hf SwarmandBee/LocalLegal-27B
Use Docker
docker model run hf.co/SwarmandBee/LocalLegal-27B
- LM Studio
- Jan
- vLLM
How to use SwarmandBee/LocalLegal-27B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "SwarmandBee/LocalLegal-27B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SwarmandBee/LocalLegal-27B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/SwarmandBee/LocalLegal-27B
- Ollama
How to use SwarmandBee/LocalLegal-27B with Ollama:
ollama run hf.co/SwarmandBee/LocalLegal-27B
- Unsloth Desktop
- Pi
How to use SwarmandBee/LocalLegal-27B with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf SwarmandBee/LocalLegal-27B
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "SwarmandBee/LocalLegal-27B" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use SwarmandBee/LocalLegal-27B with Docker Model Runner:
docker model run hf.co/SwarmandBee/LocalLegal-27B
- Lemonade
How to use SwarmandBee/LocalLegal-27B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull SwarmandBee/LocalLegal-27B
Run and chat with the model
lemonade run user.LocalLegal-27B-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use SwarmandBee/LocalLegal-27B with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf SwarmandBee/LocalLegal-27B
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default SwarmandBee/LocalLegal-27B
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use SwarmandBee/LocalLegal-27B with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf SwarmandBee/LocalLegal-27B
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "SwarmandBee/LocalLegal-27B" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen3.6-27B
|
| 4 |
+
language:
|
| 5 |
+
- en
|
| 6 |
+
pipeline_tag: text-generation
|
| 7 |
+
library_name: gguf
|
| 8 |
+
tags:
|
| 9 |
+
- legal
|
| 10 |
+
- consumer-protection
|
| 11 |
+
- fdcpa
|
| 12 |
+
- fcra
|
| 13 |
+
- letter-writing
|
| 14 |
+
- grant-writing
|
| 15 |
+
- gguf
|
| 16 |
+
- llama-cpp
|
| 17 |
+
- ollama
|
| 18 |
+
- qwen3
|
| 19 |
+
- lora
|
| 20 |
+
---
|
| 21 |
+
|
| 22 |
+
# LocalLegal-27B
|
| 23 |
+
|
| 24 |
+
**A calm, dignified consumer-rights letter-writer & grant writer.** It *organizes* the facts a person gives it and *drafts* clear, statute-grounded documents they can review, sign, and send themselves β debt-validation and cease-contact letters (FDCPA), credit-report disputes and method-of-verification letters (FCRA), goodwill and medical-billing letters, identity-theft blocks, and grant/assistance narratives.
|
| 25 |
+
|
| 26 |
+
> **This is document preparation, not legal advice.** LocalLegal is not a lawyer, does not give legal advice, and never predicts the outcome of a case. It states what a statute *says* and what a letter *requests* β and hands anything past the letter stage (lawsuits, garnishment, liens, court dates) to free legal aid or a licensed attorney.
|
| 27 |
+
|
| 28 |
+
Built by **[Swarm & Bee](https://huggingface.co/SwarmandBee)** β the brain behind the LocalLegal "write + send certified from home" flow.
|
| 29 |
+
|
| 30 |
+
---
|
| 31 |
+
|
| 32 |
+
## π It beat base β decisively, on both domains
|
| 33 |
+
|
| 34 |
+
Stage-5 gate: **held-out, per-domain, deterministic** evaluation. Teacher-forced cross-entropy / perplexity on chat-templated eval, base vs cooked, **same tokenizer & template, N=400/domain, seed 1117, seq 4096, bf16** on an RTX PRO 6000. **No LLM-as-judge** β a number anyone can re-derive.
|
| 35 |
+
|
| 36 |
+
| Domain | Base ppl | LocalLegal-27B ppl | Ξ | Base CE β Cooked CE |
|
| 37 |
+
|---|--:|--:|--:|---|
|
| 38 |
+
| **Legal** (FDCPA/FCRA letters) | 16.662 | **2.058** | **β87.65%** | 2.813 β 0.722 |
|
| 39 |
+
| **Grant** (proposal narratives) | 3.250 | **2.022** | **β37.80%** | 1.179 β 0.704 |
|
| 40 |
+
|
| 41 |
+
**Beat base on both. Killed domains: none.** Cooked CE (~0.70β0.72) matches the training landing (~0.64β0.77) β coherent, **no overfit**. The legal gain exceeds our DiabeticAnchor-27B reference (+57%). The 44%-share, 9%-truncation grant tail passed clean β not undercooked.
|
| 42 |
+
|
| 43 |
+
Receipts (deterministic, re-runnable): `beat_base_27b.py` Β· `beat_base_27b.log` Β· `beat_base_27b_verdict.json`.
|
| 44 |
+
|
| 45 |
+
---
|
| 46 |
+
|
| 47 |
+
## π§ Usage
|
| 48 |
+
|
| 49 |
+
This repo ships a **Q4 GGUF** (~16 GB) β runs on llama.cpp / Ollama / LM Studio. Qwen3.6 is a **thinking model**, so the chat template prefills an empty `<think>` block; the included `Modelfile` handles this for you.
|
| 50 |
+
|
| 51 |
+
### Ollama
|
| 52 |
+
```bash
|
| 53 |
+
# with the included Modelfile (carries the template + system prompt + params)
|
| 54 |
+
ollama create locallegal-27b -f locallegal-27b.Modelfile
|
| 55 |
+
ollama run locallegal-27b "Draft an FDCPA debt-validation letter. Collector: Midland Credit. Account #4402, $1,284 medical debt I don't recognize."
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
### llama.cpp
|
| 59 |
+
```bash
|
| 60 |
+
./llama-cli -m locallegal-27b-q4.gguf -c 8192 --temp 0.6 --top-p 0.9 \
|
| 61 |
+
-p "<your chat-templated prompt>"
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
**Recommended sampling:** `temperature 0.6`, `top_p 0.9`, `num_ctx 8192`. Stop tokens `<|im_start|>` / `<|im_end|>` (ChatML).
|
| 65 |
+
|
| 66 |
+
### System prompt (its identity β every letter includes header, RE: line, statute cite, specific request, response window, signature block)
|
| 67 |
+
The model is trained to the LocalLegal persona: calm, factual, never shaming, uses the verbs *organize / draft / prepare / review / track*, and refuses to say "legal advice," "sue them," "you'll win," or "guaranteed." Full system prompt ships in the `Modelfile`.
|
| 68 |
+
|
| 69 |
+
---
|
| 70 |
+
|
| 71 |
+
## π³ How it was cooked
|
| 72 |
+
|
| 73 |
+
- **Base:** [`Qwen/Qwen3.6-27B`](https://huggingface.co/Qwen/Qwen3.6-27B) (Apache-2.0) β hybrid Gated-DeltaNet + Gated-Attention arch, thinking model.
|
| 74 |
+
- **Method:** LoRA r32 / Ξ±16 on attn+mlp Β· LR 1e-5 Β· cosine Β· seq 4096 Β· bf16 Β· Unsloth + TRL. Clean 16-bit merge β Q4 GGUF.
|
| 75 |
+
- **Corpus:** **78,231 train / 2,500 eval**, curated down from ~340K raw rows via cross-domain dedup, eval carve-out, hash-scrub of contaminants, and near-dup pruning. Split **Legal 57.8% / Grant 42.2%** (natural balance, zero synthetic upsampling). **Per-domain true holdout** eval (legal 1,500 / grant 1,000).
|
| 76 |
+
- **Loss:** 2.156 β 0.768 (β64%), smooth monotone, grad_norm 0.20 β textbook curve, no spikes/NaN, no overcook.
|
| 77 |
+
- **Rig:** SwarmRails (owned) Β· 1Γ RTX PRO 6000 Blackwell 96 GB Β· 350 W thermal cap Β· 45.7 h wall. Sovereign compute β cooked on our own iron.
|
| 78 |
+
- **Discipline:** full canary-then-cook (5-stage senior-hack review) β beat-base-or-kill, per domain, no blended half-truths.
|
| 79 |
+
|
| 80 |
+
## π Files
|
| 81 |
+
- `locallegal-27b-q4.gguf` β Q4 quantized weights (~16 GB)
|
| 82 |
+
- `locallegal-27b.Modelfile` β Ollama template + system prompt + params
|
| 83 |
+
|
| 84 |
+
## π Defendable
|
| 85 |
+
Every claim here has a receipt: deterministic per-domain beat-base eval (no LLM judge), monotone loss curve, hash-verified corpus, and a full cook flightsheet. Show the math, verify it yourself.
|
| 86 |
+
|
| 87 |
+
## βοΈ Scope & safety
|
| 88 |
+
Document preparation only. Not legal advice, not a lawyer, no outcome predictions. Anything beyond the letter stage β free legal aid (LawHelp.org) / a licensed consumer-protection attorney / your state Attorney General. As a fine-tune of Qwen3.6-27B, it inherits the base model's Apache-2.0 terms and general LLM limitations (it can be wrong β a human reviews and signs every letter).
|
| 89 |
+
|
| 90 |
+
## Citation
|
| 91 |
+
```
|
| 92 |
+
@misc{locallegal27b2026,
|
| 93 |
+
title = {LocalLegal-27B: a statute-grounded consumer-rights letter-writer},
|
| 94 |
+
author = {Swarm and Bee},
|
| 95 |
+
year = {2026},
|
| 96 |
+
note = {LoRA fine-tune of Qwen3.6-27B; deterministic per-domain beat-base eval},
|
| 97 |
+
url = {https://huggingface.co/SwarmandBee/LocalLegal-27B}
|
| 98 |
+
}
|
| 99 |
+
```
|