Text Generation
GGUF
llama.cpp
agent
coding
reasoning
tool-use
function-calling
quantized
cuda
metal
conversational
Instructions to use badtheorylabs/BTL-3-Compact with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use badtheorylabs/BTL-3-Compact with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="badtheorylabs/BTL-3-Compact", filename="model/BTL-3-Compact-AVQ2.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use badtheorylabs/BTL-3-Compact with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf badtheorylabs/BTL-3-Compact # Run inference directly in the terminal: llama cli -hf badtheorylabs/BTL-3-Compact
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf badtheorylabs/BTL-3-Compact # Run inference directly in the terminal: llama cli -hf badtheorylabs/BTL-3-Compact
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf badtheorylabs/BTL-3-Compact # Run inference directly in the terminal: ./llama-cli -hf badtheorylabs/BTL-3-Compact
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf badtheorylabs/BTL-3-Compact # Run inference directly in the terminal: ./build/bin/llama-cli -hf badtheorylabs/BTL-3-Compact
Use Docker
docker model run hf.co/badtheorylabs/BTL-3-Compact
- LM Studio
- Jan
- vLLM
How to use badtheorylabs/BTL-3-Compact with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "badtheorylabs/BTL-3-Compact" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "badtheorylabs/BTL-3-Compact", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/badtheorylabs/BTL-3-Compact
- Ollama
How to use badtheorylabs/BTL-3-Compact with Ollama:
ollama run hf.co/badtheorylabs/BTL-3-Compact
- Unsloth Studio
How to use badtheorylabs/BTL-3-Compact with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for badtheorylabs/BTL-3-Compact to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for badtheorylabs/BTL-3-Compact to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for badtheorylabs/BTL-3-Compact to start chatting
- Pi
How to use badtheorylabs/BTL-3-Compact with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf badtheorylabs/BTL-3-Compact
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "badtheorylabs/BTL-3-Compact" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use badtheorylabs/BTL-3-Compact with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf badtheorylabs/BTL-3-Compact
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default badtheorylabs/BTL-3-Compact
Run Hermes
hermes
- Atomic Chat new
- OpenClaw new
How to use badtheorylabs/BTL-3-Compact with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf badtheorylabs/BTL-3-Compact
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "badtheorylabs/BTL-3-Compact" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use badtheorylabs/BTL-3-Compact with Docker Model Runner:
docker model run hf.co/badtheorylabs/BTL-3-Compact
- Lemonade
How to use badtheorylabs/BTL-3-Compact with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull badtheorylabs/BTL-3-Compact
Run and chat with the model
lemonade run user.BTL-3-Compact-{{QUANT_TAG}}List all available models
lemonade list
| """Install a verified BTL-3 consumer runtime and its external model.""" | |
| from __future__ import annotations | |
| import argparse | |
| import hashlib | |
| import json | |
| import os | |
| from pathlib import Path | |
| import platform | |
| import shutil | |
| import tempfile | |
| MODEL_NAME = "BTL-3-Compact-AVQ2.gguf" | |
| MODEL_BYTES = 8_392_369_600 | |
| MODEL_SHA256 = "2ddf9527620a17a2a6739d184a7096c45712092e6589128792ec6254e94dc30c" | |
| class InstallError(ValueError): | |
| pass | |
| def sha256(path: Path) -> str: | |
| digest = hashlib.sha256() | |
| with path.open("rb") as stream: | |
| for chunk in iter(lambda: stream.read(4 * 1024 * 1024), b""): | |
| digest.update(chunk) | |
| return digest.hexdigest() | |
| def default_prefix() -> Path: | |
| if os.name == "nt": | |
| root = os.environ.get("LOCALAPPDATA") | |
| if not root: | |
| raise InstallError("LOCALAPPDATA is unavailable; pass --prefix") | |
| return Path(root) / "BTL3" | |
| root = os.environ.get("XDG_DATA_HOME") | |
| return Path(root) / "btl3" if root else Path.home() / ".local/share/btl3" | |
| def read_manifest(runtime: Path) -> dict: | |
| path = runtime / "bundle-manifest.json" | |
| try: | |
| manifest = json.loads(path.read_text()) | |
| except (OSError, json.JSONDecodeError) as error: | |
| raise InstallError(f"invalid runtime manifest: {error}") from error | |
| external = manifest.get("external_model", {}) | |
| expected = { | |
| "filename": MODEL_NAME, | |
| "bytes": MODEL_BYTES, | |
| "sha256": MODEL_SHA256, | |
| } | |
| if any(external.get(key) != value for key, value in expected.items()): | |
| raise InstallError("runtime expects a different BTL-3 model") | |
| return manifest | |
| def verify_runtime(runtime: Path, manifest: dict) -> None: | |
| for name, expected in manifest.get("files", {}).items(): | |
| path = runtime / name | |
| if "symlink" in expected: | |
| if not path.is_symlink() or os.readlink(path) != expected["symlink"]: | |
| raise InstallError(f"runtime symlink mismatch: {name}") | |
| continue | |
| if not path.is_file(): | |
| raise InstallError(f"runtime file is missing: {name}") | |
| if path.stat().st_size != expected["bytes"]: | |
| raise InstallError(f"runtime file size mismatch: {name}") | |
| if sha256(path) != expected["sha256"]: | |
| raise InstallError(f"runtime file checksum mismatch: {name}") | |
| def verify_model(model: Path) -> None: | |
| if not model.is_file(): | |
| raise InstallError(f"model is missing: {model}") | |
| if model.stat().st_size != MODEL_BYTES: | |
| raise InstallError(f"model size mismatch: {model.stat().st_size}") | |
| if sha256(model) != MODEL_SHA256: | |
| raise InstallError("model SHA-256 mismatch") | |
| def install(runtime: Path, model: Path, prefix: Path, replace: bool) -> Path: | |
| runtime, model, prefix = runtime.resolve(), model.resolve(), prefix.resolve() | |
| manifest = read_manifest(runtime) | |
| verify_runtime(runtime, manifest) | |
| verify_model(model) | |
| if prefix.exists() and not replace: | |
| raise InstallError(f"install already exists (use --replace): {prefix}") | |
| prefix.parent.mkdir(parents=True, exist_ok=True) | |
| staging = Path(tempfile.mkdtemp(prefix=f".{prefix.name}-", dir=prefix.parent)) | |
| try: | |
| shutil.copytree(runtime, staging, dirs_exist_ok=True, symlinks=True) | |
| destination = staging / "model" / MODEL_NAME | |
| destination.parent.mkdir(parents=True, exist_ok=True) | |
| shutil.copy2(model, destination) | |
| receipt = { | |
| "schema_version": 1, | |
| "installed_by": "BTL-3 consumer installer", | |
| "platform": platform.platform(), | |
| "runtime_target": manifest.get("target") or manifest.get("platform"), | |
| "model": { | |
| "path": f"model/{MODEL_NAME}", | |
| "bytes": MODEL_BYTES, | |
| "sha256": MODEL_SHA256, | |
| }, | |
| } | |
| (staging / "install-receipt.json").write_text( | |
| json.dumps(receipt, indent=2) + "\n" | |
| ) | |
| if prefix.exists(): | |
| backup = prefix.with_name(f".{prefix.name}.previous") | |
| shutil.rmtree(backup, ignore_errors=True) | |
| prefix.rename(backup) | |
| try: | |
| staging.rename(prefix) | |
| except Exception: | |
| backup.rename(prefix) | |
| raise | |
| shutil.rmtree(backup) | |
| else: | |
| staging.rename(prefix) | |
| except Exception: | |
| shutil.rmtree(staging, ignore_errors=True) | |
| raise | |
| return prefix | |
| def main() -> None: | |
| parser = argparse.ArgumentParser() | |
| parser.add_argument("--runtime", type=Path, required=True) | |
| parser.add_argument("--model", type=Path, required=True) | |
| parser.add_argument("--prefix", type=Path, default=None) | |
| parser.add_argument("--replace", action="store_true") | |
| args = parser.parse_args() | |
| try: | |
| print(install( | |
| args.runtime, | |
| args.model, | |
| args.prefix or default_prefix(), | |
| args.replace, | |
| )) | |
| except InstallError as error: | |
| parser.error(str(error)) | |
| if __name__ == "__main__": | |
| main() | |