Instructions to use leeroy-jankins/gipity with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use leeroy-jankins/gipity with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("leeroy-jankins/gipity", dtype="auto")

llama-cpp-python

How to use leeroy-jankins/gipity with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="leeroy-jankins/gipity",
	filename="gipity-oss-20b-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use leeroy-jankins/gipity with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf leeroy-jankins/gipity:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf leeroy-jankins/gipity:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf leeroy-jankins/gipity:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf leeroy-jankins/gipity:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf leeroy-jankins/gipity:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf leeroy-jankins/gipity:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf leeroy-jankins/gipity:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf leeroy-jankins/gipity:Q4_K_M

Use Docker

docker model run hf.co/leeroy-jankins/gipity:Q4_K_M

LM Studio
Jan
Ollama
How to use leeroy-jankins/gipity with Ollama:
```
ollama run hf.co/leeroy-jankins/gipity:Q4_K_M
```

Unsloth Studio new

How to use leeroy-jankins/gipity with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for leeroy-jankins/gipity to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for leeroy-jankins/gipity to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for leeroy-jankins/gipity to start chatting

Pi new

How to use leeroy-jankins/gipity with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf leeroy-jankins/gipity:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "leeroy-jankins/gipity:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use leeroy-jankins/gipity with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf leeroy-jankins/gipity:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default leeroy-jankins/gipity:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use leeroy-jankins/gipity with Docker Model Runner:
```
docker model run hf.co/leeroy-jankins/gipity:Q4_K_M
```

Lemonade

How to use leeroy-jankins/gipity with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull leeroy-jankins/gipity:Q4_K_M

Run and chat with the model

lemonade run user.gipity-Q4_K_M

List all available models

lemonade list

Gipity (gipity-oss-20b-Q4_K_XL.GGUF) is a fine-tuned LLM based on OpenAI’s Chat GPT-5. This release packages the fine-tuned weights (or adapters) for practical, low-latency instruction following, summarization, reasoning, and light code generation. It is intended for local or self-hosted environments and RAG (Retrieval-Augmented Generation) stacks that require predictable, fast outputs.

Quantized, and fine-tuned GGUF based on OpenAI’s gpt-oss-20b Format: GGUF (for llama.cpp and compatible runtimes) • Quantization: Q4_K_XL (4-bit, K-grouped, extra-low loss) Gipity is a multimodal LLM for AI workflows based on OpenAI GPT-5.x. It is designed to provide a unified workspace for text, image and vision, audio, embeddings, files, vector stores, prompt engineering, and document-grounded analysis that comes with an optional UI.

📥 Download the Gipity Model

Download the GGUF file:
```
gipity-oss-20b.Q4_K_M.gguf
```

Place the file anywhere on your system, for example:

C:\Users\<you>\leeroy-jankins\gipity\gipity-oss-20b.Q4_K_M.gguf

⚙️ Streamlit UI

Highlights

Permissive Apache 2.0 license: Build freely without copyleft restrictions or patent risk—ideal for experimentation, customization, and commercial deployment.
Configurable reasoning effort: Easily adjust the reasoning effort (low, medium, high) based on your specific use case and latency needs.
Full chain-of-thought: Gain complete access to the model’s reasoning process, facilitating easier debugging and increased trust in outputs. It’s not intended to be shown to end users.
Fine-tunable: Fully customize models to your specific use case through parameter fine-tuning.
Agentic capabilities: Use the models’ native capabilities for function calling, web browsing, Python code execution, and Structured Outputs.
Native MXFP4 quantization: The models are trained with native MXFP4 precision for the MoE layer, making gipity-oss-20b model run within 16GB of memory.

⚙️ Datasets

Vectorization is the process of converting textual data into numerical vectors and is a process that is usually applied once the text is cleaned. It can help improve the execution speed and reduce the training time of your code. BudgetPy provides the following vector stores on the OpenAI platform to support environmental data analysis with machine-learning

Appropriations - Enacted appropriations from 1996-2024 available for fine-tuning learning models
Regulations - Collection of federal regulations on the use of appropriated funds
SF-133 - The Report on Budget Execution and Budgetary Resources
Balances - U.S. federal agency Account Balances (File A) submitted as part of the DATA Act 2014.
Outlays - The actual disbursements of funds by the U.S. federal government from 1962 to 2025
Circular A11 - Guidance from OMB on the preparation, submission, and execution of the federal budget
Fastbook - Treasury guidance on federal ledger accounts
Title 31 CFR - Money & Finance
Redbook - The Principles of Appropriations Law (Volumes I & II).
US Standard General Ledger - Account Definitions
Treasury Appropriation Fund Symbols (TAFSs) Dataset - Collection of TAFSs used by federal agencies

Base Model Details

Read our How to GPT Guide here!

See our collection for all versions of gpt-oss including GGUF, 4-bit & 16-bit formats.

Learn to run gpt-oss correctly - Read the Guide.

See Dynamic 2.0 GGUFs for quantization benchmarks.

✨ Read our gpt-oss Guide here!

Fine-tune gpt-oss-20b for free using our Google Colab notebook
Read our Blog about gpt-oss support: unsloth.ai/blog/gpt-oss
View the rest of our notebooks in our docs here.
Thank you to the llama.cpp team for their work on supporting this model. We wouldn't be able to release quants without them!

The F32 quant is MXFP4 upcasted to BF16 for every single layer and is unquantized.

gpt-oss-20b

Try gpt-oss · Guides · System card · OpenAI blog

Inference examples

Transformers

You can use gipity-oss-20b with Transformers. If you use the Transformers chat template, it will automatically apply the harmony response format. If you use model.generate directly, you need to apply the harmony format manually using the chat template or use our openai-harmony package.

To get started, install the necessary dependencies to setup your environment:

pip install -U transformers kernels torch

Once, setup you can proceed to run the model by running the snippet below:

from transformers import pipeline
import torch

model_id = "leeroy-jankins/gipity-oss-20b"

pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Explain quantum mechanics clearly and concisely."},
]

outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Alternatively, you can run the model via Transformers Serve to spin up a OpenAI-compatible webserver:

transformers serve
transformers chat localhost:8000 --model-name-or-path leeroy-jankins/gipity-oss-20b

Learn more about how to use gpt-oss with Transformers.

vLLM

vLLM recommends using uv for Python dependency management. You can use vLLM to spin up an OpenAI-compatible webserver. The following command will automatically download the model and start the server.

uv pip install --pre vllm==0.10.1+gptoss \
    --extra-index-url https://wheels.vllm.ai/gpt-oss/ \
    --extra-index-url https://download.pytorch.org/whl/nightly/cu128 \
    --index-strategy unsafe-best-match

vllm serve openai/gipity-oss-20b

Learn more about how to use gipity-oss with vLLM.

PyTorch / Triton

To learn about how to use this model with PyTorch and Triton, check out our reference implementations in the gpt-oss repository.

Ollama

If you are trying to run gpt-oss on consumer hardware, you can use Ollama by running the following commands after installing Ollama.

# gipity-oss-20b
ollama pull gipity-oss:20b
ollama run gipity-oss:20b

Learn more about how to use gpt-oss with Ollama.

LM Studio

If you are using LM Studio you can use the following commands to download.

# gipity-oss-20b
lms get leeroy-jankins/gipity-oss-20b

Check out our awesome list for a broader collection of gpt-oss resources and inference partners.

Download the model

You can download the model weights from the Hugging Face Hub directly from Hugging Face CLI:

# gipity-oss-20b
huggingface-cli download leeroy-jankins/gipity-oss-20b --include "original/*" --local-dir gipity-oss-20b/
pip install gpt-oss
python -m gpt_oss.chat model/

Reasoning levels

You can adjust the reasoning level that suits your task across three levels:

Low: Fast responses for general dialogue.
Medium: Balanced speed and detail.
High: Deep and detailed analysis.

The reasoning level can be set in the system prompts, e.g., "Reasoning: high".

Tool use

The gpt-oss models are excellent for:

Web browsing (using built-in browsing tools)
Function calling with defined schemas
Agentic operations like browser tasks

Fine-tuning

Both gpt-oss models can be fine-tuned for a variety of specialized use cases.

This smaller model gipity-oss-20b can be fine-tuned on consumer hardware, whereas the larger gpt-oss-120b can be fine-tuned on a single H100 node.

Downloads last month: 13

GGUF

Model size

21B params

Architecture

gpt-oss

Hardware compatibility

4-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for leeroy-jankins/gipity

Base model

openai/gpt-oss-20b

Quantized

unsloth/gpt-oss-20b-GGUF