Instructions to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M

Use Docker

docker model run hf.co/aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M

LM Studio
Jan

vLLM

How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aogavrilov/diffusiongemma-agent-iq3-cuda13"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aogavrilov/diffusiongemma-agent-iq3-cuda13",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M

Ollama
How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Ollama:
```
ollama run hf.co/aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
```

Unsloth Studio

How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for aogavrilov/diffusiongemma-agent-iq3-cuda13 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for aogavrilov/diffusiongemma-agent-iq3-cuda13 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for aogavrilov/diffusiongemma-agent-iq3-cuda13 to start chatting

How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Docker Model Runner:
```
docker model run hf.co/aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
```

Lemonade

How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M

Run and chat with the model

lemonade run user.diffusiongemma-agent-iq3-cuda13-Q4_K_M

List all available models

lemonade list

diffusiongemma-agent-iq3-cuda13 / payload /app /docs /windows_wsl_agent_runtime.md

aogavrilov

Add files using upload-large-folder tool

ef2127a verified 18 days ago

preview code

Raw

History Blame Contribute Delete

5.5 kB

Windows + WSL Local Agent Runtime

This checkout runs the GPU model in WSL and exposes the agent gateway to Windows clients. The gateway keeps the inference service on 4100 unchanged and adds bounded repository tools on 8090.

Start

From PowerShell in this repository:

.\scripts\start_agent_gateway.ps1

It starts the gateway through WSL with the working Python environment at /root/diffusiongemma-agent/.venv-wsl/bin/python. The gateway forwards model requests to http://127.0.0.1:4100/v1 and is healthy at:

http://127.0.0.1:8090/healthz

Connect an IDE

Generate the project-local MCP configuration from PowerShell:

python scripts\dg_agent.py client-init --repo C:\path\to\project --client cursor --no-oss-stack

Replace cursor with claude-code, claude-desktop, or vscode as needed. The generated primary MCP server uses wsl.exe and passes both the Windows repository location and the WSL Python path explicitly. It therefore works without copying the project into /root/diffusiongemma-agent.

Use --no-oss-stack when the client needs only the primary MCP bridge. It provides repository status, files, search, bounded file reads, Git diff, context, RAG context, plans, sessions, verification, client reports, and preserved artifacts. Scoped edits can use the separate WSL Aider runtime described below; the optional Serena semantic MCP server also runs through WSL.

Agent Commands

Run a safe inspection task through WSL:

wsl.exe --exec bash -lc "cd /mnt/c/Users/alexg/Downloads/diffusiongemma-agent && DG_AGENT_PYTHON=/root/diffusiongemma-agent/.venv-wsl/bin/python ./scripts/dg_agent.sh agent --repo /mnt/c/path/to/project --task 'Find the relevant implementation' --mode read --json"

The read route first tries the model tool loop. If the current diffusion model does not emit a usable final answer or tool call, it returns a deterministic repository context pack and records both the model transcript and fallback artifacts under runlogs/dg-agent-runs/. This prevents an incomplete internal thought from being presented as an answer.

Task Plans

dg_agent.sh task is available in the WSL runtime again. It executes the existing plan format one bounded step at a time, stores an aggregate report and per-step supervisor reports, and rejects paths outside the repository. A dry-run validates and prints the planned supervisor commands without changing files:

wsl.exe --exec bash -lc "cd /mnt/c/Users/alexg/Downloads/diffusiongemma-agent && DG_AGENT_PYTHON=/root/diffusiongemma-agent/.venv-wsl/bin/python ./scripts/dg_agent.sh task --repo /mnt/c/path/to/project --plan /mnt/c/path/to/plan.json --dry-run"

Non-dry runs refuse a dirty Git worktree unless --allow-dirty is explicit. With --rollback-on-failure, the runner reverses only the tracked diff made by the task when it started from a clean tree; it does not reset the repository or delete untracked files.

For small tasks with an exact Python return or replacement constraint, the already checked-in deterministic supervisor can make and verify the change without a model edit. Other scoped code edits use Aider through scripts/run_aider_local.sh; it applies only the selected files, then the supervisor checks syntax and the requested test command. Aider history is kept outside the target repository and Python syntax validation runs in memory, so the runner does not create .aider* files or __pycache__.

OpenCode on Windows

The supported ready-made OSS agent shell on this host is upstream OpenCode, installed locally under .tools/opencode. Install it and start the existing GPU gateway from this checkout:

.\scripts\install_opencode_windows.ps1
.\scripts\start_agent_gateway.ps1

Then start it from the target repository, not from this agent repository:

Set-Location C:\path\to\project
C:\Users\alexg\Downloads\diffusiongemma-agent\scripts\run_opencode_windows.ps1

The launcher adds the working dg_agent MCP server through WSL and uses the safe gateway on http://127.0.0.1:8090/v1. Live model limits are 768 input tokens and 256 output tokens. It supports bounded inspection and scoped edit tasks, with preserved run artifacts; see docs/opencode_local.md for commands and current limits.

Qwen Code on Windows

The optional Qwen Code 0.19.10 CLI is installed locally and runs through a private Node 22.17.1 runtime because system Node 24 aborts on Qwen shutdown. The unified launcher prefers the WSL runner with DG, Repomix, and Serena MCP; native PowerShell remains an explicit read-only fallback on the safe GPU gateway. See docs/qwen_code_local.md; use Aider/session for edits.

Current Optional Gaps

The official MCP SDK is working in the WSL runtime. Native WSL Node.js is not required for the Windows OpenCode launcher, which uses the installed Windows Node runtime and invokes WSL only for dg_agent. Aider 0.86.2 is installed in a separate portable Python 3.12 environment at /root/diffusiongemma-agent/.venv-aider; the gateway remains on Python 3.14. preflight reports the core MCP/gateway runtime as ready and lists optional integrations separately. The model service and primary gateway do not depend on them.

OpenCode intentionally mounts only the DG MCP server to keep its tool schema bounded for the small-context model. IDE client profiles can additionally mount the working WSL Serena runtime.