Instructions to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M # Run inference directly in the terminal: llama cli -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M # Run inference directly in the terminal: llama cli -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
Use Docker
docker model run hf.co/aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "aogavrilov/diffusiongemma-agent-iq3-cuda13" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "aogavrilov/diffusiongemma-agent-iq3-cuda13", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
- Ollama
How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Ollama:
ollama run hf.co/aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
- Unsloth Studio
How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for aogavrilov/diffusiongemma-agent-iq3-cuda13 to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for aogavrilov/diffusiongemma-agent-iq3-cuda13 to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for aogavrilov/diffusiongemma-agent-iq3-cuda13 to start chatting
- Pi
How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
Run Hermes
hermes
- Atomic Chat new
- OpenClaw new
How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Docker Model Runner:
docker model run hf.co/aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
- Lemonade
How to use aogavrilov/diffusiongemma-agent-iq3-cuda13 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull aogavrilov/diffusiongemma-agent-iq3-cuda13:Q4_K_M
Run and chat with the model
lemonade run user.diffusiongemma-agent-iq3-cuda13-Q4_K_M
List all available models
lemonade list
DG MCP Server
The MCP layer is a local stdio bridge around the existing DiffusionGemma agent commands. It is meant for MCP-capable IDEs and OSS agents that can launch a server process and call tools.
The default server uses the official modelcontextprotocol/python-sdk
FastMCP implementation. A dependency-free legacy JSON-RPC fallback is kept for
debugging.
Run it directly:
cd /root/diffusiongemma-agent
scripts/dg_agent.sh mcp --list-tools
scripts/dg_agent.sh mcp
scripts/dg_agent.sh mcp --legacy
Repo-local launchers are created by:
scripts/dg_agent.sh workspace-init --repo /path/to/repo
Then use:
/path/to/repo/.dg-agent/bin/mcp --list-tools
/path/to/repo/.dg-agent/bin/mcp
/path/to/repo/.dg-agent/bin/mcp-http --host 127.0.0.1 --port 8765
/path/to/repo/.dg-agent/bin/serena-mcp --help-local
/path/to/repo/.dg-agent/bin/opencode-mcp --help
/path/to/repo/.dg-agent/bin/opencode-acp --help
/path/to/repo/.dg-agent/bin/goose-mcp --help-local
/path/to/repo/.dg-agent/bin/goose-acp --help
/path/to/repo/.dg-agent/bin/goose-serve --help
The checked-in OpenCode MCP profile mounts dg_agent and repomix. Serena is
available as a separate IDE MCP entry because its Pyright startup exceeds
OpenCode's fixed connection timeout on this host.
The checked-in Goose MCP profile mounts dg_agent and serena.
Client config:
configs/client_profiles/mcp-server.json
configs/client_profiles/mcp-client-snippets.json
configs/client_profiles/claude-code.mcp.json
configs/client_profiles/claude-desktop-mcp.json
configs/client_profiles/cursor.mcp.json
configs/client_profiles/vscode.mcp.json
Workspace copy:
.dg-agent/mcp-server.json
.dg-agent/mcp-client-snippets.json
.dg-agent/claude-code.mcp.json
.dg-agent/claude-desktop-mcp.json
.dg-agent/cursor.mcp.json
.dg-agent/vscode.mcp.json
.dg-agent/goose-mcp.dg.yaml
Client targets:
Claude Code project: .mcp.json <- .dg-agent/claude-code.mcp.json
Claude Desktop: claude_desktop_config.json <- .dg-agent/claude-desktop-mcp.json
Cursor project: .cursor/mcp.json <- .dg-agent/cursor.mcp.json
VS Code workspace: .vscode/mcp.json <- .dg-agent/vscode.mcp.json
mcp-client-snippets.json preserves all of those shapes in one manifest. The
Cursor/Claude-family configs use top-level mcpServers; VS Code uses top-level
servers with a stdio server type. The snippets also include optional native
Repomix MCP server entries through scripts/run_repomix_mcp.sh for clients
and Serena MCP server entries through scripts/run_serena_mcp.sh for clients
that should use upstream repository-packing and semantic/LSP tools directly
alongside the DG tools.
For clients that cannot launch a local stdio process, run the same DG MCP server over the official SDK streamable HTTP transport:
scripts/dg_agent.sh mcp-http -- --host 127.0.0.1 --port 8765
Endpoint:
http://127.0.0.1:8765/mcp
You can also merge the local server entry into a repo client config without overwriting unrelated servers:
scripts/dg_agent.sh mcp-client-config --repo /path/to/repo --client claude-code
scripts/dg_agent.sh mcp-client-config --repo /path/to/repo --client cursor
scripts/dg_agent.sh mcp-client-config --repo /path/to/repo --client cursor --with-repomix
scripts/dg_agent.sh mcp-client-config --repo /path/to/repo --client cursor --with-serena
scripts/dg_agent.sh mcp-client-config --repo /path/to/repo --client cursor --with-repomix --with-serena
scripts/dg_agent.sh mcp-client-config --repo /path/to/repo --client cursor --with-oss-stack
scripts/dg_agent.sh mcp-client-config --repo /path/to/repo --client vscode
scripts/dg_agent.sh mcp-client-config --client claude-desktop --target /path/to/claude_desktop_config.json
After workspace-init, the same command is available as:
.dg-agent/bin/mcp-client-config --client cursor
.dg-agent/bin/mcp-client-config --client cursor --with-repomix
.dg-agent/bin/mcp-client-config --client cursor --with-serena
.dg-agent/bin/mcp-client-config --client cursor --with-oss-stack
Serena is an upstream semantic/LSP MCP server. Install and smoke-test it with:
scripts/install_serena_local.sh
scripts/dg_agent.sh smoke --suite serena-mcp --timeout 300
On Windows, use scripts/run_serena_mcp.sh rather than a direct Serena binary.
The runner prefers a WSL bridge with local Linux Node/Pyright so semantic tools
avoid blocked Windows .exe shims and native .pyd wheels. The smoke includes
a live get_symbols_overview call, so it validates the Pyright-backed symbol
path as well as MCP startup.
Run it over stdio from a target repo:
.dg-agent/bin/serena-mcp
Or expose Serena over streamable HTTP when a client cannot spawn stdio:
.dg-agent/bin/serena-mcp --transport streamable-http --port 9121
For clients that read project instruction files, install matching DG/MCP usage rules:
.dg-agent/bin/agent-rules --target all
Exposed tools:
dg_repo_status: inspectgit status --short, diff stat, and untracked files.dg_list_files: list repository files throughrg --fileswith git fallback.dg_code_outline: bounded symbol outline with upstream ast-grep outline.dg_search: bounded ripgrep search with line and column locations.dg_ast_grep: bounded structural code search with upstream ast-grep.dg_read_file: bounded line-numbered file reads inside the repo.dg_git_diff: bounded git diff or diff stat reads.dg_task_note: save durable Markdown task notes underrunlogs/.dg_task_notes: list or read saved task notes.dg_status: check backend, Aider proxy, and LiteLLM health.dg_context: build the bounded repo context pack for a task.dg_rag_context: retrieve compact read-only RAG context without calling the model.dg_rag_answer: ask the local model over compact retrieved repo context.dg_repo_pack: pack filtered repo content with upstream Repomix.dg_repo_map: build a bounded upstream Aider repo-map for codebase context.dg_preflight: check repo workspace, wrappers, services, and GPU readiness.dg_plan: generate a task-runner JSON plan from a natural-language task.dg_task: execute an existing task-runner plan; usedry_runfor inspection.dg_session: run context -> plan -> task -> verify with rollback on failure.dg_verify: run or infer a repo verification command.dg_capabilities: read or run wrapper capability probes.dg_client_smoke: prepare or validate a target repo for external IDE/agent clients.dg_client_report: generate.dg-agent/CLIENT_HANDOFF.mdand.dg-agent/client-handoff.json.dg_sessions: list recent artifacted DG agent sessions.dg_session_artifact: read a preserved session artifact, defaulting to latest.
Exposed resources:
dg://client-pack: current endpoints, profiles, launchers, and limits.dg://status: live backend/proxy/LiteLLM health snapshot.dg://usage: short Markdown usage guide for local agent workflows.dg://notes: recent task notes saved by MCP clients.dg://notes/latest: latest task note.dg://sessions: recent session list.dg://sessions/latest: latestsession.json.dg://sessions/latest/diff: latestfinal.diff.dg://capabilities/latest: latest saved capability report.dg://client-handoff: repo-local handoff JSON generated bydg_client_report.dg://client-handoff/markdown: repo-local handoff Markdown generated bydg_client_report.dg://agent-hub: repo-local agent hub JSON generated byworkspace-init.dg://agent-hub/markdown: repo-local first-read agent hub Markdown.dg://command-kit: repo-local command kit JSON for reusable workflows.dg://command-kit/markdown: repo-local command kit Markdown.dg://ide-clients: repo-local IDE client snippet JSON.dg://ide-clients/markdown: repo-local IDE client profile guide.dg://codex-profile: repo-local Codex CLI profile guide.dg://codex-profile/config: repo-local Codex CLI config template.
Exposed prompts:
dg_agent_session: guide an MCP client through context -> session -> verify.dg_agent_context: guide an MCP client to gather bounded repo context before editing.dg_agent_continue_latest: guide an MCP client to inspect latest artifacts before continuing.
The server runs over MCP stdio through the official Python SDK. It deliberately
maps tools to existing reliable wrapper commands instead of relying on raw
model-generated tool syntax. The smoke test uses the SDK client
ClientSession over stdio_client, so it checks real MCP compatibility rather
than only hand-written JSON.
For full IDE-agent use, start with the repo tools (dg_repo_status,
dg_list_files, dg_repo_map, dg_code_outline, dg_search, dg_ast_grep, dg_read_file, dg_git_diff), save handoff state
with dg_task_note when useful, then run dg_preflight and dg_context.
For external clients, call dg_client_smoke or dg_client_report first; the
latter writes and exposes the repo-local handoff through dg://client-handoff.
For first-time bootstrap, read dg://agent-hub/markdown and then the matching
command kit, IDE, or Codex resource instead of manually opening repo files.
For larger repository questions, use dg_rag_context before asking the model or
planning edits. Use dg_repo_map for an Aider-style repository sketch, then dg_code_outline when a symbol map is enough to choose files or functions before reading source. Use dg_ast_grep for language-aware code patterns such as
returns, decorators, call sites, and declarations. Use dg_repo_pack with tight include filters when a Repomix
packed artifact is better than ranked snippets. Use dg_session for one-shot bounded edits. Use dg_plan followed by
dg_task when the client should review or store the plan before executing it.
Verification:
scripts/dg_agent.sh smoke --suite mcp --timeout 120
scripts/dg_agent.sh smoke --suite goose-mcp --timeout 180
scripts/dg_agent.sh smoke --suite goose-acp --timeout 180