# SoniCoder — Project Memory

> This file is the project's persistent memory. The agent reads it on every session.
> Edit it freely — it overrides defaults.

## What is SoniCoder?

SoniCoder is a local-first AI coding agent that can:
- Generate complete fullstack applications in any language/framework
- Read, write, and edit files in a sandboxed workspace
- Run shell commands (git, npm, pip, tests)
- Apply specialized skills (frontend-design, feature-dev, code-review, debugging, fullstack-scaffold, commit-workflow)
- Respond to slash commands (/commit, /review, /feature, /design, /explain, /test, /refactor, /skill, /help)
- Deploy to HuggingFace Spaces with one click

## Architecture

```
app.py                          ← Entry point: launches Gradio Server
code/
├── config/constants.py         ← App config, system prompt, language options
├── model/
│   ├── loader.py               ← Dual model loading (text + VLM)
│   └── inference.py            ← Streaming inference (text + VLM)
├── agent/__init__.py           ← Agent loop (model ↔ tools, supports custom agents)
├── tools/
│   ├── fs.py                   ← read_file, write_file, edit_file, glob, grep, list_dir
│   ├── bash.py                 ← Sandboxed shell execution
│   ├── todos.py                ← Todo list management
│   └── github.py               ← GitHub repo import (shallow clone + strip heavy dirs)
├── skills/
│   ├── __init__.py             ← Skill discovery + loading
│   └── builtins/               ← Built-in skills (markdown)
├── agents/                     ← Custom Agent system (AI-generated personas)
│   ├── __init__.py             ← Agent CRUD + system-prompt builder
│   └── builtins/               ← Built-in agents (code-reviewer, test-writer)
├── commands/
│   ├── __init__.py             ← Slash command parser + expander
│   └── builtins/               ← Built-in commands (markdown, includes /agent and /github)
├── hooks/
│   ├── __init__.py             ← Hook rule engine
│   └── builtins/               ← Built-in hook rules (markdown)
├── execution/
│   ├── code_extractor.py       ← Code extraction from model output
│   ├── python_runner.py        ← Sandboxed Python execution
│   └── gradio_runner.py        ← Gradio app subprocess runner
├── huggingface/
│   ├── dockerfile_gen.py       ← Auto Dockerfile/package.json for JS
│   └── push.py                 ← HF Hub push + ZIP packaging
├── websearch/google_scraper.py ← DuckDuckGo + Google scraping (no API)
└── server/
    ├── chat_helpers.py         ← Chat history + prompt building
    └── routes.py               ← All HTTP + API endpoints
index.html                      ← Frontend (single-file SPA)
workspace/                      ← Sandboxed agent workspace (auto-created)
```

## Conventions

- **Python**: 3.11+, type hints everywhere, `from __future__ import annotations`
- **Style**: Black formatting, 4-space indent, 100 char line limit
- **Docstrings**: Google style for modules, functions, classes
- **Error handling**: catch specific exceptions, never bare `except:`
- **Logging**: use `logging.getLogger(__name__)`, never `print()`
- **Tests**: pytest, in `tests/` directory, `test_*.py` naming
- **Frontend**: single-file HTML with inline CSS/JS, no build step

## Server rules

- All servers bind to `0.0.0.0` (never `localhost`)
- Default port: `7860` (HF Spaces convention)
- Sub-servers use `7861`, `7862`, etc.

## Model

- Default: `openbmb/MiniCPM5-1B` (text-only, 2.17 GB)
- Optional: `openbmb/MiniCPM-V-4.6` (vision + text, 2.8 GB)
- Loaded in background thread on startup
- Streaming inference via `TextIteratorStreamer`

## Tool call format

The model calls tools by emitting fenced code blocks with `tool` as the language:

```tool
read_file
path: src/app.py
```

Multi-line values use YAML block scalars:

```tool
write_file
path: src/new.py
content: |
  import os
  def main():
      pass
```

## Slash commands

| Command | Description |
|---------|-------------|
| `/commit` | Create a git commit with a generated message |
| `/review` | Review current changes for bugs and quality |
| `/feature <desc>` | Guided feature development workflow |
| `/design <brief>` | Generate a distinctive frontend design |
| `/explain <target>` | Explain how code works |
| `/test <target>` | Generate tests |
| `/refactor <target>` | Refactor code for clarity |
| `/skill <name>` | Load and apply a skill |
| `/agent create <desc>` | AI generates a custom agent from natural-language description |
| `/agent use <name>` | Activate a saved agent |
| `/agent list` | List all saved agents |
| `/agent show <name>` | Show an agent's full definition |
| `/agent delete <name>` | Delete a user-defined agent |
| `/agent reset` | Reset to default SoniCoder persona |
| `/github <url> [subdir] [--branch <name>] [--into <path>]` | Import a GitHub repo into the workspace |
| `/help` | Show available commands and skills |

## Custom Agents

Custom agents are AI-generated personas layered on top of the base SoniCoder
system prompt. They can restrict the tool whitelist, auto-load skills, and
override temperature / max-iterations.

### Agent file format

Agents live in `workspace/.sonicoder/agents/<name>/AGENT.md` (built-ins in
`code/agents/builtins/<name>/AGENT.md`). Format:

```markdown
---
name: my-agent
description: One-line description
tools: read_file, list_dir, grep, bash
skills: code-review
temperature: 0.2
max_iterations: 12
tags: review, quality
author: AI-generated
created: 2026-06-20
---

# My Agent

Full system-prompt extension here. Define the persona, workflow, output format,
and any hard rules.
```

### How `/agent create` works

1. User runs `/agent create a security reviewer that flags hardcoded secrets`.
2. The slash-command expansion substitutes `AGENT_GENERATION_PROMPT` (defined
   in `code/agents/__init__.py`) into the prompt.
3. The base SoniCoder model (NOT a custom agent) authors an `AGENT.md` file
   via `write_file` and saves it under `.sonicoder/agents/<name>/`.
4. The user can then `/agent use <name>` or click the agent in the UI.

### Built-in agents

| Agent | Description |
|-------|-------------|
| `code-reviewer` | Read-only reviewer, structured issues table output |
| `test-writer` | Generates pytest/jest tests, runs them, iterates until green |

### API endpoints

| Endpoint | Description |
|----------|-------------|
| `list_agents` | List all agents (builtins + user) and the active one |
| `get_agent(name)` | Get a single agent's full definition |
| `save_agent(...)` | Create or overwrite a user agent (manual save) |
| `delete_agent(name)` | Delete a user agent (built-ins protected) |
| `set_active_agent(name)` | Set/clear the active agent for subsequent prompts |
| `import_github(url, branch, subdir, target_subdir, depth, timeout)` | Clone a GitHub repo into the workspace (shallow, heavy dirs stripped) |
| `github_url_examples()` | Return accepted GitHub URL formats |
| `push_github(repo_name, github_token, username, branch?, commit_message?, timeout?)` | Snapshot workspace → commit → push to a GitHub repo |

The `agent_run` endpoint also intercepts `/agent use|reset|delete|list` and
dispatches them directly to the agents module, bypassing the model entirely
for instant session-state updates.

## GitHub Import

SoniCoder can clone any public GitHub repo into the workspace, allowing the
agent to read, edit, extend, or refactor real-world code.

### How it works

1. User submits a GitHub URL via the Agent tab UI box (or via `/github <url>`
   slash command in chat while Agent mode is ON).
2. The backend (`code/tools/github.py::import_github_repo`) parses the URL
   (supports HTTPS, SSH, and `/tree/<branch>/<subdir>` forms) and validates
   that the host is `github.com`.
3. The repo is shallow-cloned (`git clone --depth 1 --single-branch`) into a
   temp directory.
4. Files are *copied* into the workspace (root or `target_subdir`) with these
   directories stripped: `.git`, `.hg`, `.svn`, `node_modules`, `__pycache__`,
   `.venv`, `venv`, `env`, `.tox`, `.mypy_cache`, `.pytest_cache`,
   `.ruff_cache`, `dist`, `build`, `.next`, `.nuxt`, `.cache`, `.gradle`,
   `target`, `Pods`. `.DS_Store` and `Thumbs.db` are also dropped.
5. The workspace tree refreshes; Agent mode auto-enables if it wasn't already.

### Security

- Only `github.com` URLs are accepted (HTTPS or SSH form).
- `target_subdir` is sanitized — no path escapes.
- The upstream repo is never modified (clone happens in temp dir, then
  copied). The `.git` directory is stripped so the agent doesn't walk it.
- Default clone timeout: 120s (UI uses 180s). Max: 600s.

### Slash command

```
/github <url> [subdir] [--branch <name>] [--into <path>] [--depth <N>] [--timeout <s>]
```

The slash command (defined in `code/commands/builtins/github.md`) instructs
the agent to invoke `import_github_repo` via bash, then list the top-level
files and suggest next steps based on what was imported.

## GitHub Push

Push the current workspace back to a GitHub repo as a commit. Only 3 inputs
are required: repo name, GitHub token, username.

### How it works (`code/tools/github.py::push_to_github`)

1. Snapshot the workspace via `snapshot_workspace()` (returns a
   `{relative_path: content}` dict).
2. Create a temp dir, `git init -b <branch>` inside it, write the snapshot
   files in, `git add -A` + `git commit -m <message>`.
3. Build a push URL of the form
   `https://<username>:<token>@github.com/<owner>/<repo>.git` and run
   `git push --force-with-lease <url> <branch>`.
4. If `--force-with-lease` fails because the remote has no refs yet
   (brand-new empty repo), retry with a plain `git push`.
5. Delete the temp dir. The token is never logged; error messages scrub it
   before being returned.

### Why `--force-with-lease`

SoniCoder treats the workspace as the source of truth. `--force-with-lease`
replaces the remote tip with the workspace snapshot, but fails loudly (rather
than silently clobbering) if someone else pushed commits in the meantime —
because the temp repo has no reflog of the remote tip, the lease fails and
the user is told to pull first.

### UI

Located in the **Deploy** tab, in a "Push Update to GitHub" section below the
HuggingFace section. Only 3 fields are required: repo name, token, username.
An "Advanced" `<details>` exposes optional `branch` and `commit_message`
fields.

### Security

- Token is sent over HTTPS to the SoniCoder backend, used once for the push,
  then dropped (not stored, not logged).
- Error messages are scrubbed to remove the token before being returned.
- The temp repo is deleted at the end of the call (context manager).
- The local SoniCoder workspace is never turned into a git repo; the
  workspace's `.git` (if any, e.g. after an import — though imports strip
  `.git`) is never read.

## Skills

| Skill | Description |
|-------|-------------|
| `frontend-design` | Distinctive visual design guidance |
| `feature-dev` | Guided feature implementation workflow |
| `code-review` | High-signal code review |
| `debugging` | Systematic debugging workflow |
| `fullstack-scaffold` | Project structure scaffolding rules |
| `commit-workflow` | Git commit best practices |

## Hooks

Hooks are markdown rules that fire on events (`bash`, `file`, `prompt`, `stop`).
They can `warn` (show a message) or `block` (prevent the action).

Built-in hooks:
- `block-dangerous-rm` — blocks `rm -rf /`, `~`, `$HOME`, `..`
- `warn-debug-code` — warns on `console.log`, `debugger`, `print`, `alert`
- `warn-secrets-in-code` — warns on hardcoded API_KEY/SECRET/TOKEN/PASSWORD
- `warn-eval-exec` — warns on `eval()` and `exec()`

Users can add custom hooks in `workspace/.sonicoder/hooks/*.local.md`.

## Workspace

The agent's sandboxed filesystem lives at `./workspace/` (configurable via
`SONICODER_WORKSPACE` env var). All file tools refuse paths that escape this root.

## Deploy

Generated projects can be pushed to HuggingFace Spaces via the Deploy tab.
Supported SDKs:
- `static` — HTML/CSS/JS
- `gradio` — Python Gradio apps
- `streamlit` — Python Streamlit apps
- `docker` — JS/TS frameworks (auto-generates Dockerfile + package.json)