Buckets:
| name: python-debugpy | |
| description: "Debug Python: pdb REPL + debugpy remote (DAP)." | |
| version: 1.0.0 | |
| author: Hermes Agent | |
| license: MIT | |
| platforms: [linux, macos] | |
| metadata: | |
| hermes: | |
| tags: [debugging, python, pdb, debugpy, breakpoints, dap, post-mortem] | |
| related_skills: [systematic-debugging, node-inspect-debugger, debugging-hermes-tui-commands] | |
| # Python Debugger (pdb + debugpy) | |
| ## Overview | |
| Three tools, picked by situation: | |
| | Tool | When | | |
| |---|---| | |
| | **`breakpoint()` + pdb** | Local, interactive, simplest. Add `breakpoint()` in the source, run normally, get a REPL at that line. | | |
| | **`python -m pdb`** | Launch an existing script under pdb with no source edits. Useful for quick poking. | | |
| | **`debugpy`** | Remote / headless / "attach to already-running process." Talks DAP, scriptable from terminal, works for long-lived processes (gateway, daemon, PTY children). | | |
| **Start with `breakpoint()`.** It's the cheapest thing that works. | |
| ## When to Use | |
| - A test fails and the traceback doesn't reveal why a value is wrong | |
| - You need to step through a function and watch a collection mutate | |
| - A long-running process (hermes gateway, tui_gateway) misbehaves and you can't restart it | |
| - Post-mortem: an exception fired in prod-ish code and you want to inspect locals at the crash site | |
| - A subprocess / child (Python `_SlashWorker`, PTY bridge worker) is the actual bug site | |
| **Don't use for:** things `print()` / `logging.debug` solve in under a minute, or things `pytest -vv --tb=long --showlocals` already reveals. | |
| ## pdb Quick Reference | |
| Inside any pdb prompt (`(Pdb)`): | |
| | Command | Action | | |
| |---|---| | |
| | `h` / `h cmd` | help | | |
| | `n` | next line (step over) | | |
| | `s` | step into | | |
| | `r` | return from current function | | |
| | `c` | continue | | |
| | `unt N` | continue until line N | | |
| | `j N` | jump to line N (same function only) | | |
| | `l` / `ll` | list source around current line / full function | | |
| | `w` | where (stack trace) | | |
| | `u` / `d` | move up / down in the stack | | |
| | `a` | print args of the current function | | |
| | `p expr` / `pp expr` | print / pretty-print expression | | |
| | `display expr` | auto-print expr on every stop | | |
| | `b file:line` | set breakpoint | | |
| | `b func` | break on function entry | | |
| | `b file:line, cond` | conditional breakpoint | | |
| | `cl N` | clear breakpoint N | | |
| | `tbreak file:line` | one-shot breakpoint | | |
| | `!stmt` | execute arbitrary Python (assignments included) | | |
| | `interact` | drop into full Python REPL in current scope (Ctrl+D to exit) | | |
| | `q` | quit | | |
| The `interact` command is the most powerful — you can import anything, inspect complex objects, even call methods that mutate state. Locals are read-only by default; use `!x = 42` from the `(Pdb)` prompt to mutate. | |
| ## Recipe 1: Local breakpoint | |
| Easiest. Edit the file: | |
| ```python | |
| def compute(x, y): | |
| result = some_helper(x) | |
| breakpoint() # <-- drops into pdb here | |
| return result + y | |
| ``` | |
| Run the code normally. You land at the `breakpoint()` line with full access to locals. | |
| **Don't forget to remove `breakpoint()` before committing.** Use `git diff` or a pre-commit grep: | |
| ```bash | |
| rg -n 'breakpoint\(\)' --type py | |
| ``` | |
| ## Recipe 2: Launch a script under pdb (no source edits) | |
| ```bash | |
| python -m pdb path/to/script.py arg1 arg2 | |
| # Lands at first line of script | |
| (Pdb) b path/to/script.py:42 | |
| (Pdb) c | |
| ``` | |
| ## Recipe 3: Debug a pytest test | |
| The hermes test runner and pytest both support this: | |
| ```bash | |
| # Drop to pdb on failure (or on any raised exception): | |
| scripts/run_tests.sh tests/path/to/test_file.py::test_name --pdb | |
| # Drop to pdb at the START of the test: | |
| scripts/run_tests.sh tests/path/to/test_file.py::test_name --trace | |
| # Show locals in tracebacks without pdb: | |
| scripts/run_tests.sh tests/path/to/test_file.py --showlocals --tb=long | |
| ``` | |
| Note: `scripts/run_tests.sh` uses xdist (`-n 4`) by default, and pdb does NOT work under xdist. Add `-p no:xdist` or run a single test with `-n 0`: | |
| ```bash | |
| scripts/run_tests.sh tests/foo_test.py::test_bar --pdb -p no:xdist | |
| # or | |
| source .venv/bin/activate | |
| python -m pytest tests/foo_test.py::test_bar --pdb | |
| ``` | |
| This bypasses the hermetic-env guarantees — fine for debugging, but re-run under the wrapper to confirm before pushing. | |
| ## Recipe 4: Post-mortem on any exception | |
| ```python | |
| import pdb, sys | |
| try: | |
| run_the_thing() | |
| except Exception: | |
| pdb.post_mortem(sys.exc_info()[2]) | |
| ``` | |
| Or wrap a whole script: | |
| ```bash | |
| python -m pdb -c continue script.py | |
| # When it crashes, pdb catches it and you're in the frame of the exception | |
| ``` | |
| Or set a global hook in a repl/jupyter: | |
| ```python | |
| import sys | |
| def excepthook(etype, value, tb): | |
| import pdb; pdb.post_mortem(tb) | |
| sys.excepthook = excepthook | |
| ``` | |
| ## Recipe 5: Remote debug with debugpy (attach to running process) | |
| For long-lived processes: Hermes gateway, tui_gateway, a daemon, a process that's already misbehaving and can't be restarted clean. | |
| ### Setup | |
| ```bash | |
| source /home/bb/hermes-agent/.venv/bin/activate | |
| pip install debugpy | |
| ``` | |
| ### Pattern A: Source-edit — process waits for debugger at launch | |
| Add near the top of the entry point (or inside the function you want to debug): | |
| ```python | |
| import debugpy | |
| debugpy.listen(("127.0.0.1", 5678)) | |
| print("debugpy listening on 5678, waiting for client...", flush=True) | |
| debugpy.wait_for_client() | |
| debugpy.breakpoint() # optional: pause immediately once attached | |
| ``` | |
| Start the process; it blocks on `wait_for_client()`. | |
| ### Pattern B: No source edit — launch with `-m debugpy` | |
| ```bash | |
| python -m debugpy --listen 127.0.0.1:5678 --wait-for-client your_script.py arg1 | |
| ``` | |
| Equivalent for module entry: | |
| ```bash | |
| python -m debugpy --listen 127.0.0.1:5678 --wait-for-client -m your.module | |
| ``` | |
| ### Pattern C: Attach to an already-running process | |
| Needs the PID and debugpy preinstalled in the target's environment: | |
| ```bash | |
| python -m debugpy --listen 127.0.0.1:5678 --pid <pid> | |
| # debugpy injects itself into the process. Then attach a client as below. | |
| ``` | |
| Some kernels/security configs block the ptrace-based injection (`/proc/sys/kernel/yama/ptrace_scope`). Fix with: | |
| ```bash | |
| echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope | |
| ``` | |
| ### Connecting a client from the terminal | |
| The easiest terminal-side DAP client is VS Code CLI or a small script. From inside Hermes you have two practical options: | |
| **Option 1: `debugpy`'s own CLI REPL** — not an official feature, but a tiny DAP client script: | |
| ```python | |
| # /tmp/dap_client.py | |
| import socket, json, itertools, time, sys | |
| HOST, PORT = "127.0.0.1", 5678 | |
| s = socket.create_connection((HOST, PORT)) | |
| seq = itertools.count(1) | |
| def send(msg): | |
| msg["seq"] = next(seq) | |
| body = json.dumps(msg).encode() | |
| s.sendall(f"Content-Length: {len(body)}\r\n\r\n".encode() + body) | |
| def recv(): | |
| header = b"" | |
| while b"\r\n\r\n" not in header: | |
| header += s.recv(1) | |
| length = int(header.decode().split("Content-Length:")[1].split("\r\n")[0].strip()) | |
| body = b"" | |
| while len(body) < length: | |
| body += s.recv(length - len(body)) | |
| return json.loads(body) | |
| send({"type": "request", "command": "initialize", "arguments": {"adapterID": "python"}}) | |
| print(recv()) | |
| send({"type": "request", "command": "attach", "arguments": {}}) | |
| print(recv()) | |
| send({"type": "request", "command": "setBreakpoints", | |
| "arguments": {"source": {"path": sys.argv[1]}, | |
| "breakpoints": [{"line": int(sys.argv[2])}]}}) | |
| print(recv()) | |
| send({"type": "request", "command": "configurationDone"}) | |
| # ... loop reading events and sending continue/stepIn/etc. | |
| ``` | |
| This is fine for one-off automation but painful as an interactive UX. | |
| **Option 2: Attach from VS Code / Cursor / Zed** — if the user has one open, they can add a `launch.json`: | |
| ```json | |
| { | |
| "name": "Attach to Hermes", | |
| "type": "debugpy", | |
| "request": "attach", | |
| "connect": { "host": "127.0.0.1", "port": 5678 }, | |
| "justMyCode": false, | |
| "pathMappings": [ | |
| { "localRoot": "${workspaceFolder}", "remoteRoot": "/home/bb/hermes-agent" } | |
| ] | |
| } | |
| ``` | |
| **Option 3: Ditch DAP, use `remote-pdb`** — usually what you actually want from a terminal agent: | |
| ```bash | |
| pip install remote-pdb | |
| ``` | |
| In your code: | |
| ```python | |
| from remote_pdb import set_trace | |
| set_trace(host="127.0.0.1", port=4444) # blocks until connection | |
| ``` | |
| Then from the terminal: | |
| ```bash | |
| nc 127.0.0.1 4444 | |
| # You get a (Pdb) prompt exactly as if debugging locally. | |
| ``` | |
| `remote-pdb` is the cleanest agent-friendly choice when `debugpy`'s DAP protocol is overkill. Use `debugpy` only when you actually need IDE integration. | |
| ## Debugging Hermes-specific Processes | |
| ### Tests | |
| See Recipe 3. Always add `-p no:xdist` or run single tests without xdist. | |
| ### `run_agent.py` / CLI — one-shot | |
| Easiest: add `breakpoint()` near the suspect line, then run `hermes` normally. Control returns to your terminal at the pause point. | |
| ### `tui_gateway` subprocess (spawned by `hermes --tui`) | |
| The gateway runs as a child of the Node TUI. Options: | |
| **A. Source-edit the gateway:** | |
| ```python | |
| # tui_gateway/server.py near the top of serve() | |
| import debugpy | |
| debugpy.listen(("127.0.0.1", 5678)) | |
| debugpy.wait_for_client() | |
| ``` | |
| Start `hermes --tui`. The TUI will appear frozen (its backend is waiting). Attach a client; execution resumes when you `continue`. | |
| **B. Use `remote-pdb` at a specific handler:** | |
| ```python | |
| from remote_pdb import set_trace | |
| set_trace(host="127.0.0.1", port=4444) # in the RPC handler you want to trap | |
| ``` | |
| Trigger the matching slash command from the TUI, then `nc 127.0.0.1 4444` in another terminal. | |
| ### `_SlashWorker` subprocess | |
| Same pattern — `remote-pdb` with `set_trace()` inside the worker's `exec` path. The worker is persistent across slash commands, so the first trigger blocks until you connect; subsequent slash commands pass through normally unless you re-arm. | |
| ### Gateway (`gateway/run.py`) | |
| Long-lived. Use `remote-pdb` at a handler, or `debugpy` with `--wait-for-client` if you're restarting the gateway anyway. | |
| ## Common Pitfalls | |
| 1. **pdb under pytest-xdist silently does nothing.** You won't see the prompt, the test just hangs. Always use `-p no:xdist` or `-n 0`. | |
| 2. **`breakpoint()` in CI / non-TTY contexts hangs the process.** Safe locally; never commit it. Add a pre-commit grep as a safety net. | |
| 3. **`PYTHONBREAKPOINT=0`** disables all `breakpoint()` calls. Check the env if your breakpoint isn't hitting: | |
| ```bash | |
| echo $PYTHONBREAKPOINT | |
| ``` | |
| 4. **`debugpy.listen` blocks only if you also call `wait_for_client()`.** Without it, execution continues and your first breakpoint may fire before the client is attached. | |
| 5. **Attach to PID fails on hardened kernels.** `ptrace_scope=1` (Ubuntu default) allows only same-user ptrace of child processes. Workaround: `echo 0 > /proc/sys/kernel/yama/ptrace_scope` (needs root) or launch under `debugpy` from the start. | |
| 6. **Threads.** `pdb` only debugs the current thread. For multithreaded code, use `debugpy` (thread-aware DAP) or set `threading.settrace()` per thread. | |
| 7. **asyncio.** `pdb` works in coroutines but `await` inside pdb requires Python 3.13+ or `await` from `interact` mode on older versions. For 3.11/3.12, use `asyncio.run_coroutine_threadsafe` tricks or `!stmt`-based awaits via `asyncio.ensure_future`. | |
| 8. **`scripts/run_tests.sh` strips credentials and sets `HOME=<tmpdir>`.** If your bug depends on user config or real API keys, it won't reproduce under the wrapper. Debug with raw `pytest` first to repro, then re-confirm under the wrapper. | |
| 9. **Forking / multiprocessing.** pdb does not follow forks. Each child needs its own `breakpoint()` or `set_trace()`. For Hermes subagents, debug one process at a time. | |
| ## Verification Checklist | |
| - [ ] After `pip install debugpy`, confirm: `python -c "import debugpy; print(debugpy.__version__)"` | |
| - [ ] For remote debug, confirm the port is actually listening: `ss -tlnp | grep 5678` | |
| - [ ] First breakpoint actually hits (if it doesn't, you likely have `PYTHONBREAKPOINT=0`, you're under xdist, or execution finished before attach) | |
| - [ ] `where` / `w` shows the expected call stack | |
| - [ ] Post-debug cleanup: no stray `breakpoint()` / `set_trace()` in committed code | |
| ```bash | |
| rg -n 'breakpoint\(\)|set_trace\(|debugpy\.listen' --type py | |
| ``` | |
| ## One-Shot Recipes | |
| **"Why is this dict missing a key?"** | |
| ```python | |
| # add above the KeyError site | |
| breakpoint() | |
| # then in pdb: | |
| (Pdb) pp d | |
| (Pdb) pp list(d.keys()) | |
| (Pdb) w # how did we get here | |
| ``` | |
| **"This test passes in isolation but fails in the suite."** | |
| ```bash | |
| scripts/run_tests.sh tests/the_test.py --pdb -p no:xdist | |
| # But if it only fails WITH other tests: | |
| source .venv/bin/activate | |
| python -m pytest tests/ -x --pdb -p no:xdist | |
| # Now it pdb-traps at the exact failing test after state accumulated. | |
| ``` | |
| **"My async handler deadlocks."** | |
| ```python | |
| # Add at handler entry | |
| import remote_pdb; remote_pdb.set_trace(host="127.0.0.1", port=4444) | |
| ``` | |
| Trigger the handler. `nc 127.0.0.1 4444`, then `w` to see the suspended frame, `!import asyncio; asyncio.all_tasks()` to see what else is pending. | |
| **"Post-mortem on a crash in an Ink child process / subprocess."** | |
| ```bash | |
| PYTHONFAULTHANDLER=1 python -m pdb -c continue path/to/entrypoint.py | |
| # On crash, pdb lands at the frame of the exception with full locals | |
| ``` | |
Xet Storage Details
- Size:
- 13.2 kB
- Xet hash:
- 2253ce9ca6fd749b0b1d33ab794db9b1a88091a8eb54ba5f4d9bb9293f66b580
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.