Blame for modding-model/README.md

# Modding agent for Ollama (modular)

Portable version of the `modding` skill: a local model, a small Python agent that gives it
internet and file tools, and modular skill files you can extend for any game.

## What is in the box
```
Modelfile              builds the Ollama model (num_ctx, short system identity)
Modelfile.deepseek     local DeepSeek-R1 variant (use with --preset deepseek-r1)
Modelfile.qwen38       Qwen 3.8 27B variant (use with --preset qwen38)
Modelfile.qwen36       Qwen 3.6 35B variant (use with --preset qwen36)
modding.config.example.json   your general games root(s), protected names, context ceiling
skill/core-lean.md     short core prompt used automatically at 8k
modding_agent.py       the agent loop + tools (the model does the thinking, this does the doing)
requirements.txt
skill/core.md          principles, research rule, workflow, tracker format, trust rules
skill/modules/         one file per game or task (load on demand)
    ksp  factorio  bethesda  unity-bepinex  bg3  minecraft   (example game modules; ANY other game is researched on the fly)
    log-debugging  texture-patch-hunt  texture-editing  new-game  dependency-chain   (tasks)
    _template                                                (how to write a new game module)
examples/               sample tracker.html (open it) and tracker.xml
claude-skill/SKILL.md  the single-file version used with Claude, for reference
workspace/<session>/   per-game: tracker.xml + tracker.html (live), downloads, staging, session.json
```

## Setup
1. Install Ollama, then `pip install -r requirements.txt`.
2. Pick a model (see **Which model?** below) and build it, for example `ollama create modding-qwen38 -f Modelfile.qwen38`.
3. Tell the agent where your games live. Use a **general root**, not one game's folder (the agent finds the games inside it):
   - once, in `modding.config.json` (copy `modding.config.example.json`): `"allow": ["C:/Games", "C:/Program Files (x86)/Steam/steamapps/common"]`
   - or the environment variable `MODDING_ALLOW` (folders separated by `;` on Windows, `:` elsewhere)
   - or `--allow` on the command line (repeatable)
4. Run, one **session per game** so every game gets its own workspace, live tracker and history:
   ```
   python modding_agent.py --preset qwen38 --session mygame
   ```
   Then tell it what you want. It is game-agnostic: it calls `find_games` to see what is installed in your allowed folders, loads the matching module if one exists, and researches the game online if none does.

Each session lives in `workspace/<session>/` (tracker.xml, tracker.html, downloads, staging, session.json).

## Focusing the agent (any game)
Nothing is assumed: no default game and **no folder is protected unless you say so**. If you have not said which game or which folders are safe, the agent asks once and tells you these switches:

| In a session | Command line | Meaning |
|---|---|---|
| `/game NAME` | `--game NAME` | the game for this session (any game; a module is loaded if one exists, otherwise the agent researches it) |
| `/protect FOLDER` | `--protect FOLDER` | a folder **name** (matches anywhere) or full **path** it must never modify without asking; repeat for more |
| `/unprotect FOLDER`, `/protected` | | stop protecting / list what is protected |
| `/allow FOLDER` | `--allow FOLDER` | a general folder it may read, and write with your OK |
| | `--session NAME` | one workspace, tracker and history per game |
| `/modules`, `/load MODULE`, `/help` | | list and load modules; show this help |

Saying "do not touch the X folder" in plain words also works: the agent records it and enforces it. `modding.config.json` can hold your usual `allow` roots and `protect` list. With `--yes` and nothing protected, the agent warns you that every write inside allowed folders will be auto-approved.

## Deploying mods and editing textures
The agent does both. **Deploying**: find, download, extract, install, debug and track mods for any game. **Loose texture editing**, in bulk and finely tuned (needs `pip install pillow`):
- `texture_info`: counts, sizes, formats, alpha, and which files look like data maps (normal, specular, roughness, ...).
- `texture_batch`: resize (`scale`, `max_size`), colour (`brightness`, `contrast`, `saturation`, `gamma`, `hue_shift`, `levels`, `channel_gain`, `tint`, `grayscale`, `invert`, `sharpen`, `blur`), format conversion (png, tga, jpg, bmp, webp, dds), restricted to a pixel `region`, and **per-file `rules`** for micro-tuning inside a mass edit, e.g. `scale=0.5 rules=[{"match":"ui/*","skip":true},{"match":"*foliage*","brightness":1.1}]`.
- Safety: always a **dry run first** (nothing written until `dry_run=false`), output goes to a new folder by default, `in_place` keeps backups, `texture_undo` restores a whole job, protected folders are skipped, and colour edits never touch normal/specular/mask maps unless you ask. DDS saving depends on your Pillow version; for exact BC1/BC3/BC5/BC7 control use the game's recommended tool (texconv, nvcompress, ImageMagick).

## Dependency chain protection
Mods often bundle other mods, and "latest" is not always right: a mod can need an exact, not-latest version of something it packages, and free and paid editions can both be the newest you can get. So deploying is chain-aware:
- The agent records, per mod, which dependencies it bundles and whether each is **hard** (exact version needed) or **soft**, and keeps a **dependency chain** table (version, folder, who provided it, who hard-requires it) in the live tracker, plus a **Conflicts** list.
- `deps_check(staged, installed)` is a read-only comparison: items that are new, identical (skipped) or **different** from what is already installed (with versions read from `.version` / `info.json` files where present). `granularity="file"` is for loose-file games such as Bethesda `Data/`.
- `copy_tree` is the safety net: it copies new items, skips identical ones, and **never overwrites or mixes a differing installed folder**; those are reported as open conflicts. Only your decision unlocks them: `exclude=` keeps the installed version, `replace=` swaps the whole folder (never file by file) with a backup in `dep_backups/`. Protected folders are never replaced.
- Conflicts are not asked about one by one: the agent collects them and asks **once**, with a few overall options (keep installed and skip the bundled copies; keep installed and skip those mods; prefer the incoming versions with backups; or decide each). The answer is applied to every file.
- Soft bundled copies are simply left out; if two mods hard-require different versions the agent first looks for a compatibility patch or workaround and only then raises a conflict. The method is in `skill/modules/dependency-chain.md` (with a KSP note about EVE free/paid editions and Kopernicus-family mods as you described them: verify per mod).

## How it works like the Claude version
- **It looks things up.** `web_search` + `web_fetch` (and `browser_fetch` if Playwright is installed) are its main tools, and the core prompt tells it its own knowledge is stale and it must verify versions, links and paths online.
- **Modular.** The model sees an index of modules and calls `load_module` for the game it needs. **New game with no module?** It researches it online and writes one with `save_module` (format in `_template`), so it keeps that knowledge next time.
- **Safe on your machine.** It can only read/write inside the session workspace and your allowed folders (general roots such as `C:\Games`). Anything outside the workspace asks `[y/N]` first; protected folders always ask, even with `--yes`. Downloads of GitHub source archives are blocked; private/local network addresses are blocked; archive extraction is checked for unsafe paths.
- **Live tracker.** Open `workspace/tracker.html` in any browser while it works: it refreshes every 3 seconds and shows the current stage (intake > research > download > extract > deploy > scan > debug > handoff), what the agent is doing right now, every mod's status, issues, and a live log. `workspace/tracker.xml` holds the same data in XML (it also resumes the tracker next session). The model updates it through `tracker_*` tools, and the script also logs every tool call automatically, so the view stays live even if the model forgets.
- **Durable.** Everything goes into `workspace/` (tracker.xml/html, downloads, sessions). Sessions resume with `--session NAME`. Back this folder up: that is what you lost last time.
- **Log debugging.** `grep_file` and `read_file` (with `tail`) on game logs, guided by the `log-debugging` module.

## Context: restricted to 8k
The context window is capped at **8192 tokens** by default (`--num-ctx 8192`, and a ceiling `--max-ctx 8192` that must be raised on purpose: set `max_ctx` in `modding.config.json` or pass `--max-ctx 16384 --num-ctx 16384`). The `Modelfile`s use 8192 too.

To make 8k workable the agent switches to **lean mode** automatically below 12k:
- a short core prompt (`skill/core-lean.md`) and 20 tools with one-line descriptions: the fixed prompt is about **3,100 tokens (about 38% of 8k)**, versus 4,000 to 6,000 for the full prompt;
- `load_module(name)` returns only the essentials (folder layout and logs); ask for the rest with `load_module(name, section="Common failures")`, so a module costs about 200 tokens instead of 500 to 1,000;
- tool outputs are capped at about 2,300 characters, replies at a third of the window (2,730 tokens);
- once the conversation passes ~75% of the window, old tool results are shrunk and then the oldest steps are dropped (never the instructions, your first message or the latest steps), and the model is told to re-read `tracker.xml` for state. This is why the live tracker matters most on small contexts.

Rules of thumb at 8k: use `--think off` (the Qwen presets already do), keep jobs to a handful of mods per step, avoid `--text-tools` (adds ~500 tokens), and expect a long job to be resumed from the tracker rather than remembered. Ollama silently drops the oldest text when a prompt overflows `num_ctx`, which is what the trimming protects against. A startup line shows the real fixed-prompt size.

## Which model? (Qwen 3.8 27B or Qwen 3.6 35B)
Both presets are included, and the agent can compare them **on your machine**:
```
python modding_agent.py --compare qwen38 qwen36
```
It runs eight quick tool-use checks per model (does it pick the right tool with the right arguments: search, read a log tail, grep a log, update the tracker, download a file, find games, run a dependency check, plan a texture batch as a dry run), reports tool-call accuracy, tokens per second and seconds per step, and names the better one with the `--preset` to use. The trade-off to expect: `qwen3.8:27b` is dense (all 27B parameters work on every token), `qwen3.6:35b` is a mixture-of-experts with about 3B active per token. On a 16 GB GPU both are split with system RAM, and the mixture-of-experts usually loses less speed in that split, while the dense model may be more reliable on multi-step tool use. I could not run either model, so run the comparison before deciding. It is a screening test, not a full benchmark: run it twice if the top two are close.

## Qwen 3.8 27B (local)
```
ollama create modding-qwen38 -f Modelfile.qwen38
python modding_agent.py --preset qwen38 --session mygame
python modding_agent.py --preset qwen38 --selftest
```
- **Needs Ollama 0.32.12 or newer.** On an older Ollama the pull fails with "requires a newer version of Ollama": run `ollama --version` and update.
- **The model:** `qwen3.8:27b` is a dense 27B, Q4_K_M, about 18 GB (weights plus a vision projector), ~256K native context, with tools, vision and thinking listed on the Ollama library. Native tool calling works, so no `--text-tools`. A larger `qwen3.8:27b-q8_0` (~30 GB) also exists.
- **Thinking is the catch.** It is on by default and can run away (one test saw a 50,000-character trace). `--preset qwen38` therefore sends `think=false` (Ollama's API switch) and temperature 0.6. For a hard debugging step use `--think on` (the reply cap rises to 12000 tokens, limited to a third of the window) with a bigger window, for example `--max-ctx 32768 --num-ctx 32768`; at 8k, thinking leaves very little room.
- **Empty replies** from a thinking model usually mean it used its whole token budget thinking; the agent says so and what to change.
- **Fit on a 16 GB GPU:** the 18 GB model will not fit entirely in VRAM, so Ollama splits it between GPU and system RAM (`ollama ps` shows the split). Because it is dense, expect it to be slower than a mixture-of-experts model would be in the same split. The 8k window keeps the context memory small.

## Qwen 3.6 35B (alternative)
```
ollama create modding-qwen36 -f Modelfile.qwen36
python modding_agent.py --preset qwen36 --session mygame
```
Needs Ollama 0.30.0 or newer. `qwen3.6:35b` is a 35B-A3B mixture-of-experts (~23-24 GB, about 3B parameters active per token), with tools, vision and thinking listed in the Ollama library. The preset sends `think=false` and temperature 0.6, like the Qwen 3.8 preset. Use `--compare` above to see which of the two suits your hardware.

## DeepSeek

Two ways, same skill, same tools, same live tracker.

### A. DeepSeek API (cloud, strongest option, needs an API key)
```
# PowerShell:  $env:DEEPSEEK_API_KEY = "sk-..."        bash:  export DEEPSEEK_API_KEY=sk-...
python modding_agent.py --backend deepseek --session mygame --allow "C:\Games"
python modding_agent.py --backend deepseek --selftest
```
- Defaults: base URL `https://api.deepseek.com`, model `deepseek-flash` (use `--model deepseek-v4-pro` for the larger one). Model names change: check https://api-docs.deepseek.com and pass `--model` if needed.
- Thinking mode is on by default at DeepSeek. With tools, DeepSeek requires the model's `reasoning_content` to be sent back on every assistant message (the API returns a 400 otherwise). The agent does this automatically. `--thinking off` is faster and cheaper; `--effort low|high|max` sets reasoning effort.
- The API has native tool calling and a very large context window, so this is the closest to how I work for you. Your prompts and any file contents the agent reads go to DeepSeek's servers, so allow only game folders you are happy to share, not your whole drive.
- Any other OpenAI-compatible service works with `--backend openai --base-url ... --model ... --api-key-env NAME`.

### B. DeepSeek locally in Ollama (R1 distills)
```
ollama create modding-deepseek -f Modelfile.deepseek
python modding_agent.py --preset deepseek-r1 --session mygame
# (the preset is shorthand for: --model modding-deepseek --text-tools --fold-system --temperature 0.6)
```
- `--text-tools`: use prompt-based tool calls. This is the safest mode for R1 distills and custom GGUFs; try without it if `ollama show` lists "tools".
- `--fold-system`: moves the instructions into the first user message (DeepSeek's advice for R1).
- The agent strips `<think>...</think>` from replies and never feeds it back; `--think on|off` controls Ollama's thinking option for models that support it.
- Distills are small reasoners: expect weaker multi-step reliability than the API. Keep batches short.

## Test it first
`python modding_agent.py --selftest --model modding` checks, in order: Ollama is running and has the model, the model really emits tool calls, `web_search` returns live results, `web_fetch` works on a JSON API and an HTML page, Pillow is installed for texture editing, the workspace is writable, and the live tracker files are created. Each line says PASS or FAIL with the reason. Add `--text-tools` if the tool-calling check fails.

## Honest limits
- A local model is far weaker than a frontier model at long multi-step jobs. Keep jobs in small batches, keep the tracker open, and say "continue" when it hits the step limit.
- No logged-in browser. **Factorio:** set `FACTORIO_USERNAME` and `FACTORIO_TOKEN` (from `player-data.json`) in your environment and downloads are authenticated by the tool. **Nexus Mods** and other login/captcha sites: the agent will give you the links and you click download (or use Vortex).
- It cannot drive your own browser tabs. Paste your tab titles/URLs or a manager export as the mod list.
- It cannot launch the game for you: you launch, then give it the log path, and it debugs from there.
- Web search uses DuckDuckGo via `ddgs` (no key). Swap `web_search()` in `modding_agent.py` for SearXNG, Brave, or Ollama's web search API if you prefer.

## Add a game
Either let the agent build the module, or copy `skill/modules/_template.md` to `skill/modules/<game>.md`, fill it in, keep line 2 as a `> one-line description` (it feeds the index), then it is available to every session (the agent sees new modules in its list automatically).