docs/Development.md

<!-- ======================================================== -->
## Table Of Contents
<!-- ======================================================== -->

1. [Development](#1-development)
2. [Maintainer Workflow](#2-maintainer-workflow)
   1. [Maintainer Loop](#maintainer-loop)
   2. [Repository Routing](#repository-routing)
   3. [Before Editing](#before-editing)
   4. [Release Workflow](#release-workflow)
3. [Implementation Standards](#3-implementation-standards)
   1. [Source Editing Rules](#source-editing-rules)
   2. [Documentation Authoring](#documentation-authoring)
   3. [Kneifftags Manifest Architecture](#kneifftags-manifest-architecture)
4. [Verification And Commit](#4-verification-and-commit)
   1. [Verification](#verification)
   2. [Review Checklist](#review-checklist)
   3. [Commit Checklist](#commit-checklist)

<br>

# 1. Development

Use this file before changing `kneifftools`.

<br>

# 2. Maintainer Workflow

<!-- ======================================================== -->
## Maintainer Loop
<!-- ======================================================== -->

The maintainer loop is:

1. Read [AGENTS.md](../AGENTS.md), this file, and the relevant docs chapter.
2. Check the current worktree.
3. Locate the source owner for the behavior.
4. Edit the smallest source surface that owns the behavior.
5. Run focused verification.
6. Run broader verification when shared behavior changes.
7. Review the Git diff.
8. Commit.

> [!NOTE]
> Related links:
> - Use [How-To User Guides](How-To-User-Guides.md) for user-facing command recipes.
> - Use [References](References.md) for exact paths, commands, and public surfaces.
> - Use [Explanations](Explanations.md) for the project model behind the workflow.

<br>

<!-- ======================================================== -->
## Repository Routing
<!-- ======================================================== -->

Put changes where the repo already has an owner.

| Change Type | Owner |
|---|---|
| Runtime package behavior | [src/kneiff](../src/kneiff) |
| Maintainer-only package helpers | [src/kneiff_dev](../src/kneiff_dev) |
| Tests | [tests](../tests) |
| Documentation | [docs](.) |
| Generated docs figures | [docs/assets](assets) |
| Ignored local diagram scratch files | `assets/*.IGNORE*` |
| Ignored local research and one-off probes | `experiments/` |
| Project metadata and dependency declarations | [pyproject.toml](../pyproject.toml) |
| Development recipes | [justfile](../justfile) |
| Local agent guidance | [AGENTS.md](../AGENTS.md) |

Ask before creating a new top-level directory when one of these owners is a
reasonable fit.

> [!NOTE]
> Related: use [project paths](References.md#project-paths) for source file
> owners and [package layout](Explanations.md#package-layout) for import
> boundaries.

<br>

<!-- ======================================================== -->
## Before Editing
<!-- ======================================================== -->

Run these checks before a non-trivial change:

```bash
git status --short
rg --files
```

Then read the source owner and nearby files. For example, a CLI change usually
requires checking the package entrypoint, tests, README, and docs references.

> [!IMPORTANT]
> Do not weaken production contracts to make a test easier. Add or reuse a
> dedicated test helper when a test needs lighter setup.

> [!NOTE]
> Related: use [verification](#verification) for the local verification
> checklist.

<br>

<!-- ======================================================== -->
## Release Workflow
<!-- ======================================================== -->

Use the `justfile` release recipes for PyPI and Hugging Face releases:

```bash
just bump-version patch
PYPI_API_KEY="pypi-..." just publish-pypi
just publish-hf
just verify-pypi
```

`just bump-version <bump>` updates the project version with `uv`, creates or
refreshes `uv.lock` and `pylock.toml`, commits the version files, and creates an
annotated `v` tag. `just publish-pypi` rebuilds `dist/`, validates the package
with Twine, and uploads the artifacts to PyPI. `just verify-pypi` installs the
published `kneiff` package in a fresh plain-pip virtual environment.

`just publish-hf` checks that `origin` is a Hugging Face remote, the current
branch is `main`, the worktree is clean, and the current project version has its
matching annotated `v` tag. It then pushes `main` and follows annotated tags,
setting the `origin/main` upstream on the first publish.

> [!IMPORTANT]
> `publish-pypi` requires `PYPI_API_KEY` and a clean worktree. `publish-hf`
> requires a clean `main` worktree, a Hugging Face `origin`, and the current
> version tag.

> [!NOTE]
> Related: use [Development: verification](#verification) before publishing a
> release.

<br>

# 3. Implementation Standards

<!-- ======================================================== -->
## Source Editing Rules
<!-- ======================================================== -->

Follow these rules for normal edits:

- Keep runtime behavior in [src/kneiff](../src/kneiff).
- Keep maintainer-only package helpers in [src/kneiff_dev](../src/kneiff_dev).
- Keep tests in [tests](../tests).
- Reuse existing helpers before adding new helpers.
- Import Kneiff-owned utility helpers through the facade:
  `import kneiff.utils as ut`.
- Use explicit `ut.` prefixes at call sites, for example
  `LOG = ut.get_logger(__name__)`.
- Keep `__init__.py` files limited to imports and module docstrings.
- Prefer exact domain attributes over defensive probes against strict objects.
- Document public behavior with Sphinx-style docstrings.
- Update docs when a change affects setup, usage, CLI behavior, public APIs,
  environment variables, or user-visible workflows.

> [!NOTE]
> Related: use [public interfaces](References.md#public-interfaces) for the
> surfaces that need stable names and documentation.

<br>

<!-- ======================================================== -->
## Documentation Authoring
<!-- ======================================================== -->

Docs live in [docs](.) and follow the standards in [docs/README.md](README.md).

When editing docs:

1. Keep [docs/README.md](README.md) as the docs rules and reading map.
2. Put procedures in [How-To User Guides](How-To-User-Guides.md).
3. Put maintainer workflow in [Development](Development.md).
4. Put exact names in [References](References.md).
5. Put system concepts in [Explanations](Explanations.md).
6. Use separator comments before major sections.
7. Add direct links to source files when a reader may need to edit them.
8. Add `[!NOTE]` related-link callouts from concept sections to task recipes.
9. Update figure assets when prose changes a diagrammed concept.
10. Use [References: figure visual tokens](References.md#figure-visual-tokens)
    before choosing reusable figure colors, strokes, fills, and typography.

> [!NOTE]
> Related links:
> - Use [Markdown formatting rules](README.md#markdown-formatting-rules) for docs layout.
> - Use [link and backlink rules](README.md#link-and-backlink-rules) for related-link callouts.
> - Use [static figure rules](README.md#static-figure-rules) for docs assets.
> - Use [References: figure visual tokens](References.md#figure-visual-tokens)
>   for the repository-owned figure theme source of truth.

<br>

<!-- ======================================================== -->
## Kneifftags Manifest Architecture
<!-- ======================================================== -->

Dataset synchronization uses the schema-version-2 fixed-block workbook and
Kneifftags extension API. The legacy schema-version-1 workbook and project
vocabulary are read only inside the migration transaction. Do not add another
production path for the retired wide workbook or Kneifftools grammar.

The implementation owners are:

| Module | Responsibility |
|---|---|
| `manifest/records.py` | Immutable logical records with no pandas, openpyxl, or caption-frame types. |
| `manifest/block_schema.py` | The single authoritative 13-row A-H field and geometry contract. |
| `manifest/block_workbook.py` | Direct-openpyxl version-2 workbook reading, staging, validation, and atomic replacement. |
| `manifest/sidecar_v2.py` | Strict logical `MANIFEST.yaml` schema version 2 without derived captions. |
| `manifest/identity.py` | Exact size-plus-SHA identity and ambiguity-aware content matching. |
| `manifest/legacy.py` | Read-only schema-1 workbook loading and pure record mapping into fixed fields. |
| `manifest/block_sync.py` | Staging, exact-content reconciliation, validation, archive creation, activation, and rollback. |
| `manifest/caption_service.py` | Kneifftags analysis and rendering boundary shared by workbook, export, and validation consumers. |
| `project_vocabulary.py` | Strict schema-2 extension loading, isolated engine composition, and project identity metadata. |
| `project_vocabulary_migration.py` | Pure schema-1 vocabulary conversion and field-aware input rewrites. |

Every visible image block has these rows in this order:

1. `Relative_path`
2. `Subject`
3. `Appearance`
4. `Composition`
5. `Pose and behavior`
6. `Face`
7. `Anatomy`
8. `Sexual content`
9. `Scene`
10. `Presentation`
11. `Fallback`
12. `SFW`
13. `Notes`

Column C is never merged. `Relative_path` is generated; the other column-C
cells are user-owned. Columns A and D-H are merged across the complete block.
The merged output headers are exactly `tag`, `json`, `nlg`, `chroma`, and
`prose`. `Notes` and the SFW control never become caption input.

The very-hidden `__kneiff_manifest__` sheet stores the workbook schema,
migration source schema, record coordinates, stable record identifiers, exact
content identities, and generated-value fingerprints. The YAML sidecar stores
the same logical record identity and user input but omits derived captions.

Content matching accepts only a unique stale-record and unique current-file
pair with the same byte size and SHA-256 digest. Any duplicate group requires
review. Paths and basenames are not file identity.

Legacy close-up state is carried as deferred evidence. Migration converts an
exclusive close-up only when the opening of the old caption proves the target.
Otherwise it preserves the target in `Composition` and emits a warning.
Populated custom columns without a fixed destination are retained in
`Fallback` with a warning.

> [!IMPORTANT]
> Keep workbook, sidecar, and vocabulary migration transactional. Stage and
> validate every replacement before activating any file, archive the complete
> prior set, and restore it after an activation failure.

> [!NOTE]
> Call Kneifftags through its package root. Analyze each record once, then reuse
> that analysis for `tags`, `json`, `nlg`, `chroma`, and `natural` rendering.
> The workbook retains the visible compatibility headers `tag` and `prose`.

<br>

# 4. Verification And Commit

<!-- ======================================================== -->
## Verification
<!-- ======================================================== -->

Run verification that matches the change.

For Python changes:

```bash
.venv/bin/ruff format .
.venv/bin/ruff check .
.venv/bin/pyright
.venv/bin/pytest
just privacy-audit
```

Run `just privacy-hook` once after cloning. It sets the repository-local Kneiff
identity, disables commit and tag signing, and installs a pre-push hook. The
hook rejects unexpected identities, non-UTC commit dates, signatures, email
addresses, and known private-workspace fingerprints. Create commits with
`TZ=UTC git commit ...` so their author and committer dates satisfy the audit.

For docs-only changes:

```bash
git diff --check
rg -n "TO[D]O|FIX[M]E|content[R]eference|oai[c]ite" README.md docs
python -c "import pathlib, xml.etree.ElementTree as ET; [ET.parse(p) for p in pathlib.Path('docs/assets').glob('*.svg')]"
```

> [!NOTE]
> The exact command depends on the local environment. If a command is not
> installed, state that in the final report and verify the closest safe
> surface.

<br>

<!-- ======================================================== -->
## Review Checklist
<!-- ======================================================== -->

Before finishing, review:

- The change is in the existing owner file or directory.
- New names match existing terminology.
- Docs link to exact files and exact chapters.
- Tests cover the behavior changed.
- The diff does not include unrelated formatting churn.
- New comments follow the local comment style.

> [!NOTE]
> Related: use [Development: before editing](#before-editing) before widening
> the scope of a change.

<br>

<!-- ======================================================== -->
## Commit Checklist
<!-- ======================================================== -->

Before committing:

1. Run focused tests for the touched area.
2. Run broader checks when shared behavior changed.
3. Review `git diff --check`.
4. Review `git status --short`.
5. Write a commit message that names the user-facing behavior.

> [!NOTE]
> Related: use [How-To User Guides: run tests](How-To-User-Guides.md#run-tests)
> for the short command recipe.